path: root/docs/archive/2025-11-04-test-strategy-decision.md

**ARCHIVED: 2025-11-04**  
**Decision:** Test ngit-relay first (Option 1)  
**Rationale:** Validate test suite before implementation (1-2 day investment)

---

# Strategic Recommendation: Test-First vs TDD Approach

**Date:** 2025-11-04  
**Status:** ✅ ARCHIVED - Decision Made  
**Context:** We have ngit-relay reference implementation available with Docker

---

## The Question

Should we:
1. **Test ngit-relay first** - Build grasp-audit against working reference, then apply to ngit-grasp
2. **TDD approach** - Build grasp-audit and ngit-grasp in parallel, test-driven

---

## Option 1: Test ngit-relay First (RECOMMENDED)

### Approach
```
Phase 1: Validate Test Suite (1-2 days)
├── Run ngit-relay Docker image
├── Build grasp-audit GRASP-01 tests
├── Test against ngit-relay
└── Fix grasp-audit until all tests pass

Phase 2: Apply to ngit-grasp (2-3 weeks)
├── Implement ngit-grasp features
├── Run same grasp-audit tests
├── Fix ngit-grasp until tests pass
└── Know tests are reliable (validated against reference)
```

### Pros
✅ **Validates test suite first** - Know tests work before implementing  
✅ **Clear success criteria** - Tests pass against reference = tests are correct  
✅ **Faster feedback** - Catch test bugs early, not during implementation  
✅ **Reference behavior** - See how ngit-relay handles edge cases  
✅ **Confidence** - When ngit-grasp passes, we know it's compliant  
✅ **Documentation** - Tests become living spec examples  
✅ **Lower risk** - Don't waste time implementing against broken tests

### Cons
❌ **Sequential** - Can't start ngit-grasp until tests validated (but only 1-2 days)  
❌ **Docker dependency** - Need Docker to run ngit-relay (already have)  
❌ **Different tech stack** - ngit-relay is Go, might have quirks

### Timeline
- **Phase 1:** 1-2 days (build + validate grasp-audit)
- **Phase 2:** 2-3 weeks (implement ngit-grasp)
- **Total:** ~3 weeks

### Risk Level
🟢 **LOW** - Tests validated before implementation

---

## Option 2: TDD Parallel Development

### Approach
```
Parallel Development
├── Write grasp-audit test
├── Run against ngit-grasp (fails - not implemented)
├── Implement ngit-grasp feature
├── Run test again (should pass)
└── Repeat for each feature
```

### Pros
✅ **True TDD** - Red → Green → Refactor cycle  
✅ **Parallel work** - No waiting for test validation  
✅ **Faster start** - Begin implementation immediately  
✅ **Integrated learning** - Discover test issues during implementation

### Cons
❌ **Test uncertainty** - Don't know if test failures are test bugs or implementation bugs  
❌ **Debugging complexity** - Two moving targets (tests + implementation)  
❌ **Wasted effort** - Might implement wrong thing if test is wrong  
❌ **No reference** - Can't verify expected behavior  
❌ **Higher risk** - Could build to wrong spec

### Timeline
- **Parallel:** 2-3 weeks (but with more debugging)
- **Total:** ~3 weeks (but less confidence)

### Risk Level
🟡 **MEDIUM** - Could implement to wrong spec

---

## Comparison

| Aspect | Test ngit-relay First | TDD Parallel |
|--------|----------------------|--------------|
| **Confidence** | High (tests validated) | Medium (tests unproven) |
| **Speed to start** | 1-2 day delay | Immediate |
| **Debugging complexity** | Low (one target) | High (two targets) |
| **Risk of rework** | Low | Medium-High |
| **Learning** | See reference behavior | Discover as you go |
| **Total time** | ~3 weeks | ~3 weeks |
| **Quality** | Higher | Lower |

---

## Real-World Analogy

**Option 1 (Test First):**
- Like calibrating a measuring tape against a known standard before measuring
- Build the test rig, validate it, then use it
- Science lab approach: calibrate instruments first

**Option 2 (TDD Parallel):**
- Like building a measuring tape and the thing you're measuring at the same time
- Hope the tape is accurate while measuring
- Risky if tape is wrong

---

## Recommendation: TEST NGIT-RELAY FIRST

### Why?

1. **We already have the reference** - ngit-relay Docker image is available
2. **Low time cost** - Only 1-2 days to validate tests
3. **High confidence gain** - Know tests are correct before implementing
4. **Better debugging** - One variable at a time (test bugs, then implementation bugs)
5. **Living documentation** - Tests show how reference implementation behaves
6. **Risk mitigation** - Don't waste weeks implementing to broken tests

### Concrete Plan

#### Step 1: Setup ngit-relay (30 minutes)
```bash
# Pull and run ngit-relay
docker pull ngitrelay/ngit-relay:latest
docker run -d -p 8080:8080 -p 3000:3000 ngitrelay/ngit-relay

# Verify it's running
curl http://localhost:8080  # Nostr relay
curl http://localhost:3000  # Git HTTP backend
```

#### Step 2: Build grasp-audit GRASP-01 tests (1 day)
```bash
cd grasp-audit

# Add GRASP-01 Git tests
# - Repository creation on announcement
# - Clone via HTTP
# - Push with valid state (should succeed)
# - Push without state (should fail)
# - Push with wrong state (should fail)
# - Multi-maintainer validation
# - refs/nostr/* support

nix develop -c cargo test
```

#### Step 3: Test against ngit-relay (1 day)
```bash
# Run compliance tests
cd grasp-audit
nix develop -c cargo run -- --url ws://localhost:8080 --git-url http://localhost:3000

# Fix test bugs until all pass
# Document any ngit-relay quirks
# Create test fixtures
```

#### Step 4: Apply to ngit-grasp (2-3 weeks)
```bash
# Now implement ngit-grasp with confidence
cd ../
# Implement features
# Run grasp-audit tests
# Fix ngit-grasp until tests pass
```

---

## What We Learn from ngit-relay

By testing against the reference, we learn:

1. **Expected behavior** - How should authorization work exactly?
2. **Error messages** - What does a proper rejection look like?
3. **Edge cases** - How does it handle:
   - Empty repositories
   - Multiple refs in one push
   - Tag vs branch pushes
   - refs/nostr/* special handling
   - Concurrent pushes
   - Invalid state events
   - Circular maintainer references

4. **Protocol details** - Git Smart HTTP quirks
5. **Performance** - What's reasonable for validation time?

---

## Migration Path

### Phase 1: Validate Tests (Days 1-2)
- [ ] Setup ngit-relay Docker
- [ ] Build grasp-audit Git tests
- [ ] Test against ngit-relay
- [ ] Fix test bugs
- [ ] Document reference behavior

### Phase 2: Implement ngit-grasp (Weeks 1-3)
- [ ] Follow current_status.md plan
- [ ] Run grasp-audit after each phase
- [ ] Fix implementation bugs
- [ ] Achieve parity with ngit-relay

### Phase 3: Exceed Reference (Week 4+)
- [ ] Add Rust-specific optimizations
- [ ] Better error messages
- [ ] Inline authorization benefits
- [ ] Performance improvements

---

## Decision Criteria

Choose **Test ngit-relay First** if:
- ✅ We value confidence over speed to start
- ✅ We want to minimize rework risk
- ✅ We can spare 1-2 days upfront
- ✅ We want tests as living documentation

Choose **TDD Parallel** if:
- ❌ We can't run ngit-relay (Docker issues, etc.)
- ❌ We need to start implementation TODAY
- ❌ We're comfortable with higher debugging complexity
- ❌ We're okay with potential rework

---

## My Recommendation

**🎯 Test ngit-relay first**

**Reasoning:**
1. Only 1-2 days upfront investment
2. Massively reduces risk of wasted effort
3. Provides living documentation
4. Gives confidence in test suite
5. We already have Docker and ngit-relay available
6. Total timeline is same (~3 weeks) but with higher quality

**The 1-2 day investment in test validation will save us days or weeks of debugging "is it the test or the implementation?"**

---

## Next Steps

If you agree with this recommendation:

1. **Today:** Setup ngit-relay Docker
2. **Tomorrow:** Build GRASP-01 Git tests in grasp-audit
3. **Day 3:** Validate tests against ngit-relay
4. **Week 2-4:** Implement ngit-grasp with confidence

If you prefer TDD parallel:
1. **Today:** Start implementing ngit-grasp Git backend
2. **Ongoing:** Write tests alongside implementation
3. **Risk:** Accept higher debugging complexity

---

## Questions?

- Is Docker available for ngit-relay?
- Any blockers to testing against reference?
- Time constraints that require immediate implementation?
- Other considerations I'm missing?

---

**Recommendation:** 🎯 **Test ngit-relay first** (1-2 day investment, weeks of confidence)

**Confidence Level:** 95% - This is the right approach