upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/archive/2025-11-04-test-strategy-decision.md
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2025-12-03 11:19:40 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2025-12-03 11:19:40 +0000
commit2eaff5b79fed364d5eba5eb38e4b7bf76326884d (patch)
treedeacd6294f8860096ee82ee76930204efd65e33c /docs/archive/2025-11-04-test-strategy-decision.md
parent57bc8cd9c021feaf08e139e8fb62800bc476068e (diff)
remove docs archive
Diffstat (limited to 'docs/archive/2025-11-04-test-strategy-decision.md')
-rw-r--r--docs/archive/2025-11-04-test-strategy-decision.md290
1 files changed, 0 insertions, 290 deletions
diff --git a/docs/archive/2025-11-04-test-strategy-decision.md b/docs/archive/2025-11-04-test-strategy-decision.md
deleted file mode 100644
index 63a6961..0000000
--- a/docs/archive/2025-11-04-test-strategy-decision.md
+++ /dev/null
@@ -1,290 +0,0 @@
1**ARCHIVED: 2025-11-04**
2**Decision:** Test ngit-relay first (Option 1)
3**Rationale:** Validate test suite before implementation (1-2 day investment)
4
5---
6
7# Strategic Recommendation: Test-First vs TDD Approach
8
9**Date:** 2025-11-04
10**Status:** ✅ ARCHIVED - Decision Made
11**Context:** We have ngit-relay reference implementation available with Docker
12
13---
14
15## The Question
16
17Should we:
181. **Test ngit-relay first** - Build grasp-audit against working reference, then apply to ngit-grasp
192. **TDD approach** - Build grasp-audit and ngit-grasp in parallel, test-driven
20
21---
22
23## Option 1: Test ngit-relay First (RECOMMENDED)
24
25### Approach
26```
27Phase 1: Validate Test Suite (1-2 days)
28├── Run ngit-relay Docker image
29├── Build grasp-audit GRASP-01 tests
30├── Test against ngit-relay
31└── Fix grasp-audit until all tests pass
32
33Phase 2: Apply to ngit-grasp (2-3 weeks)
34├── Implement ngit-grasp features
35├── Run same grasp-audit tests
36├── Fix ngit-grasp until tests pass
37└── Know tests are reliable (validated against reference)
38```
39
40### Pros
41✅ **Validates test suite first** - Know tests work before implementing
42✅ **Clear success criteria** - Tests pass against reference = tests are correct
43✅ **Faster feedback** - Catch test bugs early, not during implementation
44✅ **Reference behavior** - See how ngit-relay handles edge cases
45✅ **Confidence** - When ngit-grasp passes, we know it's compliant
46✅ **Documentation** - Tests become living spec examples
47✅ **Lower risk** - Don't waste time implementing against broken tests
48
49### Cons
50❌ **Sequential** - Can't start ngit-grasp until tests validated (but only 1-2 days)
51❌ **Docker dependency** - Need Docker to run ngit-relay (already have)
52❌ **Different tech stack** - ngit-relay is Go, might have quirks
53
54### Timeline
55- **Phase 1:** 1-2 days (build + validate grasp-audit)
56- **Phase 2:** 2-3 weeks (implement ngit-grasp)
57- **Total:** ~3 weeks
58
59### Risk Level
60🟢 **LOW** - Tests validated before implementation
61
62---
63
64## Option 2: TDD Parallel Development
65
66### Approach
67```
68Parallel Development
69├── Write grasp-audit test
70├── Run against ngit-grasp (fails - not implemented)
71├── Implement ngit-grasp feature
72├── Run test again (should pass)
73└── Repeat for each feature
74```
75
76### Pros
77✅ **True TDD** - Red → Green → Refactor cycle
78✅ **Parallel work** - No waiting for test validation
79✅ **Faster start** - Begin implementation immediately
80✅ **Integrated learning** - Discover test issues during implementation
81
82### Cons
83❌ **Test uncertainty** - Don't know if test failures are test bugs or implementation bugs
84❌ **Debugging complexity** - Two moving targets (tests + implementation)
85❌ **Wasted effort** - Might implement wrong thing if test is wrong
86❌ **No reference** - Can't verify expected behavior
87❌ **Higher risk** - Could build to wrong spec
88
89### Timeline
90- **Parallel:** 2-3 weeks (but with more debugging)
91- **Total:** ~3 weeks (but less confidence)
92
93### Risk Level
94🟡 **MEDIUM** - Could implement to wrong spec
95
96---
97
98## Comparison
99
100| Aspect | Test ngit-relay First | TDD Parallel |
101|--------|----------------------|--------------|
102| **Confidence** | High (tests validated) | Medium (tests unproven) |
103| **Speed to start** | 1-2 day delay | Immediate |
104| **Debugging complexity** | Low (one target) | High (two targets) |
105| **Risk of rework** | Low | Medium-High |
106| **Learning** | See reference behavior | Discover as you go |
107| **Total time** | ~3 weeks | ~3 weeks |
108| **Quality** | Higher | Lower |
109
110---
111
112## Real-World Analogy
113
114**Option 1 (Test First):**
115- Like calibrating a measuring tape against a known standard before measuring
116- Build the test rig, validate it, then use it
117- Science lab approach: calibrate instruments first
118
119**Option 2 (TDD Parallel):**
120- Like building a measuring tape and the thing you're measuring at the same time
121- Hope the tape is accurate while measuring
122- Risky if tape is wrong
123
124---
125
126## Recommendation: TEST NGIT-RELAY FIRST
127
128### Why?
129
1301. **We already have the reference** - ngit-relay Docker image is available
1312. **Low time cost** - Only 1-2 days to validate tests
1323. **High confidence gain** - Know tests are correct before implementing
1334. **Better debugging** - One variable at a time (test bugs, then implementation bugs)
1345. **Living documentation** - Tests show how reference implementation behaves
1356. **Risk mitigation** - Don't waste weeks implementing to broken tests
136
137### Concrete Plan
138
139#### Step 1: Setup ngit-relay (30 minutes)
140```bash
141# Pull and run ngit-relay
142docker pull ngitrelay/ngit-relay:latest
143docker run -d -p 8080:8080 -p 3000:3000 ngitrelay/ngit-relay
144
145# Verify it's running
146curl http://localhost:8080 # Nostr relay
147curl http://localhost:3000 # Git HTTP backend
148```
149
150#### Step 2: Build grasp-audit GRASP-01 tests (1 day)
151```bash
152cd grasp-audit
153
154# Add GRASP-01 Git tests
155# - Repository creation on announcement
156# - Clone via HTTP
157# - Push with valid state (should succeed)
158# - Push without state (should fail)
159# - Push with wrong state (should fail)
160# - Multi-maintainer validation
161# - refs/nostr/* support
162
163nix develop -c cargo test
164```
165
166#### Step 3: Test against ngit-relay (1 day)
167```bash
168# Run compliance tests
169cd grasp-audit
170nix develop -c cargo run -- --url ws://localhost:8080 --git-url http://localhost:3000
171
172# Fix test bugs until all pass
173# Document any ngit-relay quirks
174# Create test fixtures
175```
176
177#### Step 4: Apply to ngit-grasp (2-3 weeks)
178```bash
179# Now implement ngit-grasp with confidence
180cd ../
181# Implement features
182# Run grasp-audit tests
183# Fix ngit-grasp until tests pass
184```
185
186---
187
188## What We Learn from ngit-relay
189
190By testing against the reference, we learn:
191
1921. **Expected behavior** - How should authorization work exactly?
1932. **Error messages** - What does a proper rejection look like?
1943. **Edge cases** - How does it handle:
195 - Empty repositories
196 - Multiple refs in one push
197 - Tag vs branch pushes
198 - refs/nostr/* special handling
199 - Concurrent pushes
200 - Invalid state events
201 - Circular maintainer references
202
2034. **Protocol details** - Git Smart HTTP quirks
2045. **Performance** - What's reasonable for validation time?
205
206---
207
208## Migration Path
209
210### Phase 1: Validate Tests (Days 1-2)
211- [ ] Setup ngit-relay Docker
212- [ ] Build grasp-audit Git tests
213- [ ] Test against ngit-relay
214- [ ] Fix test bugs
215- [ ] Document reference behavior
216
217### Phase 2: Implement ngit-grasp (Weeks 1-3)
218- [ ] Follow current_status.md plan
219- [ ] Run grasp-audit after each phase
220- [ ] Fix implementation bugs
221- [ ] Achieve parity with ngit-relay
222
223### Phase 3: Exceed Reference (Week 4+)
224- [ ] Add Rust-specific optimizations
225- [ ] Better error messages
226- [ ] Inline authorization benefits
227- [ ] Performance improvements
228
229---
230
231## Decision Criteria
232
233Choose **Test ngit-relay First** if:
234- ✅ We value confidence over speed to start
235- ✅ We want to minimize rework risk
236- ✅ We can spare 1-2 days upfront
237- ✅ We want tests as living documentation
238
239Choose **TDD Parallel** if:
240- ❌ We can't run ngit-relay (Docker issues, etc.)
241- ❌ We need to start implementation TODAY
242- ❌ We're comfortable with higher debugging complexity
243- ❌ We're okay with potential rework
244
245---
246
247## My Recommendation
248
249**🎯 Test ngit-relay first**
250
251**Reasoning:**
2521. Only 1-2 days upfront investment
2532. Massively reduces risk of wasted effort
2543. Provides living documentation
2554. Gives confidence in test suite
2565. We already have Docker and ngit-relay available
2576. Total timeline is same (~3 weeks) but with higher quality
258
259**The 1-2 day investment in test validation will save us days or weeks of debugging "is it the test or the implementation?"**
260
261---
262
263## Next Steps
264
265If you agree with this recommendation:
266
2671. **Today:** Setup ngit-relay Docker
2682. **Tomorrow:** Build GRASP-01 Git tests in grasp-audit
2693. **Day 3:** Validate tests against ngit-relay
2704. **Week 2-4:** Implement ngit-grasp with confidence
271
272If you prefer TDD parallel:
2731. **Today:** Start implementing ngit-grasp Git backend
2742. **Ongoing:** Write tests alongside implementation
2753. **Risk:** Accept higher debugging complexity
276
277---
278
279## Questions?
280
281- Is Docker available for ngit-relay?
282- Any blockers to testing against reference?
283- Time constraints that require immediate implementation?
284- Other considerations I'm missing?
285
286---
287
288**Recommendation:** 🎯 **Test ngit-relay first** (1-2 day investment, weeks of confidence)
289
290**Confidence Level:** 95% - This is the right approach