diff options
| author | DanConwayDev <DanConwayDev@protonmail.com> | 2025-12-03 11:19:40 +0000 |
|---|---|---|
| committer | DanConwayDev <DanConwayDev@protonmail.com> | 2025-12-03 11:19:40 +0000 |
| commit | 2eaff5b79fed364d5eba5eb38e4b7bf76326884d (patch) | |
| tree | deacd6294f8860096ee82ee76930204efd65e33c /docs/archive/2025-11-04-test-strategy-decision.md | |
| parent | 57bc8cd9c021feaf08e139e8fb62800bc476068e (diff) | |
remove docs archive
Diffstat (limited to 'docs/archive/2025-11-04-test-strategy-decision.md')
| -rw-r--r-- | docs/archive/2025-11-04-test-strategy-decision.md | 290 |
1 files changed, 0 insertions, 290 deletions
diff --git a/docs/archive/2025-11-04-test-strategy-decision.md b/docs/archive/2025-11-04-test-strategy-decision.md deleted file mode 100644 index 63a6961..0000000 --- a/docs/archive/2025-11-04-test-strategy-decision.md +++ /dev/null | |||
| @@ -1,290 +0,0 @@ | |||
| 1 | **ARCHIVED: 2025-11-04** | ||
| 2 | **Decision:** Test ngit-relay first (Option 1) | ||
| 3 | **Rationale:** Validate test suite before implementation (1-2 day investment) | ||
| 4 | |||
| 5 | --- | ||
| 6 | |||
| 7 | # Strategic Recommendation: Test-First vs TDD Approach | ||
| 8 | |||
| 9 | **Date:** 2025-11-04 | ||
| 10 | **Status:** ✅ ARCHIVED - Decision Made | ||
| 11 | **Context:** We have ngit-relay reference implementation available with Docker | ||
| 12 | |||
| 13 | --- | ||
| 14 | |||
| 15 | ## The Question | ||
| 16 | |||
| 17 | Should we: | ||
| 18 | 1. **Test ngit-relay first** - Build grasp-audit against working reference, then apply to ngit-grasp | ||
| 19 | 2. **TDD approach** - Build grasp-audit and ngit-grasp in parallel, test-driven | ||
| 20 | |||
| 21 | --- | ||
| 22 | |||
| 23 | ## Option 1: Test ngit-relay First (RECOMMENDED) | ||
| 24 | |||
| 25 | ### Approach | ||
| 26 | ``` | ||
| 27 | Phase 1: Validate Test Suite (1-2 days) | ||
| 28 | ├── Run ngit-relay Docker image | ||
| 29 | ├── Build grasp-audit GRASP-01 tests | ||
| 30 | ├── Test against ngit-relay | ||
| 31 | └── Fix grasp-audit until all tests pass | ||
| 32 | |||
| 33 | Phase 2: Apply to ngit-grasp (2-3 weeks) | ||
| 34 | ├── Implement ngit-grasp features | ||
| 35 | ├── Run same grasp-audit tests | ||
| 36 | ├── Fix ngit-grasp until tests pass | ||
| 37 | └── Know tests are reliable (validated against reference) | ||
| 38 | ``` | ||
| 39 | |||
| 40 | ### Pros | ||
| 41 | ✅ **Validates test suite first** - Know tests work before implementing | ||
| 42 | ✅ **Clear success criteria** - Tests pass against reference = tests are correct | ||
| 43 | ✅ **Faster feedback** - Catch test bugs early, not during implementation | ||
| 44 | ✅ **Reference behavior** - See how ngit-relay handles edge cases | ||
| 45 | ✅ **Confidence** - When ngit-grasp passes, we know it's compliant | ||
| 46 | ✅ **Documentation** - Tests become living spec examples | ||
| 47 | ✅ **Lower risk** - Don't waste time implementing against broken tests | ||
| 48 | |||
| 49 | ### Cons | ||
| 50 | ❌ **Sequential** - Can't start ngit-grasp until tests validated (but only 1-2 days) | ||
| 51 | ❌ **Docker dependency** - Need Docker to run ngit-relay (already have) | ||
| 52 | ❌ **Different tech stack** - ngit-relay is Go, might have quirks | ||
| 53 | |||
| 54 | ### Timeline | ||
| 55 | - **Phase 1:** 1-2 days (build + validate grasp-audit) | ||
| 56 | - **Phase 2:** 2-3 weeks (implement ngit-grasp) | ||
| 57 | - **Total:** ~3 weeks | ||
| 58 | |||
| 59 | ### Risk Level | ||
| 60 | 🟢 **LOW** - Tests validated before implementation | ||
| 61 | |||
| 62 | --- | ||
| 63 | |||
| 64 | ## Option 2: TDD Parallel Development | ||
| 65 | |||
| 66 | ### Approach | ||
| 67 | ``` | ||
| 68 | Parallel Development | ||
| 69 | ├── Write grasp-audit test | ||
| 70 | ├── Run against ngit-grasp (fails - not implemented) | ||
| 71 | ├── Implement ngit-grasp feature | ||
| 72 | ├── Run test again (should pass) | ||
| 73 | └── Repeat for each feature | ||
| 74 | ``` | ||
| 75 | |||
| 76 | ### Pros | ||
| 77 | ✅ **True TDD** - Red → Green → Refactor cycle | ||
| 78 | ✅ **Parallel work** - No waiting for test validation | ||
| 79 | ✅ **Faster start** - Begin implementation immediately | ||
| 80 | ✅ **Integrated learning** - Discover test issues during implementation | ||
| 81 | |||
| 82 | ### Cons | ||
| 83 | ❌ **Test uncertainty** - Don't know if test failures are test bugs or implementation bugs | ||
| 84 | ❌ **Debugging complexity** - Two moving targets (tests + implementation) | ||
| 85 | ❌ **Wasted effort** - Might implement wrong thing if test is wrong | ||
| 86 | ❌ **No reference** - Can't verify expected behavior | ||
| 87 | ❌ **Higher risk** - Could build to wrong spec | ||
| 88 | |||
| 89 | ### Timeline | ||
| 90 | - **Parallel:** 2-3 weeks (but with more debugging) | ||
| 91 | - **Total:** ~3 weeks (but less confidence) | ||
| 92 | |||
| 93 | ### Risk Level | ||
| 94 | 🟡 **MEDIUM** - Could implement to wrong spec | ||
| 95 | |||
| 96 | --- | ||
| 97 | |||
| 98 | ## Comparison | ||
| 99 | |||
| 100 | | Aspect | Test ngit-relay First | TDD Parallel | | ||
| 101 | |--------|----------------------|--------------| | ||
| 102 | | **Confidence** | High (tests validated) | Medium (tests unproven) | | ||
| 103 | | **Speed to start** | 1-2 day delay | Immediate | | ||
| 104 | | **Debugging complexity** | Low (one target) | High (two targets) | | ||
| 105 | | **Risk of rework** | Low | Medium-High | | ||
| 106 | | **Learning** | See reference behavior | Discover as you go | | ||
| 107 | | **Total time** | ~3 weeks | ~3 weeks | | ||
| 108 | | **Quality** | Higher | Lower | | ||
| 109 | |||
| 110 | --- | ||
| 111 | |||
| 112 | ## Real-World Analogy | ||
| 113 | |||
| 114 | **Option 1 (Test First):** | ||
| 115 | - Like calibrating a measuring tape against a known standard before measuring | ||
| 116 | - Build the test rig, validate it, then use it | ||
| 117 | - Science lab approach: calibrate instruments first | ||
| 118 | |||
| 119 | **Option 2 (TDD Parallel):** | ||
| 120 | - Like building a measuring tape and the thing you're measuring at the same time | ||
| 121 | - Hope the tape is accurate while measuring | ||
| 122 | - Risky if tape is wrong | ||
| 123 | |||
| 124 | --- | ||
| 125 | |||
| 126 | ## Recommendation: TEST NGIT-RELAY FIRST | ||
| 127 | |||
| 128 | ### Why? | ||
| 129 | |||
| 130 | 1. **We already have the reference** - ngit-relay Docker image is available | ||
| 131 | 2. **Low time cost** - Only 1-2 days to validate tests | ||
| 132 | 3. **High confidence gain** - Know tests are correct before implementing | ||
| 133 | 4. **Better debugging** - One variable at a time (test bugs, then implementation bugs) | ||
| 134 | 5. **Living documentation** - Tests show how reference implementation behaves | ||
| 135 | 6. **Risk mitigation** - Don't waste weeks implementing to broken tests | ||
| 136 | |||
| 137 | ### Concrete Plan | ||
| 138 | |||
| 139 | #### Step 1: Setup ngit-relay (30 minutes) | ||
| 140 | ```bash | ||
| 141 | # Pull and run ngit-relay | ||
| 142 | docker pull ngitrelay/ngit-relay:latest | ||
| 143 | docker run -d -p 8080:8080 -p 3000:3000 ngitrelay/ngit-relay | ||
| 144 | |||
| 145 | # Verify it's running | ||
| 146 | curl http://localhost:8080 # Nostr relay | ||
| 147 | curl http://localhost:3000 # Git HTTP backend | ||
| 148 | ``` | ||
| 149 | |||
| 150 | #### Step 2: Build grasp-audit GRASP-01 tests (1 day) | ||
| 151 | ```bash | ||
| 152 | cd grasp-audit | ||
| 153 | |||
| 154 | # Add GRASP-01 Git tests | ||
| 155 | # - Repository creation on announcement | ||
| 156 | # - Clone via HTTP | ||
| 157 | # - Push with valid state (should succeed) | ||
| 158 | # - Push without state (should fail) | ||
| 159 | # - Push with wrong state (should fail) | ||
| 160 | # - Multi-maintainer validation | ||
| 161 | # - refs/nostr/* support | ||
| 162 | |||
| 163 | nix develop -c cargo test | ||
| 164 | ``` | ||
| 165 | |||
| 166 | #### Step 3: Test against ngit-relay (1 day) | ||
| 167 | ```bash | ||
| 168 | # Run compliance tests | ||
| 169 | cd grasp-audit | ||
| 170 | nix develop -c cargo run -- --url ws://localhost:8080 --git-url http://localhost:3000 | ||
| 171 | |||
| 172 | # Fix test bugs until all pass | ||
| 173 | # Document any ngit-relay quirks | ||
| 174 | # Create test fixtures | ||
| 175 | ``` | ||
| 176 | |||
| 177 | #### Step 4: Apply to ngit-grasp (2-3 weeks) | ||
| 178 | ```bash | ||
| 179 | # Now implement ngit-grasp with confidence | ||
| 180 | cd ../ | ||
| 181 | # Implement features | ||
| 182 | # Run grasp-audit tests | ||
| 183 | # Fix ngit-grasp until tests pass | ||
| 184 | ``` | ||
| 185 | |||
| 186 | --- | ||
| 187 | |||
| 188 | ## What We Learn from ngit-relay | ||
| 189 | |||
| 190 | By testing against the reference, we learn: | ||
| 191 | |||
| 192 | 1. **Expected behavior** - How should authorization work exactly? | ||
| 193 | 2. **Error messages** - What does a proper rejection look like? | ||
| 194 | 3. **Edge cases** - How does it handle: | ||
| 195 | - Empty repositories | ||
| 196 | - Multiple refs in one push | ||
| 197 | - Tag vs branch pushes | ||
| 198 | - refs/nostr/* special handling | ||
| 199 | - Concurrent pushes | ||
| 200 | - Invalid state events | ||
| 201 | - Circular maintainer references | ||
| 202 | |||
| 203 | 4. **Protocol details** - Git Smart HTTP quirks | ||
| 204 | 5. **Performance** - What's reasonable for validation time? | ||
| 205 | |||
| 206 | --- | ||
| 207 | |||
| 208 | ## Migration Path | ||
| 209 | |||
| 210 | ### Phase 1: Validate Tests (Days 1-2) | ||
| 211 | - [ ] Setup ngit-relay Docker | ||
| 212 | - [ ] Build grasp-audit Git tests | ||
| 213 | - [ ] Test against ngit-relay | ||
| 214 | - [ ] Fix test bugs | ||
| 215 | - [ ] Document reference behavior | ||
| 216 | |||
| 217 | ### Phase 2: Implement ngit-grasp (Weeks 1-3) | ||
| 218 | - [ ] Follow current_status.md plan | ||
| 219 | - [ ] Run grasp-audit after each phase | ||
| 220 | - [ ] Fix implementation bugs | ||
| 221 | - [ ] Achieve parity with ngit-relay | ||
| 222 | |||
| 223 | ### Phase 3: Exceed Reference (Week 4+) | ||
| 224 | - [ ] Add Rust-specific optimizations | ||
| 225 | - [ ] Better error messages | ||
| 226 | - [ ] Inline authorization benefits | ||
| 227 | - [ ] Performance improvements | ||
| 228 | |||
| 229 | --- | ||
| 230 | |||
| 231 | ## Decision Criteria | ||
| 232 | |||
| 233 | Choose **Test ngit-relay First** if: | ||
| 234 | - ✅ We value confidence over speed to start | ||
| 235 | - ✅ We want to minimize rework risk | ||
| 236 | - ✅ We can spare 1-2 days upfront | ||
| 237 | - ✅ We want tests as living documentation | ||
| 238 | |||
| 239 | Choose **TDD Parallel** if: | ||
| 240 | - ❌ We can't run ngit-relay (Docker issues, etc.) | ||
| 241 | - ❌ We need to start implementation TODAY | ||
| 242 | - ❌ We're comfortable with higher debugging complexity | ||
| 243 | - ❌ We're okay with potential rework | ||
| 244 | |||
| 245 | --- | ||
| 246 | |||
| 247 | ## My Recommendation | ||
| 248 | |||
| 249 | **🎯 Test ngit-relay first** | ||
| 250 | |||
| 251 | **Reasoning:** | ||
| 252 | 1. Only 1-2 days upfront investment | ||
| 253 | 2. Massively reduces risk of wasted effort | ||
| 254 | 3. Provides living documentation | ||
| 255 | 4. Gives confidence in test suite | ||
| 256 | 5. We already have Docker and ngit-relay available | ||
| 257 | 6. Total timeline is same (~3 weeks) but with higher quality | ||
| 258 | |||
| 259 | **The 1-2 day investment in test validation will save us days or weeks of debugging "is it the test or the implementation?"** | ||
| 260 | |||
| 261 | --- | ||
| 262 | |||
| 263 | ## Next Steps | ||
| 264 | |||
| 265 | If you agree with this recommendation: | ||
| 266 | |||
| 267 | 1. **Today:** Setup ngit-relay Docker | ||
| 268 | 2. **Tomorrow:** Build GRASP-01 Git tests in grasp-audit | ||
| 269 | 3. **Day 3:** Validate tests against ngit-relay | ||
| 270 | 4. **Week 2-4:** Implement ngit-grasp with confidence | ||
| 271 | |||
| 272 | If you prefer TDD parallel: | ||
| 273 | 1. **Today:** Start implementing ngit-grasp Git backend | ||
| 274 | 2. **Ongoing:** Write tests alongside implementation | ||
| 275 | 3. **Risk:** Accept higher debugging complexity | ||
| 276 | |||
| 277 | --- | ||
| 278 | |||
| 279 | ## Questions? | ||
| 280 | |||
| 281 | - Is Docker available for ngit-relay? | ||
| 282 | - Any blockers to testing against reference? | ||
| 283 | - Time constraints that require immediate implementation? | ||
| 284 | - Other considerations I'm missing? | ||
| 285 | |||
| 286 | --- | ||
| 287 | |||
| 288 | **Recommendation:** 🎯 **Test ngit-relay first** (1-2 day investment, weeks of confidence) | ||
| 289 | |||
| 290 | **Confidence Level:** 95% - This is the right approach | ||