upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/archive/2025-11-03-test-breakdown.md
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2025-12-03 11:19:40 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2025-12-03 11:19:40 +0000
commit2eaff5b79fed364d5eba5eb38e4b7bf76326884d (patch)
treedeacd6294f8860096ee82ee76930204efd65e33c /docs/archive/2025-11-03-test-breakdown.md
parent57bc8cd9c021feaf08e139e8fb62800bc476068e (diff)
remove docs archive
Diffstat (limited to 'docs/archive/2025-11-03-test-breakdown.md')
-rw-r--r--docs/archive/2025-11-03-test-breakdown.md203
1 files changed, 0 insertions, 203 deletions
diff --git a/docs/archive/2025-11-03-test-breakdown.md b/docs/archive/2025-11-03-test-breakdown.md
deleted file mode 100644
index c054cd3..0000000
--- a/docs/archive/2025-11-03-test-breakdown.md
+++ /dev/null
@@ -1,203 +0,0 @@
1# GRASP-01 Test Breakdown: First Requirement
2
3**Requirement:** "MUST serve a NIP-01 compliant nostr relay at / that accepts git repository announcements and their corresponding repo state announcements."
4
5---
6
7## Test Summary
8
9| Category | Count | Purpose | Time Investment |
10|----------|-------|---------|-----------------|
11| NIP-01 Smoke Tests | 6 | Verify basic relay works | 1-2 days |
12| GRASP-01 Specific | 12 | Verify GRASP protocol | 3-4 days |
13| **Total** | **18** | **Prove the concept** | **1 week** |
14
15---
16
17## NIP-01 Smoke Tests (6 tests)
18
19### Why Only Smoke Tests?
20
21**rust-nostr already has 1000+ tests for:**
22- ✅ Event structure validation
23- ✅ Signature verification (Schnorr/secp256k1)
24- ✅ Event ID calculation (SHA256)
25- ✅ WebSocket message handling
26- ✅ Subscription management
27- ✅ Filter matching
28
29**We don't need to re-test this.** We just verify the relay works at all.
30
31### The 6 Smoke Tests
32
33| # | Test Name | What It Tests | Why It Matters |
34|---|-----------|---------------|----------------|
35| 1 | `websocket_connection` | Can connect to `/` via WebSocket | Relay is running and accepting connections |
36| 2 | `send_receive_event` | Can send EVENT, get OK response | Basic event submission works |
37| 3 | `create_subscription` | Can send REQ, receive EOSE | Subscription system works |
38| 4 | `close_subscription` | Can close subscriptions | Cleanup works |
39| 5 | `reject_invalid_event` | Rejects bad signatures | Validation is enabled |
40| 6 | `reject_invalid_event_id` | Rejects wrong IDs | ID verification works |
41
42**Coverage:** Basic Nostr relay functionality (not GRASP-specific)
43
44---
45
46## GRASP-01 Specific Tests (12 tests)
47
48### Why These Tests?
49
50These test **our code**, not rust-nostr's code. They verify:
51- GRASP policy enforcement
52- Repository announcement acceptance
53- Integration between Nostr relay and Git service
54
55### The 12 GRASP Tests
56
57| # | Test Name | Spec Ref | What It Tests |
58|---|-----------|----------|---------------|
59| 7 | `accepts_repository_announcement` | GRASP-01:9-10 | Accepts NIP-34 kind 30617 events |
60| 8 | `accepts_repository_state` | GRASP-01:9-10 | Accepts NIP-34 kind 30618 events |
61| 9 | `rejects_announcement_without_clone_tag` | GRASP-01:12-13 | Enforces clone tag requirement |
62| 10 | `rejects_announcement_without_relay_tag` | GRASP-01:12-13 | Enforces relay tag requirement |
63| 11 | `accepts_announcement_with_multiple_clones` | GRASP-01:12-13 | Handles multiple clone URLs |
64| 12 | `accepts_events_tagging_announcement` | GRASP-01:17-20 | Accepts issues/PRs tagging repos |
65| 13 | `accepts_events_tagged_by_announcement` | GRASP-01:17-20 | Accepts events tagged by repos |
66| 14 | `rejects_events_tagging_rejected_announcement` | GRASP-01:17-20 | Rejects orphaned events |
67| 15 | `query_announcements_by_identifier` | GRASP-01 (implied) | Can query repos by identifier |
68| 16 | `query_state_events` | GRASP-01 (implied) | Can query repository state |
69| 17 | `state_replaces_previous` | NIP-01 replaceable | Latest state wins |
70| 18 | `concurrent_event_submission` | General reliability | No race conditions |
71
72**Coverage:** GRASP protocol requirements and policy enforcement
73
74---
75
76## What We're NOT Testing (and Why)
77
78### Not Testing: NIP-01 Core Protocol
79
80**Reason:** rust-nostr already tests this extensively
81
82| What | Why Not Testing |
83|------|-----------------|
84| Event signature verification | rust-nostr has 100+ tests |
85| Event ID calculation | rust-nostr has 50+ tests |
86| WebSocket message parsing | rust-nostr has 200+ tests |
87| Subscription filter matching | rust-nostr has 150+ tests |
88| Event serialization | rust-nostr has 75+ tests |
89
90**Estimated time saved:** 2-3 weeks of redundant work
91
92### Not Testing: Git Protocol Details
93
94**Reason:** Will test in separate Git service tests
95
96| What | Where It's Tested |
97|------|-------------------|
98| Git pack parsing | Git service unit tests |
99| Ref update parsing | Git service unit tests |
100| Git authorization | Git integration tests |
101| Push/pull operations | E2E tests |
102
103---
104
105## Test Implementation Estimate
106
107### Week 1: Test Tool Foundation
108- **Day 1-2**: Set up `grasp-compliance-tests/` crate
109- **Day 3**: Implement test client (HTTP/WebSocket)
110- **Day 4**: Implement NIP-01 smoke tests (6 tests)
111- **Day 5**: Test fixtures and builders
112
113### Week 2: GRASP Tests
114- **Day 1-2**: Implement announcement tests (7-11)
115- **Day 3**: Implement related event tests (12-14)
116- **Day 4**: Implement query tests (15-17)
117- **Day 5**: Implement concurrent test (18) + polish
118
119### Week 3: Integration
120- **Day 1-2**: Create minimal ngit-grasp skeleton
121- **Day 3-4**: Wire up nostr-relay-builder
122- **Day 5**: First test run (expect failures)
123
124### Week 4: Iteration
125- **Day 1-3**: Fix failing tests
126- **Day 4**: Documentation
127- **Day 5**: Polish and prepare for next requirement
128
129**Total:** 4 weeks to prove the concept
130
131---
132
133## Success Criteria
134
135### Phase 1: Test Tool Works
136- ✅ Can connect to any WebSocket relay
137- ✅ Can send events and subscriptions
138- ✅ Can assert on responses
139- ✅ All 18 tests can execute (even if they fail)
140
141### Phase 2: Smoke Tests Pass
142- ✅ Basic NIP-01 functionality works
143- ✅ Can send/receive events
144- ✅ Subscriptions work
145- ✅ Invalid events rejected
146
147### Phase 3: GRASP Tests Pass
148- ✅ Repository announcements accepted
149- ✅ State events accepted
150- ✅ Policy enforcement works (clone/relay tags)
151- ✅ Related events accepted
152- ✅ Queries work
153
154### Phase 4: Concept Proven
155- ✅ All 18 tests pass
156- ✅ Test tool is reusable
157- ✅ Architecture validated
158- ✅ Ready for next GRASP-01 requirements
159
160---
161
162## Comparison: Our Approach vs. Comprehensive
163
164| Aspect | Our Approach | Comprehensive Approach |
165|--------|--------------|------------------------|
166| NIP-01 Tests | 6 smoke tests | 50-100 full tests |
167| GRASP Tests | 12 focused tests | 12 focused tests |
168| Total Tests | **18** | **62-112** |
169| Time to Implement | **1 week** | **3-4 weeks** |
170| Maintenance Burden | **Low** | **High** |
171| Redundancy | **Minimal** | **Significant** |
172| Value-Add | **High** (GRASP-specific) | **Low** (mostly redundant) |
173
174**Conclusion:** Our approach is 3-4x faster with same GRASP coverage.
175
176---
177
178## Next Steps
179
1801. ✅ **Review this breakdown** - Confirm scope
1812. ✅ **Choose approach** - Test-first, parallel, or implementation-first
1823. ✅ **Start implementation** - Create test tool skeleton
1834. ✅ **Iterate** - Build until all tests pass
184
185---
186
187## Questions?
188
189- **Q: Is 6 smoke tests enough?**
190 A: Yes, because rust-nostr already tests NIP-01 comprehensively.
191
192- **Q: Should we test more NIP-01 features?**
193 A: Only if we find bugs in rust-nostr (unlikely).
194
195- **Q: Can other implementations use this?**
196 A: Yes! That's the point of making it standalone.
197
198- **Q: What about GRASP-02 and GRASP-05?**
199 A: We'll add those test modules later, same structure.
200
201---
202
203**Ready to proceed?** See REPORT_COMPLIANCE_TESTING.md for full details.