upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/archive/2025-11-03-compliance-testing-report.md
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2025-12-03 11:19:40 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2025-12-03 11:19:40 +0000
commit2eaff5b79fed364d5eba5eb38e4b7bf76326884d (patch)
treedeacd6294f8860096ee82ee76930204efd65e33c /docs/archive/2025-11-03-compliance-testing-report.md
parent57bc8cd9c021feaf08e139e8fb62800bc476068e (diff)
remove docs archive
Diffstat (limited to 'docs/archive/2025-11-03-compliance-testing-report.md')
-rw-r--r--docs/archive/2025-11-03-compliance-testing-report.md330
1 files changed, 0 insertions, 330 deletions
diff --git a/docs/archive/2025-11-03-compliance-testing-report.md b/docs/archive/2025-11-03-compliance-testing-report.md
deleted file mode 100644
index d850f73..0000000
--- a/docs/archive/2025-11-03-compliance-testing-report.md
+++ /dev/null
@@ -1,330 +0,0 @@
1# Report: GRASP Compliance Testing Strategy
2
3**Date:** November 3, 2025
4**Subject:** Exportable Test Tool for GRASP-01 First Requirement
5**Status:** Proposal Ready for Review
6
7---
8
9## Executive Summary
10
11I've analyzed the requirements for testing GRASP-01's first requirement: *"MUST serve a NIP-01 compliant nostr relay at / that accepts git repository announcements and their corresponding repo state announcements."*
12
13**Key Finding:** We should NOT extensively test NIP-01 compliance because `rust-nostr` already has 1000+ tests for this. Instead, we should:
14- ✅ Write **6 smoke tests** for basic NIP-01 functionality
15- ✅ Write **12 GRASP-specific tests** for repository announcements
16- ✅ Create a **reusable compliance testing tool** that any GRASP implementation can use
17
18This focused approach saves significant time while ensuring comprehensive GRASP protocol testing.
19
20---
21
22## The NIP-01 Testing Question
23
24### What is NIP-01?
25
26NIP-01 defines the basic Nostr protocol:
27- Event structure (id, pubkey, sig, kind, tags, content)
28- Event validation (signature verification, ID calculation)
29- WebSocket messages (EVENT, REQ, CLOSE, NOTICE, OK, EOSE)
30- Subscription filters
31
32### What Does rust-nostr Already Test?
33
34The `nostr-relay-builder` crate we're using includes:
35- ✅ Complete event validation
36- ✅ Signature verification (Schnorr on secp256k1)
37- ✅ Event ID validation (SHA256)
38- ✅ WebSocket message handling
39- ✅ Subscription management
40- ✅ 1000+ unit and integration tests
41
42### Recommendation: Smoke Tests Only
43
44**We should NOT re-test what rust-nostr already tests.**
45
46Instead of writing 50+ tests for NIP-01 compliance, we write:
47- **6 smoke tests** to verify the relay works at all
48- **12 GRASP-specific tests** for repository announcement logic
49
50This is pragmatic because:
511. We're using a battle-tested library, not implementing NIP-01 from scratch
522. Our value is GRASP protocol logic, not Nostr basics
533. Comprehensive NIP-01 testing would be 80% redundant work
544. Other GRASP implementations (Go, Python) will also use tested Nostr libraries
55
56---
57
58## Proposed Test Structure
59
60### NIP-01 Smoke Tests (6 tests)
61
62**Purpose:** Verify basic relay functionality
63
641. ✅ `websocket_connection` - Can connect to `/`
652. ✅ `send_receive_event` - Can send EVENT, get OK response
663. ✅ `create_subscription` - Can send REQ, receive EOSE
674. ✅ `close_subscription` - Can close subscriptions
685. ✅ `reject_invalid_event` - Rejects events with bad signatures
696. ✅ `reject_invalid_event_id` - Rejects events with wrong IDs
70
71**Coverage:** Basic relay works, events can be sent/received
72
73### GRASP-01 Specific Tests (12 tests)
74
75**Purpose:** Verify GRASP protocol requirements
76
777. ✅ `accepts_repository_announcement` - Accepts NIP-34 kind 30617
788. ✅ `accepts_repository_state` - Accepts NIP-34 kind 30618
799. ✅ `rejects_announcement_without_clone_tag` - Enforces clone tag
8010. ✅ `rejects_announcement_without_relay_tag` - Enforces relay tag
8111. ✅ `accepts_announcement_with_multiple_clones` - Handles multiple URLs
8212. ✅ `accepts_events_tagging_announcement` - Accepts related events
8313. ✅ `accepts_events_tagged_by_announcement` - Accepts tagged events
8414. ✅ `rejects_events_tagging_rejected_announcement` - Rejects orphans
8515. ✅ `query_announcements_by_identifier` - Can query repos
8616. ✅ `query_state_events` - Can query state
8717. ✅ `state_replaces_previous` - Replaceable events work
8818. ✅ `concurrent_event_submission` - No race conditions
89
90**Coverage:** GRASP policy enforcement, repository lifecycle
91
92---
93
94## Proposed Implementation
95
96### Structure
97
98```
99grasp-compliance-tests/ ← Standalone, reusable crate
100├── src/
101│ ├── lib.rs ← Public API
102│ ├── client.rs ← Test client (HTTP/WS/Git)
103│ ├── assertions.rs ← Spec-based assertions
104│ ├── fixtures.rs ← Event/repo builders
105│ └── specs/
106│ ├── nip01_smoke.rs ← 6 smoke tests
107│ └── grasp_01.rs ← 12 GRASP tests
108└── examples/
109 └── test_server.rs ← Test any GRASP server
110```
111
112### Key Features
113
1141. **Reusable**: Can test ngit-grasp, ngit-relay, or any GRASP implementation
1152. **Spec-Mirrored**: Test names and comments cite exact spec lines
1163. **Clear Failures**: Failures show requirement + what went wrong
1174. **Exportable**: Publish as `grasp-compliance-tests` crate
118
119### Example Usage
120
121```rust
122use grasp_compliance_tests::*;
123
124#[tokio::main]
125async fn main() {
126 let client = GraspTestClient::new("http://localhost:8080");
127
128 // Run smoke tests
129 let smoke = test_nip01_smoke(&client).await;
130 smoke.print_report();
131
132 // Run GRASP tests
133 let grasp = test_grasp_01_relay(&client).await;
134 grasp.print_report();
135}
136```
137
138### Example Output
139
140```
141GRASP-01: Relay Requirements
142════════════════════════════════════════════════════════════
143
144✓ accepts_repository_announcement (GRASP-01:9-10)
145 Requirement: MUST accept NIP-34 repository announcements
146 Duration: 45ms
147
148✗ rejects_announcement_without_clone_tag (GRASP-01:12-13)
149 Requirement: MUST reject announcements without clone tag
150 Error: Event was accepted but should have been rejected
151 Duration: 28ms
152
153Results: 11/12 passed (91.7%)
154```
155
156---
157
158## Can We Reuse rust-nostr Tests?
159
160### Direct Reuse: No
161
162- Their tests are internal to their crates
163- They test library functions, not running servers
164- Not designed for external use
165
166### Indirect Reuse: Yes
167
168We can leverage their patterns:
169
170```rust
171// Use their event builders
172use nostr_sdk::prelude::*;
173
174let event = EventBuilder::new(Kind::Custom(30617), "", [
175 Tag::identifier("my-repo"),
176 Tag::custom(TagKind::Custom("clone".into()), vec![domain]),
177])
178.to_event(&keys)?;
179
180// But test server acceptance, not library validation
181assert!(client.send_event(event).await?.ok);
182```
183
184**What we leverage:**
185- ✅ Event building utilities from `nostr-sdk`
186- ✅ Key generation patterns
187- ✅ Confidence that underlying validation works
188
189**What we test:**
190- 🎯 GRASP policy enforcement (our code)
191- 🎯 Repository announcement acceptance (our code)
192- 🎯 Integration between relay and Git service (our code)
193
194---
195
196## Timeline & Approach
197
198### Option A: Test-First (Recommended)
199
200**Week 1:**
201- Create `grasp-compliance-tests/` crate
202- Implement test client (HTTP/WebSocket)
203- Write all 18 tests (they will fail)
204
205**Week 2:**
206- Create ngit-grasp skeleton
207- Wire up nostr-relay-builder
208- Implement GRASP policies
209
210**Week 3:**
211- Fix failing tests
212- Add missing functionality
213- Iterate until green
214
215**Week 4:**
216- Polish and document
217- Extract reusable patterns
218- Prepare for next GRASP-01 requirements
219
220### Option B: Parallel Development
221
222Build test tool and implementation simultaneously.
223
224### Option C: Implementation-First
225
226Build ngit-grasp first, then create tests.
227
228**I recommend Option A** because:
229- Tests serve as executable specification
230- Forces thinking through edge cases early
231- Ensures testability from day one
232- Tests are immediately reusable by others
233
234---
235
236## Benefits of This Approach
237
238### 1. Focused Testing
239- 18 tests vs. 100+ redundant tests
240- Test GRASP logic, not generic Nostr
241- Fast execution (seconds, not minutes)
242
243### 2. Reusable Tool
244- Any GRASP implementation can use it
245- Go, Rust, Python, JavaScript
246- Publish as standalone crate
247- Community contribution opportunity
248
249### 3. Clear Failures
250- Cite exact spec requirements
251- Show expected vs. actual
252- Actionable error messages
253
254### 4. Maintainable
255- Tests mirror spec structure
256- Easy to add GRASP-02, GRASP-05 tests
257- Update tests when spec updates
258
259### 5. Proof of Concept
260- Demonstrates architecture viability
261- Validates inline authorization approach
262- Shows rust-nostr integration works
263
264---
265
266## Questions for Decision
267
268### 1. Scope Confirmation
269**Do you agree with smoke tests for NIP-01 rather than comprehensive testing?**
270
271- ✅ Yes: 6 smoke tests + 12 GRASP tests (18 total)
272- ❌ No: Write comprehensive NIP-01 tests (50+ tests)
273
274### 2. Implementation Approach
275**Which approach should we take?**
276
277- **A**: Test-first (write tests, then implement)
278- **B**: Parallel (tests and implementation together)
279- **C**: Implementation-first (code first, tests later)
280
281### 3. Crate Structure
282**Should the compliance tests be separate from day one?**
283
284- **Separate**: `grasp-compliance-tests/` as standalone crate
285- **Integrated**: Start in `ngit-grasp/tests/`, extract later
286- **Hybrid**: Some in both places
287
288### 4. Fixture Strategy
289**How should we generate test data?**
290
291- **Deterministic**: Same keys/events every run (reproducible)
292- **Random**: New keys each run (finds more bugs)
293- **Configurable**: Support both modes
294
295---
296
297## Recommended Next Steps
298
2991. ✅ **Review this proposal** - Confirm approach and scope
3002. ✅ **Answer decision questions** - Guide implementation direction
3013. ✅ **Create test tool skeleton** - Set up project structure
3024. ✅ **Implement smoke tests** - Verify basic connectivity
3035. ✅ **Implement GRASP tests** - Test repository announcements
3046. ✅ **Create minimal ngit-grasp** - Wire up nostr-relay-builder
3057. ✅ **Iterate until green** - Fix failing tests
3068. ✅ **Document and polish** - Prepare for next requirements
307
308---
309
310## Files Created
311
3121. **COMPLIANCE_TEST_PROPOSAL.md** - Detailed proposal with code examples
3132. **REPORT_COMPLIANCE_TESTING.md** - This executive summary
314
315---
316
317## Ready to Proceed?
318
319Please review and advise on:
320
3211. ✅ **Scope**: Agree with smoke tests approach?
3222. ✅ **Approach**: Test-first (A), parallel (B), or implementation-first (C)?
3233. ✅ **Priority**: Any specific tests to prioritize?
3244. ✅ **Changes**: Any modifications to the 18 proposed tests?
325
326Once you confirm the approach, I'll begin implementation immediately.
327
328---
329
330**Status:** ⏸️ Awaiting your decision on approach and scope