upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md
diff options
context:
space:
mode:
authorDanConwayDev <DanConwayDev@protonmail.com>2025-12-03 11:19:40 +0000
committerDanConwayDev <DanConwayDev@protonmail.com>2025-12-03 11:19:40 +0000
commit2eaff5b79fed364d5eba5eb38e4b7bf76326884d (patch)
treedeacd6294f8860096ee82ee76930204efd65e33c /docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md
parent57bc8cd9c021feaf08e139e8fb62800bc476068e (diff)
remove docs archive
Diffstat (limited to 'docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md')
-rw-r--r--docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md590
1 files changed, 0 insertions, 590 deletions
diff --git a/docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md b/docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md
deleted file mode 100644
index aaca16b..0000000
--- a/docs/archive/2025-11-04-evening/2025-11-04-ngit-grasp-implementation-plan.md
+++ /dev/null
@@ -1,590 +0,0 @@
1**ARCHIVED: 2025-11-04**
2**Reason:** Decided to validate grasp-audit against ngit-relay first
3**See:** docs/archive/2025-11-04-test-strategy-decision.md for rationale
4
5---
6
7# TDD Plan for GRASP-01 Git Backend
8
9**Date:** 2025-11-04
10**Status:** ARCHIVED - Superseded by test-first approach
11**Goal:** Implement Git Smart HTTP backend with inline authorization using TDD
12
13---
14
15## Current State
16
17### What We Have
18- ✅ NIP-01 compliant Nostr relay (working, tested)
19- ✅ NIP-34 event handling (announcements accepted)
20- ✅ Storage layer (in-memory + disk paths configured)
21- ✅ Test infrastructure (integration tests with auto relay management)
22- ✅ grasp-audit compliance testing library
23
24### What We Need
25- ❌ Git Smart HTTP protocol handler
26- ❌ Git repository management (init, receive-pack, upload-pack)
27- ❌ Push authorization (validate against Nostr state events)
28- ❌ Integration with existing Nostr relay
29
30---
31
32## Tool Selection Analysis
33
34### Option 1: git2-rs (libgit2 bindings)
35**Pros:**
36- ✅ Pure Rust bindings to battle-tested libgit2
37- ✅ Full Git functionality (init, push, pull, refs, objects)
38- ✅ Thread-safe, well-maintained
39- ✅ Used by cargo, widely deployed
40- ✅ Can intercept and validate operations programmatically
41
42**Cons:**
43- ❌ Requires libgit2 system dependency
44- ❌ Higher-level API - may be overkill for our needs
45- ❌ Harder to intercept low-level protocol for inline auth
46
47**Use Cases:**
48- Repository initialization
49- Reading/writing refs
50- Object storage queries
51- Validation of commits/trees
52
53### Option 2: Standard git in subprocess
54**Pros:**
55- ✅ Uses system git (already available)
56- ✅ No additional dependencies
57- ✅ Battle-tested Git implementation
58- ✅ Easy to spawn for upload-pack/receive-pack
59
60**Cons:**
61- ❌ Harder to intercept for inline authorization
62- ❌ Must parse git protocol to validate pushes
63- ❌ Subprocess overhead
64- ❌ Complex error handling
65
66**Use Cases:**
67- git-upload-pack (clone, fetch)
68- git-receive-pack (push) - but need to intercept
69
70### Option 3: git-http-backend crate
71**Pros:**
72- ✅ Purpose-built for Git Smart HTTP
73- ✅ Handles protocol parsing
74- ✅ Works with system git
75- ✅ Can intercept receive-pack before spawning git
76
77**Cons:**
78- ❌ Less mature (but we're already planning to use it per README)
79- ❌ Still need to parse pack protocol for validation
80
81**Use Cases:**
82- HTTP endpoint handling
83- Protocol negotiation
84- Spawning git processes
85
86### Option 4: Hybrid Approach (RECOMMENDED)
87**Combination:**
881. **git-http-backend** - HTTP protocol handling
892. **git2-rs** - Repository management, ref validation
903. **System git** - Actual pack operations (upload-pack/receive-pack)
91
92**Why Hybrid:**
93- git-http-backend handles HTTP → Git protocol translation
94- git2 for safe repository operations (init, refs, validation)
95- System git for pack operations (proven, fast)
96- We intercept at the HTTP layer before spawning git
97
98**Architecture:**
99```
100HTTP Request → git-http-backend → Our Auth Layer → git2/system git
101
102 Nostr Relay
103 (state validation)
104```
105
106---
107
108## Recommended Approach: Hybrid
109
110### Dependencies to Add
111```toml
112[dependencies]
113# Git operations
114git2 = "0.20" # Repository management, refs
115# git-http-backend - TBD (research if available, or implement minimal)
116
117[dev-dependencies]
118tempfile = "3.8" # Temporary repos for testing
119```
120
121### Why This Works
122
1231. **git2 for Repository Management:**
124 - Initialize bare repos when announcements arrive
125 - Read/write refs safely
126 - Query repository state
127 - Validate commits exist
128
1292. **System git for Pack Operations:**
130 - Spawn `git-upload-pack` for clones/fetches (read-only, safe)
131 - Spawn `git-receive-pack` ONLY after auth passes
132 - Leverage proven pack protocol implementation
133
1343. **Inline Authorization:**
135 - Parse HTTP request to extract ref updates
136 - Query Nostr relay for latest state event
137 - Validate push matches state
138 - Only spawn git-receive-pack if authorized
139
140---
141
142## TDD Implementation Plan
143
144### Phase 1: Repository Management (git2)
145**Goal:** Create and manage bare Git repositories
146
147**Tests:**
1481. ✅ Create bare repository when announcement received
1492. ✅ Initialize with proper config (bare, shared)
1503. ✅ Set HEAD from state event
1514. ✅ Read refs from repository
1525. ✅ Write refs to repository
1536. ✅ Query if commit exists in repository
154
155**Implementation:**
156```rust
157// src/git/repository.rs
158
159pub struct GitRepository {
160 path: PathBuf,
161 repo: git2::Repository,
162}
163
164impl GitRepository {
165 pub fn init_bare(path: PathBuf) -> Result<Self>;
166 pub fn set_head(ref_name: &str) -> Result<()>;
167 pub fn get_ref(&self, name: &str) -> Result<Option<String>>;
168 pub fn set_ref(&self, name: &str, oid: &str) -> Result<()>;
169 pub fn has_commit(&self, oid: &str) -> Result<bool>;
170}
171```
172
173**Test Example:**
174```rust
175#[test]
176fn test_init_bare_repository() {
177 let temp = TempDir::new().unwrap();
178 let repo = GitRepository::init_bare(temp.path().to_path_buf()).unwrap();
179
180 assert!(temp.path().join("HEAD").exists());
181 assert!(temp.path().join("config").exists());
182 assert!(temp.path().join("objects").exists());
183 assert!(temp.path().join("refs").exists());
184}
185```
186
187### Phase 2: Git Protocol Parsing
188**Goal:** Parse Git Smart HTTP protocol for authorization
189
190**Tests:**
1911. ✅ Parse info/refs request
1922. ✅ Parse upload-pack request (clone/fetch)
1933. ✅ Parse receive-pack request (push)
1944. ✅ Extract ref updates from receive-pack
1955. ✅ Extract capabilities from request
196
197**Implementation:**
198```rust
199// src/git/protocol.rs
200
201pub struct RefUpdate {
202 pub old_oid: String,
203 pub new_oid: String,
204 pub ref_name: String,
205}
206
207pub fn parse_receive_pack_request(body: &[u8]) -> Result<Vec<RefUpdate>>;
208pub fn parse_capabilities(body: &[u8]) -> Result<Vec<String>>;
209```
210
211**Test Example:**
212```rust
213#[test]
214fn test_parse_receive_pack_single_ref() {
215 let body = b"0000000000000000000000000000000000000000 \
216 a1b2c3d4e5f6789012345678901234567890abcd \
217 refs/heads/main\0 report-status\n";
218
219 let updates = parse_receive_pack_request(body).unwrap();
220 assert_eq!(updates.len(), 1);
221 assert_eq!(updates[0].ref_name, "refs/heads/main");
222 assert_eq!(updates[0].new_oid, "a1b2c3d4e5f6789012345678901234567890abcd");
223}
224```
225
226### Phase 3: Authorization Logic
227**Goal:** Validate pushes against Nostr state events
228
229**Tests:**
2301. ✅ Get maintainers from announcement
2312. ✅ Get maintainers recursively
2323. ✅ Handle circular maintainer references
2334. ✅ Validate ref update matches state
2345. ✅ Validate branch push matches state
2356. ✅ Validate tag push matches state
2367. ✅ Accept push to refs/nostr/*
2378. ✅ Reject push not matching state
2389. ✅ Reject push from non-maintainer
239
240**Implementation:**
241```rust
242// src/git/authorization.rs
243
244pub struct PushValidator {
245 storage: Storage,
246}
247
248impl PushValidator {
249 pub async fn validate_push(
250 &self,
251 npub: &str,
252 identifier: &str,
253 updates: &[RefUpdate],
254 ) -> Result<()>;
255
256 async fn get_maintainers(&self, npub: &str, identifier: &str) -> Vec<String>;
257 async fn get_latest_state(&self, npub: &str, identifier: &str) -> Option<StateEvent>;
258 fn validate_ref_update(&self, state: &StateEvent, update: &RefUpdate) -> Result<()>;
259}
260```
261
262**Test Example:**
263```rust
264#[tokio::test]
265async fn test_validate_matching_push() {
266 let storage = test_storage().await;
267
268 // Create announcement and state
269 let announcement = create_announcement("alice", "repo1");
270 let state = create_state("alice", "repo1")
271 .branch("main", "a1b2c3d4...");
272
273 storage.store_event(announcement).await.unwrap();
274 storage.store_event(state).await.unwrap();
275
276 // Validate matching push
277 let validator = PushValidator::new(storage);
278 let update = RefUpdate {
279 old_oid: "0000...".into(),
280 new_oid: "a1b2c3d4...".into(),
281 ref_name: "refs/heads/main".into(),
282 };
283
284 let result = validator.validate_push("alice", "repo1", &[update]).await;
285 assert!(result.is_ok());
286}
287
288#[tokio::test]
289async fn test_reject_mismatched_push() {
290 let storage = test_storage().await;
291
292 // State points to commit A
293 let state = create_state("alice", "repo1")
294 .branch("main", "aaaa1111...");
295 storage.store_event(state).await.unwrap();
296
297 // Try to push commit B
298 let validator = PushValidator::new(storage);
299 let update = RefUpdate {
300 old_oid: "0000...".into(),
301 new_oid: "bbbb2222...".into(), // Different!
302 ref_name: "refs/heads/main".into(),
303 };
304
305 let result = validator.validate_push("alice", "repo1", &[update]).await;
306 assert!(result.is_err());
307 assert!(result.unwrap_err().to_string().contains("state"));
308}
309```
310
311### Phase 4: HTTP Handlers
312**Goal:** Serve Git Smart HTTP protocol
313
314**Tests:**
3151. ✅ GET /npub/repo.git/info/refs?service=git-upload-pack
3162. ✅ POST /npub/repo.git/git-upload-pack (clone/fetch)
3173. ✅ POST /npub/repo.git/git-receive-pack (push with auth)
3184. ✅ Return 403 for unauthorized push
3195. ✅ Return 404 for non-existent repository
3206. ✅ Set correct content-type headers
3217. ✅ Include CORS headers
322
323**Implementation:**
324```rust
325// src/git/handler.rs
326
327pub async fn handle_info_refs(
328 npub: String,
329 identifier: String,
330 service: String,
331) -> Result<Response>;
332
333pub async fn handle_upload_pack(
334 npub: String,
335 identifier: String,
336 body: Bytes,
337) -> Result<Response>;
338
339pub async fn handle_receive_pack(
340 npub: String,
341 identifier: String,
342 body: Bytes,
343 validator: PushValidator,
344) -> Result<Response>;
345```
346
347**Test Example:**
348```rust
349#[tokio::test]
350async fn test_info_refs_returns_correct_headers() {
351 let app = test_app().await;
352
353 // Create repository
354 app.create_repo("alice", "test-repo").await;
355
356 // Request info/refs
357 let response = app.get("/alice-npub/test-repo.git/info/refs?service=git-upload-pack").await;
358
359 assert_eq!(response.status(), 200);
360 assert_eq!(
361 response.headers().get("content-type").unwrap(),
362 "application/x-git-upload-pack-advertisement"
363 );
364 assert_eq!(
365 response.headers().get("access-control-allow-origin").unwrap(),
366 "*"
367 );
368}
369
370#[tokio::test]
371async fn test_receive_pack_rejects_unauthorized() {
372 let app = test_app().await;
373
374 // Create repo with state
375 let (announcement, state) = app.create_repo_with_state()
376 .branch("main", "aaaa1111...")
377 .build()
378 .await;
379
380 // Try to push different commit
381 let body = create_receive_pack_request()
382 .ref_update("refs/heads/main", "0000...", "bbbb2222...")
383 .build();
384
385 let response = app.post(
386 &format!("/{}/repo.git/git-receive-pack", announcement.author_npub()),
387 body
388 ).await;
389
390 assert_eq!(response.status(), 403);
391}
392```
393
394### Phase 5: Integration with Nostr Events
395**Goal:** Automatic repository creation and state updates
396
397**Tests:**
3981. ✅ Create repository when announcement received
3992. ✅ Update HEAD when state event received
4003. ✅ Handle announcement updates (maintainers change)
4014. ✅ Clean up orphaned refs/nostr/* refs
402
403**Implementation:**
404```rust
405// src/nostr/events.rs (extend existing)
406
407async fn handle_repository_announcement(event: &Event, storage: &Storage) -> Result<()> {
408 // Extract npub and identifier
409 // Create bare repository
410 // Store event
411}
412
413async fn handle_repository_state(event: &Event, storage: &Storage) -> Result<()> {
414 // Find repository
415 // Update refs to match state
416 // Update HEAD
417}
418```
419
420**Test Example:**
421```rust
422#[tokio::test]
423async fn test_repository_created_on_announcement() {
424 let app = test_app().await;
425
426 let announcement = create_announcement("alice", "test-repo")
427 .with_clone_tag(app.domain())
428 .build();
429
430 app.send_event(announcement).await.unwrap();
431
432 // Wait for async processing
433 tokio::time::sleep(Duration::from_millis(100)).await;
434
435 // Verify repository exists
436 let repo_path = app.git_data_path()
437 .join("alice-npub")
438 .join("test-repo.git");
439
440 assert!(repo_path.exists());
441 assert!(repo_path.join("HEAD").exists());
442}
443```
444
445### Phase 6: End-to-End Testing
446**Goal:** Test with real Git client
447
448**Tests:**
4491. ✅ Clone repository with git client
4502. ✅ Fetch from repository
4513. ✅ Push to repository (authorized)
4524. ✅ Push rejected (unauthorized)
4535. ✅ Multiple concurrent operations
454
455**Implementation:**
456```rust
457// tests/e2e/git_client.rs
458
459#[tokio::test]
460async fn test_real_git_clone() {
461 let app = test_app().await;
462
463 // Setup repository with content
464 let (announcement, _) = app.create_repo_with_commits()
465 .commit("Initial", "README.md", "# Test")
466 .build()
467 .await;
468
469 // Clone with real git
470 let temp = TempDir::new().unwrap();
471 let url = format!(
472 "http://{}/{}/{}.git",
473 app.domain(),
474 announcement.author_npub(),
475 announcement.identifier()
476 );
477
478 let output = Command::new("git")
479 .args(&["clone", &url])
480 .current_dir(&temp)
481 .output()
482 .await
483 .unwrap();
484
485 assert!(output.status.success());
486
487 let cloned_path = temp.path().join(announcement.identifier());
488 assert!(cloned_path.exists());
489 assert!(cloned_path.join("README.md").exists());
490}
491```
492
493---
494
495## Testing Strategy
496
497### Unit Tests (40%)
498- Git repository operations (git2)
499- Protocol parsing
500- Authorization logic
501- Pure functions, no I/O
502
503### Integration Tests (30%)
504- HTTP handlers with test server
505- Repository + Nostr event interaction
506- Multi-maintainer flows
507- State validation
508
509### Compliance Tests (20%)
510- GRASP-01 Git requirements
511- Use grasp-audit library
512- Spec-driven assertions
513
514### E2E Tests (10%)
515- Real git client operations
516- End-to-end workflows
517- Performance testing
518
519---
520
521## Implementation Order
522
523### Week 1: Foundation
5241. Add git2 dependency
5252. Implement GitRepository (Phase 1)
5263. Write unit tests for repository operations
5274. Test repository creation from announcements
528
529### Week 2: Protocol & Authorization
5301. Implement protocol parsing (Phase 2)
5312. Implement authorization logic (Phase 3)
5323. Write unit tests for both
5334. Integration tests for validation
534
535### Week 3: HTTP & Integration
5361. Implement HTTP handlers (Phase 4)
5372. Integrate with Nostr events (Phase 5)
5383. Integration tests for full flow
5394. CORS and error handling
540
541### Week 4: E2E & Polish
5421. E2E tests with real git (Phase 6)
5432. Performance testing
5443. GRASP-01 compliance testing
5454. Documentation and examples
546
547---
548
549## Success Criteria
550
551### Functional
552- ✅ Clone repository via HTTP
553- ✅ Push authorized commits
554- ✅ Reject unauthorized pushes
555- ✅ Support multi-maintainer
556- ✅ Support refs/nostr/* for PRs
557
558### Quality
559- ✅ >80% unit test coverage
560- ✅ All integration tests pass
561- ✅ GRASP-01 compliance 100%
562- ✅ E2E tests with real git
563
564### Performance
565- ✅ Clone 1MB repo < 1s
566- ✅ Push validation < 100ms
567- ✅ 100 concurrent ops without errors
568
569---
570
571## Next Steps
572
5731. **Review this plan** - Does the hybrid approach make sense?
5742. **Start Phase 1** - Add git2, implement GitRepository
5753. **Write first test** - test_init_bare_repository
5764. **Iterate with TDD** - Red → Green → Refactor
577
578---
579
580## Questions for Review
581
5821. **Hybrid approach?** git2 + system git + HTTP layer - good balance?
5832. **git-http-backend crate?** Worth using or implement minimal HTTP layer?
5843. **Authorization granularity?** Validate per-ref or entire push?
5854. **Error messages?** How detailed for push rejections?
5865. **Testing scope?** Is 6 phases reasonable for first iteration?
587
588---
589
590**Ready to proceed?** Let me know if this plan looks good, or if you'd like to adjust the approach!