| Age | Commit message (Collapse) | Author |
|---|
|
Improves observability when pushes are rejected due to state events that
only partially match the pushed refs. Previously, logs only showed 'No
state event found' even when state events existed but didn't match.
Changes:
- Add diagnose_state_mismatch() to explain why state events don't match
- Log specific reasons: missing refs, wrong SHAs, or extra refs
- Update rejection message to 'No matching state event found' (more accurate)
- Add 4 unit tests for diagnostic function
Example diagnostic output:
WARN State event abc123 from authorized author doesn't match push:
refs/heads/main missing (state declares 9cc3d93b)
This addresses the issue where a push with only refs/heads/test was
rejected because the state event also declared refs/heads/main, but
logs didn't explain why the match failed.
|
|
This merge includes critical bug fixes and comprehensive migration tooling
developed during the relay.ngit.dev migration effort.
Bug Fixes:
- Fix git protocol error handling to return HTTP 200 with ERR pkt-line
- Fix naughty list false positives and DNS failure identification
- Fix database query filters in load_existing_events (remove .since())
- Fix OID fetch tracking to distinguish 0 OIDs from successful fetches
- Fix purgatory event source tracking for filtered expiry logging
- Implement OID retry logic for 'not our ref' errors
Migration Tools & Documentation:
- Complete 5-phase migration analysis pipeline with orchestration script
- Phase 1: Event fetching from source relay
- Phase 2: Git sync verification
- Phase 3: Categorization and relay comparison
- Phase 4: Log extraction (parse failures, purgatory expiry)
- Phase 5: Action classification for migration decisions
- Comprehensive migration guide with lessons learned
- Troubleshooting guide for permission and corruption issues
Configuration:
- Add NGIT_LOG_LEVEL configuration option
- Update git throttle limits to 60/minute
- Improve logging throughout for better observability
|
|
Add EventSource enum (Direct/Sync) to purgatory entries to distinguish
between user-submitted events and sync-fetched events. This enables:
- WARN-level logging for direct submissions that expire (user should know)
- DEBUG-level logging for sync-fetched expirations (expected behavior)
- Source upgrade from Sync→Direct if user submits after sync
- Expiry timer reset on source upgrade (fresh 30-min window for user)
The source is included in [PURGATORY_EXPIRED] logs as source=direct or
source=sync for easy filtering.
|
|
- Add [PARSE_FAIL] logging when event parsing fails
- Add [PURGATORY_EXPIRED] logging when repos expire from purgatory
- Logs include: kind, event_id, repo, npub, reason
- Supports Phase 4 migration scripts (30-extract-*.sh)
- All 382 tests pass
|
|
Implement save/restore functionality for purgatory state to prevent
event loss during relay restarts. Events in purgatory (state events,
PR events, and expired events) are now saved to disk on graceful
shutdown and restored on startup.
Key features:
- Serialize purgatory state to JSON (purgatory-state.json)
- Time conversion helpers for Instant <-> Duration serialization
- Restore with downtime adjustment (preserves remaining TTL)
- Graceful degradation (missing/corrupted files don't crash)
- File cleanup after successful restore
- get_all_identifiers() for re-queueing after restore
Files:
- src/purgatory/persistence.rs: Time conversion helpers
- src/purgatory/types.rs: Serialization derives
- src/purgatory/mod.rs: save_to_disk/restore_from_disk methods
Tests: 15 unit tests covering serialization, downtime, edge cases
|
|
|
|
Adds expired event tracking to prevent proactive sync from repeatedly
fetching and re-adding events that expired from purgatory without
finding git data.
Key features:
- Track expired events for 7 days to prevent re-sync loops
- Distinguish synced vs user-submitted events (via socket address)
- Allow users to retry expired events (git data might now be available)
- Reject synced expired events (prevents infinite loop)
- Daily cleanup of expired event records older than 7 days
Implementation:
- Added expired_events: DashMap<EventId, Instant> to Purgatory
- Updated event_ids() to include both purgatory + expired events
- Added is_expired(), mark_expired(), cleanup_expired_events()
- Updated cleanup() to mark expired events automatically
- Added is_synced detection in WritePolicy (localhost:0 = synced)
- Policy layer checks is_synced && is_expired() before rejecting
Behavior:
- Negentropy: Filters expired events before fetching (optimal)
- REQ+EOSE: Rejects synced expired events at policy layer
- User submissions: Always allowed to retry (skip expired check)
Testing:
- Added 5 new tests for expired event tracking
- All 222 tests passing
Fixes the infinite re-sync loop where events without git data would
expire, get synced again, expire again, repeat forever.
|
|
- Prefix unused variable auth_result with underscore
- Prefix unused field git_data_path with underscore in Purgatory struct
- Add #[allow(clippy::too_many_arguments)] to handle_receive_pack
- Replace len() >= 1 with !is_empty()
- Replace .last() with .next_back() on DoubleEndedIterator
- Fix doc list item overindentation
- Replace map_or(true, ...) with is_none_or(...)
- Replace map_or(false, ...) with is_some_and(...)
|
|
Implements Phase 3 of the purgatory sync integration test plan.
Key changes:
- Add immediate sync triggering for sync-received events that go to
purgatory (instead of default 3-minute delay for user-submitted events)
- TestRelay now respects RUST_LOG environment variable for debugging
- New test verifies end-to-end flow: state event syncs from source relay,
enters purgatory, git data is fetched from source's clone URL, and
event is released and served
|
|
Phase 13 of purgatory-sync-redesign:
- Add sync loop startup in main.rs (RealSyncContext + ThrottleManager + start_sync_loop)
- Update add_state() and add_pr() to automatically enqueue for background sync
- Remove start_state_sync() call from state.rs (now handled by sync loop)
- Remove orphaned legacy functions: sync_state_git_data, fetch_missing_oids_from_server,
get_most_complete_local_repo, identify_missing_oids, get_date_of_most_recent_commit_on_default_branch
- Clean up unused imports in purgatory/mod.rs
|
|
|
|
Implement the unified function that handles all post-git-data-available
processing, regardless of how data arrived (git push or purgatory sync).
This function:
- Discovers satisfiable events from purgatory (state and PR events)
- Syncs OIDs to authorized owner repos
- Aligns refs and sets HEAD
- Saves events to database
- Notifies WebSocket subscribers
- Removes from purgatory
New additions:
- ProcessResult struct for tracking processing outcomes
- process_newly_available_git_data async function in src/git/sync.rs
- Helper functions: extract_identifier_from_repo_path, extract_identifier_from_pr_event
- Purgatory::find_prs_for_identifier method for PR event discovery
- Unit tests for all helper functions
Also fixes:
- Simplified extract_domain to avoid url crate dependency
- Removed unused imports in sync/loop.rs
|
|
- Add sync_queue field to Purgatory struct for tracking identifiers that need
background git data fetching
- Implement enqueue_sync() with debouncing - resets attempt_count and updates
next_attempt when new events arrive for an identifier already in queue
- Add enqueue_sync_default() for user-submitted events (3 minute delay to wait
for git push)
- Add enqueue_sync_immediate() for sync-triggered events (500ms delay for
batching burst arrivals)
- Implement has_pending_events() to check if an identifier has state events
or PR events in purgatory
- Add helper methods: sync_queue(), remove_from_sync_queue(), sync_queue_size()
- Add unit tests for debouncing behavior and pending event detection
|
|
Implement the sync queue entry struct that tracks sync state per identifier:
- next_attempt: when the next sync should be attempted
- attempt_count: for backoff calculation (resets on new events)
- in_progress: prevents concurrent syncs for same identifier
Backoff schedule: 20s → 40s → 80s → 120s (capped at 2 minutes)
This is the foundation for the identifier-based purgatory sync system
that will replace the current per-event syncing approach.
|
|
|
|
|
|
|
|
|
|
don't save new events destined for purgatory events directly to db
or serve on websockets
don't download events already in purgatory via negentropy sync
|
|
|