| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
Main lib (src/):
- Add #[allow(dead_code)] for build_info field (stored to prevent Prometheus unregistration)
- Add #[allow(dead_code)] for first_seen field (reserved for future rate limiting)
- Replace .or_insert_with(RelaySyncNeeds::default) with .or_default()
- Replace manual div_ceil implementations with .div_ceil(100)
Test code (tests/):
- Replace .expect(&format!(...)) with .unwrap_or_else(|_| panic!(...))
- Remove needless borrows in fetch_metrics() calls
- Add #[allow(dead_code)] and #[allow(unused_imports)] to test helpers module
grasp-audit:
- Apply cargo fmt to fix formatting
|
|
|
|
|
|
The catchup sync mechanism (reconnection with since filter) is implemented
in src/sync/mod.rs handle_connect_or_reconnect(), but cannot be reliably
integration tested with current infrastructure:
- TestRelay uses in-memory database (events lost on stop)
- No way to force WebSocket disconnection without stopping relay
- Stopping syncing relay creates new instance (fresh sync, not catchup)
Convert the skeleton test file to comprehensive documentation explaining:
- How catchup sync works (since filter on reconnect)
- The 15-minute quick reconnect window logic
- Why integration testing is not feasible
- Alternative approaches that could enable testing
- Related tests that cover adjacent functionality
|
|
|
|
Previously, events were classified as 'startup' or 'live' based on whether
they came from a bootstrap relay (is_bootstrap flag). This meant ALL events
from bootstrap relays were counted as 'startup', even events received after
the initial sync completed.
Now events are classified based on whether EOSE (End Of Stored Events) has
been received for that connection:
- Events BEFORE EOSE → 'startup' (historical events during initial sync)
- Events AFTER EOSE → 'live' (new events via real-time subscription)
This enables the test_live_sync_event_count test which validates that events
received after sync connection is established are counted as live events.
Also removed the #[ignore] attribute from test_live_sync_event_count since
the metrics are now properly wired up.
|
|
- Add periodic health check in RelayConnection::run_event_loop that polls
nostr-sdk's relay.is_connected() every second to detect dead connections
- When event channel closes without explicit Closed/Shutdown, send
DisconnectNotification to SyncManager (fixes case where TCP drops silently)
- Enable test_relay_connected_status test which validates the
ngit_sync_relay_connected metric correctly reflects connection state
The issue was that when a remote relay stops abruptly, nostr-sdk's
notification receiver blocks indefinitely waiting for data. TCP disconnect
detection without keepalive can take minutes. The health check polls
nostr-sdk's internal relay status which detects disconnection promptly.
|
|
Root cause: Both Metrics::new() and SyncManager::new() were trying to register
SyncMetrics with the same Prometheus registry. The second registration failed
silently, leaving SyncManager.metrics = None, so record_connection_attempt()
calls were no-ops.
Changes:
- SyncManager::new() now accepts Option<SyncMetrics> instead of Option<&Registry>
- main.rs passes already-registered sync metrics from Metrics to SyncManager
- Simplified test_connection_failure_increments_counter assertion
- Marked 3 tests as #[ignore] pending relay tracking metrics wiring
Tests fixed:
- test_connection_failure_increments_counter (now counts failures)
- test_health_state_degrades_on_failure (now tracks health state)
- test_live_sync_layer3_events (already working, confirmed)
Tests ignored (future work):
- test_live_sync_event_count
- test_multi_source_aggregate_counts
- test_relay_connected_status
|
|
|
|
Changes:
- Fix connection attempt metrics: record success/failure based on actual
connection result instead of pre-emptively recording failure
- Add health tracker integration on connection failure: call
record_failure() and record_health_state() in error path
- Add connection verification in relay_connection.rs: wait 500ms after
connect() then verify is_connected() to detect silent failures
- Add configurable disconnect check interval via
NGIT_SYNC_DISCONNECT_CHECK_INTERVAL_SECS env var
- Update TestRelay with fast test settings: startup_delay=0, jitter=0,
disconnect_check_interval=1s
- Add debug output to metrics tests for investigation
Note: Tests may still fail due to 5-second base backoff in health tracker.
A follow-up task will add NGIT_SYNC_BASE_BACKOFF_SECS config parameter
to allow faster test cycles.
Related: metrics-wiring-plan.md Tasks 1 & 2
|
|
|
|
Deleted 12 existence-only tests that provided zero confidence:
- test_sync_metrics_exposed
- test_sync_metric_names_present
- test_connection_metrics_on_success
- test_event_sync_metrics
- test_health_state_metrics
- test_gap_event_tracking
- test_connection_failure_metrics
- test_failure_counter_increments
- test_relay_count_metrics
- test_event_source_labels_in_metrics
- test_multi_relay_load
- test_gap_events_tracked_separately
Kept 5 valuable tests:
- test_prometheus_format_valid
- test_concurrent_metrics_requests
- test_metric_values_are_numeric
- test_startup_sync_event_count
- test_metrics_availability_during_sync
Added 3 real value-checking tests (currently ignored):
- test_connection_failure_increments_counter
- test_live_sync_event_count
- test_relay_connected_status
Test results: 6 passed, 0 failed, 3 ignored
|
|
|
|
|
|
|
|
|
|
When root events (issues/patches) are received via self-subscription,
handle_root_event() was only updating the repo_sync_index directly.
This caused process_batch() to early-return when pending.is_empty(),
so Layer 3 filters for comments/replies were never created.
The fix adds root events to both:
1. repo_sync_index (for immediate availability)
2. pending queue (to trigger Layer 3 filter creation in next batch)
Critical: The pending entry must include relays from repo_sync_index
so derive_relay_targets() knows where to send Layer 3 subscriptions.
The Layer 3 test now verifies that events sent BEFORE the subscription
is established are still synced - proving subscriptions without 'since'
correctly fetch historical events.
Enabled 4 previously ignored Layer 3 tests:
- test_live_sync_layer3_events
- test_layer3_sync_with_lowercase_e_tag
- test_layer3_sync_with_uppercase_e_tag
- test_layer3_sync_with_q_tag
|
|
Enable recursive relay discovery by broadcasting synced events to
WebSocket subscribers via LocalRelay.notify_event(). This allows the
SelfSubscriber to receive 30617 announcements synced from external
relays and discover additional relay URLs to connect to.
Changes:
- Pass LocalRelay to SyncManager::new() from main.rs
- Add local_relay field to SyncManager struct
- Call notify_event() after saving synced events to database
- Enable test_recursive_relay_discovery_syncs_announcement test
The test verifies that when relay_a syncs announcement_x from bootstrap
relay_b (which lists relay_c), relay_a discovers and connects to
relay_c to sync announcement_y.
Fixes recursive relay discovery from bootstrap sync.
|
|
|
|
|
|
|
|
|
|
All tests from this file have been migrated to the new modular structure:
- tests/sync/bootstrap.rs (Tests 1, 4)
- tests/sync/discovery.rs (Tests 2, 3)
- tests/sync/live_sync.rs (Tests 5, 6, 7)
- tests/sync/tag_variations.rs (Tests 8a-c, 9a-c)
- tests/sync/catchup.rs (Test 0)
Verified: cargo test --test sync shows 19 passed, 5 ignored
|
|
|
|
Add comprehensive tests for different Layer 2 and Layer 3 tag variations:
Layer 2 tests (Tests 8a-c) - all pass:
- test_layer2_sync_with_lowercase_a_tag (standard NIP-01)
- test_layer2_sync_with_uppercase_a_tag (NIP-33)
- test_layer2_sync_with_q_tag (NIP-18 quotes)
Layer 3 tests (Tests 9a-c) - marked #[ignore]:
- test_layer3_sync_with_lowercase_e_tag (NIP-01)
- test_layer3_sync_with_uppercase_e_tag (NIP-22)
- test_layer3_sync_with_q_tag (NIP-18)
Layer 3 tests have full implementation but are ignored until
Layer 3 sync is enabled in the relay.
|
|
|
|
Create organized test structure for proactive sync:
tests/common/sync_helpers.rs (from Phase 4):
- TestClient with retry logic for connect/send
- Event builders: build_layer2_issue_event, build_layer3_comment_event
- Tag variants (a/A/q for Layer 2, e/E/q for Layer 3)
- wait_for_event_on_relay() assertion helper
- repo_coord() utility function
- Unit tests for all builders
tests/sync/mod.rs:
- Module organization for sync tests
- Documentation of test categories
tests/sync.rs:
- Main test harness including common and sync modules
tests/sync/bootstrap.rs:
- test_bootstrap_syncs_existing_layer2_events (Test 1)
- test_relay_replays_events_after_restart (Test 4)
tests/sync/discovery.rs:
- test_discovers_layer3_via_layer2 (Test 2)
- test_layer2_discovery_with_chain (Test 3 - simplified)
All 14 tests pass: cargo test --test sync
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Add SyncMetrics with full Prometheus integration
- Track sync gaps via catchup events
- Update Grafana dashboard with sync panels
- Document all sync configuration options
- Update design doc with implementation notes
|
|
- Add NegentropyService for set reconciliation
- Implement startup catchup with warm-up delay
- Implement reconnect catchup (last 3 days)
- Add daily catchup schedule with stagger
|
|
- Add SubscriptionManager for per-connection tracking
- Trigger subscription updates on new repo/PR events
- Implement consolidation when filter count > 150
|
|
- Add RelayHealthTracker with DashMap
- Implement exponential backoff (5s -> 1h max)
- Handle dead relays (24h failures -> daily retry)
- Add startup jitter to prevent thundering herd
- Add NGIT_SYNC_MAX_BACKOFF_SECS config
|
|
- Add relay discovery from stored announcements
- Implement FilterService with three-layer strategy
- Support multiple simultaneous relay connections
- Filter batching for large tag sets
|
|
- Add src/sync/ module with SyncManager
- Add NGIT_SYNC_RELAY_URL config option
- Subscribe to kind 30617 on configured relay
- Validate synced events through Nip34WritePolicy
- Integration test with two TestRelay instances
|
|
|
|
|
|
|
|
|
|
currently failing as branch isn't pushed (we should auto create this branch as we have the ref)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
allow-tip-sha1-in-want
|
|
|
|
|
|
|
|
|
|
we dont need to check the git files exist locally
|
|
|
|
|
|
|
|
|
|
|
|
- Add isolated_test! macro pattern to nip34_announcements.rs and nip01_compliance.rs
- Each test runs with its own fresh relay instance for complete isolation
- Make all individual test functions public in grasp-audit library (nip01_smoke.rs, event_acceptance_policy.rs)
- Eliminates 122 lines of boilerplate across integration tests
- Tests: 15 GRASP-01 event acceptance policy tests + 6 NIP-01 smoke tests
- Ensures tests don't interfere with each other, preventing flakiness
|
|
|
|
|
|
|
|
Replace nix::sys::signal usage with standard library process::kill().
This completes the removal of the unnecessary nix dev dependency.
All tests passing:
- NIP-01: 6 passed, 1 ignored
- NIP-34: 12 passed, 1 ignored
|
|
- Remove unnecessary 'nix' dev dependency (Unix syscalls crate, not needed)
- Migrate announcement tests to new TestRelay fixture pattern
- Delete legacy test files (announcement_tests.rs, test_relay.sh)
- Add comprehensive test documentation (docs/how-to/test-compliance.md)
- Update README.md with new test commands
- All 18 integration tests passing (NIP-01 + NIP-34)
Benefits:
- Automatic relay lifecycle management
- No manual setup required
- Pure Rust integration tests
- Better developer experience
- CI/CD ready
|
|
|