upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
Diffstat (limited to 'docs')
-rw-r--r--docs/dependencies.md18
-rw-r--r--docs/fair-launch.md70
-rw-r--r--docs/mining-calculator.md108
-rw-r--r--docs/pool-directory.md61
-rw-r--r--docs/release-verification.md94
-rw-r--r--docs/rendezvous.md336
-rw-r--r--docs/site.md115
-rw-r--r--docs/social.pngbin208957 -> 162267 bytes
-rw-r--r--docs/storage.md159
-rw-r--r--docs/tor.md231
-rw-r--r--docs/utxo-commitment.md125
-rw-r--r--docs/utxo-proofs.md61
12 files changed, 1351 insertions, 27 deletions
diff --git a/docs/dependencies.md b/docs/dependencies.md
new file mode 100644
index 0000000..9fd45b6
--- /dev/null
+++ b/docs/dependencies.md
@@ -0,0 +1,18 @@
1# Build Dependencies
2
3Bitflash links consensus-adjacent cryptographic and proof-of-work code from
4external repositories. Fresh builds must not follow upstream `master` silently:
5a changed upstream branch can break reproducible builds or change behavior in
6code that every node relies on.
7
8The top-level `Makefile` pins source dependencies by full commit hash:
9
10| dependency | repository | pinned commit |
11|---|---|---|
12| libsecp256k1 | https://github.com/bitcoin-core/secp256k1 | `7fecac74aed8e1fd9078380d67dd04663705c989` |
13| RandomX | https://github.com/tevador/RandomX | `1e9d4b2df63fa6edf46b06789486e23c0fcaa55a` |
14
15When updating either dependency, rebuild from a clean dependency directory and
16run the full selftest suite before opening the PR. The PR should explain why the
17dependency changed and whether the update can affect consensus, addresses,
18signatures, proof-of-work validation, or mining.
diff --git a/docs/fair-launch.md b/docs/fair-launch.md
new file mode 100644
index 0000000..9603910
--- /dev/null
+++ b/docs/fair-launch.md
@@ -0,0 +1,70 @@
1# Bitflash fair-launch verifier
2
3Bitflash ships a deterministic fair-launch audit script:
4
5```bash
6python3 scripts/verify-fair-launch.py --max-blocks 1000 --out bitflash-fair-launch-report.json --json
7```
8
9The verifier reads `blk*.dat` directly from the local Bitflash data directory.
10It does not use Berkeley DB, does not call the node binary, and does not need
11network access. It uses the same shared Python chain parser as the UTXO
12commitment and proof tools, so block serialization fixes land in one place.
13
14The report proves the launch baseline that can be checked from the chain:
15
16- the genesis block hash matches the source constant;
17- the genesis merkle root matches the source constant;
18- the genesis transaction is a single coinbase transaction;
19- the genesis output total is exactly `50.00000000 BTF`;
20- the scanned block range and mined coinbase total are reproducible.
21
22For machines that do not have a synced data directory yet, the source constants
23can be checked without disk blocks:
24
25```bash
26python3 scripts/verify-fair-launch.py --reconstruct-genesis
27```
28
29## Deterministic output
30
31Use `--json` or `--out` for canonical JSON. The JSON is written with sorted
32keys, fixed separators, ASCII escaping, and one trailing newline. It deliberately
33does not include local paths or wall-clock generation time, so two auditors
34scanning the same block range produce the same bytes.
35
36That makes the report shareable:
37
38```bash
39python3 scripts/verify-fair-launch.py --max-blocks 1000 --out report.json --json
40sha256sum report.json
41```
42
43If another auditor runs the same command against the same chain prefix, the
44`sha256sum` should match.
45
46## OpenTimestamps
47
48OpenTimestamps is optional and external. It anchors the report in Bitcoin
49without adding any bridge or dependency to Bitflash consensus.
50
51After generating a canonical JSON report:
52
53```bash
54ots stamp report.json
55ots verify report.json.ots
56```
57
58The `.ots` file proves that the exact report bytes existed no later than the
59Bitcoin block that confirms the timestamp.
60
61## What this does not claim
62
63This is not a retroactive genesis commitment. The Bitflash genesis block already
64exists, so adding a new on-chain commitment to genesis would create a different
65chain. This verifier instead audits the launch state that is already on-chain
66and makes the result reproducible and timestampable.
67
68It also does not prove real-world miner identity. It proves the genesis state,
69the absence of non-coinbase premine in genesis, and the deterministic coinbase
70totals over the scanned prefix.
diff --git a/docs/mining-calculator.md b/docs/mining-calculator.md
new file mode 100644
index 0000000..e6cb61b
--- /dev/null
+++ b/docs/mining-calculator.md
@@ -0,0 +1,108 @@
1# Bitflash mining calculator
2
3`scripts/bitflash-mining-calculator.py` is a local WhatToMine-style calculator
4for Bitflash.
5
6It does not fetch market prices, does not talk to exchanges, and does not
7change consensus. Price is a manual input until Bitflash has a reliable public
8market.
9
10## Examples
11
12Estimate from network hashrate:
13
14```bash
15python3 scripts/bitflash-mining-calculator.py \
16 --hashrate 21 --hashrate-unit kh/s \
17 --network-hashrate 1.56 --network-hashrate-unit mh/s \
18 --watts 140 --kwh-cost 0.10 \
19 --price 0.001 --pool-fee 1
20```
21
22Estimate from Bitflash difficulty:
23
24```bash
25python3 scripts/bitflash-mining-calculator.py \
26 --hashrate 21 --hashrate-unit kh/s \
27 --difficulty 45600 \
28 --watts 140 --kwh-cost 0.10
29```
30
31Machine-readable output:
32
33```bash
34python3 scripts/bitflash-mining-calculator.py \
35 --hashrate 21 --hashrate-unit kh/s \
36 --network-hashrate 1.56 --network-hashrate-unit mh/s \
37 --json
38```
39
40## Bitflash Difficulty Scale
41
42Bitflash does not use Bitcoin's SHA256 difficulty-1 constant for pool shares.
43The chain's easiest RandomX target is `bnProofOfWorkLimit = ~uint256(0) >> 12`,
44so difficulty 1 expects about `4096` hashes.
45
46The calculator uses:
47
48```text
49network_hashrate = difficulty * 4096 / block_time_seconds
50```
51
52At the default two-minute block target:
53
54```text
55network_hashrate = difficulty * 4096 / 120
56```
57
58If you already know network hashrate, pass `--network-hashrate` directly and
59the script derives the equivalent Bitflash difficulty for display.
60
61## Reward Estimate
62
63The expected daily reward is:
64
65```text
66blocks_per_day = 86400 / block_time_seconds
67miner_share = miner_hashrate / network_hashrate
68gross_btf_per_day = blocks_per_day * block_reward * miner_share
69net_btf_per_day = gross_btf_per_day * (1 - pool_fee_percent / 100)
70```
71
72Electricity is:
73
74```text
75electricity_per_day = watts / 1000 * 24 * kwh_cost
76```
77
78This is an expectation, not a promise. Solo mining variance can be very large,
79especially when the miner has a small share of the network.
80
81## Future Website Use
82
83The script is intentionally standalone so the same math can later back a
84website widget or status dashboard. Until a public price exists, a website
85calculator should keep price as a manual field and label fiat profit as
86speculative.
87
88## Website Fields
89
90A public calculator should ask for:
91
92- miner hashrate and unit;
93- power draw in watts;
94- electricity price per kWh;
95- pool fee percent;
96- optional BTF price;
97- either current Bitflash difficulty or current network hashrate.
98
99The output should separate:
100
101- estimated BTF per day;
102- estimated gross fiat revenue, if a manual price was provided;
103- electricity cost;
104- net profit/loss;
105- the difficulty/hashrate assumptions used.
106
107Never present the fiat number as a market quote unless it comes from a real
108public market. Before that exists, label it as a manual scenario.
diff --git a/docs/pool-directory.md b/docs/pool-directory.md
new file mode 100644
index 0000000..ff0640a
--- /dev/null
+++ b/docs/pool-directory.md
@@ -0,0 +1,61 @@
1# Public Pool Directory
2
3`pool.bitflash.network` should be a public directory, not a network dependency.
4Nodes discover pools through Nostr and `.btf`; the website only helps humans
5compare operators.
6
7Operators can publish the `pool_status.json` and `pool_rounds.json` files
8produced by:
9
10```bash
11bitflash -nogui -gen -operator \
12 -poolstatusfile=/var/www/pool_status.json \
13 -poolroundsfile=/var/www/pool_rounds.json
14```
15
16Build a static directory from one or more status files:
17
18```bash
19python3 scripts/build-pool-index.py --out-dir public/pool \
20 https://operator-one.example/pool_status.json \
21 https://operator-two.example/pool_status.json
22```
23
24The generated output contains:
25
26- `index.html`: human-readable ranking page.
27- `pools.json`: machine-readable API for `api.bitflash.network/pools.json`.
28
29The website is a convenience layer, not a consensus or discovery dependency.
30Nodes discover pools through Nostr descriptors and `.btf`; a missing website
31must not stop mining or pool operation.
32
33When a source is an HTTP(S) `pool_status.json`, the directory links to a
34`pool_rounds.json` proof file next to it. That file records the block height,
35block hash, coinbase txid, coinbase value, pool fee, total shares, per-address
36share counts, computed payouts, and payout txids once paid.
37
38Ranking order is intentionally simple:
39
401. online pools first;
412. higher reported hashrate;
423. more authorized miners;
434. lower fee;
445. name as a stable tie-breaker.
45
46An entry is considered online when the source says the pool is running and its
47status file is no more than 120 seconds old.
48
49## Recommended Hostnames
50
51Use:
52
53```text
54pool.bitflash.network human-readable pool directory
55api.bitflash.network/pools.json
56status.bitflash.network relay/node health, not pool ranking
57docs.bitflash.network operator setup guide
58```
59
60The GUI may use the API as a shortcut for humans, but it should keep the
61existing decentralized pool discovery path as the source of truth.
diff --git a/docs/release-verification.md b/docs/release-verification.md
index ecacd5f..b8eaf5e 100644
--- a/docs/release-verification.md
+++ b/docs/release-verification.md
@@ -3,60 +3,100 @@
3Bitflash releases publish binaries and a `SHA256SUMS` file. The checksum file 3Bitflash releases publish binaries and a `SHA256SUMS` file. The checksum file
4proves that the file you downloaded matches the file the release page names. 4proves that the file you downloaded matches the file the release page names.
5A detached signature, `SHA256SUMS.asc`, proves that the checksum file itself was 5A detached signature, `SHA256SUMS.asc`, proves that the checksum file itself was
6signed by a trusted release key. 6signed by the Bitflash release key.
7 7
8That gives users two separate checks: 8That gives users two separate checks:
9 9
101. `gpg --verify SHA256SUMS.asc SHA256SUMS` checks who signed the checksums. 101. `gpg --verify SHA256SUMS.asc SHA256SUMS` — who signed the checksums.
112. `sha256sum -c SHA256SUMS` checks the binaries against those checksums. 112. `sha256sum -c SHA256SUMS` — the binaries against those checksums.
12 12
13## User Check 13But the first check is only worth something if you require a **specific** key.
14A good signature by *some* key proves nothing; anyone can make one.
14 15
15From a shell with `curl` or `wget` and `sha256sum`: 16## The release signing key
17
18```text
19fingerprint 910A 2B4C CA87 9E81 FB4B 2AEA 1D4A 53D3 B78A A4B8
20uid Bitflash Releases (bitflash.network release signing)
21 <releases@bitflash.network>
22```
23
24The public key is published at
25`https://releases.bitflash.network/bitflash-release-key.asc` and embedded in
26`scripts/verify-release.sh`, so the tool's trust anchor lives in this
27repository rather than on the server it verifies.
28
29## User check
30
31From a shell with `curl` or `wget`, `sha256sum` and `gpg`:
16 32
17```bash 33```bash
18scripts/verify-release.sh latest 34scripts/verify-release.sh latest
19``` 35```
20 36
21For a specific release: 37The public release host should expose latest-file aliases for convenience:
22 38
23```bash 39```text
24scripts/verify-release.sh v1.2.13 40https://releases.bitflash.network/SHA256SUMS
41https://releases.bitflash.network/SHA256SUMS.asc
25``` 42```
26 43
27For release audits, require the signature: 44Those aliases must point at the current versioned directory. They do not replace
45signature verification; they only make the common manual path shorter.
46
47For a specific release:
28 48
29```bash 49```bash
30scripts/verify-release.sh v1.2.13 --require-signature 50scripts/verify-release.sh v1.2.15
31``` 51```
32 52
33Older releases may not have `SHA256SUMS.asc`. In that case the script warns and 53The script downloads the release, imports the pinned key into a throwaway
34still checks file integrity. New release audits should use `--require-signature`. 54keyring, and **refuses anything that is not a valid signature by the key above**
35 55before checking the hashes. Releases from before signing was introduced ship
36## Maintainer Flow 56only `SHA256SUMS`; without `--require-signature` those are checked for integrity
57and reported as unsigned.
37 58
38After building release assets in the repository root: 59By hand, if you would rather not run the script:
39 60
40```bash 61```bash
41scripts/make-release-checksums.sh --sign --local-user RELEASE_KEY_ID 62curl -O https://releases.bitflash.network/bitflash-release-key.asc
63gpg --import bitflash-release-key.asc
64gpg --verify SHA256SUMS.asc SHA256SUMS # must name the fingerprint above
65sha256sum -c SHA256SUMS
42``` 66```
43 67
44Upload all built assets plus: 68## Maintainer flow
45 69
46```text 70The signing key lives on a dedicated location, not in the build tree. After
47SHA256SUMS 71building the release assets in the repository root, write and sign the
48SHA256SUMS.asc 72checksums:
73
74```bash
75sha256sum Bitflash-*.zip Bitflash-*.AppImage bitflash-node-* > SHA256SUMS
76gpg --armor --detach-sign --output SHA256SUMS.asc SHA256SUMS
49``` 77```
50 78
51Keep the private signing key offline or on a dedicated release machine. Publish 79Publish, alongside the assets, `SHA256SUMS` and `SHA256SUMS.asc`, and update
52the public key fingerprint in the release notes and keep using the same key for 80`latest.txt` on the releases host to the new tag. Keep using the same key; if it
53future releases unless there is a clearly announced rotation. 81is ever rotated, announce the new fingerprint clearly and update the embedded
82key in `verify-release.sh`.
83
84For Windows releases with managed Tor built in, run `make windows-tor`. It
85creates `Bitflash-*-windows-with-tor.zip` by downloading the Tor Expert Bundle,
86checking the pinned SHA256 in `scripts/package-windows-tor.sh`, optionally
87verifying the Tor Project `.asc` signature, and placing `tor/tor.exe` next to
88`Bitflash.exe`. For release builds, run `TOR_VERIFY_GPG=required make
89windows-tor` so the signature check is mandatory. Include that zip in
90`SHA256SUMS` like any other release asset.
54 91
55## Why This Matters 92## Why this matters
56 93
57`SHA256SUMS` alone protects against a broken download, but not against someone 94`SHA256SUMS` alone protects against a broken download, but not against someone
58replacing both a binary and the checksum file. Signing `SHA256SUMS` means an 95replacing both a binary and the checksum file. Signing `SHA256SUMS`, and
59attacker must also have the release signing key to make the replacement verify. 96requiring a pinned key, means an attacker must also hold the release signing key
97to make the replacement verify. Serving from bitflash.network makes this the
98guarantee that matters: without a pinned key, "trust the site" is the only
99thing standing behind a download.
60 100
61This is not reproducible builds yet. It is the smaller, immediate step that 101This is not reproducible builds yet. It is the smaller, immediate step that
62makes every release asset auditable by users before they run it. 102makes every release asset auditable before anyone runs it.
diff --git a/docs/rendezvous.md b/docs/rendezvous.md
new file mode 100644
index 0000000..68b5df3
--- /dev/null
+++ b/docs/rendezvous.md
@@ -0,0 +1,336 @@
1# Bitflash `.btf` Rendezvous Protocol
2
3This is the specification of how Bitflash nodes reach each other, and how miners
4reach pools, without revealing an IP address and without any central directory.
5It documents the protocol as implemented in `src/btfaddr.cpp`, `src/btfchan.cpp`,
6`src/btfrv.cpp`, `src/btftunnel.cpp` and `src/nostr.cpp`.
7
8The design is ported from the Itzal hidden service design. A `.btf` address names
9a public key, a node proves ownership of that key by signing a service
10descriptor, other nodes find the descriptor over Nostr, and the two sides then
11meet through an untrusted relay over an end to end encrypted tunnel. The relay
12carries ciphertext only and never learns the plaintext.
13
14## 1. Overview
15
16A connection to a `.btf` service has three layers:
17
181. **Discovery (Nostr).** The service publishes a signed descriptor as a Nostr
19 event. A caller fetches it by the service public key and verifies the
20 signature. The descriptor says which rendezvous relay the service is waiting
21 at, which encryption key to use, and optionally which Tor hidden service can
22 be dialed directly.
232. **Rendezvous (TCP).** Caller and service both connect out to the same relay
24 and are paired by public key. Neither needs an inbound port or a public IP.
25 After pairing the relay is a blind byte pipe.
263. **Direct onion fallback bypass.** If the descriptor carries a valid signed
27 `.onion:port` endpoint and the caller is running with Tor mode, the caller
28 tries that endpoint before using rendezvous. This removes the rendezvous
29 relay from the data path for peers that choose to run hidden services.
304. **Encrypted channel (X25519 + XChaCha20-Poly1305).** Over the paired pipe the
31 caller and service run an ephemeral to static handshake and then exchange
32 authenticated frames. The relay sees only ciphertext.
33
34The caller ends up holding an ordinary local socket that behaves like a direct
35TCP connection to the peer. The node treats it exactly like any other peer
36socket.
37
38## 2. Identities and keys
39
40A node holds two independent keypairs.
41
42- **Signing / identity key**: a secp256k1 x-only key (BIP340 Schnorr). This is
43 the `.btf` identity and also the node's Nostr identity. Persisted in
44 `nostr.key`.
45- **Encryption key**: an X25519 (Curve25519) keypair. Persisted in
46 `btf_enc.key`. Its public half is published in the descriptor as `enc` and is
47 used for the tunnel key agreement.
48
49The identity key names the service and authenticates the descriptor. The
50encryption key secures the tunnel. They are separate so the long term identity
51never performs Diffie-Hellman and the encryption key can be rotated in a new
52signed descriptor.
53
54## 3. The `.btf` address
55
56An address is the identity public key plus a short checksum, Base32 encoded, with
57the suffix `.btf`.
58
59```
60label = Base32( pubkey[32] || checksum[2] )
61address = label + ".btf"
62```
63
64- `pubkey` is the 32 byte secp256k1 x-only public key.
65- `checksum` is the first two bytes of `SHA256(".btf-checksum-v1" || pubkey)`.
66- Base32 uses the lowercase alphabet `abcdefghijklmnopqrstuvwxyz234567`, no
67 padding. The 34 byte payload encodes to 55 characters.
68- Parsing is case insensitive, trims surrounding whitespace, and strips a
69 trailing dot. A wrong checksum is rejected.
70
71The address is self certifying: it *is* the public key, so no registry has to be
72trusted to map a name to a key.
73
74## 4. Discovery over Nostr
75
76Nodes publish and read three kinds of parameterized replaceable Nostr events.
77Parameterized replaceable means the newest event for a given author and `d` tag
78supersedes the older one, so a descriptor is always the latest state.
79
80| Purpose | kind | `d` tag |
81|---|---|---|
82| Service descriptor | 38501 | `btf-descriptor-2` |
83| Rendezvous relay announcement | 38502 | `btf-relay-2` |
84| Pool announcement | 38503 | `btf-pool-2` |
85
86The Nostr event is signed with the node's identity key, so the event author
87equals the `.btf` public key. That binds the descriptor to the address at the
88Nostr layer, and the descriptor is *also* signed inside its own content (see
89below), so a relay operator cannot forge or swap a descriptor for an address it
90does not control.
91
92**Resolving an address.** To reach `X.btf`, a caller parses the address to the 32
93byte key and sends a Nostr `REQ` filtered by `authors = [X]`, `kinds = [38501]`,
94`#d = ["btf-descriptor-2"]`, `limit = 1`. It verifies the returned descriptor and
95uses its `meeting_node` and `enc`.
96
97**Relay auto discovery.** A relay operator runs a rendezvous relay and publishes a
98kind 38502 event carrying `host:port`. Every node that reads it adds the relay to
99the set it can register at and dial through. No seed list edit or maintainer
100approval is needed. A small curated seed list is still shipped for bootstrap:
101
102```
10392.246.128.180:8434
10431.44.4.249:8434
10590.156.222.107:8434
106```
107
108The default rendezvous port is `8434`.
109
110## 5. The service descriptor
111
112The descriptor is a JSON object, signed with the identity key over a domain
113separated digest of its own fields.
114
115Legacy descriptor:
116
117```json
118{
119 "v": 2,
120 "pubkey": "<hex 32-byte x-only identity key>",
121 "enc": "<hex 32-byte X25519 public key>",
122 "meeting_node": "host:port",
123 "created": <unix seconds>,
124 "sig": "<hex 64-byte BIP340 Schnorr signature>"
125}
126```
127
128Onion-capable descriptor:
129
130```json
131{
132 "v": 3,
133 "pubkey": "<hex 32-byte x-only identity key>",
134 "enc": "<hex 32-byte X25519 public key>",
135 "meeting_node": "host:port",
136 "onion": "<56-char-v3-name>.onion:8433",
137 "created": <unix seconds>,
138 "sig": "<legacy v2 signature>",
139 "sig3": "<signature over the onion-capable digest>"
140}
141```
142
143The legacy signed digest is:
144
145```
146digest = SHA256( ".btf-descriptor-v2"
147 || pubkey[32]
148 || enc (raw bytes of the string)
149 || meeting_node (raw bytes of the string)
150 || created (8 bytes, big endian) )
151```
152
153`sig` is kept on v3 descriptors so old nodes can still resolve and dial through
154the rendezvous relay. The onion-capable digest is:
155
156```
157digest3 = SHA256( ".btf-descriptor-v3-onion"
158 || pubkey[32]
159 || enc (raw bytes of the string)
160 || meeting_node (raw bytes of the string)
161 || onion (raw bytes of the normalized string)
162 || created (8 bytes, big endian) )
163```
164
165`sig` and `sig3` are deterministic BIP340 Schnorr signatures (no auxiliary
166randomness) over their respective digests by the identity key.
167
168**Verification** (untrusted input, must never throw): parse the JSON, require all
169fields with the right types, confirm `pubkey` equals the address being resolved,
170recompute the digest, and verify the Schnorr signature. Any failure rejects the
171descriptor.
172
173If `onion` is present, readers accept it only when it is a normalized v3 onion
174endpoint and `sig3` verifies. A bad or tampered onion field is ignored; the v2
175descriptor can still be used for rendezvous fallback if `sig` verifies.
176
177`created` lets a reader prefer the freshest descriptor when more than one is
178seen. A node without a direct onion endpoint only publishes a descriptor once it
179has actually registered at a `meeting_node`, so it never advertises a relay it
180cannot be reached at. A node with a direct onion endpoint may publish before
181rendezvous registration using `meeting_node = "rendezvous-pending"`; Tor peers
182can still dial the signed onion endpoint, and rendezvous becomes fallback once
183registration succeeds.
184
185## 6. Rendezvous transport
186
187The relay is a small TCP server. Every connection opens with a fixed 33 byte
188header:
189
190```
191byte 0 role: 'S' (service) or 'C' (client)
192bytes 1..32 target public key (the .btf identity, 32 bytes)
193```
194
195**Service registration (`S`).** The service connects out to the relay and sends
196`'S' || own_pubkey`. The relay stores the connection in a table keyed by the
197public key and leaves it open, waiting. There is one slot per key: a new
198registration for the same key replaces and closes the old one. The service does
199not block waiting to be paired at registration time; it is considered registered
200the moment the relay reads the header, which is what lets it publish a descriptor
201pointing at this relay.
202
203**Client dial (`C`).** The caller connects out to the same relay and sends
204`'C' || target_pubkey`. The relay looks up the waiting service:
205
206- If present, it removes it from the table, writes a single `0x01` byte to both
207 sockets, and then forwards raw bytes in both directions until either side
208 closes. From this point the relay is a blind pipe.
209- If absent, it writes `0x00` to the caller and closes.
210
211Both sides connect *outbound* to the relay, so neither needs a public IP or an
212open inbound port. The relay never sees the identity of the caller (the caller
213sends no key of its own, only the target it wants).
214
215**Liveness.** A registration socket is idle by design and can outlive the network
216path that created it (a NAT drops the idle mapping, the relay restarts, a route
217changes) with no FIN and nothing readable. TCP keepalive was measured and does
218not reliably detect this. The service therefore imposes its own timeout on the
219wait for the pairing byte and re-registers on expiry. Expiring is not a failure;
220it costs one reconnect per interval and returns the loop to a known state.
221
222## 7. The encrypted channel
223
224Once paired through the relay, the caller and service run an end to end handshake
225and then exchange authenticated frames. The primitive is libsodium
226`crypto_box_curve25519xchacha20poly1305` (X25519 key agreement, XChaCha20-Poly1305
227AEAD), the same construction as Rust's `crypto_box` ChaChaBox.
228
229**Handshake (ephemeral to static).**
230
2311. The caller generates a fresh ephemeral X25519 keypair, computes
232 `shared = X25519(eph_sk, service_enc_pub)` reduced to a box key
233 (`crypto_box_beforenm`), and sends the 32 byte ephemeral public key over the
234 paired pipe.
2352. The service reads the ephemeral public key and computes the same
236 `shared = X25519(enc_sk, eph_pub)`.
237
238Because the caller uses a new ephemeral key per connection, each session has an
239independent key and forward secrecy against later compromise of the service
240encryption key.
241
242**Framing.** Every message is length prefixed:
243
244```
245frame = len[4] (big endian, length of the body that follows)
246 body = nonce[24] || ciphertext || mac[16]
247```
248
249The nonce is 24 random bytes per frame (XChaCha20), the MAC is 16 bytes
250(Poly1305). A body shorter than `nonce + mac` or larger than 1 MiB is rejected.
251Decryption that fails authentication drops the frame and tears the tunnel down.
252
253The tunnel is transparent: the application is handed a local socket pair end and
254reads and writes plaintext. Two pump threads move data between that socket and
255the relay socket, encrypting outbound and decrypting inbound.
256
257## 8. End to end connection flow
258
259To dial `X.btf`:
260
2611. Parse `X.btf` to the 32 byte identity key.
2622. Fetch and verify the descriptor over Nostr (kind 38501). Obtain
263 `meeting_node` and `enc`.
2643. Connect to `meeting_node`, send `'C' || X`, and wait for `0x01`.
2654. Generate an ephemeral X25519 key, derive the shared key against `enc`, send
266 the ephemeral public key.
2675. Exchange XChaCha20-Poly1305 frames. Hand the application a local socket that
268 speaks the ordinary peer protocol.
269
270A service, symmetrically: registers at a relay (`'S' || own_pubkey`), publishes a
271descriptor pointing at that relay, waits for `0x01`, reads the caller ephemeral
272public key, derives the shared key with its encryption secret, and serves the
273tunnel.
274
275## 9. Security properties
276
277- **Address unforgeability.** The address is the identity key and the descriptor
278 is Schnorr signed by it, and the Nostr event is signed by it. A relay or Nostr
279 operator cannot publish a working descriptor for an address it does not own.
280- **Relay confidentiality.** The relay forwards ciphertext and never holds the
281 channel key. It cannot read or modify traffic undetected; a tampered frame
282 fails the Poly1305 tag and the tunnel closes.
283- **Caller anonymity to the service and relay.** The caller presents no identity
284 key, only the target it wants to reach. It connects outbound, so its address is
285 not published anywhere.
286- **Forward secrecy.** The caller's per connection ephemeral key means a later
287 compromise of the service encryption key does not decrypt past sessions.
288- **IP privacy.** Neither side needs a public IP or an inbound port for
289 rendezvous. With a signed `.onion` endpoint, the reachable side exposes only a
290 Tor hidden service and the relay is not in the peer connection path.
291
292## 10. Threat model and limitations
293
294- **The relay is trusted for availability, not for confidentiality.** It can
295 refuse to pair, drop a connection, or go offline. This is why relays are
296 plural, auto discovered, and re-registration is periodic. It cannot read
297 traffic.
298- **Traffic analysis is not addressed.** A network observer, or the relay, can
299 see connection timing and byte volumes. There is no padding or mixing.
300- **Single slot registration causes churn.** One waiting slot per key per relay
301 means a race or a stale registration can briefly make a service unreachable at
302 that relay. Multiple relays and the wait timeout bound the damage, but this is
303 a known source of connection churn.
304- **Descriptor freshness is best effort.** `created` is a self reported
305 timestamp; readers prefer the newest descriptor they have seen but there is no
306 global clock.
307- **Discovery depends on public Nostr relays.** If the relays a node uses are all
308 unreachable, discovery stalls until one is reachable again. Relay TLS is
309 verified, which closes a man in the middle path on discovery, but availability
310 still rests on the relay set.
311- **Direct onion needs Tor.** With `-managedtor`, Bitflash starts a Tor process,
312 writes a local hidden-service config, and advertises the generated onion
313 endpoint. With `-onionservice`, an operator can still run Tor manually and
314 provide the endpoint. In both cases Tor owns the hidden-service private key and
315 uptime.
316
317## 11. Wire reference
318
319| Item | Value |
320|---|---|
321| `.btf` Base32 alphabet | `abcdefghijklmnopqrstuvwxyz234567`, no padding |
322| Address payload | `pubkey[32] || checksum[2]`, 55 Base32 chars |
323| Address checksum | `SHA256(".btf-checksum-v1" || pubkey)[0..2]` |
324| Descriptor domain sep | `.btf-descriptor-v2`; optional onion sep `.btf-descriptor-v3-onion` |
325| Descriptor version | `2`; `3` when a signed onion endpoint is present |
326| Descriptor signature | BIP340 Schnorr, deterministic (no aux rand), v3 keeps `sig` plus `sig3` |
327| Nostr descriptor event | kind `38501`, `d = btf-descriptor-2` |
328| Nostr relay event | kind `38502`, `d = btf-relay-2` |
329| Nostr pool event | kind `38503`, `d = btf-pool-2` |
330| Rendezvous port (default) | `8434` |
331| Rendezvous header | `role[1] ('S'/'C') || pubkey[32]`, 33 bytes |
332| Rendezvous pairing byte | `0x01` paired, `0x00` no service |
333| Channel primitive | `crypto_box_curve25519xchacha20poly1305` |
334| Nonce / MAC / key sizes | 24 / 16 / 32 bytes |
335| Frame | `len[4 big endian] || nonce[24] || ct || mac[16]` |
336| Max frame body | 1 MiB |
diff --git a/docs/site.md b/docs/site.md
new file mode 100644
index 0000000..9526a61
--- /dev/null
+++ b/docs/site.md
@@ -0,0 +1,115 @@
1# Public Site Contract
2
3The public Bitflash sites are generated from this repository. If a deployed
4page and the source disagree, the deployed page is stale.
5
6## Canonical Navigation
7
8Every public page should use the same top navigation:
9
10| label | target |
11|---|---|
12| home | `https://bitflash.network/` |
13| code | `https://git.bitflash.network/bitflash/bitflash` |
14| docs | `https://docs.bitflash.network/` |
15| downloads | `https://releases.bitflash.network/` |
16| explorer | `https://explorer.bitflash.network/` |
17| status | `https://status.bitflash.network/` |
18
19The `code` link intentionally points at the repository, not the Forgejo
20instance root. A visitor choosing "code" wants the source tree.
21
22## Documentation Pages To Publish
23
24The docs index should expose these source files:
25
26| page | source | why it matters |
27|---|---|---|
28| Manual | `README.md` | run, mine, recover, operate |
29| Key derivation and address format | `docs/derivation.md` | independent wallet recovery |
30| Release verification | `docs/release-verification.md` | pinned signing-key flow |
31| Dependencies | `docs/dependencies.md` | pinned secp256k1/RandomX supply chain |
32| Rendezvous and .btf addressing | `docs/rendezvous.md` | network design and caveats |
33| Tor mode | `docs/tor.md` | `-tor`, `-socks`, onion relay operation |
34| Fair launch verifier | `docs/fair-launch.md` | no-premine launch audit |
35| UTXO set commitment | `docs/utxo-commitment.md` | deterministic supply/root audit |
36| UTXO inclusion proofs | `docs/utxo-proofs.md` | proof for one unspent output |
37| Mining calculator | `docs/mining-calculator.md` | local WhatToMine-style estimates |
38| Public pool directory | `docs/pool-directory.md` | `pool_status.json`, `pools.json`, ranking |
39
40## Release Mirror Aliases
41
42Each release remains under its versioned directory:
43
44```text
45https://releases.bitflash.network/v1.2.19/SHA256SUMS
46https://releases.bitflash.network/v1.2.19/SHA256SUMS.asc
47```
48
49The release root should also alias the latest files:
50
51```text
52https://releases.bitflash.network/SHA256SUMS
53https://releases.bitflash.network/SHA256SUMS.asc
54https://releases.bitflash.network/Bitflash-latest-windows.zip
55https://releases.bitflash.network/Bitflash-latest-x86_64.AppImage
56https://releases.bitflash.network/bitflash-node-latest-x86_64
57```
58
59The aliases are convenience only. Verification must still require the pinned
60release key documented in `docs/release-verification.md`.
61
62## Static Explorer
63
64`scripts/build-explorer.py` writes a static explorer with no backend:
65
66```text
67index.html
68style.css
69explorer.js
70logo.png
71blocks.json
72block/<height>.json
73```
74
75Serve it with a strict content policy:
76
77```text
78default-src 'none';
79img-src 'self';
80style-src 'self';
81script-src 'self';
82connect-src 'self';
83base-uri 'none';
84form-action 'none';
85frame-ancestors 'none';
86```
87
88The viewer must never use `innerHTML` for chain data. A malicious transaction
89must be rendered as text, not interpreted as markup.
90
91## Cache Policy
92
93Recommended defaults:
94
95| path | cache |
96|---|---|
97| HTML pages | `public, max-age=0, must-revalidate` |
98| `status.json` | `no-store` |
99| `blocks.json`, `block/*.json` | `public, max-age=0, must-revalidate` |
100| release assets | `public, max-age=14400, must-revalidate` |
101| `SHA256SUMS`, signatures, keys | `public, max-age=0, must-revalidate` |
102
103## Security Headers
104
105Recommended for all static pages:
106
107```text
108Strict-Transport-Security: max-age=31536000; includeSubDomains
109X-Content-Type-Options: nosniff
110Referrer-Policy: no-referrer
111Permissions-Policy: interest-cohort=(), geolocation=(), camera=(), microphone=()
112```
113
114Forgejo has its own application headers and should not blindly inherit the
115static-site CSP.
diff --git a/docs/social.png b/docs/social.png
index 19e2a3b..9a08b02 100644
--- a/docs/social.png
+++ b/docs/social.png
Binary files differ
diff --git a/docs/storage.md b/docs/storage.md
new file mode 100644
index 0000000..c7de4e7
--- /dev/null
+++ b/docs/storage.md
@@ -0,0 +1,159 @@
1# Bitflash Storage Inventory
2
3This document records the Berkeley DB surface that still exists in Bitflash.
4It is a migration map, not a promise that all stores should move at once.
5
6The immediate priority is `wallet.dat`: it holds spend authority and recovery
7metadata. `blkindex.dat` can be rebuilt from block files, so it is important for
8startup correctness but less dangerous than wallet storage.
9
10## Berkeley DB Environment
11
12All `CDB` stores use one Berkeley DB environment rooted at:
13
14```text
15<datadir>/database/
16```
17
18The environment creates log files such as:
19
20```text
21database/log.0000000001
22```
23
24Those logs are part of the legacy BDB risk model. They are not user-facing
25wallet files, but they can contain historical records while the environment is
26live. Wallet encryption already rewrites `wallet.dat` and purges BDB environment
27logs after the encrypted file is installed.
28
29## Common Records
30
31Every `CDB` file can carry:
32
33| Key | Value | Purpose |
34|---|---|---|
35| `"version"` | `int` | Database format version written when a DB is opened in create mode. |
36
37Keys and values are serialized with the existing `CDataStream` format. A future
38SQLite migration must preserve byte-for-byte value semantics until a higher
39level schema migration is intentionally introduced.
40
41## `wallet.dat`
42
43`wallet.dat` is accessed through `CWalletDB`.
44Read-only tooling should go through `ScanWalletRecords()` instead of opening a
45Berkeley DB cursor directly. That keeps migration checks, audits, and future
46backends behind one record-stream boundary. Wallet rewrite code should also use
47that boundary when it needs to copy existing records into a newly written
48wallet file.
49
50| Key shape | Value | Secret? | Meaning |
51|---|---|---:|---|
52| `("name", address)` | `string` | No | Address book label. |
53| `("tx", txid)` | `CWalletTx` | No | Wallet transaction state, including spend flags and metadata. |
54| `("key", pubkey)` | `CPrivKey` | Yes | Plain private key record for unencrypted wallets or old backups. |
55| `"defaultkey"` | `vector<unsigned char>` | No | Default public key shown by the wallet. |
56| `"hdmaster"` | `vector<unsigned char>` | Yes | Plain BIP32 master private key material. |
57| `"hdchaincode"` | `vector<unsigned char>` | Yes | Plain BIP32 chain code. |
58| `"hdnext"` | `unsigned int` | No | Legacy deterministic derivation next index. |
59| `"hdschema"` | `int` | No | Deterministic derivation schema (`legacy` or `bip44`). |
60| `"hdcointype"` | `unsigned int` | No | BIP44 coin type. Bitflash is `4346950`. |
61| `"hdreceivenext"` | `unsigned int` | No | BIP44 external receive-chain next index. |
62| `"hdchangenext"` | `unsigned int` | No | BIP44 internal change-chain next index. |
63| `("mkey", id)` | `CWalletMasterKey` | Sensitive | Encrypted wallet master-key metadata. |
64| `("ckey", pubkey)` | `vector<unsigned char>` | Yes | Private key encrypted under the wallet master key. |
65| `"cryptedhdmaster"` | `vector<unsigned char>` | Yes | Encrypted BIP32 master private key material. |
66| `"cryptedhdchaincode"` | `vector<unsigned char>` | Yes | Encrypted BIP32 chain code. |
67| `"walletminversion"` | `int` | No | Minimum wallet format version required to open this wallet safely. |
68| `("pool", index)` | `vector<unsigned char>` | No | Public key reserved in the key pool. The private key lives in `key`/`ckey`. |
69| `("setting", name)` | typed setting value | Usually no | Wallet/UI settings such as mining mode, fees, and backup timestamps. |
70
71### Wallet Migration Rules
72
73A future wallet backend must keep these properties:
74
751. Unknown future wallet formats fail closed with a clear message.
762. Encrypted wallets must encrypt private keys and HD seed material.
773. A locked encrypted wallet may expose public keys and balances, but not
78 private keys or decrypted HD seed material.
794. `-backupwallet` must produce a wallet file that opens in a different datadir
80 without copying `database/`, and must not leave a half-written final backup
81 path if the copy or install step fails.
825. `-recoveryaudit` must distinguish:
83 - no recovery phrase installed;
84 - recovery phrase present;
85 - recovery phrase encrypted and locked, requiring unlock before coverage can
86 be audited.
87
88## SQLite Wallet Store
89
90Since v1.2.20 SQLite is the default wallet backend. It is a byte-preserving
91record store, not a new wallet format -- the same serialized `CDataStream` bytes
92`wallet.dat` uses:
93
94```sql
95CREATE TABLE wallet_records (
96 key BLOB PRIMARY KEY NOT NULL,
97 value BLOB NOT NULL
98);
99PRAGMA user_version=1;
100```
101
102A fresh data directory starts on `wallet.sqlite`. An existing `wallet.dat` keeps
103running on Berkeley DB until it is converted: the GUI offers a one-click
104conversion on first run, and `-walletbackend=sqlite` / `-walletbackend=bdb`
105(recorded in a plain-text `wallet-backend` marker file in the datadir) select a
106backend explicitly. Conversion never modifies `wallet.dat`; it stays as a
107fallback, and the change takes effect on restart.
108
109Every wallet operation runs on either backend: read, write, erase, key-pool and
110HD-seed writes (the two halves of the seed bracketed in one SQLite transaction),
111`-backupwallet` (checkpoint then copy the self-contained file), `-encryptwallet`
112(an atomic rebuild-and-swap in one transaction), and the `-walletstorageaudit` /
113`-walletstoragecheck` diagnostics, which read the active backend's file.
114
115The migration tooling remains available for scripted use. `-walletsqliteexport=FILE`
116streams every raw `wallet.dat` record through `ScanWalletRecords()` into a SQLite
117file in one transaction, refusing to overwrite. `-walletsqliteverify=FILE`
118compares the two streams and prints only counts. `-walletsqliteloadcheck=FILE`
119parses an export with the same record types the loader uses.
120`-walletsqliterestore=FILE` rebuilds a `wallet.dat` from an export in an empty
121data directory. Failed exports and restores remove their incomplete output.
122
123## `blkindex.dat`
124
125`blkindex.dat` is accessed through `CTxDB`.
126
127| Key shape | Value | Rebuildable? | Meaning |
128|---|---|---:|---|
129| `("tx", txid)` | `CTxIndex` | Yes | Transaction disk position and spent-output index state. |
130| `("owner", hash160, CDiskTxPos)` | `int height` | Yes | Legacy owner transaction index cursor range. |
131| `("blockindex", blockhash)` | `CDiskBlockIndex` | Yes | Block index entry. |
132| `"hashBestChain"` | `uint256` | Yes | Current best-chain tip hash. |
133
134`blkindex.dat` can be rebuilt from `blk*.dat` plus consensus validation. A
135future migration may therefore prefer a rebuild path over a byte-preserving
136conversion path.
137
138## Other Legacy Stores
139
140None. Upstream still documents `reviews.dat` (`CReviewDB`) and `market.dat`
141(`CMarketDB`) here; this fork deleted the 2009 marketplace outright, along
142with `CReviewDB`, `market.cpp` and `market.h`, so neither file is opened or
143created by this tree. A datadir carrying them from an older build is simply
144ignored.
145
146The old IP address database (`addr.dat`) has been removed. Peer discovery is
147handled by Nostr, `.btf`, rendezvous relays, and direct onion peers.
148
149## Migration Order (shipped in v1.2.20)
150
1511. Storage self-tests were added around Berkeley DB behavior first.
1522. The SQLite backend landed behind an explicit flag, then behind a
153 `wallet-backend` marker and a GUI conversion wizard.
1543. BDB-to-SQLite migration is a one-way copy that leaves `wallet.dat` as a backup.
1554. SQLite became the default for new wallets only after cross-version restore,
156 encryption, backup, and recovery-audit tests passed -- and a live coin-proof
157 of receive, spend, restart, and encrypt on real mainnet funds.
1585. Treat `blkindex.dat` separately; it is safer to rebuild than to migrate under
159 pressure.
diff --git a/docs/tor.md b/docs/tor.md
new file mode 100644
index 0000000..0016f8b
--- /dev/null
+++ b/docs/tor.md
@@ -0,0 +1,231 @@
1# Tor mode
2
3Bitflash can route outbound discovery traffic through a local Tor SOCKS5 proxy.
4This is transport plumbing only: it does not change consensus, wallet keys,
5mining, `.btf` addresses, or the encrypted rendezvous protocol.
6
7## Client mode
8
9Start Tor locally, then start Bitflash with:
10
11```bash
12bitflash -tor
13```
14
15`-tor` is shorthand for:
16
17```bash
18bitflash -socks=127.0.0.1:9050
19```
20
21If Tor listens somewhere else:
22
23```bash
24bitflash -tor=127.0.0.1:9150
25bitflash -tor=[::1]:9050
26```
27
28For a SOCKS5 proxy that is not Tor, use `-socks` directly:
29
30```bash
31bitflash -socks=127.0.0.1:9050
32bitflash -socks=[::1]:9050
33```
34
35If both `-tor` and `-socks` are present, Tor mode wins and the node prints a
36warning. That keeps a command line with `-tor` from silently becoming a custom
37proxy route.
38
39## Managed Tor mode
40
41`-managedtor` starts a Tor process for this node, writes a local `torrc`, creates
42a v3 hidden service for the normal Bitflash P2P listener, routes outbound
43discovery through that Tor instance, and signs the generated `.onion:8433`
44endpoint into the node's `.btf` descriptor.
45
46```bash
47bitflash -managedtor
48```
49
50Without a value, Bitflash looks for Tor in this order:
51
521. `tor/tor.exe` or `tor/tor` beside the Bitflash binary;
532. `tor.exe` or `tor` beside the Bitflash binary;
543. `tor.exe` or `tor` on `PATH`.
55
56Windows releases can also be built as `Bitflash-*-windows-with-tor.zip`. That
57package includes the official Tor Expert Bundle under `tor/`, so the first path
58above exists immediately after extraction and `bitflash -managedtor` needs no
59separate Tor install.
60
61The bundled-Tor package is created by:
62
63```bash
64make windows-tor
65```
66
67The packaging script verifies a pinned SHA256 for the downloaded Tor Expert
68Bundle before copying it into the release directory. When `gpg` is available it
69also verifies the matching `.asc` signature against the pinned Tor Browser
70Developers signing-key fingerprint. Release builds should use
71`TOR_VERIFY_GPG=required make windows-tor` to make that second check mandatory.
72The Tor binary is bundled; the runtime data directory, hidden-service key, and
73control cookie are still created under the user's Bitflash data directory.
74
75To point at a specific Tor executable:
76
77```bash
78bitflash -managedtor=/opt/tor/bin/tor
79bitflash -managedtor=C:\Tor Browser\Browser\TorBrowser\Tor\tor.exe
80```
81
82Managed Tor writes its state under the Bitflash data directory:
83
84```text
85managed-tor/
86 torrc
87 data/
88 onion-service/
89```
90
91The private key for the onion address lives in `managed-tor/onion-service/`.
92Back up that directory if you want the same onion address after moving the node.
93Delete it only if you intentionally want a fresh onion endpoint.
94
95`-managedtor` takes precedence over `-tor` and `-socks`, because it must choose
96the local SOCKS port it starts. Diagnostics report the current managed Tor
97state, including the SOCKS port and onion endpoint once Tor has generated it.
98
99When Tor mode is enabled:
100
101- outbound Nostr relay dials use Tor;
102- outbound `.btf` rendezvous relay dials use Tor;
103- outbound direct peer dials prefer signed `.onion` endpoints when a peer
104 advertises one;
105- destination names are sent to Tor as SOCKS5 domain-name CONNECT requests, so
106 the local DNS resolver does not see them;
107- direct `.onion` dials are refused unless a SOCKS5/Tor proxy is enabled;
108- plain HTTP external-IP probes are skipped.
109
110Tor mode affects outbound discovery and relay dials. It does not encrypt
111`wallet.dat`, hide mining rewards on-chain, or change the `.btf` identity. A
112relay still sees the TCP client that connected to it; with Tor mode that client
113is the Tor exit or onion circuit endpoint instead of the user's direct IP.
114
115## What to test
116
117Use the public status page only as a rough health check. To prove the local node
118is using Tor/SOCKS, test from the node machine:
119
120```bash
121bitflash -tor -debug
122```
123
124Then confirm in `debug.log` that Nostr and rendezvous dials report the proxy
125path. A DNS leak test should show no local resolver lookup for relay hostnames:
126the SOCKS5 request uses the domain-name form and asks the proxy to resolve it.
127
128## Onion rendezvous relay
129
130An operator can expose a Bitflash rendezvous relay as a Tor hidden service. The
131node does not create hidden services itself; Tor owns that configuration.
132
133Example `torrc`:
134
135```text
136HiddenServiceDir /var/lib/tor/bitflash-rendezvous/
137HiddenServicePort 8434 127.0.0.1:8434
138```
139
140After restarting Tor, read the generated onion name:
141
142```bash
143sudo cat /var/lib/tor/bitflash-rendezvous/hostname
144```
145
146Then run the relay on localhost/VPS as usual and announce the onion endpoint from
147nodes that use Tor:
148
149```bash
150bitflash -tor -rvrelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434
151bitflash -tor -announcerelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434
152```
153
154For a pool operator that wants the pool descriptor to point at an onion
155rendezvous relay, use both:
156
157```bash
158bitflash -operator -tor \
159 -rvrelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 \
160 -announcerelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434
161```
162
163## Manual direct full-node hidden service
164
165Managed Tor is the preferred path for ordinary nodes. Operators can still expose
166the P2P listener through a separately managed Tor service:
167
168```text
169HiddenServiceDir /var/lib/tor/bitflash-node/
170HiddenServicePort 8433 127.0.0.1:8433
171```
172
173After restarting Tor, read the generated onion name:
174
175```bash
176sudo cat /var/lib/tor/bitflash-node/hostname
177```
178
179Then start Bitflash with Tor outbound enabled and advertise that hidden service:
180
181```bash
182bitflash -tor \
183 -onionservice=exampleexampleexampleexampleexampleexampleexampleexample.onion:8433
184```
185
186That onion endpoint is signed into this node's self-certifying `.btf`
187descriptor. New peers that resolve the descriptor and also run with `-tor` try
188the `.onion:8433` P2P connection first. If it fails, they fall back to the
189ordinary encrypted rendezvous path. Older nodes ignore the onion field and still
190use the rendezvous relay because the descriptor keeps the legacy signature too.
191
192If the node has `-onionservice` but has not registered at any rendezvous relay
193yet, it can still publish an onion-capable descriptor with
194`meeting_node = rendezvous-pending`. Tor-capable peers can dial the hidden
195service directly from that descriptor. Rendezvous becomes a fallback, not a
196precondition for being discoverable.
197
198This means rendezvous relays stop being the only way to reach a node that has
199published a hidden service. They remain useful bootstrap/fallback infrastructure,
200but an already-discovered onion-capable peer can be reached directly over Tor.
201
202Tor owns the private key under `HiddenServiceDir`; back it up if you want the
203onion name to stay stable.
204
205## Release smoke test
206
207After building a Windows binary with managed Tor support, run the two-node smoke
208test before cutting a release:
209
210```powershell
211powershell -ExecutionPolicy Bypass -File scripts\smoke-managed-tor-onion.ps1 `
212 -Bitflash .\src\bitflash.exe `
213 -Tor C:\path\to\tor.exe
214```
215
216The script creates two temporary datadirs, starts node A with `-managedtor`,
217waits for it to publish its `.btf` identity and generated `.onion` endpoint,
218then starts node B with its own `-managedtor`, `-oniononly`, and
219`-connectbtf=<node-a>`. It passes only when node B logs that it connected to
220node A `via direct onion`; rendezvous fallback cannot satisfy the test.
221
222By default the script waits 120 seconds after node A's descriptor self-resolves
223before it starts node B. Tor can create the local `hostname` before the hidden
224service descriptor is visible enough for client circuits, and early dials often
225fail with SOCKS5 reply `0x01`. Override this with `-OnionWarmupSec N` when
226testing a slow or already-warmed Tor environment.
227
228This test still uses Nostr for descriptor discovery, because that is how nodes
229learn each other's signed `.btf` descriptors. It does not require a Bitflash
230rendezvous relay to carry the P2P connection: once the descriptor includes the
231signed onion endpoint, the actual node connection goes directly through Tor.
diff --git a/docs/utxo-commitment.md b/docs/utxo-commitment.md
new file mode 100644
index 0000000..ab43deb
--- /dev/null
+++ b/docs/utxo-commitment.md
@@ -0,0 +1,125 @@
1# Bitflash UTXO-set commitment
2
3`scripts/verify-utxo-set.py` reconstructs a deterministic commitment to the
4current UTXO set from local `blk*.dat` files.
5
6This is intentionally not a consensus rule yet. It is an audit tool: independent
7operators can run it against the same chain prefix and compare the resulting
8root and canonical JSON hash.
9
10```bash
11python3 scripts/verify-utxo-set.py --max-blocks 1000 --out utxo-report.json --json
12```
13
14Use `--max-blocks 0` or omit the option to scan every available block file:
15
16```bash
17python3 scripts/verify-utxo-set.py --out utxo-report.json --json
18```
19
20## What It Verifies
21
22The verifier:
23
24- parses Bitflash block files directly with the shared Python chain parser;
25- streams `blk*.dat` records instead of loading full block files into memory;
26- reconstructs the best chain rooted at the Bitflash genesis block;
27- ignores side blocks that may also be present in `blk*.dat`;
28- verifies every scanned block merkle root;
29- applies spends and creates outputs in deterministic order;
30- reports current UTXO count, current supply, mined coinbase total, fees, and
31 historical duplicate-output overwrites;
32- computes a deterministic UTXO root.
33
34## Commitment Algorithm
35
36The UTXO map is keyed by:
37
38```text
39txid:vout
40```
41
42Keys are sorted lexicographically. Each leaf is:
43
44```text
45SHA256("BTFUTXO1|" || outpoint || "|" || value_satoshis || "|" ||
46 script_pub_key_hex || "|" || created_height || "|" || created_txid)
47```
48
49The tree is then reduced pairwise. If a level has an odd number of hashes, the
50last hash is duplicated. Each parent is:
51
52```text
53SHA256("BTFNODE1|" || left_hash || right_hash)
54```
55
56The final hash is the UTXO-set root.
57
58## Inclusion Proofs
59
60After a root has been computed, `scripts/prove-utxo.py` can create a compact
61inclusion proof for one unspent output:
62
63```bash
64python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json
65python3 scripts/verify-utxo-proof.py utxo-proof.json
66```
67
68The verifier does not need local block files. It recomputes the target leaf,
69walks the sibling path, and compares the result with the expected root. See
70[UTXO inclusion proofs](utxo-proofs.md).
71
72## Deterministic Output
73
74`--json` and `--out` write canonical JSON with sorted keys, fixed separators,
75ASCII escaping, and one trailing newline. It contains no local path and no
76generation timestamp, so two auditors scanning the same chain prefix should
77produce byte-identical reports.
78
79```bash
80python3 scripts/verify-utxo-set.py --out utxo-report.json --json
81sha256sum utxo-report.json
82```
83
84## OpenTimestamps
85
86The canonical JSON can be timestamped in Bitcoin without adding any dependency
87to Bitflash consensus:
88
89```bash
90ots stamp utxo-report.json
91ots verify utxo-report.json.ots
92```
93
94## Historical Duplicates
95
96Bitflash inherited early Bitcoin behavior around duplicate transaction IDs.
97The verifier reports duplicate-output overwrites as warnings and uses
98last-write-wins semantics for the UTXO map, matching the legacy index model.
99Those warnings are part of the report so auditors can see the historical debt
100instead of relying on a polished summary.
101
102For synthetic chains or future clean consensus ranges, auditors can make
103duplicates a hard failure:
104
105```bash
106python3 scripts/verify-utxo-set.py --strict-duplicates
107```
108
109The default remains compatible with the current historical chain because it has
110known legacy duplicate-output overwrites.
111
112## Shared Parser
113
114The audit scripts share `scripts/bitflash_chain.py` for block parsing,
115main-chain selection, UTXO application, canonical JSON, and genesis
116reconstruction. That keeps `verify-utxo-set.py`, `verify-fair-launch.py`,
117`prove-utxo.py`, and the explorer on the same serialization rules.
118
119## Future Consensus Path
120
121This tool is the low-risk first step toward consensus commitments. A future
122hard-fork design can commit this root, or a Utreexo-style accumulator root, in
123coinbase or block headers after a fixed activation height. Until then this
124script provides reproducible supply and state commitments without changing what
125nodes accept.
diff --git a/docs/utxo-proofs.md b/docs/utxo-proofs.md
new file mode 100644
index 0000000..4947c3c
--- /dev/null
+++ b/docs/utxo-proofs.md
@@ -0,0 +1,61 @@
1# Bitflash UTXO inclusion proofs
2
3`scripts/prove-utxo.py` creates a deterministic inclusion proof for one
4currently unspent output. `scripts/verify-utxo-proof.py` verifies that proof
5against the UTXO root produced by `scripts/verify-utxo-set.py`.
6
7This is intentionally outside consensus. It turns the audit root into something
8useful for a specific coin without changing what nodes accept.
9
10## Create A Proof
11
12Use an unspent output in `txid:vout` form:
13
14```bash
15python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json
16```
17
18The proof contains:
19
20- the target UTXO;
21- its sorted leaf index;
22- the leaf hash;
23- the Merkle sibling path;
24- the UTXO root and chain tip used to create it.
25
26It does not contain the full UTXO set.
27
28## Verify A Proof
29
30Verify against the root embedded in the proof:
31
32```bash
33python3 scripts/verify-utxo-proof.py utxo-proof.json
34```
35
36Or verify against a root copied from another machine, dashboard, block explorer,
37or future node export:
38
39```bash
40python3 scripts/verify-utxo-proof.py utxo-proof.json \
41 --root ROOT_FROM_REPORT_OR_EXPORT
42```
43
44The verifier does not need local `blk*.dat` files. It only needs the proof and
45the expected root.
46
47## Scope
48
49This first proof format supports inclusion only. Absence and range proofs are
50left for a later design because they require a more careful commitment scheme
51and are easier to get subtly wrong.
52
53The proof uses the exact same leaf and node domains as the UTXO commitment:
54
55```text
56BTFUTXO1
57BTFNODE1
58```
59
60That keeps the verifier language-neutral and gives future implementations a
61small, stable target.