diff options
| author | bitflash-maintainer <noreply@bitflash.invalid> | 2026-09-06 01:53:03 +0000 |
|---|---|---|
| committer | bitflash-maintainer <noreply@bitflash.invalid> | 2026-09-06 01:53:03 +0000 |
| commit | ef9f06d43fc6577c15df2256abbb25b20d4227b8 (patch) | |
| tree | 7e9132aa871fdc01cb95268d2f5ea3bb11853ddf /docs | |
| parent | be63e83d79e7e00e6b8ca889a70e6a19886aa1d8 (diff) | |
| parent | ceae3d076b5ad11ee84729ae43f14e9312896646 (diff) | |
Bring main onto the line this project actually ships (incl. the 1.2.23 compactblock port)
Closes #6, #7, #8, #11, #35, #34, #29, #28, #27, and #16
See merge request !16
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/dependencies.md | 18 | ||||
| -rw-r--r-- | docs/fair-launch.md | 70 | ||||
| -rw-r--r-- | docs/mining-calculator.md | 108 | ||||
| -rw-r--r-- | docs/pool-directory.md | 61 | ||||
| -rw-r--r-- | docs/release-verification.md | 94 | ||||
| -rw-r--r-- | docs/rendezvous.md | 336 | ||||
| -rw-r--r-- | docs/site.md | 115 | ||||
| -rw-r--r-- | docs/social.png | bin | 208957 -> 162267 bytes | |||
| -rw-r--r-- | docs/storage.md | 159 | ||||
| -rw-r--r-- | docs/tor.md | 231 | ||||
| -rw-r--r-- | docs/utxo-commitment.md | 125 | ||||
| -rw-r--r-- | docs/utxo-proofs.md | 61 |
12 files changed, 1351 insertions, 27 deletions
diff --git a/docs/dependencies.md b/docs/dependencies.md new file mode 100644 index 0000000..9fd45b6 --- /dev/null +++ b/docs/dependencies.md | |||
| @@ -0,0 +1,18 @@ | |||
| 1 | # Build Dependencies | ||
| 2 | |||
| 3 | Bitflash links consensus-adjacent cryptographic and proof-of-work code from | ||
| 4 | external repositories. Fresh builds must not follow upstream `master` silently: | ||
| 5 | a changed upstream branch can break reproducible builds or change behavior in | ||
| 6 | code that every node relies on. | ||
| 7 | |||
| 8 | The top-level `Makefile` pins source dependencies by full commit hash: | ||
| 9 | |||
| 10 | | dependency | repository | pinned commit | | ||
| 11 | |---|---|---| | ||
| 12 | | libsecp256k1 | https://github.com/bitcoin-core/secp256k1 | `7fecac74aed8e1fd9078380d67dd04663705c989` | | ||
| 13 | | RandomX | https://github.com/tevador/RandomX | `1e9d4b2df63fa6edf46b06789486e23c0fcaa55a` | | ||
| 14 | |||
| 15 | When updating either dependency, rebuild from a clean dependency directory and | ||
| 16 | run the full selftest suite before opening the PR. The PR should explain why the | ||
| 17 | dependency changed and whether the update can affect consensus, addresses, | ||
| 18 | signatures, proof-of-work validation, or mining. | ||
diff --git a/docs/fair-launch.md b/docs/fair-launch.md new file mode 100644 index 0000000..9603910 --- /dev/null +++ b/docs/fair-launch.md | |||
| @@ -0,0 +1,70 @@ | |||
| 1 | # Bitflash fair-launch verifier | ||
| 2 | |||
| 3 | Bitflash ships a deterministic fair-launch audit script: | ||
| 4 | |||
| 5 | ```bash | ||
| 6 | python3 scripts/verify-fair-launch.py --max-blocks 1000 --out bitflash-fair-launch-report.json --json | ||
| 7 | ``` | ||
| 8 | |||
| 9 | The verifier reads `blk*.dat` directly from the local Bitflash data directory. | ||
| 10 | It does not use Berkeley DB, does not call the node binary, and does not need | ||
| 11 | network access. It uses the same shared Python chain parser as the UTXO | ||
| 12 | commitment and proof tools, so block serialization fixes land in one place. | ||
| 13 | |||
| 14 | The report proves the launch baseline that can be checked from the chain: | ||
| 15 | |||
| 16 | - the genesis block hash matches the source constant; | ||
| 17 | - the genesis merkle root matches the source constant; | ||
| 18 | - the genesis transaction is a single coinbase transaction; | ||
| 19 | - the genesis output total is exactly `50.00000000 BTF`; | ||
| 20 | - the scanned block range and mined coinbase total are reproducible. | ||
| 21 | |||
| 22 | For machines that do not have a synced data directory yet, the source constants | ||
| 23 | can be checked without disk blocks: | ||
| 24 | |||
| 25 | ```bash | ||
| 26 | python3 scripts/verify-fair-launch.py --reconstruct-genesis | ||
| 27 | ``` | ||
| 28 | |||
| 29 | ## Deterministic output | ||
| 30 | |||
| 31 | Use `--json` or `--out` for canonical JSON. The JSON is written with sorted | ||
| 32 | keys, fixed separators, ASCII escaping, and one trailing newline. It deliberately | ||
| 33 | does not include local paths or wall-clock generation time, so two auditors | ||
| 34 | scanning the same block range produce the same bytes. | ||
| 35 | |||
| 36 | That makes the report shareable: | ||
| 37 | |||
| 38 | ```bash | ||
| 39 | python3 scripts/verify-fair-launch.py --max-blocks 1000 --out report.json --json | ||
| 40 | sha256sum report.json | ||
| 41 | ``` | ||
| 42 | |||
| 43 | If another auditor runs the same command against the same chain prefix, the | ||
| 44 | `sha256sum` should match. | ||
| 45 | |||
| 46 | ## OpenTimestamps | ||
| 47 | |||
| 48 | OpenTimestamps is optional and external. It anchors the report in Bitcoin | ||
| 49 | without adding any bridge or dependency to Bitflash consensus. | ||
| 50 | |||
| 51 | After generating a canonical JSON report: | ||
| 52 | |||
| 53 | ```bash | ||
| 54 | ots stamp report.json | ||
| 55 | ots verify report.json.ots | ||
| 56 | ``` | ||
| 57 | |||
| 58 | The `.ots` file proves that the exact report bytes existed no later than the | ||
| 59 | Bitcoin block that confirms the timestamp. | ||
| 60 | |||
| 61 | ## What this does not claim | ||
| 62 | |||
| 63 | This is not a retroactive genesis commitment. The Bitflash genesis block already | ||
| 64 | exists, so adding a new on-chain commitment to genesis would create a different | ||
| 65 | chain. This verifier instead audits the launch state that is already on-chain | ||
| 66 | and makes the result reproducible and timestampable. | ||
| 67 | |||
| 68 | It also does not prove real-world miner identity. It proves the genesis state, | ||
| 69 | the absence of non-coinbase premine in genesis, and the deterministic coinbase | ||
| 70 | totals over the scanned prefix. | ||
diff --git a/docs/mining-calculator.md b/docs/mining-calculator.md new file mode 100644 index 0000000..e6cb61b --- /dev/null +++ b/docs/mining-calculator.md | |||
| @@ -0,0 +1,108 @@ | |||
| 1 | # Bitflash mining calculator | ||
| 2 | |||
| 3 | `scripts/bitflash-mining-calculator.py` is a local WhatToMine-style calculator | ||
| 4 | for Bitflash. | ||
| 5 | |||
| 6 | It does not fetch market prices, does not talk to exchanges, and does not | ||
| 7 | change consensus. Price is a manual input until Bitflash has a reliable public | ||
| 8 | market. | ||
| 9 | |||
| 10 | ## Examples | ||
| 11 | |||
| 12 | Estimate from network hashrate: | ||
| 13 | |||
| 14 | ```bash | ||
| 15 | python3 scripts/bitflash-mining-calculator.py \ | ||
| 16 | --hashrate 21 --hashrate-unit kh/s \ | ||
| 17 | --network-hashrate 1.56 --network-hashrate-unit mh/s \ | ||
| 18 | --watts 140 --kwh-cost 0.10 \ | ||
| 19 | --price 0.001 --pool-fee 1 | ||
| 20 | ``` | ||
| 21 | |||
| 22 | Estimate from Bitflash difficulty: | ||
| 23 | |||
| 24 | ```bash | ||
| 25 | python3 scripts/bitflash-mining-calculator.py \ | ||
| 26 | --hashrate 21 --hashrate-unit kh/s \ | ||
| 27 | --difficulty 45600 \ | ||
| 28 | --watts 140 --kwh-cost 0.10 | ||
| 29 | ``` | ||
| 30 | |||
| 31 | Machine-readable output: | ||
| 32 | |||
| 33 | ```bash | ||
| 34 | python3 scripts/bitflash-mining-calculator.py \ | ||
| 35 | --hashrate 21 --hashrate-unit kh/s \ | ||
| 36 | --network-hashrate 1.56 --network-hashrate-unit mh/s \ | ||
| 37 | --json | ||
| 38 | ``` | ||
| 39 | |||
| 40 | ## Bitflash Difficulty Scale | ||
| 41 | |||
| 42 | Bitflash does not use Bitcoin's SHA256 difficulty-1 constant for pool shares. | ||
| 43 | The chain's easiest RandomX target is `bnProofOfWorkLimit = ~uint256(0) >> 12`, | ||
| 44 | so difficulty 1 expects about `4096` hashes. | ||
| 45 | |||
| 46 | The calculator uses: | ||
| 47 | |||
| 48 | ```text | ||
| 49 | network_hashrate = difficulty * 4096 / block_time_seconds | ||
| 50 | ``` | ||
| 51 | |||
| 52 | At the default two-minute block target: | ||
| 53 | |||
| 54 | ```text | ||
| 55 | network_hashrate = difficulty * 4096 / 120 | ||
| 56 | ``` | ||
| 57 | |||
| 58 | If you already know network hashrate, pass `--network-hashrate` directly and | ||
| 59 | the script derives the equivalent Bitflash difficulty for display. | ||
| 60 | |||
| 61 | ## Reward Estimate | ||
| 62 | |||
| 63 | The expected daily reward is: | ||
| 64 | |||
| 65 | ```text | ||
| 66 | blocks_per_day = 86400 / block_time_seconds | ||
| 67 | miner_share = miner_hashrate / network_hashrate | ||
| 68 | gross_btf_per_day = blocks_per_day * block_reward * miner_share | ||
| 69 | net_btf_per_day = gross_btf_per_day * (1 - pool_fee_percent / 100) | ||
| 70 | ``` | ||
| 71 | |||
| 72 | Electricity is: | ||
| 73 | |||
| 74 | ```text | ||
| 75 | electricity_per_day = watts / 1000 * 24 * kwh_cost | ||
| 76 | ``` | ||
| 77 | |||
| 78 | This is an expectation, not a promise. Solo mining variance can be very large, | ||
| 79 | especially when the miner has a small share of the network. | ||
| 80 | |||
| 81 | ## Future Website Use | ||
| 82 | |||
| 83 | The script is intentionally standalone so the same math can later back a | ||
| 84 | website widget or status dashboard. Until a public price exists, a website | ||
| 85 | calculator should keep price as a manual field and label fiat profit as | ||
| 86 | speculative. | ||
| 87 | |||
| 88 | ## Website Fields | ||
| 89 | |||
| 90 | A public calculator should ask for: | ||
| 91 | |||
| 92 | - miner hashrate and unit; | ||
| 93 | - power draw in watts; | ||
| 94 | - electricity price per kWh; | ||
| 95 | - pool fee percent; | ||
| 96 | - optional BTF price; | ||
| 97 | - either current Bitflash difficulty or current network hashrate. | ||
| 98 | |||
| 99 | The output should separate: | ||
| 100 | |||
| 101 | - estimated BTF per day; | ||
| 102 | - estimated gross fiat revenue, if a manual price was provided; | ||
| 103 | - electricity cost; | ||
| 104 | - net profit/loss; | ||
| 105 | - the difficulty/hashrate assumptions used. | ||
| 106 | |||
| 107 | Never present the fiat number as a market quote unless it comes from a real | ||
| 108 | public market. Before that exists, label it as a manual scenario. | ||
diff --git a/docs/pool-directory.md b/docs/pool-directory.md new file mode 100644 index 0000000..ff0640a --- /dev/null +++ b/docs/pool-directory.md | |||
| @@ -0,0 +1,61 @@ | |||
| 1 | # Public Pool Directory | ||
| 2 | |||
| 3 | `pool.bitflash.network` should be a public directory, not a network dependency. | ||
| 4 | Nodes discover pools through Nostr and `.btf`; the website only helps humans | ||
| 5 | compare operators. | ||
| 6 | |||
| 7 | Operators can publish the `pool_status.json` and `pool_rounds.json` files | ||
| 8 | produced by: | ||
| 9 | |||
| 10 | ```bash | ||
| 11 | bitflash -nogui -gen -operator \ | ||
| 12 | -poolstatusfile=/var/www/pool_status.json \ | ||
| 13 | -poolroundsfile=/var/www/pool_rounds.json | ||
| 14 | ``` | ||
| 15 | |||
| 16 | Build a static directory from one or more status files: | ||
| 17 | |||
| 18 | ```bash | ||
| 19 | python3 scripts/build-pool-index.py --out-dir public/pool \ | ||
| 20 | https://operator-one.example/pool_status.json \ | ||
| 21 | https://operator-two.example/pool_status.json | ||
| 22 | ``` | ||
| 23 | |||
| 24 | The generated output contains: | ||
| 25 | |||
| 26 | - `index.html`: human-readable ranking page. | ||
| 27 | - `pools.json`: machine-readable API for `api.bitflash.network/pools.json`. | ||
| 28 | |||
| 29 | The website is a convenience layer, not a consensus or discovery dependency. | ||
| 30 | Nodes discover pools through Nostr descriptors and `.btf`; a missing website | ||
| 31 | must not stop mining or pool operation. | ||
| 32 | |||
| 33 | When a source is an HTTP(S) `pool_status.json`, the directory links to a | ||
| 34 | `pool_rounds.json` proof file next to it. That file records the block height, | ||
| 35 | block hash, coinbase txid, coinbase value, pool fee, total shares, per-address | ||
| 36 | share counts, computed payouts, and payout txids once paid. | ||
| 37 | |||
| 38 | Ranking order is intentionally simple: | ||
| 39 | |||
| 40 | 1. online pools first; | ||
| 41 | 2. higher reported hashrate; | ||
| 42 | 3. more authorized miners; | ||
| 43 | 4. lower fee; | ||
| 44 | 5. name as a stable tie-breaker. | ||
| 45 | |||
| 46 | An entry is considered online when the source says the pool is running and its | ||
| 47 | status file is no more than 120 seconds old. | ||
| 48 | |||
| 49 | ## Recommended Hostnames | ||
| 50 | |||
| 51 | Use: | ||
| 52 | |||
| 53 | ```text | ||
| 54 | pool.bitflash.network human-readable pool directory | ||
| 55 | api.bitflash.network/pools.json | ||
| 56 | status.bitflash.network relay/node health, not pool ranking | ||
| 57 | docs.bitflash.network operator setup guide | ||
| 58 | ``` | ||
| 59 | |||
| 60 | The GUI may use the API as a shortcut for humans, but it should keep the | ||
| 61 | existing decentralized pool discovery path as the source of truth. | ||
diff --git a/docs/release-verification.md b/docs/release-verification.md index ecacd5f..b8eaf5e 100644 --- a/docs/release-verification.md +++ b/docs/release-verification.md | |||
| @@ -3,60 +3,100 @@ | |||
| 3 | Bitflash releases publish binaries and a `SHA256SUMS` file. The checksum file | 3 | Bitflash releases publish binaries and a `SHA256SUMS` file. The checksum file |
| 4 | proves that the file you downloaded matches the file the release page names. | 4 | proves that the file you downloaded matches the file the release page names. |
| 5 | A detached signature, `SHA256SUMS.asc`, proves that the checksum file itself was | 5 | A detached signature, `SHA256SUMS.asc`, proves that the checksum file itself was |
| 6 | signed by a trusted release key. | 6 | signed by the Bitflash release key. |
| 7 | 7 | ||
| 8 | That gives users two separate checks: | 8 | That gives users two separate checks: |
| 9 | 9 | ||
| 10 | 1. `gpg --verify SHA256SUMS.asc SHA256SUMS` checks who signed the checksums. | 10 | 1. `gpg --verify SHA256SUMS.asc SHA256SUMS` — who signed the checksums. |
| 11 | 2. `sha256sum -c SHA256SUMS` checks the binaries against those checksums. | 11 | 2. `sha256sum -c SHA256SUMS` — the binaries against those checksums. |
| 12 | 12 | ||
| 13 | ## User Check | 13 | But the first check is only worth something if you require a **specific** key. |
| 14 | A good signature by *some* key proves nothing; anyone can make one. | ||
| 14 | 15 | ||
| 15 | From a shell with `curl` or `wget` and `sha256sum`: | 16 | ## The release signing key |
| 17 | |||
| 18 | ```text | ||
| 19 | fingerprint 910A 2B4C CA87 9E81 FB4B 2AEA 1D4A 53D3 B78A A4B8 | ||
| 20 | uid Bitflash Releases (bitflash.network release signing) | ||
| 21 | <releases@bitflash.network> | ||
| 22 | ``` | ||
| 23 | |||
| 24 | The public key is published at | ||
| 25 | `https://releases.bitflash.network/bitflash-release-key.asc` and embedded in | ||
| 26 | `scripts/verify-release.sh`, so the tool's trust anchor lives in this | ||
| 27 | repository rather than on the server it verifies. | ||
| 28 | |||
| 29 | ## User check | ||
| 30 | |||
| 31 | From a shell with `curl` or `wget`, `sha256sum` and `gpg`: | ||
| 16 | 32 | ||
| 17 | ```bash | 33 | ```bash |
| 18 | scripts/verify-release.sh latest | 34 | scripts/verify-release.sh latest |
| 19 | ``` | 35 | ``` |
| 20 | 36 | ||
| 21 | For a specific release: | 37 | The public release host should expose latest-file aliases for convenience: |
| 22 | 38 | ||
| 23 | ```bash | 39 | ```text |
| 24 | scripts/verify-release.sh v1.2.13 | 40 | https://releases.bitflash.network/SHA256SUMS |
| 41 | https://releases.bitflash.network/SHA256SUMS.asc | ||
| 25 | ``` | 42 | ``` |
| 26 | 43 | ||
| 27 | For release audits, require the signature: | 44 | Those aliases must point at the current versioned directory. They do not replace |
| 45 | signature verification; they only make the common manual path shorter. | ||
| 46 | |||
| 47 | For a specific release: | ||
| 28 | 48 | ||
| 29 | ```bash | 49 | ```bash |
| 30 | scripts/verify-release.sh v1.2.13 --require-signature | 50 | scripts/verify-release.sh v1.2.15 |
| 31 | ``` | 51 | ``` |
| 32 | 52 | ||
| 33 | Older releases may not have `SHA256SUMS.asc`. In that case the script warns and | 53 | The script downloads the release, imports the pinned key into a throwaway |
| 34 | still checks file integrity. New release audits should use `--require-signature`. | 54 | keyring, and **refuses anything that is not a valid signature by the key above** |
| 35 | 55 | before checking the hashes. Releases from before signing was introduced ship | |
| 36 | ## Maintainer Flow | 56 | only `SHA256SUMS`; without `--require-signature` those are checked for integrity |
| 57 | and reported as unsigned. | ||
| 37 | 58 | ||
| 38 | After building release assets in the repository root: | 59 | By hand, if you would rather not run the script: |
| 39 | 60 | ||
| 40 | ```bash | 61 | ```bash |
| 41 | scripts/make-release-checksums.sh --sign --local-user RELEASE_KEY_ID | 62 | curl -O https://releases.bitflash.network/bitflash-release-key.asc |
| 63 | gpg --import bitflash-release-key.asc | ||
| 64 | gpg --verify SHA256SUMS.asc SHA256SUMS # must name the fingerprint above | ||
| 65 | sha256sum -c SHA256SUMS | ||
| 42 | ``` | 66 | ``` |
| 43 | 67 | ||
| 44 | Upload all built assets plus: | 68 | ## Maintainer flow |
| 45 | 69 | ||
| 46 | ```text | 70 | The signing key lives on a dedicated location, not in the build tree. After |
| 47 | SHA256SUMS | 71 | building the release assets in the repository root, write and sign the |
| 48 | SHA256SUMS.asc | 72 | checksums: |
| 73 | |||
| 74 | ```bash | ||
| 75 | sha256sum Bitflash-*.zip Bitflash-*.AppImage bitflash-node-* > SHA256SUMS | ||
| 76 | gpg --armor --detach-sign --output SHA256SUMS.asc SHA256SUMS | ||
| 49 | ``` | 77 | ``` |
| 50 | 78 | ||
| 51 | Keep the private signing key offline or on a dedicated release machine. Publish | 79 | Publish, alongside the assets, `SHA256SUMS` and `SHA256SUMS.asc`, and update |
| 52 | the public key fingerprint in the release notes and keep using the same key for | 80 | `latest.txt` on the releases host to the new tag. Keep using the same key; if it |
| 53 | future releases unless there is a clearly announced rotation. | 81 | is ever rotated, announce the new fingerprint clearly and update the embedded |
| 82 | key in `verify-release.sh`. | ||
| 83 | |||
| 84 | For Windows releases with managed Tor built in, run `make windows-tor`. It | ||
| 85 | creates `Bitflash-*-windows-with-tor.zip` by downloading the Tor Expert Bundle, | ||
| 86 | checking the pinned SHA256 in `scripts/package-windows-tor.sh`, optionally | ||
| 87 | verifying the Tor Project `.asc` signature, and placing `tor/tor.exe` next to | ||
| 88 | `Bitflash.exe`. For release builds, run `TOR_VERIFY_GPG=required make | ||
| 89 | windows-tor` so the signature check is mandatory. Include that zip in | ||
| 90 | `SHA256SUMS` like any other release asset. | ||
| 54 | 91 | ||
| 55 | ## Why This Matters | 92 | ## Why this matters |
| 56 | 93 | ||
| 57 | `SHA256SUMS` alone protects against a broken download, but not against someone | 94 | `SHA256SUMS` alone protects against a broken download, but not against someone |
| 58 | replacing both a binary and the checksum file. Signing `SHA256SUMS` means an | 95 | replacing both a binary and the checksum file. Signing `SHA256SUMS`, and |
| 59 | attacker must also have the release signing key to make the replacement verify. | 96 | requiring a pinned key, means an attacker must also hold the release signing key |
| 97 | to make the replacement verify. Serving from bitflash.network makes this the | ||
| 98 | guarantee that matters: without a pinned key, "trust the site" is the only | ||
| 99 | thing standing behind a download. | ||
| 60 | 100 | ||
| 61 | This is not reproducible builds yet. It is the smaller, immediate step that | 101 | This is not reproducible builds yet. It is the smaller, immediate step that |
| 62 | makes every release asset auditable by users before they run it. | 102 | makes every release asset auditable before anyone runs it. |
diff --git a/docs/rendezvous.md b/docs/rendezvous.md new file mode 100644 index 0000000..68b5df3 --- /dev/null +++ b/docs/rendezvous.md | |||
| @@ -0,0 +1,336 @@ | |||
| 1 | # Bitflash `.btf` Rendezvous Protocol | ||
| 2 | |||
| 3 | This is the specification of how Bitflash nodes reach each other, and how miners | ||
| 4 | reach pools, without revealing an IP address and without any central directory. | ||
| 5 | It documents the protocol as implemented in `src/btfaddr.cpp`, `src/btfchan.cpp`, | ||
| 6 | `src/btfrv.cpp`, `src/btftunnel.cpp` and `src/nostr.cpp`. | ||
| 7 | |||
| 8 | The design is ported from the Itzal hidden service design. A `.btf` address names | ||
| 9 | a public key, a node proves ownership of that key by signing a service | ||
| 10 | descriptor, other nodes find the descriptor over Nostr, and the two sides then | ||
| 11 | meet through an untrusted relay over an end to end encrypted tunnel. The relay | ||
| 12 | carries ciphertext only and never learns the plaintext. | ||
| 13 | |||
| 14 | ## 1. Overview | ||
| 15 | |||
| 16 | A connection to a `.btf` service has three layers: | ||
| 17 | |||
| 18 | 1. **Discovery (Nostr).** The service publishes a signed descriptor as a Nostr | ||
| 19 | event. A caller fetches it by the service public key and verifies the | ||
| 20 | signature. The descriptor says which rendezvous relay the service is waiting | ||
| 21 | at, which encryption key to use, and optionally which Tor hidden service can | ||
| 22 | be dialed directly. | ||
| 23 | 2. **Rendezvous (TCP).** Caller and service both connect out to the same relay | ||
| 24 | and are paired by public key. Neither needs an inbound port or a public IP. | ||
| 25 | After pairing the relay is a blind byte pipe. | ||
| 26 | 3. **Direct onion fallback bypass.** If the descriptor carries a valid signed | ||
| 27 | `.onion:port` endpoint and the caller is running with Tor mode, the caller | ||
| 28 | tries that endpoint before using rendezvous. This removes the rendezvous | ||
| 29 | relay from the data path for peers that choose to run hidden services. | ||
| 30 | 4. **Encrypted channel (X25519 + XChaCha20-Poly1305).** Over the paired pipe the | ||
| 31 | caller and service run an ephemeral to static handshake and then exchange | ||
| 32 | authenticated frames. The relay sees only ciphertext. | ||
| 33 | |||
| 34 | The caller ends up holding an ordinary local socket that behaves like a direct | ||
| 35 | TCP connection to the peer. The node treats it exactly like any other peer | ||
| 36 | socket. | ||
| 37 | |||
| 38 | ## 2. Identities and keys | ||
| 39 | |||
| 40 | A node holds two independent keypairs. | ||
| 41 | |||
| 42 | - **Signing / identity key**: a secp256k1 x-only key (BIP340 Schnorr). This is | ||
| 43 | the `.btf` identity and also the node's Nostr identity. Persisted in | ||
| 44 | `nostr.key`. | ||
| 45 | - **Encryption key**: an X25519 (Curve25519) keypair. Persisted in | ||
| 46 | `btf_enc.key`. Its public half is published in the descriptor as `enc` and is | ||
| 47 | used for the tunnel key agreement. | ||
| 48 | |||
| 49 | The identity key names the service and authenticates the descriptor. The | ||
| 50 | encryption key secures the tunnel. They are separate so the long term identity | ||
| 51 | never performs Diffie-Hellman and the encryption key can be rotated in a new | ||
| 52 | signed descriptor. | ||
| 53 | |||
| 54 | ## 3. The `.btf` address | ||
| 55 | |||
| 56 | An address is the identity public key plus a short checksum, Base32 encoded, with | ||
| 57 | the suffix `.btf`. | ||
| 58 | |||
| 59 | ``` | ||
| 60 | label = Base32( pubkey[32] || checksum[2] ) | ||
| 61 | address = label + ".btf" | ||
| 62 | ``` | ||
| 63 | |||
| 64 | - `pubkey` is the 32 byte secp256k1 x-only public key. | ||
| 65 | - `checksum` is the first two bytes of `SHA256(".btf-checksum-v1" || pubkey)`. | ||
| 66 | - Base32 uses the lowercase alphabet `abcdefghijklmnopqrstuvwxyz234567`, no | ||
| 67 | padding. The 34 byte payload encodes to 55 characters. | ||
| 68 | - Parsing is case insensitive, trims surrounding whitespace, and strips a | ||
| 69 | trailing dot. A wrong checksum is rejected. | ||
| 70 | |||
| 71 | The address is self certifying: it *is* the public key, so no registry has to be | ||
| 72 | trusted to map a name to a key. | ||
| 73 | |||
| 74 | ## 4. Discovery over Nostr | ||
| 75 | |||
| 76 | Nodes publish and read three kinds of parameterized replaceable Nostr events. | ||
| 77 | Parameterized replaceable means the newest event for a given author and `d` tag | ||
| 78 | supersedes the older one, so a descriptor is always the latest state. | ||
| 79 | |||
| 80 | | Purpose | kind | `d` tag | | ||
| 81 | |---|---|---| | ||
| 82 | | Service descriptor | 38501 | `btf-descriptor-2` | | ||
| 83 | | Rendezvous relay announcement | 38502 | `btf-relay-2` | | ||
| 84 | | Pool announcement | 38503 | `btf-pool-2` | | ||
| 85 | |||
| 86 | The Nostr event is signed with the node's identity key, so the event author | ||
| 87 | equals the `.btf` public key. That binds the descriptor to the address at the | ||
| 88 | Nostr layer, and the descriptor is *also* signed inside its own content (see | ||
| 89 | below), so a relay operator cannot forge or swap a descriptor for an address it | ||
| 90 | does not control. | ||
| 91 | |||
| 92 | **Resolving an address.** To reach `X.btf`, a caller parses the address to the 32 | ||
| 93 | byte key and sends a Nostr `REQ` filtered by `authors = [X]`, `kinds = [38501]`, | ||
| 94 | `#d = ["btf-descriptor-2"]`, `limit = 1`. It verifies the returned descriptor and | ||
| 95 | uses its `meeting_node` and `enc`. | ||
| 96 | |||
| 97 | **Relay auto discovery.** A relay operator runs a rendezvous relay and publishes a | ||
| 98 | kind 38502 event carrying `host:port`. Every node that reads it adds the relay to | ||
| 99 | the set it can register at and dial through. No seed list edit or maintainer | ||
| 100 | approval is needed. A small curated seed list is still shipped for bootstrap: | ||
| 101 | |||
| 102 | ``` | ||
| 103 | 92.246.128.180:8434 | ||
| 104 | 31.44.4.249:8434 | ||
| 105 | 90.156.222.107:8434 | ||
| 106 | ``` | ||
| 107 | |||
| 108 | The default rendezvous port is `8434`. | ||
| 109 | |||
| 110 | ## 5. The service descriptor | ||
| 111 | |||
| 112 | The descriptor is a JSON object, signed with the identity key over a domain | ||
| 113 | separated digest of its own fields. | ||
| 114 | |||
| 115 | Legacy descriptor: | ||
| 116 | |||
| 117 | ```json | ||
| 118 | { | ||
| 119 | "v": 2, | ||
| 120 | "pubkey": "<hex 32-byte x-only identity key>", | ||
| 121 | "enc": "<hex 32-byte X25519 public key>", | ||
| 122 | "meeting_node": "host:port", | ||
| 123 | "created": <unix seconds>, | ||
| 124 | "sig": "<hex 64-byte BIP340 Schnorr signature>" | ||
| 125 | } | ||
| 126 | ``` | ||
| 127 | |||
| 128 | Onion-capable descriptor: | ||
| 129 | |||
| 130 | ```json | ||
| 131 | { | ||
| 132 | "v": 3, | ||
| 133 | "pubkey": "<hex 32-byte x-only identity key>", | ||
| 134 | "enc": "<hex 32-byte X25519 public key>", | ||
| 135 | "meeting_node": "host:port", | ||
| 136 | "onion": "<56-char-v3-name>.onion:8433", | ||
| 137 | "created": <unix seconds>, | ||
| 138 | "sig": "<legacy v2 signature>", | ||
| 139 | "sig3": "<signature over the onion-capable digest>" | ||
| 140 | } | ||
| 141 | ``` | ||
| 142 | |||
| 143 | The legacy signed digest is: | ||
| 144 | |||
| 145 | ``` | ||
| 146 | digest = SHA256( ".btf-descriptor-v2" | ||
| 147 | || pubkey[32] | ||
| 148 | || enc (raw bytes of the string) | ||
| 149 | || meeting_node (raw bytes of the string) | ||
| 150 | || created (8 bytes, big endian) ) | ||
| 151 | ``` | ||
| 152 | |||
| 153 | `sig` is kept on v3 descriptors so old nodes can still resolve and dial through | ||
| 154 | the rendezvous relay. The onion-capable digest is: | ||
| 155 | |||
| 156 | ``` | ||
| 157 | digest3 = SHA256( ".btf-descriptor-v3-onion" | ||
| 158 | || pubkey[32] | ||
| 159 | || enc (raw bytes of the string) | ||
| 160 | || meeting_node (raw bytes of the string) | ||
| 161 | || onion (raw bytes of the normalized string) | ||
| 162 | || created (8 bytes, big endian) ) | ||
| 163 | ``` | ||
| 164 | |||
| 165 | `sig` and `sig3` are deterministic BIP340 Schnorr signatures (no auxiliary | ||
| 166 | randomness) over their respective digests by the identity key. | ||
| 167 | |||
| 168 | **Verification** (untrusted input, must never throw): parse the JSON, require all | ||
| 169 | fields with the right types, confirm `pubkey` equals the address being resolved, | ||
| 170 | recompute the digest, and verify the Schnorr signature. Any failure rejects the | ||
| 171 | descriptor. | ||
| 172 | |||
| 173 | If `onion` is present, readers accept it only when it is a normalized v3 onion | ||
| 174 | endpoint and `sig3` verifies. A bad or tampered onion field is ignored; the v2 | ||
| 175 | descriptor can still be used for rendezvous fallback if `sig` verifies. | ||
| 176 | |||
| 177 | `created` lets a reader prefer the freshest descriptor when more than one is | ||
| 178 | seen. A node without a direct onion endpoint only publishes a descriptor once it | ||
| 179 | has actually registered at a `meeting_node`, so it never advertises a relay it | ||
| 180 | cannot be reached at. A node with a direct onion endpoint may publish before | ||
| 181 | rendezvous registration using `meeting_node = "rendezvous-pending"`; Tor peers | ||
| 182 | can still dial the signed onion endpoint, and rendezvous becomes fallback once | ||
| 183 | registration succeeds. | ||
| 184 | |||
| 185 | ## 6. Rendezvous transport | ||
| 186 | |||
| 187 | The relay is a small TCP server. Every connection opens with a fixed 33 byte | ||
| 188 | header: | ||
| 189 | |||
| 190 | ``` | ||
| 191 | byte 0 role: 'S' (service) or 'C' (client) | ||
| 192 | bytes 1..32 target public key (the .btf identity, 32 bytes) | ||
| 193 | ``` | ||
| 194 | |||
| 195 | **Service registration (`S`).** The service connects out to the relay and sends | ||
| 196 | `'S' || own_pubkey`. The relay stores the connection in a table keyed by the | ||
| 197 | public key and leaves it open, waiting. There is one slot per key: a new | ||
| 198 | registration for the same key replaces and closes the old one. The service does | ||
| 199 | not block waiting to be paired at registration time; it is considered registered | ||
| 200 | the moment the relay reads the header, which is what lets it publish a descriptor | ||
| 201 | pointing at this relay. | ||
| 202 | |||
| 203 | **Client dial (`C`).** The caller connects out to the same relay and sends | ||
| 204 | `'C' || target_pubkey`. The relay looks up the waiting service: | ||
| 205 | |||
| 206 | - If present, it removes it from the table, writes a single `0x01` byte to both | ||
| 207 | sockets, and then forwards raw bytes in both directions until either side | ||
| 208 | closes. From this point the relay is a blind pipe. | ||
| 209 | - If absent, it writes `0x00` to the caller and closes. | ||
| 210 | |||
| 211 | Both sides connect *outbound* to the relay, so neither needs a public IP or an | ||
| 212 | open inbound port. The relay never sees the identity of the caller (the caller | ||
| 213 | sends no key of its own, only the target it wants). | ||
| 214 | |||
| 215 | **Liveness.** A registration socket is idle by design and can outlive the network | ||
| 216 | path that created it (a NAT drops the idle mapping, the relay restarts, a route | ||
| 217 | changes) with no FIN and nothing readable. TCP keepalive was measured and does | ||
| 218 | not reliably detect this. The service therefore imposes its own timeout on the | ||
| 219 | wait for the pairing byte and re-registers on expiry. Expiring is not a failure; | ||
| 220 | it costs one reconnect per interval and returns the loop to a known state. | ||
| 221 | |||
| 222 | ## 7. The encrypted channel | ||
| 223 | |||
| 224 | Once paired through the relay, the caller and service run an end to end handshake | ||
| 225 | and then exchange authenticated frames. The primitive is libsodium | ||
| 226 | `crypto_box_curve25519xchacha20poly1305` (X25519 key agreement, XChaCha20-Poly1305 | ||
| 227 | AEAD), the same construction as Rust's `crypto_box` ChaChaBox. | ||
| 228 | |||
| 229 | **Handshake (ephemeral to static).** | ||
| 230 | |||
| 231 | 1. The caller generates a fresh ephemeral X25519 keypair, computes | ||
| 232 | `shared = X25519(eph_sk, service_enc_pub)` reduced to a box key | ||
| 233 | (`crypto_box_beforenm`), and sends the 32 byte ephemeral public key over the | ||
| 234 | paired pipe. | ||
| 235 | 2. The service reads the ephemeral public key and computes the same | ||
| 236 | `shared = X25519(enc_sk, eph_pub)`. | ||
| 237 | |||
| 238 | Because the caller uses a new ephemeral key per connection, each session has an | ||
| 239 | independent key and forward secrecy against later compromise of the service | ||
| 240 | encryption key. | ||
| 241 | |||
| 242 | **Framing.** Every message is length prefixed: | ||
| 243 | |||
| 244 | ``` | ||
| 245 | frame = len[4] (big endian, length of the body that follows) | ||
| 246 | body = nonce[24] || ciphertext || mac[16] | ||
| 247 | ``` | ||
| 248 | |||
| 249 | The nonce is 24 random bytes per frame (XChaCha20), the MAC is 16 bytes | ||
| 250 | (Poly1305). A body shorter than `nonce + mac` or larger than 1 MiB is rejected. | ||
| 251 | Decryption that fails authentication drops the frame and tears the tunnel down. | ||
| 252 | |||
| 253 | The tunnel is transparent: the application is handed a local socket pair end and | ||
| 254 | reads and writes plaintext. Two pump threads move data between that socket and | ||
| 255 | the relay socket, encrypting outbound and decrypting inbound. | ||
| 256 | |||
| 257 | ## 8. End to end connection flow | ||
| 258 | |||
| 259 | To dial `X.btf`: | ||
| 260 | |||
| 261 | 1. Parse `X.btf` to the 32 byte identity key. | ||
| 262 | 2. Fetch and verify the descriptor over Nostr (kind 38501). Obtain | ||
| 263 | `meeting_node` and `enc`. | ||
| 264 | 3. Connect to `meeting_node`, send `'C' || X`, and wait for `0x01`. | ||
| 265 | 4. Generate an ephemeral X25519 key, derive the shared key against `enc`, send | ||
| 266 | the ephemeral public key. | ||
| 267 | 5. Exchange XChaCha20-Poly1305 frames. Hand the application a local socket that | ||
| 268 | speaks the ordinary peer protocol. | ||
| 269 | |||
| 270 | A service, symmetrically: registers at a relay (`'S' || own_pubkey`), publishes a | ||
| 271 | descriptor pointing at that relay, waits for `0x01`, reads the caller ephemeral | ||
| 272 | public key, derives the shared key with its encryption secret, and serves the | ||
| 273 | tunnel. | ||
| 274 | |||
| 275 | ## 9. Security properties | ||
| 276 | |||
| 277 | - **Address unforgeability.** The address is the identity key and the descriptor | ||
| 278 | is Schnorr signed by it, and the Nostr event is signed by it. A relay or Nostr | ||
| 279 | operator cannot publish a working descriptor for an address it does not own. | ||
| 280 | - **Relay confidentiality.** The relay forwards ciphertext and never holds the | ||
| 281 | channel key. It cannot read or modify traffic undetected; a tampered frame | ||
| 282 | fails the Poly1305 tag and the tunnel closes. | ||
| 283 | - **Caller anonymity to the service and relay.** The caller presents no identity | ||
| 284 | key, only the target it wants to reach. It connects outbound, so its address is | ||
| 285 | not published anywhere. | ||
| 286 | - **Forward secrecy.** The caller's per connection ephemeral key means a later | ||
| 287 | compromise of the service encryption key does not decrypt past sessions. | ||
| 288 | - **IP privacy.** Neither side needs a public IP or an inbound port for | ||
| 289 | rendezvous. With a signed `.onion` endpoint, the reachable side exposes only a | ||
| 290 | Tor hidden service and the relay is not in the peer connection path. | ||
| 291 | |||
| 292 | ## 10. Threat model and limitations | ||
| 293 | |||
| 294 | - **The relay is trusted for availability, not for confidentiality.** It can | ||
| 295 | refuse to pair, drop a connection, or go offline. This is why relays are | ||
| 296 | plural, auto discovered, and re-registration is periodic. It cannot read | ||
| 297 | traffic. | ||
| 298 | - **Traffic analysis is not addressed.** A network observer, or the relay, can | ||
| 299 | see connection timing and byte volumes. There is no padding or mixing. | ||
| 300 | - **Single slot registration causes churn.** One waiting slot per key per relay | ||
| 301 | means a race or a stale registration can briefly make a service unreachable at | ||
| 302 | that relay. Multiple relays and the wait timeout bound the damage, but this is | ||
| 303 | a known source of connection churn. | ||
| 304 | - **Descriptor freshness is best effort.** `created` is a self reported | ||
| 305 | timestamp; readers prefer the newest descriptor they have seen but there is no | ||
| 306 | global clock. | ||
| 307 | - **Discovery depends on public Nostr relays.** If the relays a node uses are all | ||
| 308 | unreachable, discovery stalls until one is reachable again. Relay TLS is | ||
| 309 | verified, which closes a man in the middle path on discovery, but availability | ||
| 310 | still rests on the relay set. | ||
| 311 | - **Direct onion needs Tor.** With `-managedtor`, Bitflash starts a Tor process, | ||
| 312 | writes a local hidden-service config, and advertises the generated onion | ||
| 313 | endpoint. With `-onionservice`, an operator can still run Tor manually and | ||
| 314 | provide the endpoint. In both cases Tor owns the hidden-service private key and | ||
| 315 | uptime. | ||
| 316 | |||
| 317 | ## 11. Wire reference | ||
| 318 | |||
| 319 | | Item | Value | | ||
| 320 | |---|---| | ||
| 321 | | `.btf` Base32 alphabet | `abcdefghijklmnopqrstuvwxyz234567`, no padding | | ||
| 322 | | Address payload | `pubkey[32] || checksum[2]`, 55 Base32 chars | | ||
| 323 | | Address checksum | `SHA256(".btf-checksum-v1" || pubkey)[0..2]` | | ||
| 324 | | Descriptor domain sep | `.btf-descriptor-v2`; optional onion sep `.btf-descriptor-v3-onion` | | ||
| 325 | | Descriptor version | `2`; `3` when a signed onion endpoint is present | | ||
| 326 | | Descriptor signature | BIP340 Schnorr, deterministic (no aux rand), v3 keeps `sig` plus `sig3` | | ||
| 327 | | Nostr descriptor event | kind `38501`, `d = btf-descriptor-2` | | ||
| 328 | | Nostr relay event | kind `38502`, `d = btf-relay-2` | | ||
| 329 | | Nostr pool event | kind `38503`, `d = btf-pool-2` | | ||
| 330 | | Rendezvous port (default) | `8434` | | ||
| 331 | | Rendezvous header | `role[1] ('S'/'C') || pubkey[32]`, 33 bytes | | ||
| 332 | | Rendezvous pairing byte | `0x01` paired, `0x00` no service | | ||
| 333 | | Channel primitive | `crypto_box_curve25519xchacha20poly1305` | | ||
| 334 | | Nonce / MAC / key sizes | 24 / 16 / 32 bytes | | ||
| 335 | | Frame | `len[4 big endian] || nonce[24] || ct || mac[16]` | | ||
| 336 | | Max frame body | 1 MiB | | ||
diff --git a/docs/site.md b/docs/site.md new file mode 100644 index 0000000..9526a61 --- /dev/null +++ b/docs/site.md | |||
| @@ -0,0 +1,115 @@ | |||
| 1 | # Public Site Contract | ||
| 2 | |||
| 3 | The public Bitflash sites are generated from this repository. If a deployed | ||
| 4 | page and the source disagree, the deployed page is stale. | ||
| 5 | |||
| 6 | ## Canonical Navigation | ||
| 7 | |||
| 8 | Every public page should use the same top navigation: | ||
| 9 | |||
| 10 | | label | target | | ||
| 11 | |---|---| | ||
| 12 | | home | `https://bitflash.network/` | | ||
| 13 | | code | `https://git.bitflash.network/bitflash/bitflash` | | ||
| 14 | | docs | `https://docs.bitflash.network/` | | ||
| 15 | | downloads | `https://releases.bitflash.network/` | | ||
| 16 | | explorer | `https://explorer.bitflash.network/` | | ||
| 17 | | status | `https://status.bitflash.network/` | | ||
| 18 | |||
| 19 | The `code` link intentionally points at the repository, not the Forgejo | ||
| 20 | instance root. A visitor choosing "code" wants the source tree. | ||
| 21 | |||
| 22 | ## Documentation Pages To Publish | ||
| 23 | |||
| 24 | The docs index should expose these source files: | ||
| 25 | |||
| 26 | | page | source | why it matters | | ||
| 27 | |---|---|---| | ||
| 28 | | Manual | `README.md` | run, mine, recover, operate | | ||
| 29 | | Key derivation and address format | `docs/derivation.md` | independent wallet recovery | | ||
| 30 | | Release verification | `docs/release-verification.md` | pinned signing-key flow | | ||
| 31 | | Dependencies | `docs/dependencies.md` | pinned secp256k1/RandomX supply chain | | ||
| 32 | | Rendezvous and .btf addressing | `docs/rendezvous.md` | network design and caveats | | ||
| 33 | | Tor mode | `docs/tor.md` | `-tor`, `-socks`, onion relay operation | | ||
| 34 | | Fair launch verifier | `docs/fair-launch.md` | no-premine launch audit | | ||
| 35 | | UTXO set commitment | `docs/utxo-commitment.md` | deterministic supply/root audit | | ||
| 36 | | UTXO inclusion proofs | `docs/utxo-proofs.md` | proof for one unspent output | | ||
| 37 | | Mining calculator | `docs/mining-calculator.md` | local WhatToMine-style estimates | | ||
| 38 | | Public pool directory | `docs/pool-directory.md` | `pool_status.json`, `pools.json`, ranking | | ||
| 39 | |||
| 40 | ## Release Mirror Aliases | ||
| 41 | |||
| 42 | Each release remains under its versioned directory: | ||
| 43 | |||
| 44 | ```text | ||
| 45 | https://releases.bitflash.network/v1.2.19/SHA256SUMS | ||
| 46 | https://releases.bitflash.network/v1.2.19/SHA256SUMS.asc | ||
| 47 | ``` | ||
| 48 | |||
| 49 | The release root should also alias the latest files: | ||
| 50 | |||
| 51 | ```text | ||
| 52 | https://releases.bitflash.network/SHA256SUMS | ||
| 53 | https://releases.bitflash.network/SHA256SUMS.asc | ||
| 54 | https://releases.bitflash.network/Bitflash-latest-windows.zip | ||
| 55 | https://releases.bitflash.network/Bitflash-latest-x86_64.AppImage | ||
| 56 | https://releases.bitflash.network/bitflash-node-latest-x86_64 | ||
| 57 | ``` | ||
| 58 | |||
| 59 | The aliases are convenience only. Verification must still require the pinned | ||
| 60 | release key documented in `docs/release-verification.md`. | ||
| 61 | |||
| 62 | ## Static Explorer | ||
| 63 | |||
| 64 | `scripts/build-explorer.py` writes a static explorer with no backend: | ||
| 65 | |||
| 66 | ```text | ||
| 67 | index.html | ||
| 68 | style.css | ||
| 69 | explorer.js | ||
| 70 | logo.png | ||
| 71 | blocks.json | ||
| 72 | block/<height>.json | ||
| 73 | ``` | ||
| 74 | |||
| 75 | Serve it with a strict content policy: | ||
| 76 | |||
| 77 | ```text | ||
| 78 | default-src 'none'; | ||
| 79 | img-src 'self'; | ||
| 80 | style-src 'self'; | ||
| 81 | script-src 'self'; | ||
| 82 | connect-src 'self'; | ||
| 83 | base-uri 'none'; | ||
| 84 | form-action 'none'; | ||
| 85 | frame-ancestors 'none'; | ||
| 86 | ``` | ||
| 87 | |||
| 88 | The viewer must never use `innerHTML` for chain data. A malicious transaction | ||
| 89 | must be rendered as text, not interpreted as markup. | ||
| 90 | |||
| 91 | ## Cache Policy | ||
| 92 | |||
| 93 | Recommended defaults: | ||
| 94 | |||
| 95 | | path | cache | | ||
| 96 | |---|---| | ||
| 97 | | HTML pages | `public, max-age=0, must-revalidate` | | ||
| 98 | | `status.json` | `no-store` | | ||
| 99 | | `blocks.json`, `block/*.json` | `public, max-age=0, must-revalidate` | | ||
| 100 | | release assets | `public, max-age=14400, must-revalidate` | | ||
| 101 | | `SHA256SUMS`, signatures, keys | `public, max-age=0, must-revalidate` | | ||
| 102 | |||
| 103 | ## Security Headers | ||
| 104 | |||
| 105 | Recommended for all static pages: | ||
| 106 | |||
| 107 | ```text | ||
| 108 | Strict-Transport-Security: max-age=31536000; includeSubDomains | ||
| 109 | X-Content-Type-Options: nosniff | ||
| 110 | Referrer-Policy: no-referrer | ||
| 111 | Permissions-Policy: interest-cohort=(), geolocation=(), camera=(), microphone=() | ||
| 112 | ``` | ||
| 113 | |||
| 114 | Forgejo has its own application headers and should not blindly inherit the | ||
| 115 | static-site CSP. | ||
diff --git a/docs/social.png b/docs/social.png index 19e2a3b..9a08b02 100644 --- a/docs/social.png +++ b/docs/social.png | |||
| Binary files differ | |||
diff --git a/docs/storage.md b/docs/storage.md new file mode 100644 index 0000000..c7de4e7 --- /dev/null +++ b/docs/storage.md | |||
| @@ -0,0 +1,159 @@ | |||
| 1 | # Bitflash Storage Inventory | ||
| 2 | |||
| 3 | This document records the Berkeley DB surface that still exists in Bitflash. | ||
| 4 | It is a migration map, not a promise that all stores should move at once. | ||
| 5 | |||
| 6 | The immediate priority is `wallet.dat`: it holds spend authority and recovery | ||
| 7 | metadata. `blkindex.dat` can be rebuilt from block files, so it is important for | ||
| 8 | startup correctness but less dangerous than wallet storage. | ||
| 9 | |||
| 10 | ## Berkeley DB Environment | ||
| 11 | |||
| 12 | All `CDB` stores use one Berkeley DB environment rooted at: | ||
| 13 | |||
| 14 | ```text | ||
| 15 | <datadir>/database/ | ||
| 16 | ``` | ||
| 17 | |||
| 18 | The environment creates log files such as: | ||
| 19 | |||
| 20 | ```text | ||
| 21 | database/log.0000000001 | ||
| 22 | ``` | ||
| 23 | |||
| 24 | Those logs are part of the legacy BDB risk model. They are not user-facing | ||
| 25 | wallet files, but they can contain historical records while the environment is | ||
| 26 | live. Wallet encryption already rewrites `wallet.dat` and purges BDB environment | ||
| 27 | logs after the encrypted file is installed. | ||
| 28 | |||
| 29 | ## Common Records | ||
| 30 | |||
| 31 | Every `CDB` file can carry: | ||
| 32 | |||
| 33 | | Key | Value | Purpose | | ||
| 34 | |---|---|---| | ||
| 35 | | `"version"` | `int` | Database format version written when a DB is opened in create mode. | | ||
| 36 | |||
| 37 | Keys and values are serialized with the existing `CDataStream` format. A future | ||
| 38 | SQLite migration must preserve byte-for-byte value semantics until a higher | ||
| 39 | level schema migration is intentionally introduced. | ||
| 40 | |||
| 41 | ## `wallet.dat` | ||
| 42 | |||
| 43 | `wallet.dat` is accessed through `CWalletDB`. | ||
| 44 | Read-only tooling should go through `ScanWalletRecords()` instead of opening a | ||
| 45 | Berkeley DB cursor directly. That keeps migration checks, audits, and future | ||
| 46 | backends behind one record-stream boundary. Wallet rewrite code should also use | ||
| 47 | that boundary when it needs to copy existing records into a newly written | ||
| 48 | wallet file. | ||
| 49 | |||
| 50 | | Key shape | Value | Secret? | Meaning | | ||
| 51 | |---|---|---:|---| | ||
| 52 | | `("name", address)` | `string` | No | Address book label. | | ||
| 53 | | `("tx", txid)` | `CWalletTx` | No | Wallet transaction state, including spend flags and metadata. | | ||
| 54 | | `("key", pubkey)` | `CPrivKey` | Yes | Plain private key record for unencrypted wallets or old backups. | | ||
| 55 | | `"defaultkey"` | `vector<unsigned char>` | No | Default public key shown by the wallet. | | ||
| 56 | | `"hdmaster"` | `vector<unsigned char>` | Yes | Plain BIP32 master private key material. | | ||
| 57 | | `"hdchaincode"` | `vector<unsigned char>` | Yes | Plain BIP32 chain code. | | ||
| 58 | | `"hdnext"` | `unsigned int` | No | Legacy deterministic derivation next index. | | ||
| 59 | | `"hdschema"` | `int` | No | Deterministic derivation schema (`legacy` or `bip44`). | | ||
| 60 | | `"hdcointype"` | `unsigned int` | No | BIP44 coin type. Bitflash is `4346950`. | | ||
| 61 | | `"hdreceivenext"` | `unsigned int` | No | BIP44 external receive-chain next index. | | ||
| 62 | | `"hdchangenext"` | `unsigned int` | No | BIP44 internal change-chain next index. | | ||
| 63 | | `("mkey", id)` | `CWalletMasterKey` | Sensitive | Encrypted wallet master-key metadata. | | ||
| 64 | | `("ckey", pubkey)` | `vector<unsigned char>` | Yes | Private key encrypted under the wallet master key. | | ||
| 65 | | `"cryptedhdmaster"` | `vector<unsigned char>` | Yes | Encrypted BIP32 master private key material. | | ||
| 66 | | `"cryptedhdchaincode"` | `vector<unsigned char>` | Yes | Encrypted BIP32 chain code. | | ||
| 67 | | `"walletminversion"` | `int` | No | Minimum wallet format version required to open this wallet safely. | | ||
| 68 | | `("pool", index)` | `vector<unsigned char>` | No | Public key reserved in the key pool. The private key lives in `key`/`ckey`. | | ||
| 69 | | `("setting", name)` | typed setting value | Usually no | Wallet/UI settings such as mining mode, fees, and backup timestamps. | | ||
| 70 | |||
| 71 | ### Wallet Migration Rules | ||
| 72 | |||
| 73 | A future wallet backend must keep these properties: | ||
| 74 | |||
| 75 | 1. Unknown future wallet formats fail closed with a clear message. | ||
| 76 | 2. Encrypted wallets must encrypt private keys and HD seed material. | ||
| 77 | 3. A locked encrypted wallet may expose public keys and balances, but not | ||
| 78 | private keys or decrypted HD seed material. | ||
| 79 | 4. `-backupwallet` must produce a wallet file that opens in a different datadir | ||
| 80 | without copying `database/`, and must not leave a half-written final backup | ||
| 81 | path if the copy or install step fails. | ||
| 82 | 5. `-recoveryaudit` must distinguish: | ||
| 83 | - no recovery phrase installed; | ||
| 84 | - recovery phrase present; | ||
| 85 | - recovery phrase encrypted and locked, requiring unlock before coverage can | ||
| 86 | be audited. | ||
| 87 | |||
| 88 | ## SQLite Wallet Store | ||
| 89 | |||
| 90 | Since v1.2.20 SQLite is the default wallet backend. It is a byte-preserving | ||
| 91 | record store, not a new wallet format -- the same serialized `CDataStream` bytes | ||
| 92 | `wallet.dat` uses: | ||
| 93 | |||
| 94 | ```sql | ||
| 95 | CREATE TABLE wallet_records ( | ||
| 96 | key BLOB PRIMARY KEY NOT NULL, | ||
| 97 | value BLOB NOT NULL | ||
| 98 | ); | ||
| 99 | PRAGMA user_version=1; | ||
| 100 | ``` | ||
| 101 | |||
| 102 | A fresh data directory starts on `wallet.sqlite`. An existing `wallet.dat` keeps | ||
| 103 | running on Berkeley DB until it is converted: the GUI offers a one-click | ||
| 104 | conversion on first run, and `-walletbackend=sqlite` / `-walletbackend=bdb` | ||
| 105 | (recorded in a plain-text `wallet-backend` marker file in the datadir) select a | ||
| 106 | backend explicitly. Conversion never modifies `wallet.dat`; it stays as a | ||
| 107 | fallback, and the change takes effect on restart. | ||
| 108 | |||
| 109 | Every wallet operation runs on either backend: read, write, erase, key-pool and | ||
| 110 | HD-seed writes (the two halves of the seed bracketed in one SQLite transaction), | ||
| 111 | `-backupwallet` (checkpoint then copy the self-contained file), `-encryptwallet` | ||
| 112 | (an atomic rebuild-and-swap in one transaction), and the `-walletstorageaudit` / | ||
| 113 | `-walletstoragecheck` diagnostics, which read the active backend's file. | ||
| 114 | |||
| 115 | The migration tooling remains available for scripted use. `-walletsqliteexport=FILE` | ||
| 116 | streams every raw `wallet.dat` record through `ScanWalletRecords()` into a SQLite | ||
| 117 | file in one transaction, refusing to overwrite. `-walletsqliteverify=FILE` | ||
| 118 | compares the two streams and prints only counts. `-walletsqliteloadcheck=FILE` | ||
| 119 | parses an export with the same record types the loader uses. | ||
| 120 | `-walletsqliterestore=FILE` rebuilds a `wallet.dat` from an export in an empty | ||
| 121 | data directory. Failed exports and restores remove their incomplete output. | ||
| 122 | |||
| 123 | ## `blkindex.dat` | ||
| 124 | |||
| 125 | `blkindex.dat` is accessed through `CTxDB`. | ||
| 126 | |||
| 127 | | Key shape | Value | Rebuildable? | Meaning | | ||
| 128 | |---|---|---:|---| | ||
| 129 | | `("tx", txid)` | `CTxIndex` | Yes | Transaction disk position and spent-output index state. | | ||
| 130 | | `("owner", hash160, CDiskTxPos)` | `int height` | Yes | Legacy owner transaction index cursor range. | | ||
| 131 | | `("blockindex", blockhash)` | `CDiskBlockIndex` | Yes | Block index entry. | | ||
| 132 | | `"hashBestChain"` | `uint256` | Yes | Current best-chain tip hash. | | ||
| 133 | |||
| 134 | `blkindex.dat` can be rebuilt from `blk*.dat` plus consensus validation. A | ||
| 135 | future migration may therefore prefer a rebuild path over a byte-preserving | ||
| 136 | conversion path. | ||
| 137 | |||
| 138 | ## Other Legacy Stores | ||
| 139 | |||
| 140 | None. Upstream still documents `reviews.dat` (`CReviewDB`) and `market.dat` | ||
| 141 | (`CMarketDB`) here; this fork deleted the 2009 marketplace outright, along | ||
| 142 | with `CReviewDB`, `market.cpp` and `market.h`, so neither file is opened or | ||
| 143 | created by this tree. A datadir carrying them from an older build is simply | ||
| 144 | ignored. | ||
| 145 | |||
| 146 | The old IP address database (`addr.dat`) has been removed. Peer discovery is | ||
| 147 | handled by Nostr, `.btf`, rendezvous relays, and direct onion peers. | ||
| 148 | |||
| 149 | ## Migration Order (shipped in v1.2.20) | ||
| 150 | |||
| 151 | 1. Storage self-tests were added around Berkeley DB behavior first. | ||
| 152 | 2. The SQLite backend landed behind an explicit flag, then behind a | ||
| 153 | `wallet-backend` marker and a GUI conversion wizard. | ||
| 154 | 3. BDB-to-SQLite migration is a one-way copy that leaves `wallet.dat` as a backup. | ||
| 155 | 4. SQLite became the default for new wallets only after cross-version restore, | ||
| 156 | encryption, backup, and recovery-audit tests passed -- and a live coin-proof | ||
| 157 | of receive, spend, restart, and encrypt on real mainnet funds. | ||
| 158 | 5. Treat `blkindex.dat` separately; it is safer to rebuild than to migrate under | ||
| 159 | pressure. | ||
diff --git a/docs/tor.md b/docs/tor.md new file mode 100644 index 0000000..0016f8b --- /dev/null +++ b/docs/tor.md | |||
| @@ -0,0 +1,231 @@ | |||
| 1 | # Tor mode | ||
| 2 | |||
| 3 | Bitflash can route outbound discovery traffic through a local Tor SOCKS5 proxy. | ||
| 4 | This is transport plumbing only: it does not change consensus, wallet keys, | ||
| 5 | mining, `.btf` addresses, or the encrypted rendezvous protocol. | ||
| 6 | |||
| 7 | ## Client mode | ||
| 8 | |||
| 9 | Start Tor locally, then start Bitflash with: | ||
| 10 | |||
| 11 | ```bash | ||
| 12 | bitflash -tor | ||
| 13 | ``` | ||
| 14 | |||
| 15 | `-tor` is shorthand for: | ||
| 16 | |||
| 17 | ```bash | ||
| 18 | bitflash -socks=127.0.0.1:9050 | ||
| 19 | ``` | ||
| 20 | |||
| 21 | If Tor listens somewhere else: | ||
| 22 | |||
| 23 | ```bash | ||
| 24 | bitflash -tor=127.0.0.1:9150 | ||
| 25 | bitflash -tor=[::1]:9050 | ||
| 26 | ``` | ||
| 27 | |||
| 28 | For a SOCKS5 proxy that is not Tor, use `-socks` directly: | ||
| 29 | |||
| 30 | ```bash | ||
| 31 | bitflash -socks=127.0.0.1:9050 | ||
| 32 | bitflash -socks=[::1]:9050 | ||
| 33 | ``` | ||
| 34 | |||
| 35 | If both `-tor` and `-socks` are present, Tor mode wins and the node prints a | ||
| 36 | warning. That keeps a command line with `-tor` from silently becoming a custom | ||
| 37 | proxy route. | ||
| 38 | |||
| 39 | ## Managed Tor mode | ||
| 40 | |||
| 41 | `-managedtor` starts a Tor process for this node, writes a local `torrc`, creates | ||
| 42 | a v3 hidden service for the normal Bitflash P2P listener, routes outbound | ||
| 43 | discovery through that Tor instance, and signs the generated `.onion:8433` | ||
| 44 | endpoint into the node's `.btf` descriptor. | ||
| 45 | |||
| 46 | ```bash | ||
| 47 | bitflash -managedtor | ||
| 48 | ``` | ||
| 49 | |||
| 50 | Without a value, Bitflash looks for Tor in this order: | ||
| 51 | |||
| 52 | 1. `tor/tor.exe` or `tor/tor` beside the Bitflash binary; | ||
| 53 | 2. `tor.exe` or `tor` beside the Bitflash binary; | ||
| 54 | 3. `tor.exe` or `tor` on `PATH`. | ||
| 55 | |||
| 56 | Windows releases can also be built as `Bitflash-*-windows-with-tor.zip`. That | ||
| 57 | package includes the official Tor Expert Bundle under `tor/`, so the first path | ||
| 58 | above exists immediately after extraction and `bitflash -managedtor` needs no | ||
| 59 | separate Tor install. | ||
| 60 | |||
| 61 | The bundled-Tor package is created by: | ||
| 62 | |||
| 63 | ```bash | ||
| 64 | make windows-tor | ||
| 65 | ``` | ||
| 66 | |||
| 67 | The packaging script verifies a pinned SHA256 for the downloaded Tor Expert | ||
| 68 | Bundle before copying it into the release directory. When `gpg` is available it | ||
| 69 | also verifies the matching `.asc` signature against the pinned Tor Browser | ||
| 70 | Developers signing-key fingerprint. Release builds should use | ||
| 71 | `TOR_VERIFY_GPG=required make windows-tor` to make that second check mandatory. | ||
| 72 | The Tor binary is bundled; the runtime data directory, hidden-service key, and | ||
| 73 | control cookie are still created under the user's Bitflash data directory. | ||
| 74 | |||
| 75 | To point at a specific Tor executable: | ||
| 76 | |||
| 77 | ```bash | ||
| 78 | bitflash -managedtor=/opt/tor/bin/tor | ||
| 79 | bitflash -managedtor=C:\Tor Browser\Browser\TorBrowser\Tor\tor.exe | ||
| 80 | ``` | ||
| 81 | |||
| 82 | Managed Tor writes its state under the Bitflash data directory: | ||
| 83 | |||
| 84 | ```text | ||
| 85 | managed-tor/ | ||
| 86 | torrc | ||
| 87 | data/ | ||
| 88 | onion-service/ | ||
| 89 | ``` | ||
| 90 | |||
| 91 | The private key for the onion address lives in `managed-tor/onion-service/`. | ||
| 92 | Back up that directory if you want the same onion address after moving the node. | ||
| 93 | Delete it only if you intentionally want a fresh onion endpoint. | ||
| 94 | |||
| 95 | `-managedtor` takes precedence over `-tor` and `-socks`, because it must choose | ||
| 96 | the local SOCKS port it starts. Diagnostics report the current managed Tor | ||
| 97 | state, including the SOCKS port and onion endpoint once Tor has generated it. | ||
| 98 | |||
| 99 | When Tor mode is enabled: | ||
| 100 | |||
| 101 | - outbound Nostr relay dials use Tor; | ||
| 102 | - outbound `.btf` rendezvous relay dials use Tor; | ||
| 103 | - outbound direct peer dials prefer signed `.onion` endpoints when a peer | ||
| 104 | advertises one; | ||
| 105 | - destination names are sent to Tor as SOCKS5 domain-name CONNECT requests, so | ||
| 106 | the local DNS resolver does not see them; | ||
| 107 | - direct `.onion` dials are refused unless a SOCKS5/Tor proxy is enabled; | ||
| 108 | - plain HTTP external-IP probes are skipped. | ||
| 109 | |||
| 110 | Tor mode affects outbound discovery and relay dials. It does not encrypt | ||
| 111 | `wallet.dat`, hide mining rewards on-chain, or change the `.btf` identity. A | ||
| 112 | relay still sees the TCP client that connected to it; with Tor mode that client | ||
| 113 | is the Tor exit or onion circuit endpoint instead of the user's direct IP. | ||
| 114 | |||
| 115 | ## What to test | ||
| 116 | |||
| 117 | Use the public status page only as a rough health check. To prove the local node | ||
| 118 | is using Tor/SOCKS, test from the node machine: | ||
| 119 | |||
| 120 | ```bash | ||
| 121 | bitflash -tor -debug | ||
| 122 | ``` | ||
| 123 | |||
| 124 | Then confirm in `debug.log` that Nostr and rendezvous dials report the proxy | ||
| 125 | path. A DNS leak test should show no local resolver lookup for relay hostnames: | ||
| 126 | the SOCKS5 request uses the domain-name form and asks the proxy to resolve it. | ||
| 127 | |||
| 128 | ## Onion rendezvous relay | ||
| 129 | |||
| 130 | An operator can expose a Bitflash rendezvous relay as a Tor hidden service. The | ||
| 131 | node does not create hidden services itself; Tor owns that configuration. | ||
| 132 | |||
| 133 | Example `torrc`: | ||
| 134 | |||
| 135 | ```text | ||
| 136 | HiddenServiceDir /var/lib/tor/bitflash-rendezvous/ | ||
| 137 | HiddenServicePort 8434 127.0.0.1:8434 | ||
| 138 | ``` | ||
| 139 | |||
| 140 | After restarting Tor, read the generated onion name: | ||
| 141 | |||
| 142 | ```bash | ||
| 143 | sudo cat /var/lib/tor/bitflash-rendezvous/hostname | ||
| 144 | ``` | ||
| 145 | |||
| 146 | Then run the relay on localhost/VPS as usual and announce the onion endpoint from | ||
| 147 | nodes that use Tor: | ||
| 148 | |||
| 149 | ```bash | ||
| 150 | bitflash -tor -rvrelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 | ||
| 151 | bitflash -tor -announcerelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 | ||
| 152 | ``` | ||
| 153 | |||
| 154 | For a pool operator that wants the pool descriptor to point at an onion | ||
| 155 | rendezvous relay, use both: | ||
| 156 | |||
| 157 | ```bash | ||
| 158 | bitflash -operator -tor \ | ||
| 159 | -rvrelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 \ | ||
| 160 | -announcerelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 | ||
| 161 | ``` | ||
| 162 | |||
| 163 | ## Manual direct full-node hidden service | ||
| 164 | |||
| 165 | Managed Tor is the preferred path for ordinary nodes. Operators can still expose | ||
| 166 | the P2P listener through a separately managed Tor service: | ||
| 167 | |||
| 168 | ```text | ||
| 169 | HiddenServiceDir /var/lib/tor/bitflash-node/ | ||
| 170 | HiddenServicePort 8433 127.0.0.1:8433 | ||
| 171 | ``` | ||
| 172 | |||
| 173 | After restarting Tor, read the generated onion name: | ||
| 174 | |||
| 175 | ```bash | ||
| 176 | sudo cat /var/lib/tor/bitflash-node/hostname | ||
| 177 | ``` | ||
| 178 | |||
| 179 | Then start Bitflash with Tor outbound enabled and advertise that hidden service: | ||
| 180 | |||
| 181 | ```bash | ||
| 182 | bitflash -tor \ | ||
| 183 | -onionservice=exampleexampleexampleexampleexampleexampleexampleexample.onion:8433 | ||
| 184 | ``` | ||
| 185 | |||
| 186 | That onion endpoint is signed into this node's self-certifying `.btf` | ||
| 187 | descriptor. New peers that resolve the descriptor and also run with `-tor` try | ||
| 188 | the `.onion:8433` P2P connection first. If it fails, they fall back to the | ||
| 189 | ordinary encrypted rendezvous path. Older nodes ignore the onion field and still | ||
| 190 | use the rendezvous relay because the descriptor keeps the legacy signature too. | ||
| 191 | |||
| 192 | If the node has `-onionservice` but has not registered at any rendezvous relay | ||
| 193 | yet, it can still publish an onion-capable descriptor with | ||
| 194 | `meeting_node = rendezvous-pending`. Tor-capable peers can dial the hidden | ||
| 195 | service directly from that descriptor. Rendezvous becomes a fallback, not a | ||
| 196 | precondition for being discoverable. | ||
| 197 | |||
| 198 | This means rendezvous relays stop being the only way to reach a node that has | ||
| 199 | published a hidden service. They remain useful bootstrap/fallback infrastructure, | ||
| 200 | but an already-discovered onion-capable peer can be reached directly over Tor. | ||
| 201 | |||
| 202 | Tor owns the private key under `HiddenServiceDir`; back it up if you want the | ||
| 203 | onion name to stay stable. | ||
| 204 | |||
| 205 | ## Release smoke test | ||
| 206 | |||
| 207 | After building a Windows binary with managed Tor support, run the two-node smoke | ||
| 208 | test before cutting a release: | ||
| 209 | |||
| 210 | ```powershell | ||
| 211 | powershell -ExecutionPolicy Bypass -File scripts\smoke-managed-tor-onion.ps1 ` | ||
| 212 | -Bitflash .\src\bitflash.exe ` | ||
| 213 | -Tor C:\path\to\tor.exe | ||
| 214 | ``` | ||
| 215 | |||
| 216 | The script creates two temporary datadirs, starts node A with `-managedtor`, | ||
| 217 | waits for it to publish its `.btf` identity and generated `.onion` endpoint, | ||
| 218 | then starts node B with its own `-managedtor`, `-oniononly`, and | ||
| 219 | `-connectbtf=<node-a>`. It passes only when node B logs that it connected to | ||
| 220 | node A `via direct onion`; rendezvous fallback cannot satisfy the test. | ||
| 221 | |||
| 222 | By default the script waits 120 seconds after node A's descriptor self-resolves | ||
| 223 | before it starts node B. Tor can create the local `hostname` before the hidden | ||
| 224 | service descriptor is visible enough for client circuits, and early dials often | ||
| 225 | fail with SOCKS5 reply `0x01`. Override this with `-OnionWarmupSec N` when | ||
| 226 | testing a slow or already-warmed Tor environment. | ||
| 227 | |||
| 228 | This test still uses Nostr for descriptor discovery, because that is how nodes | ||
| 229 | learn each other's signed `.btf` descriptors. It does not require a Bitflash | ||
| 230 | rendezvous relay to carry the P2P connection: once the descriptor includes the | ||
| 231 | signed onion endpoint, the actual node connection goes directly through Tor. | ||
diff --git a/docs/utxo-commitment.md b/docs/utxo-commitment.md new file mode 100644 index 0000000..ab43deb --- /dev/null +++ b/docs/utxo-commitment.md | |||
| @@ -0,0 +1,125 @@ | |||
| 1 | # Bitflash UTXO-set commitment | ||
| 2 | |||
| 3 | `scripts/verify-utxo-set.py` reconstructs a deterministic commitment to the | ||
| 4 | current UTXO set from local `blk*.dat` files. | ||
| 5 | |||
| 6 | This is intentionally not a consensus rule yet. It is an audit tool: independent | ||
| 7 | operators can run it against the same chain prefix and compare the resulting | ||
| 8 | root and canonical JSON hash. | ||
| 9 | |||
| 10 | ```bash | ||
| 11 | python3 scripts/verify-utxo-set.py --max-blocks 1000 --out utxo-report.json --json | ||
| 12 | ``` | ||
| 13 | |||
| 14 | Use `--max-blocks 0` or omit the option to scan every available block file: | ||
| 15 | |||
| 16 | ```bash | ||
| 17 | python3 scripts/verify-utxo-set.py --out utxo-report.json --json | ||
| 18 | ``` | ||
| 19 | |||
| 20 | ## What It Verifies | ||
| 21 | |||
| 22 | The verifier: | ||
| 23 | |||
| 24 | - parses Bitflash block files directly with the shared Python chain parser; | ||
| 25 | - streams `blk*.dat` records instead of loading full block files into memory; | ||
| 26 | - reconstructs the best chain rooted at the Bitflash genesis block; | ||
| 27 | - ignores side blocks that may also be present in `blk*.dat`; | ||
| 28 | - verifies every scanned block merkle root; | ||
| 29 | - applies spends and creates outputs in deterministic order; | ||
| 30 | - reports current UTXO count, current supply, mined coinbase total, fees, and | ||
| 31 | historical duplicate-output overwrites; | ||
| 32 | - computes a deterministic UTXO root. | ||
| 33 | |||
| 34 | ## Commitment Algorithm | ||
| 35 | |||
| 36 | The UTXO map is keyed by: | ||
| 37 | |||
| 38 | ```text | ||
| 39 | txid:vout | ||
| 40 | ``` | ||
| 41 | |||
| 42 | Keys are sorted lexicographically. Each leaf is: | ||
| 43 | |||
| 44 | ```text | ||
| 45 | SHA256("BTFUTXO1|" || outpoint || "|" || value_satoshis || "|" || | ||
| 46 | script_pub_key_hex || "|" || created_height || "|" || created_txid) | ||
| 47 | ``` | ||
| 48 | |||
| 49 | The tree is then reduced pairwise. If a level has an odd number of hashes, the | ||
| 50 | last hash is duplicated. Each parent is: | ||
| 51 | |||
| 52 | ```text | ||
| 53 | SHA256("BTFNODE1|" || left_hash || right_hash) | ||
| 54 | ``` | ||
| 55 | |||
| 56 | The final hash is the UTXO-set root. | ||
| 57 | |||
| 58 | ## Inclusion Proofs | ||
| 59 | |||
| 60 | After a root has been computed, `scripts/prove-utxo.py` can create a compact | ||
| 61 | inclusion proof for one unspent output: | ||
| 62 | |||
| 63 | ```bash | ||
| 64 | python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json | ||
| 65 | python3 scripts/verify-utxo-proof.py utxo-proof.json | ||
| 66 | ``` | ||
| 67 | |||
| 68 | The verifier does not need local block files. It recomputes the target leaf, | ||
| 69 | walks the sibling path, and compares the result with the expected root. See | ||
| 70 | [UTXO inclusion proofs](utxo-proofs.md). | ||
| 71 | |||
| 72 | ## Deterministic Output | ||
| 73 | |||
| 74 | `--json` and `--out` write canonical JSON with sorted keys, fixed separators, | ||
| 75 | ASCII escaping, and one trailing newline. It contains no local path and no | ||
| 76 | generation timestamp, so two auditors scanning the same chain prefix should | ||
| 77 | produce byte-identical reports. | ||
| 78 | |||
| 79 | ```bash | ||
| 80 | python3 scripts/verify-utxo-set.py --out utxo-report.json --json | ||
| 81 | sha256sum utxo-report.json | ||
| 82 | ``` | ||
| 83 | |||
| 84 | ## OpenTimestamps | ||
| 85 | |||
| 86 | The canonical JSON can be timestamped in Bitcoin without adding any dependency | ||
| 87 | to Bitflash consensus: | ||
| 88 | |||
| 89 | ```bash | ||
| 90 | ots stamp utxo-report.json | ||
| 91 | ots verify utxo-report.json.ots | ||
| 92 | ``` | ||
| 93 | |||
| 94 | ## Historical Duplicates | ||
| 95 | |||
| 96 | Bitflash inherited early Bitcoin behavior around duplicate transaction IDs. | ||
| 97 | The verifier reports duplicate-output overwrites as warnings and uses | ||
| 98 | last-write-wins semantics for the UTXO map, matching the legacy index model. | ||
| 99 | Those warnings are part of the report so auditors can see the historical debt | ||
| 100 | instead of relying on a polished summary. | ||
| 101 | |||
| 102 | For synthetic chains or future clean consensus ranges, auditors can make | ||
| 103 | duplicates a hard failure: | ||
| 104 | |||
| 105 | ```bash | ||
| 106 | python3 scripts/verify-utxo-set.py --strict-duplicates | ||
| 107 | ``` | ||
| 108 | |||
| 109 | The default remains compatible with the current historical chain because it has | ||
| 110 | known legacy duplicate-output overwrites. | ||
| 111 | |||
| 112 | ## Shared Parser | ||
| 113 | |||
| 114 | The audit scripts share `scripts/bitflash_chain.py` for block parsing, | ||
| 115 | main-chain selection, UTXO application, canonical JSON, and genesis | ||
| 116 | reconstruction. That keeps `verify-utxo-set.py`, `verify-fair-launch.py`, | ||
| 117 | `prove-utxo.py`, and the explorer on the same serialization rules. | ||
| 118 | |||
| 119 | ## Future Consensus Path | ||
| 120 | |||
| 121 | This tool is the low-risk first step toward consensus commitments. A future | ||
| 122 | hard-fork design can commit this root, or a Utreexo-style accumulator root, in | ||
| 123 | coinbase or block headers after a fixed activation height. Until then this | ||
| 124 | script provides reproducible supply and state commitments without changing what | ||
| 125 | nodes accept. | ||
diff --git a/docs/utxo-proofs.md b/docs/utxo-proofs.md new file mode 100644 index 0000000..4947c3c --- /dev/null +++ b/docs/utxo-proofs.md | |||
| @@ -0,0 +1,61 @@ | |||
| 1 | # Bitflash UTXO inclusion proofs | ||
| 2 | |||
| 3 | `scripts/prove-utxo.py` creates a deterministic inclusion proof for one | ||
| 4 | currently unspent output. `scripts/verify-utxo-proof.py` verifies that proof | ||
| 5 | against the UTXO root produced by `scripts/verify-utxo-set.py`. | ||
| 6 | |||
| 7 | This is intentionally outside consensus. It turns the audit root into something | ||
| 8 | useful for a specific coin without changing what nodes accept. | ||
| 9 | |||
| 10 | ## Create A Proof | ||
| 11 | |||
| 12 | Use an unspent output in `txid:vout` form: | ||
| 13 | |||
| 14 | ```bash | ||
| 15 | python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json | ||
| 16 | ``` | ||
| 17 | |||
| 18 | The proof contains: | ||
| 19 | |||
| 20 | - the target UTXO; | ||
| 21 | - its sorted leaf index; | ||
| 22 | - the leaf hash; | ||
| 23 | - the Merkle sibling path; | ||
| 24 | - the UTXO root and chain tip used to create it. | ||
| 25 | |||
| 26 | It does not contain the full UTXO set. | ||
| 27 | |||
| 28 | ## Verify A Proof | ||
| 29 | |||
| 30 | Verify against the root embedded in the proof: | ||
| 31 | |||
| 32 | ```bash | ||
| 33 | python3 scripts/verify-utxo-proof.py utxo-proof.json | ||
| 34 | ``` | ||
| 35 | |||
| 36 | Or verify against a root copied from another machine, dashboard, block explorer, | ||
| 37 | or future node export: | ||
| 38 | |||
| 39 | ```bash | ||
| 40 | python3 scripts/verify-utxo-proof.py utxo-proof.json \ | ||
| 41 | --root ROOT_FROM_REPORT_OR_EXPORT | ||
| 42 | ``` | ||
| 43 | |||
| 44 | The verifier does not need local `blk*.dat` files. It only needs the proof and | ||
| 45 | the expected root. | ||
| 46 | |||
| 47 | ## Scope | ||
| 48 | |||
| 49 | This first proof format supports inclusion only. Absence and range proofs are | ||
| 50 | left for a later design because they require a more careful commitment scheme | ||
| 51 | and are easier to get subtly wrong. | ||
| 52 | |||
| 53 | The proof uses the exact same leaf and node domains as the UTXO commitment: | ||
| 54 | |||
| 55 | ```text | ||
| 56 | BTFUTXO1 | ||
| 57 | BTFNODE1 | ||
| 58 | ``` | ||
| 59 | |||
| 60 | That keeps the verifier language-neutral and gives future implementations a | ||
| 61 | small, stable target. | ||