diff options
| author | bitflash-maintainer <noreply@bitflash.invalid> | 2026-09-05 22:46:52 +0000 |
|---|---|---|
| committer | bitflash-maintainer <noreply@bitflash.invalid> | 2026-09-05 22:46:52 +0000 |
| commit | ceae3d076b5ad11ee84729ae43f14e9312896646 (patch) | |
| tree | 7e9132aa871fdc01cb95268d2f5ea3bb11853ddf /docs | |
| parent | be63e83d79e7e00e6b8ca889a70e6a19886aa1d8 (diff) | |
| parent | 66e0b9158d050c46b1ff2509faeee3f4573cac1c (diff) | |
Bring main onto the line this project actually ships
main had drifted to a history with no ancestor in common with the tree we
build and deploy. It sat at be63e83 (2026-08-09) with 28 commits; the shipped
line, deploy/soak-2026-08-21, had 383. The two shared nothing -- git
merge-base returns no result at all -- so anyone cloning this repository
landed on a default branch that was neither current nor related to the
running node.
The 28 commits on main were audited before this merge: every one is already
present in the tree by content, including the two that are hardest to see
because they were rewritten afterwards -- the payout reorg-safety work
(PayoutBlockStillOnChain, since evolved to re-check on every retry rather
than once at maturity) and the nostr relay certificate verification
(SSL_set1_host, now with a Windows root-store path). Nothing on main is lost
by this merge.
Recorded as a merge rather than a force-push: main only ever moves forward,
its old history stays reachable as the first parent, and this is revertible.
The tree is taken wholesale from the merged branch, so main's content is now
exactly what was built and tested.
What this brings onto main:
- compactblock: BIP152-style compact block relay ported from upstream 1.2.23,
primitives and unit test only, with the RAND_bytes failure path closed and
the reconstruction allocation bounded.
- upstream/main 17741d9: isolated testnet parameters and genesis, optional
2 MB large pages for RandomX. Mainnet genesis verified byte-identical
through upstream's BuildGenesisBlock() refactor.
- deploy: every image built on bookworm so binaries run on Debian 12, and the
container build now honours the RandomX and libsecp256k1 pins it had been
silently ignoring since before 1.2.15.
Gates at this tree: 15/15 selftests, compactblock's 17 checks, consensus
constants unchanged against e4f898c apart from the pre-existing
MAX_BLOCK_SIZE_GEN, mainnet genesis intact, money-log check ok.
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/dependencies.md | 18 | ||||
| -rw-r--r-- | docs/fair-launch.md | 70 | ||||
| -rw-r--r-- | docs/mining-calculator.md | 108 | ||||
| -rw-r--r-- | docs/pool-directory.md | 61 | ||||
| -rw-r--r-- | docs/release-verification.md | 94 | ||||
| -rw-r--r-- | docs/rendezvous.md | 336 | ||||
| -rw-r--r-- | docs/site.md | 115 | ||||
| -rw-r--r-- | docs/social.png | bin | 208957 -> 162267 bytes | |||
| -rw-r--r-- | docs/storage.md | 159 | ||||
| -rw-r--r-- | docs/tor.md | 231 | ||||
| -rw-r--r-- | docs/utxo-commitment.md | 125 | ||||
| -rw-r--r-- | docs/utxo-proofs.md | 61 |
12 files changed, 1351 insertions, 27 deletions
diff --git a/docs/dependencies.md b/docs/dependencies.md new file mode 100644 index 0000000..9fd45b6 --- /dev/null +++ b/docs/dependencies.md | |||
| @@ -0,0 +1,18 @@ | |||
| 1 | # Build Dependencies | ||
| 2 | |||
| 3 | Bitflash links consensus-adjacent cryptographic and proof-of-work code from | ||
| 4 | external repositories. Fresh builds must not follow upstream `master` silently: | ||
| 5 | a changed upstream branch can break reproducible builds or change behavior in | ||
| 6 | code that every node relies on. | ||
| 7 | |||
| 8 | The top-level `Makefile` pins source dependencies by full commit hash: | ||
| 9 | |||
| 10 | | dependency | repository | pinned commit | | ||
| 11 | |---|---|---| | ||
| 12 | | libsecp256k1 | https://github.com/bitcoin-core/secp256k1 | `7fecac74aed8e1fd9078380d67dd04663705c989` | | ||
| 13 | | RandomX | https://github.com/tevador/RandomX | `1e9d4b2df63fa6edf46b06789486e23c0fcaa55a` | | ||
| 14 | |||
| 15 | When updating either dependency, rebuild from a clean dependency directory and | ||
| 16 | run the full selftest suite before opening the PR. The PR should explain why the | ||
| 17 | dependency changed and whether the update can affect consensus, addresses, | ||
| 18 | signatures, proof-of-work validation, or mining. | ||
diff --git a/docs/fair-launch.md b/docs/fair-launch.md new file mode 100644 index 0000000..9603910 --- /dev/null +++ b/docs/fair-launch.md | |||
| @@ -0,0 +1,70 @@ | |||
| 1 | # Bitflash fair-launch verifier | ||
| 2 | |||
| 3 | Bitflash ships a deterministic fair-launch audit script: | ||
| 4 | |||
| 5 | ```bash | ||
| 6 | python3 scripts/verify-fair-launch.py --max-blocks 1000 --out bitflash-fair-launch-report.json --json | ||
| 7 | ``` | ||
| 8 | |||
| 9 | The verifier reads `blk*.dat` directly from the local Bitflash data directory. | ||
| 10 | It does not use Berkeley DB, does not call the node binary, and does not need | ||
| 11 | network access. It uses the same shared Python chain parser as the UTXO | ||
| 12 | commitment and proof tools, so block serialization fixes land in one place. | ||
| 13 | |||
| 14 | The report proves the launch baseline that can be checked from the chain: | ||
| 15 | |||
| 16 | - the genesis block hash matches the source constant; | ||
| 17 | - the genesis merkle root matches the source constant; | ||
| 18 | - the genesis transaction is a single coinbase transaction; | ||
| 19 | - the genesis output total is exactly `50.00000000 BTF`; | ||
| 20 | - the scanned block range and mined coinbase total are reproducible. | ||
| 21 | |||
| 22 | For machines that do not have a synced data directory yet, the source constants | ||
| 23 | can be checked without disk blocks: | ||
| 24 | |||
| 25 | ```bash | ||
| 26 | python3 scripts/verify-fair-launch.py --reconstruct-genesis | ||
| 27 | ``` | ||
| 28 | |||
| 29 | ## Deterministic output | ||
| 30 | |||
| 31 | Use `--json` or `--out` for canonical JSON. The JSON is written with sorted | ||
| 32 | keys, fixed separators, ASCII escaping, and one trailing newline. It deliberately | ||
| 33 | does not include local paths or wall-clock generation time, so two auditors | ||
| 34 | scanning the same block range produce the same bytes. | ||
| 35 | |||
| 36 | That makes the report shareable: | ||
| 37 | |||
| 38 | ```bash | ||
| 39 | python3 scripts/verify-fair-launch.py --max-blocks 1000 --out report.json --json | ||
| 40 | sha256sum report.json | ||
| 41 | ``` | ||
| 42 | |||
| 43 | If another auditor runs the same command against the same chain prefix, the | ||
| 44 | `sha256sum` should match. | ||
| 45 | |||
| 46 | ## OpenTimestamps | ||
| 47 | |||
| 48 | OpenTimestamps is optional and external. It anchors the report in Bitcoin | ||
| 49 | without adding any bridge or dependency to Bitflash consensus. | ||
| 50 | |||
| 51 | After generating a canonical JSON report: | ||
| 52 | |||
| 53 | ```bash | ||
| 54 | ots stamp report.json | ||
| 55 | ots verify report.json.ots | ||
| 56 | ``` | ||
| 57 | |||
| 58 | The `.ots` file proves that the exact report bytes existed no later than the | ||
| 59 | Bitcoin block that confirms the timestamp. | ||
| 60 | |||
| 61 | ## What this does not claim | ||
| 62 | |||
| 63 | This is not a retroactive genesis commitment. The Bitflash genesis block already | ||
| 64 | exists, so adding a new on-chain commitment to genesis would create a different | ||
| 65 | chain. This verifier instead audits the launch state that is already on-chain | ||
| 66 | and makes the result reproducible and timestampable. | ||
| 67 | |||
| 68 | It also does not prove real-world miner identity. It proves the genesis state, | ||
| 69 | the absence of non-coinbase premine in genesis, and the deterministic coinbase | ||
| 70 | totals over the scanned prefix. | ||
diff --git a/docs/mining-calculator.md b/docs/mining-calculator.md new file mode 100644 index 0000000..e6cb61b --- /dev/null +++ b/docs/mining-calculator.md | |||
| @@ -0,0 +1,108 @@ | |||
| 1 | # Bitflash mining calculator | ||
| 2 | |||
| 3 | `scripts/bitflash-mining-calculator.py` is a local WhatToMine-style calculator | ||
| 4 | for Bitflash. | ||
| 5 | |||
| 6 | It does not fetch market prices, does not talk to exchanges, and does not | ||
| 7 | change consensus. Price is a manual input until Bitflash has a reliable public | ||
| 8 | market. | ||
| 9 | |||
| 10 | ## Examples | ||
| 11 | |||
| 12 | Estimate from network hashrate: | ||
| 13 | |||
| 14 | ```bash | ||
| 15 | python3 scripts/bitflash-mining-calculator.py \ | ||
| 16 | --hashrate 21 --hashrate-unit kh/s \ | ||
| 17 | --network-hashrate 1.56 --network-hashrate-unit mh/s \ | ||
| 18 | --watts 140 --kwh-cost 0.10 \ | ||
| 19 | --price 0.001 --pool-fee 1 | ||
| 20 | ``` | ||
| 21 | |||
| 22 | Estimate from Bitflash difficulty: | ||
| 23 | |||
| 24 | ```bash | ||
| 25 | python3 scripts/bitflash-mining-calculator.py \ | ||
| 26 | --hashrate 21 --hashrate-unit kh/s \ | ||
| 27 | --difficulty 45600 \ | ||
| 28 | --watts 140 --kwh-cost 0.10 | ||
| 29 | ``` | ||
| 30 | |||
| 31 | Machine-readable output: | ||
| 32 | |||
| 33 | ```bash | ||
| 34 | python3 scripts/bitflash-mining-calculator.py \ | ||
| 35 | --hashrate 21 --hashrate-unit kh/s \ | ||
| 36 | --network-hashrate 1.56 --network-hashrate-unit mh/s \ | ||
| 37 | --json | ||
| 38 | ``` | ||
| 39 | |||
| 40 | ## Bitflash Difficulty Scale | ||
| 41 | |||
| 42 | Bitflash does not use Bitcoin's SHA256 difficulty-1 constant for pool shares. | ||
| 43 | The chain's easiest RandomX target is `bnProofOfWorkLimit = ~uint256(0) >> 12`, | ||
| 44 | so difficulty 1 expects about `4096` hashes. | ||
| 45 | |||
| 46 | The calculator uses: | ||
| 47 | |||
| 48 | ```text | ||
| 49 | network_hashrate = difficulty * 4096 / block_time_seconds | ||
| 50 | ``` | ||
| 51 | |||
| 52 | At the default two-minute block target: | ||
| 53 | |||
| 54 | ```text | ||
| 55 | network_hashrate = difficulty * 4096 / 120 | ||
| 56 | ``` | ||
| 57 | |||
| 58 | If you already know network hashrate, pass `--network-hashrate` directly and | ||
| 59 | the script derives the equivalent Bitflash difficulty for display. | ||
| 60 | |||
| 61 | ## Reward Estimate | ||
| 62 | |||
| 63 | The expected daily reward is: | ||
| 64 | |||
| 65 | ```text | ||
| 66 | blocks_per_day = 86400 / block_time_seconds | ||
| 67 | miner_share = miner_hashrate / network_hashrate | ||
| 68 | gross_btf_per_day = blocks_per_day * block_reward * miner_share | ||
| 69 | net_btf_per_day = gross_btf_per_day * (1 - pool_fee_percent / 100) | ||
| 70 | ``` | ||
| 71 | |||
| 72 | Electricity is: | ||
| 73 | |||
| 74 | ```text | ||
| 75 | electricity_per_day = watts / 1000 * 24 * kwh_cost | ||
| 76 | ``` | ||
| 77 | |||
| 78 | This is an expectation, not a promise. Solo mining variance can be very large, | ||
| 79 | especially when the miner has a small share of the network. | ||
| 80 | |||
| 81 | ## Future Website Use | ||
| 82 | |||
| 83 | The script is intentionally standalone so the same math can later back a | ||
| 84 | website widget or status dashboard. Until a public price exists, a website | ||
| 85 | calculator should keep price as a manual field and label fiat profit as | ||
| 86 | speculative. | ||
| 87 | |||
| 88 | ## Website Fields | ||
| 89 | |||
| 90 | A public calculator should ask for: | ||
| 91 | |||
| 92 | - miner hashrate and unit; | ||
| 93 | - power draw in watts; | ||
| 94 | - electricity price per kWh; | ||
| 95 | - pool fee percent; | ||
| 96 | - optional BTF price; | ||
| 97 | - either current Bitflash difficulty or current network hashrate. | ||
| 98 | |||
| 99 | The output should separate: | ||
| 100 | |||
| 101 | - estimated BTF per day; | ||
| 102 | - estimated gross fiat revenue, if a manual price was provided; | ||
| 103 | - electricity cost; | ||
| 104 | - net profit/loss; | ||
| 105 | - the difficulty/hashrate assumptions used. | ||
| 106 | |||
| 107 | Never present the fiat number as a market quote unless it comes from a real | ||
| 108 | public market. Before that exists, label it as a manual scenario. | ||
diff --git a/docs/pool-directory.md b/docs/pool-directory.md new file mode 100644 index 0000000..ff0640a --- /dev/null +++ b/docs/pool-directory.md | |||
| @@ -0,0 +1,61 @@ | |||
| 1 | # Public Pool Directory | ||
| 2 | |||
| 3 | `pool.bitflash.network` should be a public directory, not a network dependency. | ||
| 4 | Nodes discover pools through Nostr and `.btf`; the website only helps humans | ||
| 5 | compare operators. | ||
| 6 | |||
| 7 | Operators can publish the `pool_status.json` and `pool_rounds.json` files | ||
| 8 | produced by: | ||
| 9 | |||
| 10 | ```bash | ||
| 11 | bitflash -nogui -gen -operator \ | ||
| 12 | -poolstatusfile=/var/www/pool_status.json \ | ||
| 13 | -poolroundsfile=/var/www/pool_rounds.json | ||
| 14 | ``` | ||
| 15 | |||
| 16 | Build a static directory from one or more status files: | ||
| 17 | |||
| 18 | ```bash | ||
| 19 | python3 scripts/build-pool-index.py --out-dir public/pool \ | ||
| 20 | https://operator-one.example/pool_status.json \ | ||
| 21 | https://operator-two.example/pool_status.json | ||
| 22 | ``` | ||
| 23 | |||
| 24 | The generated output contains: | ||
| 25 | |||
| 26 | - `index.html`: human-readable ranking page. | ||
| 27 | - `pools.json`: machine-readable API for `api.bitflash.network/pools.json`. | ||
| 28 | |||
| 29 | The website is a convenience layer, not a consensus or discovery dependency. | ||
| 30 | Nodes discover pools through Nostr descriptors and `.btf`; a missing website | ||
| 31 | must not stop mining or pool operation. | ||
| 32 | |||
| 33 | When a source is an HTTP(S) `pool_status.json`, the directory links to a | ||
| 34 | `pool_rounds.json` proof file next to it. That file records the block height, | ||
| 35 | block hash, coinbase txid, coinbase value, pool fee, total shares, per-address | ||
| 36 | share counts, computed payouts, and payout txids once paid. | ||
| 37 | |||
| 38 | Ranking order is intentionally simple: | ||
| 39 | |||
| 40 | 1. online pools first; | ||
| 41 | 2. higher reported hashrate; | ||
| 42 | 3. more authorized miners; | ||
| 43 | 4. lower fee; | ||
| 44 | 5. name as a stable tie-breaker. | ||
| 45 | |||
| 46 | An entry is considered online when the source says the pool is running and its | ||
| 47 | status file is no more than 120 seconds old. | ||
| 48 | |||
| 49 | ## Recommended Hostnames | ||
| 50 | |||
| 51 | Use: | ||
| 52 | |||
| 53 | ```text | ||
| 54 | pool.bitflash.network human-readable pool directory | ||
| 55 | api.bitflash.network/pools.json | ||
| 56 | status.bitflash.network relay/node health, not pool ranking | ||
| 57 | docs.bitflash.network operator setup guide | ||
| 58 | ``` | ||
| 59 | |||
| 60 | The GUI may use the API as a shortcut for humans, but it should keep the | ||
| 61 | existing decentralized pool discovery path as the source of truth. | ||
diff --git a/docs/release-verification.md b/docs/release-verification.md index ecacd5f..b8eaf5e 100644 --- a/docs/release-verification.md +++ b/docs/release-verification.md | |||
| @@ -3,60 +3,100 @@ | |||
| 3 | Bitflash releases publish binaries and a `SHA256SUMS` file. The checksum file | 3 | Bitflash releases publish binaries and a `SHA256SUMS` file. The checksum file |
| 4 | proves that the file you downloaded matches the file the release page names. | 4 | proves that the file you downloaded matches the file the release page names. |
| 5 | A detached signature, `SHA256SUMS.asc`, proves that the checksum file itself was | 5 | A detached signature, `SHA256SUMS.asc`, proves that the checksum file itself was |
| 6 | signed by a trusted release key. | 6 | signed by the Bitflash release key. |
| 7 | 7 | ||
| 8 | That gives users two separate checks: | 8 | That gives users two separate checks: |
| 9 | 9 | ||
| 10 | 1. `gpg --verify SHA256SUMS.asc SHA256SUMS` checks who signed the checksums. | 10 | 1. `gpg --verify SHA256SUMS.asc SHA256SUMS` — who signed the checksums. |
| 11 | 2. `sha256sum -c SHA256SUMS` checks the binaries against those checksums. | 11 | 2. `sha256sum -c SHA256SUMS` — the binaries against those checksums. |
| 12 | 12 | ||
| 13 | ## User Check | 13 | But the first check is only worth something if you require a **specific** key. |
| 14 | A good signature by *some* key proves nothing; anyone can make one. | ||
| 14 | 15 | ||
| 15 | From a shell with `curl` or `wget` and `sha256sum`: | 16 | ## The release signing key |
| 17 | |||
| 18 | ```text | ||
| 19 | fingerprint 910A 2B4C CA87 9E81 FB4B 2AEA 1D4A 53D3 B78A A4B8 | ||
| 20 | uid Bitflash Releases (bitflash.network release signing) | ||
| 21 | <releases@bitflash.network> | ||
| 22 | ``` | ||
| 23 | |||
| 24 | The public key is published at | ||
| 25 | `https://releases.bitflash.network/bitflash-release-key.asc` and embedded in | ||
| 26 | `scripts/verify-release.sh`, so the tool's trust anchor lives in this | ||
| 27 | repository rather than on the server it verifies. | ||
| 28 | |||
| 29 | ## User check | ||
| 30 | |||
| 31 | From a shell with `curl` or `wget`, `sha256sum` and `gpg`: | ||
| 16 | 32 | ||
| 17 | ```bash | 33 | ```bash |
| 18 | scripts/verify-release.sh latest | 34 | scripts/verify-release.sh latest |
| 19 | ``` | 35 | ``` |
| 20 | 36 | ||
| 21 | For a specific release: | 37 | The public release host should expose latest-file aliases for convenience: |
| 22 | 38 | ||
| 23 | ```bash | 39 | ```text |
| 24 | scripts/verify-release.sh v1.2.13 | 40 | https://releases.bitflash.network/SHA256SUMS |
| 41 | https://releases.bitflash.network/SHA256SUMS.asc | ||
| 25 | ``` | 42 | ``` |
| 26 | 43 | ||
| 27 | For release audits, require the signature: | 44 | Those aliases must point at the current versioned directory. They do not replace |
| 45 | signature verification; they only make the common manual path shorter. | ||
| 46 | |||
| 47 | For a specific release: | ||
| 28 | 48 | ||
| 29 | ```bash | 49 | ```bash |
| 30 | scripts/verify-release.sh v1.2.13 --require-signature | 50 | scripts/verify-release.sh v1.2.15 |
| 31 | ``` | 51 | ``` |
| 32 | 52 | ||
| 33 | Older releases may not have `SHA256SUMS.asc`. In that case the script warns and | 53 | The script downloads the release, imports the pinned key into a throwaway |
| 34 | still checks file integrity. New release audits should use `--require-signature`. | 54 | keyring, and **refuses anything that is not a valid signature by the key above** |
| 35 | 55 | before checking the hashes. Releases from before signing was introduced ship | |
| 36 | ## Maintainer Flow | 56 | only `SHA256SUMS`; without `--require-signature` those are checked for integrity |
| 57 | and reported as unsigned. | ||
| 37 | 58 | ||
| 38 | After building release assets in the repository root: | 59 | By hand, if you would rather not run the script: |
| 39 | 60 | ||
| 40 | ```bash | 61 | ```bash |
| 41 | scripts/make-release-checksums.sh --sign --local-user RELEASE_KEY_ID | 62 | curl -O https://releases.bitflash.network/bitflash-release-key.asc |
| 63 | gpg --import bitflash-release-key.asc | ||
| 64 | gpg --verify SHA256SUMS.asc SHA256SUMS # must name the fingerprint above | ||
| 65 | sha256sum -c SHA256SUMS | ||
| 42 | ``` | 66 | ``` |
| 43 | 67 | ||
| 44 | Upload all built assets plus: | 68 | ## Maintainer flow |
| 45 | 69 | ||
| 46 | ```text | 70 | The signing key lives on a dedicated location, not in the build tree. After |
| 47 | SHA256SUMS | 71 | building the release assets in the repository root, write and sign the |
| 48 | SHA256SUMS.asc | 72 | checksums: |
| 73 | |||
| 74 | ```bash | ||
| 75 | sha256sum Bitflash-*.zip Bitflash-*.AppImage bitflash-node-* > SHA256SUMS | ||
| 76 | gpg --armor --detach-sign --output SHA256SUMS.asc SHA256SUMS | ||
| 49 | ``` | 77 | ``` |
| 50 | 78 | ||
| 51 | Keep the private signing key offline or on a dedicated release machine. Publish | 79 | Publish, alongside the assets, `SHA256SUMS` and `SHA256SUMS.asc`, and update |
| 52 | the public key fingerprint in the release notes and keep using the same key for | 80 | `latest.txt` on the releases host to the new tag. Keep using the same key; if it |
| 53 | future releases unless there is a clearly announced rotation. | 81 | is ever rotated, announce the new fingerprint clearly and update the embedded |
| 82 | key in `verify-release.sh`. | ||
| 83 | |||
| 84 | For Windows releases with managed Tor built in, run `make windows-tor`. It | ||
| 85 | creates `Bitflash-*-windows-with-tor.zip` by downloading the Tor Expert Bundle, | ||
| 86 | checking the pinned SHA256 in `scripts/package-windows-tor.sh`, optionally | ||
| 87 | verifying the Tor Project `.asc` signature, and placing `tor/tor.exe` next to | ||
| 88 | `Bitflash.exe`. For release builds, run `TOR_VERIFY_GPG=required make | ||
| 89 | windows-tor` so the signature check is mandatory. Include that zip in | ||
| 90 | `SHA256SUMS` like any other release asset. | ||
| 54 | 91 | ||
| 55 | ## Why This Matters | 92 | ## Why this matters |
| 56 | 93 | ||
| 57 | `SHA256SUMS` alone protects against a broken download, but not against someone | 94 | `SHA256SUMS` alone protects against a broken download, but not against someone |
| 58 | replacing both a binary and the checksum file. Signing `SHA256SUMS` means an | 95 | replacing both a binary and the checksum file. Signing `SHA256SUMS`, and |
| 59 | attacker must also have the release signing key to make the replacement verify. | 96 | requiring a pinned key, means an attacker must also hold the release signing key |
| 97 | to make the replacement verify. Serving from bitflash.network makes this the | ||
| 98 | guarantee that matters: without a pinned key, "trust the site" is the only | ||
| 99 | thing standing behind a download. | ||
| 60 | 100 | ||
| 61 | This is not reproducible builds yet. It is the smaller, immediate step that | 101 | This is not reproducible builds yet. It is the smaller, immediate step that |
| 62 | makes every release asset auditable by users before they run it. | 102 | makes every release asset auditable before anyone runs it. |
diff --git a/docs/rendezvous.md b/docs/rendezvous.md new file mode 100644 index 0000000..68b5df3 --- /dev/null +++ b/docs/rendezvous.md | |||
| @@ -0,0 +1,336 @@ | |||
| 1 | # Bitflash `.btf` Rendezvous Protocol | ||
| 2 | |||
| 3 | This is the specification of how Bitflash nodes reach each other, and how miners | ||
| 4 | reach pools, without revealing an IP address and without any central directory. | ||
| 5 | It documents the protocol as implemented in `src/btfaddr.cpp`, `src/btfchan.cpp`, | ||
| 6 | `src/btfrv.cpp`, `src/btftunnel.cpp` and `src/nostr.cpp`. | ||
| 7 | |||
| 8 | The design is ported from the Itzal hidden service design. A `.btf` address names | ||
| 9 | a public key, a node proves ownership of that key by signing a service | ||
| 10 | descriptor, other nodes find the descriptor over Nostr, and the two sides then | ||
| 11 | meet through an untrusted relay over an end to end encrypted tunnel. The relay | ||
| 12 | carries ciphertext only and never learns the plaintext. | ||
| 13 | |||
| 14 | ## 1. Overview | ||
| 15 | |||
| 16 | A connection to a `.btf` service has three layers: | ||
| 17 | |||
| 18 | 1. **Discovery (Nostr).** The service publishes a signed descriptor as a Nostr | ||
| 19 | event. A caller fetches it by the service public key and verifies the | ||
| 20 | signature. The descriptor says which rendezvous relay the service is waiting | ||
| 21 | at, which encryption key to use, and optionally which Tor hidden service can | ||
| 22 | be dialed directly. | ||
| 23 | 2. **Rendezvous (TCP).** Caller and service both connect out to the same relay | ||
| 24 | and are paired by public key. Neither needs an inbound port or a public IP. | ||
| 25 | After pairing the relay is a blind byte pipe. | ||
| 26 | 3. **Direct onion fallback bypass.** If the descriptor carries a valid signed | ||
| 27 | `.onion:port` endpoint and the caller is running with Tor mode, the caller | ||
| 28 | tries that endpoint before using rendezvous. This removes the rendezvous | ||
| 29 | relay from the data path for peers that choose to run hidden services. | ||
| 30 | 4. **Encrypted channel (X25519 + XChaCha20-Poly1305).** Over the paired pipe the | ||
| 31 | caller and service run an ephemeral to static handshake and then exchange | ||
| 32 | authenticated frames. The relay sees only ciphertext. | ||
| 33 | |||
| 34 | The caller ends up holding an ordinary local socket that behaves like a direct | ||
| 35 | TCP connection to the peer. The node treats it exactly like any other peer | ||
| 36 | socket. | ||
| 37 | |||
| 38 | ## 2. Identities and keys | ||
| 39 | |||
| 40 | A node holds two independent keypairs. | ||
| 41 | |||
| 42 | - **Signing / identity key**: a secp256k1 x-only key (BIP340 Schnorr). This is | ||
| 43 | the `.btf` identity and also the node's Nostr identity. Persisted in | ||
| 44 | `nostr.key`. | ||
| 45 | - **Encryption key**: an X25519 (Curve25519) keypair. Persisted in | ||
| 46 | `btf_enc.key`. Its public half is published in the descriptor as `enc` and is | ||
| 47 | used for the tunnel key agreement. | ||
| 48 | |||
| 49 | The identity key names the service and authenticates the descriptor. The | ||
| 50 | encryption key secures the tunnel. They are separate so the long term identity | ||
| 51 | never performs Diffie-Hellman and the encryption key can be rotated in a new | ||
| 52 | signed descriptor. | ||
| 53 | |||
| 54 | ## 3. The `.btf` address | ||
| 55 | |||
| 56 | An address is the identity public key plus a short checksum, Base32 encoded, with | ||
| 57 | the suffix `.btf`. | ||
| 58 | |||
| 59 | ``` | ||
| 60 | label = Base32( pubkey[32] || checksum[2] ) | ||
| 61 | address = label + ".btf" | ||
| 62 | ``` | ||
| 63 | |||
| 64 | - `pubkey` is the 32 byte secp256k1 x-only public key. | ||
| 65 | - `checksum` is the first two bytes of `SHA256(".btf-checksum-v1" || pubkey)`. | ||
| 66 | - Base32 uses the lowercase alphabet `abcdefghijklmnopqrstuvwxyz234567`, no | ||
| 67 | padding. The 34 byte payload encodes to 55 characters. | ||
| 68 | - Parsing is case insensitive, trims surrounding whitespace, and strips a | ||
| 69 | trailing dot. A wrong checksum is rejected. | ||
| 70 | |||
| 71 | The address is self certifying: it *is* the public key, so no registry has to be | ||
| 72 | trusted to map a name to a key. | ||
| 73 | |||
| 74 | ## 4. Discovery over Nostr | ||
| 75 | |||
| 76 | Nodes publish and read three kinds of parameterized replaceable Nostr events. | ||
| 77 | Parameterized replaceable means the newest event for a given author and `d` tag | ||
| 78 | supersedes the older one, so a descriptor is always the latest state. | ||
| 79 | |||
| 80 | | Purpose | kind | `d` tag | | ||
| 81 | |---|---|---| | ||
| 82 | | Service descriptor | 38501 | `btf-descriptor-2` | | ||
| 83 | | Rendezvous relay announcement | 38502 | `btf-relay-2` | | ||
| 84 | | Pool announcement | 38503 | `btf-pool-2` | | ||
| 85 | |||
| 86 | The Nostr event is signed with the node's identity key, so the event author | ||
| 87 | equals the `.btf` public key. That binds the descriptor to the address at the | ||
| 88 | Nostr layer, and the descriptor is *also* signed inside its own content (see | ||
| 89 | below), so a relay operator cannot forge or swap a descriptor for an address it | ||
| 90 | does not control. | ||
| 91 | |||
| 92 | **Resolving an address.** To reach `X.btf`, a caller parses the address to the 32 | ||
| 93 | byte key and sends a Nostr `REQ` filtered by `authors = [X]`, `kinds = [38501]`, | ||
| 94 | `#d = ["btf-descriptor-2"]`, `limit = 1`. It verifies the returned descriptor and | ||
| 95 | uses its `meeting_node` and `enc`. | ||
| 96 | |||
| 97 | **Relay auto discovery.** A relay operator runs a rendezvous relay and publishes a | ||
| 98 | kind 38502 event carrying `host:port`. Every node that reads it adds the relay to | ||
| 99 | the set it can register at and dial through. No seed list edit or maintainer | ||
| 100 | approval is needed. A small curated seed list is still shipped for bootstrap: | ||
| 101 | |||
| 102 | ``` | ||
| 103 | 92.246.128.180:8434 | ||
| 104 | 31.44.4.249:8434 | ||
| 105 | 90.156.222.107:8434 | ||
| 106 | ``` | ||
| 107 | |||
| 108 | The default rendezvous port is `8434`. | ||
| 109 | |||
| 110 | ## 5. The service descriptor | ||
| 111 | |||
| 112 | The descriptor is a JSON object, signed with the identity key over a domain | ||
| 113 | separated digest of its own fields. | ||
| 114 | |||
| 115 | Legacy descriptor: | ||
| 116 | |||
| 117 | ```json | ||
| 118 | { | ||
| 119 | "v": 2, | ||
| 120 | "pubkey": "<hex 32-byte x-only identity key>", | ||
| 121 | "enc": "<hex 32-byte X25519 public key>", | ||
| 122 | "meeting_node": "host:port", | ||
| 123 | "created": <unix seconds>, | ||
| 124 | "sig": "<hex 64-byte BIP340 Schnorr signature>" | ||
| 125 | } | ||
| 126 | ``` | ||
| 127 | |||
| 128 | Onion-capable descriptor: | ||
| 129 | |||
| 130 | ```json | ||
| 131 | { | ||
| 132 | "v": 3, | ||
| 133 | "pubkey": "<hex 32-byte x-only identity key>", | ||
| 134 | "enc": "<hex 32-byte X25519 public key>", | ||
| 135 | "meeting_node": "host:port", | ||
| 136 | "onion": "<56-char-v3-name>.onion:8433", | ||
| 137 | "created": <unix seconds>, | ||
| 138 | "sig": "<legacy v2 signature>", | ||
| 139 | "sig3": "<signature over the onion-capable digest>" | ||
| 140 | } | ||
| 141 | ``` | ||
| 142 | |||
| 143 | The legacy signed digest is: | ||
| 144 | |||
| 145 | ``` | ||
| 146 | digest = SHA256( ".btf-descriptor-v2" | ||
| 147 | || pubkey[32] | ||
| 148 | || enc (raw bytes of the string) | ||
| 149 | || meeting_node (raw bytes of the string) | ||
| 150 | || created (8 bytes, big endian) ) | ||
| 151 | ``` | ||
| 152 | |||
| 153 | `sig` is kept on v3 descriptors so old nodes can still resolve and dial through | ||
| 154 | the rendezvous relay. The onion-capable digest is: | ||
| 155 | |||
| 156 | ``` | ||
| 157 | digest3 = SHA256( ".btf-descriptor-v3-onion" | ||
| 158 | || pubkey[32] | ||
| 159 | || enc (raw bytes of the string) | ||
| 160 | || meeting_node (raw bytes of the string) | ||
| 161 | || onion (raw bytes of the normalized string) | ||
| 162 | || created (8 bytes, big endian) ) | ||
| 163 | ``` | ||
| 164 | |||
| 165 | `sig` and `sig3` are deterministic BIP340 Schnorr signatures (no auxiliary | ||
| 166 | randomness) over their respective digests by the identity key. | ||
| 167 | |||
| 168 | **Verification** (untrusted input, must never throw): parse the JSON, require all | ||
| 169 | fields with the right types, confirm `pubkey` equals the address being resolved, | ||
| 170 | recompute the digest, and verify the Schnorr signature. Any failure rejects the | ||
| 171 | descriptor. | ||
| 172 | |||
| 173 | If `onion` is present, readers accept it only when it is a normalized v3 onion | ||
| 174 | endpoint and `sig3` verifies. A bad or tampered onion field is ignored; the v2 | ||
| 175 | descriptor can still be used for rendezvous fallback if `sig` verifies. | ||
| 176 | |||
| 177 | `created` lets a reader prefer the freshest descriptor when more than one is | ||
| 178 | seen. A node without a direct onion endpoint only publishes a descriptor once it | ||
| 179 | has actually registered at a `meeting_node`, so it never advertises a relay it | ||
| 180 | cannot be reached at. A node with a direct onion endpoint may publish before | ||
| 181 | rendezvous registration using `meeting_node = "rendezvous-pending"`; Tor peers | ||
| 182 | can still dial the signed onion endpoint, and rendezvous becomes fallback once | ||
| 183 | registration succeeds. | ||
| 184 | |||
| 185 | ## 6. Rendezvous transport | ||
| 186 | |||
| 187 | The relay is a small TCP server. Every connection opens with a fixed 33 byte | ||
| 188 | header: | ||
| 189 | |||
| 190 | ``` | ||
| 191 | byte 0 role: 'S' (service) or 'C' (client) | ||
| 192 | bytes 1..32 target public key (the .btf identity, 32 bytes) | ||
| 193 | ``` | ||
| 194 | |||
| 195 | **Service registration (`S`).** The service connects out to the relay and sends | ||
| 196 | `'S' || own_pubkey`. The relay stores the connection in a table keyed by the | ||
| 197 | public key and leaves it open, waiting. There is one slot per key: a new | ||
| 198 | registration for the same key replaces and closes the old one. The service does | ||
| 199 | not block waiting to be paired at registration time; it is considered registered | ||
| 200 | the moment the relay reads the header, which is what lets it publish a descriptor | ||
| 201 | pointing at this relay. | ||
| 202 | |||
| 203 | **Client dial (`C`).** The caller connects out to the same relay and sends | ||
| 204 | `'C' || target_pubkey`. The relay looks up the waiting service: | ||
| 205 | |||
| 206 | - If present, it removes it from the table, writes a single `0x01` byte to both | ||
| 207 | sockets, and then forwards raw bytes in both directions until either side | ||
| 208 | closes. From this point the relay is a blind pipe. | ||
| 209 | - If absent, it writes `0x00` to the caller and closes. | ||
| 210 | |||
| 211 | Both sides connect *outbound* to the relay, so neither needs a public IP or an | ||
| 212 | open inbound port. The relay never sees the identity of the caller (the caller | ||
| 213 | sends no key of its own, only the target it wants). | ||
| 214 | |||
| 215 | **Liveness.** A registration socket is idle by design and can outlive the network | ||
| 216 | path that created it (a NAT drops the idle mapping, the relay restarts, a route | ||
| 217 | changes) with no FIN and nothing readable. TCP keepalive was measured and does | ||
| 218 | not reliably detect this. The service therefore imposes its own timeout on the | ||
| 219 | wait for the pairing byte and re-registers on expiry. Expiring is not a failure; | ||
| 220 | it costs one reconnect per interval and returns the loop to a known state. | ||
| 221 | |||
| 222 | ## 7. The encrypted channel | ||
| 223 | |||
| 224 | Once paired through the relay, the caller and service run an end to end handshake | ||
| 225 | and then exchange authenticated frames. The primitive is libsodium | ||
| 226 | `crypto_box_curve25519xchacha20poly1305` (X25519 key agreement, XChaCha20-Poly1305 | ||
| 227 | AEAD), the same construction as Rust's `crypto_box` ChaChaBox. | ||
| 228 | |||
| 229 | **Handshake (ephemeral to static).** | ||
| 230 | |||
| 231 | 1. The caller generates a fresh ephemeral X25519 keypair, computes | ||
| 232 | `shared = X25519(eph_sk, service_enc_pub)` reduced to a box key | ||
| 233 | (`crypto_box_beforenm`), and sends the 32 byte ephemeral public key over the | ||
| 234 | paired pipe. | ||
| 235 | 2. The service reads the ephemeral public key and computes the same | ||
| 236 | `shared = X25519(enc_sk, eph_pub)`. | ||
| 237 | |||
| 238 | Because the caller uses a new ephemeral key per connection, each session has an | ||
| 239 | independent key and forward secrecy against later compromise of the service | ||
| 240 | encryption key. | ||
| 241 | |||
| 242 | **Framing.** Every message is length prefixed: | ||
| 243 | |||
| 244 | ``` | ||
| 245 | frame = len[4] (big endian, length of the body that follows) | ||
| 246 | body = nonce[24] || ciphertext || mac[16] | ||
| 247 | ``` | ||
| 248 | |||
| 249 | The nonce is 24 random bytes per frame (XChaCha20), the MAC is 16 bytes | ||
| 250 | (Poly1305). A body shorter than `nonce + mac` or larger than 1 MiB is rejected. | ||
| 251 | Decryption that fails authentication drops the frame and tears the tunnel down. | ||
| 252 | |||
| 253 | The tunnel is transparent: the application is handed a local socket pair end and | ||
| 254 | reads and writes plaintext. Two pump threads move data between that socket and | ||
| 255 | the relay socket, encrypting outbound and decrypting inbound. | ||
| 256 | |||
| 257 | ## 8. End to end connection flow | ||
| 258 | |||
| 259 | To dial `X.btf`: | ||
| 260 | |||
| 261 | 1. Parse `X.btf` to the 32 byte identity key. | ||
| 262 | 2. Fetch and verify the descriptor over Nostr (kind 38501). Obtain | ||
| 263 | `meeting_node` and `enc`. | ||
| 264 | 3. Connect to `meeting_node`, send `'C' || X`, and wait for `0x01`. | ||
| 265 | 4. Generate an ephemeral X25519 key, derive the shared key against `enc`, send | ||
| 266 | the ephemeral public key. | ||
| 267 | 5. Exchange XChaCha20-Poly1305 frames. Hand the application a local socket that | ||
| 268 | speaks the ordinary peer protocol. | ||
| 269 | |||
| 270 | A service, symmetrically: registers at a relay (`'S' || own_pubkey`), publishes a | ||
| 271 | descriptor pointing at that relay, waits for `0x01`, reads the caller ephemeral | ||
| 272 | public key, derives the shared key with its encryption secret, and serves the | ||
| 273 | tunnel. | ||
| 274 | |||
| 275 | ## 9. Security properties | ||
| 276 | |||
| 277 | - **Address unforgeability.** The address is the identity key and the descriptor | ||
| 278 | is Schnorr signed by it, and the Nostr event is signed by it. A relay or Nostr | ||
| 279 | operator cannot publish a working descriptor for an address it does not own. | ||
| 280 | - **Relay confidentiality.** The relay forwards ciphertext and never holds the | ||
| 281 | channel key. It cannot read or modify traffic undetected; a tampered frame | ||
| 282 | fails the Poly1305 tag and the tunnel closes. | ||
| 283 | - **Caller anonymity to the service and relay.** The caller presents no identity | ||
| 284 | key, only the target it wants to reach. It connects outbound, so its address is | ||
| 285 | not published anywhere. | ||
| 286 | - **Forward secrecy.** The caller's per connection ephemeral key means a later | ||
| 287 | compromise of the service encryption key does not decrypt past sessions. | ||
| 288 | - **IP privacy.** Neither side needs a public IP or an inbound port for | ||
| 289 | rendezvous. With a signed `.onion` endpoint, the reachable side exposes only a | ||
| 290 | Tor hidden service and the relay is not in the peer connection path. | ||
| 291 | |||
| 292 | ## 10. Threat model and limitations | ||
| 293 | |||
| 294 | - **The relay is trusted for availability, not for confidentiality.** It can | ||
| 295 | refuse to pair, drop a connection, or go offline. This is why relays are | ||
| 296 | plural, auto discovered, and re-registration is periodic. It cannot read | ||
| 297 | traffic. | ||
| 298 | - **Traffic analysis is not addressed.** A network observer, or the relay, can | ||
| 299 | see connection timing and byte volumes. There is no padding or mixing. | ||
| 300 | - **Single slot registration causes churn.** One waiting slot per key per relay | ||
| 301 | means a race or a stale registration can briefly make a service unreachable at | ||
| 302 | that relay. Multiple relays and the wait timeout bound the damage, but this is | ||
| 303 | a known source of connection churn. | ||
| 304 | - **Descriptor freshness is best effort.** `created` is a self reported | ||
| 305 | timestamp; readers prefer the newest descriptor they have seen but there is no | ||
| 306 | global clock. | ||
| 307 | - **Discovery depends on public Nostr relays.** If the relays a node uses are all | ||
| 308 | unreachable, discovery stalls until one is reachable again. Relay TLS is | ||
| 309 | verified, which closes a man in the middle path on discovery, but availability | ||
| 310 | still rests on the relay set. | ||
| 311 | - **Direct onion needs Tor.** With `-managedtor`, Bitflash starts a Tor process, | ||
| 312 | writes a local hidden-service config, and advertises the generated onion | ||
| 313 | endpoint. With `-onionservice`, an operator can still run Tor manually and | ||
| 314 | provide the endpoint. In both cases Tor owns the hidden-service private key and | ||
| 315 | uptime. | ||
| 316 | |||
| 317 | ## 11. Wire reference | ||
| 318 | |||
| 319 | | Item | Value | | ||
| 320 | |---|---| | ||
| 321 | | `.btf` Base32 alphabet | `abcdefghijklmnopqrstuvwxyz234567`, no padding | | ||
| 322 | | Address payload | `pubkey[32] || checksum[2]`, 55 Base32 chars | | ||
| 323 | | Address checksum | `SHA256(".btf-checksum-v1" || pubkey)[0..2]` | | ||
| 324 | | Descriptor domain sep | `.btf-descriptor-v2`; optional onion sep `.btf-descriptor-v3-onion` | | ||
| 325 | | Descriptor version | `2`; `3` when a signed onion endpoint is present | | ||
| 326 | | Descriptor signature | BIP340 Schnorr, deterministic (no aux rand), v3 keeps `sig` plus `sig3` | | ||
| 327 | | Nostr descriptor event | kind `38501`, `d = btf-descriptor-2` | | ||
| 328 | | Nostr relay event | kind `38502`, `d = btf-relay-2` | | ||
| 329 | | Nostr pool event | kind `38503`, `d = btf-pool-2` | | ||
| 330 | | Rendezvous port (default) | `8434` | | ||
| 331 | | Rendezvous header | `role[1] ('S'/'C') || pubkey[32]`, 33 bytes | | ||
| 332 | | Rendezvous pairing byte | `0x01` paired, `0x00` no service | | ||
| 333 | | Channel primitive | `crypto_box_curve25519xchacha20poly1305` | | ||
| 334 | | Nonce / MAC / key sizes | 24 / 16 / 32 bytes | | ||
| 335 | | Frame | `len[4 big endian] || nonce[24] || ct || mac[16]` | | ||
| 336 | | Max frame body | 1 MiB | | ||
diff --git a/docs/site.md b/docs/site.md new file mode 100644 index 0000000..9526a61 --- /dev/null +++ b/docs/site.md | |||
| @@ -0,0 +1,115 @@ | |||
| 1 | # Public Site Contract | ||
| 2 | |||
| 3 | The public Bitflash sites are generated from this repository. If a deployed | ||
| 4 | page and the source disagree, the deployed page is stale. | ||
| 5 | |||
| 6 | ## Canonical Navigation | ||
| 7 | |||
| 8 | Every public page should use the same top navigation: | ||
| 9 | |||
| 10 | | label | target | | ||
| 11 | |---|---| | ||
| 12 | | home | `https://bitflash.network/` | | ||
| 13 | | code | `https://git.bitflash.network/bitflash/bitflash` | | ||
| 14 | | docs | `https://docs.bitflash.network/` | | ||
| 15 | | downloads | `https://releases.bitflash.network/` | | ||
| 16 | | explorer | `https://explorer.bitflash.network/` | | ||
| 17 | | status | `https://status.bitflash.network/` | | ||
| 18 | |||
| 19 | The `code` link intentionally points at the repository, not the Forgejo | ||
| 20 | instance root. A visitor choosing "code" wants the source tree. | ||
| 21 | |||
| 22 | ## Documentation Pages To Publish | ||
| 23 | |||
| 24 | The docs index should expose these source files: | ||
| 25 | |||
| 26 | | page | source | why it matters | | ||
| 27 | |---|---|---| | ||
| 28 | | Manual | `README.md` | run, mine, recover, operate | | ||
| 29 | | Key derivation and address format | `docs/derivation.md` | independent wallet recovery | | ||
| 30 | | Release verification | `docs/release-verification.md` | pinned signing-key flow | | ||
| 31 | | Dependencies | `docs/dependencies.md` | pinned secp256k1/RandomX supply chain | | ||
| 32 | | Rendezvous and .btf addressing | `docs/rendezvous.md` | network design and caveats | | ||
| 33 | | Tor mode | `docs/tor.md` | `-tor`, `-socks`, onion relay operation | | ||
| 34 | | Fair launch verifier | `docs/fair-launch.md` | no-premine launch audit | | ||
| 35 | | UTXO set commitment | `docs/utxo-commitment.md` | deterministic supply/root audit | | ||
| 36 | | UTXO inclusion proofs | `docs/utxo-proofs.md` | proof for one unspent output | | ||
| 37 | | Mining calculator | `docs/mining-calculator.md` | local WhatToMine-style estimates | | ||
| 38 | | Public pool directory | `docs/pool-directory.md` | `pool_status.json`, `pools.json`, ranking | | ||
| 39 | |||
| 40 | ## Release Mirror Aliases | ||
| 41 | |||
| 42 | Each release remains under its versioned directory: | ||
| 43 | |||
| 44 | ```text | ||
| 45 | https://releases.bitflash.network/v1.2.19/SHA256SUMS | ||
| 46 | https://releases.bitflash.network/v1.2.19/SHA256SUMS.asc | ||
| 47 | ``` | ||
| 48 | |||
| 49 | The release root should also alias the latest files: | ||
| 50 | |||
| 51 | ```text | ||
| 52 | https://releases.bitflash.network/SHA256SUMS | ||
| 53 | https://releases.bitflash.network/SHA256SUMS.asc | ||
| 54 | https://releases.bitflash.network/Bitflash-latest-windows.zip | ||
| 55 | https://releases.bitflash.network/Bitflash-latest-x86_64.AppImage | ||
| 56 | https://releases.bitflash.network/bitflash-node-latest-x86_64 | ||
| 57 | ``` | ||
| 58 | |||
| 59 | The aliases are convenience only. Verification must still require the pinned | ||
| 60 | release key documented in `docs/release-verification.md`. | ||
| 61 | |||
| 62 | ## Static Explorer | ||
| 63 | |||
| 64 | `scripts/build-explorer.py` writes a static explorer with no backend: | ||
| 65 | |||
| 66 | ```text | ||
| 67 | index.html | ||
| 68 | style.css | ||
| 69 | explorer.js | ||
| 70 | logo.png | ||
| 71 | blocks.json | ||
| 72 | block/<height>.json | ||
| 73 | ``` | ||
| 74 | |||
| 75 | Serve it with a strict content policy: | ||
| 76 | |||
| 77 | ```text | ||
| 78 | default-src 'none'; | ||
| 79 | img-src 'self'; | ||
| 80 | style-src 'self'; | ||
| 81 | script-src 'self'; | ||
| 82 | connect-src 'self'; | ||
| 83 | base-uri 'none'; | ||
| 84 | form-action 'none'; | ||
| 85 | frame-ancestors 'none'; | ||
| 86 | ``` | ||
| 87 | |||
| 88 | The viewer must never use `innerHTML` for chain data. A malicious transaction | ||
| 89 | must be rendered as text, not interpreted as markup. | ||
| 90 | |||
| 91 | ## Cache Policy | ||
| 92 | |||
| 93 | Recommended defaults: | ||
| 94 | |||
| 95 | | path | cache | | ||
| 96 | |---|---| | ||
| 97 | | HTML pages | `public, max-age=0, must-revalidate` | | ||
| 98 | | `status.json` | `no-store` | | ||
| 99 | | `blocks.json`, `block/*.json` | `public, max-age=0, must-revalidate` | | ||
| 100 | | release assets | `public, max-age=14400, must-revalidate` | | ||
| 101 | | `SHA256SUMS`, signatures, keys | `public, max-age=0, must-revalidate` | | ||
| 102 | |||
| 103 | ## Security Headers | ||
| 104 | |||
| 105 | Recommended for all static pages: | ||
| 106 | |||
| 107 | ```text | ||
| 108 | Strict-Transport-Security: max-age=31536000; includeSubDomains | ||
| 109 | X-Content-Type-Options: nosniff | ||
| 110 | Referrer-Policy: no-referrer | ||
| 111 | Permissions-Policy: interest-cohort=(), geolocation=(), camera=(), microphone=() | ||
| 112 | ``` | ||
| 113 | |||
| 114 | Forgejo has its own application headers and should not blindly inherit the | ||
| 115 | static-site CSP. | ||
diff --git a/docs/social.png b/docs/social.png index 19e2a3b..9a08b02 100644 --- a/docs/social.png +++ b/docs/social.png | |||
| Binary files differ | |||
diff --git a/docs/storage.md b/docs/storage.md new file mode 100644 index 0000000..c7de4e7 --- /dev/null +++ b/docs/storage.md | |||
| @@ -0,0 +1,159 @@ | |||
| 1 | # Bitflash Storage Inventory | ||
| 2 | |||
| 3 | This document records the Berkeley DB surface that still exists in Bitflash. | ||
| 4 | It is a migration map, not a promise that all stores should move at once. | ||
| 5 | |||
| 6 | The immediate priority is `wallet.dat`: it holds spend authority and recovery | ||
| 7 | metadata. `blkindex.dat` can be rebuilt from block files, so it is important for | ||
| 8 | startup correctness but less dangerous than wallet storage. | ||
| 9 | |||
| 10 | ## Berkeley DB Environment | ||
| 11 | |||
| 12 | All `CDB` stores use one Berkeley DB environment rooted at: | ||
| 13 | |||
| 14 | ```text | ||
| 15 | <datadir>/database/ | ||
| 16 | ``` | ||
| 17 | |||
| 18 | The environment creates log files such as: | ||
| 19 | |||
| 20 | ```text | ||
| 21 | database/log.0000000001 | ||
| 22 | ``` | ||
| 23 | |||
| 24 | Those logs are part of the legacy BDB risk model. They are not user-facing | ||
| 25 | wallet files, but they can contain historical records while the environment is | ||
| 26 | live. Wallet encryption already rewrites `wallet.dat` and purges BDB environment | ||
| 27 | logs after the encrypted file is installed. | ||
| 28 | |||
| 29 | ## Common Records | ||
| 30 | |||
| 31 | Every `CDB` file can carry: | ||
| 32 | |||
| 33 | | Key | Value | Purpose | | ||
| 34 | |---|---|---| | ||
| 35 | | `"version"` | `int` | Database format version written when a DB is opened in create mode. | | ||
| 36 | |||
| 37 | Keys and values are serialized with the existing `CDataStream` format. A future | ||
| 38 | SQLite migration must preserve byte-for-byte value semantics until a higher | ||
| 39 | level schema migration is intentionally introduced. | ||
| 40 | |||
| 41 | ## `wallet.dat` | ||
| 42 | |||
| 43 | `wallet.dat` is accessed through `CWalletDB`. | ||
| 44 | Read-only tooling should go through `ScanWalletRecords()` instead of opening a | ||
| 45 | Berkeley DB cursor directly. That keeps migration checks, audits, and future | ||
| 46 | backends behind one record-stream boundary. Wallet rewrite code should also use | ||
| 47 | that boundary when it needs to copy existing records into a newly written | ||
| 48 | wallet file. | ||
| 49 | |||
| 50 | | Key shape | Value | Secret? | Meaning | | ||
| 51 | |---|---|---:|---| | ||
| 52 | | `("name", address)` | `string` | No | Address book label. | | ||
| 53 | | `("tx", txid)` | `CWalletTx` | No | Wallet transaction state, including spend flags and metadata. | | ||
| 54 | | `("key", pubkey)` | `CPrivKey` | Yes | Plain private key record for unencrypted wallets or old backups. | | ||
| 55 | | `"defaultkey"` | `vector<unsigned char>` | No | Default public key shown by the wallet. | | ||
| 56 | | `"hdmaster"` | `vector<unsigned char>` | Yes | Plain BIP32 master private key material. | | ||
| 57 | | `"hdchaincode"` | `vector<unsigned char>` | Yes | Plain BIP32 chain code. | | ||
| 58 | | `"hdnext"` | `unsigned int` | No | Legacy deterministic derivation next index. | | ||
| 59 | | `"hdschema"` | `int` | No | Deterministic derivation schema (`legacy` or `bip44`). | | ||
| 60 | | `"hdcointype"` | `unsigned int` | No | BIP44 coin type. Bitflash is `4346950`. | | ||
| 61 | | `"hdreceivenext"` | `unsigned int` | No | BIP44 external receive-chain next index. | | ||
| 62 | | `"hdchangenext"` | `unsigned int` | No | BIP44 internal change-chain next index. | | ||
| 63 | | `("mkey", id)` | `CWalletMasterKey` | Sensitive | Encrypted wallet master-key metadata. | | ||
| 64 | | `("ckey", pubkey)` | `vector<unsigned char>` | Yes | Private key encrypted under the wallet master key. | | ||
| 65 | | `"cryptedhdmaster"` | `vector<unsigned char>` | Yes | Encrypted BIP32 master private key material. | | ||
| 66 | | `"cryptedhdchaincode"` | `vector<unsigned char>` | Yes | Encrypted BIP32 chain code. | | ||
| 67 | | `"walletminversion"` | `int` | No | Minimum wallet format version required to open this wallet safely. | | ||
| 68 | | `("pool", index)` | `vector<unsigned char>` | No | Public key reserved in the key pool. The private key lives in `key`/`ckey`. | | ||
| 69 | | `("setting", name)` | typed setting value | Usually no | Wallet/UI settings such as mining mode, fees, and backup timestamps. | | ||
| 70 | |||
| 71 | ### Wallet Migration Rules | ||
| 72 | |||
| 73 | A future wallet backend must keep these properties: | ||
| 74 | |||
| 75 | 1. Unknown future wallet formats fail closed with a clear message. | ||
| 76 | 2. Encrypted wallets must encrypt private keys and HD seed material. | ||
| 77 | 3. A locked encrypted wallet may expose public keys and balances, but not | ||
| 78 | private keys or decrypted HD seed material. | ||
| 79 | 4. `-backupwallet` must produce a wallet file that opens in a different datadir | ||
| 80 | without copying `database/`, and must not leave a half-written final backup | ||
| 81 | path if the copy or install step fails. | ||
| 82 | 5. `-recoveryaudit` must distinguish: | ||
| 83 | - no recovery phrase installed; | ||
| 84 | - recovery phrase present; | ||
| 85 | - recovery phrase encrypted and locked, requiring unlock before coverage can | ||
| 86 | be audited. | ||
| 87 | |||
| 88 | ## SQLite Wallet Store | ||
| 89 | |||
| 90 | Since v1.2.20 SQLite is the default wallet backend. It is a byte-preserving | ||
| 91 | record store, not a new wallet format -- the same serialized `CDataStream` bytes | ||
| 92 | `wallet.dat` uses: | ||
| 93 | |||
| 94 | ```sql | ||
| 95 | CREATE TABLE wallet_records ( | ||
| 96 | key BLOB PRIMARY KEY NOT NULL, | ||
| 97 | value BLOB NOT NULL | ||
| 98 | ); | ||
| 99 | PRAGMA user_version=1; | ||
| 100 | ``` | ||
| 101 | |||
| 102 | A fresh data directory starts on `wallet.sqlite`. An existing `wallet.dat` keeps | ||
| 103 | running on Berkeley DB until it is converted: the GUI offers a one-click | ||
| 104 | conversion on first run, and `-walletbackend=sqlite` / `-walletbackend=bdb` | ||
| 105 | (recorded in a plain-text `wallet-backend` marker file in the datadir) select a | ||
| 106 | backend explicitly. Conversion never modifies `wallet.dat`; it stays as a | ||
| 107 | fallback, and the change takes effect on restart. | ||
| 108 | |||
| 109 | Every wallet operation runs on either backend: read, write, erase, key-pool and | ||
| 110 | HD-seed writes (the two halves of the seed bracketed in one SQLite transaction), | ||
| 111 | `-backupwallet` (checkpoint then copy the self-contained file), `-encryptwallet` | ||
| 112 | (an atomic rebuild-and-swap in one transaction), and the `-walletstorageaudit` / | ||
| 113 | `-walletstoragecheck` diagnostics, which read the active backend's file. | ||
| 114 | |||
| 115 | The migration tooling remains available for scripted use. `-walletsqliteexport=FILE` | ||
| 116 | streams every raw `wallet.dat` record through `ScanWalletRecords()` into a SQLite | ||
| 117 | file in one transaction, refusing to overwrite. `-walletsqliteverify=FILE` | ||
| 118 | compares the two streams and prints only counts. `-walletsqliteloadcheck=FILE` | ||
| 119 | parses an export with the same record types the loader uses. | ||
| 120 | `-walletsqliterestore=FILE` rebuilds a `wallet.dat` from an export in an empty | ||
| 121 | data directory. Failed exports and restores remove their incomplete output. | ||
| 122 | |||
| 123 | ## `blkindex.dat` | ||
| 124 | |||
| 125 | `blkindex.dat` is accessed through `CTxDB`. | ||
| 126 | |||
| 127 | | Key shape | Value | Rebuildable? | Meaning | | ||
| 128 | |---|---|---:|---| | ||
| 129 | | `("tx", txid)` | `CTxIndex` | Yes | Transaction disk position and spent-output index state. | | ||
| 130 | | `("owner", hash160, CDiskTxPos)` | `int height` | Yes | Legacy owner transaction index cursor range. | | ||
| 131 | | `("blockindex", blockhash)` | `CDiskBlockIndex` | Yes | Block index entry. | | ||
| 132 | | `"hashBestChain"` | `uint256` | Yes | Current best-chain tip hash. | | ||
| 133 | |||
| 134 | `blkindex.dat` can be rebuilt from `blk*.dat` plus consensus validation. A | ||
| 135 | future migration may therefore prefer a rebuild path over a byte-preserving | ||
| 136 | conversion path. | ||
| 137 | |||
| 138 | ## Other Legacy Stores | ||
| 139 | |||
| 140 | None. Upstream still documents `reviews.dat` (`CReviewDB`) and `market.dat` | ||
| 141 | (`CMarketDB`) here; this fork deleted the 2009 marketplace outright, along | ||
| 142 | with `CReviewDB`, `market.cpp` and `market.h`, so neither file is opened or | ||
| 143 | created by this tree. A datadir carrying them from an older build is simply | ||
| 144 | ignored. | ||
| 145 | |||
| 146 | The old IP address database (`addr.dat`) has been removed. Peer discovery is | ||
| 147 | handled by Nostr, `.btf`, rendezvous relays, and direct onion peers. | ||
| 148 | |||
| 149 | ## Migration Order (shipped in v1.2.20) | ||
| 150 | |||
| 151 | 1. Storage self-tests were added around Berkeley DB behavior first. | ||
| 152 | 2. The SQLite backend landed behind an explicit flag, then behind a | ||
| 153 | `wallet-backend` marker and a GUI conversion wizard. | ||
| 154 | 3. BDB-to-SQLite migration is a one-way copy that leaves `wallet.dat` as a backup. | ||
| 155 | 4. SQLite became the default for new wallets only after cross-version restore, | ||
| 156 | encryption, backup, and recovery-audit tests passed -- and a live coin-proof | ||
| 157 | of receive, spend, restart, and encrypt on real mainnet funds. | ||
| 158 | 5. Treat `blkindex.dat` separately; it is safer to rebuild than to migrate under | ||
| 159 | pressure. | ||
diff --git a/docs/tor.md b/docs/tor.md new file mode 100644 index 0000000..0016f8b --- /dev/null +++ b/docs/tor.md | |||
| @@ -0,0 +1,231 @@ | |||
| 1 | # Tor mode | ||
| 2 | |||
| 3 | Bitflash can route outbound discovery traffic through a local Tor SOCKS5 proxy. | ||
| 4 | This is transport plumbing only: it does not change consensus, wallet keys, | ||
| 5 | mining, `.btf` addresses, or the encrypted rendezvous protocol. | ||
| 6 | |||
| 7 | ## Client mode | ||
| 8 | |||
| 9 | Start Tor locally, then start Bitflash with: | ||
| 10 | |||
| 11 | ```bash | ||
| 12 | bitflash -tor | ||
| 13 | ``` | ||
| 14 | |||
| 15 | `-tor` is shorthand for: | ||
| 16 | |||
| 17 | ```bash | ||
| 18 | bitflash -socks=127.0.0.1:9050 | ||
| 19 | ``` | ||
| 20 | |||
| 21 | If Tor listens somewhere else: | ||
| 22 | |||
| 23 | ```bash | ||
| 24 | bitflash -tor=127.0.0.1:9150 | ||
| 25 | bitflash -tor=[::1]:9050 | ||
| 26 | ``` | ||
| 27 | |||
| 28 | For a SOCKS5 proxy that is not Tor, use `-socks` directly: | ||
| 29 | |||
| 30 | ```bash | ||
| 31 | bitflash -socks=127.0.0.1:9050 | ||
| 32 | bitflash -socks=[::1]:9050 | ||
| 33 | ``` | ||
| 34 | |||
| 35 | If both `-tor` and `-socks` are present, Tor mode wins and the node prints a | ||
| 36 | warning. That keeps a command line with `-tor` from silently becoming a custom | ||
| 37 | proxy route. | ||
| 38 | |||
| 39 | ## Managed Tor mode | ||
| 40 | |||
| 41 | `-managedtor` starts a Tor process for this node, writes a local `torrc`, creates | ||
| 42 | a v3 hidden service for the normal Bitflash P2P listener, routes outbound | ||
| 43 | discovery through that Tor instance, and signs the generated `.onion:8433` | ||
| 44 | endpoint into the node's `.btf` descriptor. | ||
| 45 | |||
| 46 | ```bash | ||
| 47 | bitflash -managedtor | ||
| 48 | ``` | ||
| 49 | |||
| 50 | Without a value, Bitflash looks for Tor in this order: | ||
| 51 | |||
| 52 | 1. `tor/tor.exe` or `tor/tor` beside the Bitflash binary; | ||
| 53 | 2. `tor.exe` or `tor` beside the Bitflash binary; | ||
| 54 | 3. `tor.exe` or `tor` on `PATH`. | ||
| 55 | |||
| 56 | Windows releases can also be built as `Bitflash-*-windows-with-tor.zip`. That | ||
| 57 | package includes the official Tor Expert Bundle under `tor/`, so the first path | ||
| 58 | above exists immediately after extraction and `bitflash -managedtor` needs no | ||
| 59 | separate Tor install. | ||
| 60 | |||
| 61 | The bundled-Tor package is created by: | ||
| 62 | |||
| 63 | ```bash | ||
| 64 | make windows-tor | ||
| 65 | ``` | ||
| 66 | |||
| 67 | The packaging script verifies a pinned SHA256 for the downloaded Tor Expert | ||
| 68 | Bundle before copying it into the release directory. When `gpg` is available it | ||
| 69 | also verifies the matching `.asc` signature against the pinned Tor Browser | ||
| 70 | Developers signing-key fingerprint. Release builds should use | ||
| 71 | `TOR_VERIFY_GPG=required make windows-tor` to make that second check mandatory. | ||
| 72 | The Tor binary is bundled; the runtime data directory, hidden-service key, and | ||
| 73 | control cookie are still created under the user's Bitflash data directory. | ||
| 74 | |||
| 75 | To point at a specific Tor executable: | ||
| 76 | |||
| 77 | ```bash | ||
| 78 | bitflash -managedtor=/opt/tor/bin/tor | ||
| 79 | bitflash -managedtor=C:\Tor Browser\Browser\TorBrowser\Tor\tor.exe | ||
| 80 | ``` | ||
| 81 | |||
| 82 | Managed Tor writes its state under the Bitflash data directory: | ||
| 83 | |||
| 84 | ```text | ||
| 85 | managed-tor/ | ||
| 86 | torrc | ||
| 87 | data/ | ||
| 88 | onion-service/ | ||
| 89 | ``` | ||
| 90 | |||
| 91 | The private key for the onion address lives in `managed-tor/onion-service/`. | ||
| 92 | Back up that directory if you want the same onion address after moving the node. | ||
| 93 | Delete it only if you intentionally want a fresh onion endpoint. | ||
| 94 | |||
| 95 | `-managedtor` takes precedence over `-tor` and `-socks`, because it must choose | ||
| 96 | the local SOCKS port it starts. Diagnostics report the current managed Tor | ||
| 97 | state, including the SOCKS port and onion endpoint once Tor has generated it. | ||
| 98 | |||
| 99 | When Tor mode is enabled: | ||
| 100 | |||
| 101 | - outbound Nostr relay dials use Tor; | ||
| 102 | - outbound `.btf` rendezvous relay dials use Tor; | ||
| 103 | - outbound direct peer dials prefer signed `.onion` endpoints when a peer | ||
| 104 | advertises one; | ||
| 105 | - destination names are sent to Tor as SOCKS5 domain-name CONNECT requests, so | ||
| 106 | the local DNS resolver does not see them; | ||
| 107 | - direct `.onion` dials are refused unless a SOCKS5/Tor proxy is enabled; | ||
| 108 | - plain HTTP external-IP probes are skipped. | ||
| 109 | |||
| 110 | Tor mode affects outbound discovery and relay dials. It does not encrypt | ||
| 111 | `wallet.dat`, hide mining rewards on-chain, or change the `.btf` identity. A | ||
| 112 | relay still sees the TCP client that connected to it; with Tor mode that client | ||
| 113 | is the Tor exit or onion circuit endpoint instead of the user's direct IP. | ||
| 114 | |||
| 115 | ## What to test | ||
| 116 | |||
| 117 | Use the public status page only as a rough health check. To prove the local node | ||
| 118 | is using Tor/SOCKS, test from the node machine: | ||
| 119 | |||
| 120 | ```bash | ||
| 121 | bitflash -tor -debug | ||
| 122 | ``` | ||
| 123 | |||
| 124 | Then confirm in `debug.log` that Nostr and rendezvous dials report the proxy | ||
| 125 | path. A DNS leak test should show no local resolver lookup for relay hostnames: | ||
| 126 | the SOCKS5 request uses the domain-name form and asks the proxy to resolve it. | ||
| 127 | |||
| 128 | ## Onion rendezvous relay | ||
| 129 | |||
| 130 | An operator can expose a Bitflash rendezvous relay as a Tor hidden service. The | ||
| 131 | node does not create hidden services itself; Tor owns that configuration. | ||
| 132 | |||
| 133 | Example `torrc`: | ||
| 134 | |||
| 135 | ```text | ||
| 136 | HiddenServiceDir /var/lib/tor/bitflash-rendezvous/ | ||
| 137 | HiddenServicePort 8434 127.0.0.1:8434 | ||
| 138 | ``` | ||
| 139 | |||
| 140 | After restarting Tor, read the generated onion name: | ||
| 141 | |||
| 142 | ```bash | ||
| 143 | sudo cat /var/lib/tor/bitflash-rendezvous/hostname | ||
| 144 | ``` | ||
| 145 | |||
| 146 | Then run the relay on localhost/VPS as usual and announce the onion endpoint from | ||
| 147 | nodes that use Tor: | ||
| 148 | |||
| 149 | ```bash | ||
| 150 | bitflash -tor -rvrelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 | ||
| 151 | bitflash -tor -announcerelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 | ||
| 152 | ``` | ||
| 153 | |||
| 154 | For a pool operator that wants the pool descriptor to point at an onion | ||
| 155 | rendezvous relay, use both: | ||
| 156 | |||
| 157 | ```bash | ||
| 158 | bitflash -operator -tor \ | ||
| 159 | -rvrelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 \ | ||
| 160 | -announcerelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 | ||
| 161 | ``` | ||
| 162 | |||
| 163 | ## Manual direct full-node hidden service | ||
| 164 | |||
| 165 | Managed Tor is the preferred path for ordinary nodes. Operators can still expose | ||
| 166 | the P2P listener through a separately managed Tor service: | ||
| 167 | |||
| 168 | ```text | ||
| 169 | HiddenServiceDir /var/lib/tor/bitflash-node/ | ||
| 170 | HiddenServicePort 8433 127.0.0.1:8433 | ||
| 171 | ``` | ||
| 172 | |||
| 173 | After restarting Tor, read the generated onion name: | ||
| 174 | |||
| 175 | ```bash | ||
| 176 | sudo cat /var/lib/tor/bitflash-node/hostname | ||
| 177 | ``` | ||
| 178 | |||
| 179 | Then start Bitflash with Tor outbound enabled and advertise that hidden service: | ||
| 180 | |||
| 181 | ```bash | ||
| 182 | bitflash -tor \ | ||
| 183 | -onionservice=exampleexampleexampleexampleexampleexampleexampleexample.onion:8433 | ||
| 184 | ``` | ||
| 185 | |||
| 186 | That onion endpoint is signed into this node's self-certifying `.btf` | ||
| 187 | descriptor. New peers that resolve the descriptor and also run with `-tor` try | ||
| 188 | the `.onion:8433` P2P connection first. If it fails, they fall back to the | ||
| 189 | ordinary encrypted rendezvous path. Older nodes ignore the onion field and still | ||
| 190 | use the rendezvous relay because the descriptor keeps the legacy signature too. | ||
| 191 | |||
| 192 | If the node has `-onionservice` but has not registered at any rendezvous relay | ||
| 193 | yet, it can still publish an onion-capable descriptor with | ||
| 194 | `meeting_node = rendezvous-pending`. Tor-capable peers can dial the hidden | ||
| 195 | service directly from that descriptor. Rendezvous becomes a fallback, not a | ||
| 196 | precondition for being discoverable. | ||
| 197 | |||
| 198 | This means rendezvous relays stop being the only way to reach a node that has | ||
| 199 | published a hidden service. They remain useful bootstrap/fallback infrastructure, | ||
| 200 | but an already-discovered onion-capable peer can be reached directly over Tor. | ||
| 201 | |||
| 202 | Tor owns the private key under `HiddenServiceDir`; back it up if you want the | ||
| 203 | onion name to stay stable. | ||
| 204 | |||
| 205 | ## Release smoke test | ||
| 206 | |||
| 207 | After building a Windows binary with managed Tor support, run the two-node smoke | ||
| 208 | test before cutting a release: | ||
| 209 | |||
| 210 | ```powershell | ||
| 211 | powershell -ExecutionPolicy Bypass -File scripts\smoke-managed-tor-onion.ps1 ` | ||
| 212 | -Bitflash .\src\bitflash.exe ` | ||
| 213 | -Tor C:\path\to\tor.exe | ||
| 214 | ``` | ||
| 215 | |||
| 216 | The script creates two temporary datadirs, starts node A with `-managedtor`, | ||
| 217 | waits for it to publish its `.btf` identity and generated `.onion` endpoint, | ||
| 218 | then starts node B with its own `-managedtor`, `-oniononly`, and | ||
| 219 | `-connectbtf=<node-a>`. It passes only when node B logs that it connected to | ||
| 220 | node A `via direct onion`; rendezvous fallback cannot satisfy the test. | ||
| 221 | |||
| 222 | By default the script waits 120 seconds after node A's descriptor self-resolves | ||
| 223 | before it starts node B. Tor can create the local `hostname` before the hidden | ||
| 224 | service descriptor is visible enough for client circuits, and early dials often | ||
| 225 | fail with SOCKS5 reply `0x01`. Override this with `-OnionWarmupSec N` when | ||
| 226 | testing a slow or already-warmed Tor environment. | ||
| 227 | |||
| 228 | This test still uses Nostr for descriptor discovery, because that is how nodes | ||
| 229 | learn each other's signed `.btf` descriptors. It does not require a Bitflash | ||
| 230 | rendezvous relay to carry the P2P connection: once the descriptor includes the | ||
| 231 | signed onion endpoint, the actual node connection goes directly through Tor. | ||
diff --git a/docs/utxo-commitment.md b/docs/utxo-commitment.md new file mode 100644 index 0000000..ab43deb --- /dev/null +++ b/docs/utxo-commitment.md | |||
| @@ -0,0 +1,125 @@ | |||
| 1 | # Bitflash UTXO-set commitment | ||
| 2 | |||
| 3 | `scripts/verify-utxo-set.py` reconstructs a deterministic commitment to the | ||
| 4 | current UTXO set from local `blk*.dat` files. | ||
| 5 | |||
| 6 | This is intentionally not a consensus rule yet. It is an audit tool: independent | ||
| 7 | operators can run it against the same chain prefix and compare the resulting | ||
| 8 | root and canonical JSON hash. | ||
| 9 | |||
| 10 | ```bash | ||
| 11 | python3 scripts/verify-utxo-set.py --max-blocks 1000 --out utxo-report.json --json | ||
| 12 | ``` | ||
| 13 | |||
| 14 | Use `--max-blocks 0` or omit the option to scan every available block file: | ||
| 15 | |||
| 16 | ```bash | ||
| 17 | python3 scripts/verify-utxo-set.py --out utxo-report.json --json | ||
| 18 | ``` | ||
| 19 | |||
| 20 | ## What It Verifies | ||
| 21 | |||
| 22 | The verifier: | ||
| 23 | |||
| 24 | - parses Bitflash block files directly with the shared Python chain parser; | ||
| 25 | - streams `blk*.dat` records instead of loading full block files into memory; | ||
| 26 | - reconstructs the best chain rooted at the Bitflash genesis block; | ||
| 27 | - ignores side blocks that may also be present in `blk*.dat`; | ||
| 28 | - verifies every scanned block merkle root; | ||
| 29 | - applies spends and creates outputs in deterministic order; | ||
| 30 | - reports current UTXO count, current supply, mined coinbase total, fees, and | ||
| 31 | historical duplicate-output overwrites; | ||
| 32 | - computes a deterministic UTXO root. | ||
| 33 | |||
| 34 | ## Commitment Algorithm | ||
| 35 | |||
| 36 | The UTXO map is keyed by: | ||
| 37 | |||
| 38 | ```text | ||
| 39 | txid:vout | ||
| 40 | ``` | ||
| 41 | |||
| 42 | Keys are sorted lexicographically. Each leaf is: | ||
| 43 | |||
| 44 | ```text | ||
| 45 | SHA256("BTFUTXO1|" || outpoint || "|" || value_satoshis || "|" || | ||
| 46 | script_pub_key_hex || "|" || created_height || "|" || created_txid) | ||
| 47 | ``` | ||
| 48 | |||
| 49 | The tree is then reduced pairwise. If a level has an odd number of hashes, the | ||
| 50 | last hash is duplicated. Each parent is: | ||
| 51 | |||
| 52 | ```text | ||
| 53 | SHA256("BTFNODE1|" || left_hash || right_hash) | ||
| 54 | ``` | ||
| 55 | |||
| 56 | The final hash is the UTXO-set root. | ||
| 57 | |||
| 58 | ## Inclusion Proofs | ||
| 59 | |||
| 60 | After a root has been computed, `scripts/prove-utxo.py` can create a compact | ||
| 61 | inclusion proof for one unspent output: | ||
| 62 | |||
| 63 | ```bash | ||
| 64 | python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json | ||
| 65 | python3 scripts/verify-utxo-proof.py utxo-proof.json | ||
| 66 | ``` | ||
| 67 | |||
| 68 | The verifier does not need local block files. It recomputes the target leaf, | ||
| 69 | walks the sibling path, and compares the result with the expected root. See | ||
| 70 | [UTXO inclusion proofs](utxo-proofs.md). | ||
| 71 | |||
| 72 | ## Deterministic Output | ||
| 73 | |||
| 74 | `--json` and `--out` write canonical JSON with sorted keys, fixed separators, | ||
| 75 | ASCII escaping, and one trailing newline. It contains no local path and no | ||
| 76 | generation timestamp, so two auditors scanning the same chain prefix should | ||
| 77 | produce byte-identical reports. | ||
| 78 | |||
| 79 | ```bash | ||
| 80 | python3 scripts/verify-utxo-set.py --out utxo-report.json --json | ||
| 81 | sha256sum utxo-report.json | ||
| 82 | ``` | ||
| 83 | |||
| 84 | ## OpenTimestamps | ||
| 85 | |||
| 86 | The canonical JSON can be timestamped in Bitcoin without adding any dependency | ||
| 87 | to Bitflash consensus: | ||
| 88 | |||
| 89 | ```bash | ||
| 90 | ots stamp utxo-report.json | ||
| 91 | ots verify utxo-report.json.ots | ||
| 92 | ``` | ||
| 93 | |||
| 94 | ## Historical Duplicates | ||
| 95 | |||
| 96 | Bitflash inherited early Bitcoin behavior around duplicate transaction IDs. | ||
| 97 | The verifier reports duplicate-output overwrites as warnings and uses | ||
| 98 | last-write-wins semantics for the UTXO map, matching the legacy index model. | ||
| 99 | Those warnings are part of the report so auditors can see the historical debt | ||
| 100 | instead of relying on a polished summary. | ||
| 101 | |||
| 102 | For synthetic chains or future clean consensus ranges, auditors can make | ||
| 103 | duplicates a hard failure: | ||
| 104 | |||
| 105 | ```bash | ||
| 106 | python3 scripts/verify-utxo-set.py --strict-duplicates | ||
| 107 | ``` | ||
| 108 | |||
| 109 | The default remains compatible with the current historical chain because it has | ||
| 110 | known legacy duplicate-output overwrites. | ||
| 111 | |||
| 112 | ## Shared Parser | ||
| 113 | |||
| 114 | The audit scripts share `scripts/bitflash_chain.py` for block parsing, | ||
| 115 | main-chain selection, UTXO application, canonical JSON, and genesis | ||
| 116 | reconstruction. That keeps `verify-utxo-set.py`, `verify-fair-launch.py`, | ||
| 117 | `prove-utxo.py`, and the explorer on the same serialization rules. | ||
| 118 | |||
| 119 | ## Future Consensus Path | ||
| 120 | |||
| 121 | This tool is the low-risk first step toward consensus commitments. A future | ||
| 122 | hard-fork design can commit this root, or a Utreexo-style accumulator root, in | ||
| 123 | coinbase or block headers after a fixed activation height. Until then this | ||
| 124 | script provides reproducible supply and state commitments without changing what | ||
| 125 | nodes accept. | ||
diff --git a/docs/utxo-proofs.md b/docs/utxo-proofs.md new file mode 100644 index 0000000..4947c3c --- /dev/null +++ b/docs/utxo-proofs.md | |||
| @@ -0,0 +1,61 @@ | |||
| 1 | # Bitflash UTXO inclusion proofs | ||
| 2 | |||
| 3 | `scripts/prove-utxo.py` creates a deterministic inclusion proof for one | ||
| 4 | currently unspent output. `scripts/verify-utxo-proof.py` verifies that proof | ||
| 5 | against the UTXO root produced by `scripts/verify-utxo-set.py`. | ||
| 6 | |||
| 7 | This is intentionally outside consensus. It turns the audit root into something | ||
| 8 | useful for a specific coin without changing what nodes accept. | ||
| 9 | |||
| 10 | ## Create A Proof | ||
| 11 | |||
| 12 | Use an unspent output in `txid:vout` form: | ||
| 13 | |||
| 14 | ```bash | ||
| 15 | python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json | ||
| 16 | ``` | ||
| 17 | |||
| 18 | The proof contains: | ||
| 19 | |||
| 20 | - the target UTXO; | ||
| 21 | - its sorted leaf index; | ||
| 22 | - the leaf hash; | ||
| 23 | - the Merkle sibling path; | ||
| 24 | - the UTXO root and chain tip used to create it. | ||
| 25 | |||
| 26 | It does not contain the full UTXO set. | ||
| 27 | |||
| 28 | ## Verify A Proof | ||
| 29 | |||
| 30 | Verify against the root embedded in the proof: | ||
| 31 | |||
| 32 | ```bash | ||
| 33 | python3 scripts/verify-utxo-proof.py utxo-proof.json | ||
| 34 | ``` | ||
| 35 | |||
| 36 | Or verify against a root copied from another machine, dashboard, block explorer, | ||
| 37 | or future node export: | ||
| 38 | |||
| 39 | ```bash | ||
| 40 | python3 scripts/verify-utxo-proof.py utxo-proof.json \ | ||
| 41 | --root ROOT_FROM_REPORT_OR_EXPORT | ||
| 42 | ``` | ||
| 43 | |||
| 44 | The verifier does not need local `blk*.dat` files. It only needs the proof and | ||
| 45 | the expected root. | ||
| 46 | |||
| 47 | ## Scope | ||
| 48 | |||
| 49 | This first proof format supports inclusion only. Absence and range proofs are | ||
| 50 | left for a later design because they require a more careful commitment scheme | ||
| 51 | and are easier to get subtly wrong. | ||
| 52 | |||
| 53 | The proof uses the exact same leaf and node domains as the UTXO commitment: | ||
| 54 | |||
| 55 | ```text | ||
| 56 | BTFUTXO1 | ||
| 57 | BTFNODE1 | ||
| 58 | ``` | ||
| 59 | |||
| 60 | That keeps the verifier language-neutral and gives future implementations a | ||
| 61 | small, stable target. | ||