From 51d617e70f90a972185917ea0096c1ecef2bfa8b Mon Sep 17 00:00:00 2001 From: Bitflash-sh Date: Fri, 24 Jul 2026 11:18:11 -0300 Subject: Bitflash: fair-launch release Satoshi's bitcoin-0.1.0 reborn with RandomX CPU proof of work, anonymous .btf addressing over Nostr with CGNAT rendezvous, automatic peer discovery, a headless Linux node, and Windows GUI wallet. Fair launch, no premine, 21M cap. --- docs/logo.png | Bin 0 -> 14622 bytes 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 docs/logo.png (limited to 'docs') diff --git a/docs/logo.png b/docs/logo.png new file mode 100644 index 0000000..0f5d0f5 Binary files /dev/null and b/docs/logo.png differ -- cgit v1.2.3 From a150aeba3a9753317fa8485804230dc73ee32d0d Mon Sep 17 00:00:00 2001 From: Bitflash-sh Date: Fri, 24 Jul 2026 15:07:24 -0300 Subject: README: add live wallet + rendezvous/mining screenshots --- README.md | 14 ++++++++++++++ docs/discovery.png | Bin 0 -> 36535 bytes docs/wallet.png | Bin 0 -> 239723 bytes 3 files changed, 14 insertions(+) create mode 100644 docs/discovery.png create mode 100644 docs/wallet.png (limited to 'docs') diff --git a/README.md b/README.md index 33eab2b..38e8f5c 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,12 @@ that hides your IP and punches through CGNAT, and a **fair launch** with no prem **[โฌ‡๏ธ Download for Windows](../../releases/latest)**  ยท  **[๐Ÿง Build for Linux](#-linux-node--miner)**  ยท  **[๐Ÿง  How it works](#-how-it-works)** +
+ +Bitflash wallet mining with RandomX + +The Bitflash wallet โ€” mining with RandomX, connected to the network, coins accruing. No pools, no ASICs, no gatekeepers. + --- @@ -131,6 +137,14 @@ The relay is dumb on purpose: it pairs peers by their `.btf` key and forwards by real connection is end-to-end encrypted (X25519 + XChaCha20-Poly1305) and authenticated to each peer's static key, so a malicious relay can't read or tamper โ€” it can only refuse. +
+ +Bitflash node connecting by .btf through the rendezvous and mining RandomX + +A node tunneling to a peer by its .btf address through the rendezvous, and finding a RandomX proof of work โ€” automatic, no configuration, no IP exposed. + +
+ ### RandomX proof of work Blocks are mined with **RandomX**, a memory-hard algorithm that runs efficiently on diff --git a/docs/discovery.png b/docs/discovery.png new file mode 100644 index 0000000..8dda52c Binary files /dev/null and b/docs/discovery.png differ diff --git a/docs/wallet.png b/docs/wallet.png new file mode 100644 index 0000000..addf21e Binary files /dev/null and b/docs/wallet.png differ -- cgit v1.2.3 From 56df9830d5a002ff9ce3fac47e497673f8159e39 Mon Sep 17 00:00:00 2001 From: Bitflash-sh Date: Fri, 24 Jul 2026 16:14:29 -0300 Subject: Add social banner; use it as README hero --- README.md | 2 +- docs/social.png | Bin 0 -> 208957 bytes 2 files changed, 1 insertion(+), 1 deletion(-) create mode 100644 docs/social.png (limited to 'docs') diff --git a/README.md b/README.md index 38e8f5c..5df2a16 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@
-Bitflash +Bitflash โ€” Satoshi's Bitcoin reborn, mined with your CPU # Bitflash โšก  `BTF` diff --git a/docs/social.png b/docs/social.png new file mode 100644 index 0000000..19e2a3b Binary files /dev/null and b/docs/social.png differ -- cgit v1.2.3 From f6c5b813075cae4228d241a3567e70b210504749 Mon Sep 17 00:00:00 2001 From: mtg0001 <106211915+mtg0001@users.noreply.github.com> Date: Tue, 4 Aug 2026 10:18:01 -0300 Subject: Add release verification tooling (#108) Co-authored-by: Gilcimar Duarte --- .gitignore | 1 + Makefile | 14 ++++- README.md | 19 ++++++- docs/release-verification.md | 62 ++++++++++++++++++++ scripts/make-release-checksums.sh | 78 ++++++++++++++++++++++++++ scripts/verify-release.sh | 115 ++++++++++++++++++++++++++++++++++++++ 6 files changed, 287 insertions(+), 2 deletions(-) create mode 100644 docs/release-verification.md create mode 100755 scripts/make-release-checksums.sh create mode 100755 scripts/verify-release.sh (limited to 'docs') diff --git a/.gitignore b/.gitignore index 1ec9099..111f7ba 100644 --- a/.gitignore +++ b/.gitignore @@ -13,6 +13,7 @@ Bitflash-*-windows.zip Bitflash-*.AppImage bitflash-node-*-x86_64 SHA256SUMS +SHA256SUMS.asc # Runtime data / logs debug.log diff --git a/Makefile b/Makefile index a9630ed..ea720f1 100644 --- a/Makefile +++ b/Makefile @@ -3,6 +3,8 @@ # make linux build Bitflash-*-x86_64.AppImage # make windows build Bitflash-*-windows.zip (from MSYS2 UCRT64) # make tests build and run standalone unit tests +# make checksums create SHA256SUMS for release assets +# make verify-release TAG=v1.2.13 # make clean remove build artifacts ROOT := $(shell pwd) @@ -160,5 +162,15 @@ clean: tests: $(MAKE) -C src -f Makefile tests +checksums: + ./scripts/make-release-checksums.sh + +sign-checksums: + ./scripts/make-release-checksums.sh --sign $(if $(KEY),--local-user $(KEY),) + +verify-release: + ./scripts/verify-release.sh $(if $(TAG),$(TAG),latest) + .PHONY: linux windows clean appimage \ - tests deps-linux deps-windows deps-apt deps-secp256k1 deps-randomx + tests checksums sign-checksums verify-release \ + deps-linux deps-windows deps-apt deps-secp256k1 deps-randomx diff --git a/README.md b/README.md index 6601384..6446b3d 100644 --- a/README.md +++ b/README.md @@ -49,13 +49,30 @@ configuration โ€” it connects automatically and starts syncing. **Windows:** extract the `-windows.zip` and run `Bitflash.exe`. -Every release ships a `SHA256SUMS` covering both assets. Verifying takes a second +Every release ships a `SHA256SUMS` covering the assets. Verifying takes a second and is worth doing: ```bash sha256sum -c SHA256SUMS ``` +For newer signed releases, verify the checksum file itself first: + +```bash +gpg --verify SHA256SUMS.asc SHA256SUMS +sha256sum -c SHA256SUMS +``` + +The helper below downloads the release assets, verifies `SHA256SUMS.asc` when it +is present, then checks the hashes: + +```bash +scripts/verify-release.sh latest +``` + +See [release verification](docs/release-verification.md) for the full release +audit flow and the maintainer signing step. + **Keep your node current.** Consensus rules have changed since the first releases โ€” 1.2.1 fixed a bug that let anyone spend anyone's coins, and 1.2.2 added a per-block signature-operation cap. A node on an older build will accept diff --git a/docs/release-verification.md b/docs/release-verification.md new file mode 100644 index 0000000..ecacd5f --- /dev/null +++ b/docs/release-verification.md @@ -0,0 +1,62 @@ +# Release Verification + +Bitflash releases publish binaries and a `SHA256SUMS` file. The checksum file +proves that the file you downloaded matches the file the release page names. +A detached signature, `SHA256SUMS.asc`, proves that the checksum file itself was +signed by a trusted release key. + +That gives users two separate checks: + +1. `gpg --verify SHA256SUMS.asc SHA256SUMS` checks who signed the checksums. +2. `sha256sum -c SHA256SUMS` checks the binaries against those checksums. + +## User Check + +From a shell with `curl` or `wget` and `sha256sum`: + +```bash +scripts/verify-release.sh latest +``` + +For a specific release: + +```bash +scripts/verify-release.sh v1.2.13 +``` + +For release audits, require the signature: + +```bash +scripts/verify-release.sh v1.2.13 --require-signature +``` + +Older releases may not have `SHA256SUMS.asc`. In that case the script warns and +still checks file integrity. New release audits should use `--require-signature`. + +## Maintainer Flow + +After building release assets in the repository root: + +```bash +scripts/make-release-checksums.sh --sign --local-user RELEASE_KEY_ID +``` + +Upload all built assets plus: + +```text +SHA256SUMS +SHA256SUMS.asc +``` + +Keep the private signing key offline or on a dedicated release machine. Publish +the public key fingerprint in the release notes and keep using the same key for +future releases unless there is a clearly announced rotation. + +## Why This Matters + +`SHA256SUMS` alone protects against a broken download, but not against someone +replacing both a binary and the checksum file. Signing `SHA256SUMS` means an +attacker must also have the release signing key to make the replacement verify. + +This is not reproducible builds yet. It is the smaller, immediate step that +makes every release asset auditable by users before they run it. diff --git a/scripts/make-release-checksums.sh b/scripts/make-release-checksums.sh new file mode 100755 index 0000000..5b29e2d --- /dev/null +++ b/scripts/make-release-checksums.sh @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +set -euo pipefail + +sign=0 +key="" + +usage() { + cat <<'EOF' +Usage: scripts/make-release-checksums.sh [--sign] [--local-user KEYID] + +Creates SHA256SUMS for release assets in the current directory: + Bitflash-*-windows.zip + Bitflash-*-x86_64.AppImage + bitflash-node-*-x86_64 + +With --sign, also creates SHA256SUMS.asc as a detached ASCII-armored GPG +signature. Upload both files with the release assets. +EOF +} + +while [ "$#" -gt 0 ]; do + case "$1" in + --sign) sign=1 ;; + --local-user) + [ "$#" -ge 2 ] || { echo "--local-user needs a key id" >&2; exit 2; } + key="$2" + shift + ;; + -h|--help) usage; exit 0 ;; + *) echo "unknown option: $1" >&2; usage >&2; exit 2 ;; + esac + shift +done + +if command -v sha256sum >/dev/null 2>&1; then + sha_cmd=(sha256sum) +elif command -v shasum >/dev/null 2>&1; then + sha_cmd=(shasum -a 256) +else + echo "missing required command: sha256sum or shasum" >&2 + exit 1 +fi + +assets=() +for pattern in \ + "Bitflash-*-windows.zip" \ + "Bitflash-*-x86_64.AppImage" \ + "bitflash-node-*-x86_64" +do + for file in $pattern; do + [ -e "$file" ] || continue + assets+=("$file") + done +done + +if [ "${#assets[@]}" -eq 0 ]; then + echo "no release assets found in $(pwd)" >&2 + exit 1 +fi + +printf '%s\n' "${assets[@]}" | LC_ALL=C sort | while IFS= read -r file; do + "${sha_cmd[@]}" "$file" +done > SHA256SUMS + +echo "Wrote SHA256SUMS" + +if [ "$sign" -eq 1 ]; then + if ! command -v gpg >/dev/null 2>&1; then + echo "missing required command: gpg" >&2 + exit 1 + fi + gpg_args=(--armor --detach-sign --output SHA256SUMS.asc) + if [ -n "$key" ]; then + gpg_args+=(--local-user "$key") + fi + gpg "${gpg_args[@]}" SHA256SUMS + echo "Wrote SHA256SUMS.asc" +fi diff --git a/scripts/verify-release.sh b/scripts/verify-release.sh new file mode 100755 index 0000000..23e71d1 --- /dev/null +++ b/scripts/verify-release.sh @@ -0,0 +1,115 @@ +#!/usr/bin/env bash +set -euo pipefail + +repo="Bitflash-sh/bitflash" +tag="${1:-latest}" +require_signature=0 + +usage() { + cat <<'EOF' +Usage: scripts/verify-release.sh [tag|latest] [--require-signature] + +Downloads the release checksum file and every asset named in it, then verifies: + 1. SHA256SUMS.asc, if present, against SHA256SUMS with gpg. + 2. SHA256SUMS against the downloaded assets. + +Use --require-signature for release audits. Without it, older releases that +ship only SHA256SUMS are checked for integrity and reported as unsigned. +EOF +} + +for arg in "${@:2}"; do + case "$arg" in + --require-signature) require_signature=1 ;; + -h|--help) usage; exit 0 ;; + *) echo "unknown option: $arg" >&2; usage >&2; exit 2 ;; + esac +done + +if [ "$tag" = "-h" ] || [ "$tag" = "--help" ]; then + usage + exit 0 +fi + +need() { + if ! command -v "$1" >/dev/null 2>&1; then + echo "missing required command: $1" >&2 + exit 1 + fi +} + +need sed +need awk + +if command -v sha256sum >/dev/null 2>&1; then + sha_check=(sha256sum -c SHA256SUMS) +elif command -v shasum >/dev/null 2>&1; then + sha_check=(shasum -a 256 -c SHA256SUMS) +else + echo "missing required command: sha256sum or shasum" >&2 + exit 1 +fi + +download() { + local url="$1" + local out="$2" + if command -v curl >/dev/null 2>&1; then + curl -fsSL "$url" -o "$out" + elif command -v wget >/dev/null 2>&1; then + wget -q "$url" -O "$out" + else + echo "missing required command: curl or wget" >&2 + exit 1 + fi +} + +if [ "$tag" = "latest" ]; then + tag="$(download "https://api.github.com/repos/$repo/releases/latest" - | + sed -n 's/.*"tag_name"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' | + head -1)" + if [ -z "$tag" ]; then + echo "could not resolve latest release tag" >&2 + exit 1 + fi +fi + +workdir="$(mktemp -d "${TMPDIR:-/tmp}/bitflash-release-verify.XXXXXX")" +cleanup() { rm -rf "$workdir"; } +trap cleanup EXIT + +base_url="https://github.com/$repo/releases/download/$tag" + +echo "Verifying Bitflash $tag" +echo "Working directory: $workdir" +cd "$workdir" + +download "$base_url/SHA256SUMS" SHA256SUMS + +if download "$base_url/SHA256SUMS.asc" SHA256SUMS.asc 2>/dev/null; then + if ! command -v gpg >/dev/null 2>&1; then + echo "SHA256SUMS.asc exists, but gpg is not installed" >&2 + exit 1 + fi + gpg --verify SHA256SUMS.asc SHA256SUMS + echo "Signature OK" +else + if [ "$require_signature" -eq 1 ]; then + echo "release does not publish SHA256SUMS.asc" >&2 + exit 1 + fi + echo "WARNING: no SHA256SUMS.asc found; checking hashes only" +fi + +awk '{print $2}' SHA256SUMS | while IFS= read -r file; do + file="${file#\*}" + file="${file#./}" + [ -z "$file" ] && continue + case "$file" in + */*|..*) echo "refusing unexpected checksum path: $file" >&2; exit 1 ;; + esac + echo "Downloading $file" + download "$base_url/$file" "$file" +done + +"${sha_check[@]}" +echo "Release assets OK" -- cgit v1.2.3 From a2592156207ed04671d6532cec4997fb1dab1595 Mon Sep 17 00:00:00 2001 From: Bitflash Date: Tue, 4 Aug 2026 10:19:38 -0300 Subject: docs: one documentation directory, not two (#137) docs/ has held the README's images since the fair-launch release. I added doc/ yesterday for the derivation spec without looking, and #108 then correctly used docs/ -- leaving the repository with both, one letter apart, which is the kind of thing that quietly costs someone ten minutes a year forever. The mistake was mine, so the move is mine: derivation.md and verify-derivation.py go to docs/, and the two README links follow. No content changes. The verifier still reproduces the wallet from its new path, which is the only thing here that can actually break. --- README.md | 2 +- doc/derivation.md | 125 ------------------------------------ doc/verify-derivation.py | 159 ---------------------------------------------- docs/derivation.md | 125 ++++++++++++++++++++++++++++++++++++ docs/verify-derivation.py | 159 ++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 285 insertions(+), 285 deletions(-) delete mode 100644 doc/derivation.md delete mode 100644 doc/verify-derivation.py create mode 100644 docs/derivation.md create mode 100644 docs/verify-derivation.py (limited to 'docs') diff --git a/README.md b/README.md index 6446b3d..6746e02 100644 --- a/README.md +++ b/README.md @@ -107,7 +107,7 @@ words also cover the address the window shows you. **Both still matter** โ€” the phrase and the file back up different things, and the difference is where people lose money. -The derivation is written down in [doc/derivation.md](doc/derivation.md), with +The derivation is written down in [docs/derivation.md](docs/derivation.md), with test vectors and a script that reproduces them from scratch. It is there so the twelve words keep working even if this software does not: paths, address format and encoding, enough to recover the keys with ordinary tools and no Bitflash diff --git a/doc/derivation.md b/doc/derivation.md deleted file mode 100644 index 3ca8339..0000000 --- a/doc/derivation.md +++ /dev/null @@ -1,125 +0,0 @@ -# Key derivation and address format - -This document exists so that a Bitflash wallet can be rebuilt **without this -software**. If the project stops, the binaries are lost, or you simply do not -trust them, the twelve words plus what is written here are enough to recover -your private keys with ordinary cryptographic tools. - -Everything below is verified by `doc/verify-derivation.py`, a self-contained -script that implements this document from scratch โ€” no Bitflash code, no -libraries beyond Python's standard library โ€” and checks that it reproduces the -addresses the wallet produces. If the two ever disagree, this document is wrong -and the script will say so. - -## Addresses - -An address is the Base58Check encoding of a version byte followed by -`RIPEMD160(SHA256(public key))`. - -| | | -|---|---| -| curve | secp256k1 | -| public key encoding | **uncompressed**, 65 bytes, `0x04 โ€– X โ€– Y` | -| address version byte | **25** (`0x19`), which is why addresses begin with `B` | -| checksum | first 4 bytes of `SHA256(SHA256(payload))`, appended before Base58 | - -The public key is uncompressed. This matters more than anything else here: the -same private key hashed in compressed form gives a completely different, valid -looking address that holds nothing. Bitflash descends from Bitcoin 0.1.0, which -predates compressed keys. - -Outputs paying an address use the usual script: - -``` -OP_DUP OP_HASH160 OP_EQUALVERIFY OP_CHECKSIG -``` - -Mined coins and payments made directly to a public key use the older form -` OP_CHECKSIG`. A wallet scanning the chain must recognise both. - -## From twelve words to a master key - -Standard BIP-39 and BIP-32, with no modifications: - -1. **Seed.** `PBKDF2-HMAC-SHA512(password = mnemonic, salt = "mnemonic" + passphrase, iterations = 2048, length = 64 bytes)`. Bitflash always uses an empty passphrase, so the salt is the literal string `mnemonic`. -2. **Master key.** `I = HMAC-SHA512(key = "Bitcoin seed", data = seed)`. The master private key is `I[0..32]`, the master chain code is `I[32..64]`. - -Child derivation is BIP-32 `CKDpriv`, hardened and non-hardened as usual. - -## Two layouts - -A wallet records which layout it uses in `wallet.dat` under the key `hdschema`. -Wallets created before that field existed are treated as legacy. - -### Legacy (`hdschema` absent or 1) - -``` -m/i' -``` - -Hardened children of the master key, one flat sequence. Index 0 is the wallet's -default receiving address; the key pool and change take the following indices. - -Used by every wallet created before BIP-44 support existed, and kept forever: -those phrases must keep working. - -### BIP-44 (`hdschema` = 2) - -``` -m/44'/4346950'/0'/0/i external chain (receiving) -m/44'/4346950'/0'/1/i internal chain (change) -``` - -Coin type **4346950** is `0x425446`, the ASCII bytes `BTF`. It is -[registered for Bitflash under the symbol BITFLASH](https://github.com/satoshilabs/slips/blob/master/slip-0044.md). -The account level is fixed at `0'`; Bitflash does not expose multiple accounts. - -A wallet restored from a phrase scans **both** layouts, because the words alone -do not say which one produced them. - -## Test vectors - -Using the public BIP-39 test mnemonic: - -``` -abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about -``` - -### Legacy - -| path | address | -|---|---| -| `m/0'` | `BHJNmVvkVv59fgn7gmgP78BtciQi2HY8Hy` | -| `m/1'` | `B5qh9opG8sH8Mv9k2xzHoTeSUd9BkT6XEZ` | -| `m/2'` | `BCNH8uGyvkiz3uXjhA9iiF27PWAaTStNin` | -| `m/3'` | `BC1QNaXuTPGDgLZy53PghZMHEmZcp5GLgD` | - -### BIP-44 external chain - -| path | address | -|---|---| -| `m/44'/4346950'/0'/0/0` | `BF1ukigB1afmqxH64gJymPwUqFBrRqWbrb` | -| `m/44'/4346950'/0'/0/1` | `BACY8pcuCapNQr897eRHroNrgN38GSsEzV` | -| `m/44'/4346950'/0'/0/2` | `BMzkMN2k1phNBqy8RPb4Pv1i2QpwsWkKPa` | -| `m/44'/4346950'/0'/0/3` | `BNPZ8U6FHXr9Ererk8psyM2RqEVtF6sfBQ` | - -Check them yourself: - -``` -python doc/verify-derivation.py -``` - -Both tables were produced by the wallet โ€” the legacy one by the v1.2.15 binary, -which only knows that layout โ€” and then reproduced independently by the script. - -## What this document does not give you - -Deriving the keys is not the same as spending. To move recovered coins you still -need a node that knows the network: the chain, the peer protocol, and the -transaction format. This document is about **not losing the keys**, which is the -part that cannot be reconstructed later by reading source code that no longer -exists anywhere. - -Note also that `wallet.dat` stores private keys unencrypted. A recovery phrase -protects you against losing the file; it does not protect the file. Anyone who -can read it can spend the coins. diff --git a/doc/verify-derivation.py b/doc/verify-derivation.py deleted file mode 100644 index ba8e883..0000000 --- a/doc/verify-derivation.py +++ /dev/null @@ -1,159 +0,0 @@ -#!/usr/bin/env python3 -"""Check doc/derivation.md against the addresses the wallet really produces. - -This implements the document and nothing else: BIP-39 to seed, BIP-32 to child -keys, uncompressed secp256k1 public keys, and Base58Check with version byte 25. -It shares no code with Bitflash and needs nothing outside the standard library, -so agreement between the two is evidence that the document is enough to recover -a wallet without this project's software. - - python doc/verify-derivation.py - -Exits non-zero if any address disagrees. -""" - -import hashlib -import hmac -import sys - -MNEMONIC = ("abandon abandon abandon abandon abandon abandon " - "abandon abandon abandon abandon abandon about") -COIN_TYPE = 4346950 # 0x425446, the ASCII bytes "BTF" -ADDRESS_VERSION = 25 # 0x19, so addresses start with "B" -HARDENED = 0x80000000 - -LEGACY = [ - "BHJNmVvkVv59fgn7gmgP78BtciQi2HY8Hy", - "B5qh9opG8sH8Mv9k2xzHoTeSUd9BkT6XEZ", - "BCNH8uGyvkiz3uXjhA9iiF27PWAaTStNin", - "BC1QNaXuTPGDgLZy53PghZMHEmZcp5GLgD", -] - -BIP44_EXTERNAL = [ - "BF1ukigB1afmqxH64gJymPwUqFBrRqWbrb", - "BACY8pcuCapNQr897eRHroNrgN38GSsEzV", - "BMzkMN2k1phNBqy8RPb4Pv1i2QpwsWkKPa", - "BNPZ8U6FHXr9Ererk8psyM2RqEVtF6sfBQ", -] - -# ---- secp256k1, plain and slow on purpose: it should be readable ------------ - -P = 2**256 - 2**32 - 977 -N = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141 -G = (0x79BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798, - 0x483ADA7726A3C4655DA4FBFC0E1108A8FD17B448A68554199C47D08FFB10D4B8) - - -def point_add(p, q): - if p is None: - return q - if q is None: - return p - if p[0] == q[0] and (p[1] + q[1]) % P == 0: - return None - if p == q: - lam = 3 * p[0] * p[0] * pow(2 * p[1], P - 2, P) % P - else: - lam = (q[1] - p[1]) * pow(q[0] - p[0], P - 2, P) % P - x = (lam * lam - p[0] - q[0]) % P - return (x, (lam * (p[0] - x) - p[1]) % P) - - -def point_mul(k, p=G): - result = None - while k: - if k & 1: - result = point_add(result, p) - p = point_add(p, p) - k >>= 1 - return result - - -def pub_uncompressed(point): - return b"\x04" + point[0].to_bytes(32, "big") + point[1].to_bytes(32, "big") - - -def pub_compressed(point): - return bytes([2 + (point[1] & 1)]) + point[0].to_bytes(32, "big") - - -# ---- encoding -------------------------------------------------------------- - -def base58check(payload): - alphabet = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz" - checksum = hashlib.sha256(hashlib.sha256(payload).digest()).digest()[:4] - data = payload + checksum - number = int.from_bytes(data, "big") - out = "" - while number: - number, rem = divmod(number, 58) - out = alphabet[rem] + out - leading_zeros = len(data) - len(data.lstrip(b"\x00")) - return "1" * leading_zeros + out - - -def address(privkey): - pub = pub_uncompressed(point_mul(privkey)) - h160 = hashlib.new("ripemd160", hashlib.sha256(pub).digest()).digest() - return base58check(bytes([ADDRESS_VERSION]) + h160) - - -# ---- BIP-39 and BIP-32 ----------------------------------------------------- - -def master_from_mnemonic(mnemonic, passphrase=""): - seed = hashlib.pbkdf2_hmac("sha512", mnemonic.encode("utf-8"), - ("mnemonic" + passphrase).encode("utf-8"), - 2048, 64) - i = hmac.new(b"Bitcoin seed", seed, hashlib.sha512).digest() - return int.from_bytes(i[:32], "big"), i[32:] - - -def ckd_priv(key, chain, index): - if index >= HARDENED: - data = b"\x00" + key.to_bytes(32, "big") - else: - data = pub_compressed(point_mul(key)) - i = hmac.new(chain, data + index.to_bytes(4, "big"), hashlib.sha512).digest() - return (int.from_bytes(i[:32], "big") + key) % N, i[32:] - - -def derive(path): - key, chain = master_from_mnemonic(MNEMONIC) - for index in path: - key, chain = ckd_priv(key, chain, index) - return key - - -# ---- the check ------------------------------------------------------------- - -def main(): - failures = 0 - - print("legacy m/i'") - for i, expected in enumerate(LEGACY): - got = address(derive([i + HARDENED])) - ok = got == expected - failures += 0 if ok else 1 - print(" m/%d' %s %s" % (i, got, "ok" if ok else - "MISMATCH, wallet says " + expected)) - - print() - print("bip-44 m/44'/%d'/0'/0/i" % COIN_TYPE) - for i, expected in enumerate(BIP44_EXTERNAL): - got = address(derive([44 + HARDENED, COIN_TYPE + HARDENED, - 0 + HARDENED, 0, i])) - ok = got == expected - failures += 0 if ok else 1 - print(" .../%d %s %s" % (i, got, "ok" if ok else - "MISMATCH, wallet says " + expected)) - - print() - if failures: - print("%d address(es) disagree -- doc/derivation.md is wrong" % failures) - return 1 - print("all addresses match: the document reproduces the wallet") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/docs/derivation.md b/docs/derivation.md new file mode 100644 index 0000000..c5ee3ad --- /dev/null +++ b/docs/derivation.md @@ -0,0 +1,125 @@ +# Key derivation and address format + +This document exists so that a Bitflash wallet can be rebuilt **without this +software**. If the project stops, the binaries are lost, or you simply do not +trust them, the twelve words plus what is written here are enough to recover +your private keys with ordinary cryptographic tools. + +Everything below is verified by `docs/verify-derivation.py`, a self-contained +script that implements this document from scratch โ€” no Bitflash code, no +libraries beyond Python's standard library โ€” and checks that it reproduces the +addresses the wallet produces. If the two ever disagree, this document is wrong +and the script will say so. + +## Addresses + +An address is the Base58Check encoding of a version byte followed by +`RIPEMD160(SHA256(public key))`. + +| | | +|---|---| +| curve | secp256k1 | +| public key encoding | **uncompressed**, 65 bytes, `0x04 โ€– X โ€– Y` | +| address version byte | **25** (`0x19`), which is why addresses begin with `B` | +| checksum | first 4 bytes of `SHA256(SHA256(payload))`, appended before Base58 | + +The public key is uncompressed. This matters more than anything else here: the +same private key hashed in compressed form gives a completely different, valid +looking address that holds nothing. Bitflash descends from Bitcoin 0.1.0, which +predates compressed keys. + +Outputs paying an address use the usual script: + +``` +OP_DUP OP_HASH160 OP_EQUALVERIFY OP_CHECKSIG +``` + +Mined coins and payments made directly to a public key use the older form +` OP_CHECKSIG`. A wallet scanning the chain must recognise both. + +## From twelve words to a master key + +Standard BIP-39 and BIP-32, with no modifications: + +1. **Seed.** `PBKDF2-HMAC-SHA512(password = mnemonic, salt = "mnemonic" + passphrase, iterations = 2048, length = 64 bytes)`. Bitflash always uses an empty passphrase, so the salt is the literal string `mnemonic`. +2. **Master key.** `I = HMAC-SHA512(key = "Bitcoin seed", data = seed)`. The master private key is `I[0..32]`, the master chain code is `I[32..64]`. + +Child derivation is BIP-32 `CKDpriv`, hardened and non-hardened as usual. + +## Two layouts + +A wallet records which layout it uses in `wallet.dat` under the key `hdschema`. +Wallets created before that field existed are treated as legacy. + +### Legacy (`hdschema` absent or 1) + +``` +m/i' +``` + +Hardened children of the master key, one flat sequence. Index 0 is the wallet's +default receiving address; the key pool and change take the following indices. + +Used by every wallet created before BIP-44 support existed, and kept forever: +those phrases must keep working. + +### BIP-44 (`hdschema` = 2) + +``` +m/44'/4346950'/0'/0/i external chain (receiving) +m/44'/4346950'/0'/1/i internal chain (change) +``` + +Coin type **4346950** is `0x425446`, the ASCII bytes `BTF`. It is +[registered for Bitflash under the symbol BITFLASH](https://github.com/satoshilabs/slips/blob/master/slip-0044.md). +The account level is fixed at `0'`; Bitflash does not expose multiple accounts. + +A wallet restored from a phrase scans **both** layouts, because the words alone +do not say which one produced them. + +## Test vectors + +Using the public BIP-39 test mnemonic: + +``` +abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about +``` + +### Legacy + +| path | address | +|---|---| +| `m/0'` | `BHJNmVvkVv59fgn7gmgP78BtciQi2HY8Hy` | +| `m/1'` | `B5qh9opG8sH8Mv9k2xzHoTeSUd9BkT6XEZ` | +| `m/2'` | `BCNH8uGyvkiz3uXjhA9iiF27PWAaTStNin` | +| `m/3'` | `BC1QNaXuTPGDgLZy53PghZMHEmZcp5GLgD` | + +### BIP-44 external chain + +| path | address | +|---|---| +| `m/44'/4346950'/0'/0/0` | `BF1ukigB1afmqxH64gJymPwUqFBrRqWbrb` | +| `m/44'/4346950'/0'/0/1` | `BACY8pcuCapNQr897eRHroNrgN38GSsEzV` | +| `m/44'/4346950'/0'/0/2` | `BMzkMN2k1phNBqy8RPb4Pv1i2QpwsWkKPa` | +| `m/44'/4346950'/0'/0/3` | `BNPZ8U6FHXr9Ererk8psyM2RqEVtF6sfBQ` | + +Check them yourself: + +``` +python docs/verify-derivation.py +``` + +Both tables were produced by the wallet โ€” the legacy one by the v1.2.15 binary, +which only knows that layout โ€” and then reproduced independently by the script. + +## What this document does not give you + +Deriving the keys is not the same as spending. To move recovered coins you still +need a node that knows the network: the chain, the peer protocol, and the +transaction format. This document is about **not losing the keys**, which is the +part that cannot be reconstructed later by reading source code that no longer +exists anywhere. + +Note also that `wallet.dat` stores private keys unencrypted. A recovery phrase +protects you against losing the file; it does not protect the file. Anyone who +can read it can spend the coins. diff --git a/docs/verify-derivation.py b/docs/verify-derivation.py new file mode 100644 index 0000000..ba8e883 --- /dev/null +++ b/docs/verify-derivation.py @@ -0,0 +1,159 @@ +#!/usr/bin/env python3 +"""Check doc/derivation.md against the addresses the wallet really produces. + +This implements the document and nothing else: BIP-39 to seed, BIP-32 to child +keys, uncompressed secp256k1 public keys, and Base58Check with version byte 25. +It shares no code with Bitflash and needs nothing outside the standard library, +so agreement between the two is evidence that the document is enough to recover +a wallet without this project's software. + + python doc/verify-derivation.py + +Exits non-zero if any address disagrees. +""" + +import hashlib +import hmac +import sys + +MNEMONIC = ("abandon abandon abandon abandon abandon abandon " + "abandon abandon abandon abandon abandon about") +COIN_TYPE = 4346950 # 0x425446, the ASCII bytes "BTF" +ADDRESS_VERSION = 25 # 0x19, so addresses start with "B" +HARDENED = 0x80000000 + +LEGACY = [ + "BHJNmVvkVv59fgn7gmgP78BtciQi2HY8Hy", + "B5qh9opG8sH8Mv9k2xzHoTeSUd9BkT6XEZ", + "BCNH8uGyvkiz3uXjhA9iiF27PWAaTStNin", + "BC1QNaXuTPGDgLZy53PghZMHEmZcp5GLgD", +] + +BIP44_EXTERNAL = [ + "BF1ukigB1afmqxH64gJymPwUqFBrRqWbrb", + "BACY8pcuCapNQr897eRHroNrgN38GSsEzV", + "BMzkMN2k1phNBqy8RPb4Pv1i2QpwsWkKPa", + "BNPZ8U6FHXr9Ererk8psyM2RqEVtF6sfBQ", +] + +# ---- secp256k1, plain and slow on purpose: it should be readable ------------ + +P = 2**256 - 2**32 - 977 +N = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141 +G = (0x79BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798, + 0x483ADA7726A3C4655DA4FBFC0E1108A8FD17B448A68554199C47D08FFB10D4B8) + + +def point_add(p, q): + if p is None: + return q + if q is None: + return p + if p[0] == q[0] and (p[1] + q[1]) % P == 0: + return None + if p == q: + lam = 3 * p[0] * p[0] * pow(2 * p[1], P - 2, P) % P + else: + lam = (q[1] - p[1]) * pow(q[0] - p[0], P - 2, P) % P + x = (lam * lam - p[0] - q[0]) % P + return (x, (lam * (p[0] - x) - p[1]) % P) + + +def point_mul(k, p=G): + result = None + while k: + if k & 1: + result = point_add(result, p) + p = point_add(p, p) + k >>= 1 + return result + + +def pub_uncompressed(point): + return b"\x04" + point[0].to_bytes(32, "big") + point[1].to_bytes(32, "big") + + +def pub_compressed(point): + return bytes([2 + (point[1] & 1)]) + point[0].to_bytes(32, "big") + + +# ---- encoding -------------------------------------------------------------- + +def base58check(payload): + alphabet = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz" + checksum = hashlib.sha256(hashlib.sha256(payload).digest()).digest()[:4] + data = payload + checksum + number = int.from_bytes(data, "big") + out = "" + while number: + number, rem = divmod(number, 58) + out = alphabet[rem] + out + leading_zeros = len(data) - len(data.lstrip(b"\x00")) + return "1" * leading_zeros + out + + +def address(privkey): + pub = pub_uncompressed(point_mul(privkey)) + h160 = hashlib.new("ripemd160", hashlib.sha256(pub).digest()).digest() + return base58check(bytes([ADDRESS_VERSION]) + h160) + + +# ---- BIP-39 and BIP-32 ----------------------------------------------------- + +def master_from_mnemonic(mnemonic, passphrase=""): + seed = hashlib.pbkdf2_hmac("sha512", mnemonic.encode("utf-8"), + ("mnemonic" + passphrase).encode("utf-8"), + 2048, 64) + i = hmac.new(b"Bitcoin seed", seed, hashlib.sha512).digest() + return int.from_bytes(i[:32], "big"), i[32:] + + +def ckd_priv(key, chain, index): + if index >= HARDENED: + data = b"\x00" + key.to_bytes(32, "big") + else: + data = pub_compressed(point_mul(key)) + i = hmac.new(chain, data + index.to_bytes(4, "big"), hashlib.sha512).digest() + return (int.from_bytes(i[:32], "big") + key) % N, i[32:] + + +def derive(path): + key, chain = master_from_mnemonic(MNEMONIC) + for index in path: + key, chain = ckd_priv(key, chain, index) + return key + + +# ---- the check ------------------------------------------------------------- + +def main(): + failures = 0 + + print("legacy m/i'") + for i, expected in enumerate(LEGACY): + got = address(derive([i + HARDENED])) + ok = got == expected + failures += 0 if ok else 1 + print(" m/%d' %s %s" % (i, got, "ok" if ok else + "MISMATCH, wallet says " + expected)) + + print() + print("bip-44 m/44'/%d'/0'/0/i" % COIN_TYPE) + for i, expected in enumerate(BIP44_EXTERNAL): + got = address(derive([44 + HARDENED, COIN_TYPE + HARDENED, + 0 + HARDENED, 0, i])) + ok = got == expected + failures += 0 if ok else 1 + print(" .../%d %s %s" % (i, got, "ok" if ok else + "MISMATCH, wallet says " + expected)) + + print() + if failures: + print("%d address(es) disagree -- doc/derivation.md is wrong" % failures) + return 1 + print("all addresses match: the document reproduces the wallet") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) -- cgit v1.2.3 From 5ac1fd55eea9d4b5e9b34be50d3b684693e3a935 Mon Sep 17 00:00:00 2001 From: mtg0001 <106211915+mtg0001@users.noreply.github.com> Date: Tue, 4 Aug 2026 15:25:34 -0300 Subject: build: pin external source dependencies (#139) Co-authored-by: Gilcimar Duarte --- Makefile | 43 ++++++++++++++++++++++++++++++++++++------- docs/dependencies.md | 18 ++++++++++++++++++ 2 files changed, 54 insertions(+), 7 deletions(-) create mode 100644 docs/dependencies.md (limited to 'docs') diff --git a/Makefile b/Makefile index ea720f1..0788b61 100644 --- a/Makefile +++ b/Makefile @@ -12,6 +12,11 @@ NPROC := $(shell nproc 2>/dev/null || echo 2) SUDO := $(shell [ "$$(id -u)" = "0" ] && echo "" || echo "sudo") VERSION := 1.2.15 +SECP256K1_REPO := https://github.com/bitcoin-core/secp256k1 +SECP256K1_COMMIT := 7fecac74aed8e1fd9078380d67dd04663705c989 +RANDOMX_REPO := https://github.com/tevador/RandomX +RANDOMX_COMMIT := 1e9d4b2df63fa6edf46b06789486e23c0fcaa55a + # ---- Linux ---------------------------------------------------------------- linux: deps-linux @@ -47,7 +52,13 @@ deps-secp256k1: @if [ ! -f /usr/local/lib/libsecp256k1.a ]; then \ echo "==> building libsecp256k1"; \ rm -rf $(ROOT)/secp256k1-build; \ - git clone --depth 1 https://github.com/bitcoin-core/secp256k1 $(ROOT)/secp256k1-build; \ + git init -q $(ROOT)/secp256k1-build; \ + cd $(ROOT)/secp256k1-build && \ + git remote add origin $(SECP256K1_REPO) && \ + git fetch -q --depth 1 origin $(SECP256K1_COMMIT) && \ + git checkout -q --detach FETCH_HEAD && \ + test "$$(git rev-parse HEAD)" = "$(SECP256K1_COMMIT)" && \ + echo "==> libsecp256k1 pinned at $(SECP256K1_COMMIT)" && \ cd $(ROOT)/secp256k1-build && ./autogen.sh && \ ./configure --enable-module-schnorrsig --enable-module-extrakeys \ --disable-shared --with-pic --disable-benchmark --disable-tests && \ @@ -61,7 +72,13 @@ deps-randomx: @if [ ! -f $(HOME)/RandomX/build/librandomx.a ]; then \ echo "==> building RandomX"; \ rm -rf $(HOME)/RandomX; \ - git clone --depth 1 https://github.com/tevador/RandomX $(HOME)/RandomX; \ + git init -q $(HOME)/RandomX; \ + cd $(HOME)/RandomX && \ + git remote add origin $(RANDOMX_REPO) && \ + git fetch -q --depth 1 origin $(RANDOMX_COMMIT) && \ + git checkout -q --detach FETCH_HEAD && \ + test "$$(git rev-parse HEAD)" = "$(RANDOMX_COMMIT)" && \ + echo "==> RandomX pinned at $(RANDOMX_COMMIT)" && \ mkdir -p $(HOME)/RandomX/build; \ cd $(HOME)/RandomX/build && cmake .. -DCMAKE_BUILD_TYPE=Release && \ make -j$(NPROC) randomx; \ @@ -120,8 +137,14 @@ deps-windows: autoconf automake libtool @if [ ! -f deps/lib/libsecp256k1.a ]; then \ rm -rf secp256k1-build; \ - git clone --depth 1 https://github.com/bitcoin-core/secp256k1 secp256k1-build; \ - cd secp256k1-build && ./autogen.sh && \ + git init -q secp256k1-build; \ + cd secp256k1-build && \ + git remote add origin $(SECP256K1_REPO) && \ + git fetch -q --depth 1 origin $(SECP256K1_COMMIT) && \ + git checkout -q --detach FETCH_HEAD && \ + test "$$(git rev-parse HEAD)" = "$(SECP256K1_COMMIT)" && \ + echo "==> secp256k1 pinned at $(SECP256K1_COMMIT)" && \ + cd $(ROOT)/secp256k1-build && ./autogen.sh && \ ./configure --prefix=$(ROOT)/deps \ --enable-module-schnorrsig --enable-module-extrakeys \ --disable-shared --with-pic --disable-benchmark --disable-tests && \ @@ -132,9 +155,15 @@ deps-windows: fi @if [ ! -f RandomX/build/librandomx.a ]; then \ rm -rf RandomX; \ - git clone --depth 1 https://github.com/tevador/RandomX; \ - mkdir -p RandomX/build; \ - cd RandomX/build && cmake .. -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release && \ + git init -q RandomX; \ + cd RandomX && \ + git remote add origin $(RANDOMX_REPO) && \ + git fetch -q --depth 1 origin $(RANDOMX_COMMIT) && \ + git checkout -q --detach FETCH_HEAD && \ + test "$$(git rev-parse HEAD)" = "$(RANDOMX_COMMIT)" && \ + echo "==> RandomX pinned at $(RANDOMX_COMMIT)" && \ + mkdir -p $(ROOT)/RandomX/build; \ + cd $(ROOT)/RandomX/build && cmake .. -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release && \ make randomx; \ echo "==> RandomX done"; \ else \ diff --git a/docs/dependencies.md b/docs/dependencies.md new file mode 100644 index 0000000..9fd45b6 --- /dev/null +++ b/docs/dependencies.md @@ -0,0 +1,18 @@ +# Build Dependencies + +Bitflash links consensus-adjacent cryptographic and proof-of-work code from +external repositories. Fresh builds must not follow upstream `master` silently: +a changed upstream branch can break reproducible builds or change behavior in +code that every node relies on. + +The top-level `Makefile` pins source dependencies by full commit hash: + +| dependency | repository | pinned commit | +|---|---|---| +| libsecp256k1 | https://github.com/bitcoin-core/secp256k1 | `7fecac74aed8e1fd9078380d67dd04663705c989` | +| RandomX | https://github.com/tevador/RandomX | `1e9d4b2df63fa6edf46b06789486e23c0fcaa55a` | + +When updating either dependency, rebuild from a clean dependency directory and +run the full selftest suite before opening the PR. The PR should explain why the +dependency changed and whether the update can affect consensus, addresses, +signatures, proof-of-work validation, or mining. -- cgit v1.2.3 From c3f6b68f907b03865d2c417ee39fe9526787196f Mon Sep 17 00:00:00 2001 From: Bitflash Date: Thu, 6 Aug 2026 10:46:26 -0300 Subject: release: sign SHA256SUMS with a pinned key, verify against it verify-release.sh embeds the Bitflash release signing key and now requires that the checksum signature validate against that exact key, not merely against some key in the caller's keyring (#138). It also fetches releases from releases.bitflash.network instead of the retired GitHub host, resolving 'latest' from latest.txt. Release signing key fingerprint: 910A 2B4C CA87 9E81 FB4B 2AEA 1D4A 53D3 B78A A4B8 --- docs/release-verification.md | 78 ++++++++++++++++++++----------- scripts/verify-release.sh | 108 +++++++++++++++++++++++-------------------- 2 files changed, 108 insertions(+), 78 deletions(-) (limited to 'docs') diff --git a/docs/release-verification.md b/docs/release-verification.md index ecacd5f..32de74f 100644 --- a/docs/release-verification.md +++ b/docs/release-verification.md @@ -3,60 +3,82 @@ Bitflash releases publish binaries and a `SHA256SUMS` file. The checksum file proves that the file you downloaded matches the file the release page names. A detached signature, `SHA256SUMS.asc`, proves that the checksum file itself was -signed by a trusted release key. +signed by the Bitflash release key. That gives users two separate checks: -1. `gpg --verify SHA256SUMS.asc SHA256SUMS` checks who signed the checksums. -2. `sha256sum -c SHA256SUMS` checks the binaries against those checksums. +1. `gpg --verify SHA256SUMS.asc SHA256SUMS` โ€” who signed the checksums. +2. `sha256sum -c SHA256SUMS` โ€” the binaries against those checksums. -## User Check +But the first check is only worth something if you require a **specific** key. +A good signature by *some* key proves nothing; anyone can make one. -From a shell with `curl` or `wget` and `sha256sum`: +## The release signing key -```bash -scripts/verify-release.sh latest +```text +fingerprint 910A 2B4C CA87 9E81 FB4B 2AEA 1D4A 53D3 B78A A4B8 +uid Bitflash Releases (bitflash.network release signing) + ``` -For a specific release: +The public key is published at +`https://releases.bitflash.network/bitflash-release-key.asc` and embedded in +`scripts/verify-release.sh`, so the tool's trust anchor lives in this +repository rather than on the server it verifies. + +## User check + +From a shell with `curl` or `wget`, `sha256sum` and `gpg`: ```bash -scripts/verify-release.sh v1.2.13 +scripts/verify-release.sh latest ``` -For release audits, require the signature: +For a specific release: ```bash -scripts/verify-release.sh v1.2.13 --require-signature +scripts/verify-release.sh v1.2.15 ``` -Older releases may not have `SHA256SUMS.asc`. In that case the script warns and -still checks file integrity. New release audits should use `--require-signature`. - -## Maintainer Flow +The script downloads the release, imports the pinned key into a throwaway +keyring, and **refuses anything that is not a valid signature by the key above** +before checking the hashes. Releases from before signing was introduced ship +only `SHA256SUMS`; without `--require-signature` those are checked for integrity +and reported as unsigned. -After building release assets in the repository root: +By hand, if you would rather not run the script: ```bash -scripts/make-release-checksums.sh --sign --local-user RELEASE_KEY_ID +curl -O https://releases.bitflash.network/bitflash-release-key.asc +gpg --import bitflash-release-key.asc +gpg --verify SHA256SUMS.asc SHA256SUMS # must name the fingerprint above +sha256sum -c SHA256SUMS ``` -Upload all built assets plus: +## Maintainer flow -```text -SHA256SUMS -SHA256SUMS.asc +The signing key lives on a dedicated location, not in the build tree. After +building the release assets in the repository root, write and sign the +checksums: + +```bash +sha256sum Bitflash-*.zip Bitflash-*.AppImage bitflash-node-* > SHA256SUMS +gpg --armor --detach-sign --output SHA256SUMS.asc SHA256SUMS ``` -Keep the private signing key offline or on a dedicated release machine. Publish -the public key fingerprint in the release notes and keep using the same key for -future releases unless there is a clearly announced rotation. +Publish, alongside the assets, `SHA256SUMS` and `SHA256SUMS.asc`, and update +`latest.txt` on the releases host to the new tag. Keep using the same key; if it +is ever rotated, announce the new fingerprint clearly and update the embedded +key in `verify-release.sh`. -## Why This Matters +## Why this matters `SHA256SUMS` alone protects against a broken download, but not against someone -replacing both a binary and the checksum file. Signing `SHA256SUMS` means an -attacker must also have the release signing key to make the replacement verify. +replacing both a binary and the checksum file. Signing `SHA256SUMS`, and +requiring a pinned key, means an attacker must also hold the release signing key +to make the replacement verify. Serving from bitflash.network makes this the +guarantee that matters: without a pinned key, "trust the site" is the only +thing standing behind a download. This is not reproducible builds yet. It is the smaller, immediate step that -makes every release asset auditable by users before they run it. +makes every release asset auditable before anyone runs it. diff --git a/scripts/verify-release.sh b/scripts/verify-release.sh index 23e71d1..4fd4558 100755 --- a/scripts/verify-release.sh +++ b/scripts/verify-release.sh @@ -1,23 +1,50 @@ #!/usr/bin/env bash set -euo pipefail -repo="Bitflash-sh/bitflash" -tag="${1:-latest}" +# Verifies a Bitflash release: the checksum file's signature against a pinned +# key, then the assets against the checksums. +# +# The signing key is embedded below, not fetched. That is the whole point of a +# pinned key: the trust anchor ships with this script (in git), so it does not +# depend on the same server that serves the release. A server that is +# compromised can swap the binaries and the checksums, but it cannot forge a +# signature by this key. + +host="https://releases.bitflash.network" require_signature=0 +# The Bitflash release signing key. Verification requires this exact key -- +# gpg's own "good signature" is not enough, because a good signature by *some* +# key proves nothing. See issue #138. +RELEASE_KEY_FP="910A2B4CCA879E81FB4B2AEA1D4A53D3B78AA4B8" +read -r -d '' RELEASE_KEY <<'KEY' || true +-----BEGIN PGP PUBLIC KEY BLOCK----- + +mDMEanSOmRYJKwYBBAHaRw8BAQdAXj24i2BYUcAmOpoXwjE4ZTynHsifCZ+niwQK +HqO0nMa0UEJpdGZsYXNoIFJlbGVhc2VzIChiaXRmbGFzaC5uZXR3b3JrIHJlbGVh +c2Ugc2lnbmluZykgPHJlbGVhc2VzQGJpdGZsYXNoLm5ldHdvcms+iJAEExYKADgW +IQSRCitMyoeegftLKuodSlPTt4qkuAUCanSOmQIbAwULCQgHAgYVCgkICwIEFgID +AQIeAQIXgAAKCRAdSlPTt4qkuIi9AP96X3OcstiCTTgKOr16Vhds3LyuZ19q3Apq +4KAcNuOUMAD+NqgJ1CqkLGPXfZLCtYBCRHXNYWxMMEk+ViLcRtdLMg4= +=mxic +-----END PGP PUBLIC KEY BLOCK----- +KEY + usage() { cat <<'EOF' Usage: scripts/verify-release.sh [tag|latest] [--require-signature] Downloads the release checksum file and every asset named in it, then verifies: - 1. SHA256SUMS.asc, if present, against SHA256SUMS with gpg. + 1. SHA256SUMS.asc against SHA256SUMS, requiring the pinned Bitflash release + key (fingerprint 910A 2B4C CA87 9E81 FB4B 2AEA 1D4A 53D3 B78A A4B8). 2. SHA256SUMS against the downloaded assets. -Use --require-signature for release audits. Without it, older releases that -ship only SHA256SUMS are checked for integrity and reported as unsigned. +Releases from before signing was introduced ship only SHA256SUMS; without +--require-signature those are checked for integrity and reported as unsigned. EOF } +tag="${1:-latest}" for arg in "${@:2}"; do case "$arg" in --require-signature) require_signature=1 ;; @@ -25,19 +52,9 @@ for arg in "${@:2}"; do *) echo "unknown option: $arg" >&2; usage >&2; exit 2 ;; esac done +if [ "$tag" = "-h" ] || [ "$tag" = "--help" ]; then usage; exit 0; fi -if [ "$tag" = "-h" ] || [ "$tag" = "--help" ]; then - usage - exit 0 -fi - -need() { - if ! command -v "$1" >/dev/null 2>&1; then - echo "missing required command: $1" >&2 - exit 1 - fi -} - +need() { command -v "$1" >/dev/null 2>&1 || { echo "missing required command: $1" >&2; exit 1; }; } need sed need awk @@ -46,39 +63,25 @@ if command -v sha256sum >/dev/null 2>&1; then elif command -v shasum >/dev/null 2>&1; then sha_check=(shasum -a 256 -c SHA256SUMS) else - echo "missing required command: sha256sum or shasum" >&2 - exit 1 + echo "missing required command: sha256sum or shasum" >&2; exit 1 fi download() { - local url="$1" - local out="$2" - if command -v curl >/dev/null 2>&1; then - curl -fsSL "$url" -o "$out" - elif command -v wget >/dev/null 2>&1; then - wget -q "$url" -O "$out" - else - echo "missing required command: curl or wget" >&2 - exit 1 - fi + if command -v curl >/dev/null 2>&1; then curl -fsSL "$1" -o "$2" + elif command -v wget >/dev/null 2>&1; then wget -q "$1" -O "$2" + else echo "missing required command: curl or wget" >&2; exit 1; fi } if [ "$tag" = "latest" ]; then - tag="$(download "https://api.github.com/repos/$repo/releases/latest" - | - sed -n 's/.*"tag_name"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' | - head -1)" - if [ -z "$tag" ]; then - echo "could not resolve latest release tag" >&2 - exit 1 - fi + tag="$(download "$host/latest.txt" - 2>/dev/null | tr -d '[:space:]' || true)" + [ -n "$tag" ] || { echo "could not resolve the latest release tag from $host/latest.txt" >&2; exit 1; } fi workdir="$(mktemp -d "${TMPDIR:-/tmp}/bitflash-release-verify.XXXXXX")" cleanup() { rm -rf "$workdir"; } trap cleanup EXIT -base_url="https://github.com/$repo/releases/download/$tag" - +base_url="$host/$tag" echo "Verifying Bitflash $tag" echo "Working directory: $workdir" cd "$workdir" @@ -86,27 +89,32 @@ cd "$workdir" download "$base_url/SHA256SUMS" SHA256SUMS if download "$base_url/SHA256SUMS.asc" SHA256SUMS.asc 2>/dev/null; then - if ! command -v gpg >/dev/null 2>&1; then - echo "SHA256SUMS.asc exists, but gpg is not installed" >&2 + need gpg + # Verify in a throwaway keyring holding ONLY the pinned key, and require that + # the signature validates against it with the pinned fingerprint. This is the + # difference between "signed" and "signed by Bitflash". + keyring="$workdir/keyring" + mkdir -p "$keyring"; chmod 700 "$keyring" + printf '%s\n' "$RELEASE_KEY" | GNUPGHOME="$keyring" gpg --batch --quiet --import 2>/dev/null + status="$(GNUPGHOME="$keyring" gpg --batch --status-fd=1 --verify SHA256SUMS.asc SHA256SUMS 2>/dev/null || true)" + if printf '%s' "$status" | grep -q "VALIDSIG.*$RELEASE_KEY_FP"; then + echo "Signature OK -- signed by the pinned Bitflash release key" + else + echo "SIGNATURE REJECTED: not a valid signature by the pinned release key" >&2 + echo " expected key: $RELEASE_KEY_FP" >&2 exit 1 fi - gpg --verify SHA256SUMS.asc SHA256SUMS - echo "Signature OK" else if [ "$require_signature" -eq 1 ]; then - echo "release does not publish SHA256SUMS.asc" >&2 - exit 1 + echo "release does not publish SHA256SUMS.asc" >&2; exit 1 fi - echo "WARNING: no SHA256SUMS.asc found; checking hashes only" + echo "WARNING: no SHA256SUMS.asc found; checking hashes only (unsigned release)" fi awk '{print $2}' SHA256SUMS | while IFS= read -r file; do - file="${file#\*}" - file="${file#./}" + file="${file#\*}"; file="${file#./}" [ -z "$file" ] && continue - case "$file" in - */*|..*) echo "refusing unexpected checksum path: $file" >&2; exit 1 ;; - esac + case "$file" in */*|..*) echo "refusing unexpected checksum path: $file" >&2; exit 1 ;; esac echo "Downloading $file" download "$base_url/$file" "$file" done -- cgit v1.2.3 From a0355492c3bae37ad57ef3e05302a3e5b1a80e3e Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Fri, 7 Aug 2026 11:42:30 -0300 Subject: tools: build a static pool directory --- docs/pool-directory.md | 35 ++++++++++ scripts/build-pool-index.py | 153 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 188 insertions(+) create mode 100644 docs/pool-directory.md create mode 100644 scripts/build-pool-index.py (limited to 'docs') diff --git a/docs/pool-directory.md b/docs/pool-directory.md new file mode 100644 index 0000000..6456084 --- /dev/null +++ b/docs/pool-directory.md @@ -0,0 +1,35 @@ +# Public Pool Directory + +`pool.bitflash.network` should be a public directory, not a network dependency. +Nodes discover pools through Nostr and `.btf`; the website only helps humans +compare operators. + +Operators can publish the `pool_status.json` file produced by: + +```bash +bitflash -nogui -gen -operator -poolstatusfile=/var/www/pool_status.json +``` + +Build a static directory from one or more status files: + +```bash +python3 scripts/build-pool-index.py --out-dir public/pool \ + https://operator-one.example/pool_status.json \ + https://operator-two.example/pool_status.json +``` + +The generated output contains: + +- `index.html`: human-readable ranking page. +- `pools.json`: machine-readable API for `api.bitflash.network/pools.json`. + +Ranking order is intentionally simple: + +1. online pools first; +2. higher reported hashrate; +3. more authorized miners; +4. lower fee; +5. name as a stable tie-breaker. + +An entry is considered online when the source says the pool is running and its +status file is no more than 120 seconds old. diff --git a/scripts/build-pool-index.py b/scripts/build-pool-index.py new file mode 100644 index 0000000..40eb8f4 --- /dev/null +++ b/scripts/build-pool-index.py @@ -0,0 +1,153 @@ +#!/usr/bin/env python3 +# +# Build a static public pool directory from one or more Bitflash pool_status.json +# files. Inputs may be local paths or http(s) URLs. + +import argparse +import html +import json +import os +import time +import urllib.request + + +def load_json(source): + if source.startswith("http://") or source.startswith("https://"): + with urllib.request.urlopen(source, timeout=10) as r: + return json.loads(r.read().decode("utf-8")) + with open(source, "r", encoding="utf-8-sig") as f: + return json.load(f) + + +def normalize_pool(source, obj, now): + operator = obj.get("operator", {}) + pool = obj.get("pool", {}) + node = obj.get("node", {}) + updated = int(obj.get("updatedAt") or 0) + age = max(0, now - updated) if updated else None + online = bool(node.get("poolRunning")) and age is not None and age <= 120 + return { + "source": source, + "name": operator.get("name") or "Bitflash Pool", + "btfAddress": operator.get("btfAddress") or "", + "feePercent": float(operator.get("feePercent") or 0.0), + "dashboardUrl": operator.get("dashboardUrl") or "", + "height": int(node.get("height") or 0), + "online": online, + "ageSeconds": age, + "authorizedMiners": int(pool.get("authorizedMiners") or 0), + "blocksFoundSession": int(pool.get("blocksFoundSession") or 0), + "roundShares": int(pool.get("roundShares") or 0), + "hashRate": float(pool.get("hashRate") or 0.0), + } + + +def rank_key(pool): + return ( + 1 if pool["online"] else 0, + pool["hashRate"], + pool["authorizedMiners"], + -pool["feePercent"], + pool["name"].lower(), + ) + + +def render_html(pools, generated_at): + rows = [] + for p in pools: + dash = p["dashboardUrl"] + dash_html = ( + 'dashboard'.format(html.escape(dash, quote=True)) + if dash else "-" + ) + rows.append( + "" + "{status}" + "{name}
{addr}" + "{fee:.2f}%" + "{miners}" + "{hashrate:.2f}" + "{blocks}" + "{height}" + "{age}" + "{dash}" + "".format( + status="online" if p["online"] else "stale", + name=html.escape(p["name"]), + addr=html.escape(p["btfAddress"]), + fee=p["feePercent"], + miners=p["authorizedMiners"], + hashrate=p["hashRate"], + blocks=p["blocksFoundSession"], + height=p["height"], + age="{}s".format(p["ageSeconds"]) if p["ageSeconds"] is not None else "-", + dash=dash_html, + ) + ) + + return """ + + + + + Bitflash Pools + + + +

Bitflash Pools

+

This directory is informational. Bitflash nodes discover pools through Nostr and .btf; this page only helps humans compare public operators.

+ + + + + + {rows} + +
StatusPoolFeeMinersH/sBlocksHeightAgeLink
+

Generated at {generated_at}. JSON: pools.json.

+ + +""".format(rows="\n ".join(rows), generated_at=generated_at) + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--out-dir", default="pool-site") + ap.add_argument("sources", nargs="+", help="pool_status.json paths or URLs") + args = ap.parse_args() + + now = int(time.time()) + pools = [] + errors = [] + for source in args.sources: + try: + pools.append(normalize_pool(source, load_json(source), now)) + except Exception as e: + errors.append({"source": source, "error": str(e)}) + + pools.sort(key=rank_key, reverse=True) + os.makedirs(args.out_dir, exist_ok=True) + + generated_at = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(now)) + with open(os.path.join(args.out_dir, "pools.json"), "w", encoding="utf-8") as f: + json.dump({ + "schema": "bitflash-pool-directory-1", + "generatedAt": generated_at, + "pools": pools, + "errors": errors, + }, f, indent=2, sort_keys=True) + f.write("\n") + + with open(os.path.join(args.out_dir, "index.html"), "w", encoding="utf-8") as f: + f.write(render_html(pools, generated_at)) + + +if __name__ == "__main__": + main() -- cgit v1.2.3 From b1eb8b66b8d7b47c7fb2e85d5d4db86a11823366 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Fri, 7 Aug 2026 17:14:36 -0300 Subject: pool: publish round proof ledger --- README.md | 14 +- docs/pool-directory.md | 12 +- scripts/build-pool-index.py | 18 ++- src/main.cpp | 1 + src/main.h | 1 + src/main_gui.cpp | 5 + src/readme.txt | 4 + src/rpc.cpp | 316 +++++++++++++++++++++++++++++++++++++++++++- 8 files changed, 359 insertions(+), 12 deletions(-) (limited to 'docs') diff --git a/README.md b/README.md index 7f4d9b0..ec8e9c4 100644 --- a/README.md +++ b/README.md @@ -219,10 +219,12 @@ Open **Options** from the menu bar. Under Mining Mode: Pool announcements are published on Nostr with live status fields, so external tools can query the latest pool state by `.btf` address. -Pool operators also write a local `pool_status.json` every ten seconds. By -default it is created in the data directory; use `-poolstatusfile=PATH` to write -it somewhere a dashboard or web sync can read. The file is informational only: -nodes still discover pools through Nostr and `.btf`, not through any web domain. +Pool operators also write a local `pool_status.json` every ten seconds and a +`pool_rounds.json` proof ledger whenever they find blocks or pay miners. By +default both are created in the data directory; use `-poolstatusfile=PATH` and +`-poolroundsfile=PATH` to write them somewhere a dashboard or web sync can read. +The files are informational only: nodes still discover pools through Nostr and +`.btf`, not through any web domain. **Participant** โ€” mine to someone else's pool. Enter or select the pool's `.btf` address and enable Start Mining. @@ -243,7 +245,9 @@ xmrig -a rx/0 -o 127.0.0.1:3333 -u YOUR_BTF_ADDRESS -p x ./bitflash -nogui # node only ./bitflash -nogui -gen # node + solo mining ./bitflash -nogui -gen -operator # pool operator -./bitflash -nogui -gen -operator -poolstatusfile=/var/www/api/pools.json +./bitflash -nogui -gen -operator \ + -poolstatusfile=/var/www/pool_status.json \ + -poolroundsfile=/var/www/pool_rounds.json ./bitflash -nogui -gen -participant=POOL_BTF_ADDRESS # mine to pool ./bitflash -nogui -stratumbridge=POOL_BTF_ADDRESS # local bridge for XMRig/SRBMiner ``` diff --git a/docs/pool-directory.md b/docs/pool-directory.md index 6456084..76269b9 100644 --- a/docs/pool-directory.md +++ b/docs/pool-directory.md @@ -4,10 +4,13 @@ Nodes discover pools through Nostr and `.btf`; the website only helps humans compare operators. -Operators can publish the `pool_status.json` file produced by: +Operators can publish the `pool_status.json` and `pool_rounds.json` files +produced by: ```bash -bitflash -nogui -gen -operator -poolstatusfile=/var/www/pool_status.json +bitflash -nogui -gen -operator \ + -poolstatusfile=/var/www/pool_status.json \ + -poolroundsfile=/var/www/pool_rounds.json ``` Build a static directory from one or more status files: @@ -23,6 +26,11 @@ The generated output contains: - `index.html`: human-readable ranking page. - `pools.json`: machine-readable API for `api.bitflash.network/pools.json`. +When a source is an HTTP(S) `pool_status.json`, the directory links to a +`pool_rounds.json` proof file next to it. That file records the block height, +block hash, coinbase txid, coinbase value, pool fee, total shares, per-address +share counts, computed payouts, and payout txids once paid. + Ranking order is intentionally simple: 1. online pools first; diff --git a/scripts/build-pool-index.py b/scripts/build-pool-index.py index 382b6c3..d3eb386 100644 --- a/scripts/build-pool-index.py +++ b/scripts/build-pool-index.py @@ -8,6 +8,7 @@ import html import json import os import time +import urllib.parse import urllib.request @@ -23,6 +24,10 @@ def normalize_pool(source, obj, now): operator = obj.get("operator", {}) pool = obj.get("pool", {}) node = obj.get("node", {}) + rounds_file = operator.get("roundsFile") or "pool_rounds.json" + rounds_url = "" + if source.startswith("http://") or source.startswith("https://"): + rounds_url = urllib.parse.urljoin(source, rounds_file) updated = int(obj.get("updatedAt") or 0) age = max(0, now - updated) if updated else None online = bool(node.get("poolRunning")) and age is not None and age <= 120 @@ -39,6 +44,8 @@ def normalize_pool(source, obj, now): "blocksFoundSession": int(pool.get("blocksFoundSession") or 0), "roundShares": int(pool.get("roundShares") or 0), "hashRate": float(pool.get("hashRate") or 0.0), + "roundsUrl": rounds_url, + "recentRounds": obj.get("recentRounds", []), } @@ -56,6 +63,7 @@ def render_html(pools, generated_at): rows = [] for p in pools: dash = p["dashboardUrl"] + rounds = p.get("roundsUrl", "") # Only linkify http(s). The dashboard URL comes from an operator's # self-published status file, so a "javascript:" or "data:" scheme # would become a clickable script-injection on this public page. @@ -65,6 +73,12 @@ def render_html(pools, generated_at): html.escape(dash, quote=True)) else: dash_html = "-" + rounds_lower = rounds.lower() + if rounds_lower.startswith("http://") or rounds_lower.startswith("https://"): + rounds_html = 'proof'.format( + html.escape(rounds, quote=True)) + else: + rounds_html = "-" rows.append( "" "{status}" @@ -76,6 +90,7 @@ def render_html(pools, generated_at): "{height}" "{age}" "{dash}" + "{rounds}" "".format( status="online" if p["online"] else "stale", name=html.escape(p["name"]), @@ -87,6 +102,7 @@ def render_html(pools, generated_at): height=p["height"], age="{}s".format(p["ageSeconds"]) if p["ageSeconds"] is not None else "-", dash=dash_html, + rounds=rounds_html, ) ) @@ -110,7 +126,7 @@ def render_html(pools, generated_at):

This directory is informational. Bitflash nodes discover pools through Nostr and .btf; this page only helps humans compare public operators.

- + {rows} diff --git a/src/main.cpp b/src/main.cpp index 2be9c54..2eb0e3a 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -105,6 +105,7 @@ string strParticipantPool; // participant mode: pool .btf address string strPoolName = "Bitflash Pool"; string strPoolDashboardUrl; string strPoolStatusFile; +string strPoolRoundsFile; double dPoolFeePercent = 0.0; bool fStratumBridge = false; int nStratumBridgePort = 3333; diff --git a/src/main.h b/src/main.h index 2786a4a..eafbf16 100644 --- a/src/main.h +++ b/src/main.h @@ -87,6 +87,7 @@ extern string strParticipantPool; // participant: pool .btf address extern string strPoolName; // operator: announced pool name extern string strPoolDashboardUrl; // operator: optional dashboard URL extern string strPoolStatusFile; // operator: optional JSON status output path +extern string strPoolRoundsFile; // operator: optional public round proof path extern double dPoolFeePercent; // operator: announced fee percent extern bool fStratumBridge; // participant helper: local Stratum bridge extern int nStratumBridgePort; // local bridge listen port, default 3333 diff --git a/src/main_gui.cpp b/src/main_gui.cpp index ce6d2b5..7ad2075 100644 --- a/src/main_gui.cpp +++ b/src/main_gui.cpp @@ -126,6 +126,7 @@ static void PrintUsage() printf(" /poolfee=PCT\n"); printf(" /pooldashboard=URL (alias: /pooldash=URL)\n"); printf(" /poolstatusfile=PATH (write pool status JSON for dashboards)\n"); + printf(" /poolroundsfile=PATH (write public pool round proofs JSON)\n"); printf("\n"); printf(".btf and rendezvous:\n"); printf(" /connectbtf=PEER_BTF_ADDRESS\n"); @@ -262,6 +263,10 @@ static void ParseStartupArguments(int argc, char* argv[]) if (!poolStatusFile.empty()) strPoolStatusFile = poolStatusFile; + string poolRoundsFile = argval2(argc, argv, "/poolroundsfile", "-poolroundsfile"); + if (!poolRoundsFile.empty()) + strPoolRoundsFile = poolRoundsFile; + string btfConnect = argval2(argc, argv, "/connectbtf", "-connectbtf"); if (!btfConnect.empty()) strBtfConnect = btfConnect; diff --git a/src/readme.txt b/src/readme.txt index fa58b6e..b8b89c1 100644 --- a/src/readme.txt +++ b/src/readme.txt @@ -55,6 +55,10 @@ Command-line options (in addition to the original ones) /poolfee=PCT operator announcement fee percent (example: 0.75) /pooldashboard=URL operator announcement dashboard URL + /poolstatusfile=PATH + write pool status JSON for dashboards + /poolroundsfile=PATH + write public pool round proofs JSON /connectbtf=ADDR keep an outbound .btf connection to ADDR /rvrelay=HOST:PORT diff --git a/src/rpc.cpp b/src/rpc.cpp index 1aa4f9b..813f1d1 100644 --- a/src/rpc.cpp +++ b/src/rpc.cpp @@ -263,11 +263,47 @@ struct PendingPayout { static std::vector gPendingPayouts; static std::mutex gPayoutLedgerMutex; +struct PoolRoundProof { + std::string id; + int blockHeight; + int matureAtHeight; + int64 createdAt; + std::string blockHash; + std::string coinbaseTxid; + std::string foundBy; + double feePercent; + uint64 totalShares; + int64 coinbaseValue; + int64 subsidy; + int64 minerReward; + int64 operatorReward; + int skippedDust; + int64 forfeitedBadAddress; + std::map shares; + std::map payouts; + std::map paidTxids; + + PoolRoundProof() + : blockHeight(0), matureAtHeight(0), createdAt(0), feePercent(0.0), + totalShares(0), coinbaseValue(0), subsidy(0), minerReward(0), operatorReward(0), + skippedDust(0), forfeitedBadAddress(0) {} +}; +static std::vector gPoolRounds; +static std::mutex gPoolRoundsMutex; +static const size_t MAX_POOL_ROUNDS = 200; + static std::string PayoutFilePath() { return GetAppDir() + "/pending_payouts.json"; } +static std::string PoolRoundsFilePath() +{ + if (!strPoolRoundsFile.empty()) + return strPoolRoundsFile; + return GetAppDir() + "/pool_rounds.json"; +} + static std::string PayoutIdForBlock(int blockHeight) { return strprintf("block-%d", blockHeight); @@ -362,6 +398,167 @@ static void LoadPendingPayouts() } } +static nlohmann::json PoolRoundToJson(const PoolRoundProof& r) +{ + nlohmann::json j; + j["id"] = r.id; + j["blockHeight"] = r.blockHeight; + j["matureAtHeight"] = r.matureAtHeight; + j["createdAt"] = r.createdAt; + j["blockHash"] = r.blockHash; + j["coinbaseTxid"] = r.coinbaseTxid; + j["foundBy"] = r.foundBy; + j["feePercent"] = r.feePercent; + j["totalShares"] = r.totalShares; + j["coinbaseValueSatoshis"] = r.coinbaseValue; + j["coinbaseValueBtf"] = FormatMoney(r.coinbaseValue); + j["subsidySatoshis"] = r.subsidy; + j["subsidyBtf"] = FormatMoney(r.subsidy); + j["minerRewardSatoshis"] = r.minerReward; + j["minerRewardBtf"] = FormatMoney(r.minerReward); + j["operatorRewardSatoshis"] = r.operatorReward; + j["operatorRewardBtf"] = FormatMoney(r.operatorReward); + j["skippedDust"] = r.skippedDust; + j["forfeitedBadAddressSatoshis"] = r.forfeitedBadAddress; + j["forfeitedBadAddressBtf"] = FormatMoney(r.forfeitedBadAddress); + + j["shares"] = nlohmann::json::object(); + for (const auto& kv : r.shares) + j["shares"][kv.first] = kv.second; + + j["payouts"] = nlohmann::json::object(); + for (const auto& kv : r.payouts) { + nlohmann::json p; + p["amountSatoshis"] = kv.second; + p["amountBtf"] = FormatMoney(kv.second); + std::map::const_iterator tx = r.paidTxids.find(kv.first); + if (tx != r.paidTxids.end()) + p["txid"] = tx->second; + else + p["txid"] = nullptr; + j["payouts"][kv.first] = p; + } + return j; +} + +static void SavePoolRoundsLocked() +{ + nlohmann::json root; + root["schema"] = "bitflash-pool-rounds-1"; + root["updatedAt"] = GetTime(); + root["rounds"] = nlohmann::json::array(); + for (const PoolRoundProof& r : gPoolRounds) + root["rounds"].push_back(PoolRoundToJson(r)); + + std::string path = PoolRoundsFilePath(); + std::string tmp = path + ".tmp"; + FILE* f = fopen(tmp.c_str(), "w"); + if (!f) { + LogPrint("payout", "[payout] WARNING: could not write %s\n", tmp.c_str()); + return; + } + std::string s = root.dump(2); + fwrite(s.c_str(), 1, s.size(), f); + if (fclose(f) != 0) { + LogPrint("payout", "[payout] WARNING: could not close %s\n", tmp.c_str()); + return; + } +#ifdef _WIN32 + remove(path.c_str()); +#endif + if (rename(tmp.c_str(), path.c_str()) != 0) + LogPrint("payout", "[payout] WARNING: could not replace %s\n", path.c_str()); +} + +static void LoadPoolRounds() +{ + std::string path = PoolRoundsFilePath(); + FILE* f = fopen(path.c_str(), "r"); + if (!f) return; + fseek(f, 0, SEEK_END); + long sz = ftell(f); + fseek(f, 0, SEEK_SET); + std::string s(sz, '\0'); + fread(&s[0], 1, sz, f); + fclose(f); + + try { + nlohmann::json root = nlohmann::json::parse(s); + nlohmann::json arr = root.is_array() ? root : root["rounds"]; + std::lock_guard lk(gPoolRoundsMutex); + for (const auto& obj : arr) { + PoolRoundProof r; + if (obj.contains("id") && obj["id"].is_string()) r.id = obj["id"].get(); + if (obj.contains("blockHeight")) r.blockHeight = obj["blockHeight"].get(); + if (obj.contains("matureAtHeight")) r.matureAtHeight = obj["matureAtHeight"].get(); + if (obj.contains("createdAt")) r.createdAt = obj["createdAt"].get(); + if (obj.contains("blockHash") && obj["blockHash"].is_string()) r.blockHash = obj["blockHash"].get(); + if (obj.contains("coinbaseTxid") && obj["coinbaseTxid"].is_string()) r.coinbaseTxid = obj["coinbaseTxid"].get(); + if (obj.contains("foundBy") && obj["foundBy"].is_string()) r.foundBy = obj["foundBy"].get(); + if (obj.contains("feePercent")) r.feePercent = obj["feePercent"].get(); + if (obj.contains("totalShares")) r.totalShares = obj["totalShares"].get(); + if (obj.contains("coinbaseValueSatoshis")) r.coinbaseValue = obj["coinbaseValueSatoshis"].get(); + if (obj.contains("subsidySatoshis")) r.subsidy = obj["subsidySatoshis"].get(); + if (obj.contains("minerRewardSatoshis")) r.minerReward = obj["minerRewardSatoshis"].get(); + if (obj.contains("operatorRewardSatoshis")) r.operatorReward = obj["operatorRewardSatoshis"].get(); + if (obj.contains("skippedDust")) r.skippedDust = obj["skippedDust"].get(); + if (obj.contains("forfeitedBadAddressSatoshis")) r.forfeitedBadAddress = obj["forfeitedBadAddressSatoshis"].get(); + if (obj.contains("shares") && obj["shares"].is_object()) + for (auto it = obj["shares"].begin(); it != obj["shares"].end(); ++it) + r.shares[it.key()] = it.value().get(); + if (obj.contains("payouts") && obj["payouts"].is_object()) + for (auto it = obj["payouts"].begin(); it != obj["payouts"].end(); ++it) { + if (it.value().is_object()) { + r.payouts[it.key()] = it.value().value("amountSatoshis", (int64)0); + if (it.value().contains("txid") && it.value()["txid"].is_string()) + r.paidTxids[it.key()] = it.value()["txid"].get(); + } else if (it.value().is_number_integer()) { + r.payouts[it.key()] = it.value().get(); + } + } + if (!r.id.empty()) + gPoolRounds.push_back(r); + } + while (gPoolRounds.size() > MAX_POOL_ROUNDS) + gPoolRounds.erase(gPoolRounds.begin()); + LogPrint("payout", "[payout] loaded %zu pool round proof(s) from disk\n", + gPoolRounds.size()); + } catch (const std::exception& e) { + LogPrint("payout", "[payout] WARNING: failed to parse %s: %s\n", + path.c_str(), e.what()); + } +} + +static void RecordPoolRound(const PoolRoundProof& round) +{ + std::lock_guard lk(gPoolRoundsMutex); + for (PoolRoundProof& existing : gPoolRounds) { + if (existing.id == round.id) { + existing = round; + SavePoolRoundsLocked(); + return; + } + } + gPoolRounds.push_back(round); + while (gPoolRounds.size() > MAX_POOL_ROUNDS) + gPoolRounds.erase(gPoolRounds.begin()); + SavePoolRoundsLocked(); +} + +static void MarkPoolRoundPaid(const std::string& id, + const std::string& address, + const std::string& txid) +{ + std::lock_guard lk(gPoolRoundsMutex); + for (PoolRoundProof& r : gPoolRounds) { + if (r.id != id) + continue; + r.paidTxids[address] = txid; + SavePoolRoundsLocked(); + return; + } +} + // Payouts ready to execute are moved here under gPayoutExecMutex. // A dedicated thread drains this list so SendMoney (which takes cs_main) // never runs on the event loop thread and can't stall miner I/O. @@ -399,10 +596,28 @@ static bool FindRecordedPoolPayout(const std::string& id, } static void QueuePayouts(int blockHeight, + const std::string& blockHash, + const std::string& coinbaseTxid, + int64 coinbaseValue, + const std::string& foundBy, const std::map& shareCount, uint64 total) { + PoolRoundProof round; + round.id = PayoutIdForBlock(blockHeight); + round.blockHeight = blockHeight; + round.matureAtHeight = blockHeight + COINBASE_MATURITY; + round.createdAt = GetTime(); + round.blockHash = blockHash; + round.coinbaseTxid = coinbaseTxid; + round.coinbaseValue = coinbaseValue; + round.foundBy = foundBy; + round.feePercent = dPoolFeePercent; + round.totalShares = total; + round.shares = shareCount; + if (total == 0) { + RecordPoolRound(round); LogPrint("payout", "[payout] block %d: zero shares recorded -- nothing to pay out\n", blockHeight); return; @@ -418,9 +633,13 @@ static void QueuePayouts(int blockHeight, int64 nSubsidy = 50 * COIN; nSubsidy >>= (blockHeight / 210000); // halving schedule int64 minerReward = (int64)((double)nSubsidy * (1.0 - feeFrac)); + int64 operatorCut = nSubsidy - minerReward; + round.subsidy = nSubsidy; + round.minerReward = minerReward; + round.operatorReward = operatorCut; PendingPayout pp; - pp.id = PayoutIdForBlock(blockHeight); + pp.id = round.id; pp.blockHeight = blockHeight; pp.matureAtHeight = blockHeight + COINBASE_MATURITY; @@ -443,15 +662,18 @@ static void QueuePayouts(int blockHeight, continue; } pp.amounts[kv.first] = amount; + round.payouts[kv.first] = amount; } + round.skippedDust = skipped; + round.forfeitedBadAddress = burned; { std::lock_guard lk(gPayoutLedgerMutex); gPendingPayouts.push_back(pp); SavePendingPayoutsLocked(); } + RecordPoolRound(round); - int64 operatorCut = nSubsidy - minerReward; LogPrint("payout", "[payout] block %d: subsidy %s BTF, miners get %s" " (fee %.2f%% = %s to operator)," @@ -548,12 +770,14 @@ static void PayoutThreadFn(void*) LogPrint("payout", "[payout] recovered recorded payout %s to %s" " from wallet tx %s\n", FormatMoney(kv.second).c_str(), kv.first.c_str(), txid.c_str()); + MarkPoolRoundPaid(ppQueued.id, kv.first, txid); } else if (SendMoney(sc, kv.second, wtx)) { txid = wtx.GetHash().GetHex(); LogPrint("payout", "[payout] paid %s to %s\n", FormatMoney(kv.second).c_str(), kv.first.c_str()); + MarkPoolRoundPaid(ppQueued.id, kv.first, txid); } else { @@ -664,7 +888,8 @@ static void WritePoolStatusJsonLocked() {"name", strPoolName}, {"btfAddress", BtfLocalAddress()}, {"feePercent", dPoolFeePercent}, - {"dashboardUrl", strPoolDashboardUrl} + {"dashboardUrl", strPoolDashboardUrl}, + {"roundsFile", "pool_rounds.json"} }; j["node"] = { @@ -702,6 +927,30 @@ static void WritePoolStatusJsonLocked() }); } + j["recentRounds"] = nlohmann::json::array(); + { + std::lock_guard lk(gPoolRoundsMutex); + size_t start = gPoolRounds.size() > 10 ? gPoolRounds.size() - 10 : 0; + for (size_t i = start; i < gPoolRounds.size(); i++) { + const PoolRoundProof& r = gPoolRounds[i]; + int unpaid = 0; + for (const auto& kv : r.payouts) + if (r.paidTxids.count(kv.first) == 0) + unpaid++; + j["recentRounds"].push_back({ + {"id", r.id}, + {"blockHeight", r.blockHeight}, + {"blockHash", r.blockHash}, + {"coinbaseTxid", r.coinbaseTxid}, + {"totalShares", r.totalShares}, + {"payouts", (int)r.payouts.size()}, + {"unpaidPayouts", unpaid}, + {"coinbaseValueBtf", FormatMoney(r.coinbaseValue)}, + {"minerRewardBtf", FormatMoney(r.minerReward)} + }); + } + } + std::string path = PoolStatusFilePath(); std::string tmp = path + ".tmp"; FILE* f = fopen(tmp.c_str(), "w"); @@ -1231,7 +1480,9 @@ static bool HandleLine(Miner* m, const std::string& rawLine, if (accepted) { // Snapshot height before RebuildJob() (which increments it) int foundHeight = gCurrentJob.height; - QueuePayouts(foundHeight, roundShareCount, roundShareTotal); + QueuePayouts(foundHeight, b.GetHash().GetHex(), b.vtx[0].GetHash().GetHex(), + b.vtx[0].vout[0].nValue, m->address, + roundShareCount, roundShareTotal); // Reset round shares roundShareCount.clear(); roundShareTotal = 0; @@ -1333,6 +1584,62 @@ int RunPoolStratumSelfTest() HandleLine(&miner, stale.dump(), roundShareCount, roundShareTotal, blocksFound); check(roundShareTotal == 1, "stale submit does not increment shares"); + std::vector savedRounds = gPoolRounds; + std::string savedRoundsFile = strPoolRoundsFile; + std::string proofPath = "pool_rounds_selftest.json"; + remove(proofPath.c_str()); + remove((proofPath + ".tmp").c_str()); + gPoolRounds.clear(); + strPoolRoundsFile = proofPath; + + PoolRoundProof proof; + proof.id = "selftest-round"; + proof.blockHeight = 42; + proof.matureAtHeight = 42 + COINBASE_MATURITY; + proof.createdAt = GetTime(); + proof.blockHash = block.GetHash().GetHex(); + proof.coinbaseTxid = coinbase.GetHash().GetHex(); + proof.foundBy = miner.address; + proof.feePercent = 1.25; + proof.totalShares = 3; + proof.coinbaseValue = 50 * COIN; + proof.subsidy = 50 * COIN; + proof.minerReward = 49 * COIN; + proof.operatorReward = 1 * COIN; + proof.shares[miner.address] = 3; + proof.payouts[miner.address] = 49 * COIN; + RecordPoolRound(proof); + MarkPoolRoundPaid(proof.id, miner.address, "selftest-txid"); + + bool proofOk = false; + FILE* pf = fopen(proofPath.c_str(), "r"); + if (pf) { + fseek(pf, 0, SEEK_END); + long sz = ftell(pf); + fseek(pf, 0, SEEK_SET); + std::string s(sz, '\0'); + fread(&s[0], 1, sz, pf); + fclose(pf); + try { + json root = json::parse(s); + proofOk = + root.value("schema", "") == "bitflash-pool-rounds-1" && + root["rounds"].is_array() && + root["rounds"].size() == 1 && + root["rounds"][0].value("id", "") == proof.id && + root["rounds"][0]["shares"].value(miner.address, (uint64)0) == 3 && + root["rounds"][0]["payouts"][miner.address].value("txid", "") == "selftest-txid"; + } catch (...) { + proofOk = false; + } + } + check(proofOk, "pool round proof ledger is written and paid txid is recorded"); + + remove(proofPath.c_str()); + remove((proofPath + ".tmp").c_str()); + strPoolRoundsFile = savedRoundsFile; + gPoolRounds = savedRounds; + gCurrentJob = savedJob; gHaveJob = savedHaveJob; @@ -1754,6 +2061,7 @@ void ThreadRPCServer(void*) } LoadPendingPayouts(); + LoadPoolRounds(); _beginthread(AcceptLoopFn, 0, NULL); _beginthread(ThreadBtfPoolAnnouncer, 0, NULL); -- cgit v1.2.3 From 64317cc5a3416d77c9171351c367ab8c24c4dff1 Mon Sep 17 00:00:00 2001 From: Bitflash Date: Mon, 10 Aug 2026 11:21:39 -0300 Subject: docs: specify the .btf rendezvous protocol --- docs/rendezvous.md | 288 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 288 insertions(+) create mode 100644 docs/rendezvous.md (limited to 'docs') diff --git a/docs/rendezvous.md b/docs/rendezvous.md new file mode 100644 index 0000000..ac8e598 --- /dev/null +++ b/docs/rendezvous.md @@ -0,0 +1,288 @@ +# Bitflash `.btf` Rendezvous Protocol + +This is the specification of how Bitflash nodes reach each other, and how miners +reach pools, without revealing an IP address and without any central directory. +It documents the protocol as implemented in `src/btfaddr.cpp`, `src/btfchan.cpp`, +`src/btfrv.cpp`, `src/btftunnel.cpp` and `src/nostr.cpp`. + +The design is ported from the Itzal hidden service design. A `.btf` address names +a public key, a node proves ownership of that key by signing a service +descriptor, other nodes find the descriptor over Nostr, and the two sides then +meet through an untrusted relay over an end to end encrypted tunnel. The relay +carries ciphertext only and never learns the plaintext. + +## 1. Overview + +A connection to a `.btf` service has three layers: + +1. **Discovery (Nostr).** The service publishes a signed descriptor as a Nostr + event. A caller fetches it by the service public key and verifies the + signature. The descriptor says which rendezvous relay the service is waiting + at and which encryption key to use. +2. **Rendezvous (TCP).** Caller and service both connect out to the same relay + and are paired by public key. Neither needs an inbound port or a public IP. + After pairing the relay is a blind byte pipe. +3. **Encrypted channel (X25519 + XChaCha20-Poly1305).** Over the paired pipe the + caller and service run an ephemeral to static handshake and then exchange + authenticated frames. The relay sees only ciphertext. + +The caller ends up holding an ordinary local socket that behaves like a direct +TCP connection to the peer. The node treats it exactly like any other peer +socket. + +## 2. Identities and keys + +A node holds two independent keypairs. + +- **Signing / identity key**: a secp256k1 x-only key (BIP340 Schnorr). This is + the `.btf` identity and also the node's Nostr identity. Persisted in + `nostr.key`. +- **Encryption key**: an X25519 (Curve25519) keypair. Persisted in + `btf_enc.key`. Its public half is published in the descriptor as `enc` and is + used for the tunnel key agreement. + +The identity key names the service and authenticates the descriptor. The +encryption key secures the tunnel. They are separate so the long term identity +never performs Diffie-Hellman and the encryption key can be rotated in a new +signed descriptor. + +## 3. The `.btf` address + +An address is the identity public key plus a short checksum, Base32 encoded, with +the suffix `.btf`. + +``` +label = Base32( pubkey[32] || checksum[2] ) +address = label + ".btf" +``` + +- `pubkey` is the 32 byte secp256k1 x-only public key. +- `checksum` is the first two bytes of `SHA256(".btf-checksum-v1" || pubkey)`. +- Base32 uses the lowercase alphabet `abcdefghijklmnopqrstuvwxyz234567`, no + padding. The 34 byte payload encodes to 55 characters. +- Parsing is case insensitive, trims surrounding whitespace, and strips a + trailing dot. A wrong checksum is rejected. + +The address is self certifying: it *is* the public key, so no registry has to be +trusted to map a name to a key. + +## 4. Discovery over Nostr + +Nodes publish and read three kinds of parameterized replaceable Nostr events. +Parameterized replaceable means the newest event for a given author and `d` tag +supersedes the older one, so a descriptor is always the latest state. + +| Purpose | kind | `d` tag | +|---|---|---| +| Service descriptor | 38501 | `btf-descriptor-2` | +| Rendezvous relay announcement | 38502 | `btf-relay-2` | +| Pool announcement | 38503 | `btf-pool-2` | + +The Nostr event is signed with the node's identity key, so the event author +equals the `.btf` public key. That binds the descriptor to the address at the +Nostr layer, and the descriptor is *also* signed inside its own content (see +below), so a relay operator cannot forge or swap a descriptor for an address it +does not control. + +**Resolving an address.** To reach `X.btf`, a caller parses the address to the 32 +byte key and sends a Nostr `REQ` filtered by `authors = [X]`, `kinds = [38501]`, +`#d = ["btf-descriptor-2"]`, `limit = 1`. It verifies the returned descriptor and +uses its `meeting_node` and `enc`. + +**Relay auto discovery.** A relay operator runs a rendezvous relay and publishes a +kind 38502 event carrying `host:port`. Every node that reads it adds the relay to +the set it can register at and dial through. No seed list edit or maintainer +approval is needed. A small curated seed list is still shipped for bootstrap: + +``` +92.246.128.180:8434 +31.44.4.249:8434 +90.156.222.107:8434 +``` + +The default rendezvous port is `8434`. + +## 5. The service descriptor + +The descriptor is a JSON object, signed with the identity key over a domain +separated digest of its own fields. + +```json +{ + "v": 2, + "pubkey": "", + "enc": "", + "meeting_node": "host:port", + "created": , + "sig": "" +} +``` + +The signed digest is: + +``` +digest = SHA256( ".btf-descriptor-v2" + || pubkey[32] + || enc (raw bytes of the string) + || meeting_node (raw bytes of the string) + || created (8 bytes, big endian) ) +``` + +The signature is a deterministic BIP340 Schnorr signature (no auxiliary +randomness) over `digest` by the identity key. + +**Verification** (untrusted input, must never throw): parse the JSON, require all +fields with the right types, confirm `pubkey` equals the address being resolved, +recompute the digest, and verify the Schnorr signature. Any failure rejects the +descriptor. + +`created` lets a reader prefer the freshest descriptor when more than one is +seen. A node only publishes a descriptor once it has actually registered at a +`meeting_node`, so it never advertises a relay it cannot be reached at. + +## 6. Rendezvous transport + +The relay is a small TCP server. Every connection opens with a fixed 33 byte +header: + +``` +byte 0 role: 'S' (service) or 'C' (client) +bytes 1..32 target public key (the .btf identity, 32 bytes) +``` + +**Service registration (`S`).** The service connects out to the relay and sends +`'S' || own_pubkey`. The relay stores the connection in a table keyed by the +public key and leaves it open, waiting. There is one slot per key: a new +registration for the same key replaces and closes the old one. The service does +not block waiting to be paired at registration time; it is considered registered +the moment the relay reads the header, which is what lets it publish a descriptor +pointing at this relay. + +**Client dial (`C`).** The caller connects out to the same relay and sends +`'C' || target_pubkey`. The relay looks up the waiting service: + +- If present, it removes it from the table, writes a single `0x01` byte to both + sockets, and then forwards raw bytes in both directions until either side + closes. From this point the relay is a blind pipe. +- If absent, it writes `0x00` to the caller and closes. + +Both sides connect *outbound* to the relay, so neither needs a public IP or an +open inbound port. The relay never sees the identity of the caller (the caller +sends no key of its own, only the target it wants). + +**Liveness.** A registration socket is idle by design and can outlive the network +path that created it (a NAT drops the idle mapping, the relay restarts, a route +changes) with no FIN and nothing readable. TCP keepalive was measured and does +not reliably detect this. The service therefore imposes its own timeout on the +wait for the pairing byte and re-registers on expiry. Expiring is not a failure; +it costs one reconnect per interval and returns the loop to a known state. + +## 7. The encrypted channel + +Once paired through the relay, the caller and service run an end to end handshake +and then exchange authenticated frames. The primitive is libsodium +`crypto_box_curve25519xchacha20poly1305` (X25519 key agreement, XChaCha20-Poly1305 +AEAD), the same construction as Rust's `crypto_box` ChaChaBox. + +**Handshake (ephemeral to static).** + +1. The caller generates a fresh ephemeral X25519 keypair, computes + `shared = X25519(eph_sk, service_enc_pub)` reduced to a box key + (`crypto_box_beforenm`), and sends the 32 byte ephemeral public key over the + paired pipe. +2. The service reads the ephemeral public key and computes the same + `shared = X25519(enc_sk, eph_pub)`. + +Because the caller uses a new ephemeral key per connection, each session has an +independent key and forward secrecy against later compromise of the service +encryption key. + +**Framing.** Every message is length prefixed: + +``` +frame = len[4] (big endian, length of the body that follows) + body = nonce[24] || ciphertext || mac[16] +``` + +The nonce is 24 random bytes per frame (XChaCha20), the MAC is 16 bytes +(Poly1305). A body shorter than `nonce + mac` or larger than 1 MiB is rejected. +Decryption that fails authentication drops the frame and tears the tunnel down. + +The tunnel is transparent: the application is handed a local socket pair end and +reads and writes plaintext. Two pump threads move data between that socket and +the relay socket, encrypting outbound and decrypting inbound. + +## 8. End to end connection flow + +To dial `X.btf`: + +1. Parse `X.btf` to the 32 byte identity key. +2. Fetch and verify the descriptor over Nostr (kind 38501). Obtain + `meeting_node` and `enc`. +3. Connect to `meeting_node`, send `'C' || X`, and wait for `0x01`. +4. Generate an ephemeral X25519 key, derive the shared key against `enc`, send + the ephemeral public key. +5. Exchange XChaCha20-Poly1305 frames. Hand the application a local socket that + speaks the ordinary peer protocol. + +A service, symmetrically: registers at a relay (`'S' || own_pubkey`), publishes a +descriptor pointing at that relay, waits for `0x01`, reads the caller ephemeral +public key, derives the shared key with its encryption secret, and serves the +tunnel. + +## 9. Security properties + +- **Address unforgeability.** The address is the identity key and the descriptor + is Schnorr signed by it, and the Nostr event is signed by it. A relay or Nostr + operator cannot publish a working descriptor for an address it does not own. +- **Relay confidentiality.** The relay forwards ciphertext and never holds the + channel key. It cannot read or modify traffic undetected; a tampered frame + fails the Poly1305 tag and the tunnel closes. +- **Caller anonymity to the service and relay.** The caller presents no identity + key, only the target it wants to reach. It connects outbound, so its address is + not published anywhere. +- **Forward secrecy.** The caller's per connection ephemeral key means a later + compromise of the service encryption key does not decrypt past sessions. +- **IP privacy.** Neither side needs a public IP or an inbound port. The observable + network fact is that each side has a TCP connection to a relay. + +## 10. Threat model and limitations + +- **The relay is trusted for availability, not for confidentiality.** It can + refuse to pair, drop a connection, or go offline. This is why relays are + plural, auto discovered, and re-registration is periodic. It cannot read + traffic. +- **Traffic analysis is not addressed.** A network observer, or the relay, can + see connection timing and byte volumes. There is no padding or mixing. +- **Single slot registration causes churn.** One waiting slot per key per relay + means a race or a stale registration can briefly make a service unreachable at + that relay. Multiple relays and the wait timeout bound the damage, but this is + a known source of connection churn. +- **Descriptor freshness is best effort.** `created` is a self reported + timestamp; readers prefer the newest descriptor they have seen but there is no + global clock. +- **Discovery depends on public Nostr relays.** If the relays a node uses are all + unreachable, discovery stalls until one is reachable again. Relay TLS is + verified, which closes a man in the middle path on discovery, but availability + still rests on the relay set. + +## 11. Wire reference + +| Item | Value | +|---|---| +| `.btf` Base32 alphabet | `abcdefghijklmnopqrstuvwxyz234567`, no padding | +| Address payload | `pubkey[32] || checksum[2]`, 55 Base32 chars | +| Address checksum | `SHA256(".btf-checksum-v1" || pubkey)[0..2]` | +| Descriptor domain sep | `.btf-descriptor-v2` | +| Descriptor version | `2` | +| Descriptor signature | BIP340 Schnorr, deterministic (no aux rand) | +| Nostr descriptor event | kind `38501`, `d = btf-descriptor-2` | +| Nostr relay event | kind `38502`, `d = btf-relay-2` | +| Nostr pool event | kind `38503`, `d = btf-pool-2` | +| Rendezvous port (default) | `8434` | +| Rendezvous header | `role[1] ('S'/'C') || pubkey[32]`, 33 bytes | +| Rendezvous pairing byte | `0x01` paired, `0x00` no service | +| Channel primitive | `crypto_box_curve25519xchacha20poly1305` | +| Nonce / MAC / key sizes | 24 / 16 / 32 bytes | +| Frame | `len[4 big endian] || nonce[24] || ct || mac[16]` | +| Max frame body | 1 MiB | -- cgit v1.2.3 From 5550481af68add9616a3410bb4db35abfdb70d9b Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Mon, 10 Aug 2026 12:31:38 -0300 Subject: tools: add fair launch verifier --- README.md | 11 + docs/fair-launch.md | 69 +++++++ scripts/verify-fair-launch.py | 452 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 532 insertions(+) create mode 100644 docs/fair-launch.md create mode 100644 scripts/verify-fair-launch.py (limited to 'docs') diff --git a/README.md b/README.md index ec8e9c4..b4c3b8b 100644 --- a/README.md +++ b/README.md @@ -73,6 +73,17 @@ scripts/verify-release.sh latest See [release verification](docs/release-verification.md) for the full release audit flow and the maintainer signing step. +Bitflash also ships a deterministic fair-launch verifier: + +```bash +python3 scripts/verify-fair-launch.py --max-blocks 1000 --out bitflash-fair-launch-report.json --json +``` + +The report reads the local chain directly, checks the genesis launch baseline, +and produces canonical JSON whose hash can be compared by independent auditors +or timestamped externally with OpenTimestamps. See +[fair-launch verification](docs/fair-launch.md). + **Keep your node current.** Consensus rules have changed since the first releases โ€” 1.2.1 fixed a bug that let anyone spend anyone's coins, and 1.2.2 added a per-block signature-operation cap. A node on an older build will accept diff --git a/docs/fair-launch.md b/docs/fair-launch.md new file mode 100644 index 0000000..79eca8e --- /dev/null +++ b/docs/fair-launch.md @@ -0,0 +1,69 @@ +# Bitflash fair-launch verifier + +Bitflash ships a deterministic fair-launch audit script: + +```bash +python3 scripts/verify-fair-launch.py --max-blocks 1000 --out bitflash-fair-launch-report.json --json +``` + +The verifier reads `blk*.dat` directly from the local Bitflash data directory. +It does not use Berkeley DB, does not call the node binary, and does not need +network access. + +The report proves the launch baseline that can be checked from the chain: + +- the genesis block hash matches the source constant; +- the genesis merkle root matches the source constant; +- the genesis transaction is a single coinbase transaction; +- the genesis output total is exactly `50.00000000 BTF`; +- the scanned block range and mined coinbase total are reproducible. + +For machines that do not have a synced data directory yet, the source constants +can be checked without disk blocks: + +```bash +python3 scripts/verify-fair-launch.py --reconstruct-genesis +``` + +## Deterministic output + +Use `--json` or `--out` for canonical JSON. The JSON is written with sorted +keys, fixed separators, ASCII escaping, and one trailing newline. It deliberately +does not include local paths or wall-clock generation time, so two auditors +scanning the same block range produce the same bytes. + +That makes the report shareable: + +```bash +python3 scripts/verify-fair-launch.py --max-blocks 1000 --out report.json --json +sha256sum report.json +``` + +If another auditor runs the same command against the same chain prefix, the +`sha256sum` should match. + +## OpenTimestamps + +OpenTimestamps is optional and external. It anchors the report in Bitcoin +without adding any bridge or dependency to Bitflash consensus. + +After generating a canonical JSON report: + +```bash +ots stamp report.json +ots verify report.json.ots +``` + +The `.ots` file proves that the exact report bytes existed no later than the +Bitcoin block that confirms the timestamp. + +## What this does not claim + +This is not a retroactive genesis commitment. The Bitflash genesis block already +exists, so adding a new on-chain commitment to genesis would create a different +chain. This verifier instead audits the launch state that is already on-chain +and makes the result reproducible and timestampable. + +It also does not prove real-world miner identity. It proves the genesis state, +the absence of non-coinbase premine in genesis, and the deterministic coinbase +totals over the scanned prefix. diff --git a/scripts/verify-fair-launch.py b/scripts/verify-fair-launch.py new file mode 100644 index 0000000..ce396fb --- /dev/null +++ b/scripts/verify-fair-launch.py @@ -0,0 +1,452 @@ +#!/usr/bin/env python3 +"""Produce a deterministic Bitflash fair-launch audit report. + +This verifier is intentionally self-contained. It parses Bitflash block files +directly, without Berkeley DB, without the node binary, and without network +access. The canonical JSON output is stable for the same inputs and can be +timestamped later with OpenTimestamps or any other external notary. +""" + +import argparse +import datetime as _dt +import hashlib +import json +import os +import struct +import sys +from pathlib import Path + + +MAGIC = bytes([0xBF, 0x20, 0x5C, 0xFD]) +COIN = 100_000_000 +GENESIS_HASH = "5bd7cb255d814e48cebcdfb72da4dc87b34bd774227f8ceb546c5640f4bdc169" +GENESIS_MERKLE = "1a45b4482532abb29b10e234d3f13132230525a339ecea91658ffa675a5b1325" +GENESIS_TIME = 1_753_315_200 +GENESIS_BITS = 0x1F0FFFFF +GENESIS_NONCE = 3141 +GENESIS_TEXT = "Bitflash 24/Jul/2026 Fair launch: one CPU one vote, no premine" + + +class ParseError(Exception): + pass + + +class Reader: + def __init__(self, data): + self.data = data + self.pos = 0 + + def take(self, n): + if self.pos + n > len(self.data): + raise ParseError("unexpected end of data") + out = self.data[self.pos:self.pos + n] + self.pos += n + return out + + def u8(self): + return self.take(1)[0] + + def u32(self): + return struct.unpack(" 1: + nxt = [] + for i in range(0, len(layer), 2): + left = layer[i] + right = layer[i + 1] if i + 1 < len(layer) else left + nxt.append(sha256d(left + right)) + layer = nxt + return layer[0][::-1].hex() + + +def parse_block(payload, file_name="", offset=0): + r = Reader(payload) + header = r.take(80) + version, = struct.unpack(" len(data): + break + size = struct.unpack(" len(data): + break + blocks.append(parse_block(data[start:end], path.name, start)) + if limit and len(blocks) >= limit: + return blocks + pos = end + return blocks + + +def genesis_transaction(): + script_sig = ( + bytes.fromhex("04ffff001d") + + b"\x01\x04" + + bytes([len(GENESIS_TEXT)]) + + GENESIS_TEXT.encode("ascii") + ) + pubkey_num = bytes.fromhex( + "5f1df16b2b704c8a578d0bbaf74d385cde12c11ee50455f3c438ef4c3fbcf649" + "b6de611feae06279a60939e028a8d65c10b73071a6f16719274855feb0fd8a6704" + ) + # The genesis source uses CScript << CBigNum("0x..."), and CBigNum::getvch() + # returns the minimal little-endian script number representation. + script_pub_key = bytes([65]) + pubkey_num[::-1] + b"\xac" + tx = { + "version": 1, + "vin": [{ + "prevout_hash": "0" * 64, + "prevout_n": 0xFFFFFFFF, + "script_sig_hex": script_sig.hex(), + "sequence": 0xFFFFFFFF, + }], + "vout": [{ + "value_satoshis": 50 * COIN, + "value_btf": format_money(50 * COIN), + "script_pub_key_hex": script_pub_key.hex(), + }], + "lock_time": 0, + } + tx["txid"] = hash_hex(ser_tx(tx)) + return tx + + +def reconstructed_genesis(): + tx = genesis_transaction() + header = ( + struct.pack("") + + +def main(argv): + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--datadir", default=default_datadir(), help="Bitflash data directory") + ap.add_argument("--max-blocks", type=int, default=100, help="number of blocks to scan from blk*.dat") + ap.add_argument("--json", action="store_true", help="print canonical JSON") + ap.add_argument("--out", help="write canonical JSON to this path") + ap.add_argument("--reconstruct-genesis", action="store_true", + help="do not read disk; reconstruct only the genesis block from source constants") + args = ap.parse_args(argv) + + if args.reconstruct_genesis: + blocks = [reconstructed_genesis()] + source_mode = "reconstructed-genesis" + datadir = "" + else: + datadir = Path(args.datadir) + blocks = read_blocks(datadir, args.max_blocks) + if not blocks: + raise SystemExit("no Bitflash blocks found in %s; use --reconstruct-genesis for a source-only check" % datadir) + source_mode = "local-block-files" + + report = build_report(blocks, source_mode) + data = canonical_bytes(report) + + if args.out: + Path(args.out).write_bytes(data) + if args.json: + sys.stdout.buffer.write(data) + else: + print_text(report) + return 0 if report["verdict"] == "FAIR_LAUNCH_BASELINE_VERIFIED" else 1 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv[1:])) -- cgit v1.2.3 From ccf4489dbdf9e9d691e02e867562518c92168007 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Mon, 10 Aug 2026 14:49:21 -0300 Subject: tools: add deterministic UTXO commitment verifier --- README.md | 11 ++ docs/utxo-commitment.md | 93 +++++++++ scripts/verify-utxo-set.py | 464 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 568 insertions(+) create mode 100644 docs/utxo-commitment.md create mode 100644 scripts/verify-utxo-set.py (limited to 'docs') diff --git a/README.md b/README.md index ec8e9c4..0827ac1 100644 --- a/README.md +++ b/README.md @@ -73,6 +73,17 @@ scripts/verify-release.sh latest See [release verification](docs/release-verification.md) for the full release audit flow and the maintainer signing step. +Bitflash also ships a deterministic UTXO-set commitment tool: + +```bash +python3 scripts/verify-utxo-set.py --out utxo-report.json --json +``` + +It reconstructs the best chain from local block files, computes the current +UTXO root and supply, and writes canonical JSON that independent auditors can +compare or timestamp externally with OpenTimestamps. See +[UTXO-set commitments](docs/utxo-commitment.md). + **Keep your node current.** Consensus rules have changed since the first releases โ€” 1.2.1 fixed a bug that let anyone spend anyone's coins, and 1.2.2 added a per-block signature-operation cap. A node on an older build will accept diff --git a/docs/utxo-commitment.md b/docs/utxo-commitment.md new file mode 100644 index 0000000..ddf5450 --- /dev/null +++ b/docs/utxo-commitment.md @@ -0,0 +1,93 @@ +# Bitflash UTXO-set commitment + +`scripts/verify-utxo-set.py` reconstructs a deterministic commitment to the +current UTXO set from local `blk*.dat` files. + +This is intentionally not a consensus rule yet. It is an audit tool: independent +operators can run it against the same chain prefix and compare the resulting +root and canonical JSON hash. + +```bash +python3 scripts/verify-utxo-set.py --max-blocks 1000 --out utxo-report.json --json +``` + +Use `--max-blocks 0` or omit the option to scan every available block file: + +```bash +python3 scripts/verify-utxo-set.py --out utxo-report.json --json +``` + +## What It Verifies + +The verifier: + +- parses Bitflash block files directly; +- reconstructs the best chain rooted at the Bitflash genesis block; +- ignores side blocks that may also be present in `blk*.dat`; +- verifies every scanned block merkle root; +- applies spends and creates outputs in deterministic order; +- reports current UTXO count, current supply, mined coinbase total, fees, and + historical duplicate-output overwrites; +- computes a deterministic UTXO root. + +## Commitment Algorithm + +The UTXO map is keyed by: + +```text +txid:vout +``` + +Keys are sorted lexicographically. Each leaf is: + +```text +SHA256("BTFUTXO1|" || outpoint || "|" || value_satoshis || "|" || + script_pub_key_hex || "|" || created_height || "|" || created_txid) +``` + +The tree is then reduced pairwise. If a level has an odd number of hashes, the +last hash is duplicated. Each parent is: + +```text +SHA256("BTFNODE1|" || left_hash || right_hash) +``` + +The final hash is the UTXO-set root. + +## Deterministic Output + +`--json` and `--out` write canonical JSON with sorted keys, fixed separators, +ASCII escaping, and one trailing newline. It contains no local path and no +generation timestamp, so two auditors scanning the same chain prefix should +produce byte-identical reports. + +```bash +python3 scripts/verify-utxo-set.py --out utxo-report.json --json +sha256sum utxo-report.json +``` + +## OpenTimestamps + +The canonical JSON can be timestamped in Bitcoin without adding any dependency +to Bitflash consensus: + +```bash +ots stamp utxo-report.json +ots verify utxo-report.json.ots +``` + +## Historical Duplicates + +Bitflash inherited early Bitcoin behavior around duplicate transaction IDs. +The verifier reports duplicate-output overwrites as warnings and uses +last-write-wins semantics for the UTXO map, matching the legacy index model. +Those warnings are part of the report so auditors can see the historical debt +instead of relying on a polished summary. + +## Future Consensus Path + +This tool is the low-risk first step toward consensus commitments. A future +hard-fork design can commit this root, or a Utreexo-style accumulator root, in +coinbase or block headers after a fixed activation height. Until then this +script provides reproducible supply and state commitments without changing what +nodes accept. diff --git a/scripts/verify-utxo-set.py b/scripts/verify-utxo-set.py new file mode 100644 index 0000000..12b024f --- /dev/null +++ b/scripts/verify-utxo-set.py @@ -0,0 +1,464 @@ +#!/usr/bin/env python3 +"""Build a deterministic Bitflash UTXO-set commitment from local blk*.dat. + +This is deliberately outside consensus. It lets auditors reconstruct the chain +state from disk blocks and compare the same canonical UTXO root independently. +""" + +import argparse +import datetime as _dt +import hashlib +import json +import os +import struct +import sys +from pathlib import Path + + +MAGIC = bytes([0xBF, 0x20, 0x5C, 0xFD]) +COIN = 100_000_000 +GENESIS_HASH = "5bd7cb255d814e48cebcdfb72da4dc87b34bd774227f8ceb546c5640f4bdc169" +GENESIS_MERKLE = "1a45b4482532abb29b10e234d3f13132230525a339ecea91658ffa675a5b1325" + + +class ParseError(Exception): + pass + + +class Reader: + def __init__(self, data): + self.data = data + self.pos = 0 + + def take(self, n): + if self.pos + n > len(self.data): + raise ParseError("unexpected end of data") + out = self.data[self.pos:self.pos + n] + self.pos += n + return out + + def u8(self): + return self.take(1)[0] + + def u32(self): + return struct.unpack(" 1: + nxt = [] + for i in range(0, len(layer), 2): + left = layer[i] + right = layer[i + 1] if i + 1 < len(layer) else left + nxt.append(sha256d(left + right)) + layer = nxt + return layer[0][::-1].hex() + + +def parse_block(payload, file_name="", offset=0): + r = Reader(payload) + header = r.take(80) + version, = struct.unpack(" len(data): + break + size = struct.unpack(" len(data): + break + blocks.append(parse_block(data[start:end], path.name, start)) + if max_blocks and len(blocks) >= max_blocks: + return blocks + pos = end + return blocks + + +def select_main_chain(blocks): + by_hash = {} + for block in blocks: + by_hash.setdefault(block["hash"], block) + if GENESIS_HASH not in by_hash: + raise ParseError("genesis block not found in block files") + + children = {} + for block in by_hash.values(): + children.setdefault(block["previous_hash"], []).append(block) + for v in children.values(): + v.sort(key=lambda b: (b["hash"], b["source_file"], b["source_offset"])) + + best = [] + stack = [(by_hash[GENESIS_HASH], [by_hash[GENESIS_HASH]])] + while stack: + block, path = stack.pop() + if len(path) > len(best) or (len(path) == len(best) and block["hash"] < best[-1]["hash"]): + best = path + for child in children.get(block["hash"], []): + stack.append((child, path + [child])) + return best + + +def is_coinbase(tx): + return ( + len(tx["vin"]) == 1 + and tx["vin"][0]["prevout_hash"] == "0" * 64 + and tx["vin"][0]["prevout_n"] == 0xFFFFFFFF + ) + + +def utxo_leaf_hash(outpoint, utxo): + # Text fields are length-delimited by separators and all variable data is + # hex or decimal ASCII. That keeps the commitment language-neutral. + payload = "|".join([ + "BTFUTXO1", + outpoint, + str(utxo["value_satoshis"]), + utxo["script_pub_key_hex"], + str(utxo["created_height"]), + utxo["created_txid"], + ]).encode("ascii") + return sha256(payload) + + +def utxo_root(utxos): + leaves = [utxo_leaf_hash(k, utxos[k]) for k in sorted(utxos)] + if not leaves: + return sha256(b"BTFUTXO1|empty").hex() + layer = leaves + while len(layer) > 1: + nxt = [] + for i in range(0, len(layer), 2): + left = layer[i] + right = layer[i + 1] if i + 1 < len(layer) else left + nxt.append(sha256(b"BTFNODE1|" + left + right)) + layer = nxt + return layer[0].hex() + + +def apply_blocks(blocks): + utxos = {} + spent = 0 + missing_spends = [] + duplicate_outputs = [] + total_coinbase = 0 + total_fees = 0 + normal_txs = 0 + + for height, block in enumerate(blocks): + if block["computed_merkle_root"] != block["merkle_root"]: + raise ParseError("bad merkle root at height %d" % height) + for tx_index, tx in enumerate(block["transactions"]): + coinbase = is_coinbase(tx) + input_value = 0 + output_value = sum(out["value_satoshis"] for out in tx["vout"]) + if coinbase: + total_coinbase += output_value + else: + normal_txs += 1 + for txin in tx["vin"]: + prev = "%s:%d" % (txin["prevout_hash"], txin["prevout_n"]) + prev_utxo = utxos.pop(prev, None) + if prev_utxo is None: + missing_spends.append({ + "height": height, + "txid": tx["txid"], + "missing_outpoint": prev, + }) + else: + spent += prev_utxo["value_satoshis"] + input_value += prev_utxo["value_satoshis"] + if input_value >= output_value: + total_fees += input_value - output_value + for n, out in enumerate(tx["vout"]): + key = "%s:%d" % (tx["txid"], n) + if key in utxos: + duplicate_outputs.append({"height": height, "outpoint": key}) + utxos[key] = { + "txid": tx["txid"], + "vout": n, + "value_satoshis": out["value_satoshis"], + "value_btf": out["value_btf"], + "script_pub_key_hex": out["script_pub_key_hex"], + "created_height": height, + "created_block_hash": block["hash"], + "created_txid": tx["txid"], + "coinbase": coinbase, + } + + total_unspent = sum(u["value_satoshis"] for u in utxos.values()) + return { + "utxos": utxos, + "stats": { + "utxo_count": len(utxos), + "spent_outputs": spent, + "total_unspent_satoshis": total_unspent, + "total_unspent_btf": format_money(total_unspent), + "total_coinbase_satoshis": total_coinbase, + "total_coinbase_btf": format_money(total_coinbase), + "coinbase_minus_unspent_satoshis": total_coinbase - total_unspent, + "coinbase_minus_unspent_btf": format_money(total_coinbase - total_unspent), + "total_fees_satoshis": total_fees, + "total_fees_btf": format_money(total_fees), + "normal_transactions": normal_txs, + "missing_spends": missing_spends, + "duplicate_outputs": duplicate_outputs, + }, + } + + +def format_money(value): + sign = "-" if value < 0 else "" + value = abs(value) + return "%s%d.%08d" % (sign, value // COIN, value % COIN) + + +def utc(ts): + return _dt.datetime.fromtimestamp(ts, _dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") + + +def default_datadir(): + if os.name == "nt" and os.environ.get("APPDATA"): + return os.path.join(os.environ["APPDATA"], "Bitflash") + if os.environ.get("HOME"): + return os.path.join(os.environ["HOME"], ".bitflash") + return "." + + +def canonical_bytes(report): + return (json.dumps(report, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii") + + +def build_report(raw_blocks, include_utxos=False): + blocks = select_main_chain(raw_blocks) + state = apply_blocks(blocks) + utxos = state["utxos"] + stats = state["stats"] + tip = blocks[-1] + root = utxo_root(utxos) + checks = { + "genesis_hash_matches_source": blocks[0]["hash"] == GENESIS_HASH, + "genesis_merkle_matches_source": blocks[0]["merkle_root"] == GENESIS_MERKLE, + "all_scanned_merkle_roots_match": all(b["computed_merkle_root"] == b["merkle_root"] for b in blocks), + "no_missing_spends": len(stats["missing_spends"]) == 0, + } + warnings = { + "legacy_duplicate_output_overwrites": len(stats["duplicate_outputs"]), + } + report = { + "schema": "bitflash-utxo-set-commitment-1", + "deterministic": True, + "chain": { + "name": "Bitflash", + "ticker": "BTF", + "message_start_hex": MAGIC.hex(), + "coin": COIN, + }, + "commitment": { + "algorithm": "sorted-outpoint-merkle-sha256", + "leaf_domain": "BTFUTXO1", + "node_domain": "BTFNODE1", + "root": root, + }, + "scanned": { + "raw_block_count": len(raw_blocks), + "side_blocks_ignored": len(raw_blocks) - len(blocks), + "main_chain_block_count": len(blocks), + "first_height": 0, + "last_height": len(blocks) - 1, + "tip_hash": tip["hash"], + "tip_time": tip["time"], + "tip_time_utc": tip["time_utc"], + }, + "stats": stats, + "checks": checks, + "warnings": warnings, + "verdict": "UTXO_COMMITMENT_VERIFIED" if all(checks.values()) else "CHECKS_FAILED", + } + if include_utxos: + report["utxos"] = {k: utxos[k] for k in sorted(utxos)} + return report + + +def print_text(report): + data_hash = hashlib.sha256(canonical_bytes(report)).hexdigest() + print("Bitflash UTXO-set commitment verifier") + print("schema: %s" % report["schema"]) + print("canonical report sha256: %s" % data_hash) + print() + print("Commitment") + print(" root: %s" % report["commitment"]["root"]) + print(" algo: %s" % report["commitment"]["algorithm"]) + print() + print("Scanned") + print(" main chain blocks: %d" % report["scanned"]["main_chain_block_count"]) + print(" raw disk blocks: %d" % report["scanned"]["raw_block_count"]) + print(" side ignored: %d" % report["scanned"]["side_blocks_ignored"]) + print(" height: %d..%d" % (report["scanned"]["first_height"], report["scanned"]["last_height"])) + print(" tip: %s" % report["scanned"]["tip_hash"]) + print() + print("Supply") + print(" utxos: %d" % report["stats"]["utxo_count"]) + print(" supply: %s BTF" % report["stats"]["total_unspent_btf"]) + print(" mined: %s BTF" % report["stats"]["total_coinbase_btf"]) + print(" delta: %s BTF" % report["stats"]["coinbase_minus_unspent_btf"]) + print(" fees: %s BTF" % report["stats"]["total_fees_btf"]) + if report["warnings"]["legacy_duplicate_output_overwrites"]: + print(" legacy duplicate overwrites: %d" % report["warnings"]["legacy_duplicate_output_overwrites"]) + print() + print("Checks") + for key in sorted(report["checks"]): + print(" %s %s" % ("OK " if report["checks"][key] else "FAIL", key)) + print() + print("Verdict: %s" % report["verdict"]) + print() + print("OpenTimestamps") + print(" canonical JSON can be stamped later with: ots stamp ") + + +def main(argv): + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--datadir", default=default_datadir(), help="Bitflash data directory") + ap.add_argument("--max-blocks", type=int, default=0, + help="number of blocks to scan; 0 means all block files") + ap.add_argument("--include-utxos", action="store_true", + help="include the full sorted UTXO map in JSON output") + ap.add_argument("--json", action="store_true", help="print canonical JSON") + ap.add_argument("--out", help="write canonical JSON to this path") + args = ap.parse_args(argv) + + blocks = read_blocks(args.datadir, args.max_blocks) + if not blocks: + raise SystemExit("no Bitflash blocks found in %s" % args.datadir) + report = build_report(blocks, include_utxos=args.include_utxos) + data = canonical_bytes(report) + if args.out: + Path(args.out).write_bytes(data) + if args.json: + sys.stdout.buffer.write(data) + else: + print_text(report) + return 0 if report["verdict"] == "UTXO_COMMITMENT_VERIFIED" else 1 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv[1:])) -- cgit v1.2.3 From 7265a381405125aad5d91c3a11e9b619c9518c10 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Mon, 10 Aug 2026 16:47:44 -0300 Subject: tools: add Bitflash mining calculator --- README.md | 13 +++ docs/mining-calculator.md | 86 ++++++++++++++ scripts/bitflash-mining-calculator.py | 206 ++++++++++++++++++++++++++++++++++ 3 files changed, 305 insertions(+) create mode 100644 docs/mining-calculator.md create mode 100644 scripts/bitflash-mining-calculator.py (limited to 'docs') diff --git a/README.md b/README.md index ec8e9c4..d2e969a 100644 --- a/README.md +++ b/README.md @@ -237,6 +237,19 @@ SRBMiner-MULTI --algorithm randomx --pool 127.0.0.1:3333 --wallet YOUR_BTF_ADDRE xmrig -a rx/0 -o 127.0.0.1:3333 -u YOUR_BTF_ADDRESS -p x ``` +Estimate rewards and electricity cost locally: + +```bash +python3 scripts/bitflash-mining-calculator.py \ + --hashrate 21 --hashrate-unit kh/s \ + --network-hashrate 1.56 --network-hashrate-unit mh/s \ + --watts 140 --kwh-cost 0.10 \ + --price 0.001 --pool-fee 1 +``` + +See [mining calculator](docs/mining-calculator.md). Price is manual until +Bitflash has a reliable public market. + --- ## Headless / server mode diff --git a/docs/mining-calculator.md b/docs/mining-calculator.md new file mode 100644 index 0000000..352eea1 --- /dev/null +++ b/docs/mining-calculator.md @@ -0,0 +1,86 @@ +# Bitflash mining calculator + +`scripts/bitflash-mining-calculator.py` is a local WhatToMine-style calculator +for Bitflash. + +It does not fetch market prices, does not talk to exchanges, and does not +change consensus. Price is a manual input until Bitflash has a reliable public +market. + +## Examples + +Estimate from network hashrate: + +```bash +python3 scripts/bitflash-mining-calculator.py \ + --hashrate 21 --hashrate-unit kh/s \ + --network-hashrate 1.56 --network-hashrate-unit mh/s \ + --watts 140 --kwh-cost 0.10 \ + --price 0.001 --pool-fee 1 +``` + +Estimate from Bitflash difficulty: + +```bash +python3 scripts/bitflash-mining-calculator.py \ + --hashrate 21 --hashrate-unit kh/s \ + --difficulty 45600 \ + --watts 140 --kwh-cost 0.10 +``` + +Machine-readable output: + +```bash +python3 scripts/bitflash-mining-calculator.py \ + --hashrate 21 --hashrate-unit kh/s \ + --network-hashrate 1.56 --network-hashrate-unit mh/s \ + --json +``` + +## Bitflash Difficulty Scale + +Bitflash does not use Bitcoin's SHA256 difficulty-1 constant for pool shares. +The chain's easiest RandomX target is `bnProofOfWorkLimit = ~uint256(0) >> 12`, +so difficulty 1 expects about `4096` hashes. + +The calculator uses: + +```text +network_hashrate = difficulty * 4096 / block_time_seconds +``` + +At the default two-minute block target: + +```text +network_hashrate = difficulty * 4096 / 120 +``` + +If you already know network hashrate, pass `--network-hashrate` directly and +the script derives the equivalent Bitflash difficulty for display. + +## Reward Estimate + +The expected daily reward is: + +```text +blocks_per_day = 86400 / block_time_seconds +miner_share = miner_hashrate / network_hashrate +gross_btf_per_day = blocks_per_day * block_reward * miner_share +net_btf_per_day = gross_btf_per_day * (1 - pool_fee_percent / 100) +``` + +Electricity is: + +```text +electricity_per_day = watts / 1000 * 24 * kwh_cost +``` + +This is an expectation, not a promise. Solo mining variance can be very large, +especially when the miner has a small share of the network. + +## Future Website Use + +The script is intentionally standalone so the same math can later back a +website widget or status dashboard. Until a public price exists, a website +calculator should keep price as a manual field and label fiat profit as +speculative. diff --git a/scripts/bitflash-mining-calculator.py b/scripts/bitflash-mining-calculator.py new file mode 100644 index 0000000..1fcc1a9 --- /dev/null +++ b/scripts/bitflash-mining-calculator.py @@ -0,0 +1,206 @@ +#!/usr/bin/env python3 +"""Estimate Bitflash mining rewards and electricity cost. + +This is a local WhatToMine-style calculator for Bitflash. It does not fetch +prices, does not talk to the node, and does not affect consensus. +""" + +import argparse +import json +import math +import sys + + +HASHES_AT_DIFF_1 = 4096.0 +DEFAULT_BLOCK_TIME = 120.0 +DEFAULT_BLOCK_REWARD = 50.0 + + +UNITS = { + "h": 1.0, + "h/s": 1.0, + "hs": 1.0, + "kh": 1_000.0, + "kh/s": 1_000.0, + "khs": 1_000.0, + "mh": 1_000_000.0, + "mh/s": 1_000_000.0, + "mhs": 1_000_000.0, + "gh": 1_000_000_000.0, + "gh/s": 1_000_000_000.0, + "ghs": 1_000_000_000.0, +} + + +def parse_hashrate(value, unit): + try: + n = float(value) + except ValueError: + raise SystemExit("invalid hashrate: %s" % value) + mul = UNITS.get(unit.lower()) + if mul is None: + raise SystemExit("unknown hashrate unit: %s" % unit) + if n < 0: + raise SystemExit("hashrate must not be negative") + return n * mul + + +def fmt_hashrate(hps): + units = [("GH/s", 1_000_000_000.0), ("MH/s", 1_000_000.0), ("kH/s", 1_000.0), ("H/s", 1.0)] + for name, div in units: + if hps >= div: + return "%.4g %s" % (hps / div, name) + return "%.4g H/s" % hps + + +def fmt_money(x): + return "%.8f" % x + + +def estimate(args): + miner_hps = parse_hashrate(args.hashrate, args.hashrate_unit) + if args.network_hashrate is not None: + network_hps = parse_hashrate(args.network_hashrate, args.network_hashrate_unit) + difficulty = network_hps * args.block_time / HASHES_AT_DIFF_1 + network_source = "network_hashrate" + else: + if args.difficulty is None: + raise SystemExit("provide either --network-hashrate or --difficulty") + difficulty = float(args.difficulty) + if difficulty <= 0: + raise SystemExit("difficulty must be positive") + network_hps = difficulty * HASHES_AT_DIFF_1 / args.block_time + network_source = "difficulty" + + if network_hps <= 0: + raise SystemExit("network hashrate must be positive") + + block_reward = float(args.block_reward) + block_time = float(args.block_time) + pool_fee_percent = float(args.pool_fee) + price = float(args.price) + watts = float(args.watts) + kwh_cost = float(args.kwh_cost) + if block_reward < 0 or block_time <= 0 or pool_fee_percent < 0 or price < 0 or watts < 0 or kwh_cost < 0: + raise SystemExit("numeric inputs must be non-negative; block time must be positive") + + share = miner_hps / network_hps + blocks_per_day = 86400.0 / block_time + gross_btf_day = blocks_per_day * block_reward * share + net_btf_day = gross_btf_day * max(0.0, 1.0 - pool_fee_percent / 100.0) + revenue_day = net_btf_day * price + cost_day = watts / 1000.0 * 24.0 * kwh_cost + profit_day = revenue_day - cost_day + expected_seconds_per_block = math.inf if miner_hps <= 0 else difficulty * HASHES_AT_DIFF_1 / miner_hps + + periods = { + "hour": 1.0 / 24.0, + "day": 1.0, + "week": 7.0, + "month_30d": 30.0, + } + rows = {} + for name, days in periods.items(): + rows[name] = { + "btf": net_btf_day * days, + "revenue": revenue_day * days, + "electricity_cost": cost_day * days, + "profit": profit_day * days, + } + + return { + "schema": "bitflash-mining-calculator-1", + "network_source": network_source, + "inputs": { + "miner_hashrate_hps": miner_hps, + "miner_hashrate": fmt_hashrate(miner_hps), + "network_hashrate_hps": network_hps, + "network_hashrate": fmt_hashrate(network_hps), + "difficulty": difficulty, + "block_time_seconds": block_time, + "block_reward_btf": block_reward, + "pool_fee_percent": pool_fee_percent, + "price_per_btf": price, + "power_watts": watts, + "electricity_per_kwh": kwh_cost, + }, + "estimates": { + "miner_network_share": share, + "blocks_per_day": blocks_per_day, + "expected_time_to_solo_block_seconds": expected_seconds_per_block, + "expected_time_to_solo_block_hours": expected_seconds_per_block / 3600.0, + "gross_btf_per_day": gross_btf_day, + "net_btf_per_day": net_btf_day, + "periods": rows, + }, + } + + +def print_text(result): + i = result["inputs"] + e = result["estimates"] + print("Bitflash mining calculator") + print("schema: %s" % result["schema"]) + print() + print("Network") + print(" source: %s" % result["network_source"]) + print(" hashrate: %s" % i["network_hashrate"]) + print(" difficulty: %.8f" % i["difficulty"]) + print(" block time: %.0fs" % i["block_time_seconds"]) + print(" reward: %s BTF" % fmt_money(i["block_reward_btf"])) + print() + print("Miner") + print(" hashrate: %s" % i["miner_hashrate"]) + print(" share: %.8f%%" % (e["miner_network_share"] * 100.0)) + if math.isinf(e["expected_time_to_solo_block_seconds"]): + print(" solo block: never") + else: + print(" solo block: %.2f hours expected average" % e["expected_time_to_solo_block_hours"]) + print() + print("Economics") + print(" price: $%.8f / BTF" % i["price_per_btf"]) + print(" power: %.2f W" % i["power_watts"]) + print(" kWh cost: $%.4f" % i["electricity_per_kwh"]) + print(" pool fee: %.2f%%" % i["pool_fee_percent"]) + print() + print("Estimated rewards") + print(" %-9s %16s %12s %12s %12s" % ("period", "BTF", "revenue", "power", "profit")) + for name in ["hour", "day", "week", "month_30d"]: + row = e["periods"][name] + print(" %-9s %16s $%11.4f $%11.4f $%11.4f" % ( + name, + fmt_money(row["btf"]), + row["revenue"], + row["electricity_cost"], + row["profit"], + )) + print() + print("Note: this is an expectation, not a promise. Solo mining variance can be huge.") + + +def main(argv): + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--hashrate", required=True, help="miner hashrate number") + ap.add_argument("--hashrate-unit", default="kh/s", choices=sorted(UNITS), help="miner hashrate unit") + ap.add_argument("--network-hashrate", help="network hashrate number") + ap.add_argument("--network-hashrate-unit", default="kh/s", choices=sorted(UNITS), help="network hashrate unit") + ap.add_argument("--difficulty", type=float, help="Bitflash difficulty; diff 1 expects 4096 hashes") + ap.add_argument("--block-time", type=float, default=DEFAULT_BLOCK_TIME, help="target block time in seconds") + ap.add_argument("--block-reward", type=float, default=DEFAULT_BLOCK_REWARD, help="block reward in BTF") + ap.add_argument("--pool-fee", type=float, default=0.0, help="pool fee percent") + ap.add_argument("--price", type=float, default=0.0, help="price per BTF in USD or your chosen fiat unit") + ap.add_argument("--watts", type=float, default=0.0, help="miner or whole-system power in watts") + ap.add_argument("--kwh-cost", type=float, default=0.0, help="electricity cost per kWh") + ap.add_argument("--json", action="store_true", help="print JSON") + args = ap.parse_args(argv) + + result = estimate(args) + if args.json: + print(json.dumps(result, sort_keys=True, indent=2)) + else: + print_text(result) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv[1:])) -- cgit v1.2.3 From 034bc9a9497a336239aadb2e52d620661e90e536 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Tue, 11 Aug 2026 09:31:58 -0300 Subject: tools: add UTXO inclusion proofs --- README.md | 10 +++ docs/utxo-commitment.md | 14 +++ docs/utxo-proofs.md | 61 +++++++++++++ scripts/prove-utxo.py | 202 +++++++++++++++++++++++++++++++++++++++++++ scripts/verify-utxo-proof.py | 163 ++++++++++++++++++++++++++++++++++ 5 files changed, 450 insertions(+) create mode 100644 docs/utxo-proofs.md create mode 100644 scripts/prove-utxo.py create mode 100644 scripts/verify-utxo-proof.py (limited to 'docs') diff --git a/README.md b/README.md index d549337..c28b0f1 100644 --- a/README.md +++ b/README.md @@ -83,6 +83,16 @@ It reconstructs the best chain from local block files, computes the current UTXO root and supply, and writes canonical JSON that independent auditors can compare or timestamp externally with OpenTimestamps. See [UTXO-set commitments](docs/utxo-commitment.md). + +The same root can prove one unspent output without sharing the full UTXO set: + +```bash +python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json +python3 scripts/verify-utxo-proof.py utxo-proof.json +``` + +See [UTXO inclusion proofs](docs/utxo-proofs.md). + Bitflash also ships a deterministic fair-launch verifier: ```bash diff --git a/docs/utxo-commitment.md b/docs/utxo-commitment.md index ddf5450..991e5d9 100644 --- a/docs/utxo-commitment.md +++ b/docs/utxo-commitment.md @@ -54,6 +54,20 @@ SHA256("BTFNODE1|" || left_hash || right_hash) The final hash is the UTXO-set root. +## Inclusion Proofs + +After a root has been computed, `scripts/prove-utxo.py` can create a compact +inclusion proof for one unspent output: + +```bash +python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json +python3 scripts/verify-utxo-proof.py utxo-proof.json +``` + +The verifier does not need local block files. It recomputes the target leaf, +walks the sibling path, and compares the result with the expected root. See +[UTXO inclusion proofs](utxo-proofs.md). + ## Deterministic Output `--json` and `--out` write canonical JSON with sorted keys, fixed separators, diff --git a/docs/utxo-proofs.md b/docs/utxo-proofs.md new file mode 100644 index 0000000..4947c3c --- /dev/null +++ b/docs/utxo-proofs.md @@ -0,0 +1,61 @@ +# Bitflash UTXO inclusion proofs + +`scripts/prove-utxo.py` creates a deterministic inclusion proof for one +currently unspent output. `scripts/verify-utxo-proof.py` verifies that proof +against the UTXO root produced by `scripts/verify-utxo-set.py`. + +This is intentionally outside consensus. It turns the audit root into something +useful for a specific coin without changing what nodes accept. + +## Create A Proof + +Use an unspent output in `txid:vout` form: + +```bash +python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json +``` + +The proof contains: + +- the target UTXO; +- its sorted leaf index; +- the leaf hash; +- the Merkle sibling path; +- the UTXO root and chain tip used to create it. + +It does not contain the full UTXO set. + +## Verify A Proof + +Verify against the root embedded in the proof: + +```bash +python3 scripts/verify-utxo-proof.py utxo-proof.json +``` + +Or verify against a root copied from another machine, dashboard, block explorer, +or future node export: + +```bash +python3 scripts/verify-utxo-proof.py utxo-proof.json \ + --root ROOT_FROM_REPORT_OR_EXPORT +``` + +The verifier does not need local `blk*.dat` files. It only needs the proof and +the expected root. + +## Scope + +This first proof format supports inclusion only. Absence and range proofs are +left for a later design because they require a more careful commitment scheme +and are easier to get subtly wrong. + +The proof uses the exact same leaf and node domains as the UTXO commitment: + +```text +BTFUTXO1 +BTFNODE1 +``` + +That keeps the verifier language-neutral and gives future implementations a +small, stable target. diff --git a/scripts/prove-utxo.py b/scripts/prove-utxo.py new file mode 100644 index 0000000..7a10d2a --- /dev/null +++ b/scripts/prove-utxo.py @@ -0,0 +1,202 @@ +#!/usr/bin/env python3 +"""Create a Bitflash UTXO inclusion proof from local blk*.dat files. + +The proof is deliberately outside consensus. It proves that one txid:vout is +included in the deterministic UTXO root produced by verify-utxo-set.py. +""" + +import argparse +import hashlib +import importlib.util +import json +import os +import sys +from pathlib import Path + + +SCHEMA = "bitflash-utxo-inclusion-proof-1" +NODE_DOMAIN = b"BTFNODE1|" + + +def load_utxo_tool(): + path = Path(__file__).with_name("verify-utxo-set.py") + spec = importlib.util.spec_from_file_location("bitflash_verify_utxo_set", path) + if spec is None or spec.loader is None: + raise SystemExit("could not load %s" % path) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod + + +def canonical_bytes(obj): + return (json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii") + + +def split_outpoint(outpoint): + parts = outpoint.split(":") + if len(parts) != 2 or len(parts[0]) != 64: + raise SystemExit("outpoint must be txid:vout") + try: + vout = int(parts[1], 10) + except ValueError: + raise SystemExit("vout must be a decimal integer") + if vout < 0: + raise SystemExit("vout must not be negative") + return parts[0].lower(), vout + + +def build_merkle_proof(keys, leaf_hashes, index): + current_index = index + layer = list(leaf_hashes) + proof = [] + level = 0 + while len(layer) > 1: + if current_index % 2 == 0: + sibling_index = current_index + 1 + duplicated = sibling_index >= len(layer) + sibling = layer[current_index] if duplicated else layer[sibling_index] + side = "right" + else: + sibling_index = current_index - 1 + duplicated = False + sibling = layer[sibling_index] + side = "left" + proof.append({ + "level": level, + "side": side, + "hash": sibling.hex(), + "duplicated": duplicated, + }) + + next_layer = [] + for i in range(0, len(layer), 2): + left = layer[i] + right = layer[i + 1] if i + 1 < len(layer) else left + next_layer.append(hashlib.sha256(NODE_DOMAIN + left + right).digest()) + layer = next_layer + current_index //= 2 + level += 1 + return proof, layer[0].hex() + + +def build_proof(args): + tool = load_utxo_tool() + txid, vout = split_outpoint(args.outpoint) + outpoint = "%s:%d" % (txid, vout) + + raw_blocks = tool.read_blocks(args.datadir, args.max_blocks) + if not raw_blocks: + raise SystemExit("no Bitflash blocks found in %s" % args.datadir) + blocks = tool.select_main_chain(raw_blocks) + state = tool.apply_blocks(blocks) + utxos = state["utxos"] + stats = state["stats"] + if outpoint not in utxos: + raise SystemExit("outpoint is not unspent in the scanned chain: %s" % outpoint) + + keys = sorted(utxos) + leaf_hashes = [tool.utxo_leaf_hash(k, utxos[k]) for k in keys] + index = keys.index(outpoint) + proof_path, root = build_merkle_proof(keys, leaf_hashes, index) + direct_root = tool.utxo_root(utxos) + if root != direct_root: + raise SystemExit("internal proof root mismatch") + + tip = blocks[-1] + utxo = utxos[outpoint] + report = { + "schema": SCHEMA, + "deterministic": True, + "chain": { + "name": "Bitflash", + "ticker": "BTF", + "message_start_hex": tool.MAGIC.hex(), + "coin": tool.COIN, + }, + "commitment": { + "algorithm": "sorted-outpoint-merkle-sha256", + "leaf_domain": "BTFUTXO1", + "node_domain": "BTFNODE1", + "root": root, + "leaf_count": len(keys), + }, + "target": { + "outpoint": outpoint, + "txid": txid, + "vout": vout, + "leaf_index": index, + "leaf_hash": leaf_hashes[index].hex(), + "utxo": utxo, + }, + "proof": proof_path, + "scanned": { + "raw_block_count": len(raw_blocks), + "side_blocks_ignored": len(raw_blocks) - len(blocks), + "main_chain_block_count": len(blocks), + "first_height": 0, + "last_height": len(blocks) - 1, + "tip_hash": tip["hash"], + "tip_time": tip["time"], + "tip_time_utc": tip["time_utc"], + }, + "warnings": { + "legacy_duplicate_output_overwrites": len(stats["duplicate_outputs"]), + }, + "verdict": "UTXO_INCLUSION_PROOF_CREATED", + } + return report + + +def print_text(proof): + data_hash = hashlib.sha256(canonical_bytes(proof)).hexdigest() + target = proof["target"] + utxo = target["utxo"] + print("Bitflash UTXO inclusion proof") + print("schema: %s" % proof["schema"]) + print("canonical proof sha256: %s" % data_hash) + print() + print("Target") + print(" outpoint: %s" % target["outpoint"]) + print(" value: %s BTF" % utxo["value_btf"]) + print(" height: %d" % utxo["created_height"]) + print(" index: %d of %d" % (target["leaf_index"], proof["commitment"]["leaf_count"])) + print() + print("Commitment") + print(" root: %s" % proof["commitment"]["root"]) + print(" path: %d sibling hashes" % len(proof["proof"])) + print(" tip: %s" % proof["scanned"]["tip_hash"]) + print() + print("Verdict: %s" % proof["verdict"]) + + +def default_datadir(): + if os.name == "nt" and os.environ.get("APPDATA"): + return os.path.join(os.environ["APPDATA"], "Bitflash") + if os.environ.get("HOME"): + return os.path.join(os.environ["HOME"], ".bitflash") + return "." + + +def main(argv): + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("outpoint", help="unspent output to prove, as txid:vout") + ap.add_argument("--datadir", default=default_datadir(), help="Bitflash data directory") + ap.add_argument("--max-blocks", type=int, default=0, + help="number of blocks to scan; 0 means all block files") + ap.add_argument("--json", action="store_true", help="print canonical JSON") + ap.add_argument("--out", help="write canonical JSON proof to this path") + args = ap.parse_args(argv) + + proof = build_proof(args) + data = canonical_bytes(proof) + if args.out: + Path(args.out).write_bytes(data) + if args.json: + sys.stdout.buffer.write(data) + else: + print_text(proof) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv[1:])) diff --git a/scripts/verify-utxo-proof.py b/scripts/verify-utxo-proof.py new file mode 100644 index 0000000..31099f7 --- /dev/null +++ b/scripts/verify-utxo-proof.py @@ -0,0 +1,163 @@ +#!/usr/bin/env python3 +"""Verify a Bitflash UTXO inclusion proof against a UTXO root.""" + +import argparse +import hashlib +import json +import sys +from pathlib import Path + + +SCHEMA = "bitflash-utxo-inclusion-proof-1" +LEAF_DOMAIN = "BTFUTXO1" +NODE_DOMAIN = b"BTFNODE1|" + + +def canonical_bytes(obj): + return (json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii") + + +def sha256(data): + return hashlib.sha256(data).digest() + + +def utxo_leaf_hash(outpoint, utxo): + payload = "|".join([ + LEAF_DOMAIN, + outpoint, + str(utxo["value_satoshis"]), + utxo["script_pub_key_hex"], + str(utxo["created_height"]), + utxo["created_txid"], + ]).encode("ascii") + return sha256(payload) + + +def load_proof(path): + try: + return json.loads(Path(path).read_text(encoding="ascii")) + except Exception as e: + raise SystemExit("could not read proof: %s" % e) + + +def verify(proof, expected_root=None): + failures = [] + if proof.get("schema") != SCHEMA: + failures.append("unexpected schema") + commitment = proof.get("commitment", {}) + if commitment.get("algorithm") != "sorted-outpoint-merkle-sha256": + failures.append("unexpected commitment algorithm") + if commitment.get("leaf_domain") != LEAF_DOMAIN: + failures.append("unexpected leaf domain") + if commitment.get("node_domain") != "BTFNODE1": + failures.append("unexpected node domain") + + target = proof.get("target", {}) + outpoint = target.get("outpoint", "") + utxo = target.get("utxo", {}) + if outpoint != "%s:%s" % (target.get("txid"), target.get("vout")): + failures.append("target outpoint does not match txid:vout") + + try: + current = utxo_leaf_hash(outpoint, utxo) + leaf_ok = target.get("leaf_hash") == current.hex() + except Exception as e: + failures.append("could not hash target leaf: %s" % e) + current = b"\x00" * 32 + leaf_ok = False + + if not leaf_ok: + failures.append("target leaf hash mismatch") + + for i, step in enumerate(proof.get("proof", [])): + try: + sibling = bytes.fromhex(step["hash"]) + except Exception: + failures.append("bad sibling hash at proof level %d" % i) + sibling = b"\x00" * 32 + if len(sibling) != 32: + failures.append("bad sibling hash length at proof level %d" % i) + side = step.get("side") + if side == "right": + current = sha256(NODE_DOMAIN + current + sibling) + elif side == "left": + current = sha256(NODE_DOMAIN + sibling + current) + else: + failures.append("bad sibling side at proof level %d" % i) + + computed_root = current.hex() + root = expected_root or commitment.get("root") + if computed_root != root: + failures.append("computed root does not match expected root") + + return { + "schema": "bitflash-utxo-proof-verification-1", + "deterministic": True, + "target": { + "outpoint": outpoint, + "value_satoshis": utxo.get("value_satoshis"), + "value_btf": utxo.get("value_btf"), + "created_height": utxo.get("created_height"), + }, + "commitment": { + "expected_root": root, + "computed_root": computed_root, + "leaf_hash": target.get("leaf_hash"), + "path_length": len(proof.get("proof", [])), + }, + "checks": { + "schema_is_supported": proof.get("schema") == SCHEMA, + "leaf_hash_matches_target": leaf_ok, + "root_matches": computed_root == root, + }, + "failures": failures, + "verdict": "UTXO_PROOF_VERIFIED" if not failures else "UTXO_PROOF_FAILED", + } + + +def print_text(result): + data_hash = hashlib.sha256(canonical_bytes(result)).hexdigest() + print("Bitflash UTXO proof verifier") + print("schema: %s" % result["schema"]) + print("canonical verification sha256: %s" % data_hash) + print() + print("Target") + print(" outpoint: %s" % result["target"]["outpoint"]) + print(" value: %s BTF" % result["target"]["value_btf"]) + print(" height: %s" % result["target"]["created_height"]) + print() + print("Commitment") + print(" expected: %s" % result["commitment"]["expected_root"]) + print(" computed: %s" % result["commitment"]["computed_root"]) + print(" path: %d sibling hashes" % result["commitment"]["path_length"]) + print() + print("Checks") + for key in sorted(result["checks"]): + print(" %s %s" % ("OK " if result["checks"][key] else "FAIL", key)) + if result["failures"]: + print() + print("Failures") + for failure in result["failures"]: + print(" - %s" % failure) + print() + print("Verdict: %s" % result["verdict"]) + + +def main(argv): + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("proof", help="canonical JSON proof from prove-utxo.py") + ap.add_argument("--root", help="expected UTXO root; defaults to root embedded in proof") + ap.add_argument("--json", action="store_true", help="print canonical JSON") + args = ap.parse_args(argv) + + result = verify(load_proof(args.proof), args.root) + data = canonical_bytes(result) + if args.json: + sys.stdout.buffer.write(data) + else: + print_text(result) + return 0 if result["verdict"] == "UTXO_PROOF_VERIFIED" else 1 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv[1:])) -- cgit v1.2.3 From 1839347459759fb3b23cc747ec4368b724fee008 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Tue, 11 Aug 2026 14:13:26 -0300 Subject: tools: harden chain audit scripts --- .github/workflows/ci.yml | 3 + README.md | 8 + docs/fair-launch.md | 3 +- docs/utxo-commitment.md | 20 +- scripts/bitflash_chain.py | 453 +++++++++++++++++++++++++++++++++++ scripts/build-explorer.py | 382 ++++++++++++++++++----------- scripts/prove-utxo.py | 43 ++-- scripts/verify-fair-launch.py | 346 +++----------------------- scripts/verify-utxo-proof.py | 23 +- scripts/verify-utxo-set.py | 375 ++--------------------------- tests/scripts/test_bitflash_tools.py | 107 +++++++++ 11 files changed, 897 insertions(+), 866 deletions(-) create mode 100644 scripts/bitflash_chain.py create mode 100644 tests/scripts/test_bitflash_tools.py (limited to 'docs') diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ba8074c..f4045f4 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -25,6 +25,9 @@ jobs: - name: Run Linux selftests run: make tests + - name: Run Python tool regressions + run: python3 tests/scripts/test_bitflash_tools.py + - name: Run P2P message fuzz smoke run: make fuzz-net-message-smoke diff --git a/README.md b/README.md index c28b0f1..9c6db0e 100644 --- a/README.md +++ b/README.md @@ -104,6 +104,14 @@ and produces canonical JSON whose hash can be compared by independent auditors or timestamped externally with OpenTimestamps. See [fair-launch verification](docs/fair-launch.md). +For a static local block explorer, generate HTML and JSON from either a data +directory or explicit block files: + +```bash +python3 scripts/build-explorer.py --datadir ~/.bitflash ./explorer-out +python3 scripts/build-explorer.py ~/.bitflash/blk0001.dat ~/.bitflash/blk0002.dat ./explorer-out +``` + **Keep your node current.** Consensus rules have changed since the first releases โ€” 1.2.1 fixed a bug that let anyone spend anyone's coins, and 1.2.2 added a per-block signature-operation cap. A node on an older build will accept diff --git a/docs/fair-launch.md b/docs/fair-launch.md index 79eca8e..9603910 100644 --- a/docs/fair-launch.md +++ b/docs/fair-launch.md @@ -8,7 +8,8 @@ python3 scripts/verify-fair-launch.py --max-blocks 1000 --out bitflash-fair-laun The verifier reads `blk*.dat` directly from the local Bitflash data directory. It does not use Berkeley DB, does not call the node binary, and does not need -network access. +network access. It uses the same shared Python chain parser as the UTXO +commitment and proof tools, so block serialization fixes land in one place. The report proves the launch baseline that can be checked from the chain: diff --git a/docs/utxo-commitment.md b/docs/utxo-commitment.md index 991e5d9..ab43deb 100644 --- a/docs/utxo-commitment.md +++ b/docs/utxo-commitment.md @@ -21,7 +21,8 @@ python3 scripts/verify-utxo-set.py --out utxo-report.json --json The verifier: -- parses Bitflash block files directly; +- parses Bitflash block files directly with the shared Python chain parser; +- streams `blk*.dat` records instead of loading full block files into memory; - reconstructs the best chain rooted at the Bitflash genesis block; - ignores side blocks that may also be present in `blk*.dat`; - verifies every scanned block merkle root; @@ -98,6 +99,23 @@ last-write-wins semantics for the UTXO map, matching the legacy index model. Those warnings are part of the report so auditors can see the historical debt instead of relying on a polished summary. +For synthetic chains or future clean consensus ranges, auditors can make +duplicates a hard failure: + +```bash +python3 scripts/verify-utxo-set.py --strict-duplicates +``` + +The default remains compatible with the current historical chain because it has +known legacy duplicate-output overwrites. + +## Shared Parser + +The audit scripts share `scripts/bitflash_chain.py` for block parsing, +main-chain selection, UTXO application, canonical JSON, and genesis +reconstruction. That keeps `verify-utxo-set.py`, `verify-fair-launch.py`, +`prove-utxo.py`, and the explorer on the same serialization rules. + ## Future Consensus Path This tool is the low-risk first step toward consensus commitments. A future diff --git a/scripts/bitflash_chain.py b/scripts/bitflash_chain.py new file mode 100644 index 0000000..f3688f1 --- /dev/null +++ b/scripts/bitflash_chain.py @@ -0,0 +1,453 @@ +#!/usr/bin/env python3 +"""Shared Bitflash block/chain parser for deterministic audit tools.""" + +import datetime as _dt +import hashlib +import json +import os +import struct +from pathlib import Path + + +MAGIC = bytes([0xBF, 0x20, 0x5C, 0xFD]) +MAX_BLOCK_PAYLOAD = 32 * 1024 * 1024 +COIN = 100_000_000 +GENESIS_HASH = "5bd7cb255d814e48cebcdfb72da4dc87b34bd774227f8ceb546c5640f4bdc169" +GENESIS_MERKLE = "1a45b4482532abb29b10e234d3f13132230525a339ecea91658ffa675a5b1325" +GENESIS_TIME = 1_753_315_200 +GENESIS_BITS = 0x1F0FFFFF +GENESIS_NONCE = 3141 +GENESIS_TEXT = "Bitflash 24/Jul/2026 Fair launch: one CPU one vote, no premine" + + +class ParseError(Exception): + pass + + +class Reader: + def __init__(self, data): + self.data = data + self.pos = 0 + + def take(self, n): + if self.pos + n > len(self.data): + raise ParseError("unexpected end of data") + out = self.data[self.pos:self.pos + n] + self.pos += n + return out + + def u8(self): + return self.take(1)[0] + + def u32(self): + return struct.unpack(" 1: + nxt = [] + for i in range(0, len(layer), 2): + left = layer[i] + right = layer[i + 1] if i + 1 < len(layer) else left + nxt.append(sha256d(left + right)) + layer = nxt + return layer[0][::-1].hex() + + +def parse_block(payload, file_name="", offset=0): + r = Reader(payload) + header = r.take(80) + version, = struct.unpack(" MAX_BLOCK_PAYLOAD: + offset += 1 + f.seek(offset) + window = f.read(4) + continue + payload_offset = offset + 8 + payload = f.read(size) + if len(payload) != size: + break + try: + yield parse_block(payload, path.name, payload_offset) + except ParseError: + offset += 1 + f.seek(offset) + window = f.read(4) + continue + + offset = f.tell() + window = f.read(4) + + +def read_blocks_from_files(paths, max_blocks=0): + blocks = [] + for path in paths: + for block in iter_blocks_from_file(path): + blocks.append(block) + if max_blocks and len(blocks) >= max_blocks: + return blocks + return blocks + + +def read_blocks(datadir, max_blocks=0): + return read_blocks_from_files(iter_block_files(datadir), max_blocks) + + +def select_main_chain(blocks): + by_hash = {} + for block in blocks: + by_hash.setdefault(block["hash"], block) + if GENESIS_HASH not in by_hash: + raise ParseError("genesis block not found in block files") + + children = {} + for block in by_hash.values(): + children.setdefault(block["previous_hash"], []).append(block) + for v in children.values(): + v.sort(key=lambda b: (b["hash"], b["source_file"], b["source_offset"])) + + best = [] + stack = [(by_hash[GENESIS_HASH], [by_hash[GENESIS_HASH]])] + while stack: + block, path = stack.pop() + if len(path) > len(best) or (len(path) == len(best) and block["hash"] < best[-1]["hash"]): + best = path + for child in children.get(block["hash"], []): + stack.append((child, path + [child])) + return best + + +def is_coinbase(tx): + return ( + len(tx["vin"]) == 1 + and tx["vin"][0]["prevout_hash"] == "0" * 64 + and tx["vin"][0]["prevout_n"] == 0xFFFFFFFF + ) + + +def utxo_leaf_hash(outpoint, utxo): + payload = "|".join([ + "BTFUTXO1", + outpoint, + str(utxo["value_satoshis"]), + utxo["script_pub_key_hex"], + str(utxo["created_height"]), + utxo["created_txid"], + ]).encode("ascii") + return sha256(payload) + + +def utxo_root(utxos): + leaves = [utxo_leaf_hash(k, utxos[k]) for k in sorted(utxos)] + if not leaves: + return sha256(b"BTFUTXO1|empty").hex() + layer = leaves + while len(layer) > 1: + nxt = [] + for i in range(0, len(layer), 2): + left = layer[i] + right = layer[i + 1] if i + 1 < len(layer) else left + nxt.append(sha256(b"BTFNODE1|" + left + right)) + layer = nxt + return layer[0].hex() + + +def apply_blocks(blocks, strict_duplicates=False): + utxos = {} + spent = 0 + missing_spends = [] + duplicate_outputs = [] + total_coinbase = 0 + total_fees = 0 + normal_txs = 0 + + for height, block in enumerate(blocks): + if block["computed_merkle_root"] != block["merkle_root"]: + raise ParseError("bad merkle root at height %d" % height) + for tx in block["transactions"]: + coinbase = is_coinbase(tx) + input_value = 0 + output_value = sum(out["value_satoshis"] for out in tx["vout"]) + if coinbase: + total_coinbase += output_value + else: + normal_txs += 1 + for txin in tx["vin"]: + prev = "%s:%d" % (txin["prevout_hash"], txin["prevout_n"]) + prev_utxo = utxos.pop(prev, None) + if prev_utxo is None: + missing_spends.append({ + "height": height, + "txid": tx["txid"], + "missing_outpoint": prev, + }) + else: + spent += prev_utxo["value_satoshis"] + input_value += prev_utxo["value_satoshis"] + if input_value >= output_value: + total_fees += input_value - output_value + for n, out in enumerate(tx["vout"]): + key = "%s:%d" % (tx["txid"], n) + if key in utxos: + duplicate_outputs.append({"height": height, "outpoint": key}) + if strict_duplicates: + raise ParseError("duplicate output at height %d: %s" % (height, key)) + utxos[key] = { + "txid": tx["txid"], + "vout": n, + "value_satoshis": out["value_satoshis"], + "value_btf": out["value_btf"], + "script_pub_key_hex": out["script_pub_key_hex"], + "created_height": height, + "created_block_hash": block["hash"], + "created_txid": tx["txid"], + "coinbase": coinbase, + } + + total_unspent = sum(u["value_satoshis"] for u in utxos.values()) + return { + "utxos": utxos, + "stats": { + "utxo_count": len(utxos), + "spent_outputs": spent, + "total_unspent_satoshis": total_unspent, + "total_unspent_btf": format_money(total_unspent), + "total_coinbase_satoshis": total_coinbase, + "total_coinbase_btf": format_money(total_coinbase), + "coinbase_minus_unspent_satoshis": total_coinbase - total_unspent, + "coinbase_minus_unspent_btf": format_money(total_coinbase - total_unspent), + "total_fees_satoshis": total_fees, + "total_fees_btf": format_money(total_fees), + "normal_transactions": normal_txs, + "missing_spends": missing_spends, + "duplicate_outputs": duplicate_outputs, + }, + } + + +def genesis_transaction(): + script_sig = ( + bytes.fromhex("04ffff001d") + + b"\x01\x04" + + bytes([len(GENESIS_TEXT)]) + + GENESIS_TEXT.encode("ascii") + ) + pubkey_num = bytes.fromhex( + "5f1df16b2b704c8a578d0bbaf74d385cde12c11ee50455f3c438ef4c3fbcf649" + "b6de611feae06279a60939e028a8d65c10b73071a6f16719274855feb0fd8a6704" + ) + script_pub_key = bytes([65]) + pubkey_num[::-1] + b"\xac" + tx = { + "version": 1, + "vin": [{ + "prevout_hash": "0" * 64, + "prevout_n": 0xFFFFFFFF, + "script_sig_hex": script_sig.hex(), + "sequence": 0xFFFFFFFF, + }], + "vout": [{ + "value_satoshis": 50 * COIN, + "value_btf": format_money(50 * COIN), + "script_pub_key_hex": script_pub_key.hex(), + }], + "lock_time": 0, + } + tx["txid"] = hash_hex(ser_tx(tx)) + return tx + + +def reconstructed_genesis(): + tx = genesis_transaction() + header = ( + struct.pack(": -# blocks.json summary of every main-chain block (newest first) -# block/.json full detail per block (header + all txs) -# index.html self-contained viewer (vanilla JS, same-origin fetch) -# -# Scope (v0, deliberately): block list, block detail, and coinbase. No mempool, -# no per-address history. On this chain the coinbase payout key rotates every -# block, so an address index would be nearly useless; block-level transparency is -# what lets miners and users confirm the chain is honest. -# -# Usage: build-explorer.py - -import sys, os, json, hashlib, struct - -MAGIC = bytes([0xbf, 0x20, 0x5c, 0xfd]) -ADDRESS_VERSION = 25 # Bitflash addresses start with "B" - -def sha256d(b): return hashlib.sha256(hashlib.sha256(b).digest()).digest() +"""Build a static Bitflash block explorer from local blk*.dat files.""" +import argparse +import hashlib +import json +import sys +from pathlib import Path + +import bitflash_chain as chain + + +ADDRESS_VERSION = 25 B58 = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz" + + def b58check(payload): v = bytes([ADDRESS_VERSION]) + payload - v += sha256d(v)[:4] - n = int.from_bytes(v, 'big') + v += chain.sha256d(v)[:4] + n = int.from_bytes(v, "big") s = "" while n > 0: - n, r = divmod(n, 58); s = B58[r] + s + n, r = divmod(n, 58) + s = B58[r] + s for byte in v: - if byte == 0: s = "1" + s - else: break + if byte == 0: + s = "1" + s + else: + break return s + def hash160(b): - return hashlib.new('ripemd160', hashlib.sha256(b).digest()).digest() - -class R: - def __init__(s, b, o=0): s.b = b; s.o = o - def u(s, n): v = s.b[s.o:s.o+n]; s.o += n; return v - def i32(s): return struct.unpack(' OP_EQUALVERIFY OP_CHECKSIG - if len(spk) == 25 and spk[0] == 0x76 and spk[1] == 0xa9 and spk[2] == 0x14 and spk[23] == 0x88 and spk[24] == 0xac: + return hashlib.new("ripemd160", hashlib.sha256(b).digest()).digest() + + +def spk_info(script_hex): + spk = bytes.fromhex(script_hex) + if ( + len(spk) == 25 and spk[0] == 0x76 and spk[1] == 0xA9 + and spk[2] == 0x14 and spk[23] == 0x88 and spk[24] == 0xAC + ): return {"type": "p2pkh", "address": b58check(spk[3:23])} - # P2PK: OP_CHECKSIG (65- or 33-byte key) - if len(spk) >= 35 and spk[-1] == 0xac: + if len(spk) >= 35 and spk[-1] == 0xAC: if spk[0] == 0x41 and len(spk) == 67: - pk = spk[1:66]; return {"type": "p2pk", "pubkey": pk.hex(), "address": b58check(hash160(pk))} + pk = spk[1:66] + return {"type": "p2pk", "pubkey": pk.hex(), "address": b58check(hash160(pk))} if spk[0] == 0x21 and len(spk) == 35: - pk = spk[1:34]; return {"type": "p2pk", "pubkey": pk.hex(), "address": b58check(hash160(pk))} - return {"type": "other", "hex": spk.hex()[:80]} - -def parse_tx(r): - start = r.o - r.i32() - nin = r.cs(); vin = [] - for _ in range(nin): - ph = r.u(32); idx = r.u32(); r.script(); r.u32() - coinbase = (ph == b'\x00' * 32 and idx == 0xffffffff) - vin.append({"coinbase": coinbase, "prev": (None if coinbase else ph[::-1].hex()), "vout": (None if coinbase else idx)}) - nout = r.cs(); vout = [] - for _ in range(nout): - val = r.i64(); spk = r.script(); vout.append(dict({"value": val}, **spk_info(spk))) - r.u32() - raw = r.b[start:r.o] - return {"txid": sha256d(raw)[::-1].hex(), "vin": vin, "vout": vout, "size": len(raw)} - -def main(): - if len(sys.argv) != 3: - print("usage: build-explorer.py ", file=sys.stderr); sys.exit(2) - blkpath, out = sys.argv[1], sys.argv[2] - data = open(blkpath, 'rb').read() - blocks = {}; o = 0; n = len(data) - while o + 8 <= n: - if data[o:o+4] != MAGIC: o += 1; continue - o += 4; size = struct.unpack('") + args.block_files = None + args.out_dir = args.paths[0] + else: + if len(args.paths) < 2: + raise SystemExit("usage: build-explorer.py ... ") + args.block_files = args.paths[:-1] + args.out_dir = args.paths[-1] + return args + + +def load_blocks(args): + if args.datadir: + return chain.read_blocks(args.datadir, args.max_blocks) + return chain.read_blocks_from_files(args.block_files, args.max_blocks) + + +def build_explorer(raw_blocks, out_dir): + try: + blocks = chain.select_main_chain(raw_blocks) + except chain.ParseError as e: + raise SystemExit(str(e)) + out = Path(out_dir) + (out / "block").mkdir(parents=True, exist_ok=True) + summaries = [] - for h in chain: - b = blocks[h]; H = hc[h] - cb = b["txs"][0]["vout"][0] if b["txs"] and b["txs"][0]["vout"] else {} - summaries.append({"height": H, "hash": h, "time": b["time"], "nTx": len(b["txs"]), - "size": b["size"], "cbValue": cb.get("value", 0), "cbAddr": cb.get("address", "")}) - json.dump(dict({"height": H}, **b), open(os.path.join(out, "block", "%d.json" % H), "w"), separators=(',', ':')) + for height, block in enumerate(blocks): + txs = [explorer_tx(tx) for tx in block["transactions"]] + detail = { + "height": height, + "hash": block["hash"], + "prev": block["previous_hash"], + "merkleRoot": block["merkle_root"], + "version": block["version"], + "time": block["time"], + "bits": int(block["bits"], 16), + "nonce": block["nonce"], + "size": block["size"], + "txs": txs, + } + cb = txs[0]["vout"][0] if txs and txs[0]["vout"] else {} + summaries.append({ + "height": height, + "hash": block["hash"], + "time": block["time"], + "nTx": len(txs), + "size": block["size"], + "cbValue": cb.get("value", 0), + "cbAddr": cb.get("address", ""), + }) + (out / "block" / ("%d.json" % height)).write_text( + json.dumps(detail, sort_keys=True, separators=(",", ":")) + "\n", + encoding="ascii", + ) + summaries.sort(key=lambda x: -x["height"]) - json.dump({"network": "bitflash", "tipHeight": hc[tip], "count": len(chain), "blocks": summaries}, - open(os.path.join(out, "blocks.json"), "w"), separators=(',', ':')) - open(os.path.join(out, "index.html"), "w").write(INDEX_HTML) - print("explorer built: %d main-chain blocks, tip %d, into %s" % (len(chain), hc[tip], out)) + (out / "blocks.json").write_text( + json.dumps({ + "network": "bitflash", + "tipHeight": len(blocks) - 1, + "count": len(blocks), + "blocks": summaries, + }, sort_keys=True, separators=(",", ":")) + "\n", + encoding="ascii", + ) + (out / "index.html").write_text(INDEX_HTML, encoding="utf-8") + return len(blocks), len(blocks) - 1 + + +def main(argv): + args = parse_args(argv) + raw_blocks = load_blocks(args) + if not raw_blocks: + raise SystemExit("no Bitflash blocks parsed") + count, tip = build_explorer(raw_blocks, args.out_dir) + print("explorer built: %d main-chain blocks, tip %d, into %s" % (count, tip, args.out_dir)) + return 0 INDEX_HTML = r""" @@ -144,6 +171,7 @@ INDEX_HTML = r""" body { font: 15px/1.5 system-ui, sans-serif; margin:0; background:var(--bg); color:var(--fg); } header { padding:20px 24px; border-bottom:1px solid var(--line); display:flex; gap:16px; align-items:baseline; flex-wrap:wrap; } h1 { font-size:18px; margin:0; } + h2 { font-size:16px; } .mut { color:var(--mut); } .wrap { padding:16px 24px; max-width:1100px; margin:0 auto; } input { width:100%; max-width:420px; padding:9px 12px; border:1px solid var(--line); border-radius:8px; background:var(--card); color:var(--fg); font-size:14px; } @@ -154,7 +182,6 @@ INDEX_HTML = r""" tr.blk:hover td { background:var(--card); } code { font-size:12px; word-break:break-all; } .scroll { overflow-x:auto; } - a { color:var(--acc); text-decoration:none; } .card { border:1px solid var(--line); border-radius:10px; background:var(--card); padding:14px 16px; margin-top:12px; } .kv { display:grid; grid-template-columns:150px 1fr; gap:4px 12px; font-size:13px; } .kv div:nth-child(odd){ color:var(--mut); } @@ -174,7 +201,7 @@ INDEX_HTML = r"""
-

Read-only view of the main chain: blocks, their transactions, and the coinbase payout. There is no mempool and no per-address history here. On Bitflash the coinbase payout key rotates every block, so an address index would say almost nothing; what matters is that every block and every coinbase is here to check.

+

Read-only view of the main chain: blocks, their transactions, and the coinbase payout. There is no mempool and no per-address history here.

StatusPoolFeeMinersH/sBlocksHeightAgeLink
StatusPoolFeeMinersH/sBlocksHeightAgeLinkProof
@@ -186,52 +213,114 @@ INDEX_HTML = r""" @@ -239,5 +328,6 @@ fetch('blocks.json').then(function(r){return r.json();}).then(function(d){ """ + if __name__ == "__main__": - main() + raise SystemExit(main(sys.argv[1:])) diff --git a/scripts/prove-utxo.py b/scripts/prove-utxo.py index 7a10d2a..9f2704b 100644 --- a/scripts/prove-utxo.py +++ b/scripts/prove-utxo.py @@ -7,27 +7,16 @@ included in the deterministic UTXO root produced by verify-utxo-set.py. import argparse import hashlib -import importlib.util import json -import os import sys from pathlib import Path +import bitflash_chain as chain SCHEMA = "bitflash-utxo-inclusion-proof-1" NODE_DOMAIN = b"BTFNODE1|" -def load_utxo_tool(): - path = Path(__file__).with_name("verify-utxo-set.py") - spec = importlib.util.spec_from_file_location("bitflash_verify_utxo_set", path) - if spec is None or spec.loader is None: - raise SystemExit("could not load %s" % path) - mod = importlib.util.module_from_spec(spec) - spec.loader.exec_module(mod) - return mod - - def canonical_bytes(obj): return (json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii") @@ -80,25 +69,27 @@ def build_merkle_proof(keys, leaf_hashes, index): def build_proof(args): - tool = load_utxo_tool() txid, vout = split_outpoint(args.outpoint) outpoint = "%s:%d" % (txid, vout) - raw_blocks = tool.read_blocks(args.datadir, args.max_blocks) + raw_blocks = chain.read_blocks(args.datadir, args.max_blocks) if not raw_blocks: raise SystemExit("no Bitflash blocks found in %s" % args.datadir) - blocks = tool.select_main_chain(raw_blocks) - state = tool.apply_blocks(blocks) + try: + blocks = chain.select_main_chain(raw_blocks) + state = chain.apply_blocks(blocks, strict_duplicates=args.strict_duplicates) + except chain.ParseError as e: + raise SystemExit(str(e)) utxos = state["utxos"] stats = state["stats"] if outpoint not in utxos: raise SystemExit("outpoint is not unspent in the scanned chain: %s" % outpoint) keys = sorted(utxos) - leaf_hashes = [tool.utxo_leaf_hash(k, utxos[k]) for k in keys] + leaf_hashes = [chain.utxo_leaf_hash(k, utxos[k]) for k in keys] index = keys.index(outpoint) proof_path, root = build_merkle_proof(keys, leaf_hashes, index) - direct_root = tool.utxo_root(utxos) + direct_root = chain.utxo_root(utxos) if root != direct_root: raise SystemExit("internal proof root mismatch") @@ -110,8 +101,8 @@ def build_proof(args): "chain": { "name": "Bitflash", "ticker": "BTF", - "message_start_hex": tool.MAGIC.hex(), - "coin": tool.COIN, + "message_start_hex": chain.MAGIC.hex(), + "coin": chain.COIN, }, "commitment": { "algorithm": "sorted-outpoint-merkle-sha256", @@ -169,20 +160,14 @@ def print_text(proof): print("Verdict: %s" % proof["verdict"]) -def default_datadir(): - if os.name == "nt" and os.environ.get("APPDATA"): - return os.path.join(os.environ["APPDATA"], "Bitflash") - if os.environ.get("HOME"): - return os.path.join(os.environ["HOME"], ".bitflash") - return "." - - def main(argv): ap = argparse.ArgumentParser(description=__doc__) ap.add_argument("outpoint", help="unspent output to prove, as txid:vout") - ap.add_argument("--datadir", default=default_datadir(), help="Bitflash data directory") + ap.add_argument("--datadir", default=chain.default_datadir(), help="Bitflash data directory") ap.add_argument("--max-blocks", type=int, default=0, help="number of blocks to scan; 0 means all block files") + ap.add_argument("--strict-duplicates", action="store_true", + help="fail instead of using legacy last-write-wins duplicate output handling") ap.add_argument("--json", action="store_true", help="print canonical JSON") ap.add_argument("--out", help="write canonical JSON proof to this path") args = ap.parse_args(argv) diff --git a/scripts/verify-fair-launch.py b/scripts/verify-fair-launch.py index ce396fb..c33c8be 100644 --- a/scripts/verify-fair-launch.py +++ b/scripts/verify-fair-launch.py @@ -1,299 +1,12 @@ #!/usr/bin/env python3 -"""Produce a deterministic Bitflash fair-launch audit report. - -This verifier is intentionally self-contained. It parses Bitflash block files -directly, without Berkeley DB, without the node binary, and without network -access. The canonical JSON output is stable for the same inputs and can be -timestamped later with OpenTimestamps or any other external notary. -""" +"""Produce a deterministic Bitflash fair-launch audit report.""" import argparse -import datetime as _dt import hashlib -import json -import os -import struct import sys from pathlib import Path - -MAGIC = bytes([0xBF, 0x20, 0x5C, 0xFD]) -COIN = 100_000_000 -GENESIS_HASH = "5bd7cb255d814e48cebcdfb72da4dc87b34bd774227f8ceb546c5640f4bdc169" -GENESIS_MERKLE = "1a45b4482532abb29b10e234d3f13132230525a339ecea91658ffa675a5b1325" -GENESIS_TIME = 1_753_315_200 -GENESIS_BITS = 0x1F0FFFFF -GENESIS_NONCE = 3141 -GENESIS_TEXT = "Bitflash 24/Jul/2026 Fair launch: one CPU one vote, no premine" - - -class ParseError(Exception): - pass - - -class Reader: - def __init__(self, data): - self.data = data - self.pos = 0 - - def take(self, n): - if self.pos + n > len(self.data): - raise ParseError("unexpected end of data") - out = self.data[self.pos:self.pos + n] - self.pos += n - return out - - def u8(self): - return self.take(1)[0] - - def u32(self): - return struct.unpack(" 1: - nxt = [] - for i in range(0, len(layer), 2): - left = layer[i] - right = layer[i + 1] if i + 1 < len(layer) else left - nxt.append(sha256d(left + right)) - layer = nxt - return layer[0][::-1].hex() - - -def parse_block(payload, file_name="", offset=0): - r = Reader(payload) - header = r.take(80) - version, = struct.unpack(" len(data): - break - size = struct.unpack(" len(data): - break - blocks.append(parse_block(data[start:end], path.name, start)) - if limit and len(blocks) >= limit: - return blocks - pos = end - return blocks - - -def genesis_transaction(): - script_sig = ( - bytes.fromhex("04ffff001d") - + b"\x01\x04" - + bytes([len(GENESIS_TEXT)]) - + GENESIS_TEXT.encode("ascii") - ) - pubkey_num = bytes.fromhex( - "5f1df16b2b704c8a578d0bbaf74d385cde12c11ee50455f3c438ef4c3fbcf649" - "b6de611feae06279a60939e028a8d65c10b73071a6f16719274855feb0fd8a6704" - ) - # The genesis source uses CScript << CBigNum("0x..."), and CBigNum::getvch() - # returns the minimal little-endian script number representation. - script_pub_key = bytes([65]) + pubkey_num[::-1] + b"\xac" - tx = { - "version": 1, - "vin": [{ - "prevout_hash": "0" * 64, - "prevout_n": 0xFFFFFFFF, - "script_sig_hex": script_sig.hex(), - "sequence": 0xFFFFFFFF, - }], - "vout": [{ - "value_satoshis": 50 * COIN, - "value_btf": format_money(50 * COIN), - "script_pub_key_hex": script_pub_key.hex(), - }], - "lock_time": 0, - } - tx["txid"] = hash_hex(ser_tx(tx)) - return tx - - -def reconstructed_genesis(): - tx = genesis_transaction() - header = ( - struct.pack(" len(self.data): - raise ParseError("unexpected end of data") - out = self.data[self.pos:self.pos + n] - self.pos += n - return out - - def u8(self): - return self.take(1)[0] - - def u32(self): - return struct.unpack(" 1: - nxt = [] - for i in range(0, len(layer), 2): - left = layer[i] - right = layer[i + 1] if i + 1 < len(layer) else left - nxt.append(sha256d(left + right)) - layer = nxt - return layer[0][::-1].hex() - - -def parse_block(payload, file_name="", offset=0): - r = Reader(payload) - header = r.take(80) - version, = struct.unpack(" len(data): - break - size = struct.unpack(" len(data): - break - blocks.append(parse_block(data[start:end], path.name, start)) - if max_blocks and len(blocks) >= max_blocks: - return blocks - pos = end - return blocks - - -def select_main_chain(blocks): - by_hash = {} - for block in blocks: - by_hash.setdefault(block["hash"], block) - if GENESIS_HASH not in by_hash: - raise ParseError("genesis block not found in block files") - - children = {} - for block in by_hash.values(): - children.setdefault(block["previous_hash"], []).append(block) - for v in children.values(): - v.sort(key=lambda b: (b["hash"], b["source_file"], b["source_offset"])) - - best = [] - stack = [(by_hash[GENESIS_HASH], [by_hash[GENESIS_HASH]])] - while stack: - block, path = stack.pop() - if len(path) > len(best) or (len(path) == len(best) and block["hash"] < best[-1]["hash"]): - best = path - for child in children.get(block["hash"], []): - stack.append((child, path + [child])) - return best - - -def is_coinbase(tx): - return ( - len(tx["vin"]) == 1 - and tx["vin"][0]["prevout_hash"] == "0" * 64 - and tx["vin"][0]["prevout_n"] == 0xFFFFFFFF - ) - - -def utxo_leaf_hash(outpoint, utxo): - # Text fields are length-delimited by separators and all variable data is - # hex or decimal ASCII. That keeps the commitment language-neutral. - payload = "|".join([ - "BTFUTXO1", - outpoint, - str(utxo["value_satoshis"]), - utxo["script_pub_key_hex"], - str(utxo["created_height"]), - utxo["created_txid"], - ]).encode("ascii") - return sha256(payload) - - -def utxo_root(utxos): - leaves = [utxo_leaf_hash(k, utxos[k]) for k in sorted(utxos)] - if not leaves: - return sha256(b"BTFUTXO1|empty").hex() - layer = leaves - while len(layer) > 1: - nxt = [] - for i in range(0, len(layer), 2): - left = layer[i] - right = layer[i + 1] if i + 1 < len(layer) else left - nxt.append(sha256(b"BTFNODE1|" + left + right)) - layer = nxt - return layer[0].hex() - - -def apply_blocks(blocks): - utxos = {} - spent = 0 - missing_spends = [] - duplicate_outputs = [] - total_coinbase = 0 - total_fees = 0 - normal_txs = 0 - - for height, block in enumerate(blocks): - if block["computed_merkle_root"] != block["merkle_root"]: - raise ParseError("bad merkle root at height %d" % height) - for tx_index, tx in enumerate(block["transactions"]): - coinbase = is_coinbase(tx) - input_value = 0 - output_value = sum(out["value_satoshis"] for out in tx["vout"]) - if coinbase: - total_coinbase += output_value - else: - normal_txs += 1 - for txin in tx["vin"]: - prev = "%s:%d" % (txin["prevout_hash"], txin["prevout_n"]) - prev_utxo = utxos.pop(prev, None) - if prev_utxo is None: - missing_spends.append({ - "height": height, - "txid": tx["txid"], - "missing_outpoint": prev, - }) - else: - spent += prev_utxo["value_satoshis"] - input_value += prev_utxo["value_satoshis"] - if input_value >= output_value: - total_fees += input_value - output_value - for n, out in enumerate(tx["vout"]): - key = "%s:%d" % (tx["txid"], n) - if key in utxos: - duplicate_outputs.append({"height": height, "outpoint": key}) - utxos[key] = { - "txid": tx["txid"], - "vout": n, - "value_satoshis": out["value_satoshis"], - "value_btf": out["value_btf"], - "script_pub_key_hex": out["script_pub_key_hex"], - "created_height": height, - "created_block_hash": block["hash"], - "created_txid": tx["txid"], - "coinbase": coinbase, - } - - total_unspent = sum(u["value_satoshis"] for u in utxos.values()) - return { - "utxos": utxos, - "stats": { - "utxo_count": len(utxos), - "spent_outputs": spent, - "total_unspent_satoshis": total_unspent, - "total_unspent_btf": format_money(total_unspent), - "total_coinbase_satoshis": total_coinbase, - "total_coinbase_btf": format_money(total_coinbase), - "coinbase_minus_unspent_satoshis": total_coinbase - total_unspent, - "coinbase_minus_unspent_btf": format_money(total_coinbase - total_unspent), - "total_fees_satoshis": total_fees, - "total_fees_btf": format_money(total_fees), - "normal_transactions": normal_txs, - "missing_spends": missing_spends, - "duplicate_outputs": duplicate_outputs, - }, - } - - -def format_money(value): - sign = "-" if value < 0 else "" - value = abs(value) - return "%s%d.%08d" % (sign, value // COIN, value % COIN) - - -def utc(ts): - return _dt.datetime.fromtimestamp(ts, _dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") - - -def default_datadir(): - if os.name == "nt" and os.environ.get("APPDATA"): - return os.path.join(os.environ["APPDATA"], "Bitflash") - if os.environ.get("HOME"): - return os.path.join(os.environ["HOME"], ".bitflash") - return "." - - -def canonical_bytes(report): - return (json.dumps(report, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii") +import bitflash_chain as chain -def build_report(raw_blocks, include_utxos=False): - blocks = select_main_chain(raw_blocks) - state = apply_blocks(blocks) +def build_report(raw_blocks, include_utxos=False, strict_duplicates=False): + blocks = chain.select_main_chain(raw_blocks) + state = chain.apply_blocks(blocks, strict_duplicates=strict_duplicates) utxos = state["utxos"] stats = state["stats"] tip = blocks[-1] - root = utxo_root(utxos) + root = chain.utxo_root(utxos) checks = { - "genesis_hash_matches_source": blocks[0]["hash"] == GENESIS_HASH, - "genesis_merkle_matches_source": blocks[0]["merkle_root"] == GENESIS_MERKLE, + "genesis_hash_matches_source": blocks[0]["hash"] == chain.GENESIS_HASH, + "genesis_merkle_matches_source": blocks[0]["merkle_root"] == chain.GENESIS_MERKLE, "all_scanned_merkle_roots_match": all(b["computed_merkle_root"] == b["merkle_root"] for b in blocks), "no_missing_spends": len(stats["missing_spends"]) == 0, } @@ -370,8 +31,8 @@ def build_report(raw_blocks, include_utxos=False): "chain": { "name": "Bitflash", "ticker": "BTF", - "message_start_hex": MAGIC.hex(), - "coin": COIN, + "message_start_hex": chain.MAGIC.hex(), + "coin": chain.COIN, }, "commitment": { "algorithm": "sorted-outpoint-merkle-sha256", @@ -400,7 +61,7 @@ def build_report(raw_blocks, include_utxos=False): def print_text(report): - data_hash = hashlib.sha256(canonical_bytes(report)).hexdigest() + data_hash = hashlib.sha256(chain.canonical_bytes(report)).hexdigest() print("Bitflash UTXO-set commitment verifier") print("schema: %s" % report["schema"]) print("canonical report sha256: %s" % data_hash) @@ -437,20 +98,26 @@ def print_text(report): def main(argv): ap = argparse.ArgumentParser(description=__doc__) - ap.add_argument("--datadir", default=default_datadir(), help="Bitflash data directory") + ap.add_argument("--datadir", default=chain.default_datadir(), help="Bitflash data directory") ap.add_argument("--max-blocks", type=int, default=0, help="number of blocks to scan; 0 means all block files") ap.add_argument("--include-utxos", action="store_true", help="include the full sorted UTXO map in JSON output") + ap.add_argument("--strict-duplicates", action="store_true", + help="fail instead of using legacy last-write-wins duplicate output handling") ap.add_argument("--json", action="store_true", help="print canonical JSON") ap.add_argument("--out", help="write canonical JSON to this path") args = ap.parse_args(argv) - blocks = read_blocks(args.datadir, args.max_blocks) + blocks = chain.read_blocks(args.datadir, args.max_blocks) if not blocks: raise SystemExit("no Bitflash blocks found in %s" % args.datadir) - report = build_report(blocks, include_utxos=args.include_utxos) - data = canonical_bytes(report) + try: + report = build_report(blocks, include_utxos=args.include_utxos, + strict_duplicates=args.strict_duplicates) + except chain.ParseError as e: + raise SystemExit(str(e)) + data = chain.canonical_bytes(report) if args.out: Path(args.out).write_bytes(data) if args.json: diff --git a/tests/scripts/test_bitflash_tools.py b/tests/scripts/test_bitflash_tools.py new file mode 100644 index 0000000..ff569e1 --- /dev/null +++ b/tests/scripts/test_bitflash_tools.py @@ -0,0 +1,107 @@ +#!/usr/bin/env python3 +"""Regression tests for deterministic Bitflash Python tooling.""" + +import json +import os +import struct +import subprocess +import sys +import tempfile +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[2] +SCRIPTS = ROOT / "scripts" +sys.path.insert(0, str(SCRIPTS)) + +import bitflash_chain as chain # noqa: E402 + + +def run(args, **kwargs): + env = dict(os.environ) + env["PYTHONIOENCODING"] = "utf-8" + return subprocess.run( + [sys.executable] + [str(a) for a in args], + cwd=str(ROOT), + env=env, + check=True, + text=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + **kwargs + ) + + +def write_block_file(path, blocks): + with Path(path).open("wb") as f: + f.write(b"noise") + f.write(chain.MAGIC) + f.write(struct.pack(" Date: Tue, 11 Aug 2026 16:03:12 -0300 Subject: net: add Tor mode for outbound discovery --- README.md | 9 +++++- docs/tor.md | 84 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/main_gui.cpp | 25 +++++++++++++---- src/proxy.cpp | 50 ++++++++++++++++++++++++++++++++- src/proxy.h | 3 ++ src/readme.txt | 3 ++ src/selftest.cpp | 25 +++++++++++++++++ 7 files changed, 192 insertions(+), 7 deletions(-) create mode 100644 docs/tor.md (limited to 'docs') diff --git a/README.md b/README.md index 353da25..66f3041 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,8 @@ Worth being precise, because the difference matters if you are relying on it. **The node still listens on 8433.** Nothing dials by IP any more, but the listener is still there, so anyone who already knows your address and can reach that port may connect directly. If that matters to you, firewall it. -This is unlinkability between peers, not anonymity against a network observer. It is not Tor. +This is unlinkability between peers, not anonymity against a network observer. +For Tor routing, start the node with `-tor`; see [Tor mode](docs/tor.md). --- @@ -311,6 +312,7 @@ Other options worth knowing: -datadir=PATH # wallet and chain data elsewhere -port=N # P2P listen port, default 8433 -socks=HOST:PORT # SOCKS5 for outbound Nostr and .btf rendezvous dials +-tor[=HOST:PORT] # Tor mode; default local Tor SOCKS5 proxy is 127.0.0.1:9050 -debug # verbose log; without it debug.log is nearly silent -help # full list ``` @@ -324,6 +326,11 @@ dials through a local SOCKS5 proxy such as Tor. When it is set, the node skips plain HTTP external-IP probes instead of leaking a direct request outside the proxy. IPv6 proxy endpoints use brackets, for example `-socks=[::1]:9050`. +`-tor` is a privacy shorthand for the usual local Tor SOCKS5 listener at +`127.0.0.1:9050`. It routes outbound Nostr discovery and `.btf` rendezvous +dials through Tor, refuses direct `.onion` resolution, and keeps external-IP +probes disabled. Use `-tor=HOST:PORT` when Tor listens somewhere else. + ### When something looks wrong **Menu > Diagnostics**, and the same report in `debug.log` every ten minutes: diff --git a/docs/tor.md b/docs/tor.md new file mode 100644 index 0000000..087c822 --- /dev/null +++ b/docs/tor.md @@ -0,0 +1,84 @@ +# Tor mode + +Bitflash can route outbound discovery traffic through a local Tor SOCKS5 proxy. +This is transport plumbing only: it does not change consensus, wallet keys, +mining, `.btf` addresses, or the encrypted rendezvous protocol. + +## Client mode + +Start Tor locally, then start Bitflash with: + +```bash +bitflash -tor +``` + +`-tor` is shorthand for: + +```bash +bitflash -socks=127.0.0.1:9050 +``` + +If Tor listens somewhere else: + +```bash +bitflash -tor=127.0.0.1:9150 +bitflash -tor=[::1]:9050 +``` + +When Tor mode is enabled: + +- outbound Nostr relay dials use Tor; +- outbound `.btf` rendezvous relay dials use Tor; +- destination names are sent to Tor as SOCKS5 domain-name CONNECT requests, so + the local DNS resolver does not see them; +- direct `.onion` dials are refused unless a SOCKS5/Tor proxy is enabled; +- plain HTTP external-IP probes are skipped. + +## Onion rendezvous relay + +An operator can expose a Bitflash rendezvous relay as a Tor hidden service. The +node does not create hidden services itself; Tor owns that configuration. + +Example `torrc`: + +```text +HiddenServiceDir /var/lib/tor/bitflash-rendezvous/ +HiddenServicePort 8434 127.0.0.1:8434 +``` + +After restarting Tor, read the generated onion name: + +```bash +sudo cat /var/lib/tor/bitflash-rendezvous/hostname +``` + +Then run the relay on localhost/VPS as usual and announce the onion endpoint from +nodes that use Tor: + +```bash +bitflash -tor -rvrelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 +bitflash -tor -announcerelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 +``` + +For a pool operator that wants the pool descriptor to point at an onion +rendezvous relay, use both: + +```bash +bitflash -operator -tor \ + -rvrelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 \ + -announcerelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 +``` + +## Full-node hidden service + +The legacy P2P listener still binds to port `8433`. You can expose it through +Tor manually: + +```text +HiddenServiceDir /var/lib/tor/bitflash-node/ +HiddenServicePort 8433 127.0.0.1:8433 +``` + +Current automatic peer discovery is `.btf`/rendezvous based, not onion-address +gossip. Treat direct full-node onion access as an operator/admin tool unless a +future release adds first-class onion peer advertisements. diff --git a/src/main_gui.cpp b/src/main_gui.cpp index 90a6050..5194023 100644 --- a/src/main_gui.cpp +++ b/src/main_gui.cpp @@ -138,6 +138,7 @@ static void PrintUsage() printf("Network:\n"); printf(" /port=N (P2P listen port, default 8433)\n"); printf(" /socks=HOST:PORT (SOCKS5 proxy for outbound Nostr and .btf relay dials)\n"); + printf(" /tor[=HOST:PORT] (Tor mode; default SOCKS5 proxy is 127.0.0.1:9050)\n"); printf(" /btfseed=ADDRESS:ENCHEX (extra bootstrap peer, repeatable)\n"); printf("\n"); printf("Wallet:\n"); @@ -285,16 +286,30 @@ static void ParseStartupArguments(int argc, char* argv[]) if (!announceRelay.empty()) strBtfAnnounceRelay = announceRelay; - string socksProxy = argval2(argc, argv, "/socks", "-socks"); - if (!socksProxy.empty()) + bool fTorMode = arg(argc, argv, "/tor") || arg(argc, argv, "-tor"); + if (fTorMode) { + string torProxy = argval2(argc, argv, "/tor", "-tor"); string err; - if (!BtfSetSocks5Proxy(socksProxy, err)) - fprintf(stderr, "Ignoring /socks=%s: %s\n", socksProxy.c_str(), err.c_str()); + if (!BtfEnableTorProxy(torProxy, err)) + fprintf(stderr, "Ignoring /tor=%s: %s\n", torProxy.c_str(), err.c_str()); else - fprintf(stderr, "SOCKS5 proxy enabled for outbound discovery/relay dials: %s\n", + fprintf(stderr, "Tor mode enabled through SOCKS5 proxy %s\n", BtfSocks5ProxyName().c_str()); } + else + { + string socksProxy = argval2(argc, argv, "/socks", "-socks"); + if (!socksProxy.empty()) + { + string err; + if (!BtfSetSocks5Proxy(socksProxy, err)) + fprintf(stderr, "Ignoring /socks=%s: %s\n", socksProxy.c_str(), err.c_str()); + else + fprintf(stderr, "SOCKS5 proxy enabled for outbound discovery/relay dials: %s\n", + BtfSocks5ProxyName().c_str()); + } + } // /btfseed=ADDRESS:ENCHEX -- extra bootstrap peers, repeatable. Useful for // testing the seed path and for private networks that ship no compiled list. diff --git a/src/proxy.cpp b/src/proxy.cpp index d7dc324..7d30911 100644 --- a/src/proxy.cpp +++ b/src/proxy.cpp @@ -9,8 +9,10 @@ static CCriticalSection cs_socks5; static bool g_fSocks5Proxy = false; +static bool g_fTorProxy = false; static std::string g_socks5Host; static unsigned short g_socks5Port = 0; +static const char* DEFAULT_TOR_SOCKS5_PROXY = "127.0.0.1:9050"; static void SetSocketTimeout(SOCKET s, int nTimeoutSecs) { @@ -67,6 +69,24 @@ static bool ValidProxyIpv6HostChar(char c) return isxdigit((unsigned char)c) || c == ':' || c == '.'; } +static std::string LowerProxyHost(std::string host) +{ + if (!host.empty() && host[host.size() - 1] == '.') + host.erase(host.size() - 1); + for (size_t i = 0; i < host.size(); i++) + host[i] = (char)tolower((unsigned char)host[i]); + return host; +} + +bool BtfIsTorOnionHost(const std::string& host) +{ + std::string lower = LowerProxyHost(host); + static const char* suffix = ".onion"; + size_t suffixLen = strlen(suffix); + return lower.size() > suffixLen && + lower.compare(lower.size() - suffixLen, suffixLen, suffix) == 0; +} + static bool ParsePort(const std::string& port, unsigned short& portOut, std::string& errOut) { @@ -176,7 +196,8 @@ bool BtfParseSocks5Proxy(const std::string& spec, std::string& hostOut, return true; } -bool BtfSetSocks5Proxy(const std::string& spec, std::string& errOut) +static bool SetSocks5Proxy(const std::string& spec, bool fTorMode, + std::string& errOut) { std::string host; unsigned short port = 0; @@ -187,15 +208,27 @@ bool BtfSetSocks5Proxy(const std::string& spec, std::string& errOut) g_socks5Host = host; g_socks5Port = port; g_fSocks5Proxy = true; + g_fTorProxy = fTorMode; } return true; } +bool BtfSetSocks5Proxy(const std::string& spec, std::string& errOut) +{ + return SetSocks5Proxy(spec, false, errOut); +} + +bool BtfEnableTorProxy(const std::string& spec, std::string& errOut) +{ + return SetSocks5Proxy(spec.empty() ? DEFAULT_TOR_SOCKS5_PROXY : spec, true, errOut); +} + void BtfClearSocks5Proxy() { CRITICAL_BLOCK(cs_socks5) { g_fSocks5Proxy = false; + g_fTorProxy = false; g_socks5Host.clear(); g_socks5Port = 0; } @@ -208,6 +241,13 @@ bool BtfSocks5ProxyEnabled() return false; } +bool BtfTorProxyEnabled() +{ + CRITICAL_BLOCK(cs_socks5) + return g_fSocks5Proxy && g_fTorProxy; + return false; +} + std::string BtfSocks5ProxyName() { CRITICAL_BLOCK(cs_socks5) @@ -315,7 +355,15 @@ SOCKET BtfConnectSocket(const std::string& destHost, unsigned short destPort, } if (!fProxy) + { + if (BtfIsTorOnionHost(destHost)) + { + error("Refusing direct .onion connection to %s:%u; use /tor or /socks\n", + destHost.c_str(), (unsigned)destPort); + return INVALID_SOCKET; + } return ConnectDirect(destHost, destPort, nSockSite, nTimeoutSecs); + } SOCKET s = ConnectDirect(proxyHost, proxyPort, nSockSite, nTimeoutSecs); if (s == INVALID_SOCKET) diff --git a/src/proxy.h b/src/proxy.h index b5e69c8..6429cf2 100644 --- a/src/proxy.h +++ b/src/proxy.h @@ -21,8 +21,11 @@ bool BtfParseSocks5Proxy(const std::string& spec, std::string& hostOut, unsigned short& portOut, std::string& errOut); bool BtfSetSocks5Proxy(const std::string& spec, std::string& errOut); +bool BtfEnableTorProxy(const std::string& spec, std::string& errOut); void BtfClearSocks5Proxy(); bool BtfSocks5ProxyEnabled(); +bool BtfTorProxyEnabled(); +bool BtfIsTorOnionHost(const std::string& host); std::string BtfSocks5ProxyName(); SOCKET BtfConnectSocket(const std::string& destHost, unsigned short destPort, diff --git a/src/readme.txt b/src/readme.txt index 1fa77df..794fe20 100644 --- a/src/readme.txt +++ b/src/readme.txt @@ -46,6 +46,9 @@ Command-line options (in addition to the original ones) route outbound Nostr and .btf rendezvous dials through a SOCKS5 proxy, e.g. /socks=127.0.0.1:9050 for local Tor or /socks=[::1]:9050 for an IPv6 loopback proxy + /tor[=HOST:PORT] + Tor mode for outbound Nostr and .btf rendezvous dials; + defaults to the local Tor SOCKS5 proxy at 127.0.0.1:9050 /solomine mine without requiring a connected peer (local testing) /operator operator mode (run pool server) /participant=ADDR diff --git a/src/selftest.cpp b/src/selftest.cpp index aa4e528..cca5335 100644 --- a/src/selftest.cpp +++ b/src/selftest.cpp @@ -1474,6 +1474,11 @@ static int RunSocks5ProxySelfTest() "rejects overflowing port") ? 0 : 1; nFail += Check(!BtfParseSocks5Proxy("user:pass@localhost:9050", host, port, err), "rejects unsupported authenticated proxy syntax") ? 0 : 1; + nFail += Check(BtfIsTorOnionHost("abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcd.onion") && + BtfIsTorOnionHost("Relay.Example.ONION.") && + !BtfIsTorOnionHost("relay.example") && + !BtfIsTorOnionHost(".onion"), + "detects .onion hosts without accepting lookalikes") ? 0 : 1; BtfClearSocks5Proxy(); nFail += Check(!BtfSocks5ProxyEnabled(), "starts disabled after clear") ? 0 : 1; @@ -1490,6 +1495,26 @@ static int RunSocks5ProxySelfTest() "enables bracketed IPv6 proxy endpoint") ? 0 : 1; BtfClearSocks5Proxy(); + nFail += Check(BtfEnableTorProxy("", err) && + BtfSocks5ProxyEnabled() && + BtfTorProxyEnabled() && + BtfSocks5ProxyName() == "127.0.0.1:9050", + "enables Tor mode on the default local SOCKS5 endpoint") ? 0 : 1; + BtfClearSocks5Proxy(); + + nFail += Check(BtfEnableTorProxy("[::1]:9050", err) && + BtfSocks5ProxyEnabled() && + BtfTorProxyEnabled() && + BtfSocks5ProxyName() == "[::1]:9050", + "enables Tor mode on a custom IPv6 endpoint") ? 0 : 1; + BtfClearSocks5Proxy(); + + nFail += Check(BtfSetSocks5Proxy("127.0.0.1:9051", err) && + BtfSocks5ProxyEnabled() && + !BtfTorProxyEnabled(), + "plain SOCKS5 mode is distinct from Tor mode") ? 0 : 1; + BtfClearSocks5Proxy(); + std::string handshakeErr; bool fHandshakeOk = RunSocks5HandshakeProbe(handshakeErr); nFail += Check(fHandshakeOk, handshakeErr.empty() ? -- cgit v1.2.3 From 4d3a5ed4a188e6d5b62c7fd82c6f7ad41e9518f4 Mon Sep 17 00:00:00 2001 From: bitflash Date: Wed, 12 Aug 2026 09:59:55 -0300 Subject: docs: update social banner to the new coin logo --- docs/social.png | Bin 208957 -> 162267 bytes 1 file changed, 0 insertions(+), 0 deletions(-) (limited to 'docs') diff --git a/docs/social.png b/docs/social.png index 19e2a3b..9a08b02 100644 Binary files a/docs/social.png and b/docs/social.png differ -- cgit v1.2.3 From b0ed2ce585d2a6fd7c6d8d4db7422734aedccb58 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Wed, 12 Aug 2026 11:58:03 -0300 Subject: site: document public deployment contract --- README.md | 17 +++++ docs/mining-calculator.md | 22 ++++++ docs/pool-directory.md | 18 +++++ docs/release-verification.md | 10 +++ docs/site.md | 115 +++++++++++++++++++++++++++++++ docs/tor.md | 29 ++++++++ scripts/build-explorer.py | 128 +++++++++++++++++++++++------------ tests/scripts/test_bitflash_tools.py | 7 +- 8 files changed, 300 insertions(+), 46 deletions(-) create mode 100644 docs/site.md (limited to 'docs') diff --git a/README.md b/README.md index ef4f39b..e671302 100644 --- a/README.md +++ b/README.md @@ -38,6 +38,8 @@ Worth being precise, because the difference matters if you are relying on it. This is unlinkability between peers, not anonymity against a network observer. For Tor routing, start the node with `-tor`; see [Tor mode](docs/tor.md). +For a plain SOCKS5 proxy without Tor-specific defaults, use +`-socks=HOST:PORT`. --- @@ -74,6 +76,14 @@ scripts/verify-release.sh latest See [release verification](docs/release-verification.md) for the full release audit flow and the maintainer signing step. +Public mirrors should make the latest checksum files available both under the +versioned release directory and at the release root: + +```text +https://releases.bitflash.network/v1.2.18/SHA256SUMS +https://releases.bitflash.network/SHA256SUMS # alias to latest +``` + Bitflash also ships a deterministic UTXO-set commitment tool: ```bash @@ -113,6 +123,12 @@ python3 scripts/build-explorer.py --datadir ~/.bitflash ./explorer-out python3 scripts/build-explorer.py ~/.bitflash/blk0001.dat ~/.bitflash/blk0002.dat ./explorer-out ``` +The generated explorer is static and self-contained: `index.html`, +`style.css`, `explorer.js`, `blocks.json`, `block/*.json`, and `logo.png`. It is +safe to host behind a strict policy such as `default-src 'none'; img-src 'self'; +style-src 'self'; script-src 'self'; connect-src 'self'; base-uri 'none'; +form-action 'none'`. + **Keep your node current.** Consensus rules have changed since the first releases โ€” 1.2.1 fixed a bug that let anyone spend anyone's coins, and 1.2.2 added a per-block signature-operation cap. A node on an older build will accept @@ -265,6 +281,7 @@ default both are created in the data directory; use `-poolstatusfile=PATH` and `-poolroundsfile=PATH` to write them somewhere a dashboard or web sync can read. The files are informational only: nodes still discover pools through Nostr and `.btf`, not through any web domain. +See [public pool directory](docs/pool-directory.md) for the website/API shape. **Participant** โ€” mine to someone else's pool. Enter or select the pool's `.btf` address and enable Start Mining. diff --git a/docs/mining-calculator.md b/docs/mining-calculator.md index 352eea1..e6cb61b 100644 --- a/docs/mining-calculator.md +++ b/docs/mining-calculator.md @@ -84,3 +84,25 @@ The script is intentionally standalone so the same math can later back a website widget or status dashboard. Until a public price exists, a website calculator should keep price as a manual field and label fiat profit as speculative. + +## Website Fields + +A public calculator should ask for: + +- miner hashrate and unit; +- power draw in watts; +- electricity price per kWh; +- pool fee percent; +- optional BTF price; +- either current Bitflash difficulty or current network hashrate. + +The output should separate: + +- estimated BTF per day; +- estimated gross fiat revenue, if a manual price was provided; +- electricity cost; +- net profit/loss; +- the difficulty/hashrate assumptions used. + +Never present the fiat number as a market quote unless it comes from a real +public market. Before that exists, label it as a manual scenario. diff --git a/docs/pool-directory.md b/docs/pool-directory.md index 76269b9..ff0640a 100644 --- a/docs/pool-directory.md +++ b/docs/pool-directory.md @@ -26,6 +26,10 @@ The generated output contains: - `index.html`: human-readable ranking page. - `pools.json`: machine-readable API for `api.bitflash.network/pools.json`. +The website is a convenience layer, not a consensus or discovery dependency. +Nodes discover pools through Nostr descriptors and `.btf`; a missing website +must not stop mining or pool operation. + When a source is an HTTP(S) `pool_status.json`, the directory links to a `pool_rounds.json` proof file next to it. That file records the block height, block hash, coinbase txid, coinbase value, pool fee, total shares, per-address @@ -41,3 +45,17 @@ Ranking order is intentionally simple: An entry is considered online when the source says the pool is running and its status file is no more than 120 seconds old. + +## Recommended Hostnames + +Use: + +```text +pool.bitflash.network human-readable pool directory +api.bitflash.network/pools.json +status.bitflash.network relay/node health, not pool ranking +docs.bitflash.network operator setup guide +``` + +The GUI may use the API as a shortcut for humans, but it should keep the +existing decentralized pool discovery path as the source of truth. diff --git a/docs/release-verification.md b/docs/release-verification.md index 32de74f..408bab9 100644 --- a/docs/release-verification.md +++ b/docs/release-verification.md @@ -34,6 +34,16 @@ From a shell with `curl` or `wget`, `sha256sum` and `gpg`: scripts/verify-release.sh latest ``` +The public release host should expose latest-file aliases for convenience: + +```text +https://releases.bitflash.network/SHA256SUMS +https://releases.bitflash.network/SHA256SUMS.asc +``` + +Those aliases must point at the current versioned directory. They do not replace +signature verification; they only make the common manual path shorter. + For a specific release: ```bash diff --git a/docs/site.md b/docs/site.md new file mode 100644 index 0000000..522da10 --- /dev/null +++ b/docs/site.md @@ -0,0 +1,115 @@ +# Public Site Contract + +The public Bitflash sites are generated from this repository. If a deployed +page and the source disagree, the deployed page is stale. + +## Canonical Navigation + +Every public page should use the same top navigation: + +| label | target | +|---|---| +| home | `https://bitflash.network/` | +| code | `https://git.bitflash.network/bitflash/bitflash` | +| docs | `https://docs.bitflash.network/` | +| downloads | `https://releases.bitflash.network/` | +| explorer | `https://explorer.bitflash.network/` | +| status | `https://status.bitflash.network/` | + +The `code` link intentionally points at the repository, not the Forgejo +instance root. A visitor choosing "code" wants the source tree. + +## Documentation Pages To Publish + +The docs index should expose these source files: + +| page | source | why it matters | +|---|---|---| +| Manual | `README.md` | run, mine, recover, operate | +| Key derivation and address format | `docs/derivation.md` | independent wallet recovery | +| Release verification | `docs/release-verification.md` | pinned signing-key flow | +| Dependencies | `docs/dependencies.md` | pinned secp256k1/RandomX supply chain | +| Rendezvous and .btf addressing | `docs/rendezvous.md` | network design and caveats | +| Tor mode | `docs/tor.md` | `-tor`, `-socks`, onion relay operation | +| Fair launch verifier | `docs/fair-launch.md` | no-premine launch audit | +| UTXO set commitment | `docs/utxo-commitment.md` | deterministic supply/root audit | +| UTXO inclusion proofs | `docs/utxo-proofs.md` | proof for one unspent output | +| Mining calculator | `docs/mining-calculator.md` | local WhatToMine-style estimates | +| Public pool directory | `docs/pool-directory.md` | `pool_status.json`, `pools.json`, ranking | + +## Release Mirror Aliases + +Each release remains under its versioned directory: + +```text +https://releases.bitflash.network/v1.2.18/SHA256SUMS +https://releases.bitflash.network/v1.2.18/SHA256SUMS.asc +``` + +The release root should also alias the latest files: + +```text +https://releases.bitflash.network/SHA256SUMS +https://releases.bitflash.network/SHA256SUMS.asc +https://releases.bitflash.network/Bitflash-latest-windows.zip +https://releases.bitflash.network/Bitflash-latest-x86_64.AppImage +https://releases.bitflash.network/bitflash-node-latest-x86_64 +``` + +The aliases are convenience only. Verification must still require the pinned +release key documented in `docs/release-verification.md`. + +## Static Explorer + +`scripts/build-explorer.py` writes a static explorer with no backend: + +```text +index.html +style.css +explorer.js +logo.png +blocks.json +block/.json +``` + +Serve it with a strict content policy: + +```text +default-src 'none'; +img-src 'self'; +style-src 'self'; +script-src 'self'; +connect-src 'self'; +base-uri 'none'; +form-action 'none'; +frame-ancestors 'none'; +``` + +The viewer must never use `innerHTML` for chain data. A malicious transaction +must be rendered as text, not interpreted as markup. + +## Cache Policy + +Recommended defaults: + +| path | cache | +|---|---| +| HTML pages | `public, max-age=0, must-revalidate` | +| `status.json` | `no-store` | +| `blocks.json`, `block/*.json` | `public, max-age=0, must-revalidate` | +| release assets | `public, max-age=14400, must-revalidate` | +| `SHA256SUMS`, signatures, keys | `public, max-age=0, must-revalidate` | + +## Security Headers + +Recommended for all static pages: + +```text +Strict-Transport-Security: max-age=31536000; includeSubDomains +X-Content-Type-Options: nosniff +Referrer-Policy: no-referrer +Permissions-Policy: interest-cohort=(), geolocation=(), camera=(), microphone=() +``` + +Forgejo has its own application headers and should not blindly inherit the +static-site CSP. diff --git a/docs/tor.md b/docs/tor.md index 087c822..69b29f1 100644 --- a/docs/tor.md +++ b/docs/tor.md @@ -25,6 +25,17 @@ bitflash -tor=127.0.0.1:9150 bitflash -tor=[::1]:9050 ``` +For a SOCKS5 proxy that is not Tor, use `-socks` directly: + +```bash +bitflash -socks=127.0.0.1:9050 +bitflash -socks=[::1]:9050 +``` + +If both `-tor` and `-socks` are present, Tor mode wins and the node prints a +warning. That keeps a command line with `-tor` from silently becoming a custom +proxy route. + When Tor mode is enabled: - outbound Nostr relay dials use Tor; @@ -34,6 +45,24 @@ When Tor mode is enabled: - direct `.onion` dials are refused unless a SOCKS5/Tor proxy is enabled; - plain HTTP external-IP probes are skipped. +Tor mode affects outbound discovery and relay dials. It does not encrypt +`wallet.dat`, hide mining rewards on-chain, or change the `.btf` identity. A +relay still sees the TCP client that connected to it; with Tor mode that client +is the Tor exit or onion circuit endpoint instead of the user's direct IP. + +## What to test + +Use the public status page only as a rough health check. To prove the local node +is using Tor/SOCKS, test from the node machine: + +```bash +bitflash -tor -debug +``` + +Then confirm in `debug.log` that Nostr and rendezvous dials report the proxy +path. A DNS leak test should show no local resolver lookup for relay hostnames: +the SOCKS5 request uses the domain-name form and asks the proxy to resolve it. + ## Onion rendezvous relay An operator can expose a Bitflash rendezvous relay as a Tor hidden service. The diff --git a/scripts/build-explorer.py b/scripts/build-explorer.py index 97a7f4e..dbb8e17 100755 --- a/scripts/build-explorer.py +++ b/scripts/build-explorer.py @@ -4,6 +4,7 @@ import argparse import hashlib import json +import shutil import sys from pathlib import Path @@ -148,6 +149,11 @@ def build_explorer(raw_blocks, out_dir): encoding="ascii", ) (out / "index.html").write_text(INDEX_HTML, encoding="utf-8") + (out / "style.css").write_text(STYLE_CSS, encoding="utf-8") + (out / "explorer.js").write_text(EXPLORER_JS, encoding="utf-8") + logo = Path(__file__).resolve().parents[1] / "docs" / "logo.png" + if logo.exists(): + shutil.copyfile(logo, out / "logo.png") return len(blocks), len(blocks) - 1 @@ -167,43 +173,26 @@ INDEX_HTML = r""" Bitflash Block Explorer - + + + + -
-

Bitflash Block Explorer

- - - +
+ Bitflash + +
+

Block explorer

Read-only view of the main chain: blocks, their transactions, and the coinbase payout. There is no mempool and no per-address history here.

@@ -213,9 +202,63 @@ INDEX_HTML = r"""

- + - - - """ diff --git a/tests/scripts/test_bitflash_tools.py b/tests/scripts/test_bitflash_tools.py index 4b18d1e..3b4c846 100644 --- a/tests/scripts/test_bitflash_tools.py +++ b/tests/scripts/test_bitflash_tools.py @@ -159,8 +159,11 @@ def compare_golden(name, actual, update): path.parent.mkdir(parents=True, exist_ok=True) path.write_bytes(actual) return - expected = path.read_bytes() - if actual != expected: + # The tools promise canonical LF-terminated JSON. On Windows checkouts a + # fixture may still appear as CRLF in the working tree; normalize only line + # endings so content, field order and compact serialization still drift-test. + expected = path.read_bytes().replace(b"\r\n", b"\n") + if actual.replace(b"\r\n", b"\n") != expected: raise AssertionError("golden mismatch: %s" % name) -- cgit v1.2.3 From d2b86473a267c960a1f0c012a983176e5d962187 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Thu, 13 Aug 2026 11:11:59 -0300 Subject: net: add signed direct onion peer endpoints --- README.md | 26 ++++++++----- docs/rendezvous.md | 70 +++++++++++++++++++++++++++++------ docs/tor.md | 39 ++++++++++++++++++-- src/btfaddr.cpp | 102 +++++++++++++++++++++++++++++++++++++++++++++++---- src/btfaddr.h | 8 +++- src/main_gui.cpp | 15 +++++++- src/net.cpp | 102 +++++++++++++++++++++++++++++++++++++++++---------- src/net.h | 3 +- src/nostr.cpp | 76 +++++++++++++++++++++++++++++++------- src/nostr.h | 12 +++++- src/readme.txt | 8 ++-- src/selftest.cpp | 9 +++++ src/sockcount.h | 1 + src/test_btfaddr.cpp | 22 +++++++++++ 14 files changed, 420 insertions(+), 73 deletions(-) (limited to 'docs') diff --git a/README.md b/README.md index e671302..181b6e8 100644 --- a/README.md +++ b/README.md @@ -22,7 +22,7 @@ Bitflash keeps Satoshi's original consensus rules and replaces two things: **RandomX proof of work.** Memory-hard algorithm used by Monero. A laptop competes equally with a server. ASICs and GPUs have no advantage. -**Anonymous addressing.** Every node has a `.btf` address derived from its public key, similar to a Tor `.onion`. Nodes reach each other through encrypted rendezvous tunnels, so no port forwarding is needed and a node behind CGNAT works normally. +**Anonymous addressing.** Every node has a `.btf` address derived from its public key, similar to a Tor `.onion`. Nodes reach each other through encrypted rendezvous tunnels by default, and can advertise a signed direct `.onion` endpoint for Tor peers. No premine. No ICO. 50 BTF per block, halving on schedule, 21M cap, ~2 minute blocks. @@ -30,14 +30,15 @@ No premine. No ICO. 50 BTF per block, halving on schedule, 21M cap, ~2 minute bl Worth being precise, because the difference matters if you are relying on it. -**A peer you reach over `.btf` does not learn your IP.** All outbound connections go through a rendezvous tunnel; there is no IP-based peer dialling left in the node. The relay forwards encrypted bytes and cannot read or alter them. +**A peer you reach over `.btf` does not learn your IP.** Outbound peer discovery uses rendezvous tunnels by default. If both sides use Tor and the peer advertises a signed `.onion` endpoint, the node connects directly to that hidden service instead. In both cases the peer does not see your clearnet IP. -**The rendezvous relay does see your IP.** It has to โ€” it is the thing your TCP connection terminates on. Relays are run by volunteers, so treat that as a party who knows you are on the network. +**The rendezvous relay sees your IP unless you use Tor.** It has to โ€” it is the thing your TCP connection terminates on. With `-tor`, the relay sees Tor instead. With a signed direct `.onion` peer endpoint, the relay is not on that connection path at all. **The node still listens on 8433.** Nothing dials by IP any more, but the listener is still there, so anyone who already knows your address and can reach that port may connect directly. If that matters to you, firewall it. This is unlinkability between peers, not anonymity against a network observer. -For Tor routing, start the node with `-tor`; see [Tor mode](docs/tor.md). +For Tor routing and direct onion peer endpoints, start the node with `-tor`; +see [Tor mode](docs/tor.md). For a plain SOCKS5 proxy without Tor-specific defaults, use `-socks=HOST:PORT`. @@ -336,8 +337,9 @@ Other options worth knowing: ```bash -datadir=PATH # wallet and chain data elsewhere -port=N # P2P listen port, default 8433 --socks=HOST:PORT # SOCKS5 for outbound Nostr and .btf rendezvous dials +-socks=HOST:PORT # SOCKS5 for outbound Nostr, .btf relay, and onion peer dials -tor[=HOST:PORT] # Tor mode; default local Tor SOCKS5 proxy is 127.0.0.1:9050 +-onionservice=HOST.onion:PORT # advertise this node's Tor hidden service -debug # verbose log; without it debug.log is nearly silent -help # full list ``` @@ -346,15 +348,19 @@ Other options worth knowing: needed โ€” the data directory takes an exclusive lock, so a second node pointed at the same one will refuse to start. -`-socks=127.0.0.1:9050` routes outbound Nostr discovery and `.btf` rendezvous -dials through a local SOCKS5 proxy such as Tor. When it is set, the node skips +`-socks=127.0.0.1:9050` routes outbound Nostr discovery, `.btf` rendezvous +dials, and direct `.onion` peer dials through a local SOCKS5 proxy such as Tor. +When it is set, the node skips plain HTTP external-IP probes instead of leaking a direct request outside the proxy. IPv6 proxy endpoints use brackets, for example `-socks=[::1]:9050`. `-tor` is a privacy shorthand for the usual local Tor SOCKS5 listener at -`127.0.0.1:9050`. It routes outbound Nostr discovery and `.btf` rendezvous -dials through Tor, refuses direct `.onion` resolution, and keeps external-IP -probes disabled. Use `-tor=HOST:PORT` when Tor listens somewhere else. +`127.0.0.1:9050`. It routes outbound Nostr discovery, `.btf` rendezvous dials, +and signed direct `.onion` peer dials through Tor, and keeps external-IP probes +disabled. Use `-tor=HOST:PORT` when Tor listens somewhere else. To make your own +node reachable without a rendezvous relay, expose port `8433` as a Tor hidden +service and pass `-onionservice=HOST.onion:8433`; peers with `-tor` will try it +before falling back to rendezvous. ### When something looks wrong diff --git a/docs/rendezvous.md b/docs/rendezvous.md index ac8e598..e1888b9 100644 --- a/docs/rendezvous.md +++ b/docs/rendezvous.md @@ -18,11 +18,16 @@ A connection to a `.btf` service has three layers: 1. **Discovery (Nostr).** The service publishes a signed descriptor as a Nostr event. A caller fetches it by the service public key and verifies the signature. The descriptor says which rendezvous relay the service is waiting - at and which encryption key to use. + at, which encryption key to use, and optionally which Tor hidden service can + be dialed directly. 2. **Rendezvous (TCP).** Caller and service both connect out to the same relay and are paired by public key. Neither needs an inbound port or a public IP. After pairing the relay is a blind byte pipe. -3. **Encrypted channel (X25519 + XChaCha20-Poly1305).** Over the paired pipe the +3. **Direct onion fallback bypass.** If the descriptor carries a valid signed + `.onion:port` endpoint and the caller is running with Tor mode, the caller + tries that endpoint before using rendezvous. This removes the rendezvous + relay from the data path for peers that choose to run hidden services. +4. **Encrypted channel (X25519 + XChaCha20-Poly1305).** Over the paired pipe the caller and service run an ephemeral to static handshake and then exchange authenticated frames. The relay sees only ciphertext. @@ -107,6 +112,8 @@ The default rendezvous port is `8434`. The descriptor is a JSON object, signed with the identity key over a domain separated digest of its own fields. +Legacy descriptor: + ```json { "v": 2, @@ -118,7 +125,22 @@ separated digest of its own fields. } ``` -The signed digest is: +Onion-capable descriptor: + +```json +{ + "v": 3, + "pubkey": "", + "enc": "", + "meeting_node": "host:port", + "onion": "<56-char-v3-name>.onion:8433", + "created": , + "sig": "", + "sig3": "" +} +``` + +The legacy signed digest is: ``` digest = SHA256( ".btf-descriptor-v2" @@ -128,17 +150,37 @@ digest = SHA256( ".btf-descriptor-v2" || created (8 bytes, big endian) ) ``` -The signature is a deterministic BIP340 Schnorr signature (no auxiliary -randomness) over `digest` by the identity key. +`sig` is kept on v3 descriptors so old nodes can still resolve and dial through +the rendezvous relay. The onion-capable digest is: + +``` +digest3 = SHA256( ".btf-descriptor-v3-onion" + || pubkey[32] + || enc (raw bytes of the string) + || meeting_node (raw bytes of the string) + || onion (raw bytes of the normalized string) + || created (8 bytes, big endian) ) +``` + +`sig` and `sig3` are deterministic BIP340 Schnorr signatures (no auxiliary +randomness) over their respective digests by the identity key. **Verification** (untrusted input, must never throw): parse the JSON, require all fields with the right types, confirm `pubkey` equals the address being resolved, recompute the digest, and verify the Schnorr signature. Any failure rejects the descriptor. +If `onion` is present, readers accept it only when it is a normalized v3 onion +endpoint and `sig3` verifies. A bad or tampered onion field is ignored; the v2 +descriptor can still be used for rendezvous fallback if `sig` verifies. + `created` lets a reader prefer the freshest descriptor when more than one is -seen. A node only publishes a descriptor once it has actually registered at a -`meeting_node`, so it never advertises a relay it cannot be reached at. +seen. A node without a direct onion endpoint only publishes a descriptor once it +has actually registered at a `meeting_node`, so it never advertises a relay it +cannot be reached at. A node with a direct onion endpoint may publish before +rendezvous registration using `meeting_node = "rendezvous-pending"`; Tor peers +can still dial the signed onion endpoint, and rendezvous becomes fallback once +registration succeeds. ## 6. Rendezvous transport @@ -243,8 +285,9 @@ tunnel. not published anywhere. - **Forward secrecy.** The caller's per connection ephemeral key means a later compromise of the service encryption key does not decrypt past sessions. -- **IP privacy.** Neither side needs a public IP or an inbound port. The observable - network fact is that each side has a TCP connection to a relay. +- **IP privacy.** Neither side needs a public IP or an inbound port for + rendezvous. With a signed `.onion` endpoint, the reachable side exposes only a + Tor hidden service and the relay is not in the peer connection path. ## 10. Threat model and limitations @@ -265,6 +308,9 @@ tunnel. unreachable, discovery stalls until one is reachable again. Relay TLS is verified, which closes a man in the middle path on discovery, but availability still rests on the relay set. +- **Direct onion is not automatic Tor configuration.** Bitflash signs and + advertises a hidden-service endpoint, and Tor-mode peers can dial it. The Tor + daemon still owns hidden-service creation, private keys, rotation and uptime. ## 11. Wire reference @@ -273,9 +319,9 @@ tunnel. | `.btf` Base32 alphabet | `abcdefghijklmnopqrstuvwxyz234567`, no padding | | Address payload | `pubkey[32] || checksum[2]`, 55 Base32 chars | | Address checksum | `SHA256(".btf-checksum-v1" || pubkey)[0..2]` | -| Descriptor domain sep | `.btf-descriptor-v2` | -| Descriptor version | `2` | -| Descriptor signature | BIP340 Schnorr, deterministic (no aux rand) | +| Descriptor domain sep | `.btf-descriptor-v2`; optional onion sep `.btf-descriptor-v3-onion` | +| Descriptor version | `2`; `3` when a signed onion endpoint is present | +| Descriptor signature | BIP340 Schnorr, deterministic (no aux rand), v3 keeps `sig` plus `sig3` | | Nostr descriptor event | kind `38501`, `d = btf-descriptor-2` | | Nostr relay event | kind `38502`, `d = btf-relay-2` | | Nostr pool event | kind `38503`, `d = btf-pool-2` | diff --git a/docs/tor.md b/docs/tor.md index 69b29f1..b4f88f9 100644 --- a/docs/tor.md +++ b/docs/tor.md @@ -40,6 +40,8 @@ When Tor mode is enabled: - outbound Nostr relay dials use Tor; - outbound `.btf` rendezvous relay dials use Tor; +- outbound direct peer dials prefer signed `.onion` endpoints when a peer + advertises one; - destination names are sent to Tor as SOCKS5 domain-name CONNECT requests, so the local DNS resolver does not see them; - direct `.onion` dials are refused unless a SOCKS5/Tor proxy is enabled; @@ -98,7 +100,7 @@ bitflash -operator -tor \ -announcerelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 ``` -## Full-node hidden service +## Direct full-node hidden service The legacy P2P listener still binds to port `8433`. You can expose it through Tor manually: @@ -108,6 +110,35 @@ HiddenServiceDir /var/lib/tor/bitflash-node/ HiddenServicePort 8433 127.0.0.1:8433 ``` -Current automatic peer discovery is `.btf`/rendezvous based, not onion-address -gossip. Treat direct full-node onion access as an operator/admin tool unless a -future release adds first-class onion peer advertisements. +After restarting Tor, read the generated onion name: + +```bash +sudo cat /var/lib/tor/bitflash-node/hostname +``` + +Then start Bitflash with Tor outbound enabled and advertise that hidden service: + +```bash +bitflash -tor \ + -onionservice=exampleexampleexampleexampleexampleexampleexampleexample.onion:8433 +``` + +That onion endpoint is signed into this node's self-certifying `.btf` +descriptor. New peers that resolve the descriptor and also run with `-tor` try +the `.onion:8433` P2P connection first. If it fails, they fall back to the +ordinary encrypted rendezvous path. Older nodes ignore the onion field and still +use the rendezvous relay because the descriptor keeps the legacy signature too. + +If the node has `-onionservice` but has not registered at any rendezvous relay +yet, it can still publish an onion-capable descriptor with +`meeting_node = rendezvous-pending`. Tor-capable peers can dial the hidden +service directly from that descriptor. Rendezvous becomes a fallback, not a +precondition for being discoverable. + +This means rendezvous relays stop being the only way to reach a node that has +published a hidden service. They remain useful bootstrap/fallback infrastructure, +but an already-discovered onion-capable peer can be reached directly over Tor. + +The node does not create or rotate Tor hidden services itself. Tor owns the +private key under `HiddenServiceDir`; back it up if you want the onion name to +stay stable. diff --git a/src/btfaddr.cpp b/src/btfaddr.cpp index 6edcc91..ce2ac86 100644 --- a/src/btfaddr.cpp +++ b/src/btfaddr.cpp @@ -19,6 +19,8 @@ #include #include #include +#include +#include using json = nlohmann::json; @@ -166,10 +168,47 @@ static bool HexDecode(const std::string& s, unsigned char* out, size_t n) return true; } +bool NormalizeOnionEndpoint(const std::string& in, std::string& out) +{ + out.clear(); + std::string s = ToLowerTrim(in); + size_t colon = s.rfind(':'); + if (colon == std::string::npos || colon == 0 || colon + 1 >= s.size()) + return false; + + std::string host = s.substr(0, colon); + std::string port = s.substr(colon + 1); + while (!host.empty() && host.back() == '.') host.pop_back(); + static const char* suffix = ".onion"; + const size_t suffixLen = strlen(suffix); + if (host.size() != 56 + suffixLen) + return false; + if (host.compare(host.size() - suffixLen, suffixLen, suffix) != 0) + return false; + for (size_t i = 0; i < 56; i++) + { + char c = host[i]; + if (!((c >= 'a' && c <= 'z') || (c >= '2' && c <= '7'))) + return false; + } + for (size_t i = 0; i < port.size(); i++) + if (!isdigit((unsigned char)port[i])) + return false; + + errno = 0; + char* end = NULL; + long nPort = strtol(port.c_str(), &end, 10); + if (errno == ERANGE || end == NULL || *end != '\0' || nPort <= 0 || nPort > 65535) + return false; + + out = host + ":" + port; + return true; +} + // Digest signed over the descriptor fields (domain-separated). -static void DescriptorDigest(const unsigned char pubkey[32], const std::string& enc, - const std::string& meeting_node, uint64_t created, - unsigned char out[32]) +static void DescriptorDigestV2(const unsigned char pubkey[32], const std::string& enc, + const std::string& meeting_node, uint64_t created, + unsigned char out[32]) { static const char* DS = ".btf-descriptor-v2"; SHA256_CTX ctx; @@ -184,9 +223,27 @@ static void DescriptorDigest(const unsigned char pubkey[32], const std::string& SHA256_Final(out, &ctx); } +static void DescriptorDigestV3(const unsigned char pubkey[32], const std::string& enc, + const std::string& meeting_node, const std::string& onion, + uint64_t created, unsigned char out[32]) +{ + static const char* DS = ".btf-descriptor-v3-onion"; + SHA256_CTX ctx; + SHA256_Init(&ctx); + SHA256_Update(&ctx, (const unsigned char*)DS, strlen(DS)); + SHA256_Update(&ctx, pubkey, 32); + SHA256_Update(&ctx, (const unsigned char*)enc.data(), enc.size()); + SHA256_Update(&ctx, (const unsigned char*)meeting_node.data(), meeting_node.size()); + SHA256_Update(&ctx, (const unsigned char*)onion.data(), onion.size()); + unsigned char be[8]; + for (int i = 0; i < 8; i++) be[i] = (unsigned char)((created >> (8 * (7 - i))) & 0xff); + SHA256_Update(&ctx, be, 8); + SHA256_Final(out, &ctx); +} + std::string SignDescriptor(void* ctxv, const unsigned char seckey[32], const std::string& enc, const std::string& meeting_node, - uint64_t created) + uint64_t created, const std::string& onion) { secp256k1_context* ctx = (secp256k1_context*)ctxv; secp256k1_keypair kp; @@ -197,17 +254,28 @@ std::string SignDescriptor(void* ctxv, const unsigned char seckey[32], secp256k1_xonly_pubkey_serialize(ctx, pubkey, &xpub); unsigned char digest[32]; - DescriptorDigest(pubkey, enc, meeting_node, created, digest); + DescriptorDigestV2(pubkey, enc, meeting_node, created, digest); unsigned char sig[64]; // Deterministic signature (NULL aux_rand), matching Itzal's no-aux-rand signing. if (!secp256k1_schnorrsig_sign32(ctx, sig, digest, &kp, NULL)) return std::string(); json v = json::object(); - v["v"] = 2; + v["v"] = onion.empty() ? 2 : 3; v["pubkey"] = HexEncode(pubkey, 32); v["enc"] = enc; v["meeting_node"] = meeting_node; + if (!onion.empty()) + { + std::string onionNorm; + if (!NormalizeOnionEndpoint(onion, onionNorm)) return std::string(); + unsigned char digest3[32]; + DescriptorDigestV3(pubkey, enc, meeting_node, onionNorm, created, digest3); + unsigned char sig3[64]; + if (!secp256k1_schnorrsig_sign32(ctx, sig3, digest3, &kp, NULL)) return std::string(); + v["onion"] = onionNorm; + v["sig3"] = HexEncode(sig3, 64); + } v["created"] = created; v["sig"] = HexEncode(sig, 64); return v.dump(); @@ -242,15 +310,35 @@ bool VerifyDescriptor(void* ctxv, const std::string& jsonStr, if (!HexDecode(v["sig"].get(), sig, 64)) return false; unsigned char digest[32]; - DescriptorDigest(expect_pubkey, enc, meeting_node, created, digest); + DescriptorDigestV2(expect_pubkey, enc, meeting_node, created, digest); secp256k1_xonly_pubkey xpub; if (!secp256k1_xonly_pubkey_parse(ctx, &xpub, expect_pubkey)) return false; if (!secp256k1_schnorrsig_verify(ctx, sig, digest, 32, &xpub)) return false; + std::string onion; + if (v.contains("onion")) + { + if (v["onion"].is_string() && v.contains("sig3") && v["sig3"].is_string()) + { + std::string onionNorm; + std::string onionRaw = v["onion"].get(); + if (NormalizeOnionEndpoint(onionRaw, onionNorm) && onionNorm == onionRaw) + { + unsigned char sig3[64]; + unsigned char digest3[32]; + DescriptorDigestV3(expect_pubkey, enc, meeting_node, onionNorm, created, digest3); + if (HexDecode(v["sig3"].get(), sig3, 64) && + secp256k1_schnorrsig_verify(ctx, sig3, digest3, 32, &xpub)) + onion = onionNorm; + } + } + } + memcpy(out.pubkey, expect_pubkey, 32); out.enc = enc; out.meeting_node = meeting_node; + out.onion = onion; out.created = created; return true; } diff --git a/src/btfaddr.h b/src/btfaddr.h index 1addf8f..55a3ebd 100644 --- a/src/btfaddr.h +++ b/src/btfaddr.h @@ -52,15 +52,21 @@ struct Descriptor unsigned char pubkey[32]; std::string enc; // service x25519 public key (hex) for the E2E channel std::string meeting_node; // rendezvous node the service is registered at + std::string onion; // optional direct Tor hidden-service endpoint (host.onion:port) uint64_t created; }; +// Normalize and validate a direct Tor hidden-service endpoint. Only v3 onion +// names are accepted: 56 base32 chars plus ".onion", followed by ":PORT". +bool NormalizeOnionEndpoint(const std::string& in, std::string& out); + // Sign a descriptor. `ctx` is a secp256k1_context* (passed as void* to keep this // header free of secp headers); `seckey` is the 32-byte service secret key. // Returns the descriptor JSON as a string, or "" on failure. std::string SignDescriptor(void* ctx, const unsigned char seckey[32], const std::string& enc, const std::string& meeting_node, - uint64_t created); + uint64_t created, + const std::string& onion = std::string()); // Verify a descriptor is validly signed by `expect_pubkey` (the key the resolved // `.btf` address encodes). Returns true and fills `out` on success. diff --git a/src/main_gui.cpp b/src/main_gui.cpp index bef8503..239fb60 100644 --- a/src/main_gui.cpp +++ b/src/main_gui.cpp @@ -1,6 +1,7 @@ // Bitflash entry point -- starts node threads then runs GUI (or headless). #include "headers_core.h" +#include "nostr.h" #include "proxy.h" #include "selftest.h" #include "walletcmd.h" @@ -134,10 +135,11 @@ static void PrintUsage() printf(" /connectbtf=PEER_BTF_ADDRESS\n"); printf(" /rvrelay=HOST:PORT\n"); printf(" /announcerelay=HOST:PORT\n"); + printf(" /onionservice=HOST.onion:PORT (advertise this node's Tor hidden service)\n"); printf("\n"); printf("Network:\n"); printf(" /port=N (P2P listen port, default 8433)\n"); - printf(" /socks=HOST:PORT (SOCKS5 proxy for outbound Nostr and .btf relay dials)\n"); + printf(" /socks=HOST:PORT (SOCKS5 proxy for Nostr, .btf relay, and onion dials)\n"); printf(" /tor[=HOST:PORT] (Tor mode; default SOCKS5 proxy is 127.0.0.1:9050)\n"); printf(" /btfseed=ADDRESS:ENCHEX (extra bootstrap peer, repeatable)\n"); printf("\n"); @@ -286,6 +288,17 @@ static void ParseStartupArguments(int argc, char* argv[]) if (!announceRelay.empty()) strBtfAnnounceRelay = announceRelay; + string onionService = argval2(argc, argv, "/onionservice", "-onionservice"); + if (!onionService.empty()) + { + string err; + if (!BtfSetLocalOnionEndpoint(onionService, err)) + fprintf(stderr, "Ignoring /onionservice=%s: %s\n", onionService.c_str(), err.c_str()); + else + fprintf(stderr, "Direct onion peer endpoint advertised: %s\n", + BtfLocalOnionEndpoint().c_str()); + } + bool fTorMode = arg(argc, argv, "/tor") || arg(argc, argv, "-tor"); if (fTorMode) { diff --git a/src/net.cpp b/src/net.cpp index c24547e..ab17332 100644 --- a/src/net.cpp +++ b/src/net.cpp @@ -17,6 +17,8 @@ #include #endif #include +#include +#include #include "btfaddr.h" #include "btftunnel.h" #include "proxy.h" @@ -170,7 +172,7 @@ static const char* pszSockSite[SOCK_SITES] = { "external-ip probe", "listen socket", "inbound accept", "rendezvous dial", "rendezvous listen", "rendezvous accept", "loopback pair listener", "loopback pair app end", "loopback pair pump end", - "nostr relay", "stratum bridge" + "nostr relay", "direct onion peer", "stratum bridge" }; static string SockAccountingText() @@ -627,6 +629,7 @@ struct CachedBtfPeer { string btfAddr; string meeting; + string onion; string encHex; int64 lastSeen; // The peer's own signed descriptor, as it announced itself. Kept verbatim @@ -685,6 +688,8 @@ void LoadCachedBtfPeers(vector& out) CachedBtfPeer p; p.btfAddr = o["btf"].get(); p.meeting = o["meeting"].get(); + if (o.contains("onion") && o["onion"].is_string()) + btf::NormalizeOnionEndpoint(o["onion"].get(), p.onion); p.encHex = o["enc"].get(); p.lastSeen = o.value("seen", (int64)0); if (o.contains("desc") && o["desc"].is_string()) @@ -708,7 +713,8 @@ static CCriticalSection cs_btfPeerCache; // learned about by exchange must not cost us the ability to pass it on. static void RememberBtfPeer(const string& strBtfAddr, const string& strMeeting, const unsigned char enc_pub[32], - const string& strDesc = string()) + const string& strDesc = string(), + const string& strOnion = string()) { CRITICAL_BLOCK(cs_btfPeerCache) { @@ -716,11 +722,15 @@ static void RememberBtfPeer(const string& strBtfAddr, const string& strMeeting, LoadCachedBtfPeers(peers); string strKeepDesc = strDesc; + string strKeepOnion; + btf::NormalizeOnionEndpoint(strOnion, strKeepOnion); for (size_t i = 0; i < peers.size(); i++) if (peers[i].btfAddr == strBtfAddr) { if (strKeepDesc.empty()) strKeepDesc = peers[i].desc; + if (strKeepOnion.empty()) + strKeepOnion = peers[i].onion; peers.erase(peers.begin() + i); break; } @@ -728,6 +738,7 @@ static void RememberBtfPeer(const string& strBtfAddr, const string& strMeeting, CachedBtfPeer p; p.btfAddr = strBtfAddr; p.meeting = strMeeting; + p.onion = strKeepOnion; p.encHex = BytesToHex(enc_pub, 32); p.lastSeen = GetTime(); p.desc = strKeepDesc; @@ -742,6 +753,8 @@ static void RememberBtfPeer(const string& strBtfAddr, const string& strMeeting, nlohmann::json o; o["btf"] = q.btfAddr; o["meeting"] = q.meeting; + if (!q.onion.empty()) + o["onion"] = q.onion; o["enc"] = q.encHex; o["seen"] = q.lastSeen; if (!q.desc.empty()) @@ -830,7 +843,7 @@ static int TryBtfSeeds() foreach(const string& strRelay, relays) { if (fShutdown) return nConnected; - if (ConnectNodeBtfResolved(s.first, strRelay, enc)) + if (ConnectNodeBtfResolved(s.first, strRelay, "", enc)) { LogPrint("net", "btfseed: reached %s via %s\n", s.first.c_str(), strRelay.c_str()); @@ -868,7 +881,7 @@ void ThreadReconnectCachedBtfPeers(void* parg) if (fShutdown) return; unsigned char enc[32]; if (!HexToBytes(p.encHex, enc, 32)) continue; - if (ConnectNodeBtfResolved(p.btfAddr, p.meeting, enc)) + if (ConnectNodeBtfResolved(p.btfAddr, p.meeting, p.onion, enc)) { nConnected++; if (nConnected >= 8) break; // enough to bootstrap; the rest can wait @@ -959,14 +972,37 @@ int BtfPexAccept(const vector& vDesc) if (!HexToBytes(d.enc, enc, 32)) continue; - RememberBtfPeer(strAddr, d.meeting_node, enc, strDesc); + RememberBtfPeer(strAddr, d.meeting_node, enc, strDesc, d.onion); nKept++; } return nKept; } +static bool SplitHostPort(const string& strHostPort, string& strHost, unsigned short& nPort) +{ + size_t colon = strHostPort.rfind(':'); + if (colon == string::npos || colon == 0 || colon + 1 >= strHostPort.size()) + return false; + string port = strHostPort.substr(colon + 1); + for (size_t i = 0; i < port.size(); i++) + if (!isdigit((unsigned char)port[i])) + return false; + + errno = 0; + char* end = NULL; + long n = strtol(port.c_str(), &end, 10); + if (errno == ERANGE || end == NULL || *end != '\0' || n <= 0 || n > 65535) + return false; + + strHost = strHostPort.substr(0, colon); + nPort = (unsigned short)n; + return true; +} + static CNode* ConnectNodeBtfTail(const string& strBtfAddr, const unsigned char pk[32], - const string& strMeeting, const unsigned char enc_pub[32]) + const string& strMeeting, const string& strOnion, + const unsigned char enc_pub[32], + const string& strDesc = string()) { CAddress addr = BtfMarkerAddr(pk); CNode* pnode = FindNode(addr.ip); @@ -976,16 +1012,42 @@ static CNode* ConnectNodeBtfTail(const string& strBtfAddr, const unsigned char p return pnode; } - BtfChurnNoteDialAttempt(strBtfAddr, strMeeting); - size_t colon = strMeeting.rfind(':'); - if (colon == string::npos) + string strOnionNorm; + if (BtfSocks5ProxyEnabled() && btf::NormalizeOnionEndpoint(strOnion, strOnionNorm)) { - BtfChurnNoteDialResult(strBtfAddr, strMeeting, false); - return NULL; + string strOnionHost; + unsigned short nOnionPort = 0; + if (SplitHostPort(strOnionNorm, strOnionHost, nOnionPort)) + { + BtfChurnNoteDialAttempt(strBtfAddr, strOnionNorm); + SOCKET hOnionSocket = BtfConnectSocket(strOnionHost, nOnionPort, SOCK_ONION_PEER, 20); + if (hOnionSocket != INVALID_SOCKET) + { + if (fDebug) + LogPrint("net", "connected %s via direct onion %s\n", + strBtfAddr.c_str(), strOnionNorm.c_str()); + BtfChurnNoteDialResult(strBtfAddr, strOnionNorm, true); + RememberBtfPeer(strBtfAddr, strMeeting, enc_pub, strDesc, strOnionNorm); + + pnode = new CNode(hOnionSocket, addr, false); + pnode->strBtfAddr = strBtfAddr; + pnode->strBtfMeeting = string("onion ") + strOnionNorm; + pnode->AddRef(); + CRITICAL_BLOCK(cs_vNodes) + vNodes.push_back(pnode); + return pnode; + } + if (fDebug) + LogPrint("net", "ConnectNodeBtf: direct onion to %s at %s failed, falling back to rendezvous\n", + strBtfAddr.c_str(), strOnionNorm.c_str()); + BtfChurnNoteDialResult(strBtfAddr, strOnionNorm, false); + } } - string strHost = strMeeting.substr(0, colon); - int nPort = atoi(strMeeting.substr(colon + 1).c_str()); - if (nPort <= 0 || nPort > 65535) + + BtfChurnNoteDialAttempt(strBtfAddr, strMeeting); + string strHost; + unsigned short nPort = 0; + if (!SplitHostPort(strMeeting, strHost, nPort)) { BtfChurnNoteDialResult(strBtfAddr, strMeeting, false); return NULL; @@ -1005,7 +1067,7 @@ static CNode* ConnectNodeBtfTail(const string& strBtfAddr, const unsigned char p BtfChurnNoteDialResult(strBtfAddr, strMeeting, true); // This one answered -- worth trying first next time we start. - RememberBtfPeer(strBtfAddr, strMeeting, enc_pub); + RememberBtfPeer(strBtfAddr, strMeeting, enc_pub, strDesc, strOnionNorm); // Add node pnode = new CNode(hSocket, addr, false); @@ -1035,9 +1097,10 @@ CNode* ConnectNodeBtf(const string& strBtfAddr) LogPrint("net", "trying %s\n", strBtfAddr.c_str()); string strMeeting; + string strOnion; unsigned char enc_pub[32]; BtfChurnNoteResolveAttempt(); - if (!BtfResolve(strBtfAddr, strMeeting, enc_pub)) + if (!BtfResolve(strBtfAddr, strMeeting, enc_pub, &strOnion)) { BtfChurnNoteResolveResult(false); if (fDebug) @@ -1045,7 +1108,7 @@ CNode* ConnectNodeBtf(const string& strBtfAddr) return NULL; } BtfChurnNoteResolveResult(true); - return ConnectNodeBtfTail(strBtfAddr, pk, strMeeting, enc_pub); + return ConnectNodeBtfTail(strBtfAddr, pk, strMeeting, strOnion, enc_pub); } // Same as ConnectNodeBtf, but skips the Nostr resolve step because the caller @@ -1054,7 +1117,8 @@ CNode* ConnectNodeBtf(const string& strBtfAddr) // address). Makes zero relay connections -- only talks to the peer's // rendezvous meeting node. CNode* ConnectNodeBtfResolved(const string& strBtfAddr, const string& strMeeting, - const unsigned char enc_pub[32]) + const string& strOnion, const unsigned char enc_pub[32], + const string& strDesc) { unsigned char pk[32]; if (!btf::ParseAddress(strBtfAddr, pk)) @@ -1064,7 +1128,7 @@ CNode* ConnectNodeBtfResolved(const string& strBtfAddr, const string& strMeeting } if (BtfLocalAddress() == strBtfAddr) return NULL; // ourselves - return ConnectNodeBtfTail(strBtfAddr, pk, strMeeting, enc_pub); + return ConnectNodeBtfTail(strBtfAddr, pk, strMeeting, strOnion, enc_pub, strDesc); } // Anonymous inbound listener (this node's `.btf` hidden service). Registers at diff --git a/src/net.h b/src/net.h index feecf96..9300169 100644 --- a/src/net.h +++ b/src/net.h @@ -28,7 +28,8 @@ enum bool GetMyExternalIP(unsigned int& ipRet); CNode* ConnectNodeBtf(const string& strBtfAddr); CNode* ConnectNodeBtfResolved(const string& strBtfAddr, const string& strMeeting, - const unsigned char enc_pub[32]); + const string& strOnion, const unsigned char enc_pub[32], + const string& strDesc=string()); void ThreadBtfAccept(void* parg); void ThreadBtfConnect(void* parg); extern string strBtfConnect; diff --git a/src/nostr.cpp b/src/nostr.cpp index d286767..0c1b4b6 100644 --- a/src/nostr.cpp +++ b/src/nostr.cpp @@ -112,6 +112,8 @@ vector vBtfMeetingRelays = { }; string strBtfActiveRelay; static CCriticalSection cs_activeRelay; +static string strBtfOnionEndpoint; +static CCriticalSection cs_onionEndpoint; // Only the rendezvous we actually registered at (BtfSetActiveRelay, called from // ThreadBtfAccept once registration succeeds). Empty until then, on purpose: it @@ -133,6 +135,26 @@ void BtfSetActiveRelay(const string& relay) strBtfActiveRelay = relay; } +bool BtfSetLocalOnionEndpoint(const string& endpoint, string& errOut) +{ + string norm; + if (!btf::NormalizeOnionEndpoint(endpoint, norm)) + { + errOut = "expected v3 onion endpoint HOST.onion:PORT"; + return false; + } + CRITICAL_BLOCK(cs_onionEndpoint) + strBtfOnionEndpoint = norm; + return true; +} + +string BtfLocalOnionEndpoint() +{ + CRITICAL_BLOCK(cs_onionEndpoint) + return strBtfOnionEndpoint; + return ""; +} + // A relay this node advertises on Nostr for others to discover (/announcerelay). string strBtfAnnounceRelay; // Relays learned from other operators' Nostr announcements. @@ -370,10 +392,16 @@ std::string BtfLocalDescriptor() string meeting; CRITICAL_BLOCK(cs_activeRelay) meeting = strBtfActiveRelay; + string onion = BtfLocalOnionEndpoint(); if (meeting.empty()) - return ""; + { + if (onion.empty()) + return ""; + meeting = "rendezvous-pending"; + } return btf::SignDescriptor(g_nostrKey.ctx, g_nostrKey.seckey, - g_nostrKey.EncPubHex(), meeting, (uint64_t)GetTime()); + g_nostrKey.EncPubHex(), meeting, (uint64_t)GetTime(), + onion); } // The shared secp256k1 context, so net.cpp can verify descriptors that arrive @@ -806,17 +834,21 @@ static void PublishDescriptor(CWebSocket& ws, CNostrKey& key) // meeting_node is the rendezvous relay this node's hidden service // (ThreadBtfAccept in net.cpp) is registered at -- where clients dial us. string meeting_node = BtfActiveRelay(); + string onion = BtfLocalOnionEndpoint(); if (meeting_node.empty()) { - // Not registered anywhere yet. Publishing now would advertise a meeting - // node we can't be reached at, and peers would waste dials on it. The - // ThreadNostrSeed loop republishes on every pass, so skipping costs - // nothing but the delay until ThreadBtfAccept registers. - LogPrint("nostr", "Nostr: skipping descriptor publish, no rendezvous registered yet\n"); - return; + if (onion.empty()) + { + // Not registered anywhere yet, and no direct onion endpoint exists. + // Publishing now would advertise a meeting node we can't be reached at. + LogPrint("nostr", "Nostr: skipping descriptor publish, no rendezvous registered yet\n"); + return; + } + meeting_node = "rendezvous-pending"; } string desc = btf::SignDescriptor(key.ctx, key.seckey, key.EncPubHex(), - meeting_node, (uint64_t)GetTime()); + meeting_node, (uint64_t)GetTime(), + onion); if (desc.empty()) return; json tags = json::array({ json::array({ "d", BTF_DESC_DTAG }) }); @@ -1018,7 +1050,7 @@ static void HandlePoolAnnouncement(const json& ev) // Resolve a `.btf` address via a relay: fetch the descriptor published by that // address's key and verify it self-certifies under the decoded pubkey. static bool ResolveDescriptor(CWebSocket& ws, void* ctx, const string& btfAddr, - btf::Descriptor& out) + btf::Descriptor& out, string* rawOut = NULL) { unsigned char pk[32]; if (!btf::ParseAddress(btfAddr, pk)) @@ -1045,7 +1077,11 @@ static bool ResolveDescriptor(CWebSocket& ws, void* ctx, const string& btfAddr, const json& ev = j[2]; if (ev.contains("content") && btf::VerifyDescriptor(ctx, ev["content"].get(), pk, out)) + { + if (rawOut) + *rawOut = ev["content"].get(); return true; + } } else if (t == "EOSE") break; @@ -1057,8 +1093,10 @@ static bool ResolveDescriptor(CWebSocket& ws, void* ctx, const string& btfAddr, // public discovery relays for the owner's self-certified descriptor. Used by // ConnectNodeBtf in net.cpp. bool BtfResolve(const std::string& btfAddr, std::string& meetingHostPort, - unsigned char enc_pub[32]) + unsigned char enc_pub[32], std::string* onionHostPort) { + if (onionHostPort) + onionHostPort->clear(); if (!EnsureNostrKey()) return false; for (int i = 0; i < nNostrRelays && !fShutdown; i++) @@ -1076,10 +1114,15 @@ bool BtfResolve(const std::string& btfAddr, std::string& meetingHostPort, } lease.fOk = true; if (d.meeting_node.empty() || d.meeting_node == "rendezvous-pending") - continue; // descriptor predates the owner's relay config + { + if (!onionHostPort || d.onion.empty()) + continue; // caller needs rendezvous, or descriptor has no direct path + } if (!HexDecode(d.enc, enc_pub, 32)) continue; meetingHostPort = d.meeting_node; + if (onionHostPort) + *onionHostPort = d.onion; return true; } CATCH_PRINT_EXCEPTION("BtfResolve") @@ -1168,12 +1211,15 @@ bool BtfResolveMany(const std::vector& btfAddrs, btf::Descriptor d; if (!btf::VerifyDescriptor(g_nostrKey.ctx, ev["content"].get(), pk, d)) continue; - if (d.meeting_node.empty() || d.meeting_node == "rendezvous-pending") + if ((d.meeting_node.empty() || d.meeting_node == "rendezvous-pending") && + d.onion.empty()) continue; BtfResolvedPeer rp; if (!HexDecode(d.enc, rp.enc_pub, 32)) continue; rp.meetingHostPort = d.meeting_node; + rp.onionHostPort = d.onion; + rp.descriptor = ev["content"].get(); out[ai->second] = rp; pending.erase(ai->second); } @@ -1650,11 +1696,13 @@ static void ConnectDiscoveredBtfPeers() DialCtx* ctx = (DialCtx*)arg; string addr = ctx->addr; string meeting = ctx->rp.meetingHostPort; + string onion = ctx->rp.onionHostPort; + string desc = ctx->rp.descriptor; unsigned char enc_pub[32]; memcpy(enc_pub, ctx->rp.enc_pub, 32); delete ctx; - CNode* pnode = ConnectNodeBtfResolved(addr, meeting, enc_pub); + CNode* pnode = ConnectNodeBtfResolved(addr, meeting, onion, enc_pub, desc); CRITICAL_BLOCK(cs_btfPeerState) { if (pnode) diff --git a/src/nostr.h b/src/nostr.h index b5eb5e3..dc4288c 100644 --- a/src/nostr.h +++ b/src/nostr.h @@ -40,6 +40,13 @@ std::string BtfActiveRelay(); // Record the relay this node just registered its service at (thread-safe). void BtfSetActiveRelay(const std::string& relay); +// Optional Tor hidden-service endpoint for this node's normal P2P listener. +// Set by /onionservice=HOST.onion:PORT. The endpoint is signed into this node's +// .btf descriptor, so peers with -tor can dial it directly before falling back +// to rendezvous relays. +bool BtfSetLocalOnionEndpoint(const std::string& endpoint, std::string& errOut); +std::string BtfLocalOnionEndpoint(); + // If set (via /announcerelay=host:port), this node announces that relay on // Nostr so other nodes discover it automatically -- no manual seed-list edit. extern std::string strBtfAnnounceRelay; @@ -94,12 +101,15 @@ void* BtfSecpContext(); // self-certified descriptor and return its meeting node ("host:port") and // x25519 public key for the end-to-end channel. bool BtfResolve(const std::string& btfAddr, std::string& meetingHostPort, - unsigned char enc_pub[32]); + unsigned char enc_pub[32], + std::string* onionHostPort = NULL); // Result of resolving one .btf address via BtfResolveMany. struct BtfResolvedPeer { std::string meetingHostPort; + std::string onionHostPort; + std::string descriptor; unsigned char enc_pub[32]; }; diff --git a/src/readme.txt b/src/readme.txt index 794fe20..1e9cffc 100644 --- a/src/readme.txt +++ b/src/readme.txt @@ -43,11 +43,11 @@ Command-line options (in addition to the original ones) ------------------------------------------------------- /port=N listen on P2P port N (default 8433) /socks=HOST:PORT - route outbound Nostr and .btf rendezvous dials through a - SOCKS5 proxy, e.g. /socks=127.0.0.1:9050 for local Tor + route outbound Nostr, .btf rendezvous, and direct onion peer + dials through a SOCKS5 proxy, e.g. /socks=127.0.0.1:9050 for local Tor or /socks=[::1]:9050 for an IPv6 loopback proxy /tor[=HOST:PORT] - Tor mode for outbound Nostr and .btf rendezvous dials; + Tor mode for outbound Nostr, .btf rendezvous, and direct onion peer dials; defaults to the local Tor SOCKS5 proxy at 127.0.0.1:9050 /solomine mine without requiring a connected peer (local testing) /operator operator mode (run pool server) @@ -72,6 +72,8 @@ Command-line options (in addition to the original ones) override meeting relay list with one relay /announcerelay=HOST:PORT announce this relay on Nostr for discovery + /onionservice=HOST.onion:PORT + advertise this node's Tor hidden service in its signed .btf descriptor Roadmap diff --git a/src/selftest.cpp b/src/selftest.cpp index bcbf9d3..a0d9edd 100644 --- a/src/selftest.cpp +++ b/src/selftest.cpp @@ -6,6 +6,7 @@ // and against a temporary data directory. #include "headers_core.h" +#include "btfaddr.h" #include "bip32.h" #include "proxy.h" #include "selftest.h" @@ -1507,6 +1508,14 @@ static int RunSocks5ProxySelfTest() !BtfIsTorOnionHost("relay.example") && !BtfIsTorOnionHost(".onion"), "detects .onion hosts without accepting lookalikes") ? 0 : 1; + std::string onion; + nFail += Check(btf::NormalizeOnionEndpoint("ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCD.onion.:8433", onion) && + onion == "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcd.onion:8433", + "normalizes v3 onion peer endpoint") ? 0 : 1; + nFail += Check(!btf::NormalizeOnionEndpoint("short.onion:8433", onion) && + !btf::NormalizeOnionEndpoint("abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcd.onion:0", onion) && + !btf::NormalizeOnionEndpoint("abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabc1.onion:8433", onion), + "rejects malformed onion peer endpoints") ? 0 : 1; BtfClearSocks5Proxy(); nFail += Check(!BtfSocks5ProxyEnabled(), "starts disabled after clear") ? 0 : 1; diff --git a/src/sockcount.h b/src/sockcount.h index 5147f55..a63c6be 100644 --- a/src/sockcount.h +++ b/src/sockcount.h @@ -63,6 +63,7 @@ enum SOCK_PAIR_APP, // loopback pair: the app end handed to CNode SOCK_PAIR_PUMP, // loopback pair: the pump end SOCK_NOSTR, // outbound to a Nostr relay + SOCK_ONION_PEER, // direct outbound to a peer's Tor hidden service SOCK_STRATUM_BRIDGE, // local Stratum bridge listener/miner sockets SOCK_SITES }; diff --git a/src/test_btfaddr.cpp b/src/test_btfaddr.cpp index b40dcc1..8de426e 100644 --- a/src/test_btfaddr.cpp +++ b/src/test_btfaddr.cpp @@ -86,6 +86,7 @@ int main() } const std::string ENC_HEX = "aabbccddeeff00112233445566778899aabbccddeeff00112233445566778899"; + const std::string ONION = "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcd.onion:8433"; // helper: fresh keypair -> (seckey, xonly pubkey) auto newKey = [&](unsigned char sk[32], unsigned char pk[32]) { @@ -108,6 +109,17 @@ int main() CHECK(!btf::VerifyDescriptor(ctx, d, pk2, bad), "different key rejected"); } + printf("descriptor_signs_optional_onion_endpoint\n"); + { + unsigned char sk[32], pk[32]; newKey(sk, pk); + std::string d = btf::SignDescriptor(ctx, sk, ENC_HEX, "meeting-node-abc", 1000, ONION); + CHECK(!d.empty(), "sign produced onion descriptor"); + btf::Descriptor got; + CHECK(btf::VerifyDescriptor(ctx, d, pk, got), "onion descriptor verifies"); + CHECK(got.meeting_node == "meeting-node-abc" && got.enc == ENC_HEX && + got.created == 1000 && got.onion == ONION, "onion fields match"); + } + printf("descriptor_rejects_tampering\n"); { unsigned char sk[32], pk[32]; newKey(sk, pk); @@ -117,6 +129,16 @@ int main() CHECK(!btf::VerifyDescriptor(ctx, j.dump(), pk, o), "tampered meeting_node rejected"); json j2 = json::parse(d); j2["enc"] = "00000000000000000000000000000000000000000000000000000000deadbeef"; CHECK(!btf::VerifyDescriptor(ctx, j2.dump(), pk, o), "tampered enc rejected"); + + std::string d3 = btf::SignDescriptor(ctx, sk, ENC_HEX, "node-1", 5, ONION); + json j3 = json::parse(d3); + j3["onion"] = "bcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcde.onion:8433"; + CHECK(btf::VerifyDescriptor(ctx, j3.dump(), pk, o) && o.onion.empty(), + "tampered onion ignored rather than trusted"); + json j4 = json::parse(d3); + j4["sig3"] = std::string(128, '0'); + CHECK(btf::VerifyDescriptor(ctx, j4.dump(), pk, o) && o.onion.empty(), + "bad onion signature ignored rather than trusted"); } printf("full_resolve_chain_from_address\n"); -- cgit v1.2.3 From 53cc17d621f55b8904812fae5df90ef089bf5aaf Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Thu, 13 Aug 2026 11:54:35 -0300 Subject: net: add managed Tor hidden service bootstrap --- README.md | 13 +- docs/rendezvous.md | 8 +- docs/tor.md | 52 +++++- src/Makefile | 5 +- src/main_gui.cpp | 65 ++++--- src/makefile.mingw | 5 +- src/net.cpp | 2 + src/readme.txt | 3 + src/selftest.cpp | 37 +++- src/tor.cpp | 492 +++++++++++++++++++++++++++++++++++++++++++++++++++++ src/tor.h | 22 +++ 11 files changed, 665 insertions(+), 39 deletions(-) create mode 100644 src/tor.cpp create mode 100644 src/tor.h (limited to 'docs') diff --git a/README.md b/README.md index 181b6e8..a7ce2b2 100644 --- a/README.md +++ b/README.md @@ -339,6 +339,7 @@ Other options worth knowing: -port=N # P2P listen port, default 8433 -socks=HOST:PORT # SOCKS5 for outbound Nostr, .btf relay, and onion peer dials -tor[=HOST:PORT] # Tor mode; default local Tor SOCKS5 proxy is 127.0.0.1:9050 +-managedtor[=PATH] # start Tor, create a hidden service, advertise its onion -onionservice=HOST.onion:PORT # advertise this node's Tor hidden service -debug # verbose log; without it debug.log is nearly silent -help # full list @@ -357,10 +358,14 @@ proxy. IPv6 proxy endpoints use brackets, for example `-socks=[::1]:9050`. `-tor` is a privacy shorthand for the usual local Tor SOCKS5 listener at `127.0.0.1:9050`. It routes outbound Nostr discovery, `.btf` rendezvous dials, and signed direct `.onion` peer dials through Tor, and keeps external-IP probes -disabled. Use `-tor=HOST:PORT` when Tor listens somewhere else. To make your own -node reachable without a rendezvous relay, expose port `8433` as a Tor hidden -service and pass `-onionservice=HOST.onion:8433`; peers with `-tor` will try it -before falling back to rendezvous. +disabled. Use `-tor=HOST:PORT` when Tor listens somewhere else. + +`-managedtor[=PATH]` starts a Tor process for this node, writes a local `torrc`, +creates a v3 hidden service for port `8433`, routes outbound discovery through +that Tor instance, and signs the generated `.onion:8433` endpoint into the +node's `.btf` descriptor. Without `PATH`, Bitflash looks for `tor/tor.exe`, +`tor.exe`, or `tor` beside the binary and then on `PATH`. The hidden-service key +is stored under the Bitflash data directory in `managed-tor/onion-service/`. ### When something looks wrong diff --git a/docs/rendezvous.md b/docs/rendezvous.md index e1888b9..68b5df3 100644 --- a/docs/rendezvous.md +++ b/docs/rendezvous.md @@ -308,9 +308,11 @@ tunnel. unreachable, discovery stalls until one is reachable again. Relay TLS is verified, which closes a man in the middle path on discovery, but availability still rests on the relay set. -- **Direct onion is not automatic Tor configuration.** Bitflash signs and - advertises a hidden-service endpoint, and Tor-mode peers can dial it. The Tor - daemon still owns hidden-service creation, private keys, rotation and uptime. +- **Direct onion needs Tor.** With `-managedtor`, Bitflash starts a Tor process, + writes a local hidden-service config, and advertises the generated onion + endpoint. With `-onionservice`, an operator can still run Tor manually and + provide the endpoint. In both cases Tor owns the hidden-service private key and + uptime. ## 11. Wire reference diff --git a/docs/tor.md b/docs/tor.md index b4f88f9..1c0f589 100644 --- a/docs/tor.md +++ b/docs/tor.md @@ -36,6 +36,47 @@ If both `-tor` and `-socks` are present, Tor mode wins and the node prints a warning. That keeps a command line with `-tor` from silently becoming a custom proxy route. +## Managed Tor mode + +`-managedtor` starts a Tor process for this node, writes a local `torrc`, creates +a v3 hidden service for the normal Bitflash P2P listener, routes outbound +discovery through that Tor instance, and signs the generated `.onion:8433` +endpoint into the node's `.btf` descriptor. + +```bash +bitflash -managedtor +``` + +Without a value, Bitflash looks for Tor in this order: + +1. `tor/tor.exe` or `tor/tor` beside the Bitflash binary; +2. `tor.exe` or `tor` beside the Bitflash binary; +3. `tor.exe` or `tor` on `PATH`. + +To point at a specific Tor executable: + +```bash +bitflash -managedtor=/opt/tor/bin/tor +bitflash -managedtor=C:\Tor Browser\Browser\TorBrowser\Tor\tor.exe +``` + +Managed Tor writes its state under the Bitflash data directory: + +```text +managed-tor/ + torrc + data/ + onion-service/ +``` + +The private key for the onion address lives in `managed-tor/onion-service/`. +Back up that directory if you want the same onion address after moving the node. +Delete it only if you intentionally want a fresh onion endpoint. + +`-managedtor` takes precedence over `-tor` and `-socks`, because it must choose +the local SOCKS port it starts. Diagnostics report the current managed Tor +state, including the SOCKS port and onion endpoint once Tor has generated it. + When Tor mode is enabled: - outbound Nostr relay dials use Tor; @@ -100,10 +141,10 @@ bitflash -operator -tor \ -announcerelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 ``` -## Direct full-node hidden service +## Manual direct full-node hidden service -The legacy P2P listener still binds to port `8433`. You can expose it through -Tor manually: +Managed Tor is the preferred path for ordinary nodes. Operators can still expose +the P2P listener through a separately managed Tor service: ```text HiddenServiceDir /var/lib/tor/bitflash-node/ @@ -139,6 +180,5 @@ This means rendezvous relays stop being the only way to reach a node that has published a hidden service. They remain useful bootstrap/fallback infrastructure, but an already-discovered onion-capable peer can be reached directly over Tor. -The node does not create or rotate Tor hidden services itself. Tor owns the -private key under `HiddenServiceDir`; back it up if you want the onion name to -stay stable. +Tor owns the private key under `HiddenServiceDir`; back it up if you want the +onion name to stay stable. diff --git a/src/Makefile b/src/Makefile index 511fe79..19ad33a 100644 --- a/src/Makefile +++ b/src/Makefile @@ -53,7 +53,7 @@ LIBS = -L$(SECP_LIB) -lsecp256k1 \ HEADERS = headers_core.h headers.h util.h main.h serialize.h uint256.h \ key.h bignum.h script.h db.h base58.h net.h nostr.h \ randomx_pow.h btfaddr.h btfchan.h btfrv.h btftunnel.h market.h \ - selftest.h walletcmd.h compat.h sockcount.h proxy.h font_roboto.h + selftest.h walletcmd.h compat.h sockcount.h proxy.h tor.h font_roboto.h IMGUI_OBJS = obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o \ obj/imgui_widgets.o obj/imgui_impl_glfw.o obj/imgui_impl_opengl3.o @@ -61,7 +61,7 @@ IMGUI_OBJS = obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o \ NODE_OBJS = obj/util.o obj/script.o obj/db.o obj/net.o obj/main.o \ obj/market.o obj/sha.o obj/nostr.o obj/randomx_pow.o \ obj/btfaddr.o obj/btfchan.o obj/btfrv.o obj/btftunnel.o obj/rpc.o \ - obj/selftest.o obj/bip32.o obj/walletcmd.o obj/proxy.o + obj/selftest.o obj/bip32.o obj/walletcmd.o obj/proxy.o obj/tor.o GUI_OBJS = obj/gui.o obj/main_gui.o @@ -194,5 +194,6 @@ tests: test_bip32 bitflash-node ./bitflash-node -selftest=pool-stratum ./bitflash-node -selftest=parse-money ./bitflash-node -selftest=socks5-proxy + ./bitflash-node -selftest=managed-tor .PHONY: tests fuzz-net-message-smoke fuzz-script-smoke diff --git a/src/main_gui.cpp b/src/main_gui.cpp index 239fb60..deed471 100644 --- a/src/main_gui.cpp +++ b/src/main_gui.cpp @@ -4,6 +4,7 @@ #include "nostr.h" #include "proxy.h" #include "selftest.h" +#include "tor.h" #include "walletcmd.h" #include // hardware_concurrency, to sanity-check /genproclimit #ifndef _WIN32 @@ -113,7 +114,7 @@ static void PrintUsage() printf(" /gen\n"); printf(" /nogui or /daemon\n"); printf(" /selftest=wallet-keypool, wallet-hd, wallet-format, wallet-crypto, wallet-encrypt,\n"); - printf(" net-message, consensus-limits, pool-stratum, parse-money, or socks5-proxy\n"); + printf(" net-message, consensus-limits, pool-stratum, parse-money, socks5-proxy, or managed-tor\n"); printf("\n"); printf("Mining mode:\n"); printf(" /operator\n"); @@ -141,6 +142,7 @@ static void PrintUsage() printf(" /port=N (P2P listen port, default 8433)\n"); printf(" /socks=HOST:PORT (SOCKS5 proxy for Nostr, .btf relay, and onion dials)\n"); printf(" /tor[=HOST:PORT] (Tor mode; default SOCKS5 proxy is 127.0.0.1:9050)\n"); + printf(" /managedtor[=PATH] (start Tor, create a hidden service, advertise its onion)\n"); printf(" /btfseed=ADDRESS:ENCHEX (extra bootstrap peer, repeatable)\n"); printf("\n"); printf("Wallet:\n"); @@ -284,6 +286,24 @@ static void ParseStartupArguments(int argc, char* argv[]) vBtfMeetingRelays.push_back(rvRelay); } + // net.cpp has described nListenPort as "tunable via /port" since it was + // written, but nothing ever read the option, so the port was fixed at 8433 + // and a second node could not start on a machine already running one. Parse + // it before Tor setup so a managed hidden service maps the real listener. + string strPort = argval2(argc, argv, "/port", "-port"); + if (!strPort.empty()) + { + int nPort = atoi(strPort.c_str()); + if (nPort <= 0 || nPort > 65535) + fprintf(stderr, "Ignoring /port=%s: not a port number\n", strPort.c_str()); + else + { + nListenPort = htons((unsigned short)nPort); + // Or we would announce a port we never bound. + addrLocalHost.port = nListenPort; + } + } + string announceRelay = argval2(argc, argv, "/announcerelay", "-announcerelay"); if (!announceRelay.empty()) strBtfAnnounceRelay = announceRelay; @@ -299,10 +319,23 @@ static void ParseStartupArguments(int argc, char* argv[]) BtfLocalOnionEndpoint().c_str()); } + bool fManagedTor = arg(argc, argv, "/managedtor") || arg(argc, argv, "-managedtor"); bool fTorMode = arg(argc, argv, "/tor") || arg(argc, argv, "-tor"); - if (fTorMode) + string socksProxy = argval2(argc, argv, "/socks", "-socks"); + if (fManagedTor) + { + if (fTorMode || !socksProxy.empty()) + fprintf(stderr, "Warning: /managedtor takes precedence over /tor and /socks\n"); + + string torPath = argval2(argc, argv, "/managedtor", "-managedtor"); + string err; + if (!BtfStartManagedTor(torPath, err)) + fprintf(stderr, "Ignoring /managedtor=%s: %s\n", torPath.c_str(), err.c_str()); + else + fprintf(stderr, "Managed Tor enabled: %s\n", BtfManagedTorStatus().c_str()); + } + else if (fTorMode) { - string socksProxy = argval2(argc, argv, "/socks", "-socks"); if (!socksProxy.empty()) fprintf(stderr, "Warning: /tor takes precedence over /socks=%s\n", socksProxy.c_str()); @@ -317,7 +350,6 @@ static void ParseStartupArguments(int argc, char* argv[]) } else { - string socksProxy = argval2(argc, argv, "/socks", "-socks"); if (!socksProxy.empty()) { string err; @@ -348,22 +380,6 @@ static void ParseStartupArguments(int argc, char* argv[]) vBtfExtraSeeds.push_back(make_pair(val.substr(0, colon), val.substr(colon + 1))); } - // net.cpp has described nListenPort as "tunable via /port" since it was - // written, but nothing ever read the option, so the port was fixed at 8433 - // and a second node could not start on a machine already running one. - string strPort = argval2(argc, argv, "/port", "-port"); - if (!strPort.empty()) - { - int nPort = atoi(strPort.c_str()); - if (nPort <= 0 || nPort > 65535) - fprintf(stderr, "Ignoring /port=%s: not a port number\n", strPort.c_str()); - else - { - nListenPort = htons((unsigned short)nPort); - // Or we would announce a port we never bound. - addrLocalHost.port = nListenPort; - } - } } // Report a startup failure somewhere the user will actually see it. @@ -715,7 +731,12 @@ int main(int argc, char* argv[]) ReacceptWalletTransactions(); - if (!StartNode(strErrors)) { fprintf(stderr,"StartNode: %s\n",strErrors.c_str()); return 1; } + if (!StartNode(strErrors)) + { + fprintf(stderr,"StartNode: %s\n",strErrors.c_str()); + BtfStopManagedTor(); + return 1; + } if (nMineMode == MINE_OPERATOR) { gPoolServerRunning = true; @@ -748,6 +769,7 @@ int main(int argc, char* argv[]) printf("Running headless. Ctrl-C to stop.\n"); while (!fShutdown) Sleep(500); StopNode(); + BtfStopManagedTor(); // Checkpoints, releases wallet.dat from this directory's Berkeley DB // environment, and closes it. Declared since 2009 and never once // called here, which is why a wallet.dat copied elsewhere would not @@ -767,6 +789,7 @@ int main(int argc, char* argv[]) int ret = RunGUI(argc, argv); fShutdown = true; StopNode(); + BtfStopManagedTor(); DBFlush(true); return ret; #endif diff --git a/src/makefile.mingw b/src/makefile.mingw index f82247a..333be96 100644 --- a/src/makefile.mingw +++ b/src/makefile.mingw @@ -37,7 +37,7 @@ IMGUI_OBJS = obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o \ NODE_OBJS = obj/util.o obj/script.o obj/db.o obj/net.o obj/main.o \ obj/market.o obj/sha.o obj/nostr.o obj/randomx_pow.o \ obj/btfaddr.o obj/btfchan.o obj/btfrv.o obj/btftunnel.o obj/rpc.o \ - obj/selftest.o obj/bip32.o obj/walletcmd.o obj/proxy.o + obj/selftest.o obj/bip32.o obj/walletcmd.o obj/proxy.o obj/tor.o GUI_OBJS = obj/gui.o obj/main_gui.o obj/bitflash_res.o @@ -51,7 +51,7 @@ GUI_OBJS = obj/gui.o obj/main_gui.o obj/bitflash_res.o HEADERS = headers_core.h headers.h util.h main.h serialize.h uint256.h \ key.h bignum.h script.h db.h base58.h net.h nostr.h \ randomx_pow.h btfaddr.h btfchan.h btfrv.h btftunnel.h market.h \ - selftest.h walletcmd.h sockcount.h proxy.h compat.h font_roboto.h + selftest.h walletcmd.h sockcount.h proxy.h tor.h compat.h font_roboto.h all: bitflash.exe @@ -113,3 +113,4 @@ tests: test_bip32.exe bitflash.exe ./bitflash.exe -selftest=consensus-limits ./bitflash.exe -selftest=pool-stratum ./bitflash.exe -selftest=socks5-proxy + ./bitflash.exe -selftest=managed-tor diff --git a/src/net.cpp b/src/net.cpp index ab17332..6bc3c1c 100644 --- a/src/net.cpp +++ b/src/net.cpp @@ -22,6 +22,7 @@ #include "btfaddr.h" #include "btftunnel.h" #include "proxy.h" +#include "tor.h" void ThreadReconnectCachedBtfPeers(void* parg); void ThreadMessageHandler2(void* parg); @@ -296,6 +297,7 @@ string GetDiagnosticsText() str += strprintf(" peers watched %d of %d held by poll%s\n", nPeersWatched, nHeld, nHeld > nPeersWatched ? " <-- NOT ALL PEERS ARE BEING READ" : ""); + str += strprintf(" managed Tor %s\n", BtfManagedTorStatus().c_str()); if (nBlocksReceived > 0) str += strprintf(" blocks received %lld (%lld arrived without a parent, %.1f%%)\n", diff --git a/src/readme.txt b/src/readme.txt index 1e9cffc..9ddbc24 100644 --- a/src/readme.txt +++ b/src/readme.txt @@ -49,6 +49,9 @@ Command-line options (in addition to the original ones) /tor[=HOST:PORT] Tor mode for outbound Nostr, .btf rendezvous, and direct onion peer dials; defaults to the local Tor SOCKS5 proxy at 127.0.0.1:9050 + /managedtor[=PATH] + start Tor, create a v3 hidden service for this node, route outbound + discovery through it, and advertise the generated onion endpoint /solomine mine without requiring a connected peer (local testing) /operator operator mode (run pool server) /participant=ADDR diff --git a/src/selftest.cpp b/src/selftest.cpp index a0d9edd..4cb872e 100644 --- a/src/selftest.cpp +++ b/src/selftest.cpp @@ -11,6 +11,7 @@ #include "proxy.h" #include "selftest.h" #include "sockcount.h" +#include "tor.h" #include "walletcmd.h" extern int RunPoolStratumSelfTest(); @@ -1565,6 +1566,38 @@ static int RunSocks5ProxySelfTest() return nFail == 0 ? 0 : 1; } +static int RunManagedTorSelfTest() +{ + printf("managed-tor self-test\n"); + int nFail = 0; + + std::string torrc = BtfBuildManagedTorrcForTest("C:/Bitflash Managed Tor/data", + "C:/Bitflash Managed Tor/onion-service", + 19050, 8433); + nFail += Check(torrc.find("DataDirectory \"C:/Bitflash Managed Tor/data\"") != std::string::npos, + "quotes the Tor data directory") ? 0 : 1; + nFail += Check(torrc.find("SocksPort 127.0.0.1:19050") != std::string::npos, + "binds SOCKS5 to localhost only") ? 0 : 1; + nFail += Check(torrc.find("IsolateSOCKSAuth") != std::string::npos && + torrc.find("IsolateClientAddr") != std::string::npos && + torrc.find("IsolateDestAddr") != std::string::npos && + torrc.find("IsolateDestPort") != std::string::npos, + "enables strict Tor stream isolation") ? 0 : 1; + nFail += Check(torrc.find("HiddenServiceDir \"C:/Bitflash Managed Tor/onion-service\"") != std::string::npos, + "writes a hidden service directory") ? 0 : 1; + nFail += Check(torrc.find("HiddenServiceVersion 3") != std::string::npos, + "requests a v3 onion service") ? 0 : 1; + nFail += Check(torrc.find("HiddenServicePort 8433 127.0.0.1:8433") != std::string::npos, + "maps the onion service to the Bitflash P2P listener") ? 0 : 1; + nFail += Check(BtfManagedTorStatus() == "disabled", + "managed Tor starts disabled") ? 0 : 1; + + printf("%s (%d failure%s)\n", nFail == 0 ? "ALL TESTS PASSED" : "TESTS FAILED", + nFail, nFail == 1 ? "" : "s"); + fflush(stdout); + return nFail == 0 ? 0 : 1; +} + int RunSelfTest(const std::string& name) { AttachTerminal(); @@ -1589,8 +1622,10 @@ int RunSelfTest(const std::string& name) return RunParseMoneySelfTest(); if (name == "socks5-proxy") return RunSocks5ProxySelfTest(); + if (name == "managed-tor") + return RunManagedTorSelfTest(); printf("Unknown self-test '%s'\n", name.c_str()); - printf("Known self-tests: wallet-keypool, wallet-hd, wallet-format, wallet-crypto, wallet-encrypt, net-message, consensus-limits, pool-stratum, parse-money, socks5-proxy\n"); + printf("Known self-tests: wallet-keypool, wallet-hd, wallet-format, wallet-crypto, wallet-encrypt, net-message, consensus-limits, pool-stratum, parse-money, socks5-proxy, managed-tor\n"); return 1; } diff --git a/src/tor.cpp b/src/tor.cpp new file mode 100644 index 0000000..30a6f0b --- /dev/null +++ b/src/tor.cpp @@ -0,0 +1,492 @@ +// Copyright (c) 2026 Bitflash developers +// Distributed under the MIT/X11 software license. + +#include "headers_core.h" +#include "tor.h" +#include "nostr.h" +#include "proxy.h" + +#include + +#ifndef _WIN32 +#include +#include +#endif + +static CCriticalSection cs_managedTor; +static bool g_fManagedTor = false; +static bool g_fManagedTorStopping = false; +static unsigned short g_managedTorSocksPort = 0; +static unsigned short g_managedTorP2PPort = 0; +static std::string g_managedTorPath; +static std::string g_managedTorDataDir; +static std::string g_managedTorHiddenServiceDir; +static std::string g_managedTorOnion; +static std::string g_managedTorStatus; +#ifdef _WIN32 +static PROCESS_INFORMATION g_managedTorProcess; +#else +static pid_t g_managedTorPid = -1; +#endif + +static std::string PathJoin(const std::string& a, const std::string& b) +{ + if (a.empty()) + return b; + char last = a[a.size() - 1]; + if (last == '/' || last == '\\') + return a + b; + return a + "/" + b; +} + +static std::string NormalizeTorrcPath(std::string path) +{ + for (size_t i = 0; i < path.size(); i++) + if (path[i] == '\\') + path[i] = '/'; + return path; +} + +static std::string QuoteTorrcPath(const std::string& path) +{ + std::string out = "\""; + for (size_t i = 0; i < path.size(); i++) + { + if (path[i] == '"' || path[i] == '\\') + out += '\\'; + out += path[i]; + } + out += "\""; + return out; +} + +static std::string QuoteCommandArg(const std::string& arg) +{ + std::string out = "\""; + for (size_t i = 0; i < arg.size(); i++) + { + if (arg[i] == '"') + out += '\\'; + out += arg[i]; + } + out += "\""; + return out; +} + +static bool EnsureDir(const std::string& path, std::string& errOut) +{ + if (path.empty()) + { + errOut = "directory path is empty"; + return false; + } +#ifdef _WIN32 + DWORD attr = GetFileAttributesA(path.c_str()); + if (attr != INVALID_FILE_ATTRIBUTES) + { + if (attr & FILE_ATTRIBUTE_DIRECTORY) + return true; + errOut = strprintf("%s exists and is not a directory", path.c_str()); + return false; + } + if (CreateDirectoryA(path.c_str(), NULL)) + return true; + DWORD err = GetLastError(); + if (err == ERROR_ALREADY_EXISTS) + return true; + errOut = strprintf("could not create %s (error %lu)", path.c_str(), (unsigned long)err); + return false; +#else + struct stat st; + if (stat(path.c_str(), &st) == 0) + { + if (S_ISDIR(st.st_mode)) + return true; + errOut = strprintf("%s exists and is not a directory", path.c_str()); + return false; + } + if (mkdir(path.c_str(), 0700) == 0 || errno == EEXIST) + return true; + errOut = strprintf("could not create %s: %s", path.c_str(), strerror(errno)); + return false; +#endif +} + +static bool WriteTextFile(const std::string& path, const std::string& data, + std::string& errOut) +{ + FILE* f = fopen(path.c_str(), "wb"); + if (!f) + { + errOut = strprintf("could not write %s", path.c_str()); + return false; + } + bool ok = fwrite(data.data(), 1, data.size(), f) == data.size(); + if (fclose(f) != 0) + ok = false; + if (!ok) + { + errOut = strprintf("could not finish writing %s", path.c_str()); + return false; + } + return true; +} + +static bool ReadOneLine(const std::string& path, std::string& out) +{ + out.clear(); + FILE* f = fopen(path.c_str(), "rb"); + if (!f) + return false; + char buf[256] = {0}; + if (!fgets(buf, sizeof(buf), f)) + { + fclose(f); + return false; + } + fclose(f); + out = buf; + while (!out.empty() && (out[out.size() - 1] == '\n' || out[out.size() - 1] == '\r' || + out[out.size() - 1] == ' ' || out[out.size() - 1] == '\t')) + out.erase(out.size() - 1); + return !out.empty(); +} + +static bool PickLocalPort(unsigned short& portOut, std::string& errOut) +{ + portOut = 0; + SOCKET s = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); + if (s == INVALID_SOCKET) + { + errOut = "could not create local port probe socket"; + return false; + } + struct sockaddr_in addr; + memset(&addr, 0, sizeof(addr)); + addr.sin_family = AF_INET; + addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + addr.sin_port = 0; + if (bind(s, (struct sockaddr*)&addr, sizeof(addr)) != 0) + { + closesocket(s); + errOut = "could not bind local port probe socket"; + return false; + } + socklen_t len = sizeof(addr); + if (getsockname(s, (struct sockaddr*)&addr, &len) != 0) + { + closesocket(s); + errOut = "could not read local port probe socket"; + return false; + } + portOut = ntohs(addr.sin_port); + closesocket(s); + if (portOut == 0) + { + errOut = "local port probe returned port zero"; + return false; + } + return true; +} + +static std::string ExecutableDir() +{ +#ifdef _WIN32 + char path[MAX_PATH + 1] = {0}; + DWORD n = GetModuleFileNameA(NULL, path, MAX_PATH); + if (n == 0 || n >= MAX_PATH) + return "."; + std::string s = path; + size_t slash = s.find_last_of("\\/"); + if (slash == std::string::npos) + return "."; + return s.substr(0, slash); +#else + char path[4096] = {0}; + ssize_t n = readlink("/proc/self/exe", path, sizeof(path) - 1); + if (n <= 0) + return "."; + path[n] = 0; + std::string s = path; + size_t slash = s.find_last_of('/'); + if (slash == std::string::npos) + return "."; + return s.substr(0, slash); +#endif +} + +static bool FileIsExecutableCandidate(const std::string& path) +{ +#ifdef _WIN32 + DWORD attr = GetFileAttributesA(path.c_str()); + return attr != INVALID_FILE_ATTRIBUTES && !(attr & FILE_ATTRIBUTE_DIRECTORY); +#else + return access(path.c_str(), X_OK) == 0; +#endif +} + +static std::string ResolveTorPath(const std::string& torPathOpt) +{ + if (!torPathOpt.empty()) + return torPathOpt; + + std::vector candidates; + std::string exeDir = ExecutableDir(); +#ifdef _WIN32 + candidates.push_back(PathJoin(exeDir, "tor/tor.exe")); + candidates.push_back(PathJoin(exeDir, "tor.exe")); + candidates.push_back("tor.exe"); +#else + candidates.push_back(PathJoin(exeDir, "tor/tor")); + candidates.push_back(PathJoin(exeDir, "tor")); + candidates.push_back("tor"); +#endif + for (size_t i = 0; i < candidates.size(); i++) + { + if (candidates[i].find('/') == std::string::npos && + candidates[i].find('\\') == std::string::npos) + return candidates[i]; // Let the OS search PATH. + if (FileIsExecutableCandidate(candidates[i])) + return candidates[i]; + } +#ifdef _WIN32 + return "tor.exe"; +#else + return "tor"; +#endif +} + +std::string BtfBuildManagedTorrcForTest(const std::string& dataDir, + const std::string& hiddenServiceDir, + unsigned short socksPort, + unsigned short p2pPort) +{ + std::string torData = NormalizeTorrcPath(dataDir); + std::string hsDir = NormalizeTorrcPath(hiddenServiceDir); + std::string logFile = NormalizeTorrcPath(PathJoin(dataDir, "tor.log")); + + std::string s; + s += "# Generated by Bitflash. Edit only while Bitflash is not managing Tor.\n"; + s += "DataDirectory " + QuoteTorrcPath(torData) + "\n"; + s += strprintf("SocksPort 127.0.0.1:%u IsolateSOCKSAuth IsolateClientAddr IsolateDestAddr IsolateDestPort\n", + (unsigned)socksPort); + s += "HiddenServiceDir " + QuoteTorrcPath(hsDir) + "\n"; + s += "HiddenServiceVersion 3\n"; + s += strprintf("HiddenServicePort %u 127.0.0.1:%u\n", + (unsigned)p2pPort, (unsigned)p2pPort); + s += "Log notice file " + QuoteTorrcPath(logFile) + "\n"; + return s; +} + +static bool StartTorProcess(const std::string& torPath, const std::string& torrcPath, + std::string& errOut) +{ +#ifdef _WIN32 + memset(&g_managedTorProcess, 0, sizeof(g_managedTorProcess)); + STARTUPINFOA si; + memset(&si, 0, sizeof(si)); + si.cb = sizeof(si); + std::string cmd = QuoteCommandArg(torPath) + " -f " + QuoteCommandArg(torrcPath); + std::vector mutableCmd(cmd.begin(), cmd.end()); + mutableCmd.push_back(0); + if (!CreateProcessA(NULL, &mutableCmd[0], NULL, NULL, FALSE, CREATE_NO_WINDOW, + NULL, NULL, &si, &g_managedTorProcess)) + { + errOut = strprintf("could not start Tor process %s (error %lu)", + torPath.c_str(), (unsigned long)GetLastError()); + return false; + } + return true; +#else + g_managedTorPid = fork(); + if (g_managedTorPid < 0) + { + errOut = strprintf("could not fork Tor process: %s", strerror(errno)); + return false; + } + if (g_managedTorPid == 0) + { + execlp(torPath.c_str(), torPath.c_str(), "-f", torrcPath.c_str(), (char*)NULL); + _exit(127); + } + return true; +#endif +} + +static void ThreadManagedTorHostname(void*) +{ + std::string hostnamePath; + unsigned short p2pPort = 0; + CRITICAL_BLOCK(cs_managedTor) + { + hostnamePath = PathJoin(g_managedTorHiddenServiceDir, "hostname"); + p2pPort = g_managedTorP2PPort; + } + + for (int i = 0; i < 90 && !fShutdown; i++) + { + std::string onion; + if (ReadOneLine(hostnamePath, onion)) + { + std::string endpoint = strprintf("%s:%u", onion.c_str(), (unsigned)p2pPort); + std::string err; + if (BtfSetLocalOnionEndpoint(endpoint, err)) + { + CRITICAL_BLOCK(cs_managedTor) + { + g_managedTorOnion = BtfLocalOnionEndpoint(); + g_managedTorStatus = strprintf("running, SOCKS5 127.0.0.1:%u, onion %s", + (unsigned)g_managedTorSocksPort, + g_managedTorOnion.c_str()); + } + printf("Managed Tor hidden service ready: %s\n", BtfLocalOnionEndpoint().c_str()); + return; + } + CRITICAL_BLOCK(cs_managedTor) + g_managedTorStatus = strprintf("running, but generated onion endpoint was rejected: %s", + err.c_str()); + return; + } + Sleep(1000); + } + + CRITICAL_BLOCK(cs_managedTor) + { + if (!g_fManagedTorStopping) + g_managedTorStatus = strprintf("running, SOCKS5 127.0.0.1:%u, waiting for hidden service hostname", + (unsigned)g_managedTorSocksPort); + } +} + +bool BtfStartManagedTor(const std::string& torPathOpt, std::string& errOut) +{ + CRITICAL_BLOCK(cs_managedTor) + { + if (g_fManagedTor) + { + errOut = "managed Tor is already running"; + return false; + } + } + + std::string root = PathJoin(GetAppDir(), "managed-tor"); + std::string dataDir = PathJoin(root, "data"); + std::string hsDir = PathJoin(root, "onion-service"); + std::string torrcPath = PathJoin(root, "torrc"); + if (!EnsureDir(root, errOut) || !EnsureDir(dataDir, errOut) || !EnsureDir(hsDir, errOut)) + return false; + + unsigned short socksPort = 0; + if (!PickLocalPort(socksPort, errOut)) + return false; + + unsigned short p2pPort = ntohs(nListenPort); + std::string torrc = BtfBuildManagedTorrcForTest(dataDir, hsDir, socksPort, p2pPort); + if (!WriteTextFile(torrcPath, torrc, errOut)) + return false; + + std::string torPath = ResolveTorPath(torPathOpt); + { + CRITICAL_BLOCK(cs_managedTor) + { + g_fManagedTor = true; + g_fManagedTorStopping = false; + g_managedTorPath = torPath; + g_managedTorDataDir = dataDir; + g_managedTorHiddenServiceDir = hsDir; + g_managedTorSocksPort = socksPort; + g_managedTorP2PPort = p2pPort; + g_managedTorOnion.clear(); + g_managedTorStatus = strprintf("starting %s, SOCKS5 127.0.0.1:%u", + torPath.c_str(), (unsigned)socksPort); + } + } + + if (!StartTorProcess(torPath, torrcPath, errOut)) + { + CRITICAL_BLOCK(cs_managedTor) + { + g_fManagedTor = false; + g_managedTorStatus = strprintf("failed: %s", errOut.c_str()); + } + return false; + } + + std::string proxyErr; + if (!BtfEnableTorProxy(strprintf("127.0.0.1:%u", (unsigned)socksPort), proxyErr)) + { + BtfStopManagedTor(); + errOut = strprintf("could not enable managed Tor SOCKS5 proxy: %s", proxyErr.c_str()); + return false; + } + + if (_beginthread(ThreadManagedTorHostname, 0, NULL) == (uintptr_t)-1) + { + BtfStopManagedTor(); + errOut = "could not start managed Tor hostname watcher"; + return false; + } + + return true; +} + +void BtfStopManagedTor() +{ + bool fWasRunning = false; + CRITICAL_BLOCK(cs_managedTor) + { + fWasRunning = g_fManagedTor; + if (!fWasRunning) + return; + g_fManagedTorStopping = true; + g_fManagedTor = false; + g_managedTorStatus = "stopping"; + } +#ifdef _WIN32 + if (g_managedTorProcess.hProcess) + { + TerminateProcess(g_managedTorProcess.hProcess, 0); + WaitForSingleObject(g_managedTorProcess.hProcess, 5000); + CloseHandle(g_managedTorProcess.hProcess); + CloseHandle(g_managedTorProcess.hThread); + memset(&g_managedTorProcess, 0, sizeof(g_managedTorProcess)); + } +#else + if (g_managedTorPid > 0) + { + kill(g_managedTorPid, SIGTERM); + for (int i = 0; i < 50; i++) + { + int status = 0; + pid_t r = waitpid(g_managedTorPid, &status, WNOHANG); + if (r == g_managedTorPid) + break; + Sleep(100); + } + g_managedTorPid = -1; + } +#endif + CRITICAL_BLOCK(cs_managedTor) + { + g_managedTorStatus = "stopped"; + g_managedTorOnion.clear(); + } +} + +bool BtfManagedTorEnabled() +{ + CRITICAL_BLOCK(cs_managedTor) + return g_fManagedTor; + return false; +} + +std::string BtfManagedTorStatus() +{ + CRITICAL_BLOCK(cs_managedTor) + { + if (!g_fManagedTor && g_managedTorStatus.empty()) + return "disabled"; + return g_managedTorStatus; + } + return "disabled"; +} diff --git a/src/tor.h b/src/tor.h new file mode 100644 index 0000000..482e773 --- /dev/null +++ b/src/tor.h @@ -0,0 +1,22 @@ +// Copyright (c) 2026 Bitflash developers +// Distributed under the MIT/X11 software license. +// +// Managed Tor process support. This is transport bootstrapping only: it does +// not touch consensus, wallet keys, mining, or the .btf identity. + +#ifndef BITFLASH_TOR_H +#define BITFLASH_TOR_H + +#include + +bool BtfStartManagedTor(const std::string& torPathOpt, std::string& errOut); +void BtfStopManagedTor(); +bool BtfManagedTorEnabled(); +std::string BtfManagedTorStatus(); + +std::string BtfBuildManagedTorrcForTest(const std::string& dataDir, + const std::string& hiddenServiceDir, + unsigned short socksPort, + unsigned short p2pPort); + +#endif -- cgit v1.2.3 From 44fbf6895bd3acd11f552514e746a2e3876cad2d Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Thu, 13 Aug 2026 14:29:14 -0300 Subject: release: package Windows builds with managed Tor --- Makefile | 6 +- README.md | 13 ++++ docs/release-verification.md | 6 ++ docs/tor.md | 16 +++++ scripts/make-release-checksums.sh | 2 + scripts/package-windows-tor.sh | 132 ++++++++++++++++++++++++++++++++++++++ 6 files changed, 174 insertions(+), 1 deletion(-) create mode 100755 scripts/package-windows-tor.sh (limited to 'docs') diff --git a/Makefile b/Makefile index b4d0cf7..d1277a4 100644 --- a/Makefile +++ b/Makefile @@ -2,6 +2,7 @@ # # make linux build Bitflash-*-x86_64.AppImage # make windows build Bitflash-*-windows.zip (from MSYS2 UCRT64) +# make windows-tor build Bitflash-*-windows-with-tor.zip # make tests build and run standalone unit tests # make checksums create SHA256SUMS for release assets # make verify-release TAG=v1.2.13 @@ -180,6 +181,9 @@ windows: deps-windows @zip -r Bitflash-$(VERSION)-windows.zip Bitflash-$(VERSION)-windows/ @echo "Built: Bitflash-$(VERSION)-windows.zip" +windows-tor: windows + ./scripts/package-windows-tor.sh + # ---- Clean ---------------------------------------------------------------- clean: @@ -206,6 +210,6 @@ sign-checksums: verify-release: ./scripts/verify-release.sh $(if $(TAG),$(TAG),latest) -.PHONY: linux windows clean appimage \ +.PHONY: linux windows windows-tor clean appimage \ tests fuzz-net-message-smoke fuzz-script-smoke checksums sign-checksums verify-release \ deps-linux deps-windows deps-apt deps-secp256k1 deps-randomx diff --git a/README.md b/README.md index a7ce2b2..c30f171 100644 --- a/README.md +++ b/README.md @@ -52,6 +52,8 @@ configuration โ€” it connects automatically and starts syncing. **Linux:** make the `.AppImage` executable and run it. **Windows:** extract the `-windows.zip` and run `Bitflash.exe`. +If you want managed Tor without a separate Tor install, use the +`-windows-with-tor.zip` package and start Bitflash with `-managedtor`. Every release ships a `SHA256SUMS` covering the assets. Verifying takes a second and is worth doing: @@ -452,6 +454,17 @@ make windows ``` Installs deps via pacman, produces `Bitflash-*-windows.zip`. +To build the Windows package that also carries the Tor Expert Bundle: + +```bash +make windows-tor +``` + +That produces `Bitflash-*-windows-with-tor.zip`. The release script downloads +the Tor Expert Bundle from the official Tor archive, verifies its pinned SHA256, +and places `tor/tor.exe` beside `Bitflash.exe` so `-managedtor` works without a +separate Tor install. + --- ## At a glance diff --git a/docs/release-verification.md b/docs/release-verification.md index 408bab9..04d18a3 100644 --- a/docs/release-verification.md +++ b/docs/release-verification.md @@ -81,6 +81,12 @@ Publish, alongside the assets, `SHA256SUMS` and `SHA256SUMS.asc`, and update is ever rotated, announce the new fingerprint clearly and update the embedded key in `verify-release.sh`. +For Windows releases with managed Tor built in, run `make windows-tor`. It +creates `Bitflash-*-windows-with-tor.zip` by downloading the Tor Expert Bundle, +checking the pinned SHA256 in `scripts/package-windows-tor.sh`, and placing +`tor/tor.exe` next to `Bitflash.exe`. Include that zip in `SHA256SUMS` like any +other release asset. + ## Why this matters `SHA256SUMS` alone protects against a broken download, but not against someone diff --git a/docs/tor.md b/docs/tor.md index 1c0f589..be6a145 100644 --- a/docs/tor.md +++ b/docs/tor.md @@ -53,6 +53,22 @@ Without a value, Bitflash looks for Tor in this order: 2. `tor.exe` or `tor` beside the Bitflash binary; 3. `tor.exe` or `tor` on `PATH`. +Windows releases can also be built as `Bitflash-*-windows-with-tor.zip`. That +package includes the official Tor Expert Bundle under `tor/`, so the first path +above exists immediately after extraction and `bitflash -managedtor` needs no +separate Tor install. + +The bundled-Tor package is created by: + +```bash +make windows-tor +``` + +The packaging script verifies a pinned SHA256 for the downloaded Tor Expert +Bundle before copying it into the release directory. The Tor binary is bundled; +the runtime data directory, hidden-service key, and control cookie are still +created under the user's Bitflash data directory. + To point at a specific Tor executable: ```bash diff --git a/scripts/make-release-checksums.sh b/scripts/make-release-checksums.sh index 5b29e2d..9c4f45a 100755 --- a/scripts/make-release-checksums.sh +++ b/scripts/make-release-checksums.sh @@ -10,6 +10,7 @@ Usage: scripts/make-release-checksums.sh [--sign] [--local-user KEYID] Creates SHA256SUMS for release assets in the current directory: Bitflash-*-windows.zip + Bitflash-*-windows-with-tor.zip Bitflash-*-x86_64.AppImage bitflash-node-*-x86_64 @@ -44,6 +45,7 @@ fi assets=() for pattern in \ "Bitflash-*-windows.zip" \ + "Bitflash-*-windows-with-tor.zip" \ "Bitflash-*-x86_64.AppImage" \ "bitflash-node-*-x86_64" do diff --git a/scripts/package-windows-tor.sh b/scripts/package-windows-tor.sh new file mode 100755 index 0000000..084fd1a --- /dev/null +++ b/scripts/package-windows-tor.sh @@ -0,0 +1,132 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +VERSION="${VERSION:-$(sed -n 's/^VERSION[[:space:]]*:= *//p' "$ROOT/Makefile" | head -1)}" + +TOR_VERSION="${TOR_VERSION:-15.0.19}" +TOR_ARCHIVE_NAME="tor-expert-bundle-windows-x86_64-${TOR_VERSION}.tar.gz" +TOR_URL="${TOR_URL:-https://archive.torproject.org/tor-package-archive/torbrowser/${TOR_VERSION}/${TOR_ARCHIVE_NAME}}" +TOR_SHA256="${TOR_SHA256:-6AC067402C7B4A3DC37887ED3754B3914B67FDC220C966190683E9CCF91ABF0F}" + +BUNDLE_DIR="${BUNDLE_DIR:-$ROOT/Bitflash-${VERSION}-windows}" +OUT_ZIP="${OUT_ZIP:-$ROOT/Bitflash-${VERSION}-windows-with-tor.zip}" +CACHE_DIR="${CACHE_DIR:-$ROOT/.cache}" +ARCHIVE="${TOR_ARCHIVE:-$CACHE_DIR/$TOR_ARCHIVE_NAME}" + +usage() { + cat </dev/null 2>&1 || { + echo "missing required command: $1" >&2 + exit 1 + } +} + +need sha256sum +need tar +need zip + +if [ ! -d "$BUNDLE_DIR" ]; then + echo "missing Windows release directory: $BUNDLE_DIR" >&2 + echo "run 'make windows' first, or set BUNDLE_DIR" >&2 + exit 1 +fi +if [ ! -f "$BUNDLE_DIR/Bitflash.exe" ]; then + echo "missing Bitflash.exe in $BUNDLE_DIR" >&2 + exit 1 +fi +BUNDLE_PARENT="$(cd "$(dirname "$BUNDLE_DIR")" && pwd)" +BUNDLE_NAME="$(basename "$BUNDLE_DIR")" +OUT_PARENT="$(mkdir -p "$(dirname "$OUT_ZIP")" && cd "$(dirname "$OUT_ZIP")" && pwd)" +OUT_NAME="$(basename "$OUT_ZIP")" +OUT_ZIP_ABS="$OUT_PARENT/$OUT_NAME" + +mkdir -p "$CACHE_DIR" +if [ ! -f "$ARCHIVE" ]; then + if command -v curl >/dev/null 2>&1; then + curl -L --fail --output "$ARCHIVE" "$TOR_URL" + elif command -v wget >/dev/null 2>&1; then + wget -O "$ARCHIVE" "$TOR_URL" + else + echo "missing required command: curl or wget" >&2 + exit 1 + fi +fi + +got="$(sha256sum "$ARCHIVE" | awk '{print tolower($1)}')" +want="$(printf '%s' "$TOR_SHA256" | tr 'A-F' 'a-f')" +if [ "$got" != "$want" ]; then + echo "Tor archive SHA256 mismatch" >&2 + echo " expected: $want" >&2 + echo " got: $got" >&2 + exit 1 +fi + +tmp="$(mktemp -d "${TMPDIR:-/tmp}/bitflash-tor-package.XXXXXX")" +trap 'rm -rf "$tmp"' EXIT + +mkdir -p "$tmp/extract" +tar -xzf "$ARCHIVE" -C "$tmp/extract" + +if [ ! -f "$tmp/extract/tor/tor.exe" ]; then + echo "Tor archive did not contain tor/tor.exe" >&2 + exit 1 +fi + +rm -rf "$BUNDLE_DIR/tor" +mkdir -p "$BUNDLE_DIR/tor" +cp -R "$tmp/extract/tor/." "$BUNDLE_DIR/tor/" +if [ -d "$tmp/extract/docs" ]; then + mkdir -p "$BUNDLE_DIR/tor/docs" + cp -R "$tmp/extract/docs/." "$BUNDLE_DIR/tor/docs/" +fi +if [ -d "$tmp/extract/data" ]; then + mkdir -p "$BUNDLE_DIR/tor/data-defaults" + cp -R "$tmp/extract/data/." "$BUNDLE_DIR/tor/data-defaults/" +fi + +cat > "$BUNDLE_DIR/TOR-EXPERT-BUNDLE.txt" < Date: Thu, 13 Aug 2026 14:35:49 -0300 Subject: release: verify bundled Tor signatures --- README.md | 6 ++-- docs/release-verification.md | 8 +++-- docs/tor.md | 9 ++++-- scripts/package-windows-tor.sh | 71 +++++++++++++++++++++++++++++++++++++----- 4 files changed, 78 insertions(+), 16 deletions(-) (limited to 'docs') diff --git a/README.md b/README.md index c30f171..bb6a4d4 100644 --- a/README.md +++ b/README.md @@ -462,8 +462,10 @@ make windows-tor That produces `Bitflash-*-windows-with-tor.zip`. The release script downloads the Tor Expert Bundle from the official Tor archive, verifies its pinned SHA256, -and places `tor/tor.exe` beside `Bitflash.exe` so `-managedtor` works without a -separate Tor install. +optionally verifies the Tor Project GPG signature when `gpg` is available, and +places `tor/tor.exe` beside `Bitflash.exe` so `-managedtor` works without a +separate Tor install. For release builds, use `TOR_VERIFY_GPG=required make +windows-tor` to require the extra signature check. --- diff --git a/docs/release-verification.md b/docs/release-verification.md index 04d18a3..b8eaf5e 100644 --- a/docs/release-verification.md +++ b/docs/release-verification.md @@ -83,9 +83,11 @@ key in `verify-release.sh`. For Windows releases with managed Tor built in, run `make windows-tor`. It creates `Bitflash-*-windows-with-tor.zip` by downloading the Tor Expert Bundle, -checking the pinned SHA256 in `scripts/package-windows-tor.sh`, and placing -`tor/tor.exe` next to `Bitflash.exe`. Include that zip in `SHA256SUMS` like any -other release asset. +checking the pinned SHA256 in `scripts/package-windows-tor.sh`, optionally +verifying the Tor Project `.asc` signature, and placing `tor/tor.exe` next to +`Bitflash.exe`. For release builds, run `TOR_VERIFY_GPG=required make +windows-tor` so the signature check is mandatory. Include that zip in +`SHA256SUMS` like any other release asset. ## Why this matters diff --git a/docs/tor.md b/docs/tor.md index be6a145..a408704 100644 --- a/docs/tor.md +++ b/docs/tor.md @@ -65,9 +65,12 @@ make windows-tor ``` The packaging script verifies a pinned SHA256 for the downloaded Tor Expert -Bundle before copying it into the release directory. The Tor binary is bundled; -the runtime data directory, hidden-service key, and control cookie are still -created under the user's Bitflash data directory. +Bundle before copying it into the release directory. When `gpg` is available it +also verifies the matching `.asc` signature against the pinned Tor Browser +Developers signing-key fingerprint. Release builds should use +`TOR_VERIFY_GPG=required make windows-tor` to make that second check mandatory. +The Tor binary is bundled; the runtime data directory, hidden-service key, and +control cookie are still created under the user's Bitflash data directory. To point at a specific Tor executable: diff --git a/scripts/package-windows-tor.sh b/scripts/package-windows-tor.sh index 084fd1a..c8cca3b 100755 --- a/scripts/package-windows-tor.sh +++ b/scripts/package-windows-tor.sh @@ -8,11 +8,15 @@ TOR_VERSION="${TOR_VERSION:-15.0.19}" TOR_ARCHIVE_NAME="tor-expert-bundle-windows-x86_64-${TOR_VERSION}.tar.gz" TOR_URL="${TOR_URL:-https://archive.torproject.org/tor-package-archive/torbrowser/${TOR_VERSION}/${TOR_ARCHIVE_NAME}}" TOR_SHA256="${TOR_SHA256:-6AC067402C7B4A3DC37887ED3754B3914B67FDC220C966190683E9CCF91ABF0F}" +TOR_ASC_URL="${TOR_ASC_URL:-${TOR_URL}.asc}" +TOR_SIGNING_KEY_FINGERPRINT="${TOR_SIGNING_KEY_FINGERPRINT:-EF6E286DDA85EA2A4BA7DE684E2C6E8793298290}" +TOR_VERIFY_GPG="${TOR_VERIFY_GPG:-auto}" BUNDLE_DIR="${BUNDLE_DIR:-$ROOT/Bitflash-${VERSION}-windows}" OUT_ZIP="${OUT_ZIP:-$ROOT/Bitflash-${VERSION}-windows-with-tor.zip}" CACHE_DIR="${CACHE_DIR:-$ROOT/.cache}" ARCHIVE="${TOR_ARCHIVE:-$CACHE_DIR/$TOR_ARCHIVE_NAME}" +ASC_FILE="${TOR_ASC:-$ARCHIVE.asc}" usage() { cat </dev/null 2>&1; then + curl -L --fail --output "$out" "$url" + elif command -v wget >/dev/null 2>&1; then + wget -O "$out" "$url" + else + echo "missing required command: curl or wget" >&2 + exit 1 + fi +} + if [ ! -d "$BUNDLE_DIR" ]; then echo "missing Windows release directory: $BUNDLE_DIR" >&2 echo "run 'make windows' first, or set BUNDLE_DIR" >&2 @@ -65,14 +85,7 @@ OUT_ZIP_ABS="$OUT_PARENT/$OUT_NAME" mkdir -p "$CACHE_DIR" if [ ! -f "$ARCHIVE" ]; then - if command -v curl >/dev/null 2>&1; then - curl -L --fail --output "$ARCHIVE" "$TOR_URL" - elif command -v wget >/dev/null 2>&1; then - wget -O "$ARCHIVE" "$TOR_URL" - else - echo "missing required command: curl or wget" >&2 - exit 1 - fi + download "$TOR_URL" "$ARCHIVE" fi got="$(sha256sum "$ARCHIVE" | awk '{print tolower($1)}')" @@ -87,6 +100,48 @@ fi tmp="$(mktemp -d "${TMPDIR:-/tmp}/bitflash-tor-package.XXXXXX")" trap 'rm -rf "$tmp"' EXIT +case "$TOR_VERIFY_GPG" in + auto|required|off) ;; + *) echo "TOR_VERIFY_GPG must be auto, required, or off" >&2; exit 2 ;; +esac + +if [ "$TOR_VERIFY_GPG" != "off" ]; then + if command -v gpg >/dev/null 2>&1; then + if [ ! -f "$ASC_FILE" ]; then + download "$TOR_ASC_URL" "$ASC_FILE" + fi + gnupg_home="$tmp/gnupg" + mkdir -p "$gnupg_home" + chmod 0700 "$gnupg_home" + gpg --homedir "$gnupg_home" --batch --auto-key-locate nodefault,wkd,keyserver \ + --locate-keys torbrowser@torproject.org >/dev/null 2>&1 || { + if [ "$TOR_VERIFY_GPG" = "required" ]; then + echo "could not import Tor Browser Developers signing key" >&2 + exit 1 + fi + echo "WARNING: skipped Tor GPG signature verification; signing key import failed" >&2 + } + if gpg --homedir "$gnupg_home" --batch --with-colons --fingerprint "$TOR_SIGNING_KEY_FINGERPRINT" \ + 2>/dev/null | grep -q "^fpr:::::::::${TOR_SIGNING_KEY_FINGERPRINT}:"; then + gpg --homedir "$gnupg_home" --batch --verify "$ASC_FILE" "$ARCHIVE" >/dev/null 2>&1 || { + echo "Tor archive GPG signature verification failed" >&2 + exit 1 + } + echo "Tor archive GPG signature verified against ${TOR_SIGNING_KEY_FINGERPRINT}" + elif [ "$TOR_VERIFY_GPG" = "required" ]; then + echo "Tor signing key fingerprint mismatch" >&2 + exit 1 + else + echo "WARNING: skipped Tor GPG signature verification; signing key fingerprint was not available" >&2 + fi + elif [ "$TOR_VERIFY_GPG" = "required" ]; then + echo "missing required command for TOR_VERIFY_GPG=required: gpg" >&2 + exit 1 + else + echo "WARNING: gpg not found; relying on pinned SHA256 only" >&2 + fi +fi + mkdir -p "$tmp/extract" tar -xzf "$ARCHIVE" -C "$tmp/extract" -- cgit v1.2.3 From 5309170e567f4a8655b3b622e7b26b6d8b382942 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Thu, 13 Aug 2026 15:24:10 -0300 Subject: net: add onion-only btf dial mode --- docs/tor.md | 28 ++++++ scripts/smoke-managed-tor-onion.ps1 | 174 ++++++++++++++++++++++++++++++++++++ src/main_gui.cpp | 4 + src/net.cpp | 25 +++++- src/net.h | 1 + src/nostr.cpp | 11 +-- src/proxy.cpp | 51 ++++++++++- 7 files changed, 283 insertions(+), 11 deletions(-) create mode 100644 scripts/smoke-managed-tor-onion.ps1 (limited to 'docs') diff --git a/docs/tor.md b/docs/tor.md index a408704..0016f8b 100644 --- a/docs/tor.md +++ b/docs/tor.md @@ -201,3 +201,31 @@ but an already-discovered onion-capable peer can be reached directly over Tor. Tor owns the private key under `HiddenServiceDir`; back it up if you want the onion name to stay stable. + +## Release smoke test + +After building a Windows binary with managed Tor support, run the two-node smoke +test before cutting a release: + +```powershell +powershell -ExecutionPolicy Bypass -File scripts\smoke-managed-tor-onion.ps1 ` + -Bitflash .\src\bitflash.exe ` + -Tor C:\path\to\tor.exe +``` + +The script creates two temporary datadirs, starts node A with `-managedtor`, +waits for it to publish its `.btf` identity and generated `.onion` endpoint, +then starts node B with its own `-managedtor`, `-oniononly`, and +`-connectbtf=`. It passes only when node B logs that it connected to +node A `via direct onion`; rendezvous fallback cannot satisfy the test. + +By default the script waits 120 seconds after node A's descriptor self-resolves +before it starts node B. Tor can create the local `hostname` before the hidden +service descriptor is visible enough for client circuits, and early dials often +fail with SOCKS5 reply `0x01`. Override this with `-OnionWarmupSec N` when +testing a slow or already-warmed Tor environment. + +This test still uses Nostr for descriptor discovery, because that is how nodes +learn each other's signed `.btf` descriptors. It does not require a Bitflash +rendezvous relay to carry the P2P connection: once the descriptor includes the +signed onion endpoint, the actual node connection goes directly through Tor. diff --git a/scripts/smoke-managed-tor-onion.ps1 b/scripts/smoke-managed-tor-onion.ps1 new file mode 100644 index 0000000..70658c7 --- /dev/null +++ b/scripts/smoke-managed-tor-onion.ps1 @@ -0,0 +1,174 @@ +param( + [string]$Bitflash, + [string]$Tor, + [string]$WorkDir, + [int]$TimeoutSec = 600, + [int]$OnionWarmupSec = 120 +) + +$ErrorActionPreference = "Stop" + +function Resolve-RepoRoot { + $scriptDir = Split-Path -Parent $PSCommandPath + return (Resolve-Path (Join-Path $scriptDir "..")).Path +} + +function Get-FreeTcpPort { + $listener = [Net.Sockets.TcpListener]::new([Net.IPAddress]::Loopback, 0) + $listener.Start() + try { + return ([Net.IPEndPoint]$listener.LocalEndpoint).Port + } + finally { + $listener.Stop() + } +} + +function Wait-RegexInFile { + param( + [string]$Path, + [string]$Pattern, + [datetime]$Deadline, + [string]$Label + ) + + while ((Get-Date) -lt $Deadline) { + if (Test-Path $Path) { + $text = Get-Content -Raw -ErrorAction SilentlyContinue $Path + if ($null -ne $text) { + $m = [regex]::Match($text, $Pattern) + if ($m.Success) { + return $m + } + } + } + Start-Sleep -Milliseconds 500 + } + throw "timed out waiting for $Label" +} + +function Stop-TestProcesses { + param( + [System.Diagnostics.Process[]]$Processes, + [string[]]$DataDirKeys + ) + + foreach ($p in $Processes) { + try { + $p.Refresh() + if (!$p.HasExited) { + Stop-Process -Id $p.Id -Force + } + } + catch {} + } + + foreach ($key in $DataDirKeys) { + if ([string]::IsNullOrWhiteSpace($key)) { + continue + } + try { + Get-CimInstance Win32_Process -Filter "name = 'tor.exe'" | + Where-Object { $_.CommandLine -like "*$key*" } | + ForEach-Object { Stop-Process -Id $_.ProcessId -Force } + } + catch {} + } +} + +$repo = Resolve-RepoRoot +if ([string]::IsNullOrWhiteSpace($Bitflash)) { + $Bitflash = Join-Path $repo "src\bitflash.exe" +} +if ([string]::IsNullOrWhiteSpace($Tor)) { + $candidates = @( + (Join-Path $repo "src\tor\tor.exe"), + (Join-Path $repo "Bitflash-$(Select-String -Path (Join-Path $repo 'Makefile') -Pattern '^VERSION[ ]*:=' | ForEach-Object { ($_.Line -split ':=')[1].Trim() })-windows\tor\tor.exe") + ) + foreach ($candidate in $candidates) { + if (Test-Path $candidate) { + $Tor = $candidate + break + } + } +} +if ([string]::IsNullOrWhiteSpace($WorkDir)) { + $WorkDir = Join-Path $env:TEMP ("bitflash-managed-tor-two-node-" + (Get-Date -Format "yyyyMMddHHmmss")) +} + +if (!(Test-Path $Bitflash)) { + throw "Bitflash binary not found: $Bitflash" +} +if (!(Test-Path $Tor)) { + throw "Tor binary not found. Pass -Tor or build the Windows-with-Tor package." +} + +New-Item -ItemType Directory -Force -Path $WorkDir | Out-Null +$dataA = Join-Path $WorkDir "node-a" +$dataB = Join-Path $WorkDir "node-b" +New-Item -ItemType Directory -Force -Path $dataA | Out-Null +New-Item -ItemType Directory -Force -Path $dataB | Out-Null + +$portA = Get-FreeTcpPort +$portB = Get-FreeTcpPort +$deadline = (Get-Date).AddSeconds($TimeoutSec) +$oldPath = $env:PATH +$env:PATH = "C:\msys64\ucrt64\bin;C:\msys64\usr\bin;" + $env:PATH + +$procA = $null +$procB = $null +$dataKeys = @((Split-Path $WorkDir -Leaf)) + +try { + $argsA = @( + "-datadir=$dataA", + "-port=$portA", + "-managedtor=$Tor", + "-nogui", + "-debug", + "-checkblocks=0" + ) + $procA = Start-Process -FilePath $Bitflash -ArgumentList $argsA -WorkingDirectory (Split-Path $Bitflash -Parent) -PassThru -WindowStyle Hidden + $logA = Join-Path $dataA "debug.log" + + $btfMatch = Wait-RegexInFile -Path $logA -Pattern "Nostr: node \.btf address = ([a-z2-7]+\.btf)" -Deadline $deadline -Label "node A .btf address" + $btfA = $btfMatch.Groups[1].Value + + $onionMatch = Wait-RegexInFile -Path $logA -Pattern "Managed Tor hidden service ready: ([a-z2-7]{56}\.onion:\d+)" -Deadline $deadline -Label "node A managed onion" + $onionA = $onionMatch.Groups[1].Value + + Wait-RegexInFile -Path $logA -Pattern "Nostr: \.btf self-resolve OK via .*" -Deadline $deadline -Label "node A descriptor self-resolve" | Out-Null + if ($OnionWarmupSec -gt 0) { + Write-Host "Waiting $OnionWarmupSec second(s) for Tor hidden-service descriptor propagation" + Start-Sleep -Seconds $OnionWarmupSec + } + + $argsB = @( + "-datadir=$dataB", + "-port=$portB", + "-managedtor=$Tor", + "-oniononly", + "-connectbtf=$btfA", + "-nogui", + "-debug", + "-checkblocks=0" + ) + $procB = Start-Process -FilePath $Bitflash -ArgumentList $argsB -WorkingDirectory (Split-Path $Bitflash -Parent) -PassThru -WindowStyle Hidden + $logB = Join-Path $dataB "debug.log" + + Wait-RegexInFile -Path $logB -Pattern ("connected " + [regex]::Escape($btfA) + " via direct onion ") -Deadline $deadline -Label "node B direct onion connection to node A" | Out-Null + + Write-Host "Managed Tor two-node onion smoke test passed" + Write-Host " node A .btf: $btfA" + Write-Host " node A onion: $onionA" + Write-Host " node A port: $portA" + Write-Host " node B port: $portB" + Write-Host " work dir: $WorkDir" +} +finally { + $env:PATH = $oldPath + $procs = @() + if ($null -ne $procA) { $procs += $procA } + if ($null -ne $procB) { $procs += $procB } + Stop-TestProcesses -Processes $procs -DataDirKeys $dataKeys +} diff --git a/src/main_gui.cpp b/src/main_gui.cpp index deed471..f0fe903 100644 --- a/src/main_gui.cpp +++ b/src/main_gui.cpp @@ -143,6 +143,7 @@ static void PrintUsage() printf(" /socks=HOST:PORT (SOCKS5 proxy for Nostr, .btf relay, and onion dials)\n"); printf(" /tor[=HOST:PORT] (Tor mode; default SOCKS5 proxy is 127.0.0.1:9050)\n"); printf(" /managedtor[=PATH] (start Tor, create a hidden service, advertise its onion)\n"); + printf(" /oniononly (do not fall back to rendezvous when a .btf onion dial fails)\n"); printf(" /btfseed=ADDRESS:ENCHEX (extra bootstrap peer, repeatable)\n"); printf("\n"); printf("Wallet:\n"); @@ -278,6 +279,7 @@ static void ParseStartupArguments(int argc, char* argv[]) string btfConnect = argval2(argc, argv, "/connectbtf", "-connectbtf"); if (!btfConnect.empty()) strBtfConnect = btfConnect; + fBtfOnionOnly = arg(argc, argv, "/oniononly") || arg(argc, argv, "-oniononly"); string rvRelay = argval2(argc, argv, "/rvrelay", "-rvrelay"); if (!rvRelay.empty()) @@ -360,6 +362,8 @@ static void ParseStartupArguments(int argc, char* argv[]) BtfSocks5ProxyName().c_str()); } } + if (fBtfOnionOnly && !BtfSocks5ProxyEnabled()) + fprintf(stderr, "Warning: /oniononly without /tor, /managedtor, or /socks cannot dial .onion peers\n"); // /btfseed=ADDRESS:ENCHEX -- extra bootstrap peers, repeatable. Useful for // testing the seed path and for private networks that ship no compiled list. diff --git a/src/net.cpp b/src/net.cpp index 6bc3c1c..997a2bd 100644 --- a/src/net.cpp +++ b/src/net.cpp @@ -65,6 +65,8 @@ deque > vRelayExpiration; CCriticalSection cs_mapRelay; map mapAlreadyAskedFor; string strBtfConnect; // .btf peer to keep connected to (from /connectbtf) +bool fBtfOnionOnly = false; // require direct onion transport for .btf peers +static const int BTF_ONION_CONNECT_TIMEOUT_SECS = 120; int nPeersWatched = 0; int64 nBlocksReceived = 0; @@ -1022,7 +1024,8 @@ static CNode* ConnectNodeBtfTail(const string& strBtfAddr, const unsigned char p if (SplitHostPort(strOnionNorm, strOnionHost, nOnionPort)) { BtfChurnNoteDialAttempt(strBtfAddr, strOnionNorm); - SOCKET hOnionSocket = BtfConnectSocket(strOnionHost, nOnionPort, SOCK_ONION_PEER, 20); + SOCKET hOnionSocket = BtfConnectSocket(strOnionHost, nOnionPort, SOCK_ONION_PEER, + BTF_ONION_CONNECT_TIMEOUT_SECS); if (hOnionSocket != INVALID_SOCKET) { if (fDebug) @@ -1040,12 +1043,23 @@ static CNode* ConnectNodeBtfTail(const string& strBtfAddr, const unsigned char p return pnode; } if (fDebug) - LogPrint("net", "ConnectNodeBtf: direct onion to %s at %s failed, falling back to rendezvous\n", - strBtfAddr.c_str(), strOnionNorm.c_str()); + LogPrint("net", "ConnectNodeBtf: direct onion to %s at %s failed%s\n", + strBtfAddr.c_str(), strOnionNorm.c_str(), + fBtfOnionOnly ? "" : ", falling back to rendezvous"); BtfChurnNoteDialResult(strBtfAddr, strOnionNorm, false); + if (fBtfOnionOnly) + return NULL; } } + if (fBtfOnionOnly) + { + if (fDebug) + LogPrint("net", "ConnectNodeBtf: onion-only mode refused rendezvous fallback for %s\n", + strBtfAddr.c_str()); + return NULL; + } + BtfChurnNoteDialAttempt(strBtfAddr, strMeeting); string strHost; unsigned short nPort = 0; @@ -2012,6 +2026,11 @@ bool StartNode(string& strError) // IP address, and port for the socket that is being bound int nRetryLimit = 15; struct sockaddr_in sockaddr = addrLocalHost.GetSockAddr(); + // Keep addrLocalHost as this node's advertised address, but listen on all + // local interfaces. Managed Tor forwards hidden-service traffic to + // 127.0.0.1:, and binding only to the LAN address made that onion + // endpoint unreachable even though the P2P listener was running. + sockaddr.sin_addr.s_addr = INADDR_ANY; if (bind(hListenSocket, (struct sockaddr*)&sockaddr, sizeof(sockaddr)) == SOCKET_ERROR) { int nErr = WSAGetLastError(); diff --git a/src/net.h b/src/net.h index 9300169..984588b 100644 --- a/src/net.h +++ b/src/net.h @@ -33,6 +33,7 @@ CNode* ConnectNodeBtfResolved(const string& strBtfAddr, const string& strMeeting void ThreadBtfAccept(void* parg); void ThreadBtfConnect(void* parg); extern string strBtfConnect; +extern bool fBtfOnionOnly; // Bootstrap seeds added at runtime with /btfseed=ADDRESS:ENCHEX (repeatable). // The compiled-in list lives in net.cpp; these are appended to it. extern std::vector > vBtfExtraSeeds; diff --git a/src/nostr.cpp b/src/nostr.cpp index 0c1b4b6..35fde43 100644 --- a/src/nostr.cpp +++ b/src/nostr.cpp @@ -1764,11 +1764,12 @@ void ThreadNostrSeed(void* parg) LogPrint("nostr", "Nostr: node pubkey = %s\n", key.PubKeyHex().c_str()); LogPrint("nostr", "Nostr: node .btf address = %s\n", key.BtfAddress().c_str()); - // The self-test below publishes a real descriptor, and PublishDescriptor now - // refuses to publish before ThreadBtfAccept has registered a rendezvous. Give - // registration a bounded head start so the test measures the publish/resolve - // chain instead of losing the race against it. - for (int i = 0; i < 60 && !fShutdown && BtfActiveRelay().empty(); i++) + // The self-test below publishes a real descriptor. Rendezvous-only nodes + // need ThreadBtfAccept to register first; Tor nodes can publish as soon as + // their signed onion endpoint exists, with rendezvous marked pending. + for (int i = 0; i < 60 && !fShutdown && + BtfActiveRelay().empty() && + BtfLocalOnionEndpoint().empty(); i++) Sleep(1000); // One-time self-test of the rendezvous: publish our descriptor to a relay, diff --git a/src/proxy.cpp b/src/proxy.cpp index 7d30911..eed054c 100644 --- a/src/proxy.cpp +++ b/src/proxy.cpp @@ -13,6 +13,7 @@ static bool g_fTorProxy = false; static std::string g_socks5Host; static unsigned short g_socks5Port = 0; static const char* DEFAULT_TOR_SOCKS5_PROXY = "127.0.0.1:9050"; +static const int TOR_CONNECT_TIMEOUT_SECS = 120; static void SetSocketTimeout(SOCKET s, int nTimeoutSecs) { @@ -297,10 +298,25 @@ static bool Socks5Connect(SOCKET s, const std::string& destHost, unsigned char hello[3] = { 0x05, 0x01, 0x00 }; // SOCKS5, one method, no auth if (!WriteN(s, hello, sizeof(hello))) + { + error("SOCKS5 hello write failed for %s:%u\n", + destHost.c_str(), (unsigned)destPort); return false; + } unsigned char choice[2] = { 0, 0 }; - if (!ReadN(s, choice, sizeof(choice)) || choice[0] != 0x05 || choice[1] != 0x00) + if (!ReadN(s, choice, sizeof(choice))) + { + error("SOCKS5 method selection read failed for %s:%u\n", + destHost.c_str(), (unsigned)destPort); + return false; + } + if (choice[0] != 0x05 || choice[1] != 0x00) + { + error("SOCKS5 proxy rejected no-auth method for %s:%u (version=0x%02x method=0x%02x)\n", + destHost.c_str(), (unsigned)destPort, + (unsigned)choice[0], (unsigned)choice[1]); return false; + } std::vector req; req.reserve(7 + destHost.size()); @@ -313,11 +329,25 @@ static bool Socks5Connect(SOCKET s, const std::string& destHost, req.push_back((unsigned char)(destPort >> 8)); req.push_back((unsigned char)(destPort & 0xff)); if (!WriteN(s, &req[0], (int)req.size())) + { + error("SOCKS5 CONNECT request write failed for %s:%u\n", + destHost.c_str(), (unsigned)destPort); return false; + } unsigned char rep[4] = {0,0,0,0}; - if (!ReadN(s, rep, sizeof(rep)) || rep[0] != 0x05) + if (!ReadN(s, rep, sizeof(rep))) + { + error("SOCKS5 CONNECT reply read failed for %s:%u\n", + destHost.c_str(), (unsigned)destPort); return false; + } + if (rep[0] != 0x05) + { + error("SOCKS5 CONNECT reply has invalid version 0x%02x for %s:%u\n", + (unsigned)rep[0], destHost.c_str(), (unsigned)destPort); + return false; + } if (rep[1] != 0x00) { error("SOCKS5 CONNECT to %s:%u failed with reply 0x%02x\n", @@ -335,10 +365,20 @@ static bool Socks5Connect(SOCKET s, const std::string& destHost, nAddr = len; } else + { + error("SOCKS5 CONNECT reply has invalid address type 0x%02x for %s:%u\n", + (unsigned)rep[3], destHost.c_str(), (unsigned)destPort); return false; + } std::vector discard(nAddr + 2); - return ReadN(s, &discard[0], (int)discard.size()); + if (!ReadN(s, &discard[0], (int)discard.size())) + { + error("SOCKS5 CONNECT bind address read failed for %s:%u\n", + destHost.c_str(), (unsigned)destPort); + return false; + } + return true; } SOCKET BtfConnectSocket(const std::string& destHost, unsigned short destPort, @@ -347,9 +387,11 @@ SOCKET BtfConnectSocket(const std::string& destHost, unsigned short destPort, std::string proxyHost; unsigned short proxyPort = 0; bool fProxy = false; + bool fTorProxy = false; CRITICAL_BLOCK(cs_socks5) { fProxy = g_fSocks5Proxy; + fTorProxy = g_fTorProxy; proxyHost = g_socks5Host; proxyPort = g_socks5Port; } @@ -365,6 +407,9 @@ SOCKET BtfConnectSocket(const std::string& destHost, unsigned short destPort, return ConnectDirect(destHost, destPort, nSockSite, nTimeoutSecs); } + if (fTorProxy && nTimeoutSecs > 0 && nTimeoutSecs < TOR_CONNECT_TIMEOUT_SECS) + nTimeoutSecs = TOR_CONNECT_TIMEOUT_SECS; + SOCKET s = ConnectDirect(proxyHost, proxyPort, nSockSite, nTimeoutSecs); if (s == INVALID_SOCKET) return INVALID_SOCKET; -- cgit v1.2.3 From 9e39a50d1f110020b0841d31e4b931aa877cc448 Mon Sep 17 00:00:00 2001 From: bitflash Date: Thu, 13 Aug 2026 16:56:01 -0300 Subject: release: v1.2.19 (nHDNext restore fix, Tor onion transport: managed Tor, direct onion peers, onion-only mode) --- Makefile | 2 +- README.md | 2 +- build-1219.log | 381 ++++++++++++++++++++ build-17.log | 78 +++++ build-appimage.log | 5 + build-node.log | 387 +++++++++++++++++++++ docs/site.md | 4 +- scripts/__pycache__/bitflash_chain.cpython-313.pyc | Bin 0 -> 22668 bytes 8 files changed, 855 insertions(+), 4 deletions(-) create mode 100644 build-1219.log create mode 100644 build-17.log create mode 100644 build-appimage.log create mode 100644 build-node.log create mode 100644 scripts/__pycache__/bitflash_chain.cpython-313.pyc (limited to 'docs') diff --git a/Makefile b/Makefile index d1277a4..60d1227 100644 --- a/Makefile +++ b/Makefile @@ -11,7 +11,7 @@ ROOT := $(shell pwd) NPROC := $(shell nproc 2>/dev/null || echo 2) SUDO := $(shell [ "$$(id -u)" = "0" ] && echo "" || echo "sudo") -VERSION := 1.2.18 +VERSION := 1.2.19 SECP256K1_REPO := https://github.com/bitcoin-core/secp256k1 SECP256K1_COMMIT := 7fecac74aed8e1fd9078380d67dd04663705c989 diff --git a/README.md b/README.md index bb6a4d4..1782bc3 100644 --- a/README.md +++ b/README.md @@ -83,7 +83,7 @@ Public mirrors should make the latest checksum files available both under the versioned release directory and at the release root: ```text -https://releases.bitflash.network/v1.2.18/SHA256SUMS +https://releases.bitflash.network/v1.2.19/SHA256SUMS https://releases.bitflash.network/SHA256SUMS # alias to latest ``` diff --git a/build-1219.log b/build-1219.log new file mode 100644 index 0000000..9aa1b25 --- /dev/null +++ b/build-1219.log @@ -0,0 +1,381 @@ +Lendo listas de pacotes... +Construindo รกrvore de dependรชncias... +Lendo informaรงรฃo de estado... +build-essential jรก รฉ a versรฃo mais nova (12.12). +cmake jรก รฉ a versรฃo mais nova (3.31.6-2). +git jรก รฉ a versรฃo mais nova (1:2.47.3-0+deb13u1). +pkg-config jรก รฉ a versรฃo mais nova (1.8.1-4). +autoconf jรก รฉ a versรฃo mais nova (2.72-3.1). +libtool jรก รฉ a versรฃo mais nova (2.5.4-4). +libssl-dev jรก รฉ a versรฃo mais nova (3.5.6-1~deb13u2). +libdb5.3++-dev jรก รฉ a versรฃo mais nova (5.3.28+dfsg2-9). +libsodium-dev jรก รฉ a versรฃo mais nova (1.0.18-1+deb13u1). +nlohmann-json3-dev jรก รฉ a versรฃo mais nova (3.11.3-2.1). +libboost-system-dev jรก รฉ a versรฃo mais nova (1.83.0.2+b2). +libglfw3-dev jรก รฉ a versรฃo mais nova (3.4-3). +libgl-dev jรก รฉ a versรฃo mais nova (1.7.0-1+b2). +python3-pil jรก รฉ a versรฃo mais nova (11.1.0-5+deb13u4). +clang jรก รฉ a versรฃo mais nova (1:19.0-63). +0 pacotes atualizados, 0 pacotes novos instalados, 0 a serem removidos e 44 nรฃo atualizados. +==> building libsecp256k1 +==> libsecp256k1 pinned at 7fecac74aed8e1fd9078380d67dd04663705c989 +libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'autotools-aux'. +libtoolize: copying file 'autotools-aux/ltmain.sh' +libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'autotools-aux/m4'. +libtoolize: copying file 'autotools-aux/m4/libtool.m4' +libtoolize: copying file 'autotools-aux/m4/ltoptions.m4' +libtoolize: copying file 'autotools-aux/m4/ltsugar.m4' +libtoolize: copying file 'autotools-aux/m4/ltversion.m4' +libtoolize: copying file 'autotools-aux/m4/lt~obsolete.m4' +configure.ac:39: installing 'autotools-aux/ar-lib' +configure.ac:37: installing 'autotools-aux/compile' +configure.ac:24: installing 'autotools-aux/config.guess' +configure.ac:24: installing 'autotools-aux/config.sub' +configure.ac:27: installing 'autotools-aux/install-sh' +configure.ac:27: installing 'autotools-aux/missing' +Makefile.am: installing 'autotools-aux/depcomp' +parallel-tests: installing 'autotools-aux/test-driver' +checking build system type... x86_64-pc-linux-gnu +checking host system type... x86_64-pc-linux-gnu +checking for a BSD-compatible install... /usr/bin/install -c +checking whether sleep supports fractional seconds... yes +checking filesystem timestamp resolution... 0.01 +checking whether build environment is sane... yes +checking for a race-free mkdir -p... /usr/bin/mkdir -p +checking for gawk... no +checking for mawk... mawk +checking whether make sets $(MAKE)... yes +checking whether make supports nested variables... yes +checking xargs -n works... yes +checking for gcc... gcc +checking whether the C compiler works... yes +checking for C compiler default output file name... a.out +checking for suffix of executables... +checking whether we are cross compiling... no +checking for suffix of object files... o +checking whether the compiler supports GNU C... yes +checking whether gcc accepts -g... yes +checking for gcc option to enable C11 features... none needed +checking whether gcc understands -c and -o together... yes +checking whether make supports the include directive... yes (GNU style) +checking dependency style of gcc... gcc3 +checking dependency style of gcc... gcc3 +checking for ar... ar +checking the archiver (ar) interface... ar +checking how to print strings... printf +checking for a sed that does not truncate output... /usr/bin/sed +checking for grep that handles long lines and -e... /usr/bin/grep +checking for egrep... /usr/bin/grep -E +checking for fgrep... /usr/bin/grep -F +checking for ld used by gcc... /usr/bin/ld +checking if the linker (/usr/bin/ld) is GNU ld... yes +checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B +checking the name lister (/usr/bin/nm -B) interface... BSD nm +checking whether ln -s works... yes +checking the maximum length of command line arguments... 1572864 +checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop +checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop +checking for /usr/bin/ld option to reload object files... -r +checking for file... file +checking for objdump... objdump +checking how to recognize dependent libraries... pass_all +checking for dlltool... no +checking how to associate runtime and link libraries... printf %s\n +checking for ranlib... ranlib +checking for archiver @FILE support... @ +checking for strip... strip +checking command to parse /usr/bin/nm -B output from gcc object... ok +checking for sysroot... no +checking for a working dd... /usr/bin/dd +checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 +checking for mt... mt +checking if mt is a manifest tool... no +checking for stdio.h... yes +checking for stdlib.h... yes +checking for string.h... yes +checking for inttypes.h... yes +checking for stdint.h... yes +checking for strings.h... yes +checking for sys/stat.h... yes +checking for sys/types.h... yes +checking for unistd.h... yes +checking for dlfcn.h... yes +checking for objdir... .libs +checking if gcc supports -fno-rtti -fno-exceptions... no +checking for gcc option to produce PIC... -fPIC -DPIC +checking if gcc PIC flag -fPIC -DPIC works... yes +checking if gcc static flag -static works... yes +checking if gcc supports -c -o file.o... yes +checking if gcc supports -c -o file.o... (cached) yes +checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes +checking dynamic linker characteristics... GNU/Linux ld.so +checking how to hardcode library paths into programs... immediate +checking whether stripping libraries is possible... yes +checking if libtool supports shared libraries... yes +checking whether to build shared libraries... no +checking whether to build static libraries... yes +checking if gcc supports -Werror... yes +checking if gcc supports -std=c89 -pedantic -Wno-long-long -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef... yes +checking if gcc supports -Wno-overlength-strings... yes +checking if gcc supports -Wall... yes +checking if gcc supports -Wno-unused-function... yes +checking if gcc supports -Wextra... yes +checking if gcc supports -Wcast-align... yes +checking if gcc supports -Wcast-align=strict... yes +checking if gcc supports -Wconditional-uninitialized... no +checking if gcc supports -Wreserved-identifier... no +checking if gcc supports -Wtrailing-whitespace=any... no +checking if gcc supports -Wleading-whitespace=spaces... no +checking for valgrind support... +checking for x86_64 assembly availability... yes +checking that generated files are newer than configure... done +configure: creating ./config.status +config.status: creating Makefile +config.status: creating libsecp256k1.pc +config.status: executing depfiles commands +config.status: executing libtool commands + +Build Options: + with external callbacks = no + with benchmarks = no + with tests = no + with exhaustive tests = yes + with ctime tests = no + with coverage = no + with examples = no + module ecdh = yes + module recovery = no + module extrakeys = yes + module schnorrsig = yes + module musig = yes + module ellswift = yes + module silentpayments = yes + + asm = x86_64 + ecmult window size = 15 + ecmult gen table size = 86 KiB + + valgrind = no + CC = gcc + CPPFLAGS = + SECP_CFLAGS = -O2 -std=c89 -pedantic -Wno-long-long -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef -Wno-overlength-strings -Wall -Wno-unused-function -Wextra -Wcast-align -Wcast-align=strict + CFLAGS = -g -O2 + LDFLAGS = +make[1]: Entrando no diretรณrio '/root/rel16/secp256k1-build' + +make all-am +make[2]: Entrando no diretรณrio '/root/rel16/secp256k1-build' + + CC src/exhaustive_tests-tests_exhaustive.o + CCLD exhaustive_tests + CC src/libsecp256k1_la-secp256k1.lo + CC src/libsecp256k1_precomputed_la-precomputed_ecmult.lo + CC src/libsecp256k1_precomputed_la-precomputed_ecmult_gen.lo + CCLD libsecp256k1_precomputed.la + CCLD libsecp256k1.la +make[2]: Saindo do diretรณrio '/root/rel16/secp256k1-build' + +make[1]: Saindo do diretรณrio '/root/rel16/secp256k1-build' + +make[1]: Entrando no diretรณrio '/root/rel16/secp256k1-build' + +make install-am +make[2]: Entrando no diretรณrio '/root/rel16/secp256k1-build' + +make[3]: Entrando no diretรณrio '/root/rel16/secp256k1-build' + + /usr/bin/mkdir -p '/usr/local/lib' + /bin/bash ./libtool --mode=install /usr/bin/install -c libsecp256k1.la '/usr/local/lib' +libtool: install: /usr/bin/install -c .libs/libsecp256k1.lai /usr/local/lib/libsecp256k1.la +libtool: install: /usr/bin/install -c .libs/libsecp256k1.a /usr/local/lib/libsecp256k1.a +libtool: install: chmod 644 /usr/local/lib/libsecp256k1.a +libtool: install: ranlib /usr/local/lib/libsecp256k1.a +libtool: finish: PATH="/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin" ldconfig -n /usr/local/lib +---------------------------------------------------------------------- +Libraries have been installed in: + /usr/local/lib + +If you ever happen to want to link against installed libraries +in a given directory, LIBDIR, you must either use libtool, and +specify the full pathname of the library, or use the '-LLIBDIR' +flag during linking and do at least one of the following: + - add LIBDIR to the 'LD_LIBRARY_PATH' environment variable + during execution + - add LIBDIR to the 'LD_RUN_PATH' environment variable + during linking + - use the '-Wl,-rpath -Wl,LIBDIR' linker flag + - have your system administrator add LIBDIR to '/etc/ld.so.conf' + +See any operating system documentation about shared libraries for +more information, such as the ld(1) and ld.so(8) manual pages. +---------------------------------------------------------------------- + /usr/bin/mkdir -p '/usr/local/include' + /usr/bin/install -c -m 644 include/secp256k1.h include/secp256k1_preallocated.h include/secp256k1_ecdh.h include/secp256k1_extrakeys.h include/secp256k1_schnorrsig.h include/secp256k1_musig.h include/secp256k1_ellswift.h include/secp256k1_silentpayments.h '/usr/local/include' + /usr/bin/mkdir -p '/usr/local/lib/pkgconfig' + /usr/bin/install -c -m 644 libsecp256k1.pc '/usr/local/lib/pkgconfig' +make[3]: Saindo do diretรณrio '/root/rel16/secp256k1-build' + +make[2]: Saindo do diretรณrio '/root/rel16/secp256k1-build' + +make[1]: Saindo do diretรณrio '/root/rel16/secp256k1-build' + +==> libsecp256k1 done +==> building RandomX +==> RandomX pinned at 1e9d4b2df63fa6edf46b06789486e23c0fcaa55a +-- The C compiler identification is GNU 14.2.0 +-- The CXX compiler identification is GNU 14.2.0 +-- Detecting C compiler ABI info +-- Detecting C compiler ABI info - done +-- Check for working C compiler: /usr/bin/cc - skipped +-- Detecting C compile features +-- Detecting C compile features - done +-- Detecting CXX compiler ABI info +-- Detecting CXX compiler ABI info - done +-- Check for working CXX compiler: /usr/bin/c++ - skipped +-- Detecting CXX compile features +-- Detecting CXX compile features - done +-- Performing Test _maes_cxx +-- Performing Test _maes_cxx - Success +-- Setting CXX flag -maes +-- Performing Test _maes_c +-- Performing Test _maes_c - Success +-- Setting C flag -maes +-- Performing Test HAVE_SSSE3 +-- Performing Test HAVE_SSSE3 - Success +-- Performing Test HAVE_AVX2 +-- Performing Test HAVE_AVX2 - Success +-- Performing Test CMAKE_HAVE_LIBC_PTHREAD +-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success +-- Found Threads: TRUE +-- Performing Test HAVE_CXX_ATOMICS +-- Performing Test HAVE_CXX_ATOMICS - Success +-- Performing Test HAVE_POSIX_MEMALIGN +-- Performing Test HAVE_POSIX_MEMALIGN - Success +-- Configuring done (0.9s) +-- Generating done (0.0s) +-- Build files have been written to: /root/RandomX/build +make[1]: Entrando no diretรณrio '/root/RandomX/build' + +[ 3%] Building CXX object CMakeFiles/randomx.dir/src/aes_hash.cpp.o +[ 7%] Building C object CMakeFiles/randomx.dir/src/argon2_ref.c.o +[ 11%] Building C object CMakeFiles/randomx.dir/src/argon2_ssse3.c.o +[ 14%] Building C object CMakeFiles/randomx.dir/src/argon2_avx2.c.o +[ 18%] Building CXX object CMakeFiles/randomx.dir/src/bytecode_machine.cpp.o +[ 22%] Building CXX object CMakeFiles/randomx.dir/src/cpu.cpp.o +[ 25%] Building CXX object CMakeFiles/randomx.dir/src/dataset.cpp.o +[ 29%] Building CXX object CMakeFiles/randomx.dir/src/soft_aes.cpp.o +[ 33%] Building C object CMakeFiles/randomx.dir/src/virtual_memory.c.o +[ 37%] Building CXX object CMakeFiles/randomx.dir/src/vm_interpreted.cpp.o +[ 40%] Building CXX object CMakeFiles/randomx.dir/src/allocator.cpp.o +[ 44%] Building CXX object CMakeFiles/randomx.dir/src/assembly_generator_x86.cpp.o +[ 48%] Building CXX object CMakeFiles/randomx.dir/src/instruction.cpp.o +[ 51%] Building CXX object CMakeFiles/randomx.dir/src/randomx.cpp.o +[ 55%] Building CXX object CMakeFiles/randomx.dir/src/superscalar.cpp.o +[ 59%] Building CXX object CMakeFiles/randomx.dir/src/vm_compiled.cpp.o +[ 62%] Building CXX object CMakeFiles/randomx.dir/src/vm_interpreted_light.cpp.o +[ 66%] Building C object CMakeFiles/randomx.dir/src/argon2_core.c.o +[ 70%] Building CXX object CMakeFiles/randomx.dir/src/blake2_generator.cpp.o +[ 74%] Building CXX object CMakeFiles/randomx.dir/src/instructions_portable.cpp.o +[ 77%] Building C object CMakeFiles/randomx.dir/src/reciprocal.c.o +[ 81%] Building CXX object CMakeFiles/randomx.dir/src/virtual_machine.cpp.o +[ 85%] Building CXX object CMakeFiles/randomx.dir/src/vm_compiled_light.cpp.o +[ 88%] Building C object CMakeFiles/randomx.dir/src/blake2/blake2b.c.o +[ 92%] Building CXX object CMakeFiles/randomx.dir/src/jit_compiler_x86.cpp.o +[ 96%] Building C object CMakeFiles/randomx.dir/src/jit_compiler_x86_static.S.o +[100%] Linking CXX static library librandomx.a +[100%] Built target randomx +make[1]: Saindo do diretรณrio '/root/RandomX/build' + +==> RandomX done +make -C src -f Makefile -j1 +make[1]: Entrando no diretรณrio '/root/rel16/src' + +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/util.o util.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/script.o script.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/db.o db.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/net.o net.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/main.o main.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/market.o market.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/nostr.o nostr.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/randomx_pow.o randomx_pow.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/btfaddr.o btfaddr.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/btfchan.o btfchan.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/btfrv.o btfrv.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/btftunnel.o btftunnel.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/rpc.o rpc.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/selftest.o selftest.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/bip32.o bip32.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/walletcmd.o walletcmd.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/proxy.o proxy.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/gui.o gui.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/main_gui.o main_gui.cpp +g++ -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o bitflash obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o obj/imgui_widgets.o obj/imgui_impl_glfw.o obj/imgui_impl_opengl3.o obj/util.o obj/script.o obj/db.o obj/net.o obj/main.o obj/market.o obj/sha.o obj/nostr.o obj/randomx_pow.o obj/btfaddr.o obj/btfchan.o obj/btfrv.o obj/btftunnel.o obj/rpc.o obj/selftest.o obj/bip32.o obj/walletcmd.o obj/proxy.o obj/gui.o obj/main_gui.o -L/usr/local/lib -lsecp256k1 -L/root/RandomX/build -lrandomx -lsodium -lssl -lcrypto -ldb_cxx -lpthread -lglfw -lGL +Built: bitflash +make[1]: Saindo do diretรณrio '/root/rel16/src' + +strip src/bitflash +make -f Makefile appimage +make[1]: Entrando no diretรณrio '/root/rel16' + +==> building AppImage + bundled libsodium.so.23 + bundled libdb_cxx-5.3.so + bundled libglfw.so.3 +Built: Bitflash-1.2.19-x86_64.AppImage +make[1]: Saindo do diretรณrio '/root/rel16' + +make -f Makefile linux-node +make[1]: Entrando no diretรณrio '/root/rel16' + +Lendo listas de pacotes... +Construindo รกrvore de dependรชncias... +Lendo informaรงรฃo de estado... +build-essential jรก รฉ a versรฃo mais nova (12.12). +cmake jรก รฉ a versรฃo mais nova (3.31.6-2). +git jรก รฉ a versรฃo mais nova (1:2.47.3-0+deb13u1). +pkg-config jรก รฉ a versรฃo mais nova (1.8.1-4). +autoconf jรก รฉ a versรฃo mais nova (2.72-3.1). +libtool jรก รฉ a versรฃo mais nova (2.5.4-4). +libssl-dev jรก รฉ a versรฃo mais nova (3.5.6-1~deb13u2). +libdb5.3++-dev jรก รฉ a versรฃo mais nova (5.3.28+dfsg2-9). +libsodium-dev jรก รฉ a versรฃo mais nova (1.0.18-1+deb13u1). +nlohmann-json3-dev jรก รฉ a versรฃo mais nova (3.11.3-2.1). +libboost-system-dev jรก รฉ a versรฃo mais nova (1.83.0.2+b2). +libglfw3-dev jรก รฉ a versรฃo mais nova (3.4-3). +libgl-dev jรก รฉ a versรฃo mais nova (1.7.0-1+b2). +python3-pil jรก รฉ a versรฃo mais nova (11.1.0-5+deb13u4). +clang jรก รฉ a versรฃo mais nova (1:19.0-63). +0 pacotes atualizados, 0 pacotes novos instalados, 0 a serem removidos e 44 nรฃo atualizados. +==> libsecp256k1 already installed +==> RandomX already built +make -C src -f Makefile bitflash-node -j1 +make[2]: Entrando no diretรณrio '/root/rel16/src' + +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/util.o util.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/script.o script.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/db.o db.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/net.o net.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/main.o main.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/market.o market.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/nostr.o nostr.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/randomx_pow.o randomx_pow.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/btfaddr.o btfaddr.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/btfchan.o btfchan.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/btfrv.o btfrv.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/btftunnel.o btftunnel.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/rpc.o rpc.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/selftest.o selftest.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/bip32.o bip32.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/walletcmd.o walletcmd.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/proxy.o proxy.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/main_gui.o main_gui.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/headless.o headless.cpp +g++ -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o bitflash-node obj-node/util.o obj-node/script.o obj-node/db.o obj-node/net.o obj-node/main.o obj-node/market.o obj-node/sha.o obj-node/nostr.o obj-node/randomx_pow.o obj-node/btfaddr.o obj-node/btfchan.o obj-node/btfrv.o obj-node/btftunnel.o obj-node/rpc.o obj-node/selftest.o obj-node/bip32.o obj-node/walletcmd.o obj-node/proxy.o obj-node/main_gui.o obj-node/headless.o -L/usr/local/lib -lsecp256k1 -L/root/RandomX/build -lrandomx -lsodium -lssl -lcrypto -ldb_cxx -lpthread +Built: bitflash-node (headless, no libGL needed) +make[2]: Saindo do diretรณrio '/root/rel16/src' + +strip src/bitflash-node +cp src/bitflash-node bitflash-node-1.2.19-x86_64 +Built: bitflash-node-1.2.19-x86_64 +make[1]: Saindo do diretรณrio '/root/rel16' + diff --git a/build-17.log b/build-17.log new file mode 100644 index 0000000..6c8c841 --- /dev/null +++ b/build-17.log @@ -0,0 +1,78 @@ +Lendo listas de pacotes... +Construindo รกrvore de dependรชncias... +Lendo informaรงรฃo de estado... +build-essential jรก รฉ a versรฃo mais nova (12.12). +cmake jรก รฉ a versรฃo mais nova (3.31.6-2). +git jรก รฉ a versรฃo mais nova (1:2.47.3-0+deb13u1). +pkg-config jรก รฉ a versรฃo mais nova (1.8.1-4). +autoconf jรก รฉ a versรฃo mais nova (2.72-3.1). +libtool jรก รฉ a versรฃo mais nova (2.5.4-4). +libssl-dev jรก รฉ a versรฃo mais nova (3.5.6-1~deb13u2). +libdb5.3++-dev jรก รฉ a versรฃo mais nova (5.3.28+dfsg2-9). +libsodium-dev jรก รฉ a versรฃo mais nova (1.0.18-1+deb13u1). +nlohmann-json3-dev jรก รฉ a versรฃo mais nova (3.11.3-2.1). +libboost-system-dev jรก รฉ a versรฃo mais nova (1.83.0.2+b2). +libglfw3-dev jรก รฉ a versรฃo mais nova (3.4-3). +libgl-dev jรก รฉ a versรฃo mais nova (1.7.0-1+b2). +python3-pil jรก รฉ a versรฃo mais nova (11.1.0-5+deb13u4). +clang jรก รฉ a versรฃo mais nova (1:19.0-63). +0 pacotes atualizados, 0 pacotes novos instalados, 0 a serem removidos e 49 nรฃo atualizados. +==> libsecp256k1 already installed +==> RandomX already built +make -C src -f Makefile -j1 +make[1]: Entrando no diretรณrio '/root/rel16/src' + +make[1]: Nada a ser feito para 'all'. +make[1]: Saindo do diretรณrio '/root/rel16/src' + +strip src/bitflash +make -f Makefile appimage +make[1]: Entrando no diretรณrio '/root/rel16' + +==> building AppImage + bundled libsodium.so.23 + bundled libdb_cxx-5.3.so + bundled libglfw.so.3 +Built: Bitflash-1.2.17-x86_64.AppImage +make[1]: Saindo do diretรณrio '/root/rel16' + +make -f Makefile linux-node +make[1]: Entrando no diretรณrio '/root/rel16' + +Lendo listas de pacotes... +Construindo รกrvore de dependรชncias... +Lendo informaรงรฃo de estado... +build-essential jรก รฉ a versรฃo mais nova (12.12). +cmake jรก รฉ a versรฃo mais nova (3.31.6-2). +git jรก รฉ a versรฃo mais nova (1:2.47.3-0+deb13u1). +pkg-config jรก รฉ a versรฃo mais nova (1.8.1-4). +autoconf jรก รฉ a versรฃo mais nova (2.72-3.1). +libtool jรก รฉ a versรฃo mais nova (2.5.4-4). +libssl-dev jรก รฉ a versรฃo mais nova (3.5.6-1~deb13u2). +libdb5.3++-dev jรก รฉ a versรฃo mais nova (5.3.28+dfsg2-9). +libsodium-dev jรก รฉ a versรฃo mais nova (1.0.18-1+deb13u1). +nlohmann-json3-dev jรก รฉ a versรฃo mais nova (3.11.3-2.1). +libboost-system-dev jรก รฉ a versรฃo mais nova (1.83.0.2+b2). +libglfw3-dev jรก รฉ a versรฃo mais nova (3.4-3). +libgl-dev jรก รฉ a versรฃo mais nova (1.7.0-1+b2). +python3-pil jรก รฉ a versรฃo mais nova (11.1.0-5+deb13u4). +clang jรก รฉ a versรฃo mais nova (1:19.0-63). +0 pacotes atualizados, 0 pacotes novos instalados, 0 a serem removidos e 49 nรฃo atualizados. +==> libsecp256k1 already installed +==> RandomX already built +make -C src -f Makefile bitflash-node -j1 +make[2]: Entrando no diretรณrio '/root/rel16/src' + +make[2]: 'bitflash-node' estรก atualizado. +make[2]: Saindo do diretรณrio '/root/rel16/src' + +strip src/bitflash-node +cp src/bitflash-node bitflash-node-1.2.17-x86_64 +Built: bitflash-node-1.2.17-x86_64 +make[1]: Saindo do diretรณrio '/root/rel16' + +==> building AppImage + bundled libsodium.so.23 + bundled libdb_cxx-5.3.so + bundled libglfw.so.3 +Built: Bitflash-1.2.17-x86_64.AppImage diff --git a/build-appimage.log b/build-appimage.log new file mode 100644 index 0000000..d6b95ed --- /dev/null +++ b/build-appimage.log @@ -0,0 +1,5 @@ +==> building AppImage + bundled libsodium.so.23 + bundled libdb_cxx-5.3.so + bundled libglfw.so.3 +Built: Bitflash-1.2.16-x86_64.AppImage diff --git a/build-node.log b/build-node.log new file mode 100644 index 0000000..951c2a0 --- /dev/null +++ b/build-node.log @@ -0,0 +1,387 @@ +Lendo listas de pacotes... +Construindo รกrvore de dependรชncias... +Lendo informaรงรฃo de estado... +build-essential jรก รฉ a versรฃo mais nova (12.12). +cmake jรก รฉ a versรฃo mais nova (3.31.6-2). +git jรก รฉ a versรฃo mais nova (1:2.47.3-0+deb13u1). +pkg-config jรก รฉ a versรฃo mais nova (1.8.1-4). +autoconf jรก รฉ a versรฃo mais nova (2.72-3.1). +libtool jรก รฉ a versรฃo mais nova (2.5.4-4). +libssl-dev jรก รฉ a versรฃo mais nova (3.5.6-1~deb13u2). +libdb5.3++-dev jรก รฉ a versรฃo mais nova (5.3.28+dfsg2-9). +libsodium-dev jรก รฉ a versรฃo mais nova (1.0.18-1+deb13u1). +nlohmann-json3-dev jรก รฉ a versรฃo mais nova (3.11.3-2.1). +libboost-system-dev jรก รฉ a versรฃo mais nova (1.83.0.2+b2). +libglfw3-dev jรก รฉ a versรฃo mais nova (3.4-3). +libgl-dev jรก รฉ a versรฃo mais nova (1.7.0-1+b2). +python3-pil jรก รฉ a versรฃo mais nova (11.1.0-5+deb13u4). +clang jรก รฉ a versรฃo mais nova (1:19.0-63). +0 pacotes atualizados, 0 pacotes novos instalados, 0 a serem removidos e 49 nรฃo atualizados. +==> building libsecp256k1 +==> libsecp256k1 pinned at 7fecac74aed8e1fd9078380d67dd04663705c989 +libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'autotools-aux'. +libtoolize: copying file 'autotools-aux/ltmain.sh' +libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'autotools-aux/m4'. +libtoolize: copying file 'autotools-aux/m4/libtool.m4' +libtoolize: copying file 'autotools-aux/m4/ltoptions.m4' +libtoolize: copying file 'autotools-aux/m4/ltsugar.m4' +libtoolize: copying file 'autotools-aux/m4/ltversion.m4' +libtoolize: copying file 'autotools-aux/m4/lt~obsolete.m4' +configure.ac:39: installing 'autotools-aux/ar-lib' +configure.ac:37: installing 'autotools-aux/compile' +configure.ac:24: installing 'autotools-aux/config.guess' +configure.ac:24: installing 'autotools-aux/config.sub' +configure.ac:27: installing 'autotools-aux/install-sh' +configure.ac:27: installing 'autotools-aux/missing' +Makefile.am: installing 'autotools-aux/depcomp' +parallel-tests: installing 'autotools-aux/test-driver' +checking build system type... x86_64-pc-linux-gnu +checking host system type... x86_64-pc-linux-gnu +checking for a BSD-compatible install... /usr/bin/install -c +checking whether sleep supports fractional seconds... yes +checking filesystem timestamp resolution... 0.01 +checking whether build environment is sane... yes +checking for a race-free mkdir -p... /usr/bin/mkdir -p +checking for gawk... no +checking for mawk... mawk +checking whether make sets $(MAKE)... yes +checking whether make supports nested variables... yes +checking xargs -n works... yes +checking for gcc... gcc +checking whether the C compiler works... yes +checking for C compiler default output file name... a.out +checking for suffix of executables... +checking whether we are cross compiling... no +checking for suffix of object files... o +checking whether the compiler supports GNU C... yes +checking whether gcc accepts -g... yes +checking for gcc option to enable C11 features... none needed +checking whether gcc understands -c and -o together... yes +checking whether make supports the include directive... yes (GNU style) +checking dependency style of gcc... gcc3 +checking dependency style of gcc... gcc3 +checking for ar... ar +checking the archiver (ar) interface... ar +checking how to print strings... printf +checking for a sed that does not truncate output... /usr/bin/sed +checking for grep that handles long lines and -e... /usr/bin/grep +checking for egrep... /usr/bin/grep -E +checking for fgrep... /usr/bin/grep -F +checking for ld used by gcc... /usr/bin/ld +checking if the linker (/usr/bin/ld) is GNU ld... yes +checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B +checking the name lister (/usr/bin/nm -B) interface... BSD nm +checking whether ln -s works... yes +checking the maximum length of command line arguments... 1572864 +checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop +checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop +checking for /usr/bin/ld option to reload object files... -r +checking for file... file +checking for objdump... objdump +checking how to recognize dependent libraries... pass_all +checking for dlltool... no +checking how to associate runtime and link libraries... printf %s\n +checking for ranlib... ranlib +checking for archiver @FILE support... @ +checking for strip... strip +checking command to parse /usr/bin/nm -B output from gcc object... ok +checking for sysroot... no +checking for a working dd... /usr/bin/dd +checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 +checking for mt... mt +checking if mt is a manifest tool... no +checking for stdio.h... yes +checking for stdlib.h... yes +checking for string.h... yes +checking for inttypes.h... yes +checking for stdint.h... yes +checking for strings.h... yes +checking for sys/stat.h... yes +checking for sys/types.h... yes +checking for unistd.h... yes +checking for dlfcn.h... yes +checking for objdir... .libs +checking if gcc supports -fno-rtti -fno-exceptions... no +checking for gcc option to produce PIC... -fPIC -DPIC +checking if gcc PIC flag -fPIC -DPIC works... yes +checking if gcc static flag -static works... yes +checking if gcc supports -c -o file.o... yes +checking if gcc supports -c -o file.o... (cached) yes +checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes +checking dynamic linker characteristics... GNU/Linux ld.so +checking how to hardcode library paths into programs... immediate +checking whether stripping libraries is possible... yes +checking if libtool supports shared libraries... yes +checking whether to build shared libraries... no +checking whether to build static libraries... yes +checking if gcc supports -Werror... yes +checking if gcc supports -std=c89 -pedantic -Wno-long-long -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef... yes +checking if gcc supports -Wno-overlength-strings... yes +checking if gcc supports -Wall... yes +checking if gcc supports -Wno-unused-function... yes +checking if gcc supports -Wextra... yes +checking if gcc supports -Wcast-align... yes +checking if gcc supports -Wcast-align=strict... yes +checking if gcc supports -Wconditional-uninitialized... no +checking if gcc supports -Wreserved-identifier... no +checking if gcc supports -Wtrailing-whitespace=any... no +checking if gcc supports -Wleading-whitespace=spaces... no +checking for valgrind support... +checking for x86_64 assembly availability... yes +checking that generated files are newer than configure... done +configure: creating ./config.status +config.status: creating Makefile +config.status: creating libsecp256k1.pc +config.status: executing depfiles commands +config.status: executing libtool commands + +Build Options: + with external callbacks = no + with benchmarks = no + with tests = no + with exhaustive tests = yes + with ctime tests = no + with coverage = no + with examples = no + module ecdh = yes + module recovery = no + module extrakeys = yes + module schnorrsig = yes + module musig = yes + module ellswift = yes + module silentpayments = yes + + asm = x86_64 + ecmult window size = 15 + ecmult gen table size = 86 KiB + + valgrind = no + CC = gcc + CPPFLAGS = + SECP_CFLAGS = -O2 -std=c89 -pedantic -Wno-long-long -Wnested-externs -Wshadow -Wstrict-prototypes -Wundef -Wno-overlength-strings -Wall -Wno-unused-function -Wextra -Wcast-align -Wcast-align=strict + CFLAGS = -g -O2 + LDFLAGS = +make[1]: Entrando no diretรณrio '/root/rel16/secp256k1-build' + +make all-am +make[2]: Entrando no diretรณrio '/root/rel16/secp256k1-build' + + CC src/exhaustive_tests-tests_exhaustive.o + CCLD exhaustive_tests + CC src/libsecp256k1_la-secp256k1.lo + CC src/libsecp256k1_precomputed_la-precomputed_ecmult.lo + CC src/libsecp256k1_precomputed_la-precomputed_ecmult_gen.lo + CCLD libsecp256k1_precomputed.la + CCLD libsecp256k1.la +make[2]: Saindo do diretรณrio '/root/rel16/secp256k1-build' + +make[1]: Saindo do diretรณrio '/root/rel16/secp256k1-build' + +make[1]: Entrando no diretรณrio '/root/rel16/secp256k1-build' + +make install-am +make[2]: Entrando no diretรณrio '/root/rel16/secp256k1-build' + +make[3]: Entrando no diretรณrio '/root/rel16/secp256k1-build' + + /usr/bin/mkdir -p '/usr/local/lib' + /bin/bash ./libtool --mode=install /usr/bin/install -c libsecp256k1.la '/usr/local/lib' +libtool: install: /usr/bin/install -c .libs/libsecp256k1.lai /usr/local/lib/libsecp256k1.la +libtool: install: /usr/bin/install -c .libs/libsecp256k1.a /usr/local/lib/libsecp256k1.a +libtool: install: chmod 644 /usr/local/lib/libsecp256k1.a +libtool: install: ranlib /usr/local/lib/libsecp256k1.a +libtool: finish: PATH="/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin" ldconfig -n /usr/local/lib +---------------------------------------------------------------------- +Libraries have been installed in: + /usr/local/lib + +If you ever happen to want to link against installed libraries +in a given directory, LIBDIR, you must either use libtool, and +specify the full pathname of the library, or use the '-LLIBDIR' +flag during linking and do at least one of the following: + - add LIBDIR to the 'LD_LIBRARY_PATH' environment variable + during execution + - add LIBDIR to the 'LD_RUN_PATH' environment variable + during linking + - use the '-Wl,-rpath -Wl,LIBDIR' linker flag + - have your system administrator add LIBDIR to '/etc/ld.so.conf' + +See any operating system documentation about shared libraries for +more information, such as the ld(1) and ld.so(8) manual pages. +---------------------------------------------------------------------- + /usr/bin/mkdir -p '/usr/local/include' + /usr/bin/install -c -m 644 include/secp256k1.h include/secp256k1_preallocated.h include/secp256k1_ecdh.h include/secp256k1_extrakeys.h include/secp256k1_schnorrsig.h include/secp256k1_musig.h include/secp256k1_ellswift.h include/secp256k1_silentpayments.h '/usr/local/include' + /usr/bin/mkdir -p '/usr/local/lib/pkgconfig' + /usr/bin/install -c -m 644 libsecp256k1.pc '/usr/local/lib/pkgconfig' +make[3]: Saindo do diretรณrio '/root/rel16/secp256k1-build' + +make[2]: Saindo do diretรณrio '/root/rel16/secp256k1-build' + +make[1]: Saindo do diretรณrio '/root/rel16/secp256k1-build' + +==> libsecp256k1 done +==> building RandomX +==> RandomX pinned at 1e9d4b2df63fa6edf46b06789486e23c0fcaa55a +-- The C compiler identification is GNU 14.2.0 +-- The CXX compiler identification is GNU 14.2.0 +-- Detecting C compiler ABI info +-- Detecting C compiler ABI info - done +-- Check for working C compiler: /usr/bin/cc - skipped +-- Detecting C compile features +-- Detecting C compile features - done +-- Detecting CXX compiler ABI info +-- Detecting CXX compiler ABI info - done +-- Check for working CXX compiler: /usr/bin/c++ - skipped +-- Detecting CXX compile features +-- Detecting CXX compile features - done +-- Performing Test _maes_cxx +-- Performing Test _maes_cxx - Success +-- Setting CXX flag -maes +-- Performing Test _maes_c +-- Performing Test _maes_c - Success +-- Setting C flag -maes +-- Performing Test HAVE_SSSE3 +-- Performing Test HAVE_SSSE3 - Success +-- Performing Test HAVE_AVX2 +-- Performing Test HAVE_AVX2 - Success +-- Performing Test CMAKE_HAVE_LIBC_PTHREAD +-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success +-- Found Threads: TRUE +-- Performing Test HAVE_CXX_ATOMICS +-- Performing Test HAVE_CXX_ATOMICS - Success +-- Performing Test HAVE_POSIX_MEMALIGN +-- Performing Test HAVE_POSIX_MEMALIGN - Success +-- Configuring done (1.0s) +-- Generating done (0.0s) +-- Build files have been written to: /root/RandomX/build +make[1]: Entrando no diretรณrio '/root/RandomX/build' + +[ 3%] Building CXX object CMakeFiles/randomx.dir/src/aes_hash.cpp.o +[ 7%] Building C object CMakeFiles/randomx.dir/src/argon2_ref.c.o +[ 11%] Building C object CMakeFiles/randomx.dir/src/argon2_ssse3.c.o +[ 14%] Building C object CMakeFiles/randomx.dir/src/argon2_avx2.c.o +[ 18%] Building CXX object CMakeFiles/randomx.dir/src/bytecode_machine.cpp.o +[ 22%] Building CXX object CMakeFiles/randomx.dir/src/cpu.cpp.o +[ 25%] Building CXX object CMakeFiles/randomx.dir/src/dataset.cpp.o +[ 29%] Building CXX object CMakeFiles/randomx.dir/src/soft_aes.cpp.o +[ 33%] Building C object CMakeFiles/randomx.dir/src/virtual_memory.c.o +[ 37%] Building CXX object CMakeFiles/randomx.dir/src/vm_interpreted.cpp.o +[ 40%] Building CXX object CMakeFiles/randomx.dir/src/allocator.cpp.o +[ 44%] Building CXX object CMakeFiles/randomx.dir/src/assembly_generator_x86.cpp.o +[ 48%] Building CXX object CMakeFiles/randomx.dir/src/instruction.cpp.o +[ 51%] Building CXX object CMakeFiles/randomx.dir/src/randomx.cpp.o +[ 55%] Building CXX object CMakeFiles/randomx.dir/src/superscalar.cpp.o +[ 59%] Building CXX object CMakeFiles/randomx.dir/src/vm_compiled.cpp.o +[ 62%] Building CXX object CMakeFiles/randomx.dir/src/vm_interpreted_light.cpp.o +[ 66%] Building C object CMakeFiles/randomx.dir/src/argon2_core.c.o +[ 70%] Building CXX object CMakeFiles/randomx.dir/src/blake2_generator.cpp.o +[ 74%] Building CXX object CMakeFiles/randomx.dir/src/instructions_portable.cpp.o +[ 77%] Building C object CMakeFiles/randomx.dir/src/reciprocal.c.o +[ 81%] Building CXX object CMakeFiles/randomx.dir/src/virtual_machine.cpp.o +[ 85%] Building CXX object CMakeFiles/randomx.dir/src/vm_compiled_light.cpp.o +[ 88%] Building C object CMakeFiles/randomx.dir/src/blake2/blake2b.c.o +[ 92%] Building CXX object CMakeFiles/randomx.dir/src/jit_compiler_x86.cpp.o +[ 96%] Building C object CMakeFiles/randomx.dir/src/jit_compiler_x86_static.S.o +[100%] Linking CXX static library librandomx.a +[100%] Built target randomx +make[1]: Saindo do diretรณrio '/root/RandomX/build' + +==> RandomX done +make -C src -f Makefile -j1 +make[1]: Entrando no diretรณrio '/root/rel16/src' + +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/imgui.o imgui/imgui.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/imgui_draw.o imgui/imgui_draw.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/imgui_tables.o imgui/imgui_tables.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/imgui_widgets.o imgui/imgui_widgets.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/imgui_impl_glfw.o imgui/imgui_impl_glfw.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/imgui_impl_opengl3.o imgui/imgui_impl_opengl3.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/util.o util.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/script.o script.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/db.o db.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/net.o net.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/main.o main.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/market.o market.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -O3 -o obj/sha.o sha.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/nostr.o nostr.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/randomx_pow.o randomx_pow.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/btfaddr.o btfaddr.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/btfchan.o btfchan.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/btfrv.o btfrv.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/btftunnel.o btftunnel.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/rpc.o rpc.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/selftest.o selftest.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/bip32.o bip32.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/walletcmd.o walletcmd.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/gui.o gui.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o obj/main_gui.o main_gui.cpp +g++ -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o bitflash obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o obj/imgui_widgets.o obj/imgui_impl_glfw.o obj/imgui_impl_opengl3.o obj/util.o obj/script.o obj/db.o obj/net.o obj/main.o obj/market.o obj/sha.o obj/nostr.o obj/randomx_pow.o obj/btfaddr.o obj/btfchan.o obj/btfrv.o obj/btftunnel.o obj/rpc.o obj/selftest.o obj/bip32.o obj/walletcmd.o obj/gui.o obj/main_gui.o -L/usr/local/lib -lsecp256k1 -L/root/RandomX/build -lrandomx -lsodium -lssl -lcrypto -ldb_cxx -lpthread -lglfw -lGL +Built: bitflash +make[1]: Saindo do diretรณrio '/root/rel16/src' + +strip src/bitflash +make -f Makefile appimage +make[1]: Entrando no diretรณrio '/root/rel16' + +==> building AppImage + bundled libsodium.so.23 + bundled libdb_cxx-5.3.so + bundled libglfw.so.3 +Built: Bitflash-1.2.16-x86_64.AppImage +make[1]: Saindo do diretรณrio '/root/rel16' + +make -f Makefile linux-node +make[1]: Entrando no diretรณrio '/root/rel16' + +Lendo listas de pacotes... +Construindo รกrvore de dependรชncias... +Lendo informaรงรฃo de estado... +build-essential jรก รฉ a versรฃo mais nova (12.12). +cmake jรก รฉ a versรฃo mais nova (3.31.6-2). +git jรก รฉ a versรฃo mais nova (1:2.47.3-0+deb13u1). +pkg-config jรก รฉ a versรฃo mais nova (1.8.1-4). +autoconf jรก รฉ a versรฃo mais nova (2.72-3.1). +libtool jรก รฉ a versรฃo mais nova (2.5.4-4). +libssl-dev jรก รฉ a versรฃo mais nova (3.5.6-1~deb13u2). +libdb5.3++-dev jรก รฉ a versรฃo mais nova (5.3.28+dfsg2-9). +libsodium-dev jรก รฉ a versรฃo mais nova (1.0.18-1+deb13u1). +nlohmann-json3-dev jรก รฉ a versรฃo mais nova (3.11.3-2.1). +libboost-system-dev jรก รฉ a versรฃo mais nova (1.83.0.2+b2). +libglfw3-dev jรก รฉ a versรฃo mais nova (3.4-3). +libgl-dev jรก รฉ a versรฃo mais nova (1.7.0-1+b2). +python3-pil jรก รฉ a versรฃo mais nova (11.1.0-5+deb13u4). +clang jรก รฉ a versรฃo mais nova (1:19.0-63). +0 pacotes atualizados, 0 pacotes novos instalados, 0 a serem removidos e 49 nรฃo atualizados. +==> libsecp256k1 already installed +==> RandomX already built +make -C src -f Makefile bitflash-node -j1 +make[2]: Entrando no diretรณrio '/root/rel16/src' + +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/util.o util.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/script.o script.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/db.o db.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/net.o net.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/main.o main.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/market.o market.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -O3 -o obj-node/sha.o sha.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/nostr.o nostr.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/randomx_pow.o randomx_pow.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/btfaddr.o btfaddr.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/btfchan.o btfchan.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/btfrv.o btfrv.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/btftunnel.o btftunnel.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/rpc.o rpc.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/selftest.o selftest.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/bip32.o bip32.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/walletcmd.o walletcmd.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/main_gui.o main_gui.cpp +g++ -c -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -DBITFLASH_NO_GUI -o obj-node/headless.o headless.cpp +g++ -std=gnu++14 -O2 -Wformat -Werror=format -Wno-deprecated-declarations -Wno-write-strings -Wno-narrowing -Wno-unused-result -fpermissive -pthread -DSECP256K1_STATIC -I/usr/local/include -I/root/RandomX/src -I/usr/include/nlohmann -Iimgui -o bitflash-node obj-node/util.o obj-node/script.o obj-node/db.o obj-node/net.o obj-node/main.o obj-node/market.o obj-node/sha.o obj-node/nostr.o obj-node/randomx_pow.o obj-node/btfaddr.o obj-node/btfchan.o obj-node/btfrv.o obj-node/btftunnel.o obj-node/rpc.o obj-node/selftest.o obj-node/bip32.o obj-node/walletcmd.o obj-node/main_gui.o obj-node/headless.o -L/usr/local/lib -lsecp256k1 -L/root/RandomX/build -lrandomx -lsodium -lssl -lcrypto -ldb_cxx -lpthread +Built: bitflash-node (headless, no libGL needed) +make[2]: Saindo do diretรณrio '/root/rel16/src' + +strip src/bitflash-node +cp src/bitflash-node bitflash-node-1.2.16-x86_64 +Built: bitflash-node-1.2.16-x86_64 +make[1]: Saindo do diretรณrio '/root/rel16' + diff --git a/docs/site.md b/docs/site.md index 522da10..9526a61 100644 --- a/docs/site.md +++ b/docs/site.md @@ -42,8 +42,8 @@ The docs index should expose these source files: Each release remains under its versioned directory: ```text -https://releases.bitflash.network/v1.2.18/SHA256SUMS -https://releases.bitflash.network/v1.2.18/SHA256SUMS.asc +https://releases.bitflash.network/v1.2.19/SHA256SUMS +https://releases.bitflash.network/v1.2.19/SHA256SUMS.asc ``` The release root should also alias the latest files: diff --git a/scripts/__pycache__/bitflash_chain.cpython-313.pyc b/scripts/__pycache__/bitflash_chain.cpython-313.pyc new file mode 100644 index 0000000..47a7995 Binary files /dev/null and b/scripts/__pycache__/bitflash_chain.cpython-313.pyc differ -- cgit v1.2.3 From 4660b7ba94b071e8e7cc69c06e04b5ff64f12dd4 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Thu, 13 Aug 2026 17:21:01 -0300 Subject: docs: inventory Berkeley DB storage --- docs/storage.md | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 115 insertions(+) create mode 100644 docs/storage.md (limited to 'docs') diff --git a/docs/storage.md b/docs/storage.md new file mode 100644 index 0000000..4dd35bd --- /dev/null +++ b/docs/storage.md @@ -0,0 +1,115 @@ +# Bitflash Storage Inventory + +This document records the Berkeley DB surface that still exists in Bitflash. +It is a migration map, not a promise that all stores should move at once. + +The immediate priority is `wallet.dat`: it holds spend authority and recovery +metadata. `blkindex.dat` can be rebuilt from block files, so it is important for +startup correctness but less dangerous than wallet storage. + +## Berkeley DB Environment + +All `CDB` stores use one Berkeley DB environment rooted at: + +```text +/database/ +``` + +The environment creates log files such as: + +```text +database/log.0000000001 +``` + +Those logs are part of the legacy BDB risk model. They are not user-facing +wallet files, but they can contain historical records while the environment is +live. Wallet encryption already rewrites `wallet.dat` and purges BDB environment +logs after the encrypted file is installed. + +## Common Records + +Every `CDB` file can carry: + +| Key | Value | Purpose | +|---|---|---| +| `"version"` | `int` | Database format version written when a DB is opened in create mode. | + +Keys and values are serialized with the existing `CDataStream` format. A future +SQLite migration must preserve byte-for-byte value semantics until a higher +level schema migration is intentionally introduced. + +## `wallet.dat` + +`wallet.dat` is accessed through `CWalletDB`. + +| Key shape | Value | Secret? | Meaning | +|---|---|---:|---| +| `("name", address)` | `string` | No | Address book label. | +| `("tx", txid)` | `CWalletTx` | No | Wallet transaction state, including spend flags and metadata. | +| `("key", pubkey)` | `CPrivKey` | Yes | Plain private key record for unencrypted wallets or old backups. | +| `"defaultkey"` | `vector` | No | Default public key shown by the wallet. | +| `"hdmaster"` | `vector` | Yes | Plain BIP32 master private key material. | +| `"hdchaincode"` | `vector` | Yes | Plain BIP32 chain code. | +| `"hdnext"` | `unsigned int` | No | Legacy deterministic derivation next index. | +| `"hdschema"` | `int` | No | Deterministic derivation schema (`legacy` or `bip44`). | +| `"hdcointype"` | `unsigned int` | No | BIP44 coin type. Bitflash is `4346950`. | +| `"hdreceivenext"` | `unsigned int` | No | BIP44 external receive-chain next index. | +| `"hdchangenext"` | `unsigned int` | No | BIP44 internal change-chain next index. | +| `("mkey", id)` | `CWalletMasterKey` | Sensitive | Encrypted wallet master-key metadata. | +| `("ckey", pubkey)` | `vector` | Yes | Private key encrypted under the wallet master key. | +| `"cryptedhdmaster"` | `vector` | Yes | Encrypted BIP32 master private key material. | +| `"cryptedhdchaincode"` | `vector` | Yes | Encrypted BIP32 chain code. | +| `"walletminversion"` | `int` | No | Minimum wallet format version required to open this wallet safely. | +| `("pool", index)` | `vector` | No | Public key reserved in the key pool. The private key lives in `key`/`ckey`. | +| `("setting", name)` | typed setting value | Usually no | Wallet/UI settings such as mining mode, fees, and backup timestamps. | + +### Wallet Migration Rules + +A future wallet backend must keep these properties: + +1. Unknown future wallet formats fail closed with a clear message. +2. Encrypted wallets must encrypt private keys and HD seed material. +3. A locked encrypted wallet may expose public keys and balances, but not + private keys or decrypted HD seed material. +4. `-backupwallet` must produce a wallet file that opens in a different datadir + without copying `database/`. +5. `-recoveryaudit` must distinguish: + - no recovery phrase installed; + - recovery phrase present; + - recovery phrase encrypted and locked, requiring unlock before coverage can + be audited. + +## `blkindex.dat` + +`blkindex.dat` is accessed through `CTxDB`. + +| Key shape | Value | Rebuildable? | Meaning | +|---|---|---:|---| +| `("tx", txid)` | `CTxIndex` | Yes | Transaction disk position and spent-output index state. | +| `("owner", hash160, CDiskTxPos)` | `int height` | Yes | Legacy owner transaction index cursor range. | +| `("blockindex", blockhash)` | `CDiskBlockIndex` | Yes | Block index entry. | +| `"hashBestChain"` | `uint256` | Yes | Current best-chain tip hash. | + +`blkindex.dat` can be rebuilt from `blk*.dat` plus consensus validation. A +future migration may therefore prefer a rebuild path over a byte-preserving +conversion path. + +## Other Legacy Stores + +| File | Class | Records | Current risk | +|---|---|---|---| +| `reviews.dat` | `CReviewDB` | `("user", hash)`, `("reviews", hash)` | Legacy market/review storage. Not part of wallet safety. | +| `market.dat` | `CMarketDB` | No explicit typed records in current wrappers. | Legacy market storage shell. | + +The old IP address database (`addr.dat`) has been removed. Peer discovery is +handled by Nostr, `.btf`, rendezvous relays, and direct onion peers. + +## Recommended Migration Order + +1. Keep adding storage self-tests around current BDB behavior. +2. Add an experimental wallet backend behind an explicit flag. +3. Implement BDB-to-new-wallet migration as a one-way copy with a backup. +4. Make the new wallet backend default only after cross-version restore, + encryption, backup, and recovery-audit tests pass. +5. Treat `blkindex.dat` separately; it is safer to rebuild than to migrate under + pressure. -- cgit v1.2.3 From 95bd9cbcb1d17f4bc6223e6bfe3a92bfb8c3067d Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Fri, 14 Aug 2026 09:08:29 -0300 Subject: wallet: add read-only record scan boundary --- docs/storage.md | 3 +++ src/db.cpp | 39 ++++++++++++++++++++++++++++++++++ src/db.h | 11 ++++++++++ src/walletcmd.cpp | 63 ++++++++++++++++++++++--------------------------------- 4 files changed, 78 insertions(+), 38 deletions(-) (limited to 'docs') diff --git a/docs/storage.md b/docs/storage.md index 4dd35bd..e533589 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -41,6 +41,9 @@ level schema migration is intentionally introduced. ## `wallet.dat` `wallet.dat` is accessed through `CWalletDB`. +Read-only tooling should go through `ScanWalletRecords()` instead of opening a +Berkeley DB cursor directly. That keeps migration checks, audits, and future +backends behind one record-stream boundary. | Key shape | Value | Secret? | Meaning | |---|---|---:|---| diff --git a/src/db.cpp b/src/db.cpp index 9f62964..4fa72af 100644 --- a/src/db.cpp +++ b/src/db.cpp @@ -932,6 +932,45 @@ bool CWalletDB::LoadWallet(vector& vchDefaultKeyRet) return true; } +bool ScanWalletRecords(CWalletRecordVisitor& visitor, string& strErrorRet) +{ + class CWalletScanDB : public CWalletDB + { + public: + CWalletScanDB() : CWalletDB("r") { } + using CDB::GetCursor; + using CDB::ReadAtCursor; + }; + + strErrorRet.clear(); + CWalletScanDB dbWallet; + Dbc* pcursor = dbWallet.GetCursor(); + if (!pcursor) + { + strErrorRet = "could not open a wallet cursor"; + return false; + } + CAutoCursor cursorGuard(pcursor); + + loop + { + CDataStream ssKey(SER_DISK); + CDataStream ssValue(SER_DISK); + int ret = dbWallet.ReadAtCursor(pcursor, ssKey, ssValue); + if (ret == DB_NOTFOUND) + break; + if (ret != 0) + { + strErrorRet = strprintf("could not read wallet record: Berkeley DB error %d", ret); + return false; + } + if (!visitor.VisitWalletRecord(ssKey, ssValue, strErrorRet)) + return false; + } + + return true; +} + bool LoadWallet() { vector vchDefaultKey; diff --git a/src/db.h b/src/db.h index 4fb5cfd..2816957 100644 --- a/src/db.h +++ b/src/db.h @@ -524,6 +524,17 @@ public: bool LoadWallet(vector& vchDefaultKeyRet); }; +class CWalletRecordVisitor +{ +public: + virtual ~CWalletRecordVisitor() { } + virtual bool VisitWalletRecord(const CDataStream& ssKey, + const CDataStream& ssValue, + string& strErrorRet) = 0; +}; + +bool ScanWalletRecords(CWalletRecordVisitor& visitor, string& strErrorRet); + bool LoadWallet(); // Writes a wallet.dat that opens on its own, anywhere -- see the definition in diff --git a/src/walletcmd.cpp b/src/walletcmd.cpp index ce0da02..71f1ec3 100644 --- a/src/walletcmd.cpp +++ b/src/walletcmd.cpp @@ -553,51 +553,38 @@ static void CountWalletStorageType(const std::string& strType, static bool ReadWalletStorageCounts(WalletStorageAuditCounts& counts, std::string& strError) { - class CWalletAuditDB : public CWalletDB + class CWalletStorageCountVisitor : public CWalletRecordVisitor { public: - CWalletAuditDB() : CWalletDB("r") { } - using CDB::GetCursor; - using CDB::ReadAtCursor; - }; - - CWalletAuditDB walletdb; - Dbc* pcursor = walletdb.GetCursor(); - if (!pcursor) - { - strError = "cannot open wallet.dat cursor"; - return false; - } + WalletStorageAuditCounts& counts; - for (;;) - { - CDataStream ssKey(SER_DISK); - CDataStream ssValue(SER_DISK); - int ret = walletdb.ReadAtCursor(pcursor, ssKey, ssValue); - if (ret == DB_NOTFOUND) - break; - if (ret != 0) - { - pcursor->close(); - strError = strprintf("Berkeley DB cursor read failed (%d)", ret); - return false; - } + explicit CWalletStorageCountVisitor(WalletStorageAuditCounts& countsIn) + : counts(countsIn) { } - counts.nTotal++; - try + bool VisitWalletRecord(const CDataStream& ssKeyIn, + const CDataStream& ssValue, + std::string& strErrorRet) { - std::string strType; - ssKey >> strType; - CountWalletStorageType(strType, counts); - } - catch (...) - { - counts.nMalformed++; + (void)ssValue; + (void)strErrorRet; + counts.nTotal++; + try + { + CDataStream ssKey = ssKeyIn; + std::string strType; + ssKey >> strType; + CountWalletStorageType(strType, counts); + } + catch (...) + { + counts.nMalformed++; + } + return true; } - } + }; - pcursor->close(); - return true; + CWalletStorageCountVisitor visitor(counts); + return ScanWalletRecords(visitor, strError); } static std::string WalletStorageAuditJson(const WalletStorageAuditCounts& counts) -- cgit v1.2.3 From 5352185c7198a183c5906898e16bc4a6bb4e8654 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Fri, 14 Aug 2026 09:55:53 -0300 Subject: wallet: route rewrite copies through record scan --- docs/storage.md | 4 ++- src/db.cpp | 105 +++++++++++++++++++++++++++----------------------------- 2 files changed, 54 insertions(+), 55 deletions(-) (limited to 'docs') diff --git a/docs/storage.md b/docs/storage.md index e533589..e4a66d2 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -43,7 +43,9 @@ level schema migration is intentionally introduced. `wallet.dat` is accessed through `CWalletDB`. Read-only tooling should go through `ScanWalletRecords()` instead of opening a Berkeley DB cursor directly. That keeps migration checks, audits, and future -backends behind one record-stream boundary. +backends behind one record-stream boundary. Wallet rewrite code should also use +that boundary when it needs to copy existing records into a newly written +wallet file. | Key shape | Value | Secret? | Meaning | |---|---|---:|---| diff --git a/src/db.cpp b/src/db.cpp index 4fa72af..b90219e 100644 --- a/src/db.cpp +++ b/src/db.cpp @@ -1354,70 +1354,68 @@ public: { return Write(make_pair(string("pool"), nIndex), vchPubKey); } +}; + +static bool IsWalletEncryptionPrivateRecord(const string& strType) +{ + return strType == "key" || + strType == "pool" || + strType == "hdmaster" || + strType == "hdchaincode" || + strType == "mkey" || + strType == "ckey" || + strType == "cryptedhdmaster" || + strType == "cryptedhdchaincode" || + strType == "walletminversion"; +} - bool CopyPublicRecordsTo(CWalletRewriteDB& dbTo, int& nCopiedRet, string& strErrorRet) +class CWalletPublicRecordCopyVisitor : public CWalletRecordVisitor +{ +public: + CWalletRewriteDB& dbTo; + int& nCopied; + + CWalletPublicRecordCopyVisitor(CWalletRewriteDB& dbToIn, int& nCopiedIn) + : dbTo(dbToIn), nCopied(nCopiedIn) { } + + bool VisitWalletRecord(const CDataStream& ssKey, + const CDataStream& ssValue, + string& strErrorRet) { - nCopiedRet = 0; - Dbc* pcursor = GetCursor(); - if (!pcursor) + string strType; + try { - strErrorRet = "could not open a wallet cursor"; - return false; + CDataStream ssType = ssKey; + ssType >> strType; } - - unsigned int fFlags = DB_NEXT; - loop + catch (...) { - CDataStream ssKey; - CDataStream ssValue; - int ret = ReadAtCursor(pcursor, ssKey, ssValue, fFlags); - if (ret == DB_NOTFOUND) - break; - if (ret != 0) - { - pcursor->close(); - strErrorRet = strprintf("could not read wallet record: Berkeley DB error %d", ret); - return false; - } - - string strType; - try - { - CDataStream ssType = ssKey; - ssType >> strType; - } - catch (...) - { - pcursor->close(); - strErrorRet = "wallet.dat has a record whose key type cannot be read"; - return false; - } + strErrorRet = "wallet.dat has a record whose key type cannot be read"; + return false; + } - if (strType == "key" || - strType == "pool" || - strType == "hdmaster" || - strType == "hdchaincode" || - strType == "mkey" || - strType == "ckey" || - strType == "cryptedhdmaster" || - strType == "cryptedhdchaincode" || - strType == "walletminversion") - continue; + if (IsWalletEncryptionPrivateRecord(strType)) + return true; - if (!dbTo.WriteRaw(ssKey, ssValue)) - { - pcursor->close(); - strErrorRet = strprintf("could not copy wallet record '%s'", strType.c_str()); - return false; - } - nCopiedRet++; + if (!dbTo.WriteRaw(ssKey, ssValue)) + { + strErrorRet = strprintf("could not copy wallet record '%s'", strType.c_str()); + return false; } - - pcursor->close(); + nCopied++; return true; } }; +static bool CopyPublicWalletRecordsTo(CWalletRewriteDB& dbTo, + int& nCopiedRet, + string& strErrorRet) +{ + nCopiedRet = 0; + CWalletPublicRecordCopyVisitor visitor(dbTo, nCopiedRet); + return ScanWalletRecords(visitor, strErrorRet); +} + static string WalletEncryptBackupPath() { string strBase = GetAppDir() + "/wallet.dat.before-encrypt."; @@ -1594,9 +1592,8 @@ bool EncryptWallet(const string& strPassphrase, string& strBackupRet, string& st int nCopied = 0; { - CWalletRewriteDB dbSource("wallet.dat", "r"); CWalletRewriteDB dbTarget(strTempFile.c_str(), "c+"); - if (!dbSource.CopyPublicRecordsTo(dbTarget, nCopied, strErrorRet)) + if (!CopyPublicWalletRecordsTo(dbTarget, nCopied, strErrorRet)) { remove(strTempPath.c_str()); return false; -- cgit v1.2.3 From dd9db5ba5407b9c30b4f41fc1153db39f0770154 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Fri, 14 Aug 2026 11:08:34 -0300 Subject: wallet: install backups via temporary file --- docs/storage.md | 3 ++- src/db.cpp | 40 ++++++++++++++++++++++++++++++++++------ src/selftest.cpp | 7 +++++++ 3 files changed, 43 insertions(+), 7 deletions(-) (limited to 'docs') diff --git a/docs/storage.md b/docs/storage.md index e4a66d2..c1cda45 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -77,7 +77,8 @@ A future wallet backend must keep these properties: 3. A locked encrypted wallet may expose public keys and balances, but not private keys or decrypted HD seed material. 4. `-backupwallet` must produce a wallet file that opens in a different datadir - without copying `database/`. + without copying `database/`, and must not leave a half-written final backup + path if the copy or install step fails. 5. `-recoveryaudit` must distinguish: - no recovery phrase installed; - recovery phrase present; diff --git a/src/db.cpp b/src/db.cpp index b90219e..0ff58e7 100644 --- a/src/db.cpp +++ b/src/db.cpp @@ -1030,6 +1030,13 @@ bool BackupWallet(const string& strDest) if (!FileExists(strSrc.c_str())) return error("BackupWallet() : %s does not exist\n", strSrc.c_str()); + string strTemp = strprintf("%s.tmp.%lld", strDest.c_str(), GetTime()); + for (int i = 1; FileExists(strTemp.c_str()); i++) + strTemp = strprintf("%s.tmp.%lld.%d", strDest.c_str(), GetTime(), i); + string strOld = strprintf("%s.old.%lld", strDest.c_str(), GetTime()); + for (int i = 1; FileExists(strOld.c_str()); i++) + strOld = strprintf("%s.old.%lld.%d", strDest.c_str(), GetTime(), i); + CRITICAL_BLOCK(cs_db) { // Committing first, so the copy is not missing the newest records. @@ -1044,11 +1051,11 @@ bool BackupWallet(const string& strDest) FILE* pfIn = fopen(strSrc.c_str(), "rb"); if (!pfIn) return error("BackupWallet() : cannot read %s\n", strSrc.c_str()); - FILE* pfOut = fopen(strDest.c_str(), "wb"); + FILE* pfOut = fopen(strTemp.c_str(), "wb"); if (!pfOut) { fclose(pfIn); - return error("BackupWallet() : cannot write %s\n", strDest.c_str()); + return error("BackupWallet() : cannot write %s\n", strTemp.c_str()); } char buf[65536]; @@ -1065,18 +1072,39 @@ bool BackupWallet(const string& strDest) if (!fOk) { - remove(strDest.c_str()); // half a wallet is worse than none - return error("BackupWallet() : copy to %s failed\n", strDest.c_str()); + remove(strTemp.c_str()); // half a wallet is worse than none + return error("BackupWallet() : copy to %s failed\n", strTemp.c_str()); } // Without this the copy is welded to this node's database/ dir. - int ret = dbenv.lsn_reset(strDest.c_str(), 0); + int ret = dbenv.lsn_reset(strTemp.c_str(), 0); if (ret != 0) printf("BackupWallet() : warning -- lsn_reset returned %d; the copy " "may only open beside this node's database/ directory\n", ret); + + bool fHadOldDest = FileExists(strDest.c_str()); + if (fHadOldDest && rename(strDest.c_str(), strOld.c_str()) != 0) + { + remove(strTemp.c_str()); + return error("BackupWallet() : could not move old %s aside\n", strDest.c_str()); + } + if (rename(strTemp.c_str(), strDest.c_str()) != 0) + { + remove(strTemp.c_str()); + if (fHadOldDest) + rename(strOld.c_str(), strDest.c_str()); + return error("BackupWallet() : could not install %s\n", strDest.c_str()); + } + if (fHadOldDest) + remove(strOld.c_str()); } catch (const std::exception& e) - { return error("BackupWallet() : %s\n", e.what()); } + { + remove(strTemp.c_str()); + if (FileExists(strOld.c_str()) && !FileExists(strDest.c_str())) + rename(strOld.c_str(), strDest.c_str()); + return error("BackupWallet() : %s\n", e.what()); + } } // Recorded here rather than by whoever called, so that a backup taken with diff --git a/src/selftest.cpp b/src/selftest.cpp index 638aa66..81c4439 100644 --- a/src/selftest.cpp +++ b/src/selftest.cpp @@ -1391,6 +1391,13 @@ static int RunWalletPortabilitySelfTest() nFail += Check(nBackupRet == 0 && FileExists(strPortableWallet.c_str()), "/backupwallet writes a portable wallet.dat") ? 0 : 1; + string strBackupOut2 = tmp + "/backupwallet-overwrite.txt"; + int nBackupRet2 = RunBitflashChild(strExe, vBackupArgs, NULL, &strBackupOut2); + nFail += Check(nBackupRet2 == 0 && FileExists(strPortableWallet.c_str()) && + !DirectoryHasFileWithPrefix(tmp, "portable-wallet.dat.tmp.") && + !DirectoryHasFileWithPrefix(tmp, "portable-wallet.dat.old."), + "/backupwallet replaces an existing backup without leaving temp files") ? 0 : 1; + nFail += Check(CopyFileLocal(strOriginal + "/wallet.dat", strRawCopy + "/wallet.dat"), "a raw wallet.dat can be copied without database logs") ? 0 : 1; -- cgit v1.2.3 From c49b92a9a79a398b51200e69f7dde8a467b9c925 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Fri, 14 Aug 2026 15:03:21 -0300 Subject: wallet: add experimental SQLite record store --- .github/workflows/ci.yml | 1 + Makefile | 8 +- docs/storage.md | 19 ++++ src/Makefile | 10 +- src/main_gui.cpp | 3 +- src/makefile.mingw | 8 +- src/selftest.cpp | 108 +++++++++++++++++++- src/wallet_sqlite.cpp | 254 +++++++++++++++++++++++++++++++++++++++++++++++ src/wallet_sqlite.h | 40 ++++++++ 9 files changed, 439 insertions(+), 12 deletions(-) create mode 100644 src/wallet_sqlite.cpp create mode 100644 src/wallet_sqlite.h (limited to 'docs') diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f4045f4..e22b5a4 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -56,6 +56,7 @@ jobs: mingw-w64-ucrt-x86_64-glfw mingw-w64-ucrt-x86_64-openssl mingw-w64-ucrt-x86_64-db + mingw-w64-ucrt-x86_64-sqlite3 mingw-w64-ucrt-x86_64-libsodium mingw-w64-ucrt-x86_64-nlohmann-json mingw-w64-ucrt-x86_64-boost diff --git a/Makefile b/Makefile index 60d1227..fab2507 100644 --- a/Makefile +++ b/Makefile @@ -43,10 +43,11 @@ deps-apt: @if command -v apt-get >/dev/null 2>&1; then \ $(SUDO) apt-get install -y build-essential cmake git pkg-config autoconf \ libtool libssl-dev libdb5.3++-dev libsodium-dev nlohmann-json3-dev \ - libboost-system-dev libglfw3-dev libgl-dev python3-pil clang; \ + libboost-system-dev libglfw3-dev libgl-dev libsqlite3-dev \ + python3-pil clang; \ else \ echo "Not an apt system โ€” install: g++ cmake git autoconf libtool"; \ - echo " libssl libdb++ libsodium nlohmann-json boost glfw3 opengl python3-pil"; \ + echo " libssl libdb++ libsodium nlohmann-json boost glfw3 opengl sqlite3 python3-pil"; \ fi deps-secp256k1: @@ -113,7 +114,7 @@ img = ico.convert('RGBA'); \ img.save('/tmp/bitflash.AppDir/bitflash.png'); \ import shutil; shutil.copy('/tmp/bitflash.AppDir/bitflash.png', \ '/tmp/bitflash.AppDir/usr/share/icons/hicolor/256x256/apps/bitflash.png')" - @for lib in libsodium.so.23 libdb_cxx-5.3.so libglfw.so.3; do \ + @for lib in libsodium.so.23 libdb_cxx-5.3.so libsqlite3.so.0 libglfw.so.3; do \ path=$$(ldconfig -p | grep " $$lib " | awk '{print $$NF}' | head -1); \ [ -z "$$path" ] && path=$$(find /usr/lib /lib -name "$$lib" 2>/dev/null | head -1); \ [ -n "$$path" ] && cp -L "$$path" /tmp/bitflash.AppDir/usr/lib/ && echo " bundled $$lib" || echo " missing $$lib"; \ @@ -131,6 +132,7 @@ deps-windows: mingw-w64-ucrt-x86_64-glfw \ mingw-w64-ucrt-x86_64-openssl \ mingw-w64-ucrt-x86_64-db \ + mingw-w64-ucrt-x86_64-sqlite3 \ mingw-w64-ucrt-x86_64-libsodium \ mingw-w64-ucrt-x86_64-nlohmann-json \ mingw-w64-ucrt-x86_64-boost \ diff --git a/docs/storage.md b/docs/storage.md index c1cda45..f75f9b9 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -85,6 +85,25 @@ A future wallet backend must keep these properties: - recovery phrase encrypted and locked, requiring unlock before coverage can be audited. +## Experimental SQLite Wallet Store + +The SQLite wallet path starts as a byte-preserving record store, not as a new +wallet format: + +```sql +CREATE TABLE wallet_records ( + key BLOB PRIMARY KEY NOT NULL, + value BLOB NOT NULL +); +PRAGMA user_version=1; +``` + +Keys and values are the same serialized `CDataStream` bytes used by `wallet.dat`. +The first implementation is exercised only by `-selftest=wallet-sqlite`; the +runtime wallet still opens Berkeley DB. The purpose is to prove schema creation, +raw record write/read, close/reopen behavior, and value preservation before any +BDB-to-SQLite migration or runtime backend flag exists. + ## `blkindex.dat` `blkindex.dat` is accessed through `CTxDB`. diff --git a/src/Makefile b/src/Makefile index 19ad33a..cfa40f6 100644 --- a/src/Makefile +++ b/src/Makefile @@ -40,7 +40,7 @@ FUZZ_LDFLAGS = -fsanitize=fuzzer,address,undefined LIBS = -L$(SECP_LIB) -lsecp256k1 \ -L$(RANDOMX_DIR)/build -lrandomx \ - -lsodium -lssl -lcrypto -ldb_cxx -lpthread \ + -lsodium -lssl -lcrypto -ldb_cxx -lsqlite3 -lpthread \ -lglfw -lGL # Every header a .cpp here includes has to be listed, not just the interesting @@ -53,7 +53,7 @@ LIBS = -L$(SECP_LIB) -lsecp256k1 \ HEADERS = headers_core.h headers.h util.h main.h serialize.h uint256.h \ key.h bignum.h script.h db.h base58.h net.h nostr.h \ randomx_pow.h btfaddr.h btfchan.h btfrv.h btftunnel.h market.h \ - selftest.h walletcmd.h compat.h sockcount.h proxy.h tor.h font_roboto.h + selftest.h walletcmd.h wallet_sqlite.h compat.h sockcount.h proxy.h tor.h font_roboto.h IMGUI_OBJS = obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o \ obj/imgui_widgets.o obj/imgui_impl_glfw.o obj/imgui_impl_opengl3.o @@ -61,7 +61,8 @@ IMGUI_OBJS = obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o \ NODE_OBJS = obj/util.o obj/script.o obj/db.o obj/net.o obj/main.o \ obj/market.o obj/sha.o obj/nostr.o obj/randomx_pow.o \ obj/btfaddr.o obj/btfchan.o obj/btfrv.o obj/btftunnel.o obj/rpc.o \ - obj/selftest.o obj/bip32.o obj/walletcmd.o obj/proxy.o obj/tor.o + obj/selftest.o obj/bip32.o obj/walletcmd.o obj/wallet_sqlite.o \ + obj/proxy.o obj/tor.o GUI_OBJS = obj/gui.o obj/main_gui.o @@ -116,7 +117,7 @@ bitflash: $(IMGUI_OBJS) $(NODE_OBJS) $(GUI_OBJS) NODE_LIBS = -L$(SECP_LIB) -lsecp256k1 \ -L$(RANDOMX_DIR)/build -lrandomx \ - -lsodium -lssl -lcrypto -ldb_cxx -lpthread + -lsodium -lssl -lcrypto -ldb_cxx -lsqlite3 -lpthread NODE_ONLY_OBJS = $(patsubst obj/%,obj-node/%,$(NODE_OBJS)) \ obj-node/main_gui.o obj-node/headless.o @@ -187,6 +188,7 @@ tests: test_bip32 bitflash-node ./bitflash-node -selftest=wallet-keypool ./bitflash-node -selftest=wallet-hd ./bitflash-node -selftest=wallet-format + ./bitflash-node -selftest=wallet-sqlite ./bitflash-node -selftest=wallet-crypto ./bitflash-node -selftest=wallet-encrypt ./bitflash-node -selftest=net-message diff --git a/src/main_gui.cpp b/src/main_gui.cpp index 705b4ef..965c818 100644 --- a/src/main_gui.cpp +++ b/src/main_gui.cpp @@ -114,7 +114,8 @@ static void PrintUsage() printf(" /gen\n"); printf(" /nogui or /daemon\n"); printf(" /selftest=wallet-keypool, wallet-hd, wallet-format, wallet-storage-sanity,\n"); - printf(" db-env-reopen, wallet-crypto, wallet-encrypt, wallet-portability,\n"); + printf(" db-env-reopen, wallet-sqlite, wallet-crypto, wallet-encrypt,\n"); + printf(" wallet-portability,\n"); printf(" net-message,\n"); printf(" consensus-limits, pool-stratum,\n"); printf(" parse-money, socks5-proxy, or managed-tor\n"); diff --git a/src/makefile.mingw b/src/makefile.mingw index 333be96..2ff4e11 100644 --- a/src/makefile.mingw +++ b/src/makefile.mingw @@ -24,7 +24,7 @@ CXXFLAGS = -std=gnu++14 -O2 $(WARNFLAGS) -fpermissive -pthread \ LIBS = -L$(DEPS)/lib -lsecp256k1 \ -L$(RANDOMX)/build -lrandomx \ - -lsodium -lssl -lcrypto -ldb_cxx -lpthread \ + -lsodium -lssl -lcrypto -ldb_cxx -lsqlite3 -lpthread \ -LC:/msys64/ucrt64/lib \ -lglfw3 -lopengl32 -lgdi32 \ -lkernel32 -luser32 -lcomdlg32 -lwinspool -lwinmm \ @@ -37,7 +37,8 @@ IMGUI_OBJS = obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o \ NODE_OBJS = obj/util.o obj/script.o obj/db.o obj/net.o obj/main.o \ obj/market.o obj/sha.o obj/nostr.o obj/randomx_pow.o \ obj/btfaddr.o obj/btfchan.o obj/btfrv.o obj/btftunnel.o obj/rpc.o \ - obj/selftest.o obj/bip32.o obj/walletcmd.o obj/proxy.o obj/tor.o + obj/selftest.o obj/bip32.o obj/walletcmd.o obj/wallet_sqlite.o \ + obj/proxy.o obj/tor.o GUI_OBJS = obj/gui.o obj/main_gui.o obj/bitflash_res.o @@ -51,7 +52,7 @@ GUI_OBJS = obj/gui.o obj/main_gui.o obj/bitflash_res.o HEADERS = headers_core.h headers.h util.h main.h serialize.h uint256.h \ key.h bignum.h script.h db.h base58.h net.h nostr.h \ randomx_pow.h btfaddr.h btfchan.h btfrv.h btftunnel.h market.h \ - selftest.h walletcmd.h sockcount.h proxy.h tor.h compat.h font_roboto.h + selftest.h walletcmd.h wallet_sqlite.h sockcount.h proxy.h tor.h compat.h font_roboto.h all: bitflash.exe @@ -107,6 +108,7 @@ tests: test_bip32.exe bitflash.exe ./bitflash.exe -selftest=wallet-keypool ./bitflash.exe -selftest=wallet-hd ./bitflash.exe -selftest=wallet-format + ./bitflash.exe -selftest=wallet-sqlite ./bitflash.exe -selftest=wallet-crypto ./bitflash.exe -selftest=wallet-encrypt ./bitflash.exe -selftest=net-message diff --git a/src/selftest.cpp b/src/selftest.cpp index 31c168f..f9c4a8e 100644 --- a/src/selftest.cpp +++ b/src/selftest.cpp @@ -13,6 +13,7 @@ #include "sockcount.h" #include "tor.h" #include "walletcmd.h" +#include "wallet_sqlite.h" extern int RunPoolStratumSelfTest(); @@ -62,6 +63,11 @@ static std::string HexStrLocal(const std::vector& v) return out; } +static vector DataStreamBytes(const CDataStream& ss) +{ + return vector(ss.begin(), ss.end()); +} + static bool MakeTempDir(std::string& out) { #ifdef _WIN32 @@ -1030,6 +1036,104 @@ static int RunDbEnvReopenSelfTest() return nFail == 0 ? 0 : 1; } +static int RunWalletSQLiteSelfTest() +{ + fflush(stdout); + printf("wallet-sqlite self-test\n"); + + std::string tmp; + if (!MakeTempDir(tmp)) + { + printf(" FAIL could not create a temporary data directory\n"); + return 1; + } + + int nFail = 0; + string strPath = tmp + "/wallet.sqlite"; + string strError; + + try + { + CDataStream ssKeySetting(SER_DISK); + ssKeySetting << make_pair(string("setting"), string("sqlite-roundtrip")); + CDataStream ssValueSetting(SER_DISK); + ssValueSetting << (int64)424242; + + vector vchPubKey; + vchPubKey.push_back(0x04); + for (int i = 0; i < 64; i++) + vchPubKey.push_back((unsigned char)i); + CDataStream ssKeyDefault(SER_DISK); + ssKeyDefault << string("defaultkey"); + CDataStream ssValueDefault(SER_DISK); + ssValueDefault << vchPubKey; + + { + CWalletDBSQLite db; + nFail += Check(db.Open(strPath, strError), + "SQLite wallet opens and creates schema") ? 0 : 1; + nFail += Check(db.BeginTransaction(strError), + "SQLite wallet begins a write transaction") ? 0 : 1; + nFail += Check(db.WriteRecord(DataStreamBytes(ssKeySetting), + DataStreamBytes(ssValueSetting), + strError), + "SQLite wallet writes a serialized setting record") ? 0 : 1; + nFail += Check(db.WriteRecord(DataStreamBytes(ssKeyDefault), + DataStreamBytes(ssValueDefault), + strError), + "SQLite wallet writes a serialized defaultkey record") ? 0 : 1; + nFail += Check(db.CommitTransaction(strError), + "SQLite wallet commits a write transaction") ? 0 : 1; + int nCount = 0; + nFail += Check(db.CountRecords(nCount, strError) && nCount == 2, + "SQLite wallet counts raw records") ? 0 : 1; + } + + { + CWalletDBSQLite db; + nFail += Check(db.Open(strPath, strError), + "SQLite wallet reopens") ? 0 : 1; + vector vchRead; + nFail += Check(db.ReadRecord(DataStreamBytes(ssKeySetting), + vchRead, + strError), + "SQLite wallet reads a serialized setting record") ? 0 : 1; + int64 nReadValue = 0; + CDataStream ssRead(vchRead, SER_DISK); + ssRead >> nReadValue; + nFail += Check(!ssRead.fail() && nReadValue == 424242, + "SQLite wallet preserves serialized values byte-for-byte") ? 0 : 1; + + vchRead.clear(); + nFail += Check(db.ReadRecord(DataStreamBytes(ssKeyDefault), + vchRead, + strError), + "SQLite wallet reads a serialized defaultkey record") ? 0 : 1; + vector vchReadPubKey; + CDataStream ssReadDefault(vchRead, SER_DISK); + ssReadDefault >> vchReadPubKey; + nFail += Check(!ssReadDefault.fail() && vchReadPubKey == vchPubKey, + "SQLite wallet preserves public key blobs") ? 0 : 1; + } + } + catch (const std::exception& e) + { + printf(" FAIL exception: %s\n", e.what()); + nFail++; + } + catch (...) + { + printf(" FAIL unknown exception\n"); + nFail++; + } + + RemoveTree(tmp); + printf("%s (%d failure%s)\n", nFail == 0 ? "ALL TESTS PASSED" : "TESTS FAILED", + nFail, nFail == 1 ? "" : "s"); + fflush(stdout); + return nFail == 0 ? 0 : 1; +} + static void ClearWalletRuntimeForTest() { CRITICAL_BLOCK(cs_mapKeys) @@ -2289,6 +2393,8 @@ int RunSelfTest(const std::string& name) return RunWalletStorageSanitySelfTest(); if (name == "db-env-reopen") return RunDbEnvReopenSelfTest(); + if (name == "wallet-sqlite") + return RunWalletSQLiteSelfTest(); if (name == "wallet-crypto") return RunWalletCryptoSelfTest(); if (name == "wallet-encrypt") @@ -2309,6 +2415,6 @@ int RunSelfTest(const std::string& name) return RunManagedTorSelfTest(); printf("Unknown self-test '%s'\n", name.c_str()); - printf("Known self-tests: wallet-keypool, wallet-hd, wallet-format, wallet-storage-sanity, db-env-reopen, wallet-crypto, wallet-encrypt, wallet-portability, net-message, consensus-limits, pool-stratum, parse-money, socks5-proxy, managed-tor\n"); + printf("Known self-tests: wallet-keypool, wallet-hd, wallet-format, wallet-storage-sanity, db-env-reopen, wallet-sqlite, wallet-crypto, wallet-encrypt, wallet-portability, net-message, consensus-limits, pool-stratum, parse-money, socks5-proxy, managed-tor\n"); return 1; } diff --git a/src/wallet_sqlite.cpp b/src/wallet_sqlite.cpp new file mode 100644 index 0000000..4104ac6 --- /dev/null +++ b/src/wallet_sqlite.cpp @@ -0,0 +1,254 @@ +// Copyright (c) 2026 Bitflash developers +// Distributed under the MIT/X11 software license. + +#include "headers.h" +#include "wallet_sqlite.h" + +static void SetSQLiteError(sqlite3* pdb, const string& strContext, + string& strError) +{ + if (pdb) + strError = strContext + ": " + sqlite3_errmsg(pdb); + else + strError = strContext; +} + +CWalletDBSQLite::CWalletDBSQLite() : pdb(NULL) +{ +} + +CWalletDBSQLite::~CWalletDBSQLite() +{ + Close(); +} + +bool CWalletDBSQLite::Open(const string& strPath, string& strError) +{ + Close(); + + int ret = sqlite3_open_v2(strPath.c_str(), &pdb, + SQLITE_OPEN_READWRITE | + SQLITE_OPEN_CREATE | + SQLITE_OPEN_FULLMUTEX, + NULL); + if (ret != SQLITE_OK) + { + SetSQLiteError(pdb, "cannot open SQLite wallet", strError); + Close(); + return false; + } + + sqlite3_busy_timeout(pdb, 5000); + + if (!Exec("PRAGMA journal_mode=WAL;", strError)) + { + Close(); + return false; + } + if (!Exec("PRAGMA synchronous=FULL;", strError)) + { + Close(); + return false; + } + if (!InitSchema(strError)) + { + Close(); + return false; + } + return true; +} + +void CWalletDBSQLite::Close() +{ + if (pdb) + { + sqlite3_close(pdb); + pdb = NULL; + } +} + +bool CWalletDBSQLite::BeginTransaction(string& strError) +{ + if (!pdb) + { + strError = "SQLite wallet is not open"; + return false; + } + return Exec("BEGIN IMMEDIATE;", strError); +} + +bool CWalletDBSQLite::CommitTransaction(string& strError) +{ + if (!pdb) + { + strError = "SQLite wallet is not open"; + return false; + } + return Exec("COMMIT;", strError); +} + +bool CWalletDBSQLite::RollbackTransaction(string& strError) +{ + if (!pdb) + { + strError = "SQLite wallet is not open"; + return false; + } + return Exec("ROLLBACK;", strError); +} + +bool CWalletDBSQLite::Exec(const char* pszSql, string& strError) +{ + char* pszErr = NULL; + int ret = sqlite3_exec(pdb, pszSql, NULL, NULL, &pszErr); + if (ret != SQLITE_OK) + { + if (pszErr) + { + strError = pszErr; + sqlite3_free(pszErr); + } + else + SetSQLiteError(pdb, "SQLite exec failed", strError); + return false; + } + return true; +} + +bool CWalletDBSQLite::InitSchema(string& strError) +{ + return Exec("CREATE TABLE IF NOT EXISTS wallet_records (" + "key BLOB PRIMARY KEY NOT NULL," + "value BLOB NOT NULL" + ");", + strError) && + Exec("PRAGMA user_version=1;", strError); +} + +bool CWalletDBSQLite::WriteRecord(const vector& vchKey, + const vector& vchValue, + string& strError) +{ + if (!pdb) + { + strError = "SQLite wallet is not open"; + return false; + } + if (vchKey.empty()) + { + strError = "SQLite wallet record key is empty"; + return false; + } + + sqlite3_stmt* stmt = NULL; + const char* pszSql = + "INSERT OR REPLACE INTO wallet_records(key, value) VALUES(?, ?);"; + int ret = sqlite3_prepare_v2(pdb, pszSql, -1, &stmt, NULL); + if (ret != SQLITE_OK) + { + SetSQLiteError(pdb, "cannot prepare SQLite wallet write", strError); + return false; + } + + ret = sqlite3_bind_blob(stmt, 1, &vchKey[0], (int)vchKey.size(), + SQLITE_TRANSIENT); + if (ret == SQLITE_OK) + ret = sqlite3_bind_blob(stmt, 2, + vchValue.empty() ? NULL : &vchValue[0], + (int)vchValue.size(), + SQLITE_TRANSIENT); + if (ret == SQLITE_OK) + ret = sqlite3_step(stmt); + + bool fOk = (ret == SQLITE_DONE); + if (!fOk) + SetSQLiteError(pdb, "cannot write SQLite wallet record", strError); + sqlite3_finalize(stmt); + return fOk; +} + +bool CWalletDBSQLite::ReadRecord(const vector& vchKey, + vector& vchValueRet, + string& strError) +{ + vchValueRet.clear(); + if (!pdb) + { + strError = "SQLite wallet is not open"; + return false; + } + if (vchKey.empty()) + { + strError = "SQLite wallet record key is empty"; + return false; + } + + sqlite3_stmt* stmt = NULL; + const char* pszSql = "SELECT value FROM wallet_records WHERE key = ?;"; + int ret = sqlite3_prepare_v2(pdb, pszSql, -1, &stmt, NULL); + if (ret != SQLITE_OK) + { + SetSQLiteError(pdb, "cannot prepare SQLite wallet read", strError); + return false; + } + + ret = sqlite3_bind_blob(stmt, 1, &vchKey[0], (int)vchKey.size(), + SQLITE_TRANSIENT); + if (ret != SQLITE_OK) + { + SetSQLiteError(pdb, "cannot bind SQLite wallet key", strError); + sqlite3_finalize(stmt); + return false; + } + + ret = sqlite3_step(stmt); + if (ret == SQLITE_ROW) + { + const unsigned char* pch = + (const unsigned char*)sqlite3_column_blob(stmt, 0); + int nBytes = sqlite3_column_bytes(stmt, 0); + if (nBytes > 0 && pch) + vchValueRet.assign(pch, pch + nBytes); + sqlite3_finalize(stmt); + return true; + } + + if (ret == SQLITE_DONE) + strError = "SQLite wallet record not found"; + else + SetSQLiteError(pdb, "cannot read SQLite wallet record", strError); + sqlite3_finalize(stmt); + return false; +} + +bool CWalletDBSQLite::CountRecords(int& nRecordsRet, string& strError) +{ + nRecordsRet = 0; + if (!pdb) + { + strError = "SQLite wallet is not open"; + return false; + } + + sqlite3_stmt* stmt = NULL; + int ret = sqlite3_prepare_v2(pdb, + "SELECT COUNT(*) FROM wallet_records;", + -1, &stmt, NULL); + if (ret != SQLITE_OK) + { + SetSQLiteError(pdb, "cannot prepare SQLite wallet count", strError); + return false; + } + + ret = sqlite3_step(stmt); + if (ret != SQLITE_ROW) + { + SetSQLiteError(pdb, "cannot count SQLite wallet records", strError); + sqlite3_finalize(stmt); + return false; + } + + nRecordsRet = sqlite3_column_int(stmt, 0); + sqlite3_finalize(stmt); + return true; +} diff --git a/src/wallet_sqlite.h b/src/wallet_sqlite.h new file mode 100644 index 0000000..9812d3a --- /dev/null +++ b/src/wallet_sqlite.h @@ -0,0 +1,40 @@ +// Copyright (c) 2026 Bitflash developers +// Distributed under the MIT/X11 software license. + +#ifndef BITFLASH_WALLET_SQLITE_H +#define BITFLASH_WALLET_SQLITE_H + +#include + +class CWalletDBSQLite +{ +public: + CWalletDBSQLite(); + ~CWalletDBSQLite(); + + bool Open(const string& strPath, string& strError); + void Close(); + + bool BeginTransaction(string& strError); + bool CommitTransaction(string& strError); + bool RollbackTransaction(string& strError); + + bool WriteRecord(const vector& vchKey, + const vector& vchValue, + string& strError); + bool ReadRecord(const vector& vchKey, + vector& vchValueRet, + string& strError); + bool CountRecords(int& nRecordsRet, string& strError); + +private: + sqlite3* pdb; + + bool Exec(const char* pszSql, string& strError); + bool InitSchema(string& strError); + + CWalletDBSQLite(const CWalletDBSQLite&); + void operator=(const CWalletDBSQLite&); +}; + +#endif -- cgit v1.2.3 From 59b9d11015f0e3a2475d0750fc534583c2ca9f77 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Fri, 14 Aug 2026 15:56:12 -0300 Subject: wallet: export BDB records to SQLite --- docs/storage.md | 6 ++ src/Makefile | 1 + src/main_gui.cpp | 15 ++++- src/makefile.mingw | 1 + src/selftest.cpp | 148 +++++++++++++++++++++++++++++++++++++++++++++++++- src/wallet_sqlite.cpp | 10 ++++ src/wallet_sqlite.h | 1 + src/walletcmd.cpp | 97 +++++++++++++++++++++++++++++++++ src/walletcmd.h | 5 ++ 9 files changed, 281 insertions(+), 3 deletions(-) (limited to 'docs') diff --git a/docs/storage.md b/docs/storage.md index f75f9b9..bb414b5 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -104,6 +104,12 @@ runtime wallet still opens Berkeley DB. The purpose is to prove schema creation, raw record write/read, close/reopen behavior, and value preservation before any BDB-to-SQLite migration or runtime backend flag exists. +`-walletsqliteexport=FILE` is the next staging tool. It streams every raw +`wallet.dat` record through `ScanWalletRecords()` and writes those same key and +value bytes into the SQLite table inside one transaction. It refuses to +overwrite an existing export and still does not change the runtime wallet +backend. + ## `blkindex.dat` `blkindex.dat` is accessed through `CTxDB`. diff --git a/src/Makefile b/src/Makefile index cfa40f6..b83ff58 100644 --- a/src/Makefile +++ b/src/Makefile @@ -189,6 +189,7 @@ tests: test_bip32 bitflash-node ./bitflash-node -selftest=wallet-hd ./bitflash-node -selftest=wallet-format ./bitflash-node -selftest=wallet-sqlite + ./bitflash-node -selftest=wallet-sqlite-migration ./bitflash-node -selftest=wallet-crypto ./bitflash-node -selftest=wallet-encrypt ./bitflash-node -selftest=net-message diff --git a/src/main_gui.cpp b/src/main_gui.cpp index 965c818..6e89270 100644 --- a/src/main_gui.cpp +++ b/src/main_gui.cpp @@ -114,8 +114,8 @@ static void PrintUsage() printf(" /gen\n"); printf(" /nogui or /daemon\n"); printf(" /selftest=wallet-keypool, wallet-hd, wallet-format, wallet-storage-sanity,\n"); - printf(" db-env-reopen, wallet-sqlite, wallet-crypto, wallet-encrypt,\n"); - printf(" wallet-portability,\n"); + printf(" db-env-reopen, wallet-sqlite, wallet-sqlite-migration,\n"); + printf(" wallet-crypto, wallet-encrypt, wallet-portability,\n"); printf(" net-message,\n"); printf(" consensus-limits, pool-stratum,\n"); printf(" parse-money, socks5-proxy, or managed-tor\n"); @@ -181,6 +181,8 @@ static void PrintUsage() printf(" (write the same audit as deterministic JSON)\n"); printf(" /walletstoragecheck (fail closed on inconsistent or unsafe\n"); printf(" wallet.dat storage state)\n"); + printf(" /walletsqliteexport=FILE (copy raw wallet.dat records into an\n"); + printf(" experimental SQLite store, then exit)\n"); printf(" /rescan (walk the chain for coins this wallet owns\n"); printf(" but never recorded, then exit)\n"); printf("\n"); @@ -529,6 +531,15 @@ int main(int argc, char* argv[]) return nRet; } + string strWalletSQLiteExport = + argval2(argc, argv, "/walletsqliteexport", "-walletsqliteexport"); + if (!strWalletSQLiteExport.empty()) + { + int nRet = CmdWalletSQLiteExport(strWalletSQLiteExport); + DBFlush(true); + return nRet; + } + // Berkeley DB reports failure by throwing, and CDB's constructor lets it // through. Nothing on this path caught anything, so an unreadable // blkindex.dat or wallet.dat -- one written by another platform's Berkeley diff --git a/src/makefile.mingw b/src/makefile.mingw index 2ff4e11..14e6b32 100644 --- a/src/makefile.mingw +++ b/src/makefile.mingw @@ -109,6 +109,7 @@ tests: test_bip32.exe bitflash.exe ./bitflash.exe -selftest=wallet-hd ./bitflash.exe -selftest=wallet-format ./bitflash.exe -selftest=wallet-sqlite + ./bitflash.exe -selftest=wallet-sqlite-migration ./bitflash.exe -selftest=wallet-crypto ./bitflash.exe -selftest=wallet-encrypt ./bitflash.exe -selftest=net-message diff --git a/src/selftest.cpp b/src/selftest.cpp index f9c4a8e..7a54271 100644 --- a/src/selftest.cpp +++ b/src/selftest.cpp @@ -1134,6 +1134,150 @@ static int RunWalletSQLiteSelfTest() return nFail == 0 ? 0 : 1; } +class CWalletRecordMapVisitor : public CWalletRecordVisitor +{ +public: + std::map, vector >& mapRecords; + + explicit CWalletRecordMapVisitor( + std::map, vector >& mapRecordsIn) + : mapRecords(mapRecordsIn) { } + + bool VisitWalletRecord(const CDataStream& ssKey, + const CDataStream& ssValue, + string& strErrorRet) + { + vector vchKey = DataStreamBytes(ssKey); + if (mapRecords.count(vchKey)) + { + strErrorRet = "duplicate serialized wallet key"; + return false; + } + mapRecords[vchKey] = DataStreamBytes(ssValue); + return true; + } +}; + +static int RunWalletSQLiteMigrationSelfTest() +{ + fflush(stdout); + printf("wallet-sqlite-migration self-test\n"); + + std::string tmp; + std::string cwd; + if (!MakeTempDir(tmp)) + { + printf(" FAIL could not create a temporary data directory\n"); + return 1; + } + if (!GetCurrentDir(cwd)) + { + printf(" FAIL could not read current directory\n"); + RemoveTree(tmp); + return 1; + } + + int nFail = 0; + printf(" temp root: %s\n", tmp.c_str()); + + try + { +#ifdef _WIN32 + string strExe = cwd + "\\bitflash.exe"; +#else + string strExe = cwd + "/bitflash-node"; +#endif + string strWalletDir = tmp + "/wallet"; + nFail += Check(MakeDirLocal(strWalletDir), + "temporary wallet directory can be created") ? 0 : 1; + + string strCreateOut = tmp + "/create.txt"; + vector vCreateArgs; + vCreateArgs.push_back("-datadir=" + strWalletDir); + vCreateArgs.push_back("-nomanagedtor"); + vCreateArgs.push_back("-nogui"); + vCreateArgs.push_back("-newphrase"); + int nCreateRet = RunBitflashChild(strExe, vCreateArgs, NULL, &strCreateOut); + nFail += Check(nCreateRet == 0 && + FileContainsText(strCreateOut, "Write these twelve words down"), + "a phrase wallet can be created for SQLite export") ? 0 : 1; + + string strSQLite = tmp + "/wallet.sqlite"; + string strExportOut = tmp + "/sqlite-export.txt"; + vector vExportArgs; + vExportArgs.push_back("-datadir=" + strWalletDir); + vExportArgs.push_back("-nomanagedtor"); + vExportArgs.push_back("-nogui"); + vExportArgs.push_back("-walletsqliteexport=" + strSQLite); + int nExportRet = RunBitflashChild(strExe, vExportArgs, NULL, &strExportOut); + nFail += Check(nExportRet == 0 && + FileExists(strSQLite.c_str()) && + FileContainsText(strExportOut, "SQLite wallet export written"), + "wallet.dat can be exported to SQLite") ? 0 : 1; + + string strOverwriteOut = tmp + "/sqlite-export-overwrite.txt"; + int nOverwriteRet = + RunBitflashChild(strExe, vExportArgs, NULL, &strOverwriteOut); + nFail += Check(nOverwriteRet != 0 && + FileContainsText(strOverwriteOut, + "Refusing to overwrite existing SQLite wallet export"), + "SQLite wallet export refuses to overwrite") ? 0 : 1; + remove((strSQLite + "-wal").c_str()); + remove((strSQLite + "-shm").c_str()); + + std::map, vector > mapBDB; + string strOldDataDir = strSetDataDir; + string strScanError; + strSetDataDir = strWalletDir; + CWalletRecordMapVisitor visitor(mapBDB); + bool fScanOk = ScanWalletRecords(visitor, strScanError); + DBFlush(true); + strSetDataDir = strOldDataDir; + nFail += Check(fScanOk && !mapBDB.empty(), + "wallet.dat records can be scanned for comparison") ? 0 : 1; + + CWalletDBSQLite db; + string strError; + nFail += Check(db.Open(strSQLite, strError), + "exported SQLite wallet opens") ? 0 : 1; + int nSQLiteRecords = 0; + nFail += Check(db.CountRecords(nSQLiteRecords, strError) && + nSQLiteRecords == (int)mapBDB.size(), + "SQLite export has the same record count as wallet.dat") ? 0 : 1; + + bool fAllMatch = true; + for (std::map, vector >::const_iterator it = + mapBDB.begin(); it != mapBDB.end(); ++it) + { + vector vchSQLiteValue; + if (!db.ReadRecord(it->first, vchSQLiteValue, strError) || + vchSQLiteValue != it->second) + { + fAllMatch = false; + break; + } + } + nFail += Check(fAllMatch, + "SQLite export preserves every wallet.dat key/value byte-for-byte") ? 0 : 1; + } + catch (const std::exception& e) + { + printf(" FAIL exception: %s\n", e.what()); + nFail++; + } + catch (...) + { + printf(" FAIL unknown exception\n"); + nFail++; + } + + RemoveTree(tmp); + printf("%s (%d failure%s)\n", nFail == 0 ? "ALL TESTS PASSED" : "TESTS FAILED", + nFail, nFail == 1 ? "" : "s"); + fflush(stdout); + return nFail == 0 ? 0 : 1; +} + static void ClearWalletRuntimeForTest() { CRITICAL_BLOCK(cs_mapKeys) @@ -2395,6 +2539,8 @@ int RunSelfTest(const std::string& name) return RunDbEnvReopenSelfTest(); if (name == "wallet-sqlite") return RunWalletSQLiteSelfTest(); + if (name == "wallet-sqlite-migration") + return RunWalletSQLiteMigrationSelfTest(); if (name == "wallet-crypto") return RunWalletCryptoSelfTest(); if (name == "wallet-encrypt") @@ -2415,6 +2561,6 @@ int RunSelfTest(const std::string& name) return RunManagedTorSelfTest(); printf("Unknown self-test '%s'\n", name.c_str()); - printf("Known self-tests: wallet-keypool, wallet-hd, wallet-format, wallet-storage-sanity, db-env-reopen, wallet-sqlite, wallet-crypto, wallet-encrypt, wallet-portability, net-message, consensus-limits, pool-stratum, parse-money, socks5-proxy, managed-tor\n"); + printf("Known self-tests: wallet-keypool, wallet-hd, wallet-format, wallet-storage-sanity, db-env-reopen, wallet-sqlite, wallet-sqlite-migration, wallet-crypto, wallet-encrypt, wallet-portability, net-message, consensus-limits, pool-stratum, parse-money, socks5-proxy, managed-tor\n"); return 1; } diff --git a/src/wallet_sqlite.cpp b/src/wallet_sqlite.cpp index 4104ac6..4369af6 100644 --- a/src/wallet_sqlite.cpp +++ b/src/wallet_sqlite.cpp @@ -97,6 +97,16 @@ bool CWalletDBSQLite::RollbackTransaction(string& strError) return Exec("ROLLBACK;", strError); } +bool CWalletDBSQLite::Checkpoint(string& strError) +{ + if (!pdb) + { + strError = "SQLite wallet is not open"; + return false; + } + return Exec("PRAGMA wal_checkpoint(TRUNCATE);", strError); +} + bool CWalletDBSQLite::Exec(const char* pszSql, string& strError) { char* pszErr = NULL; diff --git a/src/wallet_sqlite.h b/src/wallet_sqlite.h index 9812d3a..5ef862d 100644 --- a/src/wallet_sqlite.h +++ b/src/wallet_sqlite.h @@ -18,6 +18,7 @@ public: bool BeginTransaction(string& strError); bool CommitTransaction(string& strError); bool RollbackTransaction(string& strError); + bool Checkpoint(string& strError); bool WriteRecord(const vector& vchKey, const vector& vchValue, diff --git a/src/walletcmd.cpp b/src/walletcmd.cpp index 6fb09a3..20e7e33 100644 --- a/src/walletcmd.cpp +++ b/src/walletcmd.cpp @@ -12,6 +12,7 @@ #include "headers_core.h" #include "bip32.h" #include "walletcmd.h" +#include "wallet_sqlite.h" #undef printf @@ -790,6 +791,102 @@ int CmdWalletStorageAudit(const std::string& strJsonOut) return 0; } +static vector DataStreamBytesLocal(const CDataStream& ss) +{ + return vector(ss.begin(), ss.end()); +} + +class CWalletSQLiteExportVisitor : public CWalletRecordVisitor +{ +public: + CWalletDBSQLite& db; + int& nCopied; + + CWalletSQLiteExportVisitor(CWalletDBSQLite& dbIn, int& nCopiedIn) + : db(dbIn), nCopied(nCopiedIn) { } + + bool VisitWalletRecord(const CDataStream& ssKey, + const CDataStream& ssValue, + string& strErrorRet) + { + if (!db.WriteRecord(DataStreamBytesLocal(ssKey), + DataStreamBytesLocal(ssValue), + strErrorRet)) + return false; + nCopied++; + return true; + } +}; + +int CmdWalletSQLiteExport(const std::string& strDest) +{ + AttachTerminal(); + + if (strDest.empty()) + { + fprintf(stderr, "Missing SQLite wallet export path.\n"); + return 1; + } + if (FileExists(strDest.c_str())) + { + fprintf(stderr, "Refusing to overwrite existing SQLite wallet export: %s\n", + strDest.c_str()); + return 1; + } + + string strError; + CWalletDBSQLite db; + if (!db.Open(strDest, strError)) + { + fprintf(stderr, "Cannot open SQLite wallet export: %s\n", strError.c_str()); + return 1; + } + if (!db.BeginTransaction(strError)) + { + fprintf(stderr, "Cannot begin SQLite wallet export: %s\n", strError.c_str()); + return 1; + } + + int nCopied = 0; + CWalletSQLiteExportVisitor visitor(db, nCopied); + if (!ScanWalletRecords(visitor, strError)) + { + string strRollbackError; + db.RollbackTransaction(strRollbackError); + fprintf(stderr, "Cannot export wallet.dat to SQLite: %s\n", strError.c_str()); + return 1; + } + if (!db.CommitTransaction(strError)) + { + fprintf(stderr, "Cannot commit SQLite wallet export: %s\n", strError.c_str()); + return 1; + } + if (!db.Checkpoint(strError)) + { + fprintf(stderr, "Cannot checkpoint SQLite wallet export: %s\n", strError.c_str()); + return 1; + } + + int nCount = 0; + if (!db.CountRecords(nCount, strError)) + { + fprintf(stderr, "Cannot verify SQLite wallet export count: %s\n", + strError.c_str()); + return 1; + } + if (nCount != nCopied) + { + fprintf(stderr, "SQLite wallet export count mismatch: copied %d, stored %d\n", + nCopied, nCount); + return 1; + } + + printf("SQLite wallet export written to %s\n", strDest.c_str()); + printf(" records copied: %d\n", nCopied); + fflush(stdout); + return 0; +} + int CmdEncryptWallet(const std::string& strPassphrase) { AttachTerminal(); diff --git a/src/walletcmd.h b/src/walletcmd.h index 6077669..b1a29d2 100644 --- a/src/walletcmd.h +++ b/src/walletcmd.h @@ -70,6 +70,11 @@ int CmdWalletStorageAudit(const std::string& strJsonOut = ""); // or migration-unsafe mix of storage records. int CmdWalletStorageCheck(); +// Diagnostic/migration staging: copy every raw wallet.dat record into an +// experimental SQLite key/value store, preserving serialized bytes. This does +// not change which backend the runtime wallet opens. +int CmdWalletSQLiteExport(const std::string& strDest); + // Rewrite wallet.dat so private keys and the HD seed are encrypted with the // given passphrase. Returns 0 on success and exits without starting the node. int CmdEncryptWallet(const std::string& strPassphrase); -- cgit v1.2.3 From df84b82162b38dcd57114510b9651af5d85f04cb Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Fri, 14 Aug 2026 16:20:41 -0300 Subject: wallet: scan SQLite wallet records --- docs/storage.md | 4 +++- src/selftest.cpp | 6 ++++++ src/wallet_sqlite.cpp | 54 +++++++++++++++++++++++++++++++++++++++++++++++++++ src/wallet_sqlite.h | 3 +++ 4 files changed, 66 insertions(+), 1 deletion(-) (limited to 'docs') diff --git a/docs/storage.md b/docs/storage.md index bb414b5..2d748fa 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -108,7 +108,9 @@ BDB-to-SQLite migration or runtime backend flag exists. `wallet.dat` record through `ScanWalletRecords()` and writes those same key and value bytes into the SQLite table inside one transaction. It refuses to overwrite an existing export and still does not change the runtime wallet -backend. +backend. `CWalletDBSQLite::ScanRecords()` can stream those exported records back +through the same visitor shape, so tests can compare BDB and SQLite record +streams without introducing a SQLite runtime wallet loader yet. ## `blkindex.dat` diff --git a/src/selftest.cpp b/src/selftest.cpp index 7a54271..408dd41 100644 --- a/src/selftest.cpp +++ b/src/selftest.cpp @@ -1259,6 +1259,12 @@ static int RunWalletSQLiteMigrationSelfTest() } nFail += Check(fAllMatch, "SQLite export preserves every wallet.dat key/value byte-for-byte") ? 0 : 1; + + std::map, vector > mapSQLite; + CWalletRecordMapVisitor sqliteVisitor(mapSQLite); + nFail += Check(db.ScanRecords(sqliteVisitor, strError) && + mapSQLite == mapBDB, + "SQLite wallet scanner streams every exported record byte-for-byte") ? 0 : 1; } catch (const std::exception& e) { diff --git a/src/wallet_sqlite.cpp b/src/wallet_sqlite.cpp index 4369af6..8489cac 100644 --- a/src/wallet_sqlite.cpp +++ b/src/wallet_sqlite.cpp @@ -231,6 +231,60 @@ bool CWalletDBSQLite::ReadRecord(const vector& vchKey, return false; } +bool CWalletDBSQLite::ScanRecords(CWalletRecordVisitor& visitor, + string& strError) +{ + if (!pdb) + { + strError = "SQLite wallet is not open"; + return false; + } + + sqlite3_stmt* stmt = NULL; + const char* pszSql = "SELECT key, value FROM wallet_records ORDER BY rowid;"; + int ret = sqlite3_prepare_v2(pdb, pszSql, -1, &stmt, NULL); + if (ret != SQLITE_OK) + { + SetSQLiteError(pdb, "cannot prepare SQLite wallet scan", strError); + return false; + } + + while ((ret = sqlite3_step(stmt)) == SQLITE_ROW) + { + const unsigned char* pchKey = + (const unsigned char*)sqlite3_column_blob(stmt, 0); + int nKeyBytes = sqlite3_column_bytes(stmt, 0); + const unsigned char* pchValue = + (const unsigned char*)sqlite3_column_blob(stmt, 1); + int nValueBytes = sqlite3_column_bytes(stmt, 1); + + vector vchKey; + vector vchValue; + if (nKeyBytes > 0 && pchKey) + vchKey.assign(pchKey, pchKey + nKeyBytes); + if (nValueBytes > 0 && pchValue) + vchValue.assign(pchValue, pchValue + nValueBytes); + + CDataStream ssKey(vchKey, SER_DISK); + CDataStream ssValue(vchValue, SER_DISK); + if (!visitor.VisitWalletRecord(ssKey, ssValue, strError)) + { + sqlite3_finalize(stmt); + return false; + } + } + + if (ret != SQLITE_DONE) + { + SetSQLiteError(pdb, "cannot scan SQLite wallet records", strError); + sqlite3_finalize(stmt); + return false; + } + + sqlite3_finalize(stmt); + return true; +} + bool CWalletDBSQLite::CountRecords(int& nRecordsRet, string& strError) { nRecordsRet = 0; diff --git a/src/wallet_sqlite.h b/src/wallet_sqlite.h index 5ef862d..385a340 100644 --- a/src/wallet_sqlite.h +++ b/src/wallet_sqlite.h @@ -6,6 +6,8 @@ #include +class CWalletRecordVisitor; + class CWalletDBSQLite { public: @@ -26,6 +28,7 @@ public: bool ReadRecord(const vector& vchKey, vector& vchValueRet, string& strError); + bool ScanRecords(CWalletRecordVisitor& visitor, string& strError); bool CountRecords(int& nRecordsRet, string& strError); private: -- cgit v1.2.3 From 99c6b5f2971da51cca1c739b5e982ae2022acc85 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Fri, 14 Aug 2026 16:34:43 -0300 Subject: wallet: verify SQLite wallet exports --- docs/storage.md | 2 + src/main_gui.cpp | 11 ++++++ src/selftest.cpp | 41 +++++++++++++++++++++ src/walletcmd.cpp | 107 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/walletcmd.h | 4 ++ 5 files changed, 165 insertions(+) (limited to 'docs') diff --git a/docs/storage.md b/docs/storage.md index 2d748fa..0a441bc 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -111,6 +111,8 @@ overwrite an existing export and still does not change the runtime wallet backend. `CWalletDBSQLite::ScanRecords()` can stream those exported records back through the same visitor shape, so tests can compare BDB and SQLite record streams without introducing a SQLite runtime wallet loader yet. +`-walletsqliteverify=FILE` compares the current `wallet.dat` stream with a +SQLite export and prints only record counts and mismatch counts. ## `blkindex.dat` diff --git a/src/main_gui.cpp b/src/main_gui.cpp index 6e89270..caacf13 100644 --- a/src/main_gui.cpp +++ b/src/main_gui.cpp @@ -183,6 +183,8 @@ static void PrintUsage() printf(" wallet.dat storage state)\n"); printf(" /walletsqliteexport=FILE (copy raw wallet.dat records into an\n"); printf(" experimental SQLite store, then exit)\n"); + printf(" /walletsqliteverify=FILE (compare wallet.dat with a SQLite export,\n"); + printf(" printing counts only, then exit)\n"); printf(" /rescan (walk the chain for coins this wallet owns\n"); printf(" but never recorded, then exit)\n"); printf("\n"); @@ -540,6 +542,15 @@ int main(int argc, char* argv[]) return nRet; } + string strWalletSQLiteVerify = + argval2(argc, argv, "/walletsqliteverify", "-walletsqliteverify"); + if (!strWalletSQLiteVerify.empty()) + { + int nRet = CmdWalletSQLiteVerify(strWalletSQLiteVerify); + DBFlush(true); + return nRet; + } + // Berkeley DB reports failure by throwing, and CDB's constructor lets it // through. Nothing on this path caught anything, so an unreadable // blkindex.dat or wallet.dat -- one written by another platform's Berkeley diff --git a/src/selftest.cpp b/src/selftest.cpp index 408dd41..4ba90c3 100644 --- a/src/selftest.cpp +++ b/src/selftest.cpp @@ -1134,6 +1134,8 @@ static int RunWalletSQLiteSelfTest() return nFail == 0 ? 0 : 1; } +namespace { + class CWalletRecordMapVisitor : public CWalletRecordVisitor { public: @@ -1158,6 +1160,8 @@ public: } }; +} // namespace + static int RunWalletSQLiteMigrationSelfTest() { fflush(stdout); @@ -1215,6 +1219,17 @@ static int RunWalletSQLiteMigrationSelfTest() FileContainsText(strExportOut, "SQLite wallet export written"), "wallet.dat can be exported to SQLite") ? 0 : 1; + string strVerifyOut = tmp + "/sqlite-verify.txt"; + vector vVerifyArgs; + vVerifyArgs.push_back("-datadir=" + strWalletDir); + vVerifyArgs.push_back("-nomanagedtor"); + vVerifyArgs.push_back("-nogui"); + vVerifyArgs.push_back("-walletsqliteverify=" + strSQLite); + int nVerifyRet = RunBitflashChild(strExe, vVerifyArgs, NULL, &strVerifyOut); + nFail += Check(nVerifyRet == 0 && + FileContainsText(strVerifyOut, "verification: ok"), + "SQLite wallet export verifies against wallet.dat") ? 0 : 1; + string strOverwriteOut = tmp + "/sqlite-export-overwrite.txt"; int nOverwriteRet = RunBitflashChild(strExe, vExportArgs, NULL, &strOverwriteOut); @@ -1265,6 +1280,32 @@ static int RunWalletSQLiteMigrationSelfTest() nFail += Check(db.ScanRecords(sqliteVisitor, strError) && mapSQLite == mapBDB, "SQLite wallet scanner streams every exported record byte-for-byte") ? 0 : 1; + + db.Close(); + CWalletDBSQLite dbCorrupt; + vector vchBadValue; + vchBadValue.push_back(0xba); + vchBadValue.push_back(0xad); + nFail += Check(!mapBDB.empty() && + dbCorrupt.Open(strSQLite, strError) && + dbCorrupt.BeginTransaction(strError) && + dbCorrupt.WriteRecord(mapBDB.begin()->first, + vchBadValue, + strError) && + dbCorrupt.CommitTransaction(strError) && + dbCorrupt.Checkpoint(strError), + "SQLite export can be deliberately corrupted for verifier test") ? 0 : 1; + dbCorrupt.Close(); + + string strVerifyBadOut = tmp + "/sqlite-verify-bad.txt"; + int nVerifyBadRet = + RunBitflashChild(strExe, vVerifyArgs, NULL, &strVerifyBadOut); + nFail += Check(nVerifyBadRet == 2 && + FileContainsText(strVerifyBadOut, + "record value mismatches: 1") && + FileContainsText(strVerifyBadOut, + "verification: failed"), + "SQLite wallet verifier detects a mismatched record value") ? 0 : 1; } catch (const std::exception& e) { diff --git a/src/walletcmd.cpp b/src/walletcmd.cpp index 20e7e33..36f6783 100644 --- a/src/walletcmd.cpp +++ b/src/walletcmd.cpp @@ -796,6 +796,8 @@ static vector DataStreamBytesLocal(const CDataStream& ss) return vector(ss.begin(), ss.end()); } +namespace { + class CWalletSQLiteExportVisitor : public CWalletRecordVisitor { public: @@ -818,6 +820,32 @@ public: } }; +class CWalletRecordMapVisitor : public CWalletRecordVisitor +{ +public: + std::map, vector >& mapRecords; + + explicit CWalletRecordMapVisitor( + std::map, vector >& mapRecordsIn) + : mapRecords(mapRecordsIn) { } + + bool VisitWalletRecord(const CDataStream& ssKey, + const CDataStream& ssValue, + string& strErrorRet) + { + vector vchKey = DataStreamBytesLocal(ssKey); + if (mapRecords.count(vchKey)) + { + strErrorRet = "duplicate serialized wallet key"; + return false; + } + mapRecords[vchKey] = DataStreamBytesLocal(ssValue); + return true; + } +}; + +} // namespace + int CmdWalletSQLiteExport(const std::string& strDest) { AttachTerminal(); @@ -887,6 +915,85 @@ int CmdWalletSQLiteExport(const std::string& strDest) return 0; } +int CmdWalletSQLiteVerify(const std::string& strPath) +{ + AttachTerminal(); + + if (strPath.empty()) + { + fprintf(stderr, "Missing SQLite wallet export path.\n"); + return 1; + } + if (!FileExists(strPath.c_str())) + { + fprintf(stderr, "SQLite wallet export does not exist: %s\n", + strPath.c_str()); + return 1; + } + + std::map, vector > mapBDB; + std::map, vector > mapSQLite; + string strError; + + CWalletRecordMapVisitor bdbVisitor(mapBDB); + if (!ScanWalletRecords(bdbVisitor, strError)) + { + fprintf(stderr, "Cannot scan wallet.dat records: %s\n", strError.c_str()); + return 1; + } + + CWalletDBSQLite db; + if (!db.Open(strPath, strError)) + { + fprintf(stderr, "Cannot open SQLite wallet export: %s\n", strError.c_str()); + return 1; + } + CWalletRecordMapVisitor sqliteVisitor(mapSQLite); + if (!db.ScanRecords(sqliteVisitor, strError)) + { + fprintf(stderr, "Cannot scan SQLite wallet export: %s\n", strError.c_str()); + return 1; + } + + int nMissing = 0; + int nMismatched = 0; + for (std::map, vector >::const_iterator it = + mapBDB.begin(); it != mapBDB.end(); ++it) + { + std::map, vector >::const_iterator sit = + mapSQLite.find(it->first); + if (sit == mapSQLite.end()) + nMissing++; + else if (sit->second != it->second) + nMismatched++; + } + + int nExtra = 0; + for (std::map, vector >::const_iterator it = + mapSQLite.begin(); it != mapSQLite.end(); ++it) + { + if (!mapBDB.count(it->first)) + nExtra++; + } + + printf("SQLite wallet export verification\n"); + printf(" wallet.dat records: %u\n", (unsigned int)mapBDB.size()); + printf(" SQLite records: %u\n", (unsigned int)mapSQLite.size()); + printf(" missing records: %d\n", nMissing); + printf(" extra records: %d\n", nExtra); + printf(" record value mismatches: %d\n", nMismatched); + if (nMissing == 0 && nExtra == 0 && nMismatched == 0) + { + printf(" verification: ok\n"); + fflush(stdout); + return 0; + } + + printf(" verification: failed\n"); + fflush(stdout); + return 2; +} + int CmdEncryptWallet(const std::string& strPassphrase) { AttachTerminal(); diff --git a/src/walletcmd.h b/src/walletcmd.h index b1a29d2..784c74c 100644 --- a/src/walletcmd.h +++ b/src/walletcmd.h @@ -75,6 +75,10 @@ int CmdWalletStorageCheck(); // not change which backend the runtime wallet opens. int CmdWalletSQLiteExport(const std::string& strDest); +// Diagnostic/migration staging: compare a SQLite wallet export with the current +// wallet.dat raw records. Prints counts only, never record keys or values. +int CmdWalletSQLiteVerify(const std::string& strPath); + // Rewrite wallet.dat so private keys and the HD seed are encrypted with the // given passphrase. Returns 0 on success and exits without starting the node. int CmdEncryptWallet(const std::string& strPassphrase); -- cgit v1.2.3 From df129cc672b9b894ff2558d3e9361ddab082433d Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Fri, 14 Aug 2026 17:02:35 -0300 Subject: wallet: restore BDB wallets from SQLite exports --- docs/storage.md | 3 ++ src/main_gui.cpp | 11 +++++ src/selftest.cpp | 35 +++++++++++++++ src/walletcmd.cpp | 126 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/walletcmd.h | 4 ++ 5 files changed, 179 insertions(+) (limited to 'docs') diff --git a/docs/storage.md b/docs/storage.md index 0a441bc..4947575 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -113,6 +113,9 @@ through the same visitor shape, so tests can compare BDB and SQLite record streams without introducing a SQLite runtime wallet loader yet. `-walletsqliteverify=FILE` compares the current `wallet.dat` stream with a SQLite export and prints only record counts and mismatch counts. +`-walletsqliterestore=FILE` rebuilds `wallet.dat` from a SQLite export only in +an empty data directory, then the verifier can prove the restored BDB file still +matches the export byte-for-byte. ## `blkindex.dat` diff --git a/src/main_gui.cpp b/src/main_gui.cpp index caacf13..cf96822 100644 --- a/src/main_gui.cpp +++ b/src/main_gui.cpp @@ -185,6 +185,8 @@ static void PrintUsage() printf(" experimental SQLite store, then exit)\n"); printf(" /walletsqliteverify=FILE (compare wallet.dat with a SQLite export,\n"); printf(" printing counts only, then exit)\n"); + printf(" /walletsqliterestore=FILE (rebuild wallet.dat from a SQLite export in\n"); + printf(" an empty data directory, then exit)\n"); printf(" /rescan (walk the chain for coins this wallet owns\n"); printf(" but never recorded, then exit)\n"); printf("\n"); @@ -551,6 +553,15 @@ int main(int argc, char* argv[]) return nRet; } + string strWalletSQLiteRestore = + argval2(argc, argv, "/walletsqliterestore", "-walletsqliterestore"); + if (!strWalletSQLiteRestore.empty()) + { + int nRet = CmdWalletSQLiteRestore(strWalletSQLiteRestore); + DBFlush(true); + return nRet; + } + // Berkeley DB reports failure by throwing, and CDB's constructor lets it // through. Nothing on this path caught anything, so an unreadable // blkindex.dat or wallet.dat -- one written by another platform's Berkeley diff --git a/src/selftest.cpp b/src/selftest.cpp index 4ba90c3..7d3a96e 100644 --- a/src/selftest.cpp +++ b/src/selftest.cpp @@ -1230,6 +1230,41 @@ static int RunWalletSQLiteMigrationSelfTest() FileContainsText(strVerifyOut, "verification: ok"), "SQLite wallet export verifies against wallet.dat") ? 0 : 1; + string strRestoredDir = tmp + "/restored-wallet"; + nFail += Check(MakeDirLocal(strRestoredDir), + "restored wallet directory can be created") ? 0 : 1; + string strRestoreOut = tmp + "/sqlite-restore.txt"; + vector vRestoreArgs; + vRestoreArgs.push_back("-datadir=" + strRestoredDir); + vRestoreArgs.push_back("-nomanagedtor"); + vRestoreArgs.push_back("-nogui"); + vRestoreArgs.push_back("-walletsqliterestore=" + strSQLite); + int nRestoreRet = RunBitflashChild(strExe, vRestoreArgs, NULL, &strRestoreOut); + nFail += Check(nRestoreRet == 0 && + FileContainsText(strRestoreOut, "records restored:"), + "SQLite wallet export restores into a fresh wallet.dat") ? 0 : 1; + + string strVerifyRestoredOut = tmp + "/sqlite-verify-restored.txt"; + vector vVerifyRestoredArgs; + vVerifyRestoredArgs.push_back("-datadir=" + strRestoredDir); + vVerifyRestoredArgs.push_back("-nomanagedtor"); + vVerifyRestoredArgs.push_back("-nogui"); + vVerifyRestoredArgs.push_back("-walletsqliteverify=" + strSQLite); + int nVerifyRestoredRet = + RunBitflashChild(strExe, vVerifyRestoredArgs, NULL, &strVerifyRestoredOut); + nFail += Check(nVerifyRestoredRet == 0 && + FileContainsText(strVerifyRestoredOut, + "verification: ok"), + "restored wallet.dat verifies against the SQLite export") ? 0 : 1; + + string strRestoreOverwriteOut = tmp + "/sqlite-restore-overwrite.txt"; + int nRestoreOverwriteRet = + RunBitflashChild(strExe, vRestoreArgs, NULL, &strRestoreOverwriteOut); + nFail += Check(nRestoreOverwriteRet == 1 && + FileContainsText(strRestoreOverwriteOut, + "Refusing to overwrite existing wallet.dat"), + "SQLite wallet restore refuses to overwrite wallet.dat") ? 0 : 1; + string strOverwriteOut = tmp + "/sqlite-export-overwrite.txt"; int nOverwriteRet = RunBitflashChild(strExe, vExportArgs, NULL, &strOverwriteOut); diff --git a/src/walletcmd.cpp b/src/walletcmd.cpp index 36f6783..d1bad98 100644 --- a/src/walletcmd.cpp +++ b/src/walletcmd.cpp @@ -844,6 +844,52 @@ public: } }; +class CWalletRawRestoreDB : public CWalletDB +{ +public: + CWalletRawRestoreDB(const char* pszMode="cr+", bool fTxn=true) + : CWalletDB(pszMode, fTxn) { } + + bool WriteRaw(CDataStream ssKey, CDataStream ssValue) + { + if (!pdb || ssKey.empty()) + return false; + + Dbt datKey((void*)&ssKey[0], ssKey.size()); + Dbt datValue(ssValue.empty() ? NULL : (void*)&ssValue[0], + ssValue.size()); + int ret = pdb->put(GetTxn(), &datKey, &datValue, 0); + + memset(datKey.get_data(), 0, datKey.get_size()); + if (datValue.get_data()) + memset(datValue.get_data(), 0, datValue.get_size()); + return ret == 0; + } +}; + +class CWalletSQLiteRestoreVisitor : public CWalletRecordVisitor +{ +public: + CWalletRawRestoreDB& db; + int& nCopied; + + CWalletSQLiteRestoreVisitor(CWalletRawRestoreDB& dbIn, int& nCopiedIn) + : db(dbIn), nCopied(nCopiedIn) { } + + bool VisitWalletRecord(const CDataStream& ssKey, + const CDataStream& ssValue, + string& strErrorRet) + { + if (!db.WriteRaw(ssKey, ssValue)) + { + strErrorRet = "could not write raw wallet record"; + return false; + } + nCopied++; + return true; + } +}; + } // namespace int CmdWalletSQLiteExport(const std::string& strDest) @@ -994,6 +1040,86 @@ int CmdWalletSQLiteVerify(const std::string& strPath) return 2; } +int CmdWalletSQLiteRestore(const std::string& strPath) +{ + AttachTerminal(); + + if (strPath.empty()) + { + fprintf(stderr, "Missing SQLite wallet export path.\n"); + return 1; + } + if (!FileExists(strPath.c_str())) + { + fprintf(stderr, "SQLite wallet export does not exist: %s\n", + strPath.c_str()); + return 1; + } + + string strWalletPath = GetAppDir() + "/wallet.dat"; + if (FileExists(strWalletPath.c_str())) + { + fprintf(stderr, "Refusing to overwrite existing wallet.dat: %s\n", + strWalletPath.c_str()); + return 1; + } + + string strError; + CWalletDBSQLite sqlite; + if (!sqlite.Open(strPath, strError)) + { + fprintf(stderr, "Cannot open SQLite wallet export: %s\n", strError.c_str()); + return 1; + } + + int nSQLiteRecords = 0; + if (!sqlite.CountRecords(nSQLiteRecords, strError)) + { + fprintf(stderr, "Cannot count SQLite wallet export records: %s\n", + strError.c_str()); + return 1; + } + if (nSQLiteRecords <= 0) + { + fprintf(stderr, "SQLite wallet export contains no records.\n"); + return 2; + } + + CWalletRawRestoreDB bdb("cr+", true); + if (!bdb.TxnBegin()) + { + fprintf(stderr, "Cannot begin Berkeley DB restore transaction.\n"); + return 1; + } + + int nCopied = 0; + CWalletSQLiteRestoreVisitor visitor(bdb, nCopied); + if (!sqlite.ScanRecords(visitor, strError)) + { + bdb.TxnAbort(); + fprintf(stderr, "Cannot restore SQLite wallet export: %s\n", + strError.c_str()); + return 1; + } + if (nCopied != nSQLiteRecords) + { + bdb.TxnAbort(); + fprintf(stderr, "SQLite wallet restore copied %d records, expected %d.\n", + nCopied, nSQLiteRecords); + return 2; + } + if (!bdb.TxnCommit()) + { + fprintf(stderr, "Cannot commit Berkeley DB restore transaction.\n"); + return 1; + } + + printf("SQLite wallet export restored to wallet.dat\n"); + printf(" records restored: %d\n", nCopied); + fflush(stdout); + return 0; +} + int CmdEncryptWallet(const std::string& strPassphrase) { AttachTerminal(); diff --git a/src/walletcmd.h b/src/walletcmd.h index 784c74c..50dc69e 100644 --- a/src/walletcmd.h +++ b/src/walletcmd.h @@ -79,6 +79,10 @@ int CmdWalletSQLiteExport(const std::string& strDest); // wallet.dat raw records. Prints counts only, never record keys or values. int CmdWalletSQLiteVerify(const std::string& strPath); +// Diagnostic/migration staging: rebuild wallet.dat from a SQLite export in an +// empty data directory. Refuses to overwrite an existing wallet.dat. +int CmdWalletSQLiteRestore(const std::string& strPath); + // Rewrite wallet.dat so private keys and the HD seed are encrypted with the // given passphrase. Returns 0 on success and exits without starting the node. int CmdEncryptWallet(const std::string& strPassphrase); -- cgit v1.2.3 From f7f9885e89e2c2527933b99ec713592ebf930483 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Fri, 14 Aug 2026 17:23:21 -0300 Subject: wallet: clean up failed SQLite migrations --- docs/storage.md | 3 ++- src/selftest.cpp | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++ src/walletcmd.cpp | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 87 insertions(+), 1 deletion(-) (limited to 'docs') diff --git a/docs/storage.md b/docs/storage.md index 4947575..aca3f6e 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -115,7 +115,8 @@ streams without introducing a SQLite runtime wallet loader yet. SQLite export and prints only record counts and mismatch counts. `-walletsqliterestore=FILE` rebuilds `wallet.dat` from a SQLite export only in an empty data directory, then the verifier can prove the restored BDB file still -matches the export byte-for-byte. +matches the export byte-for-byte. Failed exports remove the incomplete SQLite +file and failed restores remove the incomplete `wallet.dat`. ## `blkindex.dat` diff --git a/src/selftest.cpp b/src/selftest.cpp index 7d3a96e..343d9c5 100644 --- a/src/selftest.cpp +++ b/src/selftest.cpp @@ -1162,6 +1162,39 @@ public: } // namespace +static bool CreateSQLiteWalletWithEmptyKey(const string& strPath) +{ + sqlite3* pdb = NULL; + int ret = sqlite3_open_v2(strPath.c_str(), &pdb, + SQLITE_OPEN_READWRITE | + SQLITE_OPEN_CREATE | + SQLITE_OPEN_FULLMUTEX, + NULL); + if (ret != SQLITE_OK || !pdb) + { + if (pdb) + sqlite3_close(pdb); + return false; + } + + char* pszErr = NULL; + bool fOk = + sqlite3_exec(pdb, + "CREATE TABLE wallet_records (" + "key BLOB PRIMARY KEY NOT NULL," + "value BLOB NOT NULL" + ");", + NULL, NULL, &pszErr) == SQLITE_OK && + sqlite3_exec(pdb, + "INSERT INTO wallet_records(key, value) " + "VALUES(x'', x'01');", + NULL, NULL, &pszErr) == SQLITE_OK; + if (pszErr) + sqlite3_free(pszErr); + sqlite3_close(pdb); + return fOk; +} + static int RunWalletSQLiteMigrationSelfTest() { fflush(stdout); @@ -1265,6 +1298,26 @@ static int RunWalletSQLiteMigrationSelfTest() "Refusing to overwrite existing wallet.dat"), "SQLite wallet restore refuses to overwrite wallet.dat") ? 0 : 1; + string strBadSQLite = tmp + "/bad-empty-key.sqlite"; + nFail += Check(CreateSQLiteWalletWithEmptyKey(strBadSQLite), + "bad SQLite export with an empty key can be created") ? 0 : 1; + string strBadRestoreDir = tmp + "/bad-restore-wallet"; + nFail += Check(MakeDirLocal(strBadRestoreDir), + "bad restore wallet directory can be created") ? 0 : 1; + string strBadRestoreOut = tmp + "/sqlite-restore-bad.txt"; + vector vBadRestoreArgs; + vBadRestoreArgs.push_back("-datadir=" + strBadRestoreDir); + vBadRestoreArgs.push_back("-nomanagedtor"); + vBadRestoreArgs.push_back("-nogui"); + vBadRestoreArgs.push_back("-walletsqliterestore=" + strBadSQLite); + int nBadRestoreRet = + RunBitflashChild(strExe, vBadRestoreArgs, NULL, &strBadRestoreOut); + nFail += Check(nBadRestoreRet == 1 && + FileContainsText(strBadRestoreOut, + "Cannot restore SQLite wallet export") && + !FileExists((strBadRestoreDir + "/wallet.dat").c_str()), + "failed SQLite wallet restore removes incomplete wallet.dat") ? 0 : 1; + string strOverwriteOut = tmp + "/sqlite-export-overwrite.txt"; int nOverwriteRet = RunBitflashChild(strExe, vExportArgs, NULL, &strOverwriteOut); diff --git a/src/walletcmd.cpp b/src/walletcmd.cpp index d1bad98..9e4a981 100644 --- a/src/walletcmd.cpp +++ b/src/walletcmd.cpp @@ -798,6 +798,19 @@ static vector DataStreamBytesLocal(const CDataStream& ss) namespace { +static void RemoveSQLiteExportFiles(const string& strPath) +{ + remove(strPath.c_str()); + remove((strPath + "-wal").c_str()); + remove((strPath + "-shm").c_str()); +} + +static void RemoveFailedRestoreWallet(const string& strWalletPath) +{ + DBFlush(true); + remove(strWalletPath.c_str()); +} + class CWalletSQLiteExportVisitor : public CWalletRecordVisitor { public: @@ -912,11 +925,14 @@ int CmdWalletSQLiteExport(const std::string& strDest) CWalletDBSQLite db; if (!db.Open(strDest, strError)) { + RemoveSQLiteExportFiles(strDest); fprintf(stderr, "Cannot open SQLite wallet export: %s\n", strError.c_str()); return 1; } if (!db.BeginTransaction(strError)) { + db.Close(); + RemoveSQLiteExportFiles(strDest); fprintf(stderr, "Cannot begin SQLite wallet export: %s\n", strError.c_str()); return 1; } @@ -927,16 +943,22 @@ int CmdWalletSQLiteExport(const std::string& strDest) { string strRollbackError; db.RollbackTransaction(strRollbackError); + db.Close(); + RemoveSQLiteExportFiles(strDest); fprintf(stderr, "Cannot export wallet.dat to SQLite: %s\n", strError.c_str()); return 1; } if (!db.CommitTransaction(strError)) { + db.Close(); + RemoveSQLiteExportFiles(strDest); fprintf(stderr, "Cannot commit SQLite wallet export: %s\n", strError.c_str()); return 1; } if (!db.Checkpoint(strError)) { + db.Close(); + RemoveSQLiteExportFiles(strDest); fprintf(stderr, "Cannot checkpoint SQLite wallet export: %s\n", strError.c_str()); return 1; } @@ -944,12 +966,16 @@ int CmdWalletSQLiteExport(const std::string& strDest) int nCount = 0; if (!db.CountRecords(nCount, strError)) { + db.Close(); + RemoveSQLiteExportFiles(strDest); fprintf(stderr, "Cannot verify SQLite wallet export count: %s\n", strError.c_str()); return 1; } if (nCount != nCopied) { + db.Close(); + RemoveSQLiteExportFiles(strDest); fprintf(stderr, "SQLite wallet export count mismatch: copied %d, stored %d\n", nCopied, nCount); return 1; @@ -1097,6 +1123,8 @@ int CmdWalletSQLiteRestore(const std::string& strPath) if (!sqlite.ScanRecords(visitor, strError)) { bdb.TxnAbort(); + bdb.Close(); + RemoveFailedRestoreWallet(strWalletPath); fprintf(stderr, "Cannot restore SQLite wallet export: %s\n", strError.c_str()); return 1; @@ -1104,12 +1132,16 @@ int CmdWalletSQLiteRestore(const std::string& strPath) if (nCopied != nSQLiteRecords) { bdb.TxnAbort(); + bdb.Close(); + RemoveFailedRestoreWallet(strWalletPath); fprintf(stderr, "SQLite wallet restore copied %d records, expected %d.\n", nCopied, nSQLiteRecords); return 2; } if (!bdb.TxnCommit()) { + bdb.Close(); + RemoveFailedRestoreWallet(strWalletPath); fprintf(stderr, "Cannot commit Berkeley DB restore transaction.\n"); return 1; } -- cgit v1.2.3 From 3e778dd6cc9045862e99503b5b9af80ec682826f Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Fri, 14 Aug 2026 17:44:09 -0300 Subject: wallet: check SQLite exports against loader types --- docs/storage.md | 3 + src/main_gui.cpp | 11 +++ src/selftest.cpp | 54 ++++++++++++++ src/walletcmd.cpp | 209 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/walletcmd.h | 4 ++ 5 files changed, 281 insertions(+) (limited to 'docs') diff --git a/docs/storage.md b/docs/storage.md index aca3f6e..07c7170 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -117,6 +117,9 @@ SQLite export and prints only record counts and mismatch counts. an empty data directory, then the verifier can prove the restored BDB file still matches the export byte-for-byte. Failed exports remove the incomplete SQLite file and failed restores remove the incomplete `wallet.dat`. +`-walletsqliteloadcheck=FILE` parses a SQLite export using the same record +types the wallet loader understands, so a migration can prove the export is not +just byte-preserving but loader-compatible before runtime activation. ## `blkindex.dat` diff --git a/src/main_gui.cpp b/src/main_gui.cpp index cf96822..c9aac7d 100644 --- a/src/main_gui.cpp +++ b/src/main_gui.cpp @@ -187,6 +187,8 @@ static void PrintUsage() printf(" printing counts only, then exit)\n"); printf(" /walletsqliterestore=FILE (rebuild wallet.dat from a SQLite export in\n"); printf(" an empty data directory, then exit)\n"); + printf(" /walletsqliteloadcheck=FILE\n"); + printf(" (parse a SQLite export like the wallet loader)\n"); printf(" /rescan (walk the chain for coins this wallet owns\n"); printf(" but never recorded, then exit)\n"); printf("\n"); @@ -562,6 +564,15 @@ int main(int argc, char* argv[]) return nRet; } + string strWalletSQLiteLoadCheck = + argval2(argc, argv, "/walletsqliteloadcheck", "-walletsqliteloadcheck"); + if (!strWalletSQLiteLoadCheck.empty()) + { + int nRet = CmdWalletSQLiteLoadCheck(strWalletSQLiteLoadCheck); + DBFlush(true); + return nRet; + } + // Berkeley DB reports failure by throwing, and CDB's constructor lets it // through. Nothing on this path caught anything, so an unreadable // blkindex.dat or wallet.dat -- one written by another platform's Berkeley diff --git a/src/selftest.cpp b/src/selftest.cpp index 343d9c5..feb4076 100644 --- a/src/selftest.cpp +++ b/src/selftest.cpp @@ -1195,6 +1195,28 @@ static bool CreateSQLiteWalletWithEmptyKey(const string& strPath) return fOk; } +static bool CreateSQLiteWalletWithFutureMinVersion(const string& strPath) +{ + string strError; + CWalletDBSQLite db; + CDataStream ssKey(SER_DISK); + ssKey << string("walletminversion"); + CDataStream ssValue(SER_DISK); + ssValue << (WALLET_FORMAT_SUPPORTED + 1); + + if (!db.Open(strPath, strError)) + return false; + if (!db.BeginTransaction(strError)) + return false; + if (!db.WriteRecord(DataStreamBytes(ssKey), DataStreamBytes(ssValue), strError)) + return false; + if (!db.CommitTransaction(strError)) + return false; + if (!db.Checkpoint(strError)) + return false; + return true; +} + static int RunWalletSQLiteMigrationSelfTest() { fflush(stdout); @@ -1263,6 +1285,19 @@ static int RunWalletSQLiteMigrationSelfTest() FileContainsText(strVerifyOut, "verification: ok"), "SQLite wallet export verifies against wallet.dat") ? 0 : 1; + string strLoadCheckOut = tmp + "/sqlite-loadcheck.txt"; + vector vLoadCheckArgs; + vLoadCheckArgs.push_back("-datadir=" + strWalletDir); + vLoadCheckArgs.push_back("-nomanagedtor"); + vLoadCheckArgs.push_back("-nogui"); + vLoadCheckArgs.push_back("-walletsqliteloadcheck=" + strSQLite); + int nLoadCheckRet = + RunBitflashChild(strExe, vLoadCheckArgs, NULL, &strLoadCheckOut); + nFail += Check(nLoadCheckRet == 0 && + FileContainsText(strLoadCheckOut, + "load check: ok"), + "SQLite wallet export passes the loader compatibility check") ? 0 : 1; + string strRestoredDir = tmp + "/restored-wallet"; nFail += Check(MakeDirLocal(strRestoredDir), "restored wallet directory can be created") ? 0 : 1; @@ -1318,6 +1353,25 @@ static int RunWalletSQLiteMigrationSelfTest() !FileExists((strBadRestoreDir + "/wallet.dat").c_str()), "failed SQLite wallet restore removes incomplete wallet.dat") ? 0 : 1; + string strFutureSQLite = tmp + "/future-wallet.sqlite"; + nFail += Check(CreateSQLiteWalletWithFutureMinVersion(strFutureSQLite), + "SQLite export with a future wallet format can be created") ? 0 : 1; + string strFutureLoadCheckOut = tmp + "/sqlite-loadcheck-future.txt"; + vector vFutureLoadCheckArgs; + vFutureLoadCheckArgs.push_back("-datadir=" + strWalletDir); + vFutureLoadCheckArgs.push_back("-nomanagedtor"); + vFutureLoadCheckArgs.push_back("-nogui"); + vFutureLoadCheckArgs.push_back("-walletsqliteloadcheck=" + strFutureSQLite); + int nFutureLoadCheckRet = + RunBitflashChild(strExe, vFutureLoadCheckArgs, NULL, + &strFutureLoadCheckOut); + nFail += Check(nFutureLoadCheckRet == 2 && + FileContainsText(strFutureLoadCheckOut, + "unsupported future wallet format") && + FileContainsText(strFutureLoadCheckOut, + "load check: failed"), + "SQLite wallet load check rejects future wallet formats") ? 0 : 1; + string strOverwriteOut = tmp + "/sqlite-export-overwrite.txt"; int nOverwriteRet = RunBitflashChild(strExe, vExportArgs, NULL, &strOverwriteOut); diff --git a/src/walletcmd.cpp b/src/walletcmd.cpp index 9e4a981..4dda8dd 100644 --- a/src/walletcmd.cpp +++ b/src/walletcmd.cpp @@ -903,6 +903,171 @@ public: } }; +class CWalletSQLiteLoadCheckVisitor : public CWalletRecordVisitor +{ +public: + int nRecords; + int nUnknown; + string strFailure; + + CWalletSQLiteLoadCheckVisitor() : nRecords(0), nUnknown(0) { } + + bool Fail(const string& strMessage, string& strErrorRet) + { + strFailure = strMessage; + strErrorRet = strMessage; + return false; + } + + bool VisitWalletRecord(const CDataStream& ssKeyIn, + const CDataStream& ssValueIn, + string& strErrorRet) + { + nRecords++; + try + { + CDataStream ssKey = ssKeyIn; + CDataStream ssValue = ssValueIn; + string strType; + ssKey >> strType; + if (ssKey.fail()) + return Fail("malformed wallet record key", strErrorRet); + + if (strType == "name") + { + string strAddress; + string strName; + ssKey >> strAddress; + ssValue >> strName; + } + else if (strType == "tx") + { + uint256 hash; + CWalletTx wtx; + ssKey >> hash; + ssValue >> wtx; + } + else if (strType == "key") + { + vector vchPubKey; + CPrivKey vchPrivKey; + ssKey >> vchPubKey; + ssValue >> vchPrivKey; + } + else if (strType == "mkey") + { + unsigned int nID = 0; + CWalletMasterKey kMasterKey; + ssKey >> nID; + ssValue >> kMasterKey; + } + else if (strType == "ckey") + { + vector vchPubKey; + vector vchCryptedSecret; + ssKey >> vchPubKey; + ssValue >> vchCryptedSecret; + } + else if (strType == "defaultkey" || + strType == "hdmaster" || + strType == "hdchaincode" || + strType == "cryptedhdmaster" || + strType == "cryptedhdchaincode") + { + vector vch; + ssValue >> vch; + } + else if (strType == "hdnext" || + strType == "hdreceivenext" || + strType == "hdchangenext" || + strType == "hdcointype") + { + unsigned int nValue = 0; + ssValue >> nValue; + } + else if (strType == "hdschema") + { + int nSchema = 0; + ssValue >> nSchema; + if (!ssValue.fail() && + nSchema != HD_SCHEMA_NONE && + nSchema != HD_SCHEMA_LEGACY && + nSchema != HD_SCHEMA_BIP44) + return Fail("unsupported deterministic wallet schema", + strErrorRet); + } + else if (strType == "walletminversion") + { + int nMinVersion = 0; + ssValue >> nMinVersion; + if (!ssValue.fail() && nMinVersion > WALLET_FORMAT_SUPPORTED) + return Fail("unsupported future wallet format", strErrorRet); + } + else if (strType == "pool") + { + int64 nIndex = 0; + vector vchPubKey; + ssKey >> nIndex; + ssValue >> vchPubKey; + } + else if (strType == "setting") + { + string strKey; + ssKey >> strKey; + if (strKey == "nTransactionFee") + { + int64 nValue = 0; + ssValue >> nValue; + } + else if (strKey == "addrIncoming") + { + CAddress addr; + ssValue >> addr; + } + else if (strKey == "nMineMode") + { + int nValue = 0; + ssValue >> nValue; + } + else if (strKey == "fGenerateBitcoins") + { + int fValue = 0; + ssValue >> fValue; + } + else if (strKey == "strParticipantPool") + { + string strValue; + ssValue >> strValue; + } + } + else if (strType == "version") + { + int nVersion = 0; + ssValue >> nVersion; + } + else + { + nUnknown++; + } + + if (ssKey.fail()) + return Fail("malformed wallet record key", strErrorRet); + if (ssValue.fail()) + return Fail("malformed wallet record value", strErrorRet); + } + catch (const std::exception& e) + { + return Fail(strprintf("wallet record parse exception: %s", e.what()), + strErrorRet); + } + catch (...) + { + return Fail("wallet record parse exception", strErrorRet); + } + return true; + } +}; + } // namespace int CmdWalletSQLiteExport(const std::string& strDest) @@ -1152,6 +1317,50 @@ int CmdWalletSQLiteRestore(const std::string& strPath) return 0; } +int CmdWalletSQLiteLoadCheck(const std::string& strPath) +{ + AttachTerminal(); + + if (strPath.empty()) + { + fprintf(stderr, "Missing SQLite wallet export path.\n"); + return 1; + } + if (!FileExists(strPath.c_str())) + { + fprintf(stderr, "SQLite wallet export does not exist: %s\n", + strPath.c_str()); + return 1; + } + + string strError; + CWalletDBSQLite sqlite; + if (!sqlite.Open(strPath, strError)) + { + fprintf(stderr, "Cannot open SQLite wallet export: %s\n", strError.c_str()); + return 1; + } + + CWalletSQLiteLoadCheckVisitor visitor; + if (!sqlite.ScanRecords(visitor, strError)) + { + printf("SQLite wallet load check\n"); + printf(" records checked: %d\n", visitor.nRecords); + printf(" unknown records: %d\n", visitor.nUnknown); + printf(" load check: failed\n"); + printf(" failure: %s\n", strError.c_str()); + fflush(stdout); + return 2; + } + + printf("SQLite wallet load check\n"); + printf(" records checked: %d\n", visitor.nRecords); + printf(" unknown records: %d\n", visitor.nUnknown); + printf(" load check: ok\n"); + fflush(stdout); + return 0; +} + int CmdEncryptWallet(const std::string& strPassphrase) { AttachTerminal(); diff --git a/src/walletcmd.h b/src/walletcmd.h index 50dc69e..d993856 100644 --- a/src/walletcmd.h +++ b/src/walletcmd.h @@ -83,6 +83,10 @@ int CmdWalletSQLiteVerify(const std::string& strPath); // empty data directory. Refuses to overwrite an existing wallet.dat. int CmdWalletSQLiteRestore(const std::string& strPath); +// Diagnostic/migration staging: parse a SQLite wallet export as the runtime +// wallet loader would. Prints counts only, never record keys or values. +int CmdWalletSQLiteLoadCheck(const std::string& strPath); + // Rewrite wallet.dat so private keys and the HD seed are encrypted with the // given passphrase. Returns 0 on success and exits without starting the node. int CmdEncryptWallet(const std::string& strPassphrase); -- cgit v1.2.3 From f9ee957bd6d3f28dfee166a82f389aad666a0119 Mon Sep 17 00:00:00 2001 From: Nsable-sh Date: Fri, 14 Aug 2026 18:17:23 -0300 Subject: wallet: stage SQLite runtime loading --- docs/storage.md | 4 + src/db.cpp | 361 +++++++++++++++++++++++++++++++++++++++++++++++--- src/db.h | 1 + src/main_gui.cpp | 22 +++ src/selftest.cpp | 29 ++++ src/wallet_sqlite.cpp | 19 +++ src/wallet_sqlite.h | 1 + 7 files changed, 420 insertions(+), 17 deletions(-) (limited to 'docs') diff --git a/docs/storage.md b/docs/storage.md index 07c7170..08c4a01 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -120,6 +120,10 @@ file and failed restores remove the incomplete `wallet.dat`. `-walletsqliteloadcheck=FILE` parses a SQLite export using the same record types the wallet loader understands, so a migration can prove the export is not just byte-preserving but loader-compatible before runtime activation. +`-walletsqlite=FILE` is the first runtime staging flag: it loads a SQLite export +through the startup wallet path, initializes the in-memory wallet state, reports +success, and exits before network, mining, GUI, or wallet mutation begins. It is +deliberately read-only until the write path is routed to SQLite as well. ## `blkindex.dat` diff --git a/src/db.cpp b/src/db.cpp index 8b82fe5..581e5a7 100644 --- a/src/db.cpp +++ b/src/db.cpp @@ -3,6 +3,7 @@ // file license.txt or http://www.opensource.org/licenses/mit-license.php. #include "headers.h" +#include "wallet_sqlite.h" @@ -650,6 +651,289 @@ bool CReviewDB::WriteReviews(uint256 hash, const vector& vReviews) // CWalletDB // +class CSQLiteWalletRuntimeLoadVisitor : public CWalletRecordVisitor +{ +public: + bool fHaveStoredMineMode; + bool fHaveStoredHDCoinType; + string strLastType; + + CSQLiteWalletRuntimeLoadVisitor() : + fHaveStoredMineMode(false), + fHaveStoredHDCoinType(false) + { + } + + bool VisitWalletRecord(const CDataStream& ssKeyIn, + const CDataStream& ssValueIn, + string& strErrorRet) + { + CDataStream ssKey = ssKeyIn; + CDataStream ssValue = ssValueIn; + string strType; + ssKey >> strType; + strLastType = strType; + + if (strType == "name") + { + string strAddress; + ssKey >> strAddress; + ssValue >> mapAddressBook[strAddress]; + } + else if (strType == "tx") + { + uint256 hash; + ssKey >> hash; + CWalletTx& wtx = mapWallet[hash]; + ssValue >> wtx; + + if (wtx.GetHash() != hash) + printf("Error in SQLite wallet export, hash mismatch\n"); + } + else if (strType == "key") + { + vector vchPubKey; + ssKey >> vchPubKey; + CPrivKey vchPrivKey; + ssValue >> vchPrivKey; + + mapKeys[vchPubKey] = vchPrivKey; + mapPubKeys[Hash160(vchPubKey)] = vchPubKey; + } + else if (strType == "mkey") + { + unsigned int nID = 0; + ssKey >> nID; + CWalletMasterKey kMasterKey; + ssValue >> kMasterKey; + mapMasterKeys[nID] = kMasterKey; + nWalletMasterKeyMaxID = max(nWalletMasterKeyMaxID, nID); + fWalletEncrypted = true; + fWalletLocked = true; + } + else if (strType == "ckey") + { + vector vchPubKey; + ssKey >> vchPubKey; + vector vchCryptedSecret; + ssValue >> vchCryptedSecret; + if (!AddCryptedKey(vchPubKey, vchCryptedSecret)) + { + strErrorRet = "SQLite wallet export has an unreadable encrypted key record"; + return false; + } + } + else if (strType == "defaultkey") + { + ssValue >> vchDefaultKey; + } + else if (strType == "hdmaster") + { + ssValue >> vchHDMaster; + } + else if (strType == "hdchaincode") + { + ssValue >> vchHDChainCode; + } + else if (strType == "hdnext") + { + ssValue >> nHDNext; + } + else if (strType == "hdschema") + { + ssValue >> nHDKeySchema; + } + else if (strType == "hdcointype") + { + ssValue >> nHDCoinType; + fHaveStoredHDCoinType = true; + } + else if (strType == "hdreceivenext") + { + ssValue >> nHDReceiveNext; + } + else if (strType == "hdchangenext") + { + ssValue >> nHDChangeNext; + } + else if (strType == "cryptedhdmaster") + { + ssValue >> vchCryptedHDMaster; + fWalletEncrypted = true; + fWalletLocked = true; + } + else if (strType == "cryptedhdchaincode") + { + ssValue >> vchCryptedHDChainCode; + fWalletEncrypted = true; + fWalletLocked = true; + } + else if (strType == "walletminversion") + { + int nMinVersion = 0; + ssValue >> nMinVersion; + if (nMinVersion > WALLET_FORMAT_SUPPORTED) + { + strWalletLoadError = strprintf( + "This SQLite wallet export was written by a newer version of " + "Bitflash (wallet format %d; this build understands %d). " + "Upgrade Bitflash before opening it.", + nMinVersion, WALLET_FORMAT_SUPPORTED); + strErrorRet = strWalletLoadError; + return false; + } + } + else if (strType == "pool") + { + int64 nIndex; + ssKey >> nIndex; + vector vchPubKey; + ssValue >> vchPubKey; + mapKeyPool[nIndex] = vchPubKey; + } + else if (strType == "setting") + { + string strKey; + ssKey >> strKey; + if (strKey == "nTransactionFee") ssValue >> nTransactionFee; + if (strKey == "addrIncoming") ssValue >> addrIncoming; + + if (!fMineModeFromCommandLine) + { + if (strKey == "nMineMode") { ssValue >> nMineMode; fHaveStoredMineMode = true; } + if (strKey == "fGenerateBitcoins") ssValue >> fGenerateBitcoins; + if (strKey == "strParticipantPool") ssValue >> strParticipantPool; + } + } + + if (ssKey.fail()) + { + strErrorRet = strprintf("malformed SQLite wallet record key while handling '%s'", + strLastType.c_str()); + return false; + } + if (ssValue.fail()) + { + strErrorRet = strprintf("malformed SQLite wallet record value while handling '%s'", + strLastType.c_str()); + return false; + } + return true; + } + + vector vchDefaultKey; +}; + +static bool FinishLoadedWallet(const vector& vchDefaultKey, + bool fMayCreateDefaultKey) +{ + if (!vchDefaultKey.empty() && mapPubKeys.count(Hash160(vchDefaultKey))) + { + keyUser.SetPubKey(vchDefaultKey); + if (mapKeys.count(vchDefaultKey)) + keyUser.SetPrivKey(mapKeys[vchDefaultKey]); + } + else if (IsWalletEncrypted()) + { + printf("LoadWallet: encrypted wallet has no usable default public key. " + "The file was left untouched.\n"); + return false; + } + else + { + if (!fMayCreateDefaultKey) + { + strWalletLoadError = + "SQLite wallet export has no usable default public key"; + printf("LoadWallet: %s\n", strWalletLoadError.c_str()); + return false; + } + + keyUser.MakeNewKey(); + if (!AddKey(keyUser)) + return false; + if (!SetAddressBookName(PubKeyToAddress(keyUser.GetPubKey()), "Your Address")) + return false; + CWalletDB().WriteDefaultKey(keyUser.GetPubKey()); + } + + return true; +} + +static bool ReconcileLoadedWalletMetadata(bool fHaveStoredMineMode, + bool fHaveStoredHDCoinType) +{ + // An encrypted wallet has a seed; it just has not been decrypted yet. + // + // The reset in the else branch was written when "no seed in memory after + // loading" could only mean "this wallet has no seed". With encryption that + // is no longer true, and taking the else branch here wiped the schema and + // the derivation counters that had just been read correctly out of the + // file. Unlocking later restores the seed but nothing restores those, so a + // BIP44 wallet came back deriving legacy addresses -- different addresses + // from the ones its own recovery phrase produces, while the audit reported + // the phrase as covering nothing. + if (HaveHDSeed() || !vchCryptedHDMaster.empty()) + { + if (nHDKeySchema == HD_SCHEMA_NONE) + { + nHDKeySchema = HD_SCHEMA_LEGACY; + printf("LoadWallet: deterministic seed has no schema metadata; " + "treating it as legacy m/index'\n"); + } + else if (nHDKeySchema != HD_SCHEMA_LEGACY && + nHDKeySchema != HD_SCHEMA_BIP44) + { + printf("LoadWallet: deterministic wallet schema %d (%s) is not " + "supported by this build\n", + nHDKeySchema, HDKeySchemaName(nHDKeySchema).c_str()); + return false; + } + if (!fHaveStoredHDCoinType) + { + nHDCoinType = HD_BIP44_COIN_TYPE_BITFLASH; + printf("LoadWallet: deterministic seed has no BIP44 coin type metadata; " + "using Bitflash coin type %u\n", nHDCoinType); + } + } + else + { + nHDKeySchema = HD_SCHEMA_NONE; + nHDNext = 0; + nHDReceiveNext = 0; + nHDChangeNext = 0; + nHDCoinType = HD_BIP44_COIN_TYPE_BITFLASH; + } + + // fGenerateBitcoins and nMineMode only mean anything together, and a + // wallet.dat can easily hold one without the other: Bitcoin 0.1.0 already + // wrote fGenerateBitcoins when mining was toggled, years before this fork + // had a mode at all. Restoring that lone flag put a node into the one state + // the miner cannot act on -- generate on, mode relay -- so every miner + // thread started and returned at the relay guard, and the machine sat there + // looking like it was mining while hashing nothing. That happened to a + // 32-core node on the first 1.2.11 start. + // + // So the flag is only believed when the mode it belongs to was stored with + // it, and the pair is made coherent either way. + if (!fMineModeFromCommandLine) + { + if (!fHaveStoredMineMode) + { + if (fGenerateBitcoins) + printf("LoadWallet: ignoring a stored fGenerateBitcoins with no mining mode " + "beside it -- it predates this setting and cannot be read on its own\n"); + fGenerateBitcoins = 0; + } + else + { + fGenerateBitcoins = (nMineMode != MINE_RELAY) ? 1 : 0; + } + } + + return true; +} + bool CWalletDB::LoadWallet(vector& vchDefaultKeyRet) { // Whether wallet.dat actually carried a mining mode, as opposed to just the @@ -1005,31 +1289,74 @@ bool LoadWallet() if (!CWalletDB("cr").LoadWallet(vchDefaultKey)) return false; - if (!vchDefaultKey.empty() && mapPubKeys.count(Hash160(vchDefaultKey))) + return FinishLoadedWallet(vchDefaultKey, true); +} + +bool LoadWalletFromSQLite(const string& strPath) +{ + strWalletLoadError.clear(); + + if (!FileExists(strPath.c_str())) { - // Set keyUser - keyUser.SetPubKey(vchDefaultKey); - if (mapKeys.count(vchDefaultKey)) - keyUser.SetPrivKey(mapKeys[vchDefaultKey]); + strWalletLoadError = strprintf("SQLite wallet export does not exist: %s", + strPath.c_str()); + printf("LoadWallet: %s\n", strWalletLoadError.c_str()); + return false; } - else if (IsWalletEncrypted()) + + CWalletDBSQLite sqlite; + string strError; + if (!sqlite.OpenReadOnly(strPath, strError)) { - printf("LoadWallet: encrypted wallet has no usable default public key. " - "The file was left untouched.\n"); + strWalletLoadError = strError; + printf("LoadWallet: cannot open SQLite wallet export: %s\n", + strError.c_str()); return false; } - else + + CSQLiteWalletRuntimeLoadVisitor visitor; + try { - // Create new keyUser and set as default key - keyUser.MakeNewKey(); - if (!AddKey(keyUser)) - return false; - if (!SetAddressBookName(PubKeyToAddress(keyUser.GetPubKey()), "Your Address")) - return false; - CWalletDB().WriteDefaultKey(keyUser.GetPubKey()); + CRITICAL_BLOCK(cs_mapKeys) + CRITICAL_BLOCK(cs_mapWallet) + { + if (!sqlite.ScanRecords(visitor, strError)) + { + if (strWalletLoadError.empty()) + strWalletLoadError = strError; + printf("LoadWallet: SQLite wallet export could not be read. " + "Failed while handling a '%s' record: %s\n", + visitor.strLastType.c_str(), strError.c_str()); + return false; + } + } + } + catch (const std::exception& e) + { + strWalletLoadError = strprintf( + "SQLite wallet export could not be read while handling a '%s' record: %s", + visitor.strLastType.c_str(), e.what()); + printf("LoadWallet: %s\n", strWalletLoadError.c_str()); + return false; + } + catch (...) + { + strWalletLoadError = strprintf( + "SQLite wallet export could not be read while handling a '%s' record", + visitor.strLastType.c_str()); + printf("LoadWallet: %s\n", strWalletLoadError.c_str()); + return false; } - return true; + if (!ReconcileLoadedWalletMetadata(visitor.fHaveStoredMineMode, + visitor.fHaveStoredHDCoinType)) + { + if (strWalletLoadError.empty()) + strWalletLoadError = "SQLite wallet metadata is not supported by this build"; + return false; + } + + return FinishLoadedWallet(visitor.vchDefaultKey, false); } diff --git a/src/db.h b/src/db.h index 53bda59..59ff24a 100644 --- a/src/db.h +++ b/src/db.h @@ -536,6 +536,7 @@ public: bool ScanWalletRecords(CWalletRecordVisitor& visitor, string& strErrorRet); bool LoadWallet(); +bool LoadWalletFromSQLite(const string& strPath); // Writes a wallet.dat that opens on its own, anywhere -- see the definition in // db.cpp for why a plain file copy does not. Still a point-in-time snapshot: diff --git a/src/main_gui.cpp b/src/main_gui.cpp index c9aac7d..ee875c7 100644 --- a/src/main_gui.cpp +++ b/src/main_gui.cpp @@ -189,6 +189,7 @@ static void PrintUsage() printf(" an empty data directory, then exit)\n"); printf(" /walletsqliteloadcheck=FILE\n"); printf(" (parse a SQLite export like the wallet loader)\n"); + printf(" /walletsqlite=FILE (stage-load a SQLite wallet export, then exit)\n"); printf(" /rescan (walk the chain for coins this wallet owns\n"); printf(" but never recorded, then exit)\n"); printf("\n"); @@ -617,6 +618,27 @@ int main(int argc, char* argv[]) printf("Loading wallet...\n"); try { + string strWalletSQLite = + argval2(argc, argv, "/walletsqlite", "-walletsqlite"); + if (!strWalletSQLite.empty()) + { + AttachTerminal(); + if (!LoadWalletFromSQLite(strWalletSQLite)) + { + FatalStartupError(fHeadlessStartup, + "Cannot open SQLite wallet export.", + strWalletLoadError); + return 1; + } + + fprintf(stderr, "SQLite wallet loaded: read-only staging backend\n"); + fprintf(stderr, "SQLite runtime writes are not enabled yet; exiting before " + "network, mining, GUI, or wallet mutation starts.\n"); + fflush(stderr); + DBFlush(true); + return 0; + } + if (!LoadWallet()) { // LoadWallet() explains itself into strWalletLoadError when it diff --git a/src/selftest.cpp b/src/selftest.cpp index feb4076..14bf45a 100644 --- a/src/selftest.cpp +++ b/src/selftest.cpp @@ -1298,6 +1298,19 @@ static int RunWalletSQLiteMigrationSelfTest() "load check: ok"), "SQLite wallet export passes the loader compatibility check") ? 0 : 1; + string strRuntimeOut = tmp + "/sqlite-runtime.txt"; + vector vRuntimeArgs; + vRuntimeArgs.push_back("-datadir=" + strWalletDir); + vRuntimeArgs.push_back("-nomanagedtor"); + vRuntimeArgs.push_back("-nogui"); + vRuntimeArgs.push_back("-walletsqlite=" + strSQLite); + int nRuntimeRet = + RunBitflashChild(strExe, vRuntimeArgs, NULL, &strRuntimeOut); + nFail += Check(nRuntimeRet == 0 && + FileContainsText(strRuntimeOut, + "SQLite wallet loaded: read-only staging backend"), + "SQLite wallet export loads through the runtime startup path") ? 0 : 1; + string strRestoredDir = tmp + "/restored-wallet"; nFail += Check(MakeDirLocal(strRestoredDir), "restored wallet directory can be created") ? 0 : 1; @@ -1372,6 +1385,22 @@ static int RunWalletSQLiteMigrationSelfTest() "load check: failed"), "SQLite wallet load check rejects future wallet formats") ? 0 : 1; + string strFutureRuntimeOut = tmp + "/sqlite-runtime-future.txt"; + vector vFutureRuntimeArgs; + vFutureRuntimeArgs.push_back("-datadir=" + strWalletDir); + vFutureRuntimeArgs.push_back("-nomanagedtor"); + vFutureRuntimeArgs.push_back("-nogui"); + vFutureRuntimeArgs.push_back("-walletsqlite=" + strFutureSQLite); + int nFutureRuntimeRet = + RunBitflashChild(strExe, vFutureRuntimeArgs, NULL, + &strFutureRuntimeOut); + nFail += Check(nFutureRuntimeRet == 1 && + FileContainsText(strFutureRuntimeOut, + "Cannot open SQLite wallet export") && + FileContainsText(strFutureRuntimeOut, + "newer version of Bitflash"), + "SQLite runtime startup rejects future wallet formats") ? 0 : 1; + string strOverwriteOut = tmp + "/sqlite-export-overwrite.txt"; int nOverwriteRet = RunBitflashChild(strExe, vExportArgs, NULL, &strOverwriteOut); diff --git a/src/wallet_sqlite.cpp b/src/wallet_sqlite.cpp index 8489cac..ca81573 100644 --- a/src/wallet_sqlite.cpp +++ b/src/wallet_sqlite.cpp @@ -58,6 +58,25 @@ bool CWalletDBSQLite::Open(const string& strPath, string& strError) return true; } +bool CWalletDBSQLite::OpenReadOnly(const string& strPath, string& strError) +{ + Close(); + + int ret = sqlite3_open_v2(strPath.c_str(), &pdb, + SQLITE_OPEN_READONLY | + SQLITE_OPEN_FULLMUTEX, + NULL); + if (ret != SQLITE_OK) + { + SetSQLiteError(pdb, "cannot open SQLite wallet read-only", strError); + Close(); + return false; + } + + sqlite3_busy_timeout(pdb, 5000); + return true; +} + void CWalletDBSQLite::Close() { if (pdb) diff --git a/src/wallet_sqlite.h b/src/wallet_sqlite.h index 385a340..5e6467b 100644 --- a/src/wallet_sqlite.h +++ b/src/wallet_sqlite.h @@ -15,6 +15,7 @@ public: ~CWalletDBSQLite(); bool Open(const string& strPath, string& strError); + bool OpenReadOnly(const string& strPath, string& strError); void Close(); bool BeginTransaction(string& strError); -- cgit v1.2.3 From 4c24a86a8aa01b02874e5bdb7856beed6e4cf6b6 Mon Sep 17 00:00:00 2001 From: bitflash Date: Wed, 19 Aug 2026 11:20:55 -0300 Subject: wallet/docs: SQLite is the default backend, not experimental Drop the "experimental" label from the wallet code and docs now that SQLite is the default backend for new wallets, is coin-proven, and carries encrypt, backup, atomic seed writes, and the storage diagnostics. - main_gui.cpp: -help text, the unknown-backend error (SQLite is no longer "experimental" and Berkeley DB is no longer "the default"), and the startup banner ("Wallet backend: SQLite"). - walletcmd.h: the export command comment. - docs/storage.md: rewrite the SQLite section from staging-only ("runtime still opens Berkeley DB", "no runtime backend flag exists yet") to the shipped state -- default backend, conversion wizard, per-backend ops -- and reframe the migration order as shipped in v1.2.20. --- docs/storage.md | 67 +++++++++++++++++++++++++++----------------------------- src/main_gui.cpp | 16 +++++++------- src/walletcmd.h | 6 ++--- 3 files changed, 43 insertions(+), 46 deletions(-) (limited to 'docs') diff --git a/docs/storage.md b/docs/storage.md index 08c4a01..b52a955 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -85,10 +85,11 @@ A future wallet backend must keep these properties: - recovery phrase encrypted and locked, requiring unlock before coverage can be audited. -## Experimental SQLite Wallet Store +## SQLite Wallet Store -The SQLite wallet path starts as a byte-preserving record store, not as a new -wallet format: +Since v1.2.20 SQLite is the default wallet backend. It is a byte-preserving +record store, not a new wallet format -- the same serialized `CDataStream` bytes +`wallet.dat` uses: ```sql CREATE TABLE wallet_records ( @@ -98,32 +99,26 @@ CREATE TABLE wallet_records ( PRAGMA user_version=1; ``` -Keys and values are the same serialized `CDataStream` bytes used by `wallet.dat`. -The first implementation is exercised only by `-selftest=wallet-sqlite`; the -runtime wallet still opens Berkeley DB. The purpose is to prove schema creation, -raw record write/read, close/reopen behavior, and value preservation before any -BDB-to-SQLite migration or runtime backend flag exists. - -`-walletsqliteexport=FILE` is the next staging tool. It streams every raw -`wallet.dat` record through `ScanWalletRecords()` and writes those same key and -value bytes into the SQLite table inside one transaction. It refuses to -overwrite an existing export and still does not change the runtime wallet -backend. `CWalletDBSQLite::ScanRecords()` can stream those exported records back -through the same visitor shape, so tests can compare BDB and SQLite record -streams without introducing a SQLite runtime wallet loader yet. -`-walletsqliteverify=FILE` compares the current `wallet.dat` stream with a -SQLite export and prints only record counts and mismatch counts. -`-walletsqliterestore=FILE` rebuilds `wallet.dat` from a SQLite export only in -an empty data directory, then the verifier can prove the restored BDB file still -matches the export byte-for-byte. Failed exports remove the incomplete SQLite -file and failed restores remove the incomplete `wallet.dat`. -`-walletsqliteloadcheck=FILE` parses a SQLite export using the same record -types the wallet loader understands, so a migration can prove the export is not -just byte-preserving but loader-compatible before runtime activation. -`-walletsqlite=FILE` is the first runtime staging flag: it loads a SQLite export -through the startup wallet path, initializes the in-memory wallet state, reports -success, and exits before network, mining, GUI, or wallet mutation begins. It is -deliberately read-only until the write path is routed to SQLite as well. +A fresh data directory starts on `wallet.sqlite`. An existing `wallet.dat` keeps +running on Berkeley DB until it is converted: the GUI offers a one-click +conversion on first run, and `-walletbackend=sqlite` / `-walletbackend=bdb` +(recorded in a plain-text `wallet-backend` marker file in the datadir) select a +backend explicitly. Conversion never modifies `wallet.dat`; it stays as a +fallback, and the change takes effect on restart. + +Every wallet operation runs on either backend: read, write, erase, key-pool and +HD-seed writes (the two halves of the seed bracketed in one SQLite transaction), +`-backupwallet` (checkpoint then copy the self-contained file), `-encryptwallet` +(an atomic rebuild-and-swap in one transaction), and the `-walletstorageaudit` / +`-walletstoragecheck` diagnostics, which read the active backend's file. + +The migration tooling remains available for scripted use. `-walletsqliteexport=FILE` +streams every raw `wallet.dat` record through `ScanWalletRecords()` into a SQLite +file in one transaction, refusing to overwrite. `-walletsqliteverify=FILE` +compares the two streams and prints only counts. `-walletsqliteloadcheck=FILE` +parses an export with the same record types the loader uses. +`-walletsqliterestore=FILE` rebuilds a `wallet.dat` from an export in an empty +data directory. Failed exports and restores remove their incomplete output. ## `blkindex.dat` @@ -150,12 +145,14 @@ conversion path. The old IP address database (`addr.dat`) has been removed. Peer discovery is handled by Nostr, `.btf`, rendezvous relays, and direct onion peers. -## Recommended Migration Order +## Migration Order (shipped in v1.2.20) -1. Keep adding storage self-tests around current BDB behavior. -2. Add an experimental wallet backend behind an explicit flag. -3. Implement BDB-to-new-wallet migration as a one-way copy with a backup. -4. Make the new wallet backend default only after cross-version restore, - encryption, backup, and recovery-audit tests pass. +1. Storage self-tests were added around Berkeley DB behavior first. +2. The SQLite backend landed behind an explicit flag, then behind a + `wallet-backend` marker and a GUI conversion wizard. +3. BDB-to-SQLite migration is a one-way copy that leaves `wallet.dat` as a backup. +4. SQLite became the default for new wallets only after cross-version restore, + encryption, backup, and recovery-audit tests passed -- and a live coin-proof + of receive, spend, restart, and encrypt on real mainnet funds. 5. Treat `blkindex.dat` separately; it is safer to rebuild than to migrate under pressure. diff --git a/src/main_gui.cpp b/src/main_gui.cpp index 55ca8c6..943799f 100644 --- a/src/main_gui.cpp +++ b/src/main_gui.cpp @@ -182,7 +182,7 @@ static void PrintUsage() printf(" /walletstoragecheck (fail closed on inconsistent or unsafe\n"); printf(" wallet.dat storage state)\n"); printf(" /walletsqliteexport=FILE (copy raw wallet.dat records into an\n"); - printf(" experimental SQLite store, then exit)\n"); + printf(" SQLite store, then exit)\n"); printf(" /walletsqliteverify=FILE (compare wallet.dat with a SQLite export,\n"); printf(" printing counts only, then exit)\n"); printf(" /walletsqliterestore=FILE (rebuild wallet.dat from a SQLite export in\n"); @@ -680,17 +680,17 @@ int main(int argc, char* argv[]) AttachTerminal(); FatalStartupError(fHeadlessStartup, strprintf("Unknown wallet backend '%s'.", strWalletBackend.c_str()), - "Use -walletbackend=sqlite for the experimental SQLite backend, " - "or omit -walletbackend (or -walletbackend=bdb) to use the " - "default Berkeley DB wallet.dat."); + "Use -walletbackend=sqlite for the SQLite backend, or " + "-walletbackend=bdb for the Berkeley DB wallet.dat."); return 1; } if (strWalletBackend == "sqlite") { - // Opt-in, experimental. The node runs on /wallet.sqlite and - // never opens wallet.dat, but the user has to have exported one - // first -- there is no silent migration, and wallet.dat is left + // Explicit opt-in via the flag. The node runs on + // /wallet.sqlite and never opens wallet.dat, but the user + // has to have exported one first -- there is no silent migration, + // and wallet.dat is left // exactly where it is so a plain restart goes back to it. AttachTerminal(); string strSQLitePath = GetAppDir() + "/wallet.sqlite"; @@ -724,7 +724,7 @@ int main(int argc, char* argv[]) } fprintf(stderr, - "Wallet backend: SQLite (experimental)\n" + "Wallet backend: SQLite\n" " %s\n" "Your Berkeley DB wallet.dat is left untouched. Restart without\n" "-walletbackend=sqlite to go back to it.\n", diff --git a/src/walletcmd.h b/src/walletcmd.h index 38cbfdb..7013d44 100644 --- a/src/walletcmd.h +++ b/src/walletcmd.h @@ -70,9 +70,9 @@ int CmdWalletStorageAudit(const std::string& strJsonOut = ""); // or migration-unsafe mix of storage records. int CmdWalletStorageCheck(); -// Diagnostic/migration staging: copy every raw wallet.dat record into an -// experimental SQLite key/value store, preserving serialized bytes. This does -// not change which backend the runtime wallet opens. +// Copy every raw wallet.dat record into a SQLite key/value store, preserving +// serialized bytes. A migration/scripting tool; it does not change which backend +// the runtime wallet opens. int CmdWalletSQLiteExport(const std::string& strDest); // Export the currently-loaded wallet to a SQLite file (shared by the command -- cgit v1.2.3 From 18ffcb21393c9e54653844d62c5ef97c58bc61d4 Mon Sep 17 00:00:00 2001 From: bitflash-maintainer Date: Tue, 1 Sep 2026 17:16:04 +0000 Subject: docs: this tree has no marketplace stores to document --- docs/storage.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) (limited to 'docs') diff --git a/docs/storage.md b/docs/storage.md index b52a955..c7de4e7 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -137,10 +137,11 @@ conversion path. ## Other Legacy Stores -| File | Class | Records | Current risk | -|---|---|---|---| -| `reviews.dat` | `CReviewDB` | `("user", hash)`, `("reviews", hash)` | Legacy market/review storage. Not part of wallet safety. | -| `market.dat` | `CMarketDB` | No explicit typed records in current wrappers. | Legacy market storage shell. | +None. Upstream still documents `reviews.dat` (`CReviewDB`) and `market.dat` +(`CMarketDB`) here; this fork deleted the 2009 marketplace outright, along +with `CReviewDB`, `market.cpp` and `market.h`, so neither file is opened or +created by this tree. A datadir carrying them from an older build is simply +ignored. The old IP address database (`addr.dat`) has been removed. Peer discovery is handled by Nostr, `.btf`, rendezvous relays, and direct onion peers. -- cgit v1.2.3