upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--README.md10
-rw-r--r--docs/utxo-commitment.md14
-rw-r--r--docs/utxo-proofs.md61
-rw-r--r--scripts/prove-utxo.py202
-rw-r--r--scripts/verify-utxo-proof.py163
5 files changed, 450 insertions, 0 deletions
diff --git a/README.md b/README.md
index d549337..c28b0f1 100644
--- a/README.md
+++ b/README.md
@@ -83,6 +83,16 @@ It reconstructs the best chain from local block files, computes the current
83UTXO root and supply, and writes canonical JSON that independent auditors can 83UTXO root and supply, and writes canonical JSON that independent auditors can
84compare or timestamp externally with OpenTimestamps. See 84compare or timestamp externally with OpenTimestamps. See
85[UTXO-set commitments](docs/utxo-commitment.md). 85[UTXO-set commitments](docs/utxo-commitment.md).
86
87The same root can prove one unspent output without sharing the full UTXO set:
88
89```bash
90python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json
91python3 scripts/verify-utxo-proof.py utxo-proof.json
92```
93
94See [UTXO inclusion proofs](docs/utxo-proofs.md).
95
86Bitflash also ships a deterministic fair-launch verifier: 96Bitflash also ships a deterministic fair-launch verifier:
87 97
88```bash 98```bash
diff --git a/docs/utxo-commitment.md b/docs/utxo-commitment.md
index ddf5450..991e5d9 100644
--- a/docs/utxo-commitment.md
+++ b/docs/utxo-commitment.md
@@ -54,6 +54,20 @@ SHA256("BTFNODE1|" || left_hash || right_hash)
54 54
55The final hash is the UTXO-set root. 55The final hash is the UTXO-set root.
56 56
57## Inclusion Proofs
58
59After a root has been computed, `scripts/prove-utxo.py` can create a compact
60inclusion proof for one unspent output:
61
62```bash
63python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json
64python3 scripts/verify-utxo-proof.py utxo-proof.json
65```
66
67The verifier does not need local block files. It recomputes the target leaf,
68walks the sibling path, and compares the result with the expected root. See
69[UTXO inclusion proofs](utxo-proofs.md).
70
57## Deterministic Output 71## Deterministic Output
58 72
59`--json` and `--out` write canonical JSON with sorted keys, fixed separators, 73`--json` and `--out` write canonical JSON with sorted keys, fixed separators,
diff --git a/docs/utxo-proofs.md b/docs/utxo-proofs.md
new file mode 100644
index 0000000..4947c3c
--- /dev/null
+++ b/docs/utxo-proofs.md
@@ -0,0 +1,61 @@
1# Bitflash UTXO inclusion proofs
2
3`scripts/prove-utxo.py` creates a deterministic inclusion proof for one
4currently unspent output. `scripts/verify-utxo-proof.py` verifies that proof
5against the UTXO root produced by `scripts/verify-utxo-set.py`.
6
7This is intentionally outside consensus. It turns the audit root into something
8useful for a specific coin without changing what nodes accept.
9
10## Create A Proof
11
12Use an unspent output in `txid:vout` form:
13
14```bash
15python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json
16```
17
18The proof contains:
19
20- the target UTXO;
21- its sorted leaf index;
22- the leaf hash;
23- the Merkle sibling path;
24- the UTXO root and chain tip used to create it.
25
26It does not contain the full UTXO set.
27
28## Verify A Proof
29
30Verify against the root embedded in the proof:
31
32```bash
33python3 scripts/verify-utxo-proof.py utxo-proof.json
34```
35
36Or verify against a root copied from another machine, dashboard, block explorer,
37or future node export:
38
39```bash
40python3 scripts/verify-utxo-proof.py utxo-proof.json \
41 --root ROOT_FROM_REPORT_OR_EXPORT
42```
43
44The verifier does not need local `blk*.dat` files. It only needs the proof and
45the expected root.
46
47## Scope
48
49This first proof format supports inclusion only. Absence and range proofs are
50left for a later design because they require a more careful commitment scheme
51and are easier to get subtly wrong.
52
53The proof uses the exact same leaf and node domains as the UTXO commitment:
54
55```text
56BTFUTXO1
57BTFNODE1
58```
59
60That keeps the verifier language-neutral and gives future implementations a
61small, stable target.
diff --git a/scripts/prove-utxo.py b/scripts/prove-utxo.py
new file mode 100644
index 0000000..7a10d2a
--- /dev/null
+++ b/scripts/prove-utxo.py
@@ -0,0 +1,202 @@
1#!/usr/bin/env python3
2"""Create a Bitflash UTXO inclusion proof from local blk*.dat files.
3
4The proof is deliberately outside consensus. It proves that one txid:vout is
5included in the deterministic UTXO root produced by verify-utxo-set.py.
6"""
7
8import argparse
9import hashlib
10import importlib.util
11import json
12import os
13import sys
14from pathlib import Path
15
16
17SCHEMA = "bitflash-utxo-inclusion-proof-1"
18NODE_DOMAIN = b"BTFNODE1|"
19
20
21def load_utxo_tool():
22 path = Path(__file__).with_name("verify-utxo-set.py")
23 spec = importlib.util.spec_from_file_location("bitflash_verify_utxo_set", path)
24 if spec is None or spec.loader is None:
25 raise SystemExit("could not load %s" % path)
26 mod = importlib.util.module_from_spec(spec)
27 spec.loader.exec_module(mod)
28 return mod
29
30
31def canonical_bytes(obj):
32 return (json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii")
33
34
35def split_outpoint(outpoint):
36 parts = outpoint.split(":")
37 if len(parts) != 2 or len(parts[0]) != 64:
38 raise SystemExit("outpoint must be txid:vout")
39 try:
40 vout = int(parts[1], 10)
41 except ValueError:
42 raise SystemExit("vout must be a decimal integer")
43 if vout < 0:
44 raise SystemExit("vout must not be negative")
45 return parts[0].lower(), vout
46
47
48def build_merkle_proof(keys, leaf_hashes, index):
49 current_index = index
50 layer = list(leaf_hashes)
51 proof = []
52 level = 0
53 while len(layer) > 1:
54 if current_index % 2 == 0:
55 sibling_index = current_index + 1
56 duplicated = sibling_index >= len(layer)
57 sibling = layer[current_index] if duplicated else layer[sibling_index]
58 side = "right"
59 else:
60 sibling_index = current_index - 1
61 duplicated = False
62 sibling = layer[sibling_index]
63 side = "left"
64 proof.append({
65 "level": level,
66 "side": side,
67 "hash": sibling.hex(),
68 "duplicated": duplicated,
69 })
70
71 next_layer = []
72 for i in range(0, len(layer), 2):
73 left = layer[i]
74 right = layer[i + 1] if i + 1 < len(layer) else left
75 next_layer.append(hashlib.sha256(NODE_DOMAIN + left + right).digest())
76 layer = next_layer
77 current_index //= 2
78 level += 1
79 return proof, layer[0].hex()
80
81
82def build_proof(args):
83 tool = load_utxo_tool()
84 txid, vout = split_outpoint(args.outpoint)
85 outpoint = "%s:%d" % (txid, vout)
86
87 raw_blocks = tool.read_blocks(args.datadir, args.max_blocks)
88 if not raw_blocks:
89 raise SystemExit("no Bitflash blocks found in %s" % args.datadir)
90 blocks = tool.select_main_chain(raw_blocks)
91 state = tool.apply_blocks(blocks)
92 utxos = state["utxos"]
93 stats = state["stats"]
94 if outpoint not in utxos:
95 raise SystemExit("outpoint is not unspent in the scanned chain: %s" % outpoint)
96
97 keys = sorted(utxos)
98 leaf_hashes = [tool.utxo_leaf_hash(k, utxos[k]) for k in keys]
99 index = keys.index(outpoint)
100 proof_path, root = build_merkle_proof(keys, leaf_hashes, index)
101 direct_root = tool.utxo_root(utxos)
102 if root != direct_root:
103 raise SystemExit("internal proof root mismatch")
104
105 tip = blocks[-1]
106 utxo = utxos[outpoint]
107 report = {
108 "schema": SCHEMA,
109 "deterministic": True,
110 "chain": {
111 "name": "Bitflash",
112 "ticker": "BTF",
113 "message_start_hex": tool.MAGIC.hex(),
114 "coin": tool.COIN,
115 },
116 "commitment": {
117 "algorithm": "sorted-outpoint-merkle-sha256",
118 "leaf_domain": "BTFUTXO1",
119 "node_domain": "BTFNODE1",
120 "root": root,
121 "leaf_count": len(keys),
122 },
123 "target": {
124 "outpoint": outpoint,
125 "txid": txid,
126 "vout": vout,
127 "leaf_index": index,
128 "leaf_hash": leaf_hashes[index].hex(),
129 "utxo": utxo,
130 },
131 "proof": proof_path,
132 "scanned": {
133 "raw_block_count": len(raw_blocks),
134 "side_blocks_ignored": len(raw_blocks) - len(blocks),
135 "main_chain_block_count": len(blocks),
136 "first_height": 0,
137 "last_height": len(blocks) - 1,
138 "tip_hash": tip["hash"],
139 "tip_time": tip["time"],
140 "tip_time_utc": tip["time_utc"],
141 },
142 "warnings": {
143 "legacy_duplicate_output_overwrites": len(stats["duplicate_outputs"]),
144 },
145 "verdict": "UTXO_INCLUSION_PROOF_CREATED",
146 }
147 return report
148
149
150def print_text(proof):
151 data_hash = hashlib.sha256(canonical_bytes(proof)).hexdigest()
152 target = proof["target"]
153 utxo = target["utxo"]
154 print("Bitflash UTXO inclusion proof")
155 print("schema: %s" % proof["schema"])
156 print("canonical proof sha256: %s" % data_hash)
157 print()
158 print("Target")
159 print(" outpoint: %s" % target["outpoint"])
160 print(" value: %s BTF" % utxo["value_btf"])
161 print(" height: %d" % utxo["created_height"])
162 print(" index: %d of %d" % (target["leaf_index"], proof["commitment"]["leaf_count"]))
163 print()
164 print("Commitment")
165 print(" root: %s" % proof["commitment"]["root"])
166 print(" path: %d sibling hashes" % len(proof["proof"]))
167 print(" tip: %s" % proof["scanned"]["tip_hash"])
168 print()
169 print("Verdict: %s" % proof["verdict"])
170
171
172def default_datadir():
173 if os.name == "nt" and os.environ.get("APPDATA"):
174 return os.path.join(os.environ["APPDATA"], "Bitflash")
175 if os.environ.get("HOME"):
176 return os.path.join(os.environ["HOME"], ".bitflash")
177 return "."
178
179
180def main(argv):
181 ap = argparse.ArgumentParser(description=__doc__)
182 ap.add_argument("outpoint", help="unspent output to prove, as txid:vout")
183 ap.add_argument("--datadir", default=default_datadir(), help="Bitflash data directory")
184 ap.add_argument("--max-blocks", type=int, default=0,
185 help="number of blocks to scan; 0 means all block files")
186 ap.add_argument("--json", action="store_true", help="print canonical JSON")
187 ap.add_argument("--out", help="write canonical JSON proof to this path")
188 args = ap.parse_args(argv)
189
190 proof = build_proof(args)
191 data = canonical_bytes(proof)
192 if args.out:
193 Path(args.out).write_bytes(data)
194 if args.json:
195 sys.stdout.buffer.write(data)
196 else:
197 print_text(proof)
198 return 0
199
200
201if __name__ == "__main__":
202 raise SystemExit(main(sys.argv[1:]))
diff --git a/scripts/verify-utxo-proof.py b/scripts/verify-utxo-proof.py
new file mode 100644
index 0000000..31099f7
--- /dev/null
+++ b/scripts/verify-utxo-proof.py
@@ -0,0 +1,163 @@
1#!/usr/bin/env python3
2"""Verify a Bitflash UTXO inclusion proof against a UTXO root."""
3
4import argparse
5import hashlib
6import json
7import sys
8from pathlib import Path
9
10
11SCHEMA = "bitflash-utxo-inclusion-proof-1"
12LEAF_DOMAIN = "BTFUTXO1"
13NODE_DOMAIN = b"BTFNODE1|"
14
15
16def canonical_bytes(obj):
17 return (json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii")
18
19
20def sha256(data):
21 return hashlib.sha256(data).digest()
22
23
24def utxo_leaf_hash(outpoint, utxo):
25 payload = "|".join([
26 LEAF_DOMAIN,
27 outpoint,
28 str(utxo["value_satoshis"]),
29 utxo["script_pub_key_hex"],
30 str(utxo["created_height"]),
31 utxo["created_txid"],
32 ]).encode("ascii")
33 return sha256(payload)
34
35
36def load_proof(path):
37 try:
38 return json.loads(Path(path).read_text(encoding="ascii"))
39 except Exception as e:
40 raise SystemExit("could not read proof: %s" % e)
41
42
43def verify(proof, expected_root=None):
44 failures = []
45 if proof.get("schema") != SCHEMA:
46 failures.append("unexpected schema")
47 commitment = proof.get("commitment", {})
48 if commitment.get("algorithm") != "sorted-outpoint-merkle-sha256":
49 failures.append("unexpected commitment algorithm")
50 if commitment.get("leaf_domain") != LEAF_DOMAIN:
51 failures.append("unexpected leaf domain")
52 if commitment.get("node_domain") != "BTFNODE1":
53 failures.append("unexpected node domain")
54
55 target = proof.get("target", {})
56 outpoint = target.get("outpoint", "")
57 utxo = target.get("utxo", {})
58 if outpoint != "%s:%s" % (target.get("txid"), target.get("vout")):
59 failures.append("target outpoint does not match txid:vout")
60
61 try:
62 current = utxo_leaf_hash(outpoint, utxo)
63 leaf_ok = target.get("leaf_hash") == current.hex()
64 except Exception as e:
65 failures.append("could not hash target leaf: %s" % e)
66 current = b"\x00" * 32
67 leaf_ok = False
68
69 if not leaf_ok:
70 failures.append("target leaf hash mismatch")
71
72 for i, step in enumerate(proof.get("proof", [])):
73 try:
74 sibling = bytes.fromhex(step["hash"])
75 except Exception:
76 failures.append("bad sibling hash at proof level %d" % i)
77 sibling = b"\x00" * 32
78 if len(sibling) != 32:
79 failures.append("bad sibling hash length at proof level %d" % i)
80 side = step.get("side")
81 if side == "right":
82 current = sha256(NODE_DOMAIN + current + sibling)
83 elif side == "left":
84 current = sha256(NODE_DOMAIN + sibling + current)
85 else:
86 failures.append("bad sibling side at proof level %d" % i)
87
88 computed_root = current.hex()
89 root = expected_root or commitment.get("root")
90 if computed_root != root:
91 failures.append("computed root does not match expected root")
92
93 return {
94 "schema": "bitflash-utxo-proof-verification-1",
95 "deterministic": True,
96 "target": {
97 "outpoint": outpoint,
98 "value_satoshis": utxo.get("value_satoshis"),
99 "value_btf": utxo.get("value_btf"),
100 "created_height": utxo.get("created_height"),
101 },
102 "commitment": {
103 "expected_root": root,
104 "computed_root": computed_root,
105 "leaf_hash": target.get("leaf_hash"),
106 "path_length": len(proof.get("proof", [])),
107 },
108 "checks": {
109 "schema_is_supported": proof.get("schema") == SCHEMA,
110 "leaf_hash_matches_target": leaf_ok,
111 "root_matches": computed_root == root,
112 },
113 "failures": failures,
114 "verdict": "UTXO_PROOF_VERIFIED" if not failures else "UTXO_PROOF_FAILED",
115 }
116
117
118def print_text(result):
119 data_hash = hashlib.sha256(canonical_bytes(result)).hexdigest()
120 print("Bitflash UTXO proof verifier")
121 print("schema: %s" % result["schema"])
122 print("canonical verification sha256: %s" % data_hash)
123 print()
124 print("Target")
125 print(" outpoint: %s" % result["target"]["outpoint"])
126 print(" value: %s BTF" % result["target"]["value_btf"])
127 print(" height: %s" % result["target"]["created_height"])
128 print()
129 print("Commitment")
130 print(" expected: %s" % result["commitment"]["expected_root"])
131 print(" computed: %s" % result["commitment"]["computed_root"])
132 print(" path: %d sibling hashes" % result["commitment"]["path_length"])
133 print()
134 print("Checks")
135 for key in sorted(result["checks"]):
136 print(" %s %s" % ("OK " if result["checks"][key] else "FAIL", key))
137 if result["failures"]:
138 print()
139 print("Failures")
140 for failure in result["failures"]:
141 print(" - %s" % failure)
142 print()
143 print("Verdict: %s" % result["verdict"])
144
145
146def main(argv):
147 ap = argparse.ArgumentParser(description=__doc__)
148 ap.add_argument("proof", help="canonical JSON proof from prove-utxo.py")
149 ap.add_argument("--root", help="expected UTXO root; defaults to root embedded in proof")
150 ap.add_argument("--json", action="store_true", help="print canonical JSON")
151 args = ap.parse_args(argv)
152
153 result = verify(load_proof(args.proof), args.root)
154 data = canonical_bytes(result)
155 if args.json:
156 sys.stdout.buffer.write(data)
157 else:
158 print_text(result)
159 return 0 if result["verdict"] == "UTXO_PROOF_VERIFIED" else 1
160
161
162if __name__ == "__main__":
163 raise SystemExit(main(sys.argv[1:]))