upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--.github/workflows/ci.yml3
-rw-r--r--README.md8
-rw-r--r--docs/fair-launch.md3
-rw-r--r--docs/utxo-commitment.md20
-rw-r--r--scripts/bitflash_chain.py450
-rwxr-xr-xscripts/build-explorer.py385
-rw-r--r--scripts/prove-utxo.py50
-rw-r--r--scripts/verify-fair-launch.py346
-rw-r--r--scripts/verify-utxo-proof.py29
-rw-r--r--scripts/verify-utxo-set.py375
-rw-r--r--tests/scripts/test_bitflash_tools.py107
11 files changed, 902 insertions, 874 deletions
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index ba8074c..f4045f4 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -25,6 +25,9 @@ jobs:
25 - name: Run Linux selftests 25 - name: Run Linux selftests
26 run: make tests 26 run: make tests
27 27
28 - name: Run Python tool regressions
29 run: python3 tests/scripts/test_bitflash_tools.py
30
28 - name: Run P2P message fuzz smoke 31 - name: Run P2P message fuzz smoke
29 run: make fuzz-net-message-smoke 32 run: make fuzz-net-message-smoke
30 33
diff --git a/README.md b/README.md
index c28b0f1..9c6db0e 100644
--- a/README.md
+++ b/README.md
@@ -104,6 +104,14 @@ and produces canonical JSON whose hash can be compared by independent auditors
104or timestamped externally with OpenTimestamps. See 104or timestamped externally with OpenTimestamps. See
105[fair-launch verification](docs/fair-launch.md). 105[fair-launch verification](docs/fair-launch.md).
106 106
107For a static local block explorer, generate HTML and JSON from either a data
108directory or explicit block files:
109
110```bash
111python3 scripts/build-explorer.py --datadir ~/.bitflash ./explorer-out
112python3 scripts/build-explorer.py ~/.bitflash/blk0001.dat ~/.bitflash/blk0002.dat ./explorer-out
113```
114
107**Keep your node current.** Consensus rules have changed since the first 115**Keep your node current.** Consensus rules have changed since the first
108releases — 1.2.1 fixed a bug that let anyone spend anyone's coins, and 1.2.2 116releases — 1.2.1 fixed a bug that let anyone spend anyone's coins, and 1.2.2
109added a per-block signature-operation cap. A node on an older build will accept 117added a per-block signature-operation cap. A node on an older build will accept
diff --git a/docs/fair-launch.md b/docs/fair-launch.md
index 79eca8e..9603910 100644
--- a/docs/fair-launch.md
+++ b/docs/fair-launch.md
@@ -8,7 +8,8 @@ python3 scripts/verify-fair-launch.py --max-blocks 1000 --out bitflash-fair-laun
8 8
9The verifier reads `blk*.dat` directly from the local Bitflash data directory. 9The verifier reads `blk*.dat` directly from the local Bitflash data directory.
10It does not use Berkeley DB, does not call the node binary, and does not need 10It does not use Berkeley DB, does not call the node binary, and does not need
11network access. 11network access. It uses the same shared Python chain parser as the UTXO
12commitment and proof tools, so block serialization fixes land in one place.
12 13
13The report proves the launch baseline that can be checked from the chain: 14The report proves the launch baseline that can be checked from the chain:
14 15
diff --git a/docs/utxo-commitment.md b/docs/utxo-commitment.md
index 991e5d9..ab43deb 100644
--- a/docs/utxo-commitment.md
+++ b/docs/utxo-commitment.md
@@ -21,7 +21,8 @@ python3 scripts/verify-utxo-set.py --out utxo-report.json --json
21 21
22The verifier: 22The verifier:
23 23
24- parses Bitflash block files directly; 24- parses Bitflash block files directly with the shared Python chain parser;
25- streams `blk*.dat` records instead of loading full block files into memory;
25- reconstructs the best chain rooted at the Bitflash genesis block; 26- reconstructs the best chain rooted at the Bitflash genesis block;
26- ignores side blocks that may also be present in `blk*.dat`; 27- ignores side blocks that may also be present in `blk*.dat`;
27- verifies every scanned block merkle root; 28- verifies every scanned block merkle root;
@@ -98,6 +99,23 @@ last-write-wins semantics for the UTXO map, matching the legacy index model.
98Those warnings are part of the report so auditors can see the historical debt 99Those warnings are part of the report so auditors can see the historical debt
99instead of relying on a polished summary. 100instead of relying on a polished summary.
100 101
102For synthetic chains or future clean consensus ranges, auditors can make
103duplicates a hard failure:
104
105```bash
106python3 scripts/verify-utxo-set.py --strict-duplicates
107```
108
109The default remains compatible with the current historical chain because it has
110known legacy duplicate-output overwrites.
111
112## Shared Parser
113
114The audit scripts share `scripts/bitflash_chain.py` for block parsing,
115main-chain selection, UTXO application, canonical JSON, and genesis
116reconstruction. That keeps `verify-utxo-set.py`, `verify-fair-launch.py`,
117`prove-utxo.py`, and the explorer on the same serialization rules.
118
101## Future Consensus Path 119## Future Consensus Path
102 120
103This tool is the low-risk first step toward consensus commitments. A future 121This tool is the low-risk first step toward consensus commitments. A future
diff --git a/scripts/bitflash_chain.py b/scripts/bitflash_chain.py
new file mode 100644
index 0000000..4a3231c
--- /dev/null
+++ b/scripts/bitflash_chain.py
@@ -0,0 +1,450 @@
1#!/usr/bin/env python3
2"""Shared Bitflash block/chain parser for deterministic audit tools."""
3
4import datetime as _dt
5import hashlib
6import json
7import mmap
8import os
9import struct
10from pathlib import Path
11
12
13MAGIC = bytes([0xBF, 0x20, 0x5C, 0xFD])
14MAX_BLOCK_PAYLOAD = 32 * 1024 * 1024
15UTXO_LEAF_DOMAIN = "BTFUTXO1"
16UTXO_NODE_DOMAIN_TAG = "BTFNODE1"
17UTXO_NODE_DOMAIN = (UTXO_NODE_DOMAIN_TAG + "|").encode("ascii")
18COIN = 100_000_000
19GENESIS_HASH = "5bd7cb255d814e48cebcdfb72da4dc87b34bd774227f8ceb546c5640f4bdc169"
20GENESIS_MERKLE = "1a45b4482532abb29b10e234d3f13132230525a339ecea91658ffa675a5b1325"
21GENESIS_TIME = 1_753_315_200
22GENESIS_BITS = 0x1F0FFFFF
23GENESIS_NONCE = 3141
24GENESIS_TEXT = "Bitflash 24/Jul/2026 Fair launch: one CPU one vote, no premine"
25
26
27class ParseError(Exception):
28 pass
29
30
31class Reader:
32 def __init__(self, data):
33 self.data = data
34 self.pos = 0
35
36 def take(self, n):
37 if self.pos + n > len(self.data):
38 raise ParseError("unexpected end of data")
39 out = self.data[self.pos:self.pos + n]
40 self.pos += n
41 return out
42
43 def u8(self):
44 return self.take(1)[0]
45
46 def u32(self):
47 return struct.unpack("<I", self.take(4))[0]
48
49 def i32(self):
50 return struct.unpack("<i", self.take(4))[0]
51
52 def i64(self):
53 return struct.unpack("<q", self.take(8))[0]
54
55
56def sha256(data):
57 return hashlib.sha256(data).digest()
58
59
60def sha256d(data):
61 return hashlib.sha256(hashlib.sha256(data).digest()).digest()
62
63
64def hash_hex(serialized):
65 return sha256d(serialized)[::-1].hex()
66
67
68def compact_size(r):
69 first = r.u8()
70 if first < 253:
71 return first
72 if first == 253:
73 return struct.unpack("<H", r.take(2))[0]
74 if first == 254:
75 return r.u32()
76 return struct.unpack("<Q", r.take(8))[0]
77
78
79def read_varbytes(r):
80 return r.take(compact_size(r))
81
82
83def ser_compact_size(n):
84 if n < 253:
85 return bytes([n])
86 if n <= 0xFFFF:
87 return b"\xfd" + struct.pack("<H", n)
88 if n <= 0xFFFFFFFF:
89 return b"\xfe" + struct.pack("<I", n)
90 return b"\xff" + struct.pack("<Q", n)
91
92
93def ser_varbytes(b):
94 return ser_compact_size(len(b)) + b
95
96
97def ser_tx(tx):
98 out = bytearray()
99 out += struct.pack("<i", tx["version"])
100 out += ser_compact_size(len(tx["vin"]))
101 for txin in tx["vin"]:
102 out += bytes.fromhex(txin["prevout_hash"])[::-1]
103 out += struct.pack("<I", txin["prevout_n"])
104 out += ser_varbytes(bytes.fromhex(txin["script_sig_hex"]))
105 out += struct.pack("<I", txin["sequence"])
106 out += ser_compact_size(len(tx["vout"]))
107 for txout in tx["vout"]:
108 out += struct.pack("<q", txout["value_satoshis"])
109 out += ser_varbytes(bytes.fromhex(txout["script_pub_key_hex"]))
110 out += struct.pack("<I", tx["lock_time"])
111 return bytes(out)
112
113
114def parse_tx(r):
115 start = r.pos
116 tx = {"version": r.i32(), "vin": [], "vout": [], "lock_time": 0}
117 for _ in range(compact_size(r)):
118 tx["vin"].append({
119 "prevout_hash": r.take(32)[::-1].hex(),
120 "prevout_n": r.u32(),
121 "script_sig_hex": read_varbytes(r).hex(),
122 "sequence": r.u32(),
123 })
124 for _ in range(compact_size(r)):
125 value = r.i64()
126 tx["vout"].append({
127 "value_satoshis": value,
128 "value_btf": format_money(value),
129 "script_pub_key_hex": read_varbytes(r).hex(),
130 })
131 tx["lock_time"] = r.u32()
132 raw = r.data[start:r.pos]
133 tx["txid"] = hash_hex(raw)
134 return tx
135
136
137def merkle_root(txids):
138 if not txids:
139 return "0" * 64
140 layer = [bytes.fromhex(x)[::-1] for x in txids]
141 while len(layer) > 1:
142 nxt = []
143 for i in range(0, len(layer), 2):
144 left = layer[i]
145 right = layer[i + 1] if i + 1 < len(layer) else left
146 nxt.append(sha256d(left + right))
147 layer = nxt
148 return layer[0][::-1].hex()
149
150
151def parse_block(payload, file_name="", offset=0):
152 r = Reader(payload)
153 header = r.take(80)
154 version, = struct.unpack("<i", header[0:4])
155 prev = header[4:36][::-1].hex()
156 merkle = header[36:68][::-1].hex()
157 ntime, bits, nonce = struct.unpack("<III", header[68:80])
158 txs = [parse_tx(r) for _ in range(compact_size(r))]
159 if r.pos != len(payload):
160 raise ParseError("trailing bytes inside block")
161 txids = [tx["txid"] for tx in txs]
162 return {
163 "hash": hash_hex(header),
164 "version": version,
165 "previous_hash": prev,
166 "merkle_root": merkle,
167 "computed_merkle_root": merkle_root(txids),
168 "time": ntime,
169 "time_utc": utc(ntime),
170 "bits": "0x%08x" % bits,
171 "nonce": nonce,
172 "tx_count": len(txs),
173 "transactions": txs,
174 "source_file": file_name,
175 "source_offset": offset,
176 "size": len(payload),
177 }
178
179
180def iter_block_files(datadir):
181 for path in sorted(Path(datadir).glob("blk*.dat")):
182 yield path
183
184
185def iter_blocks_from_file(path):
186 path = Path(path)
187 with path.open("rb") as f:
188 f.seek(0, os.SEEK_END)
189 file_size = f.tell()
190 if file_size == 0:
191 return
192 f.seek(0)
193 mm = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
194 try:
195 pos = 0
196 while True:
197 offset = mm.find(MAGIC, pos)
198 if offset < 0 or offset + 8 > file_size:
199 break
200 size = struct.unpack("<I", mm[offset + 4:offset + 8])[0]
201 payload_offset = offset + 8
202 end = payload_offset + size
203 if size == 0 or size > MAX_BLOCK_PAYLOAD or end > file_size:
204 pos = offset + 1
205 continue
206 payload = mm[payload_offset:end]
207 try:
208 yield parse_block(payload, path.name, payload_offset)
209 except ParseError:
210 pos = offset + 1
211 continue
212 pos = end
213 finally:
214 mm.close()
215
216
217def read_blocks_from_files(paths, max_blocks=0):
218 blocks = []
219 for path in paths:
220 for block in iter_blocks_from_file(path):
221 blocks.append(block)
222 if max_blocks and len(blocks) >= max_blocks:
223 return blocks
224 return blocks
225
226
227def read_blocks(datadir, max_blocks=0):
228 return read_blocks_from_files(iter_block_files(datadir), max_blocks)
229
230
231def select_main_chain(blocks):
232 by_hash = {}
233 for block in blocks:
234 by_hash.setdefault(block["hash"], block)
235 if GENESIS_HASH not in by_hash:
236 raise ParseError("genesis block not found in block files")
237
238 children = {}
239 for block in by_hash.values():
240 children.setdefault(block["previous_hash"], []).append(block)
241 for v in children.values():
242 v.sort(key=lambda b: (b["hash"], b["source_file"], b["source_offset"]))
243
244 best = []
245 stack = [(by_hash[GENESIS_HASH], [by_hash[GENESIS_HASH]])]
246 while stack:
247 block, path = stack.pop()
248 if len(path) > len(best) or (len(path) == len(best) and block["hash"] < best[-1]["hash"]):
249 best = path
250 for child in children.get(block["hash"], []):
251 stack.append((child, path + [child]))
252 return best
253
254
255def is_coinbase(tx):
256 return (
257 len(tx["vin"]) == 1
258 and tx["vin"][0]["prevout_hash"] == "0" * 64
259 and tx["vin"][0]["prevout_n"] == 0xFFFFFFFF
260 )
261
262
263def utxo_leaf_hash(outpoint, utxo):
264 payload = "|".join([
265 UTXO_LEAF_DOMAIN,
266 outpoint,
267 str(utxo["value_satoshis"]),
268 utxo["script_pub_key_hex"],
269 str(utxo["created_height"]),
270 utxo["created_txid"],
271 ]).encode("ascii")
272 return sha256(payload)
273
274
275def utxo_root(utxos):
276 leaves = [utxo_leaf_hash(k, utxos[k]) for k in sorted(utxos)]
277 if not leaves:
278 return sha256(b"BTFUTXO1|empty").hex()
279 layer = leaves
280 while len(layer) > 1:
281 nxt = []
282 for i in range(0, len(layer), 2):
283 left = layer[i]
284 right = layer[i + 1] if i + 1 < len(layer) else left
285 nxt.append(sha256(UTXO_NODE_DOMAIN + left + right))
286 layer = nxt
287 return layer[0].hex()
288
289
290def apply_blocks(blocks, strict_duplicates=False):
291 utxos = {}
292 spent = 0
293 missing_spends = []
294 duplicate_outputs = []
295 total_coinbase = 0
296 total_fees = 0
297 normal_txs = 0
298
299 for height, block in enumerate(blocks):
300 if block["computed_merkle_root"] != block["merkle_root"]:
301 raise ParseError("bad merkle root at height %d" % height)
302 for tx in block["transactions"]:
303 coinbase = is_coinbase(tx)
304 input_value = 0
305 output_value = sum(out["value_satoshis"] for out in tx["vout"])
306 if coinbase:
307 total_coinbase += output_value
308 else:
309 normal_txs += 1
310 for txin in tx["vin"]:
311 prev = "%s:%d" % (txin["prevout_hash"], txin["prevout_n"])
312 prev_utxo = utxos.pop(prev, None)
313 if prev_utxo is None:
314 missing_spends.append({
315 "height": height,
316 "txid": tx["txid"],
317 "missing_outpoint": prev,
318 })
319 else:
320 spent += prev_utxo["value_satoshis"]
321 input_value += prev_utxo["value_satoshis"]
322 if input_value >= output_value:
323 total_fees += input_value - output_value
324 for n, out in enumerate(tx["vout"]):
325 key = "%s:%d" % (tx["txid"], n)
326 if key in utxos:
327 if strict_duplicates:
328 raise ParseError("duplicate output at height %d: %s" % (height, key))
329 duplicate_outputs.append({"height": height, "outpoint": key})
330 utxos[key] = {
331 "txid": tx["txid"],
332 "vout": n,
333 "value_satoshis": out["value_satoshis"],
334 "value_btf": out["value_btf"],
335 "script_pub_key_hex": out["script_pub_key_hex"],
336 "created_height": height,
337 "created_block_hash": block["hash"],
338 "created_txid": tx["txid"],
339 "coinbase": coinbase,
340 }
341
342 total_unspent = sum(u["value_satoshis"] for u in utxos.values())
343 return {
344 "utxos": utxos,
345 "stats": {
346 "utxo_count": len(utxos),
347 "spent_outputs": spent,
348 "total_unspent_satoshis": total_unspent,
349 "total_unspent_btf": format_money(total_unspent),
350 "total_coinbase_satoshis": total_coinbase,
351 "total_coinbase_btf": format_money(total_coinbase),
352 "coinbase_minus_unspent_satoshis": total_coinbase - total_unspent,
353 "coinbase_minus_unspent_btf": format_money(total_coinbase - total_unspent),
354 "total_fees_satoshis": total_fees,
355 "total_fees_btf": format_money(total_fees),
356 "normal_transactions": normal_txs,
357 "missing_spends": missing_spends,
358 "duplicate_outputs": duplicate_outputs,
359 },
360 }
361
362
363def genesis_transaction():
364 script_sig = (
365 bytes.fromhex("04ffff001d")
366 + b"\x01\x04"
367 + bytes([len(GENESIS_TEXT)])
368 + GENESIS_TEXT.encode("ascii")
369 )
370 pubkey_num = bytes.fromhex(
371 "5f1df16b2b704c8a578d0bbaf74d385cde12c11ee50455f3c438ef4c3fbcf649"
372 "b6de611feae06279a60939e028a8d65c10b73071a6f16719274855feb0fd8a6704"
373 )
374 script_pub_key = bytes([65]) + pubkey_num[::-1] + b"\xac"
375 tx = {
376 "version": 1,
377 "vin": [{
378 "prevout_hash": "0" * 64,
379 "prevout_n": 0xFFFFFFFF,
380 "script_sig_hex": script_sig.hex(),
381 "sequence": 0xFFFFFFFF,
382 }],
383 "vout": [{
384 "value_satoshis": 50 * COIN,
385 "value_btf": format_money(50 * COIN),
386 "script_pub_key_hex": script_pub_key.hex(),
387 }],
388 "lock_time": 0,
389 }
390 tx["txid"] = hash_hex(ser_tx(tx))
391 return tx
392
393
394def reconstructed_genesis():
395 tx = genesis_transaction()
396 header = (
397 struct.pack("<i", 1)
398 + bytes(32)
399 + bytes.fromhex(tx["txid"])[::-1]
400 + struct.pack("<III", GENESIS_TIME, GENESIS_BITS, GENESIS_NONCE)
401 )
402 return {
403 "hash": hash_hex(header),
404 "version": 1,
405 "previous_hash": "0" * 64,
406 "merkle_root": tx["txid"],
407 "computed_merkle_root": tx["txid"],
408 "time": GENESIS_TIME,
409 "time_utc": utc(GENESIS_TIME),
410 "bits": "0x%08x" % GENESIS_BITS,
411 "nonce": GENESIS_NONCE,
412 "tx_count": 1,
413 "transactions": [tx],
414 "source_file": "reconstructed-from-source-constants",
415 "source_offset": 0,
416 "size": 80 + len(ser_compact_size(1)) + len(ser_tx(tx)),
417 }
418
419
420def ser_block(block):
421 header = (
422 struct.pack("<i", block["version"])
423 + bytes.fromhex(block["previous_hash"])[::-1]
424 + bytes.fromhex(block["merkle_root"])[::-1]
425 + struct.pack("<III", block["time"], int(block["bits"], 16), block["nonce"])
426 )
427 txs = block["transactions"]
428 return header + ser_compact_size(len(txs)) + b"".join(ser_tx(tx) for tx in txs)
429
430
431def format_money(value):
432 sign = "-" if value < 0 else ""
433 value = abs(value)
434 return "%s%d.%08d" % (sign, value // COIN, value % COIN)
435
436
437def utc(ts):
438 return _dt.datetime.fromtimestamp(ts, _dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
439
440
441def default_datadir():
442 if os.name == "nt" and os.environ.get("APPDATA"):
443 return os.path.join(os.environ["APPDATA"], "Bitflash")
444 if os.environ.get("HOME"):
445 return os.path.join(os.environ["HOME"], ".bitflash")
446 return "."
447
448
449def canonical_bytes(report):
450 return (json.dumps(report, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii")
diff --git a/scripts/build-explorer.py b/scripts/build-explorer.py
index 7849e8a..97a7f4e 100755
--- a/scripts/build-explorer.py
+++ b/scripts/build-explorer.py
@@ -1,134 +1,164 @@
1#!/usr/bin/env python3 1#!/usr/bin/env python3
2# 2"""Build a static Bitflash block explorer from local blk*.dat files."""
3# Build a static Bitflash block explorer (v0) from a node's blk0001.dat.
4# Emits into <out-dir>:
5# blocks.json summary of every main-chain block (newest first)
6# block/<height>.json full detail per block (header + all txs)
7# index.html self-contained viewer (vanilla JS, same-origin fetch)
8#
9# Scope (v0, deliberately): block list, block detail, and coinbase. No mempool,
10# no per-address history. On this chain the coinbase payout key rotates every
11# block, so an address index would be nearly useless; block-level transparency is
12# what lets miners and users confirm the chain is honest.
13#
14# Usage: build-explorer.py <blk0001.dat> <out-dir>
15
16import sys, os, json, hashlib, struct
17
18MAGIC = bytes([0xbf, 0x20, 0x5c, 0xfd])
19ADDRESS_VERSION = 25 # Bitflash addresses start with "B"
20
21def sha256d(b): return hashlib.sha256(hashlib.sha256(b).digest()).digest()
22 3
4import argparse
5import hashlib
6import json
7import sys
8from pathlib import Path
9
10import bitflash_chain as chain
11
12
13ADDRESS_VERSION = 25
23B58 = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz" 14B58 = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz"
15
16
24def b58check(payload): 17def b58check(payload):
25 v = bytes([ADDRESS_VERSION]) + payload 18 v = bytes([ADDRESS_VERSION]) + payload
26 v += sha256d(v)[:4] 19 v += chain.sha256d(v)[:4]
27 n = int.from_bytes(v, 'big') 20 n = int.from_bytes(v, "big")
28 s = "" 21 s = ""
29 while n > 0: 22 while n > 0:
30 n, r = divmod(n, 58); s = B58[r] + s 23 n, r = divmod(n, 58)
24 s = B58[r] + s
31 for byte in v: 25 for byte in v:
32 if byte == 0: s = "1" + s 26 if byte == 0:
33 else: break 27 s = "1" + s
28 else:
29 break
34 return s 30 return s
35 31
32
36def hash160(b): 33def hash160(b):
37 return hashlib.new('ripemd160', hashlib.sha256(b).digest()).digest() 34 return hashlib.new("ripemd160", hashlib.sha256(b).digest()).digest()
38 35
39class R: 36
40 def __init__(s, b, o=0): s.b = b; s.o = o 37def spk_info(script_hex):
41 def u(s, n): v = s.b[s.o:s.o+n]; s.o += n; return v 38 try:
42 def i32(s): return struct.unpack('<i', s.u(4))[0] 39 spk = bytes.fromhex(script_hex)
43 def u32(s): return struct.unpack('<I', s.u(4))[0] 40 except ValueError:
44 def i64(s): return struct.unpack('<q', s.u(8))[0] 41 return {"type": "malformed", "hex": script_hex[:80]}
45 def cs(s): 42 if (
46 c = s.b[s.o]; s.o += 1 43 len(spk) == 25 and spk[0] == 0x76 and spk[1] == 0xA9
47 if c < 253: return c 44 and spk[2] == 0x14 and spk[23] == 0x88 and spk[24] == 0xAC
48 if c == 253: return struct.unpack('<H', s.u(2))[0] 45 ):
49 if c == 254: return struct.unpack('<I', s.u(4))[0]
50 return struct.unpack('<Q', s.u(8))[0]
51 def script(s): n = s.cs(); return s.u(n)
52
53def spk_info(spk):
54 # P2PKH: OP_DUP OP_HASH160 <20> OP_EQUALVERIFY OP_CHECKSIG
55 if len(spk) == 25 and spk[0] == 0x76 and spk[1] == 0xa9 and spk[2] == 0x14 and spk[23] == 0x88 and spk[24] == 0xac:
56 return {"type": "p2pkh", "address": b58check(spk[3:23])} 46 return {"type": "p2pkh", "address": b58check(spk[3:23])}
57 # P2PK: <pubkey> OP_CHECKSIG (65- or 33-byte key) 47 if len(spk) >= 35 and spk[-1] == 0xAC:
58 if len(spk) >= 35 and spk[-1] == 0xac:
59 if spk[0] == 0x41 and len(spk) == 67: 48 if spk[0] == 0x41 and len(spk) == 67:
60 pk = spk[1:66]; return {"type": "p2pk", "pubkey": pk.hex(), "address": b58check(hash160(pk))} 49 pk = spk[1:66]
50 return {"type": "p2pk", "pubkey": pk.hex(), "address": b58check(hash160(pk))}
61 if spk[0] == 0x21 and len(spk) == 35: 51 if spk[0] == 0x21 and len(spk) == 35:
62 pk = spk[1:34]; return {"type": "p2pk", "pubkey": pk.hex(), "address": b58check(hash160(pk))} 52 pk = spk[1:34]
63 return {"type": "other", "hex": spk.hex()[:80]} 53 return {"type": "p2pk", "pubkey": pk.hex(), "address": b58check(hash160(pk))}
64 54 return {"type": "other", "hex": script_hex[:80]}
65def parse_tx(r): 55
66 start = r.o 56
67 r.i32() 57def explorer_tx(tx):
68 nin = r.cs(); vin = [] 58 vin = []
69 for _ in range(nin): 59 for txin in tx["vin"]:
70 ph = r.u(32); idx = r.u32(); r.script(); r.u32() 60 coinbase = (
71 coinbase = (ph == b'\x00' * 32 and idx == 0xffffffff) 61 txin["prevout_hash"] == "0" * 64
72 vin.append({"coinbase": coinbase, "prev": (None if coinbase else ph[::-1].hex()), "vout": (None if coinbase else idx)}) 62 and txin["prevout_n"] == 0xFFFFFFFF
73 nout = r.cs(); vout = [] 63 )
74 for _ in range(nout): 64 vin.append({
75 val = r.i64(); spk = r.script(); vout.append(dict({"value": val}, **spk_info(spk))) 65 "coinbase": coinbase,
76 r.u32() 66 "prev": None if coinbase else txin["prevout_hash"],
77 raw = r.b[start:r.o] 67 "vout": None if coinbase else txin["prevout_n"],
78 return {"txid": sha256d(raw)[::-1].hex(), "vin": vin, "vout": vout, "size": len(raw)} 68 })
79 69 vout = []
80def main(): 70 for out in tx["vout"]:
81 if len(sys.argv) != 3: 71 vout.append(dict({"value": out["value_satoshis"]}, **spk_info(out["script_pub_key_hex"])))
82 print("usage: build-explorer.py <blk0001.dat> <out-dir>", file=sys.stderr); sys.exit(2) 72 return {"txid": tx["txid"], "vin": vin, "vout": vout, "size": len(chain.ser_tx(tx))}
83 blkpath, out = sys.argv[1], sys.argv[2] 73
84 data = open(blkpath, 'rb').read() 74
85 blocks = {}; o = 0; n = len(data) 75def parse_args(argv):
86 while o + 8 <= n: 76 ap = argparse.ArgumentParser(description=__doc__)
87 if data[o:o+4] != MAGIC: o += 1; continue 77 ap.add_argument("--datadir", help="Bitflash data directory containing blk*.dat")
88 o += 4; size = struct.unpack('<I', data[o:o+4])[0]; o += 4 78 ap.add_argument("--max-blocks", type=int, default=0,
89 blk = data[o:o+size]; o += size 79 help="number of blocks to scan; 0 means all block files")
90 if len(blk) < 80: continue 80 ap.add_argument("paths", nargs="+",
91 h = sha256d(blk[:80])[::-1].hex() 81 help="with --datadir: <out-dir>; otherwise: <blk*.dat>... <out-dir>")
92 r = R(blk, 0) 82 args = ap.parse_args(argv)
93 ver = r.i32(); prev = r.u(32)[::-1].hex(); merk = r.u(32)[::-1].hex() 83 if args.datadir:
94 t = r.u32(); bits = r.u32(); nonce = r.u32() 84 if len(args.paths) != 1:
95 ntx = r.cs(); txs = [] 85 raise SystemExit("usage with --datadir: build-explorer.py --datadir DATADIR <out-dir>")
96 try: 86 args.block_files = None
97 for _ in range(ntx): txs.append(parse_tx(r)) 87 args.out_dir = args.paths[0]
98 except Exception: 88 else:
99 continue 89 if len(args.paths) < 2:
100 blocks[h] = {"hash": h, "prev": prev, "merkleRoot": merk, "version": ver, 90 raise SystemExit("usage: build-explorer.py <blk*.dat>... <out-dir>")
101 "time": t, "bits": bits, "nonce": nonce, "size": size, "txs": txs} 91 args.block_files = args.paths[:-1]
102 if not blocks: 92 args.out_dir = args.paths[-1]
103 print("no blocks parsed", file=sys.stderr); sys.exit(1) 93 return args
104 94
105 hc = {} 95
106 def height(h): 96def load_blocks(args):
107 chain = []; cur = h 97 if args.datadir:
108 while cur in blocks and cur not in hc: 98 return chain.read_blocks(args.datadir, args.max_blocks)
109 chain.append(cur); cur = blocks[cur]["prev"] 99 return chain.read_blocks_from_files(args.block_files, args.max_blocks)
110 base = hc.get(cur, 0) 100
111 for hh in reversed(chain): base += 1; hc[hh] = base 101
112 return hc[h] 102def build_explorer(raw_blocks, out_dir):
113 for h in blocks: height(h) 103 try:
114 tip = max(blocks, key=lambda h: hc[h]) 104 blocks = chain.select_main_chain(raw_blocks)
115 chain = []; cur = tip 105 except chain.ParseError as e:
116 while cur in blocks: chain.append(cur); cur = blocks[cur]["prev"] 106 raise SystemExit(str(e))
117 chain.reverse() 107 out = Path(out_dir)
118 108 (out / "block").mkdir(parents=True, exist_ok=True)
119 os.makedirs(os.path.join(out, "block"), exist_ok=True) 109
120 summaries = [] 110 summaries = []
121 for h in chain: 111 for height, block in enumerate(blocks):
122 b = blocks[h]; H = hc[h] 112 txs = [explorer_tx(tx) for tx in block["transactions"]]
123 cb = b["txs"][0]["vout"][0] if b["txs"] and b["txs"][0]["vout"] else {} 113 detail = {
124 summaries.append({"height": H, "hash": h, "time": b["time"], "nTx": len(b["txs"]), 114 "height": height,
125 "size": b["size"], "cbValue": cb.get("value", 0), "cbAddr": cb.get("address", "")}) 115 "hash": block["hash"],
126 json.dump(dict({"height": H}, **b), open(os.path.join(out, "block", "%d.json" % H), "w"), separators=(',', ':')) 116 "prev": block["previous_hash"],
117 "merkleRoot": block["merkle_root"],
118 "version": block["version"],
119 "time": block["time"],
120 "bits": int(block["bits"], 16),
121 "nonce": block["nonce"],
122 "size": block["size"],
123 "txs": txs,
124 }
125 cb = txs[0]["vout"][0] if txs and txs[0]["vout"] else {}
126 summaries.append({
127 "height": height,
128 "hash": block["hash"],
129 "time": block["time"],
130 "nTx": len(txs),
131 "size": block["size"],
132 "cbValue": cb.get("value", 0),
133 "cbAddr": cb.get("address", ""),
134 })
135 (out / "block" / ("%d.json" % height)).write_text(
136 json.dumps(detail, sort_keys=True, separators=(",", ":")) + "\n",
137 encoding="ascii",
138 )
139
127 summaries.sort(key=lambda x: -x["height"]) 140 summaries.sort(key=lambda x: -x["height"])
128 json.dump({"network": "bitflash", "tipHeight": hc[tip], "count": len(chain), "blocks": summaries}, 141 (out / "blocks.json").write_text(
129 open(os.path.join(out, "blocks.json"), "w"), separators=(',', ':')) 142 json.dumps({
130 open(os.path.join(out, "index.html"), "w").write(INDEX_HTML) 143 "network": "bitflash",
131 print("explorer built: %d main-chain blocks, tip %d, into %s" % (len(chain), hc[tip], out)) 144 "tipHeight": len(blocks) - 1,
145 "count": len(blocks),
146 "blocks": summaries,
147 }, sort_keys=True, separators=(",", ":")) + "\n",
148 encoding="ascii",
149 )
150 (out / "index.html").write_text(INDEX_HTML, encoding="utf-8")
151 return len(blocks), len(blocks) - 1
152
153
154def main(argv):
155 args = parse_args(argv)
156 raw_blocks = load_blocks(args)
157 if not raw_blocks:
158 raise SystemExit("no Bitflash blocks parsed")
159 count, tip = build_explorer(raw_blocks, args.out_dir)
160 print("explorer built: %d main-chain blocks, tip %d, into %s" % (count, tip, args.out_dir))
161 return 0
132 162
133 163
134INDEX_HTML = r"""<!doctype html> 164INDEX_HTML = r"""<!doctype html>
@@ -144,6 +174,7 @@ INDEX_HTML = r"""<!doctype html>
144 body { font: 15px/1.5 system-ui, sans-serif; margin:0; background:var(--bg); color:var(--fg); } 174 body { font: 15px/1.5 system-ui, sans-serif; margin:0; background:var(--bg); color:var(--fg); }
145 header { padding:20px 24px; border-bottom:1px solid var(--line); display:flex; gap:16px; align-items:baseline; flex-wrap:wrap; } 175 header { padding:20px 24px; border-bottom:1px solid var(--line); display:flex; gap:16px; align-items:baseline; flex-wrap:wrap; }
146 h1 { font-size:18px; margin:0; } 176 h1 { font-size:18px; margin:0; }
177 h2 { font-size:16px; }
147 .mut { color:var(--mut); } 178 .mut { color:var(--mut); }
148 .wrap { padding:16px 24px; max-width:1100px; margin:0 auto; } 179 .wrap { padding:16px 24px; max-width:1100px; margin:0 auto; }
149 input { width:100%; max-width:420px; padding:9px 12px; border:1px solid var(--line); border-radius:8px; background:var(--card); color:var(--fg); font-size:14px; } 180 input { width:100%; max-width:420px; padding:9px 12px; border:1px solid var(--line); border-radius:8px; background:var(--card); color:var(--fg); font-size:14px; }
@@ -154,7 +185,6 @@ INDEX_HTML = r"""<!doctype html>
154 tr.blk:hover td { background:var(--card); } 185 tr.blk:hover td { background:var(--card); }
155 code { font-size:12px; word-break:break-all; } 186 code { font-size:12px; word-break:break-all; }
156 .scroll { overflow-x:auto; } 187 .scroll { overflow-x:auto; }
157 a { color:var(--acc); text-decoration:none; }
158 .card { border:1px solid var(--line); border-radius:10px; background:var(--card); padding:14px 16px; margin-top:12px; } 188 .card { border:1px solid var(--line); border-radius:10px; background:var(--card); padding:14px 16px; margin-top:12px; }
159 .kv { display:grid; grid-template-columns:150px 1fr; gap:4px 12px; font-size:13px; } 189 .kv { display:grid; grid-template-columns:150px 1fr; gap:4px 12px; font-size:13px; }
160 .kv div:nth-child(odd){ color:var(--mut); } 190 .kv div:nth-child(odd){ color:var(--mut); }
@@ -174,7 +204,7 @@ INDEX_HTML = r"""<!doctype html>
174 <button class="link" id="home" style="display:none">back to blocks</button> 204 <button class="link" id="home" style="display:none">back to blocks</button>
175</header> 205</header>
176<div class="wrap"> 206<div class="wrap">
177 <p class="note">Read-only view of the main chain: blocks, their transactions, and the coinbase payout. There is no mempool and no per-address history here. On Bitflash the coinbase payout key rotates every block, so an address index would say almost nothing; what matters is that every block and every coinbase is here to check.</p> 207 <p class="note">Read-only view of the main chain: blocks, their transactions, and the coinbase payout. There is no mempool and no per-address history here.</p>
178 <div id="listview"> 208 <div id="listview">
179 <input id="q" placeholder="Search by height or block hash"> 209 <input id="q" placeholder="Search by height or block hash">
180 <div class="scroll"><table> 210 <div class="scroll"><table>
@@ -186,52 +216,114 @@ INDEX_HTML = r"""<!doctype html>
186 <div id="detail" style="display:none"></div> 216 <div id="detail" style="display:none"></div>
187</div> 217</div>
188<script> 218<script>
189var SUM = [], BYH = {}, SHOWN = 200; 219var SUM = [], SHOWN = 200;
190function btf(sat){ return (sat/1e8).toFixed(8).replace(/0+$/,'').replace(/\.$/,'')+" BTF"; } 220function btf(sat){ return (sat/1e8).toFixed(8).replace(/0+$/,'').replace(/\.$/,'')+" BTF"; }
191function ts(t){ return new Date(t*1000).toISOString().replace('T',' ').replace('.000Z',' UTC'); } 221function ts(t){ return new Date(t*1000).toISOString().replace('T',' ').replace('.000Z',' UTC'); }
192function esc(s){ return String(s).replace(/[&<>]/g,function(c){return {'&':'&amp;','<':'&lt;','>':'&gt;'}[c];}); } 222function short(h){ return h ? h.slice(0,10)+"..."+h.slice(-6) : "-"; }
193function short(h){ return h ? h.slice(0,10)+"…"+h.slice(-6) : "-"; } 223function E(tag, cls, text){
194 224 var e = document.createElement(tag);
225 if(cls) e.className = cls;
226 if(text !== undefined) e.textContent = text;
227 return e;
228}
229function code(text){ return E('code', '', text); }
230function clear(e){ while(e.firstChild) e.removeChild(e.firstChild); }
231function append(parent){
232 for(var i=1;i<arguments.length;i++) parent.appendChild(arguments[i]);
233 return parent;
234}
235function row(cells, cls){
236 var tr = E('tr', cls || '');
237 cells.forEach(function(c){ append(tr, E('td', '', c)); });
238 return tr;
239}
195function renderList(filter){ 240function renderList(filter){
196 var q=(filter||"").trim().toLowerCase(), rows=document.getElementById('rows'), out=""; 241 var q=(filter||"").trim().toLowerCase(), rows=document.getElementById('rows');
242 clear(rows);
197 var list = q ? SUM.filter(function(b){ return String(b.height)===q || b.hash.indexOf(q)===0; }) : SUM.slice(0,SHOWN); 243 var list = q ? SUM.filter(function(b){ return String(b.height)===q || b.hash.indexOf(q)===0; }) : SUM.slice(0,SHOWN);
198 for(var i=0;i<list.length;i++){ var b=list[i]; 244 if(!list.length){
199 out += '<tr class="blk" data-h="'+b.height+'"><td>'+b.height+'</td><td>'+ts(b.time)+'</td><td>'+b.nTx+'</td><td>'+b.size+' B</td><td>'+btf(b.cbValue)+'</td><td><code>'+(b.cbAddr?esc(b.cbAddr):'-')+'</code></td></tr>'; 245 var empty = E('tr');
246 var td = E('td', 'mut', 'no match');
247 td.colSpan = 6;
248 append(empty, td); append(rows, empty);
200 } 249 }
201 rows.innerHTML = out || '<tr><td colspan="6" class="mut">no match</td></tr>'; 250 list.forEach(function(b){
251 var tr = row([String(b.height), ts(b.time), String(b.nTx), b.size+' B', btf(b.cbValue), ''], 'blk');
252 tr.dataset.h = b.height;
253 clear(tr.children[5]);
254 append(tr.children[5], code(b.cbAddr || '-'));
255 tr.onclick = function(){ openBlock(tr.dataset.h); };
256 append(rows, tr);
257 });
202 document.getElementById('more').textContent = (!q && SUM.length>SHOWN) ? ("showing latest "+SHOWN+" of "+SUM.length+" blocks; search to find any") : ""; 258 document.getElementById('more').textContent = (!q && SUM.length>SHOWN) ? ("showing latest "+SHOWN+" of "+SUM.length+" blocks; search to find any") : "";
203 Array.prototype.forEach.call(rows.querySelectorAll('tr.blk'), function(tr){ tr.onclick=function(){ openBlock(tr.getAttribute('data-h')); }; });
204} 259}
205function ioSide(title, items, html){ 260function kv(parent, key, value, isCode){
206 var s='<div><div class="mut">'+title+'</div>'; for(var i=0;i<items.length;i++){ s+=html(items[i]); } return s+'</div>'; 261 append(parent, E('div', '', key));
262 append(parent, isCode ? append(E('div'), code(value)) : E('div', '', value));
263}
264function ioSide(title, items, render){
265 var side = E('div');
266 append(side, E('div', 'mut', title));
267 items.forEach(function(item){ append(side, render(item)); });
268 return side;
207} 269}
208function openBlock(h){ 270function openBlock(h){
209 fetch('block/'+h+'.json').then(function(r){return r.json();}).then(function(b){ 271 fetch('block/'+h+'.json').then(function(r){return r.json();}).then(function(b){
210 var s='<div class="card"><div class="kv">'+ 272 var detail = document.getElementById('detail');
211 '<div>Height</div><div>'+b.height+'</div>'+ 273 clear(detail);
212 '<div>Hash</div><div><code>'+esc(b.hash)+'</code></div>'+ 274 append(detail, E('h2', '', 'Block '+b.height));
213 '<div>Previous</div><div><button class="link" onclick="openBlock('+(b.height-1)+')"><code>'+esc(short(b.prev))+'</code></button></div>'+ 275 var card = E('div', 'card'), grid = E('div', 'kv');
214 '<div>Merkle root</div><div><code>'+esc(b.merkleRoot)+'</code></div>'+ 276 kv(grid, 'Height', String(b.height));
215 '<div>Time (UTC)</div><div>'+ts(b.time)+'</div>'+ 277 kv(grid, 'Hash', b.hash, true);
216 '<div>Bits / Nonce</div><div>'+b.bits.toString(16)+' / '+b.nonce+'</div>'+ 278 var prev = E('div');
217 '<div>Size / Txs</div><div>'+b.size+' B / '+b.txs.length+'</div>'+ 279 if(b.height > 0){
218 '</div></div>'; 280 var btn = E('button', 'link');
219 for(var i=0;i<b.txs.length;i++){ var t=b.txs[i]; 281 append(btn, code(short(b.prev)));
220 s+='<div class="tx"><div><code>'+esc(t.txid)+'</code> '+(i===0?'<span class="pill">coinbase</span>':'')+'</div><div class="io">'; 282 btn.onclick = function(){ openBlock(b.height - 1); };
221 s+=ioSide('Inputs', t.vin, function(v){ return '<div>'+(v.coinbase?'<span class="pill">coinbase</span>':'<code>'+esc(short(v.prev))+':'+v.vout+'</code>')+'</div>'; }); 283 append(prev, btn);
222 s+=ioSide('Outputs', t.vout, function(o){ var to=o.address?esc(o.address):(o.pubkey?'pubkey '+esc(o.pubkey.slice(0,16))+'…':esc(o.type)); return '<div>'+btf(o.value)+' → <code>'+to+'</code></div>'; }); 284 } else {
223 s+='</div></div>'; 285 append(prev, code(short(b.prev)));
224 } 286 }
287 append(grid, E('div', '', 'Previous'), prev);
288 kv(grid, 'Merkle root', b.merkleRoot, true);
289 kv(grid, 'Time (UTC)', ts(b.time));
290 kv(grid, 'Bits / Nonce', b.bits.toString(16)+' / '+b.nonce);
291 kv(grid, 'Size / Txs', b.size+' B / '+b.txs.length);
292 append(card, grid); append(detail, card);
293 b.txs.forEach(function(t, i){
294 var tx = E('div', 'tx');
295 var head = E('div');
296 append(head, code(t.txid));
297 if(i === 0) append(head, E('span', 'pill', 'coinbase'));
298 var io = E('div', 'io');
299 append(io, ioSide('Inputs', t.vin, function(v){
300 var d = E('div');
301 if(v.coinbase) append(d, E('span', 'pill', 'coinbase'));
302 else append(d, code(short(v.prev)+':'+v.vout));
303 return d;
304 }));
305 append(io, ioSide('Outputs', t.vout, function(o){
306 var d = E('div');
307 var to = o.address ? o.address : (o.pubkey ? 'pubkey '+o.pubkey.slice(0,16)+'...' : o.type);
308 append(d, document.createTextNode(btf(o.value)+' -> '), code(to));
309 return d;
310 }));
311 append(tx, head, io); append(detail, tx);
312 });
225 document.getElementById('listview').style.display='none'; 313 document.getElementById('listview').style.display='none';
226 var d=document.getElementById('detail'); d.style.display='block'; d.innerHTML='<h2 style="font-size:16px">Block '+b.height+'</h2>'+s; 314 detail.style.display='block';
227 document.getElementById('home').style.display='inline'; 315 document.getElementById('home').style.display='inline';
228 window.scrollTo(0,0); 316 window.scrollTo(0,0);
229 }); 317 });
230} 318}
231document.getElementById('home').onclick=function(){ document.getElementById('detail').style.display='none'; document.getElementById('listview').style.display='block'; this.style.display='none'; }; 319document.getElementById('home').onclick=function(){
320 document.getElementById('detail').style.display='none';
321 document.getElementById('listview').style.display='block';
322 this.style.display='none';
323};
232document.getElementById('q').oninput=function(){ renderList(this.value); }; 324document.getElementById('q').oninput=function(){ renderList(this.value); };
233fetch('blocks.json').then(function(r){return r.json();}).then(function(d){ 325fetch('blocks.json').then(function(r){return r.json();}).then(function(d){
234 SUM=d.blocks; document.getElementById('tip').textContent='tip height '+d.tipHeight+' · '+d.count+' blocks'; 326 SUM=d.blocks; document.getElementById('tip').textContent='tip height '+d.tipHeight+' - '+d.count+' blocks';
235 renderList(''); 327 renderList('');
236}); 328});
237</script> 329</script>
@@ -239,5 +331,6 @@ fetch('blocks.json').then(function(r){return r.json();}).then(function(d){
239</html> 331</html>
240""" 332"""
241 333
334
242if __name__ == "__main__": 335if __name__ == "__main__":
243 main() 336 raise SystemExit(main(sys.argv[1:]))
diff --git a/scripts/prove-utxo.py b/scripts/prove-utxo.py
index 7a10d2a..105e677 100644
--- a/scripts/prove-utxo.py
+++ b/scripts/prove-utxo.py
@@ -7,25 +7,13 @@ included in the deterministic UTXO root produced by verify-utxo-set.py.
7 7
8import argparse 8import argparse
9import hashlib 9import hashlib
10import importlib.util
11import json 10import json
12import os
13import sys 11import sys
14from pathlib import Path 12from pathlib import Path
15 13
14import bitflash_chain as chain
16 15
17SCHEMA = "bitflash-utxo-inclusion-proof-1" 16SCHEMA = "bitflash-utxo-inclusion-proof-1"
18NODE_DOMAIN = b"BTFNODE1|"
19
20
21def load_utxo_tool():
22 path = Path(__file__).with_name("verify-utxo-set.py")
23 spec = importlib.util.spec_from_file_location("bitflash_verify_utxo_set", path)
24 if spec is None or spec.loader is None:
25 raise SystemExit("could not load %s" % path)
26 mod = importlib.util.module_from_spec(spec)
27 spec.loader.exec_module(mod)
28 return mod
29 17
30 18
31def canonical_bytes(obj): 19def canonical_bytes(obj):
@@ -72,7 +60,7 @@ def build_merkle_proof(keys, leaf_hashes, index):
72 for i in range(0, len(layer), 2): 60 for i in range(0, len(layer), 2):
73 left = layer[i] 61 left = layer[i]
74 right = layer[i + 1] if i + 1 < len(layer) else left 62 right = layer[i + 1] if i + 1 < len(layer) else left
75 next_layer.append(hashlib.sha256(NODE_DOMAIN + left + right).digest()) 63 next_layer.append(hashlib.sha256(chain.UTXO_NODE_DOMAIN + left + right).digest())
76 layer = next_layer 64 layer = next_layer
77 current_index //= 2 65 current_index //= 2
78 level += 1 66 level += 1
@@ -80,25 +68,27 @@ def build_merkle_proof(keys, leaf_hashes, index):
80 68
81 69
82def build_proof(args): 70def build_proof(args):
83 tool = load_utxo_tool()
84 txid, vout = split_outpoint(args.outpoint) 71 txid, vout = split_outpoint(args.outpoint)
85 outpoint = "%s:%d" % (txid, vout) 72 outpoint = "%s:%d" % (txid, vout)
86 73
87 raw_blocks = tool.read_blocks(args.datadir, args.max_blocks) 74 raw_blocks = chain.read_blocks(args.datadir, args.max_blocks)
88 if not raw_blocks: 75 if not raw_blocks:
89 raise SystemExit("no Bitflash blocks found in %s" % args.datadir) 76 raise SystemExit("no Bitflash blocks found in %s" % args.datadir)
90 blocks = tool.select_main_chain(raw_blocks) 77 try:
91 state = tool.apply_blocks(blocks) 78 blocks = chain.select_main_chain(raw_blocks)
79 state = chain.apply_blocks(blocks, strict_duplicates=args.strict_duplicates)
80 except chain.ParseError as e:
81 raise SystemExit(str(e))
92 utxos = state["utxos"] 82 utxos = state["utxos"]
93 stats = state["stats"] 83 stats = state["stats"]
94 if outpoint not in utxos: 84 if outpoint not in utxos:
95 raise SystemExit("outpoint is not unspent in the scanned chain: %s" % outpoint) 85 raise SystemExit("outpoint is not unspent in the scanned chain: %s" % outpoint)
96 86
97 keys = sorted(utxos) 87 keys = sorted(utxos)
98 leaf_hashes = [tool.utxo_leaf_hash(k, utxos[k]) for k in keys] 88 leaf_hashes = [chain.utxo_leaf_hash(k, utxos[k]) for k in keys]
99 index = keys.index(outpoint) 89 index = keys.index(outpoint)
100 proof_path, root = build_merkle_proof(keys, leaf_hashes, index) 90 proof_path, root = build_merkle_proof(keys, leaf_hashes, index)
101 direct_root = tool.utxo_root(utxos) 91 direct_root = chain.utxo_root(utxos)
102 if root != direct_root: 92 if root != direct_root:
103 raise SystemExit("internal proof root mismatch") 93 raise SystemExit("internal proof root mismatch")
104 94
@@ -110,13 +100,13 @@ def build_proof(args):
110 "chain": { 100 "chain": {
111 "name": "Bitflash", 101 "name": "Bitflash",
112 "ticker": "BTF", 102 "ticker": "BTF",
113 "message_start_hex": tool.MAGIC.hex(), 103 "message_start_hex": chain.MAGIC.hex(),
114 "coin": tool.COIN, 104 "coin": chain.COIN,
115 }, 105 },
116 "commitment": { 106 "commitment": {
117 "algorithm": "sorted-outpoint-merkle-sha256", 107 "algorithm": "sorted-outpoint-merkle-sha256",
118 "leaf_domain": "BTFUTXO1", 108 "leaf_domain": chain.UTXO_LEAF_DOMAIN,
119 "node_domain": "BTFNODE1", 109 "node_domain": chain.UTXO_NODE_DOMAIN_TAG,
120 "root": root, 110 "root": root,
121 "leaf_count": len(keys), 111 "leaf_count": len(keys),
122 }, 112 },
@@ -169,20 +159,14 @@ def print_text(proof):
169 print("Verdict: %s" % proof["verdict"]) 159 print("Verdict: %s" % proof["verdict"])
170 160
171 161
172def default_datadir():
173 if os.name == "nt" and os.environ.get("APPDATA"):
174 return os.path.join(os.environ["APPDATA"], "Bitflash")
175 if os.environ.get("HOME"):
176 return os.path.join(os.environ["HOME"], ".bitflash")
177 return "."
178
179
180def main(argv): 162def main(argv):
181 ap = argparse.ArgumentParser(description=__doc__) 163 ap = argparse.ArgumentParser(description=__doc__)
182 ap.add_argument("outpoint", help="unspent output to prove, as txid:vout") 164 ap.add_argument("outpoint", help="unspent output to prove, as txid:vout")
183 ap.add_argument("--datadir", default=default_datadir(), help="Bitflash data directory") 165 ap.add_argument("--datadir", default=chain.default_datadir(), help="Bitflash data directory")
184 ap.add_argument("--max-blocks", type=int, default=0, 166 ap.add_argument("--max-blocks", type=int, default=0,
185 help="number of blocks to scan; 0 means all block files") 167 help="number of blocks to scan; 0 means all block files")
168 ap.add_argument("--strict-duplicates", action="store_true",
169 help="fail instead of using legacy last-write-wins duplicate output handling")
186 ap.add_argument("--json", action="store_true", help="print canonical JSON") 170 ap.add_argument("--json", action="store_true", help="print canonical JSON")
187 ap.add_argument("--out", help="write canonical JSON proof to this path") 171 ap.add_argument("--out", help="write canonical JSON proof to this path")
188 args = ap.parse_args(argv) 172 args = ap.parse_args(argv)
diff --git a/scripts/verify-fair-launch.py b/scripts/verify-fair-launch.py
index ce396fb..c33c8be 100644
--- a/scripts/verify-fair-launch.py
+++ b/scripts/verify-fair-launch.py
@@ -1,299 +1,12 @@
1#!/usr/bin/env python3 1#!/usr/bin/env python3
2"""Produce a deterministic Bitflash fair-launch audit report. 2"""Produce a deterministic Bitflash fair-launch audit report."""
3
4This verifier is intentionally self-contained. It parses Bitflash block files
5directly, without Berkeley DB, without the node binary, and without network
6access. The canonical JSON output is stable for the same inputs and can be
7timestamped later with OpenTimestamps or any other external notary.
8"""
9 3
10import argparse 4import argparse
11import datetime as _dt
12import hashlib 5import hashlib
13import json
14import os
15import struct
16import sys 6import sys
17from pathlib import Path 7from pathlib import Path
18 8
19 9import bitflash_chain as chain
20MAGIC = bytes([0xBF, 0x20, 0x5C, 0xFD])
21COIN = 100_000_000
22GENESIS_HASH = "5bd7cb255d814e48cebcdfb72da4dc87b34bd774227f8ceb546c5640f4bdc169"
23GENESIS_MERKLE = "1a45b4482532abb29b10e234d3f13132230525a339ecea91658ffa675a5b1325"
24GENESIS_TIME = 1_753_315_200
25GENESIS_BITS = 0x1F0FFFFF
26GENESIS_NONCE = 3141
27GENESIS_TEXT = "Bitflash 24/Jul/2026 Fair launch: one CPU one vote, no premine"
28
29
30class ParseError(Exception):
31 pass
32
33
34class Reader:
35 def __init__(self, data):
36 self.data = data
37 self.pos = 0
38
39 def take(self, n):
40 if self.pos + n > len(self.data):
41 raise ParseError("unexpected end of data")
42 out = self.data[self.pos:self.pos + n]
43 self.pos += n
44 return out
45
46 def u8(self):
47 return self.take(1)[0]
48
49 def u32(self):
50 return struct.unpack("<I", self.take(4))[0]
51
52 def i32(self):
53 return struct.unpack("<i", self.take(4))[0]
54
55 def i64(self):
56 return struct.unpack("<q", self.take(8))[0]
57
58
59def sha256d(data):
60 return hashlib.sha256(hashlib.sha256(data).digest()).digest()
61
62
63def hash_hex(serialized):
64 return sha256d(serialized)[::-1].hex()
65
66
67def compact_size(r):
68 first = r.u8()
69 if first < 253:
70 return first
71 if first == 253:
72 return struct.unpack("<H", r.take(2))[0]
73 if first == 254:
74 return r.u32()
75 return struct.unpack("<Q", r.take(8))[0]
76
77
78def read_varbytes(r):
79 return r.take(compact_size(r))
80
81
82def ser_compact_size(n):
83 if n < 253:
84 return bytes([n])
85 if n <= 0xFFFF:
86 return b"\xfd" + struct.pack("<H", n)
87 if n <= 0xFFFFFFFF:
88 return b"\xfe" + struct.pack("<I", n)
89 return b"\xff" + struct.pack("<Q", n)
90
91
92def ser_varbytes(b):
93 return ser_compact_size(len(b)) + b
94
95
96def ser_tx(tx):
97 out = bytearray()
98 out += struct.pack("<i", tx["version"])
99 out += ser_compact_size(len(tx["vin"]))
100 for txin in tx["vin"]:
101 out += bytes.fromhex(txin["prevout_hash"])[::-1]
102 out += struct.pack("<I", txin["prevout_n"])
103 out += ser_varbytes(bytes.fromhex(txin["script_sig_hex"]))
104 out += struct.pack("<I", txin["sequence"])
105 out += ser_compact_size(len(tx["vout"]))
106 for txout in tx["vout"]:
107 out += struct.pack("<q", txout["value_satoshis"])
108 out += ser_varbytes(bytes.fromhex(txout["script_pub_key_hex"]))
109 out += struct.pack("<I", tx["lock_time"])
110 return bytes(out)
111
112
113def parse_tx(r):
114 start = r.pos
115 tx = {"version": r.i32(), "vin": [], "vout": [], "lock_time": 0}
116 for _ in range(compact_size(r)):
117 prev_hash = r.take(32)[::-1].hex()
118 prev_n = r.u32()
119 script = read_varbytes(r).hex()
120 sequence = r.u32()
121 tx["vin"].append({
122 "prevout_hash": prev_hash,
123 "prevout_n": prev_n,
124 "script_sig_hex": script,
125 "sequence": sequence,
126 })
127 for _ in range(compact_size(r)):
128 value = r.i64()
129 script = read_varbytes(r).hex()
130 tx["vout"].append({
131 "value_satoshis": value,
132 "value_btf": format_money(value),
133 "script_pub_key_hex": script,
134 })
135 tx["lock_time"] = r.u32()
136 raw = r.data[start:r.pos]
137 tx["txid"] = hash_hex(raw)
138 return tx
139
140
141def merkle_root(txids):
142 if not txids:
143 return "0" * 64
144 layer = [bytes.fromhex(x)[::-1] for x in txids]
145 while len(layer) > 1:
146 nxt = []
147 for i in range(0, len(layer), 2):
148 left = layer[i]
149 right = layer[i + 1] if i + 1 < len(layer) else left
150 nxt.append(sha256d(left + right))
151 layer = nxt
152 return layer[0][::-1].hex()
153
154
155def parse_block(payload, file_name="", offset=0):
156 r = Reader(payload)
157 header = r.take(80)
158 version, = struct.unpack("<i", header[0:4])
159 prev = header[4:36][::-1].hex()
160 merkle = header[36:68][::-1].hex()
161 ntime, bits, nonce = struct.unpack("<III", header[68:80])
162 txs = [parse_tx(r) for _ in range(compact_size(r))]
163 if r.pos != len(payload):
164 raise ParseError("trailing bytes inside block")
165 txids = [tx["txid"] for tx in txs]
166 return {
167 "hash": hash_hex(header),
168 "version": version,
169 "previous_hash": prev,
170 "merkle_root": merkle,
171 "computed_merkle_root": merkle_root(txids),
172 "time": ntime,
173 "time_utc": utc(ntime),
174 "bits": "0x%08x" % bits,
175 "nonce": nonce,
176 "tx_count": len(txs),
177 "transactions": txs,
178 "source_file": file_name,
179 "source_offset": offset,
180 }
181
182
183def iter_block_files(datadir):
184 for path in sorted(Path(datadir).glob("blk*.dat")):
185 yield path
186
187
188def read_blocks(datadir, limit):
189 blocks = []
190 for path in iter_block_files(datadir):
191 data = path.read_bytes()
192 pos = 0
193 while True:
194 idx = data.find(MAGIC, pos)
195 if idx < 0:
196 break
197 if idx + 8 > len(data):
198 break
199 size = struct.unpack("<I", data[idx + 4:idx + 8])[0]
200 start = idx + 8
201 end = start + size
202 if end > len(data):
203 break
204 blocks.append(parse_block(data[start:end], path.name, start))
205 if limit and len(blocks) >= limit:
206 return blocks
207 pos = end
208 return blocks
209
210
211def genesis_transaction():
212 script_sig = (
213 bytes.fromhex("04ffff001d")
214 + b"\x01\x04"
215 + bytes([len(GENESIS_TEXT)])
216 + GENESIS_TEXT.encode("ascii")
217 )
218 pubkey_num = bytes.fromhex(
219 "5f1df16b2b704c8a578d0bbaf74d385cde12c11ee50455f3c438ef4c3fbcf649"
220 "b6de611feae06279a60939e028a8d65c10b73071a6f16719274855feb0fd8a6704"
221 )
222 # The genesis source uses CScript << CBigNum("0x..."), and CBigNum::getvch()
223 # returns the minimal little-endian script number representation.
224 script_pub_key = bytes([65]) + pubkey_num[::-1] + b"\xac"
225 tx = {
226 "version": 1,
227 "vin": [{
228 "prevout_hash": "0" * 64,
229 "prevout_n": 0xFFFFFFFF,
230 "script_sig_hex": script_sig.hex(),
231 "sequence": 0xFFFFFFFF,
232 }],
233 "vout": [{
234 "value_satoshis": 50 * COIN,
235 "value_btf": format_money(50 * COIN),
236 "script_pub_key_hex": script_pub_key.hex(),
237 }],
238 "lock_time": 0,
239 }
240 tx["txid"] = hash_hex(ser_tx(tx))
241 return tx
242
243
244def reconstructed_genesis():
245 tx = genesis_transaction()
246 header = (
247 struct.pack("<i", 1)
248 + bytes(32)
249 + bytes.fromhex(tx["txid"])[::-1]
250 + struct.pack("<III", GENESIS_TIME, GENESIS_BITS, GENESIS_NONCE)
251 )
252 return {
253 "hash": hash_hex(header),
254 "version": 1,
255 "previous_hash": "0" * 64,
256 "merkle_root": tx["txid"],
257 "computed_merkle_root": tx["txid"],
258 "time": GENESIS_TIME,
259 "time_utc": utc(GENESIS_TIME),
260 "bits": "0x%08x" % GENESIS_BITS,
261 "nonce": GENESIS_NONCE,
262 "tx_count": 1,
263 "transactions": [tx],
264 "source_file": "reconstructed-from-source-constants",
265 "source_offset": 0,
266 }
267
268
269def format_money(value):
270 sign = "-" if value < 0 else ""
271 value = abs(value)
272 return "%s%d.%08d" % (sign, value // COIN, value % COIN)
273
274
275def utc(ts):
276 return _dt.datetime.fromtimestamp(ts, _dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
277
278
279def default_datadir():
280 if os.name == "nt" and os.environ.get("APPDATA"):
281 return os.path.join(os.environ["APPDATA"], "Bitflash")
282 if os.environ.get("HOME"):
283 return os.path.join(os.environ["HOME"], ".bitflash")
284 return "."
285
286
287def is_coinbase(tx):
288 return (
289 len(tx["vin"]) == 1
290 and tx["vin"][0]["prevout_hash"] == "0" * 64
291 and tx["vin"][0]["prevout_n"] == 0xFFFFFFFF
292 )
293
294
295def canonical_bytes(report):
296 return (json.dumps(report, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii")
297 10
298 11
299def build_report(blocks, source_mode): 12def build_report(blocks, source_mode):
@@ -307,7 +20,7 @@ def build_report(blocks, source_mode):
307 if not block["transactions"]: 20 if not block["transactions"]:
308 continue 21 continue
309 cb = block["transactions"][0] 22 cb = block["transactions"][0]
310 if is_coinbase(cb): 23 if chain.is_coinbase(cb):
311 amount = sum(out["value_satoshis"] for out in cb["vout"]) 24 amount = sum(out["value_satoshis"] for out in cb["vout"])
312 total_mined += amount 25 total_mined += amount
313 coinbase_outputs.append({ 26 coinbase_outputs.append({
@@ -315,18 +28,18 @@ def build_report(blocks, source_mode):
315 "block_hash": block["hash"], 28 "block_hash": block["hash"],
316 "coinbase_txid": cb["txid"], 29 "coinbase_txid": cb["txid"],
317 "value_satoshis": amount, 30 "value_satoshis": amount,
318 "value_btf": format_money(amount), 31 "value_btf": chain.format_money(amount),
319 "outputs": cb["vout"], 32 "outputs": cb["vout"],
320 }) 33 })
321 non_coinbase_txs += sum(1 for tx in block["transactions"] if not is_coinbase(tx)) 34 non_coinbase_txs += sum(1 for tx in block["transactions"] if not chain.is_coinbase(tx))
322 35
323 checks = { 36 checks = {
324 "genesis_hash_matches_source": genesis["hash"] == GENESIS_HASH, 37 "genesis_hash_matches_source": genesis["hash"] == chain.GENESIS_HASH,
325 "genesis_merkle_matches_source": genesis["merkle_root"] == GENESIS_MERKLE, 38 "genesis_merkle_matches_source": genesis["merkle_root"] == chain.GENESIS_MERKLE,
326 "computed_merkle_matches_header": genesis["computed_merkle_root"] == genesis["merkle_root"], 39 "computed_merkle_matches_header": genesis["computed_merkle_root"] == genesis["merkle_root"],
327 "genesis_has_single_transaction": genesis["tx_count"] == 1, 40 "genesis_has_single_transaction": genesis["tx_count"] == 1,
328 "genesis_transaction_is_coinbase": bool(genesis["transactions"] and is_coinbase(genesis["transactions"][0])), 41 "genesis_transaction_is_coinbase": bool(genesis["transactions"] and chain.is_coinbase(genesis["transactions"][0])),
329 "genesis_supply_is_50_btf": sum(out["value_satoshis"] for out in launch_outputs) == 50 * COIN, 42 "genesis_supply_is_50_btf": sum(out["value_satoshis"] for out in launch_outputs) == 50 * chain.COIN,
330 } 43 }
331 44
332 report = { 45 report = {
@@ -336,17 +49,17 @@ def build_report(blocks, source_mode):
336 "chain": { 49 "chain": {
337 "name": "Bitflash", 50 "name": "Bitflash",
338 "ticker": "BTF", 51 "ticker": "BTF",
339 "message_start_hex": MAGIC.hex(), 52 "message_start_hex": chain.MAGIC.hex(),
340 "coin": COIN, 53 "coin": chain.COIN,
341 }, 54 },
342 "source_constants": { 55 "source_constants": {
343 "genesis_hash": GENESIS_HASH, 56 "genesis_hash": chain.GENESIS_HASH,
344 "genesis_merkle_root": GENESIS_MERKLE, 57 "genesis_merkle_root": chain.GENESIS_MERKLE,
345 "genesis_time": GENESIS_TIME, 58 "genesis_time": chain.GENESIS_TIME,
346 "genesis_time_utc": utc(GENESIS_TIME), 59 "genesis_time_utc": chain.utc(chain.GENESIS_TIME),
347 "genesis_bits": "0x%08x" % GENESIS_BITS, 60 "genesis_bits": "0x%08x" % chain.GENESIS_BITS,
348 "genesis_nonce": GENESIS_NONCE, 61 "genesis_nonce": chain.GENESIS_NONCE,
349 "genesis_message": GENESIS_TEXT, 62 "genesis_message": chain.GENESIS_TEXT,
350 }, 63 },
351 "scanned": { 64 "scanned": {
352 "block_count": len(blocks), 65 "block_count": len(blocks),
@@ -354,7 +67,7 @@ def build_report(blocks, source_mode):
354 "last_height": len(blocks) - 1, 67 "last_height": len(blocks) - 1,
355 "non_coinbase_transactions": non_coinbase_txs, 68 "non_coinbase_transactions": non_coinbase_txs,
356 "total_coinbase_satoshis": total_mined, 69 "total_coinbase_satoshis": total_mined,
357 "total_coinbase_btf": format_money(total_mined), 70 "total_coinbase_btf": chain.format_money(total_mined),
358 }, 71 },
359 "genesis": { 72 "genesis": {
360 "hash": genesis["hash"], 73 "hash": genesis["hash"],
@@ -381,7 +94,7 @@ def build_report(blocks, source_mode):
381 ], 94 ],
382 }, 95 },
383 } 96 }
384 report["attestation_payload_sha256"] = hashlib.sha256(canonical_bytes(report)).hexdigest() 97 report["attestation_payload_sha256"] = hashlib.sha256(chain.canonical_bytes(report)).hexdigest()
385 return report 98 return report
386 99
387 100
@@ -397,7 +110,7 @@ def print_text(report):
397 print(" merkle: %s" % report["genesis"]["merkle_root"]) 110 print(" merkle: %s" % report["genesis"]["merkle_root"])
398 print(" time: %s" % report["genesis"]["time_utc"]) 111 print(" time: %s" % report["genesis"]["time_utc"])
399 print(" txs: %d" % report["genesis"]["tx_count"]) 112 print(" txs: %d" % report["genesis"]["tx_count"])
400 print(" supply: %s BTF" % format_money(sum(o["value_satoshis"] for o in report["genesis"]["outputs"]))) 113 print(" supply: %s BTF" % chain.format_money(sum(o["value_satoshis"] for o in report["genesis"]["outputs"])))
401 print() 114 print()
402 print("Scanned") 115 print("Scanned")
403 print(" blocks: %d" % report["scanned"]["block_count"]) 116 print(" blocks: %d" % report["scanned"]["block_count"])
@@ -417,7 +130,7 @@ def print_text(report):
417 130
418def main(argv): 131def main(argv):
419 ap = argparse.ArgumentParser(description=__doc__) 132 ap = argparse.ArgumentParser(description=__doc__)
420 ap.add_argument("--datadir", default=default_datadir(), help="Bitflash data directory") 133 ap.add_argument("--datadir", default=chain.default_datadir(), help="Bitflash data directory")
421 ap.add_argument("--max-blocks", type=int, default=100, help="number of blocks to scan from blk*.dat") 134 ap.add_argument("--max-blocks", type=int, default=100, help="number of blocks to scan from blk*.dat")
422 ap.add_argument("--json", action="store_true", help="print canonical JSON") 135 ap.add_argument("--json", action="store_true", help="print canonical JSON")
423 ap.add_argument("--out", help="write canonical JSON to this path") 136 ap.add_argument("--out", help="write canonical JSON to this path")
@@ -426,18 +139,19 @@ def main(argv):
426 args = ap.parse_args(argv) 139 args = ap.parse_args(argv)
427 140
428 if args.reconstruct_genesis: 141 if args.reconstruct_genesis:
429 blocks = [reconstructed_genesis()] 142 blocks = [chain.reconstructed_genesis()]
430 source_mode = "reconstructed-genesis" 143 source_mode = "reconstructed-genesis"
431 datadir = ""
432 else: 144 else:
433 datadir = Path(args.datadir) 145 blocks = chain.read_blocks(args.datadir, args.max_blocks)
434 blocks = read_blocks(datadir, args.max_blocks)
435 if not blocks: 146 if not blocks:
436 raise SystemExit("no Bitflash blocks found in %s; use --reconstruct-genesis for a source-only check" % datadir) 147 raise SystemExit("no Bitflash blocks found in %s; use --reconstruct-genesis for a source-only check" % args.datadir)
437 source_mode = "local-block-files" 148 source_mode = "local-block-files"
438 149
439 report = build_report(blocks, source_mode) 150 try:
440 data = canonical_bytes(report) 151 report = build_report(blocks, source_mode)
152 except chain.ParseError as e:
153 raise SystemExit(str(e))
154 data = chain.canonical_bytes(report)
441 155
442 if args.out: 156 if args.out:
443 Path(args.out).write_bytes(data) 157 Path(args.out).write_bytes(data)
diff --git a/scripts/verify-utxo-proof.py b/scripts/verify-utxo-proof.py
index 31099f7..50d9356 100644
--- a/scripts/verify-utxo-proof.py
+++ b/scripts/verify-utxo-proof.py
@@ -7,32 +7,15 @@ import json
7import sys 7import sys
8from pathlib import Path 8from pathlib import Path
9 9
10import bitflash_chain as chain
10 11
11SCHEMA = "bitflash-utxo-inclusion-proof-1" 12SCHEMA = "bitflash-utxo-inclusion-proof-1"
12LEAF_DOMAIN = "BTFUTXO1"
13NODE_DOMAIN = b"BTFNODE1|"
14 13
15 14
16def canonical_bytes(obj): 15def canonical_bytes(obj):
17 return (json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii") 16 return (json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii")
18 17
19 18
20def sha256(data):
21 return hashlib.sha256(data).digest()
22
23
24def utxo_leaf_hash(outpoint, utxo):
25 payload = "|".join([
26 LEAF_DOMAIN,
27 outpoint,
28 str(utxo["value_satoshis"]),
29 utxo["script_pub_key_hex"],
30 str(utxo["created_height"]),
31 utxo["created_txid"],
32 ]).encode("ascii")
33 return sha256(payload)
34
35
36def load_proof(path): 19def load_proof(path):
37 try: 20 try:
38 return json.loads(Path(path).read_text(encoding="ascii")) 21 return json.loads(Path(path).read_text(encoding="ascii"))
@@ -47,9 +30,9 @@ def verify(proof, expected_root=None):
47 commitment = proof.get("commitment", {}) 30 commitment = proof.get("commitment", {})
48 if commitment.get("algorithm") != "sorted-outpoint-merkle-sha256": 31 if commitment.get("algorithm") != "sorted-outpoint-merkle-sha256":
49 failures.append("unexpected commitment algorithm") 32 failures.append("unexpected commitment algorithm")
50 if commitment.get("leaf_domain") != LEAF_DOMAIN: 33 if commitment.get("leaf_domain") != chain.UTXO_LEAF_DOMAIN:
51 failures.append("unexpected leaf domain") 34 failures.append("unexpected leaf domain")
52 if commitment.get("node_domain") != "BTFNODE1": 35 if commitment.get("node_domain") != chain.UTXO_NODE_DOMAIN_TAG:
53 failures.append("unexpected node domain") 36 failures.append("unexpected node domain")
54 37
55 target = proof.get("target", {}) 38 target = proof.get("target", {})
@@ -59,7 +42,7 @@ def verify(proof, expected_root=None):
59 failures.append("target outpoint does not match txid:vout") 42 failures.append("target outpoint does not match txid:vout")
60 43
61 try: 44 try:
62 current = utxo_leaf_hash(outpoint, utxo) 45 current = chain.utxo_leaf_hash(outpoint, utxo)
63 leaf_ok = target.get("leaf_hash") == current.hex() 46 leaf_ok = target.get("leaf_hash") == current.hex()
64 except Exception as e: 47 except Exception as e:
65 failures.append("could not hash target leaf: %s" % e) 48 failures.append("could not hash target leaf: %s" % e)
@@ -79,9 +62,9 @@ def verify(proof, expected_root=None):
79 failures.append("bad sibling hash length at proof level %d" % i) 62 failures.append("bad sibling hash length at proof level %d" % i)
80 side = step.get("side") 63 side = step.get("side")
81 if side == "right": 64 if side == "right":
82 current = sha256(NODE_DOMAIN + current + sibling) 65 current = chain.sha256(chain.UTXO_NODE_DOMAIN + current + sibling)
83 elif side == "left": 66 elif side == "left":
84 current = sha256(NODE_DOMAIN + sibling + current) 67 current = chain.sha256(chain.UTXO_NODE_DOMAIN + sibling + current)
85 else: 68 else:
86 failures.append("bad sibling side at proof level %d" % i) 69 failures.append("bad sibling side at proof level %d" % i)
87 70
diff --git a/scripts/verify-utxo-set.py b/scripts/verify-utxo-set.py
index 12b024f..b5bf3d5 100644
--- a/scripts/verify-utxo-set.py
+++ b/scripts/verify-utxo-set.py
@@ -1,363 +1,24 @@
1#!/usr/bin/env python3 1#!/usr/bin/env python3
2"""Build a deterministic Bitflash UTXO-set commitment from local blk*.dat. 2"""Build a deterministic Bitflash UTXO-set commitment from local blk*.dat."""
3
4This is deliberately outside consensus. It lets auditors reconstruct the chain
5state from disk blocks and compare the same canonical UTXO root independently.
6"""
7 3
8import argparse 4import argparse
9import datetime as _dt
10import hashlib 5import hashlib
11import json
12import os
13import struct
14import sys 6import sys
15from pathlib import Path 7from pathlib import Path
16 8
17 9import bitflash_chain as chain
18MAGIC = bytes([0xBF, 0x20, 0x5C, 0xFD])
19COIN = 100_000_000
20GENESIS_HASH = "5bd7cb255d814e48cebcdfb72da4dc87b34bd774227f8ceb546c5640f4bdc169"
21GENESIS_MERKLE = "1a45b4482532abb29b10e234d3f13132230525a339ecea91658ffa675a5b1325"
22
23
24class ParseError(Exception):
25 pass
26
27
28class Reader:
29 def __init__(self, data):
30 self.data = data
31 self.pos = 0
32
33 def take(self, n):
34 if self.pos + n > len(self.data):
35 raise ParseError("unexpected end of data")
36 out = self.data[self.pos:self.pos + n]
37 self.pos += n
38 return out
39
40 def u8(self):
41 return self.take(1)[0]
42
43 def u32(self):
44 return struct.unpack("<I", self.take(4))[0]
45
46 def i32(self):
47 return struct.unpack("<i", self.take(4))[0]
48
49 def i64(self):
50 return struct.unpack("<q", self.take(8))[0]
51
52
53def sha256(data):
54 return hashlib.sha256(data).digest()
55
56
57def sha256d(data):
58 return hashlib.sha256(hashlib.sha256(data).digest()).digest()
59
60
61def hash_hex(serialized):
62 return sha256d(serialized)[::-1].hex()
63
64
65def compact_size(r):
66 first = r.u8()
67 if first < 253:
68 return first
69 if first == 253:
70 return struct.unpack("<H", r.take(2))[0]
71 if first == 254:
72 return r.u32()
73 return struct.unpack("<Q", r.take(8))[0]
74
75
76def read_varbytes(r):
77 return r.take(compact_size(r))
78
79
80def ser_compact_size(n):
81 if n < 253:
82 return bytes([n])
83 if n <= 0xFFFF:
84 return b"\xfd" + struct.pack("<H", n)
85 if n <= 0xFFFFFFFF:
86 return b"\xfe" + struct.pack("<I", n)
87 return b"\xff" + struct.pack("<Q", n)
88
89
90def ser_varbytes(b):
91 return ser_compact_size(len(b)) + b
92
93
94def ser_tx(tx):
95 out = bytearray()
96 out += struct.pack("<i", tx["version"])
97 out += ser_compact_size(len(tx["vin"]))
98 for txin in tx["vin"]:
99 out += bytes.fromhex(txin["prevout_hash"])[::-1]
100 out += struct.pack("<I", txin["prevout_n"])
101 out += ser_varbytes(bytes.fromhex(txin["script_sig_hex"]))
102 out += struct.pack("<I", txin["sequence"])
103 out += ser_compact_size(len(tx["vout"]))
104 for txout in tx["vout"]:
105 out += struct.pack("<q", txout["value_satoshis"])
106 out += ser_varbytes(bytes.fromhex(txout["script_pub_key_hex"]))
107 out += struct.pack("<I", tx["lock_time"])
108 return bytes(out)
109
110
111def parse_tx(r):
112 tx = {"version": r.i32(), "vin": [], "vout": [], "lock_time": 0}
113 for _ in range(compact_size(r)):
114 tx["vin"].append({
115 "prevout_hash": r.take(32)[::-1].hex(),
116 "prevout_n": r.u32(),
117 "script_sig_hex": read_varbytes(r).hex(),
118 "sequence": r.u32(),
119 })
120 for _ in range(compact_size(r)):
121 value = r.i64()
122 tx["vout"].append({
123 "value_satoshis": value,
124 "value_btf": format_money(value),
125 "script_pub_key_hex": read_varbytes(r).hex(),
126 })
127 tx["lock_time"] = r.u32()
128 tx["txid"] = hash_hex(ser_tx(tx))
129 return tx
130
131
132def merkle_root(txids):
133 if not txids:
134 return "0" * 64
135 layer = [bytes.fromhex(x)[::-1] for x in txids]
136 while len(layer) > 1:
137 nxt = []
138 for i in range(0, len(layer), 2):
139 left = layer[i]
140 right = layer[i + 1] if i + 1 < len(layer) else left
141 nxt.append(sha256d(left + right))
142 layer = nxt
143 return layer[0][::-1].hex()
144
145
146def parse_block(payload, file_name="", offset=0):
147 r = Reader(payload)
148 header = r.take(80)
149 version, = struct.unpack("<i", header[0:4])
150 prev = header[4:36][::-1].hex()
151 merkle = header[36:68][::-1].hex()
152 ntime, bits, nonce = struct.unpack("<III", header[68:80])
153 txs = [parse_tx(r) for _ in range(compact_size(r))]
154 if r.pos != len(payload):
155 raise ParseError("trailing bytes inside block")
156 txids = [tx["txid"] for tx in txs]
157 return {
158 "hash": hash_hex(header),
159 "version": version,
160 "previous_hash": prev,
161 "merkle_root": merkle,
162 "computed_merkle_root": merkle_root(txids),
163 "time": ntime,
164 "time_utc": utc(ntime),
165 "bits": "0x%08x" % bits,
166 "nonce": nonce,
167 "tx_count": len(txs),
168 "transactions": txs,
169 "source_file": file_name,
170 "source_offset": offset,
171 }
172
173
174def read_blocks(datadir, max_blocks):
175 blocks = []
176 for path in sorted(Path(datadir).glob("blk*.dat")):
177 data = path.read_bytes()
178 pos = 0
179 while True:
180 idx = data.find(MAGIC, pos)
181 if idx < 0:
182 break
183 if idx + 8 > len(data):
184 break
185 size = struct.unpack("<I", data[idx + 4:idx + 8])[0]
186 start = idx + 8
187 end = start + size
188 if end > len(data):
189 break
190 blocks.append(parse_block(data[start:end], path.name, start))
191 if max_blocks and len(blocks) >= max_blocks:
192 return blocks
193 pos = end
194 return blocks
195
196
197def select_main_chain(blocks):
198 by_hash = {}
199 for block in blocks:
200 by_hash.setdefault(block["hash"], block)
201 if GENESIS_HASH not in by_hash:
202 raise ParseError("genesis block not found in block files")
203
204 children = {}
205 for block in by_hash.values():
206 children.setdefault(block["previous_hash"], []).append(block)
207 for v in children.values():
208 v.sort(key=lambda b: (b["hash"], b["source_file"], b["source_offset"]))
209
210 best = []
211 stack = [(by_hash[GENESIS_HASH], [by_hash[GENESIS_HASH]])]
212 while stack:
213 block, path = stack.pop()
214 if len(path) > len(best) or (len(path) == len(best) and block["hash"] < best[-1]["hash"]):
215 best = path
216 for child in children.get(block["hash"], []):
217 stack.append((child, path + [child]))
218 return best
219
220
221def is_coinbase(tx):
222 return (
223 len(tx["vin"]) == 1
224 and tx["vin"][0]["prevout_hash"] == "0" * 64
225 and tx["vin"][0]["prevout_n"] == 0xFFFFFFFF
226 )
227
228
229def utxo_leaf_hash(outpoint, utxo):
230 # Text fields are length-delimited by separators and all variable data is
231 # hex or decimal ASCII. That keeps the commitment language-neutral.
232 payload = "|".join([
233 "BTFUTXO1",
234 outpoint,
235 str(utxo["value_satoshis"]),
236 utxo["script_pub_key_hex"],
237 str(utxo["created_height"]),
238 utxo["created_txid"],
239 ]).encode("ascii")
240 return sha256(payload)
241
242
243def utxo_root(utxos):
244 leaves = [utxo_leaf_hash(k, utxos[k]) for k in sorted(utxos)]
245 if not leaves:
246 return sha256(b"BTFUTXO1|empty").hex()
247 layer = leaves
248 while len(layer) > 1:
249 nxt = []
250 for i in range(0, len(layer), 2):
251 left = layer[i]
252 right = layer[i + 1] if i + 1 < len(layer) else left
253 nxt.append(sha256(b"BTFNODE1|" + left + right))
254 layer = nxt
255 return layer[0].hex()
256
257
258def apply_blocks(blocks):
259 utxos = {}
260 spent = 0
261 missing_spends = []
262 duplicate_outputs = []
263 total_coinbase = 0
264 total_fees = 0
265 normal_txs = 0
266
267 for height, block in enumerate(blocks):
268 if block["computed_merkle_root"] != block["merkle_root"]:
269 raise ParseError("bad merkle root at height %d" % height)
270 for tx_index, tx in enumerate(block["transactions"]):
271 coinbase = is_coinbase(tx)
272 input_value = 0
273 output_value = sum(out["value_satoshis"] for out in tx["vout"])
274 if coinbase:
275 total_coinbase += output_value
276 else:
277 normal_txs += 1
278 for txin in tx["vin"]:
279 prev = "%s:%d" % (txin["prevout_hash"], txin["prevout_n"])
280 prev_utxo = utxos.pop(prev, None)
281 if prev_utxo is None:
282 missing_spends.append({
283 "height": height,
284 "txid": tx["txid"],
285 "missing_outpoint": prev,
286 })
287 else:
288 spent += prev_utxo["value_satoshis"]
289 input_value += prev_utxo["value_satoshis"]
290 if input_value >= output_value:
291 total_fees += input_value - output_value
292 for n, out in enumerate(tx["vout"]):
293 key = "%s:%d" % (tx["txid"], n)
294 if key in utxos:
295 duplicate_outputs.append({"height": height, "outpoint": key})
296 utxos[key] = {
297 "txid": tx["txid"],
298 "vout": n,
299 "value_satoshis": out["value_satoshis"],
300 "value_btf": out["value_btf"],
301 "script_pub_key_hex": out["script_pub_key_hex"],
302 "created_height": height,
303 "created_block_hash": block["hash"],
304 "created_txid": tx["txid"],
305 "coinbase": coinbase,
306 }
307
308 total_unspent = sum(u["value_satoshis"] for u in utxos.values())
309 return {
310 "utxos": utxos,
311 "stats": {
312 "utxo_count": len(utxos),
313 "spent_outputs": spent,
314 "total_unspent_satoshis": total_unspent,
315 "total_unspent_btf": format_money(total_unspent),
316 "total_coinbase_satoshis": total_coinbase,
317 "total_coinbase_btf": format_money(total_coinbase),
318 "coinbase_minus_unspent_satoshis": total_coinbase - total_unspent,
319 "coinbase_minus_unspent_btf": format_money(total_coinbase - total_unspent),
320 "total_fees_satoshis": total_fees,
321 "total_fees_btf": format_money(total_fees),
322 "normal_transactions": normal_txs,
323 "missing_spends": missing_spends,
324 "duplicate_outputs": duplicate_outputs,
325 },
326 }
327
328
329def format_money(value):
330 sign = "-" if value < 0 else ""
331 value = abs(value)
332 return "%s%d.%08d" % (sign, value // COIN, value % COIN)
333
334
335def utc(ts):
336 return _dt.datetime.fromtimestamp(ts, _dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
337
338
339def default_datadir():
340 if os.name == "nt" and os.environ.get("APPDATA"):
341 return os.path.join(os.environ["APPDATA"], "Bitflash")
342 if os.environ.get("HOME"):
343 return os.path.join(os.environ["HOME"], ".bitflash")
344 return "."
345
346
347def canonical_bytes(report):
348 return (json.dumps(report, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii")
349 10
350 11
351def build_report(raw_blocks, include_utxos=False): 12def build_report(raw_blocks, include_utxos=False, strict_duplicates=False):
352 blocks = select_main_chain(raw_blocks) 13 blocks = chain.select_main_chain(raw_blocks)
353 state = apply_blocks(blocks) 14 state = chain.apply_blocks(blocks, strict_duplicates=strict_duplicates)
354 utxos = state["utxos"] 15 utxos = state["utxos"]
355 stats = state["stats"] 16 stats = state["stats"]
356 tip = blocks[-1] 17 tip = blocks[-1]
357 root = utxo_root(utxos) 18 root = chain.utxo_root(utxos)
358 checks = { 19 checks = {
359 "genesis_hash_matches_source": blocks[0]["hash"] == GENESIS_HASH, 20 "genesis_hash_matches_source": blocks[0]["hash"] == chain.GENESIS_HASH,
360 "genesis_merkle_matches_source": blocks[0]["merkle_root"] == GENESIS_MERKLE, 21 "genesis_merkle_matches_source": blocks[0]["merkle_root"] == chain.GENESIS_MERKLE,
361 "all_scanned_merkle_roots_match": all(b["computed_merkle_root"] == b["merkle_root"] for b in blocks), 22 "all_scanned_merkle_roots_match": all(b["computed_merkle_root"] == b["merkle_root"] for b in blocks),
362 "no_missing_spends": len(stats["missing_spends"]) == 0, 23 "no_missing_spends": len(stats["missing_spends"]) == 0,
363 } 24 }
@@ -370,8 +31,8 @@ def build_report(raw_blocks, include_utxos=False):
370 "chain": { 31 "chain": {
371 "name": "Bitflash", 32 "name": "Bitflash",
372 "ticker": "BTF", 33 "ticker": "BTF",
373 "message_start_hex": MAGIC.hex(), 34 "message_start_hex": chain.MAGIC.hex(),
374 "coin": COIN, 35 "coin": chain.COIN,
375 }, 36 },
376 "commitment": { 37 "commitment": {
377 "algorithm": "sorted-outpoint-merkle-sha256", 38 "algorithm": "sorted-outpoint-merkle-sha256",
@@ -400,7 +61,7 @@ def build_report(raw_blocks, include_utxos=False):
400 61
401 62
402def print_text(report): 63def print_text(report):
403 data_hash = hashlib.sha256(canonical_bytes(report)).hexdigest() 64 data_hash = hashlib.sha256(chain.canonical_bytes(report)).hexdigest()
404 print("Bitflash UTXO-set commitment verifier") 65 print("Bitflash UTXO-set commitment verifier")
405 print("schema: %s" % report["schema"]) 66 print("schema: %s" % report["schema"])
406 print("canonical report sha256: %s" % data_hash) 67 print("canonical report sha256: %s" % data_hash)
@@ -437,20 +98,26 @@ def print_text(report):
437 98
438def main(argv): 99def main(argv):
439 ap = argparse.ArgumentParser(description=__doc__) 100 ap = argparse.ArgumentParser(description=__doc__)
440 ap.add_argument("--datadir", default=default_datadir(), help="Bitflash data directory") 101 ap.add_argument("--datadir", default=chain.default_datadir(), help="Bitflash data directory")
441 ap.add_argument("--max-blocks", type=int, default=0, 102 ap.add_argument("--max-blocks", type=int, default=0,
442 help="number of blocks to scan; 0 means all block files") 103 help="number of blocks to scan; 0 means all block files")
443 ap.add_argument("--include-utxos", action="store_true", 104 ap.add_argument("--include-utxos", action="store_true",
444 help="include the full sorted UTXO map in JSON output") 105 help="include the full sorted UTXO map in JSON output")
106 ap.add_argument("--strict-duplicates", action="store_true",
107 help="fail instead of using legacy last-write-wins duplicate output handling")
445 ap.add_argument("--json", action="store_true", help="print canonical JSON") 108 ap.add_argument("--json", action="store_true", help="print canonical JSON")
446 ap.add_argument("--out", help="write canonical JSON to this path") 109 ap.add_argument("--out", help="write canonical JSON to this path")
447 args = ap.parse_args(argv) 110 args = ap.parse_args(argv)
448 111
449 blocks = read_blocks(args.datadir, args.max_blocks) 112 blocks = chain.read_blocks(args.datadir, args.max_blocks)
450 if not blocks: 113 if not blocks:
451 raise SystemExit("no Bitflash blocks found in %s" % args.datadir) 114 raise SystemExit("no Bitflash blocks found in %s" % args.datadir)
452 report = build_report(blocks, include_utxos=args.include_utxos) 115 try:
453 data = canonical_bytes(report) 116 report = build_report(blocks, include_utxos=args.include_utxos,
117 strict_duplicates=args.strict_duplicates)
118 except chain.ParseError as e:
119 raise SystemExit(str(e))
120 data = chain.canonical_bytes(report)
454 if args.out: 121 if args.out:
455 Path(args.out).write_bytes(data) 122 Path(args.out).write_bytes(data)
456 if args.json: 123 if args.json:
diff --git a/tests/scripts/test_bitflash_tools.py b/tests/scripts/test_bitflash_tools.py
new file mode 100644
index 0000000..ff569e1
--- /dev/null
+++ b/tests/scripts/test_bitflash_tools.py
@@ -0,0 +1,107 @@
1#!/usr/bin/env python3
2"""Regression tests for deterministic Bitflash Python tooling."""
3
4import json
5import os
6import struct
7import subprocess
8import sys
9import tempfile
10from pathlib import Path
11
12
13ROOT = Path(__file__).resolve().parents[2]
14SCRIPTS = ROOT / "scripts"
15sys.path.insert(0, str(SCRIPTS))
16
17import bitflash_chain as chain # noqa: E402
18
19
20def run(args, **kwargs):
21 env = dict(os.environ)
22 env["PYTHONIOENCODING"] = "utf-8"
23 return subprocess.run(
24 [sys.executable] + [str(a) for a in args],
25 cwd=str(ROOT),
26 env=env,
27 check=True,
28 text=True,
29 stdout=subprocess.PIPE,
30 stderr=subprocess.PIPE,
31 **kwargs
32 )
33
34
35def write_block_file(path, blocks):
36 with Path(path).open("wb") as f:
37 f.write(b"noise")
38 f.write(chain.MAGIC)
39 f.write(struct.pack("<I", chain.MAX_BLOCK_PAYLOAD + 1))
40 f.write(b"still-noise")
41 for block in blocks:
42 payload = chain.ser_block(block)
43 f.write(chain.MAGIC)
44 f.write(struct.pack("<I", len(payload)))
45 f.write(payload)
46
47
48def assert_true(cond, msg):
49 if not cond:
50 raise AssertionError(msg)
51
52
53def main():
54 genesis = chain.reconstructed_genesis()
55 outpoint = "%s:0" % chain.GENESIS_MERKLE
56
57 with tempfile.TemporaryDirectory(prefix="bitflash-tool-tests-") as tmp:
58 tmp = Path(tmp)
59 datadir = tmp / "datadir"
60 datadir.mkdir()
61 blk1 = datadir / "blk0001.dat"
62 blk2 = datadir / "blk0002.dat"
63 write_block_file(blk1, [genesis])
64 blk2.write_bytes(b"")
65
66 fair = run([SCRIPTS / "verify-fair-launch.py", "--datadir", datadir,
67 "--max-blocks", "1", "--json"]).stdout
68 fair_report = json.loads(fair)
69 assert_true(fair_report["verdict"] == "FAIR_LAUNCH_BASELINE_VERIFIED",
70 "fair launch verifier failed synthetic genesis")
71
72 utxo = run([SCRIPTS / "verify-utxo-set.py", "--datadir", datadir,
73 "--json"]).stdout
74 utxo_report = json.loads(utxo)
75 assert_true(utxo_report["verdict"] == "UTXO_COMMITMENT_VERIFIED",
76 "UTXO verifier failed synthetic genesis")
77
78 proof_path = tmp / "proof.json"
79 run([SCRIPTS / "prove-utxo.py", outpoint, "--datadir", datadir,
80 "--out", proof_path, "--json"])
81 proof_check = run([SCRIPTS / "verify-utxo-proof.py", proof_path,
82 "--root", utxo_report["commitment"]["root"], "--json"]).stdout
83 proof_report = json.loads(proof_check)
84 assert_true(proof_report["verdict"] == "UTXO_PROOF_VERIFIED",
85 "UTXO proof verifier rejected synthetic proof")
86
87 explorer_dir = tmp / "explorer"
88 run([SCRIPTS / "build-explorer.py", blk1, blk2, explorer_dir])
89 blocks = json.loads((explorer_dir / "blocks.json").read_text(encoding="ascii"))
90 assert_true(blocks["tipHeight"] == 0 and blocks["count"] == 1,
91 "explorer did not build the synthetic chain")
92 index_html = (explorer_dir / "index.html").read_text(encoding="utf-8")
93 assert_true("innerHTML" not in index_html, "explorer viewer still uses innerHTML")
94
95 try:
96 chain.apply_blocks([genesis, genesis], strict_duplicates=True)
97 except chain.ParseError:
98 pass
99 else:
100 raise AssertionError("strict duplicate output mode did not fail")
101
102 print("Bitflash script tool regressions passed")
103 return 0
104
105
106if __name__ == "__main__":
107 raise SystemExit(main())