Decode Bitcoin CompactSize lengths with worked byte examples, distinguish Core’s internal VARINT format, and avoid unsafe parser assumptions.
Know what the number describes
A raw transaction is a sequence of fields. Some fields have a fixed width; others begin with a count or length. A parser must know which rule applies before interpreting the next byte. Reading every number as a four-byte integer shifts the cursor and corrupts everything that follows.
CompactSize is used for variable-length counts and lengths in transaction serialization. The term “varint” is sometimes used loosely for it, but Bitcoin Core also implements a different internal encoding named VARINT. Similar names do not make their byte representations interchangeable.
Work across the first boundary
CompactSize values below 253 occupy one byte. A prefix of fd selects a following two-byte unsigned little-endian value; fe selects four bytes and ff selects eight. These prefixes choose a width, rather than contributing a high digit to the encoded number.
Decode three counts
The byte fc means 252. The sequence fd fd 00 means 253. The sequence fd 00 01 means 256, because the payload is little-endian. For the second example the parser consumes three bytes in total: one selector and two payload bytes. These are counts, not amounts of bitcoin.
Keep the surrounding transaction schema
An output contains an amount, a script length and the script bytes. Its amount is a fixed-width integer, while its script length uses CompactSize. An input count determines how many input structures follow; it does not say how many bytes the entire input region occupies.
A decoder can therefore display perfectly plausible numbers while reading the wrong boundaries. A good debugging method records the cursor before each field, the number of consumed bytes and the expected next field. Compare that trace with a known decoder using public test data. Do not experiment with private wallet material to diagnose a serialization error.
Reject ambiguity and resource abuse
A decoder must enforce the format's canonical encodings, detect truncation, and apply the limits appropriate to the object being read. A declared count is not permission to allocate that much memory. Check remaining bytes and use bounded processing before creating large collections.
Imagine a short upload declaring billions of entries. The first useful result is an invalid-input response, not a lengthy allocation attempt. Input-length limits, time budgets and format validation solve different parts of that problem. The same discipline matters for an explorer's public transaction decoder.
Do not reuse a disk decoder blindly
Core's internal VARINT uses a different base-128 scheme. A program reading a Core database needs the exact database schema as well as its integer encoding. Neither a network packet diagram nor a successful CompactSize test proves that a database reader is correct.
Before choosing a decoding function, write down the source format, version, field type, byte order and expected bounds. That small checklist is more reliable than a helper function called simply “readVarInt”. Continue with Anatomy of a raw transaction to see how the fields fit together.
Sources and further reading
Key takeaways
- CompactSize prefixes select the width of a following unsigned little-endian value.
- Core internal VARINT and transaction CompactSize are distinct formats.
- Validate canonical form, available bytes and resource limits before trusting a declared count.
Common questions
Does fd mean the number 253 everywhere?
Only the surrounding schema tells you. In a CompactSize field it is a selector for a following two-byte value.
Can I use the same parser for Core database files?
Not without checking the database schema. Core uses additional encodings and storage conventions.