Inside Bitcoin

251test framework.
252
253The overall result of this is that `setgenerate` RPC call has been removed, as
254well as the `-gen` and `-genproclimit` command-line options.
255
256For testing, the `generate` call can still be used to mine a block, and a new
257RPC call `generatetoaddress` has been added to mine to a specific address. This
258works with wallet disabled.
259
260
261New bytespersigop implementation
262--------------------------------
263
264The former implementation of the bytespersigop filter accidentally broke bare
265multisig (which is meant to be controlled by the `permitbaremultisig` option),
266since the consensus protocol always counts these older transaction forms as 20
267sigops for backwards compatibility. Simply fixing this bug by counting more
268accurately would have reintroduced a vulnerability. It has therefore been
269replaced with a new implementation that rather than filter such transactions,
270instead treats them (for fee purposes only) as if they were in fact the size
271of a transaction actually using all 20 sigops.
272
273
274Low-level P2P changes
275----------------------
276
277- The optional new p2p message "feefilter" is implemented and the protocol
278  version is bumped to 70013. Upon receiving a feefilter message from a peer,
279  a node will not send invs for any transactions which do not meet the filter
280  feerate. [BIP 133](https://github.com/bitcoin/bips/blob/master/bip-0133.mediawiki)
281
282- The P2P alert system has been removed in PR #7692 and the `alert` P2P message
283  is no longer supported.
284
285- The transaction relay mechanism used to relay one quarter of all transactions
286  instantly, while queueing up the rest and sending them out in batch. As
287  this resulted in chains of dependent transactions being reordered, it
288  systematically hurt transaction relay. The relay code was redesigned in PRs
289  \#7840 and #8082, and now always batches transactions announcements while also
290  sorting them according to dependency order. This significantly reduces orphan
291  transactions. To compensate for the removal of instant relay, the frequency of
292  batch sending was doubled for outgoing peers.
293
294- Since PR #7840 the BIP35 `mempool` command is also subject to batch processing.
295  Also the `mempool` message is no longer handled for non-whitelisted peers when
296  `NODE_BLOOM` is disabled through `-peerbloomfilters=0`.
297
298- The maximum size of orphan transactions that are kept in memory until their
299  ancestors arrive has been raised in PR #8179 from 5000 to 99999 bytes. They
300  are now also removed from memory when they are included in a block, conflict
301  with a block, and time out after 20 minutes.
302
303- We respond at most once to a getaddr request during the lifetime of a
304  connection since PR #7856.
305
306- Connections to peers who have recently been the first one to give us a valid
307  new block or transaction are protected from disconnections since PR #8084.
308
309
310Low-level RPC changes
311----------------------
312
313- RPC calls have been added to output detailed statistics for individual mempool
314  entries, as well as to calculate the in-mempool ancestors or descendants of a
315  transaction: see `getmempoolentry`, `getmempoolancestors`, `getmempooldescendants`.
316
317- `gettxoutsetinfo` UTXO hash (`hash_serialized`) has changed. There was a divergence between
318  32-bit and 64-bit platforms, and the txids were missing in the hashed data. This has been
319  fixed, but this means that the output will be different than from previous versions.
320
321- Full UTF-8 support in the RPC API. Non-ASCII characters in, for example,
322  wallet labels have always been malformed because they weren't taken into account
323  properly in JSON RPC processing. This is no longer the case. This also affects
324  the GUI debug console.
325
326- Asm script outputs replacements for OP_NOP2 and OP_NOP3
327
328  - OP_NOP2 has been renamed to OP_CHECKLOCKTIMEVERIFY by [BIP 
32965](https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki)
330
331  - OP_NOP3 has been renamed to OP_CHECKSEQUENCEVERIFY by [BIP 
332112](https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki)
333
334  - The following outputs are affected by this change:
335
336    - RPC `getrawtransaction` (in verbose mode)
337    - RPC `decoderawtransaction`
338    - RPC `decodescript`
339    - REST `/rest/tx/` (JSON format)
340    - REST `/rest/block/` (JSON format when including extended tx details)
341    - `bitcoin-tx -json`
342
343- The sorting of the output of the `getrawmempool` output has changed.
344
345- New RPC commands: `generatetoaddress`, `importprunedfunds`, `removeprunedfunds`, `signmessagewithprivkey`,
346  `getmempoolancestors`, `getmempooldescendants`, `getmempoolentry`,
347  `createwitnessaddress`, `addwitnessaddress`.
348
349- Removed RPC commands: `setgenerate`, `getgenerate`.
350
351- New options were added to `fundrawtransaction`: `includeWatching`, `changeAddress`, `changePosition` and `feeRate`.
352
353
354Low-level ZMQ changes
355----------------------
356
357- Each ZMQ notification now contains an up-counting sequence number that allows
358  listeners to detect lost notifications.
359  The sequence number is always the last element in a multi-part ZMQ notification and
360  therefore backward compatible. Each message type has its own counter.
361  PR [#7762](https://github.com/bitcoin/bitcoin/pull/7762).
362
363
3640.13.0 Change log
365=================
366
367Detailed release notes follow. This overview includes changes that affect
368behavior, not code moves, refactors and string updates. For convenience in locating
369the code changes and accompanying discussion, both the pull request and
370git merge commit are mentioned.
371
372### RPC and other APIs
373
374- #7156 `9ee02cf` Remove cs_main lock from `createrawtransaction` (laanwj)
375- #7326 `2cd004b` Fix typo, wrong information in gettxout help text (paveljanik)
376- #7222 `82429d0` Indicate which transactions are signaling opt-in RBF (sdaftuar)
377- #7480 `b49a623` Changed getnetworkhps value to double to avoid overflow (instagibbs)
378- #7550 `8b958ab` Input-from-stdin mode for bitcoin-cli (laanwj)
379- #7670 `c9a1265` Use cached block hash in blockToJSON() (rat4)
380- #7726 `9af69fa` Correct importaddress help reference to importpubkey (CypherGrue)
381- #7766 `16555b6` Register calls where they are defined (laanwj)
382- #7797 `e662a76` Fix generatetoaddress failing to parse address (mruddy)
383- #7774 `916b15a` Add versionHex in getblock and getblockheader JSON results (mruddy)
384- #7863 `72c54e3` Getblockchaininfo: make bip9_softforks an object, not an array (rustyrussell)
385- #7842 `d97101e` Do not print minping time in getpeerinfo when no ping received yet (paveljanik)
386- #7518 `be14ca5` Add multiple options to fundrawtransaction (promag)
387- #7756 `9e47fce` Add cursor to iterate over utxo set, use this in `gettxoutsetinfo` (laanwj)
388- #7848 `88616d2` Divergence between 32- and 64-bit when hashing >4GB affects `gettxoutsetinfo` (laanwj)
389- #7827 `4205ad7` Speed up `getchaintips` (mrbandrews)
390- #7762 `a1eb344` Append a message sequence number to every ZMQ notification (jonasschnelli)
391- #7688 `46880ed` List solvability in listunspent output and improve help (sipa)
392- #7926 `5725807` Push back `getaddednodeinfo` dead value (instagibbs)
393- #7953 `0630353` Create `signmessagewithprivkey` rpc (achow101)
394- #8049 `c028c7b` Expose information on whether transaction relay is enabled in `getnetworkinfo` (laanwj)
395- #7967 `8c1e49b` Add feerate option to `fundrawtransaction` (jonasschnelli)
396- #8118 `9b6a48c` Reduce unnecessary hashing in `signrawtransaction` (jonasnick)
397- #7957 `79004d4` Add support for transaction sequence number (jonasschnelli)
398- #8153 `75ec320` `fundrawtransaction` feeRate: Use BTC/kB (MarcoFalke)
399- #7292 `7ce9ac5` Expose ancestor/descendant information over RPC (sdaftuar)
400- #8171 `62fcf27` Fix createrawtx sequence number unsigned int parsing (jonasschnelli)
401- #7892 `9c3d0fa` Add full UTF-8 support to RPC (laanwj)
402- #8317 `304eff3` Don't use floating point in rpcwallet (MarcoFalke)
403- #8258 `5a06ebb` Hide softfork in `getblockchaininfo` if timeout is 0 (jl2012)
404- #8244 `1922e5a` Remove unnecessary LOCK(cs_main) in getrawmempool (dcousens)
405
406### Block and transaction handling
407
408- #7056 `6a07208` Save last db read (morcos)
409- #6842 `0192806` Limitfreerelay edge case bugfix (ptschip)
410- #7084 `11d74f6` Replace maxFeeRate of 10000*minRelayTxFee with maxTxFee in mempool (MarcoFalke)
411- #7539 `9f33dba` Add tags to mempool's mapTx indices (sdaftuar)
412- #7592 `26a2a72` Re-remove ERROR logging for mempool rejects (laanwj)
413- #7187 `14d6324` Keep reorgs fast for SequenceLocks checks (morcos)
414- #7594 `01f4267` Mempool: Add tracking of ancestor packages (sdaftuar)
415- #7904 `fc9e334` Txdb: Fix assert crash in new UTXO set cursor (laanwj)
416- #7927 `f9c2ac7` Minor changes to dbwrapper to simplify support for other databases (laanwj)
417- #7933 `e26b620` Fix OOM when deserializing UTXO entries with invalid length (sipa)
418- #8020 `5e374f7` Use SipHash-2-4 for various non-cryptographic hashes (sipa)
419- #8076 `d720980` VerifyDB: don't check blocks that have been pruned (sdaftuar)
420- #8080 `862fd24` Do not use mempool for GETDATA for tx accepted after the last mempool req (gmaxwell)
421- #7997 `a82f033` Replace mapNextTx with slimmer setSpends (kazcw)
422- #8220 `1f86d64` Stop trimming when mapTx is empty (sipa)
423- #8273 `396f9d6` Bump `-dbcache` default to 300MiB (laanwj)
424- #7225 `eb33179` Eliminate unnecessary call to CheckBlock (sdaftuar)
425- #7907 `006cdf6` Optimize and Cleanup CScript::FindAndDelete (pstratem)
426- #7917 `239d419` Optimize reindex (sipa)
427- #7763 `3081fb9` Put hex-encoded version in UpdateTip (sipa)
428- #8149 `d612837` Testnet-only segregated witness (sipa)
429- #8305 `3730393` Improve handling of unconnecting headers (sdaftuar)
430- #8363 `fca1a41` Rename "block cost" to "block weight" (sdaftuar)
431- #8381 `f84ee3d` Make witness v0 outputs non-standard (jl2012)
432- #8364 `3f65ba2` Treat high-sigop transactions as larger rather than rejecting them (sipa)
433
434### P2P protocol and network code
435
436- #6589 `dc0305d` Log bytes recv/sent per command (jonasschnelli)
437- #7164 `3b43cad` Do not download transactions during initial blockchain sync (ptschip)
438- #7458 `898fedf` peers.dat, banlist.dat recreated when missing (kirkalx)
439- #7637 `3da5d1b` Fix memleak in TorController (laanwj, jonasschnelli)
440- #7553 `9f14e5a` Remove vfReachable and modify IsReachable to only use vfLimited (pstratem)
441- #7708 `9426632` De-neuter NODE_BLOOM (pstratem)
442- #7692 `29b2be6` Remove P2P alert system (btcdrak)
443- #7542 `c946a15` Implement "feefilter" P2P message (morcos)
444- #7573 `352fd57` Add `-maxtimeadjustment` command line option (mruddy)
445- #7570 `232592a` Add IPv6 Link-Local Address Support (mruddy)
446- #7874 `e6a4d48` Improve AlreadyHave (morcos)
447- #7856 `64e71b3` Only send one GetAddr response per connection (gmaxwell)
448- #7868 `7daa3ad` Split DNS resolving functionality out of net structures (theuni)
449- #7919 `7617682` Fix headers announcements edge case (sdaftuar)
450- #7514 `d9594bf` Fix IsInitialBlockDownload for testnet (jmacwhyte)
451- #7959 `03cf6e8` fix race that could fail to persist a ban (kazcw)
452- #7840 `3b9a0bf` Several performance and privacy improvements to inv/mempool handling (sipa)
453- #8011 `65aecda` Don't run ThreadMessageHandler at lowered priority (kazcw)
454- #7696 `5c3f8dd` Fix de-serialization bug where AddrMan is left corrupted (EthanHeilman)
455- #7932 `ed749bd` CAddrMan::Deserialize handle corrupt serializations better (pstratem)
456- #7906 `83121cc` Prerequisites for p2p encapsulation changes (theuni)
457- #8033 `18436d8` Fix Socks5() connect failures to be less noisy and less unnecessarily scary (wtogami)
458- #8082 `01d8359` Defer inserting into maprelay until just before relaying (gmaxwell)
459- #7960 `6a22373` Only use AddInventoryKnown for transactions (sdaftuar)
460- #8078 `2156fa2` Disable the mempool P2P command when bloom filters disabled (petertodd)
461- #8065 `67c91f8` Addrman offline attempts (gmaxwell)
462- #7703 `761cddb` Tor: Change auth order to only use password auth if -torpassword (laanwj)
463- #8083 `cd0c513` Add support for dnsseeds with option to filter by servicebits (jonasschnelli)
464- #8173 `4286f43` Use SipHash for node eviction (sipa)
465- #8154 `1445835` Drop vAddrToSend after sending big addr message (kazcw)
466- #7749 `be9711e` Enforce expected outbound services (sipa)
467- #8208 `0a64777` Do not set extra flags for unfiltered DNS seed results (sipa)
468- #8084 `e4bb4a8` Add recently accepted blocks and txn to AttemptToEvictConnection (gmaxwell)
469- #8113 `3f89a53` Rework addnode behaviour (sipa)
470- #8179 `94ab58b` Evict orphans which are included or precluded by accepted blocks (gmaxwell)
471- #8068 `e9d76a1` Compact Blocks (TheBlueMatt)
472- #8204 `0833894` Update petertodd's testnet seed (petertodd)
473- #8247 `5cd35d3` Mark my dnsseed as supporting filtering (sipa)
474- #8275 `042c323` Remove bad chain alert partition check (btcdrak)
475- #8271 `1bc9c80` Do not send witnesses in cmpctblock (sipa)
476- #8312 `ca40ef6` Fix mempool DoS vulnerability from malleated transactions (sdaftuar)
477- #7180 `16ccb74` Account for `sendheaders` `verack` messages (laanwj)
478- #8102 `425278d` Bugfix: use global ::fRelayTxes instead of CNode in version send (sipa)
479- #8408 `b7e2011` Prevent fingerprinting, disk-DoS with compact blocks (sdaftuar)
480
481### Build system
482
483- #7302 `41f1a3e` C++11 build/runtime fixes (theuni)
484- #7322 `fd9356b` c++11: add scoped enum fallbacks to CPPFLAGS rather than defining them locally (theuni)
485- #7441 `a6771fc` Use Debian 8.3 in gitian build guide (fanquake)
486- #7349 `152a821` Build against system UniValue when available (luke-jr)
487- #7520 `621940e` LibreSSL doesn't define OPENSSL_VERSION, use LIBRESSL_VERSION_TEXT instead (paveljanik)
488- #7528 `9b9bfce` autogen.sh: warn about needing autoconf if autoreconf is not found (knocte)
489- #7504 `19324cf` Crystal clean make clean (paveljanik)
490- #7619 `18b3f1b` Add missing sudo entry in gitian VM setup (btcdrak)
491- #7616 `639ec58`  [depends] Delete unused patches  (MarcoFalke)
492- #7658 `c15eb28` Add curl to Gitian setup instructions (btcdrak)
493- #7710 `909b72b` [Depends] Bump miniupnpc and config.guess+sub (fanquake)
494- #7723 `5131005` build: python 3 compatibility (laanwj)
495- #7477 `28ad4d9` Fix quoting of copyright holders in configure.ac (domob1812)
496- #7711 `a67bc5e` [build-aux] Update Boost & check macros to latest serials (fanquake)
497- #7788 `4dc1b3a` Use relative paths instead of absolute paths in protoc calls (paveljanik)
498- #7809 `bbd210d` depends: some base fixes/changes (theuni)
499- #7603 `73fc922` Build System: Use PACKAGE_TARNAME in NSIS script (JeremyRand)
500- #7905 `187186b` test: move accounting_tests and rpc_wallet_tests to wallet/test (laanwj)