A Brief History of Bitcoin Forks

This post was originally published at: https://blog.blockchain.com/2016/02/26/a-brief-history-of-bitcoin-forks/

Content was co-authored by Peter Smith.

TL;DR Summary

Soft consensus forks render previously valid blocks invalid, and create security risks for non-upgrading nodes.
Hard consensus forks render previously invalid blocks valid, and make non-upgrading nodes incompatible.
Unplanned forks pose the greatest risk, and are generally caused by poorly understood software complexity or rushed changes.
Both soft and hard consensus forks can be used to make complex changes to bitcoin; soft forks merely require a lower amount of consensus.

Introduction

At Blockchain, we monitor both the Bitcoin network and new proposed changes to it. With the current debate on how to scale the network ongoing, we thought it would be useful to provide some background and general information.

Bitcoin is useful because people all over the world can agree on the balance of their digital wallets, discriminating between valid financial transactions and attempted fraud. Funds are shifted from one address to the next with each transaction, and this action is codified when a miner adds the transaction to a permanent block in the blockchain.

With many miners competing amongst another in intentional race conditions, there are often differing opinions about which next few blocks will become permanent. However, in order for Bitcoin to work as money, these disagreements must be settled quickly. Maintaining agreement on which blocks are valid and which are not requires software consensus.

Occasionally, the Bitcoin protocol malfunctions and an unintentional consensus fork takes place. A consensus fork represents differing opinions about the valid state of the Bitcoin blockchain, and persists over some meaningful period of time. Unintentional consensus forks are resolved through a coordinated fix deployed throughout the ecosystem.

Developers regularly create intentional consensus forks by modifying Bitcoin client software. These changes upgrade or constrain the way that Bitcoin works — temporarily or permanently.

Aside from intentionality, we tend to categorize consensus forks in two ways:

(1) A hard consensus fork occurs when blocks that would have previously been considered invalid are now valid. Any Bitcoin user, miner, exchanger, etc. who wants to stay in consensus with the network must upgrade his software during a hard consensus fork; otherwise, some new block that the network accepts will appear as invalid to him.

(2) A soft consensus fork occurs when blocks that would have previously been considered valid are now invalid. Upgrading software during a consensus soft fork is forever optional to a Bitcoin user, miner or exchanger, with the following caveats:

If the soft fork introduces a new feature that you want to use as either the sender or recipient, you must upgrade in order to use it.
At least 51% of miners must upgrade to adopt the soft fork; otherwise, it will forever appear as the shortest chain and get orphaned by the network.
Refusal to accept the soft fork can reduce your security. As you would normally consider soft forked transactions invalid, Bitcoin developers use various tricks to make these transactions appear valid to you while reducing your client’s capacity to process exactly why they are valid. See “Risks of Intentional Soft Consensus Forks” below.
Given that a consensus fork can be “soft” or “hard” and “intentional” or “unintentional”, there are essentially four basic kinds of consensus forks that can take place:

x	Intentional	Unintentional
Soft	Intentional Soft Fork	Unintentional Soft Fork
Hard	Intentional Hard Fork	Unintentional Hard Fork

Fig. 1: A taxonomy of consensus forks based on compatibility and intentionality.

Some of these definitions are provided in BIP 99. [1]

Risks of Unintentional Consensus Forks

Unintentional consensus forks are unplanned and pose the most risk. During an unintentional consensus fork, significant stakeholders in the Bitcoin network develop multiple, mutually exclusive concepts of which blocks are valid. Longer contrasting chains of blocks make it more expensive to resolve the problem.

When an unintentional consensus fork occurs, the Bitcoin ecosystem must undertake these steps:

Alert stakeholders and diagnose the problem.
Temporarily disable services (e.g. exchanges, wallets) to prevent the creation of new transactions that may be struck from the record or create fraud.
Build human and economic consensus on which chain will become the official one, and which will be orphaned.
Build human and economic consensus on which validation rules everyone should be using in their Bitcoin client software.
Build human and economic consensus about whether specific transactions need to be eliminated from the record, or whether a rollback needs to occur on one or more blocks.
Deal with the consequences of intentional or unintentional double spends that may have taken place.

Miners are fearful about unintentional consensus forks. Blocks that are struck from the record in order to resolve a fork represent irrevocable financial losses for the miners who mined them.

Bitcoin investors fear unintentional forks based on the devalued reputation of the system as a value exchange, as well as the possibility of funds lost due to fraud. All miners are temporarily Bitcoin investors until they sell mined coins to recover capital investments, unless they are appropriately hedged.

Bitcoin companies fear unintentional forks because it can be costly and time-consuming to repair the mess created by a fork.

An unintentional consensus fork occurred on March 12, 2013. Gavin Andresen described the fork in BIP 50:

“A block that had a larger number of total transaction inputs than previously seen was mined and broadcasted. Bitcoin 0.8 nodes were able to handle this, but some pre-0.8 Bitcoin nodes rejected it, causing an unexpected fork of the blockchain. The pre-0.8-incompatible chain (from here on, the 0.8 chain) at that point had around 60% of the mining hash power ensuring the split did not automatically resolve (as would have occurred if the pre-0.8 chain outpaced the 0.8 chain in total work, forcing 0.8 nodes to reorganise to the pre-0.8 chain).”

At the time of the fork, this was understood as a classical hard fork resulting from incompatibility between versions of the Bitcoin client; however, later analysis determined that the client upgrade simply caused the fork to have greater effect than it would have if all clients remained on the older version. As the fork was the expression of blocks that (almost always) would have been considered invalid now being regarded as valid by the majority of hashing capacity, this is still best categorized as an unintentional hard fork using the taxonomy in Figure 1, though this is contested by some developers.

Unintentional consensus forks occur when security and quality testing have failed. Some increased factors for unintentional consensus forks include:

Software complexity, or ‘unknown unknowns’
Insufficient testing time and/or resources allocated before deployment of new software
Weak documentation for existing and new software

Risks of Intentional Consensus Forks

Intentional consensus forks are sometimes executed as planned upgrades to the network, but on rare occasions have been used as emergency fixes.

Intentional but emergency consensus forks carry some of the same risks as unintentional consensus forks in that there is limited time to test their effects on the network, but the risk is often preferable to extending the exigent emergency. On August 15, 2010, an emergency consensus fork was executed to deal with an exploited integer overflow bug that permitted an attacker to create several billion bitcoins. The small development and user community at the time decided to respond to this by editing the blockchain to remove the problematic transaction and fixing the bug; as this was an instance of a block previously considered valid being rendered invalid, this was a soft consensus fork.

Risks of Intentional Soft Consensus Forks

To date, all planned software upgrades to Bitcoin have been intentional soft consensus forks. Some of these upgrades have used a kind of trick to achieve this effect: transactions using the new feature are made to appear valid to pre-fork clients by repurposing the semantics of existing opcodes. This mechanism was used for the consensus soft fork creating P2SH transactions (BIP 16). When it is employed, only post-fork clients will understand the exact semantics of the validity of the transaction; consequently, pre-fork clients are no longer able to verify whether a transaction is valid in a post-fork context, but merely verify how many blocks have been mined on top of that transaction — an act referred to as a “block depth check.”

Soft consensus forks can yield blocks that are rejected by post-fork clients but accepted by pre-fork clients. This can lead to double spends that receive one or more confirmations from pre-fork or SPV miners, continued until post-fork miners reject the chain. This is known as the “fake confirmation” vulnerability. The risk of this vulnerability is proportional to the percent of hash power that declines to upgrade to the post-fork software, the number of SPV miners, and the number of clients who decline to upgrade. To mitigate this vulnerability, recent soft consensus forks have attempted to set consensus thresholds to ensure that a certain percentage of miners adopt a soft fork; however, even with the fairly high threshold of 95% used in the BIP 66 soft fork, a confluence of pre-fork and SPV miners lead to several blocks in a row being mined on a pre-BIP 66 fork chain until the relevant companies were contacted.

A common misconception is that soft consensus forks are necessarily minor changes while hard consensus forks are unbounded. Given demonstrations that even fundamental parameters like the currency supply can be modified via a soft consensus fork, there appears to be no upper limit to the scope of changes that can be effected through this style of fork. Theoretical soft consensus forks include modifying maximum block size, [2] the block time target, [3] and the currency cap of 21 million coins. [4].

In summary, intentional soft consensus forks do not tend to cause network splits, but can reduce payment verification capabilities for full nodes that do not upgrade, for SPV clients, and for thin clients that submit balance queries to full nodes that do not upgrade.

Risks of Intentional Hard Forks

As no planned hard forks have taken place, their risks are not well understood. Various proposals have been made over the years for planned hard consensus forks, but none have been executed. More recently, some alternative Bitcoin clients, such as Bitcoin Classic, have provided the software necessary for a hard consensus fork related to maximum block size.

Perhaps the worst case scenario would be if a substantial portion of the network — including more than 50% of the hash rate but less than 100% — refused to upgrade, and divergent chains were mined over a meaningful period of time. Without proper preparation, this would lead to confusion, double spend fraud, and a devaluation of the currency. Regardless of threats in advance, a minority of the network would only refuse to hard fork consensus if they believed that the potential disaster of multiple chains was preferable to the change incurred by the hard fork.

Statistical analysis has shown that BIP 101-style activation of hard consensus forks can be prematurely activated; for example, a hard consensus fork triggered by signals from 75% of the last 1000 blocks has an approximately 88% chance of being activated randomly with just 70% of hashrate support over the course of 10,000 blocks. [5] Higher thresholds come with the added risk of stonewalling or ransom demands from a smaller minority of hashing power.

[1] https://github.com/bitcoin/bips/blob/master/bip-0099.mediawiki
[2] [Bitcoin-development] soft-fork block size increase (extension blocks) Re: Proposed alternatives to the 20MB stepfunction
[3] Soft-forking the block time to 2 min: my primarily silly and academic (but seemingly effective) entry to the “increase the blockchain’s capacity in an arbitrarily roundabout way as long as it’s a softfork” competition
[4] Raising the 21 million BTC limit with a soft fork
[5] BIP101 implementation flaws