 |
hardfork_prototype_0 - an early prototype of a Bitcoin hard fork
Part 1 - Introduction
This is a 3-part article which describes an early prototype of a Bitcoin spin-off, called "hard fork prototype 0" (HFP0, for short). A spin-off is a special type of hard fork (HF) which preserves the entire ledger up to the point in time where it diverges to become its own currency.
The parts of this article progress from a general overview to deeper technical explanation:
- Part 1 (this post): introduction, history
- Part 2: high-level technical overview
- Part 3: detailed technical walkthrough
The HFP0 source code has been published along with these articles. This prototype is based on Bitcoin Classic v0.12.0cl1 (this commit, to be precise) and was inspired by an earlier hard fork prototype, (satoshisbitcoin), which was written by user 'rocks' on the United Bitcoin Community's Bitcoin Forum. I heartily recommend joining that forum if you're interested in gaining deeper knowledge of Bitcoin and the economic aspects of cryptocurrencies.
In March 2016 I started working on what became HFP0. I had planned to release it alongside satoshisbitcoin as a complementary spin-off which would not change Bitcoin's proof-of-work (POW) function, because I thought it would be worthwhile to offer a true hard fork which the existing miners could still support, if they wanted. The other aim was to familiarize myself with the Bitcoin source code and development process. That's why free software is free and open after all - if you don't like the way things are heading, you can pick up the code, learn about it and implement the features you want.
Even before the Ethereum fork, I discussed the need for protection against replay attacks. The ETH/ETC fork finally convinced me that a serious Bitcoin spin-off attempt would require a robust solution to this problem. Furthermore, any spin-off would require substantial support and engagement from the community to get off the ground. As it happened, members of the community created BTCfork, an organization consisting of Bitcoin users from all walks of life and professional backgrounds, united in their desire to hard-fork Bitcoin to restore the original vision of Satoshi: Bitcoin: a peer-to-peer electronic cash system.
The next two parts of the article will delve into the technical aspects of HFP0. I encourage those who are interested to study and play with HFP0, and use whichever parts of it they see fit for their own hard fork experiments, just as I learned and used code from satoshisbitcoin. There are some additions that may yet be of value, such as the per-block retargeting algorithm (Ray Dillinger's MIDAS) and the more 'realistic' implementation of BitPay's adaptive block size algorithm.
If you wish to provide feedback or ask questions about HFP0, I've opened a Reddit thread in /r/btcfork to gather comments and questions.
The BTCfork community welcomes everyone who wants to enable a hard fork to join in the discussion. We meet up on various forums and chat channels to discuss and collaborate on matters related to hard forks:
- on Reddit at /r/btcfork
- on our Slack (sign up here)
- on GitHub at https://github.com/BTCfork
Please refer to the BTCfork website for more information on how to contribute.
Why not spin off right now using this code?
HFP0 is not ready for production use. That's why I'm calling it a prototype, and releasing only source code.
It lacks a feature which developers in BTCfork, including myself, have identified as a critical requirement for a viable spin-off:
- protection against replay attacks by making the forked transaction signatures invalid on the old chain
A lack of replay attack protection led to complications in the recent Ethereum hard fork. Althought the ETH/ETC fork was very different in many respects from what a planned HF in Bitcoin would be like, it highlighted the importance of protecting against replay attacks.
I believe that the viability of an attempted Bitcoin spin-off would be seriously compromised if it failed to provide replay attack protection. My initial solution design turned out to be inadequate, but a better solution has since been proposed and will be implemented and tested in future forks.
There are other features missing in this prototype that a safe hard fork should implement:
- clean separation of the networks to avoid unnecessary processing on both sides
- creating a backup of the user's hot wallet prior to the fork, for use with an unmodified (old chain) client, to ensure user can access his entire pre-fork holdings on the old chain without problems
There are further issues around the release and deployment of a spin-off that need to be managed carefully. Without extensive quality assurance (testing), sufficient public exposure (incl. preparation time) or build-up of critical infrastructure and supportive relationships with prospective miners, pools, exchanges, wallet manufacturers, a spin-off would start with a potentially fatal disadvantage compared to the incumbent Bitcoin implementation.
More detail on other shortcomings of this prototype will be presented in the remaining parts (see Technical Overview and Details).
History of this project
I have been following Bitcoin's block size debate for quite some time on various forums, starting in /r/Bitcoin (nowadays I would recommend /r/btc).
After several elective (i.e. voting-based) hard fork proposals went nowhere due to a minority of Bitcoin users, the miners, vetoing Bitcoin Improvement Proposals (BIPs 101,109) and unknown criminal elements employing 'dirty tricks' against the network infrastructure of such forks (e.g. denial-of-service attacks against their nodes), I became determined to involve myself in development of a non-elective hard-fork as described by Satoshi. I wanted to give the market freedom of choice on the issue of block size, and that it was the only way to overcome what I perceive to be a corporate capture of the decision-making power in terms of where Bitcoin's development is headed.
I took particular interest in the satoshisbitcoin fork development in the Bitcoin Forum, and participated in the public testing of that client, which revealed some needed improvements.
The original purpose of my fork was to offer a complementary spin-off which retained Bitcoin's traditional SHA256 POW and was to be based on a more recent Classic 0.12 codebase (satoshisbitcoin was based on Classic 0.11.2 which lacked some optimizations such as libsecp256k-based signature validation).
Satoshisbitcoin's author disappeared from the Bitcoin Forum around mid-April 2016, leaving that fork software unmaintained. I decided to absorb the POW-change functionality into my actively developed fork, making it possible to build both a POW and a non-POW version of my fork, as it seemed important to offer the choice to the market in the event of a hard fork situation. I recognize that a lot of money has been invested in Bitcoin's mining infrastructure by many individuals as well as companies. Giving existing miners the option of supporting a hard fork by mining the unchanged-POW version using their existing equipment would lower the risk to the currency's security. Then the market would be able to decide whether to support the miners or not, without having to make a binary choice between the existing miners and the fork's actual improvements. It is important for the health of the Bitcoin ecosystem that decisions on critical issues are opened up to independent choice. As with a referendum, there is a danger of confusing the performance of the incumbent miners with the issue on which the referendum is being held - in this case the block size.
Starting from the premise of supporting the existing POW, my fork needed changes which satoshisbitcoin did not strictly require, such as modifying the difficulty adjustment algorithm to defend against attacks by the existing miners. Also, I wanted it to support blocks bigger than 2MB, leading to the addition of BitPay's adaptive block size algorithm (modified to have a 2MB floor and 4MB ceiling) and Bitcoin Unlimited's Xtreme Thinblocks for improved block propagation.
On this road, which I started on with zero knowledge of the Bitcoin code, I encountered the help of many generous developers and non-developers who willingly shared what they knew about Bitcoin, its code and other aspects. The list is too long to mention specific names, but I'm grateful to all the Bitcoin Unlimited, Bitcoin XT, Bitcoin Classic and Iguana developers who have helped me when I had questions, as I am to all the Bitcoin Forum members who gave input and advice on matters related to forking.
Participating in BTCfork and /r/btcfork has taught me more about what constitutes a viable, safer hard fork client. Since then my efforts have been focused on developing the requirements, design and implementation of a Minimum Viable Fork in a truly public way, as I believe it should be done to engender the necessary confidence of the wider Bitcoin community for a hard fork.
I think it is appropriate that I now publish HFP0 as it is and for what it is - a step on the road to Bitcoin's eventual first hard fork, and perhaps something that others can use as inspiration and study, just like I was able to do with satoshisbitcoin.
I will provide a description of the technical details and the shortcomings of HFP0 in the subsequent parts (Technical Overview and Details).
The back story - the block size debate
This fork code is part of a bigger story, the block size debate. Software projects sometimes split up when members find themselves holding different opinions about the directions in which the project should go. Bitcoin users have, since 2012, slowly divided into two discernable camps:
- a group who believes Bitcoin should remain P2P electronic cash, scaling its transaction rate and growing its user base through an increase of its current 1MB block size limit to a value exceeding transactional demand by a safe margin, and removing the limits in its protocol altogether where possible
- a group who believes that Bitcoin should become more like a settlement system (few transactions, higher fees), with the payment function (fast, cheap fees) offloaded onto additional layers above it (Lightning Network) or sidechains
My opinions align strongly with the first group. I believe that Bitcoin can scale on-chain quite significantly, and should do so to reach as many people as possible, giving them access to secure, disintermediated p2p electronic cash and a world of new services that can arise from this "magical Internet money".
Like many supporters of this view, I am not against exploring alternative methods to scale Bitcoin, such as the Lightning Network, sidechains etc. I strongly disagree with the way in which a roadmap for Bitcoin has been unilaterally imposed on Bitcoin by the Core developers, the most senior and influential of whom have incorporated as Blockstream, and have dismissed many prominent early Bitcoin developers and users who have been lobbying for an increase of the block size cap to prevent economic damage to Bitcoin as blocks have predictably grown fuller, fees increased and network confirmation times grown larger over time. The Core developers have shown no willingness to compromise on the block size issue at all.
This is leaving the advocates of on-chain scaling no choice but to go their own way - a hard fork - the equivalent of a divorce in free software projects.
What makes a cryptocurrency valuable is not only its technical feature set, but its distributed ledger which tracks the real-world distribution of the coins. There are countless new currencies which start from scratch, with an empty ledger.
However, if we want to hard-fork to upgrade Bitcoin's capabilities, it makes sense to preserve the distribution of everyone's coins, i.e. spin-off using the existing ledger. This still results in two currencies ("old" and "new" Bitcoin) from a certain point onwards, but like in a share split, everyone who owns bitcoins before the fork will have the same amount of coins available on the both sides of the fork (the existing currency and the new one) afterwards.
The value of the coins on the new fork may initially be substantially less than those of the original Bitcoin, but from then on it's up to the market to price them according to the value people see in the new currency. If the new currency provides the same features - and more - it could replace the old one.
It takes some consideration to understand that this process of spinning off is not harmful, but beneficial in multiple ways.
- it presents a more inclusive method of governing the evolution of the protocol
- it is an intrinsic defense mechanism to protect against capture by any subset of its economic actors (anyone can fork at any time)
- it allows using fixed time plans to upgrade instead of uncertain threshold voting
I believe these reasons are strong enough for hard forking to become the preferred means of upgrading Bitcoin. Several of the top cryptocurrency contenders (Ethereum, Monero, Dash etc.) fork hard successfully on a regular basis, demonstrating that much of the fear that has been spread around hard forks is unjustified.
Quo vadis?
My focus now will be to use the knowledge I've gained in developing this prototype to work on better fork implementations, following the BTCfork roadmap.
The first priority will be to assist the BTCfork project in the open development of a Minimum Viable Fork (MVF) - i.e. a spin-off which includes only a minimal set of changes and yet strives to be viable. Requirements and design of such a fork are being specified in collaboration with the wider community, and implementation will proceed shortly, based on the input from many supportive developers.
At the same time, I'll collaborate with others to bring about more fully-featured spin-offs which add features voted as desirable by the community.
These spin-offs will venture beyond the MVF requirements in their feature sets, providing for example:
- block sizes > 2MB
- better defensive mechanisms (e.g. against DDOS and hashrate-based attacks)
- improved output to assist users to transact more safely in the expected turbulent period after the fork
Guiding principles behind my fork development efforts
In implementing HFP0 and my future work towards an initial spin-off of Bitcoin, I want to preserve what I feel are important qualities of the current protocol. Some are economically important, some are there for technical reasons, to ensure the secure and safe operation of the system in the context of current infrastructure realities. I believe the technical ones could well be changed by forks in the far future (several years out), once Bitcoin scaling is on a good track and other aspects such as faster confirmation times move into the spotlight.
| Principle | Reason |
|---|---|
| Preserve the 21M coin limit | This is a fundamental principle of Bitcoin and part of its attractiveness, that the total supply will be limited. A spin-off that changes this would alter the economic premise entirely |
| Preserve the existing ledger | There are enough altcoins that start a new ledger. A spin-off is fundamentally not about taking ownership away (even in the form of requiring new investment in a new ledger), but enabling existing participants to exercise choice on the protocol running the existing ledger. |
| Preserve the Nakamoto consensus (valid chain with most proof-of-work) | This is part of Bitcoin's security model, and should be unchanged by a spin-off. |
| Preserve the 10 minute block period | Lowering this parameter could interfere with safe block propagation, and requires changing other parameters (e.g. block reward) to compensate to keep the economics unchanged. The complicated set of parameter changes would have downstream impact on other systems and should better not be mixed with an initial fork to increase the block size. |
| Preserve the halving schedule | The Bitcoin halving is an important factor in the existing design, and changing it requires careful consideration due to dependencies with other parameters. At the very least, it attracts great public attention to the Bitcoin market periodically. |
| Preserve the P2P qualities of the protocol | Bitcoin's peer-to-peer nature is one of its great achievements and drawing points. Changing this by introducing some form of centralization would endanger public interest in Bitcoin. |
As there is divergence of opinion in the community on whether it is preferable for a spin-off to change the POW, it makes sense that both options should be put to the market to decide. They each have advantages/disadvantages, and which way the market decides will probably depend significantly on the behavior of the existing miners and developers in the run-up to hard fork deployments.
| SHA256 variant |
Modified POW variant |
|
|---|---|---|
| Advantages | ||
| + simple, secure, proven algorithm | + re-decentralize mining (lowers the barrier of entry to participate as miners) | |
| + capitalize on existing ASIC hashing power | + reduce likelihood of future ASIC development, thereby discouraging centralization | |
| + allows existing miners to support the spin-off | + can enforce political change if miners have formed cartel | |
| Disadvantages | - exposure to hashpower attacks | - more complex algorithm, potentially vulnerable in new ways |
| - no real chance of general public participation in mining the fork due to ASICs ramping up the difficulty | - reduced security compared to ASICs | |
| - reserves seats for existing mining establishment, potentially allowing existing politics to continue | - impacts on existing mining hardware investments | |
HFP0 supports both variants through the use of a new configuration parameter: --disable-newpow .
By default, it builds with a modified POW based on Colin Percival's well-known scrypt algorithm (see also RFC 7914), which is also used by several other cryptocurrencies. HFP0 uses the modified scrypt implemented by satoshisbitcoin - refer to src/crypto/modified_scrypt_* .
By configuring the build with --disable-newpow, the new POW is completely disabled and the existing SHA256 is used.