1. Proposal for funding the development of blocktorrent
2. Bitcoin address
TBA
3. Motivation
Blocktorrent is a proposed block distribution protocol which aims to reduce block propagation times by a combination of techniques, for example, the splitting of each block into smaller, verifiable chunks. The blocktorrent network will be a fully P2P relay network which has a dynamic, emergent routing topology, and is independent of the existing Bitcoin P2P network. Blocktorrent probably isn’t really necessary for 1 MB blocks, as its performance is expected to be comparable to current state-of-the-art block propagation methods. But for block sizes beyond 8 MB, blocktorrent is expected to be significantly better.
By reducing block propagation times, blocktorrent will make larger blocks more feasible without significantly increasing miner centralization pressure. The blocktorrent network will also form a new block relay network that will have an unprecedented degree of decentralization.
Introduction
The current resource bottleneck to on-chain scaling is that larger blocks require more time to propagate across the Bitcoin network. This manifests as an increased orphan cost to miners who are unable to afford high-speed Internet connections, and modifies the incentive structure of block rewards to favour larger miners.
There have been many methods to improve block propagation. One such class of methods are those which reduce a block down to a list of shortened transaction hashes – examples include Xthin and compact blocks. These methods have been successful in reducing block propagation times. Blocktorrent will improve upon this technique by exploiting the fact that the merkle tree containing transactions can be split into chunks. Transactions themselves can also be further split into chunks.
As an example of the benefits of chunking, imagine that a miner wishes to broadcast an 8 MB block through a network full of 10 Mbit/s connections. An 8 MB block is about 128 kB in shortened transaction hashes. It takes the miner about 100 ms to send this block to one other peer. So after 100 ms, there are only two nodes with block data, thus there is 20 Mbit/s of uploading bandwidth able to continue broadcasting this block. But if this block is split into chunks of 16 kB, the miner can now send one chunk each to 8 different peers in 100 ms. These peers can broadcast those chunks to other peers, even if they haven’t received the full block. So after 100 ms, there are now 9 nodes uploading block data – 90 Mbit/s of uploading bandwidth able to continue broadcasting this block. In this way, blocktorrent’s use of chunking means that the network quickly reaches network-wide bandwidth saturation.
Blocktorrent will be a new relay network – existing examples of relay networks include FALCON and FIBRE. These existing relay networks are decentralized in the sense that anyone can set up one. However, in order to be effective, creating a new one requires the permission and participation of large miners. This means that in practice, relay networks are often set up or maintained by a small number of people, and this is a form of centralization. In contrast to this, the blocktorrent network will be a public P2P network where all nodes will have an automatic peer discovery mechanism, so that anyone wishing to join the blocktorrent network can easily do so.
Protocol overview
- The merkle tree containing transactions in a block can be broken into independent, verifiable chunks. The transactions themselves can also be broken into verifiable chunks thanks to the Merkle-Damgård structure underlying SHA-256, although those chunks are not independently verifiable. The blocktorrent protocol is broadly-speaking, like bittorrent, applied in such a way as to transfer those chunks from peer to peer.
- The protocol will be UDP-based, to avoid issues with TCP: slow start, and packet loss affecting speed. While the protocol will have a mechanism for reliable transmission of some messages, most messages will be sent with the assumption of unreliability. Using UDP also allows the use of NAT hole-punching techniques to achieve configuration-free P2P connections from behind NATs.
- Robustness will be achieved by a combination of redundancy (multiple peers) and erasure coding of data.
- The blocktorrent network will be fully P2P, with an emergent network topology. There has to be peer prioritization and banning of misbehaving peers. This makes the network robust and permissionless.
- For low-latency block propagation, peers can arrange to have some volunteered data pre-emptively sent to them. This is comparable to Bitcoin Unlimited's Xpedited technique. Peers with volunteering data arrangements have to do some sort of mempool synchronization so that volunteered data is actually useful.
- Nodes can be configured to have profiles that have bandwidth/latency tradeoffs. For example, a low-bandwidth node on a residential Internet connection would not waste bandwidth on volunteered data, with the consequence that they receive blocks later. On the other hand, miners would set higher volunteered data limits so they would inevitably end up sending and receiving more redundant data, but in return will receive blocks with lower latency.
- Blocktorrent can be used to propagate weak blocks – blocks with a lowered difficulty. Future blocks can then make compact references to these weak blocks. This can make the encoding of known transactions in a block very small – less than a byte per transaction. Miners who wish to have their blocks propagate fast should publish weak blocks which will be mostly identical to their real blocks – then block propagation will be approximately O(1).
4. Objectives
- To create a blocktorrent protocol specification that will result in a network with good block propagation performance across a large range of block distribution scenarios. This protocol will be based on the points outlined in the "Protocol overview" section.
- To implement a concrete realization of this protocol specification, appropriate for integration with Bitcoin Unlimited.
- To form an operational blocktorrent network.
5. Duration
This project will occur in two phases:
- Phase one - November 2016 to February 2017
- Phase two - March 2017 to May 2017
6. Team
Chris Chua (someone42)
7. Current work
Work on the python blocktorrent implementation has already started – see
https://github.com/jtoomim/blocktorrent-python. At this stage, the code is able to fill in and validate merkle trees, connect and exchange messages over a UDP-based protocol, and communicate one node's merkle tree state to another node. Funding will allow this work to happen at a much quicker pace, and at a much higher quality.
Simulations do indicate that blocktorrent demonstrates significantly better block propagation performance for blocks larger than 8 MB – see
https://eprint.iacr.org/2016/555.pdf
8. Description of activities
Phase one
The first phase will involve the development of a proof-of-concept blocktorrent node implementation written in Python (
400 hours). Development efforts will focus on implementing the features described in the protocol overview section, as well as instrumenting the implementation. The purpose of this phase will be to investigate protocol features and details, and to test blocktorrent under different real and simulated network conditions, and under different block propagation scenarios – e.g. larger blocks, and blocks with unknown transactions. A substantial amount of protocol design work will occur here based on the feedback from these tests.
The outcome of simulations performed using the Python implementation will be described in a series of forum or blog posts (
25 hours). These posts will attempt to characterize the conditions where blocktorrent performs well, and what kinds of situations cause performance degradation. These posts will also document some of the design decisions behind the protocol.
Phase two
The next phase will be the development of a C++ implementation (libblocktorrent), which will be a library with a well-documented interface (
350 hours). This library will allow Bitcoin Unlimited to interface with the blocktorrent network. This implementation will also be used to create a standalone client which is able to assist in block propagation without necessarily being connected to a Bitcoin node.
When the blocktorrent protocol has become stable enough, a human-readable protocol specification will be written (
15 hours). The specification will be written with the intention to submit it to various Bitcoin standardization processes. The specification will be of sufficient detail that a competent developer could read the specification and produce a working implementation.
All code from this project will be licensed under the MIT license.
