BUIP051: (passed) Add CompactBlocks support

[freetrader]

Summary

In the interest of supporting better fast block propagation between BU and other implementation which only support CompactBlocks (CB) [1] but not XthinBlocks (XTB), it is proposed to add BIP152 support to the BU client.


Motivation

Currently, bridging between XTB and CB peers is possible by connecting via a BitcoinXT node. There are few of these (46 according to nodecounter.com). Supporting CB in BU would strengthen the connections between the BU peers and the rest of the network, without having to rely on intermediates.

Also, few other (non-Core) implementation currently support XTB. It is more likely that some support CB. Being able to exchange big blocks with them might be necessary in the near/medium term, and since some of them are quite different from Satoshi clients, providing reference XTB implementation for them will not be so easy for BU developers. It makes more sense to support their existing fast propagation protocol.


Implementation considerations

The implementation should, for safety reasons, prefer to use the more tested XTB with peers that support this service, and fall back to using CompactBlocks with those that do not support XTB.

User preference switches are recommended to

1. allow disabling CB support in the client entirely (such a kill switch should be supplied anyway for security purposes, should a flaw be found in the CB implementation, similar to `-use-thinblocks` for XTB)
   
2. allow the user to prefer CB to XTB with peers that support both

Notes:

1. This has already been implemented in Bitcoin XT, so hopefully together with the BIP152 specification it should be quite easy to port this feature to BU.
2. This porting work could be encouraged by making this a funded BUIP. The author has no advice for the effort and funding level required, but is open to suggestions, esp. from the XT developers who have already implemented this.

Additional information

[1] https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki

 

[AdrianX]

Great Idea @freetrader it saw https://bitcrust.org/features it introduced me to the possibility, and at first I didn't know what to think, anyway I've processed some of the implications and have some questions.

I like bitcrust approach to molecularity similar in principal to BU but just built form the ground up.

XTB and CB two technologies should not compete but complement each other to make the bitcoin network more efficient, they compete with relay networks like http://bitcoinfibre.org/ the obvious benefit to keeping efficiency gains on the bitcoin network is it reduces the need on external influences on block sequencing.

If you receive a CB block can you transmit the block as a XTB block or is it CB in CB out and XTB in XBT out?

How are XTB blocks and CB fowarded once verified?

My thoughts I see only need to send either the Xthin and CB bloom filter - I don't see a need to send both. to make BU the best client possible we should send blocks with the version that processes and validates the fastest. (XTB has tested benchmarks so it's quantifiable and until be have apples to apples data I think we should stick with XTB.)

The needs change once the network is transmitting blocks of different sizes, miners may favor one version over the other depending on blockchain condition so i think it is a good idea to keep them both current.

 

[freetrader]

> If you receive a CB block can you transmit the block as a XTB block or is it CB in CB out and XTB in XBT out?

Yes you can, this is what XT does, if I understand correctly. With either tech, once you've reassembled the block that you got from your peer, you can break it down to a bloom filter for your peers that speak the other protocol.

> How are XTB blocks and CB fowarded once verified?

I'm not sure I understand the question... they can be forwarded to XTB peers using XTB, and to CB peers using CB, or to either using the older p2p protocol for an uncompressed block (not sure if that is subject to some limitations, but I think it would work ok for blocks which are a couple of MB, unless you are a miner and want the blocks as fast as possible.

The idea with this BUIP is that a BU node would carry on, by default, to talk XTB with whoever understands it - because yes, that code has gone through more testing than any new CB code. And XTB has probably gone through more on-the-network testing than CB overall.

It's useful to think of these block propagation protocols similar to languages that a person can speak.

If you're in a crowd, and there are some Zulu speakers, and some Albanian speakers, some of whom also speak English:

• if you can speak fluent Albanian, of course you might use that with the Albanians
• if you can speak fluent Zulu, no problem communicating with the Zulus
• you can help those Albanians and Zulus who don't speak English or the other group's language by acting as a translator of 'blocks of information'
• if all breaks down, or you encounter someone who is not Albanian or Zulu, you and your peers might still fall back to English!

 

[jbreher]

Just chiming in to state that, after letting this idea gestate for several days, I like it, and can find niofault with the concept. Any possible implementation issues aside, I think it would be a net benefit for both Bitcoin and BU.

 

[Christoph Bergmann]

I like this too. It's a shame that Core refuses to use xthin, but the result should not be that everybody has fast blocks except BU nodes.

 

[torusJKL]

I think it is in the spirit of BU and the idea that there are multiple competing implementations to adopt working features from other implementations (even if it comes from Core) that do not pose a threat to BU and its goals.

 

[Roy Badami]

I'm in favour of this.

 

[JAFO]

I am in favour of this.

 

[seweso]

Yes!

 

[freetrader]

Picking this up for implementation. Initially at least for Bitcoin Cash, since effort for implementing and testing on BTC don't seem to be warranted for me at this stage (coin.dance today shows 34 BU nodes or only 0.35% of BTC network).

For the Bitcoin Cash network, this BUIP should have positive effects on the network at least until Graphene is widely deployed in other clients since it will allow BU nodes to communicate block data efficiently with ABC nodes.

Even once Graphene is deployed, it can serve as a fallback (since ABC nodes don't have Xthin).

 

[freetrader]

@solex, if I understand correctly @ptschip has implemented Compact Blocks for the latest BU release.

If so, I would like to thank him and suggest that you can formally close this BUIP.

 

[solex]

@freetrader
Yes, iIndeed, I understand he has completed this work. It is certainly a good addition to have to the BU implementation. We appreciate all @ptschip does.