Subchains and other applications of "weak blocks"

[Peter R]

The purpose of this thread is to discuss and debate the idea of "subchains" as both a scalability technique and a method to improve zero-confirmation security. Discussion of other topics related to "weak blocks" is also welcome.

Background material:

1. Subchains paper

Abstract. Orphaning risk for large blocks limits Bitcoin’s transactional capacity while the lack of secure instant transactions restricts its usability. Progress on either front would help spur adoption. This paper considers a technique for using fractional-difficulty blocks (weak blocks) to build subchains bridging adjacent pairs of full-difficulty blocks (strong blocks). Subchains both reduce orphaning risk by propagating block contents over the entire block interval, and add security to zero-confirmation transactions due to the weak blocks built above them. Miners are incentivized to cooperate building subchains in order to process more transactions per second (thereby claiming more fee revenue) without incurring additional orphaning risk. The use of subchains also diverts fee revenue towards network hash power rather than dripping it out of the system to pay for orphaned blocks. By nesting subchains, weak block confirmation times approaching the theoretical limits imposed by speed-of-light constraints would become possible with future technology improvements. As subchains are built on top of the existing Bitcoin protocol, their implementation does not require any changes to Bitcoin’s consensus rules.

 

[sickpig]

@Peter R I'm really glad you're keep working on this subject.

We are in desperate need of scientific thinking.

Just look at Todd's last series of emails on btc dev mailing list, as @Justus Ranvier said all of suddend he feels the urge of solving a series of well known problems with proposed solutions that change bitcoin foundamentals.

 

[solex]

Congratulations @Peter R on a milestone paper, which may well become regarded as the most important on cryptocurrency since Satoshi's original work. (Hat tip to @rocks for the inspiration).

Its implications for smooth scaling are huge and its potential for fast confirmation (aka "0-conf") and short-circuiting of the recent trend to RBF is a real achievement.

Small point for Fig.1. Perhaps the phrase "idealized blockchain" is needed, or maybe even 4,3,5,4,4 subchains to avoid the superficial impression that consecutive sub-chains are of equal length.

 

[Peter R]

@solex

Glad to hear that you think the paper may be significant!

Regarding Fig. 1, I went back-and-forth on this point on how to draw it. But for some reason I didn't think to add an explanatory note that randomness will affect the number of Δ-blocks per strong block! I will do this for the next revision.

 

[Peter R]

Question for @theZerg or @Gavin Andresen:

What would it take to implement subchains as described here well-enough to begin testnet studies? I'm imagining a non-nested subchain where we can set the subchain factor to 4x, 10x, 100x, etc. We'd then set-up nodes at various locations across the globe and study how the subchains get constructed and how the nodes deal with reorgs.

 

[theZerg]

I'm sorry haven't read it carefully yet. Lot's to do in RL and releasing BU. But write up a BUIP!

 

[Peter R]

A Visual Explanation of Subchains

Subchains are a practical application of "weak blocks," which add security to zero-confirmation transactions and permit massive scaling of Bitcoin.

Fig. 1. Each time a block that satisfies the weak target is found, the subchain is extended. When a block satisfying the strong target is found, the subchain is closed, becoming a strong block, and a new subchain begins.

Fig. 2. Miners cooperate to build subchains in order to process more transactions and claim more fees without incurring additional orphaning risk. This illustration visualizes "idealized" ¼-difficulty subchains (also referred to as 4x subchains). In practice, each strong block may contain more or less than four weak blocks, due to randomness.

Fig. 3. Miners build subchains layer by layer (a – c), where each layer corresponds to the solution of a weak block. To propagate blocks (weak or strong), miners need only send their Δ-block and a reference to the subchain’s tip (f), reducing the quantity of transmitted bytes. When a nonce that satisfies the strong target is found, the subchain is closed thereby becoming a strong block (d), and miners begin working on a new subchain (e).


For further reading, please refer to "Reduce Orphaning Risk and Improve Zero-Confirmation Security With Subchains."

 

[Zangelbert Bingledack]

Fantastic, Peter.

 

[cypherdoc]

A Visual Explanation of Subchains

Subchains are a practical application of "weak blocks," which add security to zero-confirmation transactions and permit massive scaling of Bitcoin.

Fig. 1. Each time a block that satisfies the weak target is found, the subchain is extended. When a block satisfying the strong target is found, the subchain is closed, becoming a strong block, and a new subchain begins.

Fig. 2. Miners cooperate to build subchains in order to process more transactions and claim more fees without incurring additional orphaning risk. This illustration visualizes "idealized" ¼-difficulty subchains (also referred to as 4x subchains). In practice, each strong block may contain more or less than four weak blocks, due to randomness.

Fig. 3. Miners build subchains layer by layer (a – c), where each layer corresponds to the solution of a weak block. To propagate blocks (weak or strong), miners need only send their Δ-block and a reference to the subchain’s tip (f), reducing the quantity of transmitted bytes. When a nonce that satisfies the strong target is found, the subchain is closed thereby becoming a strong block (d), and miners begin working on a new subchain (e).


For further reading, please refer to "Reduce Orphaning Risk and Improve Zero-Confirmation Security With Subchains."

Reminds me of etotheipi's great diagrammatic work.

 

[solex]

Fantastic graphics, and if Peter ever gives up on Bitcoin he has a career producing hypnotic therapy for sleeplessness.

 

[Matthew Light]

This is intriguing stuff - seems like it would work well.

 

[Peter R]

Thanks, @Matthew Light. Welcome to the forum!

 

[priestc]

Any change to the way bitcoin works today needs to come with it a reasoning as to why the current way bitcoin mining is broken. I don't believe bitcoin mining as is today is broken, so I can not support this idea. Sorry.

 

[Peter R]

@priestc

I agree that things that aren't broken shouldn't be changed without a good reason, and this is why I like the subchain idea. It is a new protocol layer that sits on top of the existing Bitcoin protocol. It requires neither a hard nor soft fork to implement--only voluntary cooperation by miners and nodes. It is simply a more efficient way for miners to build and propagate their blocks, thereby allowing them to process more transactions per second. It has the nice side effect of adding some security to zero-confirmation transactions.

 

[theZerg]

Yeah @priestc it doesn't really change mining... just what u do with near misses.

 

[priestc]

@priestc
thereby allowing them to process more transactions per second.

The only factor that determines how many transactions per second a moner can process is determined by the blocksize. If the blocksize is still limited to 1MB, only 7 transactions per second can be processed, period.

It has the nice side effect of adding some security to zero-confirmation transactions.

Zero-confirm transactions are already secure enough.

It seems like your proposal is trying to decrease the block interval time without actually changing the block interval in the protocol. I think it would be better to just straight up decrease the block interval...

 

[Peter R]

"The only factor that determines how many transactions per second a moner can process is determined by the blocksize. If the blocksize is still limited to 1MB, only 7 transactions per second can be processed, period." -- @priestc

Yes, the block size limit of course must be increased too. What subchains do is allow more transactions per second to be processed at a given level of orphaning risk. If orphaning risk is lower, then miners will be more likely to produce larger blocks. Here is a chart from my paper that illustrates this:

 

[awemany]

It seems like your proposal is trying to decrease the block interval time without actually changing the block interval in the protocol. I think it would be better to just straight up decrease the block interval...

One huge advantage of @Peter R.'s / @rocks' approach here is that you do not have to signal a new block interval across the Bitcoin ecosystem, you can experiment with subchains as they are just a network thing and not in the consensus layer. Look how hard it is to change just the block size.

Another advantage is that I am not so sure about details of incentive structures etc. as the subchain block time approaches the network diameter in terms of latency. I do not think you want to come closer to that with the Bitcoin's main block interval without more thought and analysis. I think Satoshi's intuition regarding the 10min was along the lines of '10min is enough for changes to fully ripple through the network, if you reduce it significantly (towards seconds), weird things or bad things might happen'. Subchains might be a nice way to probe this region.

I could imagine, for example, that full nodes that collected transactions might be paid by miners for being served transactions.

 

[jonny1000]

Peter

I have now read your paper.

In the paper you cited one of Greg's issue with using orphan risk to drive fees:

"The use of subchains also diverts fee revenue towards network hash power rather than dripping it out of the system to pay for orphaned blocks."

This point was always contentious, but I thought you always disagreed with it. Are you now saying you agree with Greg's potential concern, since its mentioned as an issue in your paper?

The paper seems to be a good description of the interesting weak blocks idea, which is a potentially helpful way of reducing orphan risk. I do not understand how this helps with your contentions idea of using orphan risk costs to prevent fees falling to zero, all it does is make it worse as by reducing orphan risk costs and therefore fees even closer to zero.

I do not see any explanation as to why weak blocks mean orphan risk costs will be smaller relative to fee revenue than without weak blocks. Please could you provide your explanation for this as I can't see it in the paper? It seems this is now a pattern of behaviour:

1. Peter has an interesting idea
2. People criticise the idea
3. Peter denies there is an issue
4. Despite denials there was a problem, a new interesting paper is published claiming to solve the problem with a new solution
5. After reading the well written and genuinely interesting paper, it is apparent no solution to the problem is mentioned or addressed in the substance of the paper

I do see advantages of weak blocks, but these advantages are unrelated to the fee market issues we discuss.

 

[Mengerian]

Hi @Peter R

Thinking through some ideas on how to bootstrap the subchain system. Say you start with two miners, they could agree to communicate weak blocs between then, forming a subchain, both would benefit. The benefit to both would be maximized in a situation where they have have bad connection between them, say one inside China and its firewall, and one outside.

The miner producing a weak block has an incentive share it, as it will reduce his orphaning risk if the other miner downloads and verifies it. The second miner has an incentive to download the first weak block, since the fractional difficulty proves that the first miner is trying to solve that block, and he will have it quickly to start mining on top of it of he downloads and verifies it. He also has an incentive to do his mining on top of it, as then he can share his weak blocks, and full blocks, with lower orphaning risk as well.

Say the two miners each have about 5% of the total hash power on the network. If they share weak blocks at 1% difficulty, then they will on average produce about 5 blocks each, for a 10 block subchain, for every full block found on the network. About 10% of the time, one of these miners will find the full block. So if the subchain reduces the mutual orphaning risk between these two miners by 90%, for example , then the total reduction in their overall orphaning risk would be about 10% of 90%, or 9%.

Once two miners are mutually benefiting from subchains, others would have an incentive to join, so the network could expand from there. I think it would also make sense for many small miners to communicate using subchains even if big miners do not participate. They could build subchains with very low fractional difficulty to reduce their mutual orphaning, thus helping them compete against big, well connected miners.