BUIP146: (closed) Add config parameter to delay mining transactions

[Jonathan Silverblood]

BUIP146: Add config parameter to delay mining transactions
Submitted by: Jonathan Silverblood
Date: 2019/5/11

Summary
The purpose of this BUIP is to make Bitcoin Unlimited a better choice for miners when used in a high-frequency network where transactions are created at a high rate, but propagation has imperfect latencies.

Proposal
This BUIP proposes that:

• Make a short study on a testnet to determine if there are possible benefits to delaying inclusion of a transaction into the block template
• Add a new configurable parameter to determines how long the node should wait before considering a new transaction as ready to be mined.

Motivation
To make the best use of our current block propagation protocols (graphene, xthin, xthinner - all that is based on some kind of set reconciliation) it is important to not have too strong fragmentation between the mempools of the peers in the network. By not including new transactions before we have given them a reasonable chance to propagate we are creating beneficial conditions for the propagation of created blocks which should reduce the number of orphans produced when mining with Bitcoin Unlimited as the full node software of choice.

Background
During a previous stress test on Bitcoin Cash, one of several network bottlenecks discovered related to the propagation of transactions. Not all nodes in the network will have the same performance, so we will always need to be able to work with nodes of varying ability to propagate transactions at low latency.

Note: In the discussion following this proposal, some of the assumptions that this proposal relies on have been broken. For example, it is said that the delay between accepting a transaction and mining on it is already ~30 seconds long. If this is the case both today and in the future, then this proposal to add extra delay is likely pointless. I suspect it is not, however, as from practical experience I have many times sent a TX and got included in a block within just a handful of seconds.

 

[imaginary_username]

It sounds very valuable even for the research alone.

 

[Norway]

What are perfect latencies?

 

[Jonathan Silverblood]

perfect latency is the speed of light for the distance between the nodes, or extremely close.

 

[cypherdoc]

How extremely close? Can you give us a number?

 

[Norway]

Speed of light in what medium? Vacuum or fiber?
[doublepost=1557578544,1557577617][/doublepost]This proposal works against the incentives in bitcoin. Not including transactions into blocks, is giving tx fees to other miners.

 

[jtoomim]

The main transaction propagation bottleneck (INVENTORY_BROADCAST_MAX bug) in the Sep 1st stress tests was fixed. However, mempools between nodes remained pretty well synced despite this bug, and it was not responsible for poor block propagation performance. It was mostly just responsible for small block sizes (3 MB average, IIRC).

At very high transaction throughput (e.g. as seen on the gigablock testnet), the transaction broadcast rate will exceed the rate at which slower nodes can process and validate transactions. If different nodes vary in performance by 50%, this means that over a 600 second block interval, some nodes might fall behind by 300 seconds. A 3 second delay is going to do nearly nothing to address this problem.

Due to the way that pools and the stratum protocol works, there is usually already around 5 to 50 seconds of delay between a transaction arriving at the bitcoind process for the poolserver and a block actually being mined with that transaction. I don't think it's necessary to add any more delay.

I don't see any harm in having this be a command-line option, but I would recommend a default value of 0, and I don't think this should be a high priority use of developer time.

 

[Peter Tschipper]

At current transaction rates there is no indication of any problem. For instance we see very little if any re-requesting of transactions for graphene or any other of the thintype relay protocols. However at higher tx rate, 100's to 1000's per second I could see the potential for some delay in propagation so I think it's not a bad idea to have a configurable setting (but agree with @jtoomim that it should be a default of 0), so that miners can be sure their blocks will propagate as fast as possible . It would then be up to the miners to decide when/if and at what settings they want to do this kind of filtering and at what cost/profit to them. So in that sense, to give miners more freedom to choose I think is a good thing.

 

[Jonathan Silverblood]

In discussions elsewhere, I have been told that the time between transaction acceptance and actually being mined on, is closer to 30s than to 0s today. I have also been told that 95% propagation normally happen in less than 2s.

With these two data points, this suggestion is moot for the current network. I will rewrite this buip to focus on the research part rather than the implementation part as no matter how I look at it, I either don't know enough about how the network is constructed, or this should matter at some higher frequency in the future.

 

[Peter Tschipper]

I think you have a very interesting BUIP...it highlights an issue I've never really thought about. I think when we get to the next round of giga-net testing, hopefully this summer, this problem you've identified might become visible and maybe we can look at incorporating your suggested fix and testing it out.

 

[Jonathan Silverblood]

@solex this BUIP needs a number. I'm not as confident as I was at the start about the value it provides, but BU relies on the wisdom of the crowd by means of the voting process, so I still think it would make sense to include it for voting.

 

[Roy Badami]

I'm a bit confused as to what exactly this BUIP would mean if passed. Is it asking for this work to be funded? (It doesn't seem to be.) Is it directing the Developer to do this? (The latter is not something that would normally be an appropriate way of getting development like this done.)

If you (or someone) wants to do the research, I don't think any permission is required. If you (or someone) wants to add the config parameter then, since this sounds non-controversial, a PR would seem more appropriate than a BUIP vote in the first instance.

 

[theZerg]

What a BUIP does is authorize and direct the Developer to merge code that implements the BUIP if it is provided. BUIPs do not direct what individuals actually work on. This design is part of the checks and balances in the BU Articles. So if this BUIP passes, it means that some developer could go off and implement the code and be basically sure that it would get merged (provided it meets quality standards).

I personally like that idea, but do not see it as a priority item given current and expected medium-term transaction counts. So I do not intend to work on this idea right now. However, I do intend to vote "for" this BUIP so that anyone else can work on it and have some assurance that their work will be incorporated into our client.

 

[Roy Badami]

@theZerg Thanks for the clarification. I've voted in favour on that basis - although the BUIP still seems to me to be a bit unnecessary.

My understanding is that an uncontroversial change can be merged at the Developer's discretion based simply on a PR. And I still don't understand what effect the membership "approving" some uncontroversial research would have (as far as I can see, none), since carrying out research doesn't require any permission from anyone.