Do we need to mount a more extensive defense against RBF?

[freetrader]

Apologies if this has been discussed elsewhere, but I've not seen a detailed discussion or answer:

With the recent merge of opt-in RBF and the even more recent attempt to sneak in Full RBF using a new policy mechanism, I am growing increasingly disconcerted about the impact of pervasive RBF on Bitcoin's future.

It is clear to me from reading merchant opinions that it will be very detrimental to anyone who currently finds zero-conf to have an acceptable level of risk to their business.

Peter Todd has announced that he is planning to create a fork which effectively speeds up full RBF by finding like-minded policy peers:

19:08 petertodd jonasschnelli: well, devils advocate, I'll still be doing a full-RBF fork with code to make the policy option find similarly policy peers

Source: http://bitcoinstats.com/irc/bitcoin-dev/logs/2015/12/17#l1450379288.0

I don't know if Core will reconsider the existing merge, but I'm not hopeful.

What else can be done to defend against RBF? If core does not revert the merge, would it be useful for another fork (or pull request on existing fork) to be created to actively limit RBF transaction propagation?

While I realize such a move could not entirely prevent RBF from taking a hold, I think it may be better than doing nothing about it. As such it could conceivably become a desirable feature for those who wish to preserve the utility of zero-conf to the maximum degree possible.

 

[Zangelbert Bingledack]

Hmm, in some sense if Peter Todd is able to succeed in forcing RBF to be the dominant assumption in the network through some kind of peer-finding, wouldn't that in itself prove his thesis that zero-conf is a sitting duck? (Actually no, I guess that would only be true if it didn't require a fork. I'll leave this here in case there's something to this line of thinking.)

 

[freetrader]

A good question is whether it is acceptable, in the context of good software engineering, to include a proposed policy mechanism in a distributed system when the policies are not completely expressed by the software.

As we see in the case of the RBF policy functionality, which leaves a door open to outside implementations to avail themselves of an unimplemented policy (full RBF) that was, at the time the merge decision was made, not considered acceptable to implement in the product.

To me it smells of negligence.

 

[Justus Ranvier]

One of the first uses for OT voting pools will be to allow people to self-insure their on-chain zero conf payments.

The idea is that users place a small amount of bitcoins in their insurance fund tied to smart contracts that pay out anyone who can produce a proof that the user double-spent a payment they agreed to make.

The servers that make up the pool will act as automated oracles for paying out claims.

Once we have all that working, we'll be able to easily create anti-Todd insurance smart contracts. Users can continue to shop with 0-conf payments by promising not to use dangerous anti-features and backing these promises with escrowed insurance funds.

That should make the risk sufficiently predictable and low that retail Bitcoin payments can continue to work as they do now.

 

[Windowly]

@freetrader, I believe this is another reason why we need to start a minority fork (but with the economic majority) fork soon -- and put in place developers we can trust who will not add RBF to the code.

 

[jl777]

In the vast majority of confirmed transactions the sequenceid is 0xffffffff and presumably with CLTV 0xfffffffe will start appearing.

I havent investigated fully the RBF yet, but can someone explain exactly how it allows to unwind an already committed tx? With micropayment channels, most of the back and forth is offchain and only at the end pushed to the blockchain. Granted stuffing the mempool with all these transitory incremental tx is probably a bad idea, but I am not understanding why it is zero-conf apocalypse.

Currently the miner's behavior is undefined and they are even ignoring all mempool and just grabbing the blockrewards. Maybe I just need to see a specific example of tx series to better understand why RBF is evil

James

####
OK, I saw the posts about a zeroconf tx being overridden with a higher fee. But wouldnt this be precluded if the tx that is pushed to the network has a sequenceid of 0xffffffff? My understanding is that the lower sequenceid's are used during offchain negotiations and when it is finalized, the sequenceid is change to 0xffffffff to make it unchangeable. Is there a reason zero-conf cant use sequenceid of 0xffffffff?

 

[freetrader]

@jl777 : I think this document should answer your specific questions w.r.t. the current implementation of opt-in RBF:

https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki

To your question "Is there a reason zero-conf cant use sequenceid of 0xffffffff?":

What you are describing above is a way for the sender to opt out of RBF for a transaction.

This is only a feature of the opt-in implementation - it does not lessen the overall harmful systemic effects on 0-conf which would result from widespread use of RBF.

Here is a well-researched article illustrating how 0-conf can be harmed by RBF :

https://chrispacia.wordpress.com/2015/11/29/on-zero-confirmation-transactions/

 

[jl777]

OK, so the issue is that RBF as optin is not an issue, but rather if it became an optout it would be?
I read the BIP125 and with all normal tx using sequenceid 0xffffffff and only CLTV enabled ones to use 0xfffffffe, the only negative effect I see is that the upcoming CSV which has to use sequenceids that are lower is forced to use RBF. It does seem strange that only 2 values out of 2^32 are allocated to RBF disabled and when I questioned about that I got some strange non-tech based answer from gmax.

So clearly this issue is politicized, no doubt about that.

I just want to know specifically how zero conf IS harmed by optin RBF. I think if one additional restriction is added to BIP125, it would actually remove ambiguous behavior and allow using the mempool to store in progress micropayments, but as I said earlier that is probably not a good idea to encourage. what happens in a payment channel should stay in the payment channel, until it is finalized with sequenceid 0xffffffff

So what I see objectively is that 99.9999998% of the sequencid bitspace is dominated by RBF enabled, when at most it should be 50%, so it prevents using sequenceid to store out of band information, which again would probably be not a good idea.

James

P.S. The change to BIP125 that would make RBF safe is:
all nonchange output amounts must be greater than or equal to the tx being replaced
which creates a need for identifying which one is the change output, details details. the devil is there

 

[_mr_e]

One issue with RBF is that it's likely a "boil-the-frog" approach to getting full rbf merged which changes the dynamics satoshi set it. Peter Todd is already pushing for this: .
RBF is very much needed for blockstreams plans to be pushed through and it is very interesting how much they enforce the need to for unanimous consensus on big changes unless it suits their agenda.

The other reason is spelled out quite well in this thread:

https://www.reddit.com/r/Bitcoin/comments/3zju81/rbf_optin_a_man_walk_in_a_bar_order_a_coffe_drink/

 

[jl777]

it does seem to be the camel's nose, but as it is I do not see why zeroconf tx are using sequenceid's that are not 0xffffffff. any such input indicates that it isnt final. so we are mixing non-final tx with zeroconf?

maybe once I understand this requirement, I will understand better

the way I would implement zeroconf is to have the other party push to the network and monitor my peers and wait for 100% of them to have it. that way no data points are lost as nodes wont relay a tx you sent to them and short of the attacker feeding all your peers the valid tx while blasting the doublespend to all other nodes, it seems that 100% agreement on the zeroconf is a pretty good security for low value tx.

Where in that is sequenceid needed? Of course, the loss of sequenceid to store out of band info is lost for insufficient reason, but that probably is not any point that will get much sympathy outside of people trying to compress the blockchain. My understanding is to use RBF enabled tx only for offchain and set sequenceid to 0xffffffff when it is final.

All these coffee examples dont make sense to me. Why did he pay with an input flagged that it will change? You do realize 99.999% of all inputs have sequenceid of 0xffffffff

I think the real issue is the change of sequenceid to mean enable RBF and allowing it to dominate all other possible uses for that bitspace. Would be better to have 16 or 24 bits of non-RBF values, not just 2 values

James
[doublepost=1456549676][/doublepost]Of course I agree with just pushing through a permanent change without consensus is wrong, but I think making the technical argument more correct will help. As it is, assuming the zeroconf uses RBF enabled sequenceid, is a confusing point for me

 

[freetrader]

@jl777 I agree with you that opt-in RBF as implemented now stomps all over the nSequence space in a very cavalier way.

But I must admit I don't entirely follow your other arguments, and frankly some seem a little circular to me.

the way I would implement zeroconf is to have the other party push to the network and monitor my peers and wait for 100% of them to have it

Wouldn't you agree that with a design that removes the decision-making aspect of signaling from the protocol data, an attacker could poison RBF decision-making by controlling only a few nodes because of the high degree of connectedness of the p2p network?

Apart from that, it seems unlikely to me that an approach of "I'll wait for all my peers to do X before I also do it" could work well in a p2p protocol, as it produces a chicken and egg situation.

The coffee shop example is a thought experiment - what happens when people start to use RBF in real life, and what are the likely real-world knock-on effects.

I think you are looking at it from an individual user's "can i safely opt-out of RBF" point of view, and finding nothing wrong with it.

One has to acknowledge though that there are real world risks if RBF is widely available.

The fact is it makes it easier for criminals to conduct double spends.

Core developers admit that to protect against this, you need an intelligence network of monitoring nodes which assist you in decision making. They trumpet the fact that some large enterprises have such monitoring protection already. This does not help Joe Average Merchant protect his business.

Then there is the question of whether it is a good idea to introduce a mechanic which enables a bidding war for block space, and who benefits from that.

 

[jl777]

I cant agree or disagree with abstract examples that dont actually show the specific tx usage of sequenceid. I am thinking about this from an alternate core developers point of view. There is already a mechanism for miners to decide to make more money by using the tx with higher fees. I think it is called the "comparison" operation where you compare X and X+delta and decide you like X+delta more.

"high degree of connectedness of the p2p network"?

Why do you consider a high degree of connectedness? 8 to 16 nodes out of 6000?

I like to work with math, not subjective examples.

sorry but I do not see evidence either way to be able to conclude either:
###
One has to acknowledge though that there are real world risks if RBF is widely available.

The fact is it makes it easier for criminals to conduct double spends.
###

Please tell me use cases where RBF and zeroconf have to be used in conjunction. zeroconf is the least secure so why combine that with temporary sequenceid? It seems a very bad design. Also CURRENTLY the miners that are financially motivated could behave in the exact same way as with RBF. But since miners are all so altruistic I guess this is not an issue at all?

Please tell me specific cases when sequenceid other than 0xffffffff needs to be used so that zeroconf works. other sequenceids indicate it is not a permanent transaction, which means it will be changed and usually these are all exchanged offchain. By the time it is pushed to the network you change the sequenceid to 0xffffffff, avoiding RBF.

currently sequenceid has UNDEFINED behavior and as such it really cant be used other than offchain, and that has the risk the other side broadcasts it early. At that point I dont see how you can end up reversing what is already sent, but maybe following transactions wont be confirmed. that can happen NOW.

technically, RBF appears to provide a way to know what willl happen. If the BIP is adding a new constraint that doesnt allow changing the output destination, the chance of using it to reroute funds might be able to be closed. to do that some convention about the change output and which one is active would need to also be decided.

I have written a bitcoin core (without all the cases covered yet) from scratch, so I need to fully understand what the protocol and blockchain fields actually do. I thought this was the Bitcoin protocol section and not the Bitcoin politics section. I know if politics are being used against you like in this case, the instinct is to tit for tat, but to use a flawed technical argument does not help your case.

The most valid objection to RBF to me right now is the total domination of sequenceid, we might as well rename it RBFid.

I do not have time to get involved in debating non-technical presumptive declarations. I have had enough of that elsewhere. I am totally open minded as to seeing specific usecases where combining zeroconf with sequenceid's that are not 0xffffffff are needed. The only one I can think of would be that CSV would be forced to use RBF. One way that limits the RBF bitspace domination is to allocate the LSB to being its flag. this sacrifices the timestamps to 2 second resolution, but it would allow for both RBF and non RBF usage across the entire sequenceid bitspace.

So lets discuss technical details, instead of trying to bully people with declarative statements that are not having the tech details. OK?

James

P.S. On the politics issue, have you considered you are being misdirected to fight the wrong battle? Maybe you are setup to being discredited or distracted from the real issues. Clearly political shenanigans are going on, but using a weak tech argument only makes you look like an alarmist.
[doublepost=1456614176,1456613288][/doublepost]sorry to discredit another of your points, but on the chicken and egg comment, it appears you are not understanding fully how the zeroconf works. Since zeroconf is not any official documented this, you need to deduce how to implement it properly, so unless you are a core dev, it will not be expected to be common knowledge.

Here are the constraints with zero-conf.
1. Nodes that you send a tx to wont relay it back to you
2. Nodes that you are not connected to are hard to communicate directly with
3. We assume the attacker has 100% connectivity to the network
4. We want to maximize the chances that when we accept zeroconf it really did propagate
5. We assume there is an offchain communication between the two parties, ie QR code

Now the most effective attack against zeroconf would be if the attacker knew the exact peers your node has and created two sets of nodes, ones with direct contact to you and ones without. Simultaneously all the nodes you are in contact with are pushed the valid tx and all the others are pushed the double-spend. To reduce latency and get the timing as precise as possible you transfer all but the last byte for all connections. The protocol specifies how many bytes to expect, so all the nodes are now waiting for that last byte. Then you using 6000 threads to simultaneously release that last byte. This would maximize the chance for the attacker's desired pattern to be achieved, ie all your peers tell you the tx you want to see, while 99% of the network has a different one.

Notice RBF plays no part at all in this attack. and why on earth would non-permanent sequenceid be used?

Now with the above assumptions and attack scenario, I claim that the fewer of our peers that we get data from the worse it is for us. So while if the attacker does the above perfectly, it will most likely work, for low value tx it just isnt worth all the effort to set it up, we hope. The way to get the most data is to NOT relay the tx at all, but let the other side do it, that way all of our peers will INV us the new tx and we can see if all of them have the desired tx. When 100% of our peers have the tx, we conclude it is safe enough. If there is any disagreement, we simply wait for it to be confirmed.

Now that is how I plan to do my zeroconf and nowhere in there is a sequenceid that is not 0xffffffff any part of it, since it leads to UNDEFINED behavior at the mercy of a miner who might or might not want the extra fee.

This is my analysis of zeroconf, which combined with my analysis of RBF that leads me to conclude that only insecure implementations of zeroconf that for some unknown reason use RBF will be insecure.

Please enlighten me the errors of my conclusions

James

P.S. I know about 0xfffffffe being another permanent sequenceid to be used in conjunction with CLTV, in the above treat 0xffffffff as covering both cases

 

[freetrader]

@jl777 (James):

So lets discuss technical details, instead of trying to bully people with declarative statements that are not having the tech details. OK?

If you have read my opening statement, this thread is not intended to be limited to technical arguments.

Apart from that, I have tried to be helpful to you, but you don't seem to see it that way.

One of the recurring criticisms of Core developers is that they refuse to listen when potential negative economic consequences of protocol changes are highlighted. In fact, with the case of RBF, they implemented it in v0.12 despite strong community resistance on this very point.

to use a flawed technical argument does not help your case

I refer you to the above that not all arguments against RBF are technical.

I hope you understand that a technology may be perfectly implemented and suited to its aim, yet have harmful consequences to some greater good, even to itself. For example, intrusive advertising on the web has not only attracted the ire of its targets, but become a security and privacy nightmare.

I thought this was the Bitcoin protocol section and not the Bitcoin politics section

P.S. On the politics issue, have you considered you are being misdirected to fight the wrong battle? Maybe you are setup to being discredited or distracted from the real issues. Clearly political shenanigans are going on, but using a weak tech argument only makes you look like an alarmist.

Refer to what I said above.

When it comes to Bitcoin, protocol development requires cognizance of or at least an interest in the economic implications.

Perhaps your ideas can lead to an improved version of RBF.

I can only encourage you to submit it to whichever project you think is most receptive, in the form of a BIP, a BUIP or however Classic and XT accept change proposals.

I have written a bitcoin core (without all the cases covered yet) from scratch

That sounds interesting. Do you have a link that you can share?

 

[jl777]

https://github.com/jl777/SuperNET
docs.supernet.org has API bindings, but not so much text yet
I posted details at https://bitco.in/forum/threads/iguana-parallel-sync-full-btc-blockchain-in-30-minutes-uses-half-the-space-but-starts-up-instant.911/

while politics is unfortunately an unavoidable thing, I hope you can understand that I need to limit my involvement to technical issues so I can avoid politics as much as possible. I find that political debates are never ending with each side able to just ignore the other and repeating the same thing over and over again with or without any changes. This to me feels a waste of my time.

I agree that forcing RBF against community consensus is bad, I just dont agree with the zeroconf is broken on the technical level and I am certainly no expert on poliics, so I will limit myself to the tech side. If there are any valid uses of zeroconf with non-permanent sequenceid, I am interested to know of them as I am interested to know all aspects of bitcoin tech.

I missed that this thread was not about the RBF tech. sorry about that

James

 

[TrevinHofmann]

Non-RBF relies on miner altruism, or you must convince miners why it would be beneficial to them not to use RBF. Non-mining nodes are about as relevant to RBF as they are to Classic: not very.

If miners support RBF, you won't be able to stop users from getting their RBF transactions to those miners.

 

[jl777]

what stops a miner today from including the tx with a higher fee?

and still nobody answers the question, why on earth would you mix zeroconf with a non-permanent sequenceid. Find flaws in my zeroconf method above that using temporary sequenceids fixes. Short of that, there seems to be no technical link between RBF and zeroconf.

The argument that today it is possible, but nobody does it and RBF would make lazy miners who are currently leaving money on the table be able to make more fees... Is that really the strongest argument against RBF takeover of the sequenceid field? If the "we shouldnt make it easy for miners to make money" is a justification, why not just make it so all block rewards are distributed out to existing accounts? We are talking about wealth redistribution, arent we? Preventing miners from making the txfees that they are able to and therefore in some sense entitled to. I can make an equally hysterical argument that if RBF isnt made standard and we dont make it easier for miners to make more money, they will all just stop mining and bitcoin wont get any more blocks. Is that really the level of discourse that we want?

I am non-partisan. I will evaluate the tech as I understand it, regardless of it is agreeing with the party-line or not.

So your response is better than the starbucks example, but the whole making easier for someone to do something that they are already able to do? Maybe this is the equivalent to getting worked up about selling guns with bullets already in them, since that makes it that much easier for criminals to commit crimes. While technically the difficulty of independently purchasing bullets might impede the clueless criminal, is that really a strong argument?

Taking it further we would conclude, lets just eliminate the ability for RBF and make all transactions final when submitted. Sure, fine. but then please explain how you make micropayment channels work. And still there is no proven reason why zeroconf must use temporary sequenceids, so without such proof, or at least usecase, the entire "it breaks zeroconf and without zeroconf bitcoin wont be used by the masses and without that bitcoin will never scale" doesnt even have a basis. Though I really like how the nail in the horse's hoof brought the kingdom down thing is going on.

James