I really want to like Bitcoin Unlimited...

[theZerg]

@Gavin Andresen, if you are interested in helping, perhaps you could think about what should be done with the num sig ops limitations and how they can be applied to an "unlimited" philosophy? I do not have a good handle on how num sig ops can reasonably verses unreasonably expand as block size expands, and what user options (if any) should be provided to tweak these limits. Right now sigop limits are commented out -- this is probably not a good idea to leave unaddressed if the unexpectedly fast uptake continues since I hear that its possible to craft a txn that takes n^2 time to validate.

Or, you could provide some concrete, fixable feedback.

 

[Georg Engelmann]

I like the idea of having multiple implementations, but I think for now it would be better if theZerg & other BU contributors made pull requests for new features / other changes on the XT repository and only had BU specific changes in the BU repository.

New code should be reviewed thoroughly.

 

[YarkoL]

What can be fixed is the lack of unit tests for the
new code additions. I'm willing to look into this when
I have some free time.

(I'm ISTQB-certified software tester, although
it's been a while)

 

[theZerg]

Open a new topic when you have time. I can set u up with some scripts I based off of python-bitcoinlib and maybe a bunch of vps.

 

[Gavin Andresen]

What to do about sighash N^2 problem:

That's tough, because if the philosophy is "accept any hard fork that increases the block size" you need to be compatible with any of several different ways the problem could be fixed, from limiting the size of a transaction to accurate sigop counting to replacing the size limit with a unified validation cost metric.

I've been trying for many months to get developers to at least agree on how to fix the issue. Since it is an issue for segwitness, perhaps NOW they'll get around to thinking about it.

I'll segue into some concrete, fixable feedback:

You need to figure out who the customers are for Bitcoin Unlimited. Miners? Exchanges/merchants? End-users? All of the above? I think one of the biggest problems with Bitcoin Core is it is trying to be everything to everybody, and since there aren't the management or testing resources to be everything to everybody, nobody is satisfied.

It is natural and healthy for specialization to happen, and splitting out the consensus code into a separate library was supposed to make that easy... but the consensus library has been a failure (last I looked there wasn't even a final decision on whether it would be C or C++, or if it was C++ whether it was OK for it to be C++11....).

If Bitcoin Unlimited is not for miners, then you don't need to worry about the sighash/sigops limits-- just make them unlimited. If you're worried about a very expensive-to-validate block causing a temporary denial-of-service problem, then fix that problem (validate new blocks in a separate process or thread).

 

[theZerg]

> That's tough, because if the philosophy is "accept any hard fork that increases the block size" you need to be compatible with any of several different ways the problem could be fixed, from limiting the size of a transaction to accurate sigop counting to replacing the size limit with a unified validation cost metric.

It may be easier for BU than some of the other clients. I am not looking to reject transactions from a consensus perspective (as our philosophy is to accept what the mining majority produces) so our compatibility issues might be limited to making sure that we are more lenient then everyone else. But as you mentioned at the bottom, I am worried about DOS. How about a crawler that identifies BU or XT client nodes and then sends them a bogus expensive to validate txn, repeatedly? It does seem like threading could at least keep the client responsive as the validation occurred... Would be cool if we could estimate the time to validate a txn and enclosing block (if in a block) -- this would allow the client to make decisions based on its own capabilities -- for example to defer validation of a transaction until it is in a block or a block until it is not on the tip.


> You need to figure out who the customers are for Bitcoin Unlimited. Miners? Exchanges/merchants? End-users? All of the above? I think one of the biggest problems with Bitcoin Core is it is trying to be everything to everybody, and since there aren't the management or testing resources to be everything to everybody, nobody is satisfied.

I think that you have a very good point here. My personal opinion is that the wallet is not useful, except as a rarely used "savings" account.

So I originally focused the changes in BU as a node for exchanges, merchants, and people wanting to provide SPV-wallets with connectivity (hobbyists). Basically for people who don't care about the block size to just follow the mining majority. So yes I did comment out the sighash and sigopts limits.

But the situation today is that miners may be looking for new node software so there could be opportunity there. And we would want to capitalize on that opportunity if it helps break the 1MB Schelling point.

Ultimately though its not up to me; it is going to be up the the members to decide this by the BUIPs that get passed, and any development resources that these members are able to offer.

 

[Aquent]

What to do about sighash N^2 problem:

That's tough, because if the philosophy is "accept any hard fork that increases the block size" you need to be compatible with any of several different ways the problem could be fixed, from limiting the size of a transaction to accurate sigop counting to replacing the size limit with a unified validation cost metric.

I'm wondering how BIP100 solved it? I suppose it had the same base thinking that we do, but far more restrictive and gave miners far too much power which BU solves. Duno if Jeff Garzick wants to chime in to help.

 

[Peter R]

You need to figure out who the customers are for Bitcoin Unlimited. Miners? Exchanges/merchants? End-users? All of the above? I think one of the biggest problems with Bitcoin Core is it is trying to be everything to everybody, and since there aren't the management or testing resources to be everything to everybody, nobody is satisfied.

This is a great question for the obvious reason that it's important to know your customer when designing a product, but also because it helps with the metamorphosis underway with people waking up to the idea that we don't need to run Core (and that we don't need to all run the same thing).

Presently, I believe Bitcoin Unlimited customers are node operators who support the idea that Bitcoin is a creature of the market and should be governed by the code people freely choose to run. Our customers are probably presently choosing BU for ideological reasons and to cast a vote against Core.

Now let's imagine that the civil war ends and we get bigger blocks (with Qmax > Q*). Who are our customers then? Or perhaps a better question is: Who do we want them to be?

I would personally like to appeal to miners if it's possible. Intellectually, I'm most interested in Sector 3 below (with overlaps into Sectors 2 and 4).

I think Bitcoin Unlimited could become very useful to miners because:

1. It allows them to more easily control variables such as block size limit (BSL), and in the future things like #sig-ops, etc.

2. I think we're planning to add signalling to BU and this would help miners to make decisions (e.g., nodes could be polled for their BSL settings; miners could write their BSL into the coinbase TX as well as signals such as "8 MB March 1, 2016.")

3. I think it would be reasonable that miners would contribute funding to our efforts if we were focusing development on them.

4. I want an excuse to work on subchains

EDIT: Haha I see @solex just started a BUIP regarding Item 2 (signalling).

 

[cypherdoc]

It is natural and healthy for specialization to happen, and splitting out the consensus code into a separate library was supposed to make that easy... but the consensus library has been a failure (last I looked there wasn't even a final decision on whether it would be C or C++, or if it was C++ whether it was OK for it to be C++11....).

wow, that's news to me.

what happened to your idea of max tx size of 100kB? who cares if it takes 10y for you to realize it needs to be changed. we change it then.

 

[Mengerian]

You need to figure out who the customers are for Bitcoin Unlimited. Miners? Exchanges/merchants? End-users? All of the above?

Yes, excellent point. My recollection of the discussion surrounding the genesis of Bitcoin Unlimited was the concept of non-mining nodes and their dual functions of (1) robustly tracking consensus and (2) validating transactions.

So, as I see it, Bitcoin Unlimited is a way for non-mining nodes to say to the miners: "We will accept the longest proof-of-work chain, including blocks larger than 1MB, whether the miners are following BIP100, BIP101, or any other limit schedule, we will accept the blocks (subject to our own individual capabilities)".

Incidentally, I think this also ties in to the question regarding Replace-by-Fee. RBF is basically a miner policy of what transactions to include in the blocks they are building. So if BU is catering to non-mining nodes, it doesn't have much influence on RBF policy of miners (it could make policy choices on double-spend relay though).

 

[Christoph Bergmann]

stupid question from a laymen ... couldn't the problem of the huge transactions be solved by letting the user decide what's the largest transaction he's willing to propagate?

 

[Richy_T]

I don't know why I can't quote Christoph but that seems to be a fair suggestion to me. There is already a minrelayfee setting. Node operators should decide how their nodes will operate.

Clearly they should be accepted if they're in blocks but I see no reason nodes shouldn't feel free to refuse to handle them in the mempool.

As to double spends, if a transaction isn't in a block, it's valid even if it attempts to spend the same as another transaction that also isn't in a block. It's up to miners to decide which one makes it into a block (this is why I have a hard time opposing the idea of RBF (though I object to the way it's being implemented against community opinion)). Though clearly there needs to be some protection against abuse. I'm not sure what that might be. I think XT relays double spends though I'm not sure of the rationale behind that (double spend detection?)

Edit: Hmm, the quote button just appeared so don't worry about that. Another thing might be to have slow processing validations pushed into a queue in a thread with a lower priority. That way they just get in the way of each other and not smaller, lighter transactions.
[doublepost=1451755789][/doublepost]

What to do about sighash N^2 problem:
It is natural and healthy for specialization to happen, and splitting out the consensus code into a separate library was supposed to make that easy... but the consensus library has been a failure (last I looked there wasn't even a final decision on whether it would be C or C++, or if it was C++ whether it was OK for it to be C++11....).

Libconsensus needs to happen though given what has happened, I'm not sure Core are the right people for that. Seems to me that step 1 would be to implement it in whatever bitcoind is written in (some flavor of C++) and go from there. Though it wouldn't upset me to see it written in plain C.

Unfortunately, I'm not a hardcore dev but I am hoping to look more deeply within the bitcoin code. Maybe it will be possibly to build some momentum. Even Jorge has some interesting things to say on the matter (well, I say even but he is a Professor of computing after all for all his skepticism)

 

[solex]

What to do about sighash N^2 problem:
You need to figure out who the customers are for Bitcoin Unlimited. Miners? Exchanges/merchants? End-users? All of the above? I think one of the biggest problems with Bitcoin Core is it is trying to be everything to everybody, and since there aren't the management or testing resources to be everything to everybody, nobody is satisfied.

This is a great question. I am reminded of the Rime of the Ancient Mariner
http://thedancingrest.com/2013/09/30/from-the-rime-of-the-ancient-mariner-by-samuel-taylor-coleridge/:

The fair breeze blew, the white foam flew,
The furrow followed free ;
We were the first that ever burst
Into that silent sea.

The ship hath been suddenly becalmed.

Down dropt the breeze, the sails dropt down,
‘Twas sad as sad could be ;
And we did speak only to break
The silence of the sea !

All in a hot and copper sky,
The bloody Sun, at noon,
Right up above the mast did stand,
No bigger than the Moon.

Day after day, day after day,
We stuck, nor breath nor motion ;
As idle as a painted ship
Upon a painted ocean.

---------------------------------------------------------
Bitcoin Unlimited needs miners as well as full-node users. It needs to surpass 50% hashing power or it will have unfurled bright into the open sea on a fair breeze, and then become suddenly becalmed, like a painted ship on a painted ocean.

 

[awemany]

So as someone having some (very, very minor!) commits in BU, I have to say that Gavin is obviously absolutely right in his assessment what and where the code is currently lacking. He might have pulled us back from cheerful optimism to a more realistic position, but that is a service.

For that matter and the same reason, I would also still take criticism by e.g. Greg Maxwell seriously, should he suddenly join this forum and say something reasonable.

But after all, one should also clearly mention that BU made a necessary trade-off between having comprehensive test suite, cleaned up commits and being out of the door early.

I personally think that trade-off is fine so far - but one of the first things that should happen before implementing any other functionality in BU is IMO putting in all necessary regression tests, cleaning up the commits etc. so that BU becomes 'Gavin-approvable'.

As other's said, I'd be glad if @Gavin Andresen would be willing to review an updated and cleaned-up code base.

Finally, @theZerg, I don't think Gavin meant that as a personal attack at all, and as you explained well, you made the necessary trade-offs with your limited time. I am in no position to criticize that, I rather think that from purely an 'this is how the code base ought to be, with sufficient time and resources' POV, Gavin is right.

Unfortunately, resources are constrained.

I definitely appreciate the hard work you put in to pull this all together!

 

[TD-Linux]

>
It may be easier for BU than some of the other clients. I am not looking to reject transactions from a consensus perspective (as our philosophy is to accept what the mining majority produces) so our compatibility issues might be limited to making sure that we are more lenient then everyone else.

If this is the case, why not use a SPV wallet instead? Or is the idea that BU will become a SPV wallet derived from Bitcoin Core?

 

[theZerg]

I meant that I am not looking to reject transactions and cause a fork just because some transaction in a block has a few more SIGOPs than this node can easily handle. Number of SIGOPs is not a consensus breaking issue IMHO, just like block size is not. At the same time, a node could choose not to validate or relay a transaction with an excessive # of SIGOPs until it appears in a block. And if/when it does as Gavin suggested we could do the validation in a separate thread so the rest of the app doesn't stutter to a halt.

 

[Peter Tschipper]

I'm new here in this forum and I'm really liking this idea of BU. Will it succeed in the long run, who can say - but all great things come from small beginnings...

There is a great deal of dis-satisfaction with where Bitcoin Core is going. I see this project as a good and positive outlet for all those who are wanting to move in a different direction, and to push the technology forward in terms of scalability.

Lots of work to be done and edges to be ironed out with the processes. Gavin has made some suggestions regarding the maintenance of the code base and I would tend to agree. There has to be a streamlined system for submitting PR's, making code changes and then commiting and merging in a sensible way, having a huge Git log is not really a good idea - unnecessary commits need to be squashed. In addition, the regression testing is an important area: Travis builds are needed. But I don't think it's time to be overly critical, this project is just starting up. Areas of improvement need to be clearly defined so that work can proceed in bringing processes up to snuff.

 

[YarkoL]

@Peter Tschipper
I fully agree on all points. But as you noted, things
are still getting organized (However it wouldn't hurt to
setup Travis even as this early stage).

 

[freetrader]

> streamlined system for submitting PR

Ideas?

(I would really like us to have a BUIP0 as an equivalent to BIP1... - also, Process BUIPs in addition to Engineering & Informational?)

 

[Peter Tschipper]

I really think we should leverage the well oiled development processes from Core. The one thing Core does really well IMO is do a great job of maintaining the code base in a safe and reliable way. Just because we don't like their direction doesn't mean they haven't done anything right. They have a done a great deal and continue to do so, I don't think there is anything wrong in recognizing that fact. Why re-invent the wheel here. I see nothing wrong with modeling their processes and then making enhancements/changes where needed.