Bitcoin Unlimited - Development discussion

[solex]

@Inca makes the point that RBF is a contentious change in Core.
Again, I really don't want to bikeshed anything as I realise that time is of the essence, however, it just may be that BU needs another check-box early on for the RBF changes in Core (default blank and needs checking by the user to enable?).

 

[theZerg]

As far as I understood it (haven't checked the code) RBF is merged into the coming .12 release. So it won't be in our 11.2... we don't need to modify BUIP001.

 

[sickpig]

@awemany @thezerg Yep, I've done a few gitian builds. You can re-use this for BU. https://bitbucket.org/bitcartel/bitcoinxt-obb-gitian

@bitcartel, @awemany, @theZerg

after looking at the aforementioned bitbucket repo I've seen that awemany and bitcartel gpg pub keys are already there. so technically speaking we already the minimum requirement to get a deterministic build, namely more the one individual participating to the process.

do you think two people this is enough? if it's not enough I'll give it a try just tell me the preferred method to exchange my pub pgp key.

wrt windows build, I've no experience whatsoever and I think it would be better to find some window geek to speed up the process.

 

[theZerg]

@bitcartel The instructions say to set:
export SIGNER=73AEB925

but I'm imagining that this number is specific to me... where do I get it?

 

[sickpig]

@theZerg it has to be your PGP/GPG public short key ID.

I didn't look at script helpers code but it sees the your pgp public has to be stored in the gitian.sigs subdir.

this is where @awemany's is stored /0.11.0-linux/reddit_r_awemany/awemany.pub, @bitcartel's is stored here instead: bitcoinxt-obb-gitian / contrib / gitian-downloader / bitcartel-key.pgp

since your pgp key is not in the master repo you should add it locally to your cloned repo.

 

[davecgh]

@Peter R I think alternative implementations, whether officially supported by BU or not, would help support the vision in promoting a heterogeneous network and letting users choose from different software. Of course there are limited resources at this stage, so I agree with @awemany that it's probably a good idea for developers to focus on one codebase, and most folk are probably familiar with core over btcd. In the short term, this would mean core would still be in the driving seat. However in the long run, if the project proves to be successful, BU could become an umbrella like Ubuntu with different 'flavors' such as bitcore+unlimited, btcd+unlimited which all implement the specifications and features that adhere to the BU vision. @theZerg Great work so far.

It would be pretty simple to "remove" the block limit in btcsuite/btcd. I put remove in quotes because there is still obviously the maximum message size limit which effectively limits block sizes to 32MB (or whatever the constant is set to). It should also be noted that the maximum number of signature operations is based on that constant as well, so those would increase with this change (the same is true in core).

Specifically change this constant to be:

const MaxBlockPayload = MaxMessagePayload

Perhaps also of note in relation to larger block sizes, we've been working on a pretty massive database and chain infrastructure rewrite to focus on better I/O performance which will have a significant positive impact on larger blocks.

Specifically, the following PRs are related to that work:

https://github.com/btcsuite/btcd/pull/380
https://github.com/btcsuite/btcd/pull/491
https://github.com/btcsuite/btcd/pull/492

 

[awemany]

@davecgh: Are you familiar with the inner workings of btcd?

I find today's bitcoind kind of crufty in places and hard to get a good overview compared to the earlier versions by Satoshi, and I think it also grew in size by some factors.

Usually, a from scratch reimplementation of a concept is in better shape because you're less likely to get the overall architecture wrong. Is that the case with btcd?

 

[davecgh]

@awemany: I'm the lead dev on it, so yes I'm pretty familiar with it! https://github.com/btcsuite/btcd/graphs/contributors

Due to that, my answer to your question is going to have some natural inherent bias in it, but basing it on feedback from external contributors, I've heard very few negative things regarding the architecture (the database and addrmgr being the most notable culprits, but as previously mentioned the database is being completely redone currently).

All of the important things are completely modular and btcd itself has no integrated wallet or GUI code to distract from actual core network development.

For an example of the modularity, there is a wire package for handling the network protocol, a peer package which builds on top of that to provide the framework for implementing full nodes (btcd itself is built on it), SPV nodes, proxies, etc, a txscript package which implements the transaction script engine, a btcec package that implements all of the secp256k1 ECDSA crypto, a blockchain package which implements all of the chain selection rules and processing, and several others.

In addition, within the btcsuite organization, there are other repos such as the btcutil repo which provides things like the base58 package for handling the Bitcoin-specific base58 and base58check encoding, the bloom package for dealing with Bitcoin-flavor bloom filters, the hdkeychain package for generating and working with BIP32 hierarchical deterministic keys, the txsort package for providing transaction sorting according to BIPLI01 for the lightning network, etc.

As you can see by following those package links I provided, all of them are fully documented and the vast majority include examples of how to use the package. There are also an abundance of internal comments within the code to explain what's going on. I should also note the test coverage of the vast majority of the packages is extremely high. For example, the wire package has 100% coverage, the btcjson package has 100%, the btcec package has ~97%, the txscript package has ~96%, and the hdkeychain package also has ~96%.

EDIT:

Oh, and since this is a development thread, I thought I should also call out it only takes a few seconds to recompile after changes and getting setup for building from master is super easy. Here is an example of my using 'go get' to download the btcd code and all its deps into a fresh GOPATH and compile everything along with timing information:

$ ls -al /c/tmpgo
ls: can't access /c/tmpgo: No such file or directory     (Showing this is from scratch)
$ export GOPATH=/c/tmpgo
$ time go get -u -v github.com/btcsuite/btcd/...
github.com/btcsuite/btcd (download)
github.com/btcsuite/fastsha256 (download)
github.com/btcsuite/btclog (download)
...
github.com/btcsuite/btcd

real  0m40.411s

$ $GOPATH/bin/btcd -V
btcd version 0.12.0-beta

In summary, that is showing downloading and compiling btcd and all of its dependencies from scratch took ~40 seconds and is a single command.

Once that initial setup is done, 'go build' is able to recompile code changes in a few seconds.

 

[awemany]

@davecgh: Excuse my total ignorance, you guys are way too quiet and 'Keeping calm and Bitcoining on' as Gavin quipped.

That's awesome, also that you guys are around!

Sounds like I might have a try at learning go

 

[Inca]

Just reviewed all the commits from thezerg and awemany on git. I see you went with the logo with an infinity sign in it

Excellent work guys. I will compile it on one of my machines this week and run a node on main net.

Is there a particular format for the BUIP requests on that thread?

 

[bitcartel]

@davecgh:

Sounds like I might have a try at learning go

Go for it! --> http://www.informit.com/store/go-programming-language-9780134190440

 

[Mengerian]

I met up with @Mengerian for beers last night (cheers!) and he made a great point about getting people to adopt Bitcoin Unlimited:

Rather than appealing purely on a "decentralize development / support bigger blocks / free choice," what if we also find ways to make BU more useful for the people actually running nodes?

Cheers to you too! that was fun!

My thoughts are that Bitcoin Unlimited could position itself as the robust consensus-tracking node implementation. A node that is future-proof and can handle whatever block-size proposals are implemented. A node that empowers the user by giving the best information on what is happening on the Bitcoin network. This could include things like noticing new transaction types, keeping track of soft-forks, noticing if miners seem to have patterns on the kind of transactions they include in blocks, etc.

 

[AdrianX]

@theZerg not sure where to post this, but just wanted to leave some feedback.

Settings > Unlimited > Mining
Maximum Generated Block Size:

A denomination is needed it's 1000000 what?

Settings > Unlimited > Network
Block Acceptance
Excessive Block Size (denomination neede) I think MB is easy to understand as it's a unit that's been discussed.

Excessive Acceptance Depth (to be denominated in Blocks.)
[doublepost=1451199572,1451198896][/doublepost]My first test run was disappointing.

I've been using XT and Armory on Ubuntu 14.04

I'm using Armory to compose transactions with 0 fees. (All coins over 12 months.)

Armory broadcasts the transactions using XT no problem. But when I use BU Armory sends back a message that there are insufficient fees. It does not broadcast the transaction when connected to BU.

It won't broadcast a transaction with a fee up to 0.0001 (I ran out of old coins in that wallet befor I could confirm that I could send at all.)

When I switch from BU to XT (just changing the engine in the background)

Armory's transactions are broadcast via XT no problem.

Why is this?

 

[theZerg]

Thx for the unit feedback. Does your armory do the same on core? The fee code hasn't been touched.

 

[AdrianX]

@theZerg I haven't installed Core. Next time I move coins I'll test it with Core.

 

[YarkoL]

My BU node which has not quite synced with the chain
yet is returning -1.00000000 when I call estimatefee <numberofblocks>
I'm not that familiar with the inner workings of the new fee policy, but
could someone who has the entire blockchain verify that the rpc call
works?

My first post in this forum, so hello.

 

[solex]

Hello YarkoL
-1 means that it is not able to provide any estimate. You will need to wait until sync is complete. BU will take the same time as Core to do this.

 

[YarkoL]

Thanks solex. The code has indeed reference to a
nBestSeenHeight variable, which I take to be the best block
height of my peers.

Armory broadcasts the transactions using XT no problem. But when I use BU Armory sends back a message that there are insufficient fees. It does not broadcast the transaction when connected to BU.

Perhaps naive, but still necessary question: are you running BU with
-sendfreetransactions flag set to true?

 

[AdrianX]

Thx for the unit feedback. Does your armory do the same on core? The fee code hasn't been touched.

@theZerg so I have some feedback

I installed Bitcoin Core 64-bit 0.11.2, Bitcoin Unlimited 64-bit: 0.11.2, Dec 22, 2015 and BitcoinXT 64-bit: 0.11.0D on an standard Ubuntu 14.04 LTS.

I broadcast a transaction with coins from 2014 and 0 transaction fees from Armory 0.93.3 while running each of the bitcoin clients in the background. My settings in Armory were set to "Expert User" to enable coin control and I deselected the check box that says "Let Armory run Bitcoin-QT/bitcoind in the background" so I could confirm each transaction was broadcast by the relevant QT client.
(at the time of the broadcast attempt there were more than 12 connected nodes on each client.)

Bitcoin Core and Bitcoin Unlimited both rejected the armory transaction with 0 fee promoting armory to display this message:

The transactions had previously been broadcast once the fee set by the core developers was added. (promoting my original post)

Bitcoin XT broadcast the transaction with 0 fees. It took a number of blocks before being included, it was rejected by Block 391911 (and final accepted by AntPool in Block 391915)

So not an BU problem in that it's expected behavior in that you are copying Core, But it's a problem in that BU is is not enabling a free market but relaying Core's centrally controlled fee market policy.

 

[theZerg]

something for us to argue about... something makes me think that this is configurable. Can you check?