Gold collapsing. Bitcoin UP.

[adamstgbit]

@adamstgbit

his scenario makes no sense. in an unlimited situation, there will never be one superpool who dominates the entire miner space. there are too many billionaires, corporations, sovereign investment funds, and gvts who will get involved to compete in mining.

i tend to agree with you, but his scenario is at least theoretically possible, and we are NOT yet at the point where corps and gvt. are hashing..
if anything, his theory at least shows why some of the smaller miners would not want 128MB right away, and all these small pools put together might make a solid % of all bitcoin hash rate, and if some significant % of hash rate is NOT on your side, then maybe the change is Not good...

CWS is nuts, if he thinks he can win with 60% hash rate.

sry CWS i dont care much much hash power you have you CANNOT split the network because you want your changes applied 6 months earlier -_-.

 

[jtoomim]

If anyone has debug.log files from their full nodes during the stress test, and has their node configured with an accurate clock (e.g. with ntp installed), I would appreciate it if you send me your log files plus some other information so I can analyze block propagation rates.

https://www.reddit.com/r/btc/comments/9cp2av/please_send_me_your_full_node_debuglog_files_from/

 

[jtoomim]

@jtoomim
if there are 5 pools with 19% hash-rate and 1 pool with 5% hash-rate, the advantage the large pools have is probably marginal?

The 5% pool will be at a 13% orphan rate disadvantage relative to the 19% pools in that scenario. If all pools have the same hardware and software with the same configuration, and if the 5% pool has an orphan rate of 10.0%, then the 19% pools will have orphan rates of 8.8%.

Note: I'm using a slightly less approximate formula than @Peter R's below. His is easier to use and close enough.

@jtoomim
is there some maths that show the bottom line of how much of an advantage a 30-40% pool actually would have?

It depends on the block propagation rate, and on what pools you're comparing them to, and whether HFM is enabled, and what transaction fees are like, and a bunch of other things. I ran through one such scenario at the bottom of this post.

Edit: Yay, @Peter R is back! Now I'm not the only one doing math!

 

[Peter R]

@jtoomim

I think the scenario you describe only makes sense if there is 1 large pool and many smaller pools

if there are 5 pools with 19% hash-rate and 1 pool with 5% hash-rate, the advantage the large pools have is probably marginal?

is there some maths that show the bottom line of how much of an advantage a 30-40% pool actually would have?

The revenue advantage is approximately the difference in hash rate (as a fraction) multiplied by the average orphan rate.

Example 1: 20% pool versus 5% pool at 1% orphan rate:

(20% - 5%) x 1% = 0.15% more revenue _per hash_.

Example 2: 40% pool versus small miner at 10% orphan rate:

(40% - ~0%) x 10% = 4% more revenue _per hash_.

 

[Norway]

nChain or Craig did not say they are opposing ABC changes only to start a war. That is NewLiberty's own claim about the intent.

This might be NewLiberty's amalgam of the general philosophy on the nature of Bitcoin Craig has described before (that Bitcoin will need to be resilient, that it grows stronger with each attack, that this is how Bitcoin works, martial arts philosophy etc), but if it hasn't been as clear as day from Craig's Twitter feed:

Craig absolutely hates PoS.

The very idea makes him vomit. The combination of ABC seeking to change the base protocol outside the scope of its original features in 0.1 (which happen to be in his opinion, pro-Omni/wormhole supporting changes), combined with the fact that Jihan is very vocal about desiring a PoS system on BCH where coins are votes, not hashpower (appeared inside a chat screenshot that Craig or someone translated), and PoW becomes less important, COMBINED with Craig's claim that before the fork, Craig tried to gather hashpower to scale the blocksize on BTC and keep one Bitcoin, and asked Jihan it he could help, but Jihan "stabbed him in the back" and created ABC instead... When you consider all those things together, it's very obvious this isn't some random lesson for miners. It is defending Bitcoin from being sabotaged once again.

I think none of the lessons in Bitcoin need to be arbitrary or random: Bitcoin will continue to be attacked from many angles both opaquely and more insidiously, so that's more than enough to chew on, as is.

Other accusations/thoughts from Craig that you are free to consider if you want:

• Craig accuses Jihan of supporting Segwit (there may actually be some backing on this)
• Craig says Jihan supports Vitalik and Poon to build Plasma (https://plasma.io/plasma.pdf), which is kinda the end game for both BTC and BCH to have all the momentum taken off the main chain and sucked into the PoS system on top
• Craig supports building everything through the Bitcoin script system, hence even L2 apps basically are all *native* to Bitcoin:

The road to Bitcoin hell is paved by letting PoS creep into or usurp Bitcoin, imho. Stay vigilant.

Here is the screenshot where Jihan describes letting miners choose is "flawed", and it is after all the "coinholders" who should have a say and it would be "easy to implement technically" . "Pos important." " PoW can't be deserted, ofcourse".

[doublepost=1535903313,1535902373][/doublepost]

I think miners are learning, so they will deal with those facts. I think the ones that can, already move out of China. China is policy-wise so uncertain: One day the local govt clamps down on mining and tells you to wrap up your operations. Within a few months it's "hey we are not in the news anymore because price has gone down, we unofficially don't care if you mine anymore, so you can stay".

We shouldn't care if pools are economically viable or not, they work this out themselves. The ones that aren't profitable eventually stop: Bad connectivity compared to others ultimately leads to being outcompeted. If Chinese telecommunication providers don't care about miners its their own business. It's just one country, there's many more telecommunications providers around the world would would love to welcome more miners and industry in if they have the infrastructure for it or are creating better infrastructure for it.

This is an awesome post, @reina. I'm glad you became a BU member. We need to shake things up.

With BUIP 101, I am attacking a soft spot nobody knew existed. It's the default value of the EB setting you get when you download the BU client.

This value has never been an issue. The BU devs have just put a "reasonable value" as default.

After all, the user can just change it with his keyboard. Most mining pools (the user here) are big operations with CS employees. So according to economic incentives, the vast size of the economy bitcoin has become and "game theory", this initial setting should be irrelevant, right?

No! It's not irrelevant! It's very important for a reason that is never discussed on bitcoin forums.The reason is psychology.

Think about it. The miners knew that it would make sense to increase the blocksize when many of them met up in the Hong Kong meeting with core. 2 MB blocks was their "demand". They all had access to the source code. They could have changed a "1" to a "2". And those who couldn't could get binaries from the others. But they didn't do anything. They just negotiated with core, because core had definition power. Core was the boss of the protocol.

The conflict started way before the Hong Kong agreement. And the miners didn't act rational. They just wanted to comply. They acted within a fake box of options.

Eventually, some of the miners saw the dead end the core project was going and gave birth to BCH. Haipo Yang paid a high short term opportunity cost to mine the BCH difficulty down. But he gained my respect and got the name of his newborn daughter in the coinbase text.

Long story short, the miners acted in a way that hurt their own business hard. Their actions held the adoption down and the value of BTC down. Bitcoin was set back many years. BTC lost dominance to random altcoins.

All this shit happened because of psychology. You can't explain it with game theory or economic incentives. The miners regarded core devs as leaders with a common goal as themselves. They trusted core to do good.

We all know how wrong this was.

It's time to change the dynamic. If we have a system where a consensus of devs prone to conflict tell the miners what level of blockspace production quota is "safe" twice a year, we will never reach the target.

The scientific experiment testing Xthin had a big flaw. It concluded that a megabyte of data would take 15-20 seconds to send through the great firewall of China.

I don't doubt the measurements. But I doubt the motives and incentives. When performing the experiment, they didn't try to send the megabyte as fast as possible. They just rented random servers and watched the performance.

A miner / pool will solve issues like these. They will find a way. There are "global ISP"s in China. A miner / pool can have the node on the other side of the wall.

We must let go of scientists in white coats giving the green light for new blockspace quotas twice a year.

If this quota is safe for everybody, it's too low. We can't let the laggards set the tempo.

 

[jtoomim]

@Norway

The scientific experiment testing Xthin had a big flaw. It concluded that a megabyte of data would take 15-20 seconds to send through the great firewall of China.

No, that is not correct. The Gigablock Testnet Initiative did not include any nodes in China at all. The conclusion was that blocks propagate through the non-China network at a rate of 0.6 seconds per megabyte of block data, or about 15.6 seconds per megabyte of network traffic. That number refers to the total time from the miner to the last node, not the time per hop.

My 2014 tests included nodes in China, and found that blocks performed about 4x to 8x worse while crossing the firewall. If they had included nodes from China, their measurements would have been far worse.

I don't doubt the measurements. But I doubt the motives and incentives. When performing the experiment, they didn't try to send the megabyte as fast as possible. They just rented random servers and watched the performance.

Huh? You think they got bad numbers because they weren't trying hard enough? Should they have given their servers a pep talk beforehand? Sarcasm aside, I'm confused about what you mean here. Can you clarify? Exactly what does "trying harder" look like?

By the way, they weren't "random servers." They were expensive servers. The median server cost was $600/month.

Anyway, @Peter R is back, so maybe he can comment about whether they were trying to send one megabyte as fast as possible.

 

[Norway]

@jtoomim
This test showed that blocks 900kB - 1MB took 17.4 seconds to transmit through the great firewall of China. (Look for BIN 3 in the article.):

https://medium.com/@peter_r/towards-massive-on-chain-scaling-block-propagation-results-with-xthin-792a752c14c2

Huh? You think they got bad numbers because they weren't trying hard enough?

Yes. That's my point.

"Trying harder" could be using a global ISP in China, like I mentioned in the post.

 

[jtoomim]

@Norway

This test showed that blocks 900kB - 1MB took 17.4 seconds to transmit through the great firewall of China. (Look for BIN 3 in the article.):

Edit: I made a mistake in the first draft on the math. Below is the fixed version.

Edit 2: I made another mistake. Used 0.7s instead of 1.8s for GFW+Xthin. Fixed.

Yes, that number is broadly consistent with what I have been saying. The 17.4 seconds is for ~0.95 MB of uncompressed blocksize, traveling one hop across the great firewall. That translates to about 58 kB/s on that one hop. As block propagation usually takes 3-6 hops from start to finish, it will take longer (maybe 2x?) to traverse the entire network than it takes just for the firewall crossing, giving an effective bandwidth probably around 30 kB/s. That's without XThin.

With XThin, they were transmitting far less data. In the next post in that series, they show that a 1 MB block only requires transmitting 42 kB of network data. This took an average of 1.8 seconds, giving an average network throughput of 23 kB/s for one hop across the GFW. Alternately, this is 0.55 MB/s of block size for the one hop across the GFW.

If it takes 1.8 seconds for a 1 MB block, then a 100 MB block with XThin should take close to 180 seconds, assuming linear scaling. That would result in an approximate orphan rate of 26% without HFM if all miners were on the other side of the firewall. In reality, only about 50% are on the other side, so it would likely only be about 13-16% total. Also, bandwidth utilization should improve on longer transfers, so the reality might be a bit better than that. Still bad, though.

It's worth mentioning that this 13% figure applies to miners outside of China when a block is found inside China and needs to be pushed out just as much as it applies to the reverse. Everybody gets a 13% orphan rate if the hashrate distribution is 50%/50%. If the hashrate is 70% China, 30% non-China, then the China miners only get a 7.7% orphan rate, whereas the non-China miners get an 18% orphan rate.

You think they got bad numbers because they weren't trying hard enough?

Yes. That's my point.

Perhaps you'd like to try yourself? We'd all love to see an implementation that works well. We just haven't been able to get one yet.

 

[Norway]

For the third time:

"Trying harder" could be using a global ISP in China, like I mentioned in the post.

 

[jtoomim]

That would be the second time, not the third time, as you ninja-edited it into your second post after I read and quoted it.

Can you name one of these global ISPs in China? Do you mean China Mobile? Are you suggesting that people run their miners off of their cell phone hotspots? That actually might work okay, but I haven't heard of anyone trying. It might have some reliability issues, though.

It's also a bit difficult to test remotely. You're correct that the BU team was not trying hard enough to fly to China and plug a cell phone hotspot into their servers.

Besides, there's still the issue that even non-China block propagation is only 1.6 MB/s with XThin, with network throughput of about 58-62 kB/s.

 

[cypherdoc]

oh boy .bwahaha:

 

[79b79aa8]

let's see if they get that LN in working order soon, so it can be ported over to a transaction/settlement layer that works instantly with negligible fees.

 

[adamstgbit]

Can you name one of these global ISPs in China? Do you mean China Mobile? Are you suggesting that people run their miners off of their cell phone hotspots? That actually might work okay, but I haven't heard of anyone trying. It might have some reliability issues, though.

or... they could just have the node outside of chain?
cant they organize such that the generation/acceptance of blocks & TX happens outside the firewall and the mindless mining machines are in chain?

 

[Norway]

oh boy .bwahaha:

Not being able to find a payment route is a privacy feature. When noone can send money to you or receive money from you, 100% privacy is achieved. You can hodl and store value forever without anybody knowing.

 

[Norway]

@jtoomim
A miner wanting to send a megabyte of data faster than 17.4 seconds through GFC could check out this page:
https://startuplivingchina.com/best-vpn-for-china/

ExpressVPN looks like a good option. And as I wrote in the initial post and @adamstgbit wrote, miners can have nodes on the other side of the wall.

You are just proving my point that developers are not the right people to tell miners what limits are safe or not. This is the main reason I have proposed BUIP 101.

 

[cypherdoc]

not to mention that 22MB blocks do not appear to be an ATMP choke point:

https://explorer.bitcoin.com/bch/block/000000000000000001abbbf1c78115a3deeab2a21bf3a60830c6d687cff9f19c

 

[jtoomim]

The ATMP bottleneck is on the rate at which transactions can enter mempool, not the rate at which they can be committed to blocks. That 23 MB block took 1 hour 20 minutes to mine. That's why it was so big -- it had access to a large mempool built over the previous 1.3 hours.

The estimate of 22 MB was from observing that the nodes in the Gigablock tests could handle at most 100 tx/sec before hitting the ATMP bottleneck. However, it seems that the median node on mainnet is much slower than that, and hits the ATMP bottleneck at something lower, like 40 tx/sec. If you have a 10 minute block filled at 40 tx/sec and 400 bytes/tx, you'd get

40tx/sec * 400bytes/tx * 600 sec = 9.6 MB

which is in the ballpark of what we saw during the stress test, and possibly an overestimate. The longer the interval between blocks, the larger the block was, and only the blocks at very long intervals were very large.

@Norway
VPNs do not solve the packet loss issue. I've lived in China, and I've used many VPNs, and they usually make performance worse, not better. I've tried ExpressVNP, VyprVPN, AirVPN, and a couple others that I've forgotten. VPNs are also generally inferior in performance to Shadowsocks set up on a server of my own ownership. Even Shadowsocks generally worsens improve performance compared to a direct simple TCP connection through the GFW. VPNs and Shadowsocks are very useful for bypassing censorship, but they usually make non-censored performance worse.

I'm going to be going offline soon, as I'm moving to California in a few hours. Nice chatting with you guys, ttfn.

 

[NewLiberty]

Not being able to find a payment route is a privacy feature. When noone can send money to you or receive money from you, 100% privacy is achieved. You can hodl and store value forever without anybody knowing.

Further, it reduces the anonymity set rather than increasing it, and creates metadata chokepoints for data aggregation.
Its almost as if they don't know what privacy is or how it is breached.
[doublepost=1536111790][/doublepost]

nChain or Craig did not say they are opposing ABC changes only to start a war. That is NewLiberty's own claim about the intent.

Nope, that was not my claim either. Never did I claim it was their only purpose, that would be Freetrader's interpretation rather than mine. To be even more clear, I am not involved with nChain in any way other than reading what they put on social media and have no insider information to the company thoughts that is not public. These are outsider observations.

Invoking Nakamoto Consensus from Whitepaper Sec 12 is an additional benefit that it happens sooner rather than later. It would come when the cap is removed anyway, but I would suggest that nChain would likely aver that the ABC version changed the timing to sooner rather than later.

 

[adamstgbit]

The ATMP bottleneck is on the rate at which transactions can enter mempool, not the rate at which they can be committed to blocks. That 23 MB block took 1 hour 20 minutes to mine. That's why it was so big -- it had access to a large mempool built over the previous 1.3 hours.

The estimate of 22 MB was from observing that the nodes in the Gigablock tests could handle at most 100 tx/sec before hitting the ATMP bottleneck. However, it seems that the median node on mainnet is much slower than that, and hits the ATMP bottleneck at something lower, like 40 tx/sec. If you have a 10 minute block filled at 40 tx/sec and 400 bytes/tx, you'd get

40tx/sec * 400bytes/tx * 600 sec = 9.6 MB

which is in the ballpark of what we saw during the stress test, and possibly an overestimate. The longer the interval between blocks, the larger the block was, and only the blocks at very long intervals were very large.

@Norway
VPNs do not solve the packet loss issue. I've lived in China, and I've used many VPNs, and they usually make performance worse, not better. I've tried ExpressVNP, VyprVPN, AirVPN, and a couple others that I've forgotten. VPNs are also generally inferior in performance to Shadowsocks set up on a server of my own ownership. Even Shadowsocks generally worsens improve performance compared to a direct simple TCP connection through the GFW. VPNs and Shadowsocks are very useful for bypassing censorship, but they usually make non-censored performance worse.

I'm going to be going offline soon, as I'm moving to California in a few hours. Nice chatting with you guys, ttfn.

ATMP is a single threaded process. there are ways of spreading the work load around, i'd bet that the thing is, is that TX are validated and then stuffed into the mempool one by one, but TX validation could be spread out over X threads ( based on the number of cpu cores ) and only LOCK when pushing the already validated TX into the mempool.

so we could probably get a good 4-8x performance increase just by re working the code.

and then after that we fall back on moors law which of a doubling in performance every 2 years.

10MB today
80MB after code improvments
>1GB block in <6 years because moors law

thats assuming that the TX validation code itself cant be optimized. didnt nChain say they where looking into that.

But i'll bet that the REAL story behind the ATMP bottleneck is FAR more interesting then i make it out to be and that there are far more cleaver things to do to improve performance.


however we are clearly not there yet, so i'd have to agree increasing block size RIGHT NOW is probably not a good idea

 

[NewLiberty]

Node implementations will be judged on their capacity. So providing good information to your mining and other customers is vital.

What miners want to know is how well it performs on their equipment, so including some type of benchmarking toolset to assist them in choosing proper hardware is a big plus.