Syncing and RAM

With my struggle to resync the blockchain I learned that Bitcoin is something like the ultimate RAM test.

At least this was the most plausible explanation I heard about a phenomen I experienced myself and heard from two more people: The client loads and syncs and loads, and suddenly it stopps, no message, nothing, and if you close it and start again it tells you: "Corrupted database." - and you have to start with genesis again.

Explanation might be that the RAM makes so many operations while syncing, validating 100 millions of transactions, that even the tiniest damage of the ram, such kind of damage even your linux-memtest doesn't find, is enough to materialize at some time. like cancer materializes with a good chance after enough cell divisions, like you find a block if you do enough sha 256 calculations.

When you start again from genesis after the failure, you probably experience again. Not in the same block, maybe later, maybe earlier, but probably. Best solution is to buy a new ram (like I did -- success -- but I did also bought a new cpu that validates faster than I can download) - or to not run a client. Two of people that asked in my blog / in our forum decided to not buy a new ram because their old ram works in everything but bitcoin.

Syncing the client has requirements to rams you never find in "normal life". If the blockchain grows and grows and grows, especially when we have bigger blocks, the problem will grow with it. If we have a billion of transactions a node has to process in a row, it needs a perfect ram, and maybe even not damaged, but cheap sticks will fail.

I don't know nothing about the code, but I think the best solution would be to just do "saves", like in a videogame. Maybe save the index-files (?) evey 1024 blocks, and if the nodes detects corrupted databases he loads the last working database? If you don't need to start at zero, the problem would fade away - you can just try again and again, it doesn't matter, because your blockchain gets longer with every attempt.
 

sickpig

Active Member
Aug 28, 2015
926
2,541
@Christoph Bergmann

I t would be useful in this case to run a memtest session (*) on your old RAM.

Do you happen to have Linux installed in such box? if it's the case usually you could chose to test ram via memtest at boot time.

(*) http://www.memtest.org/

p.s. thanks for the help you provided on "that" BCT thread today :p
 
:) that people in "that thread" are so stupid, I can't resist, especially since I realized that it makes no sense to talk to them ...

About the RAM: Yes, I made the linux RAM-test, and it showed no mistake. But the RAM-test went on ~7-8 hours, while the snycing ~72-80 hours ... someone said that a positive memtest is no guarantee that there is no mistake.

If there is another explanation for this - it seems to happen from time to time and more often, as the blockchain grows - I'm curious. By now the ram-hypothesis seems to be most plausible.
 

sickpig

Active Member
Aug 28, 2015
926
2,541
@Christoph Bergmann

~72-80 hs? really?

which bitcoind version/flavour are you using?

Recently a BU 0.12 running on a computer (2 xeon core, 6GB) it takes ~12hs to download and validate all the block chain.
 

jl777

Active Member
Feb 26, 2016
279
345
VM swapping is the likely cause of both the slowdown and "RAM" error
i think it might be more likely to get an uncorrected bit flip from HDD vs RAM, though the specifics vary widely and totally depends on your exact hardware setup.

when you use more RAM than physical RAM, then your HDD becomes part of "RAM", so that is why even if memtest works, it fails with bitcoin.

when you got new RAM, did you get the exact same GB or did you also get more GB? if the latter, that would reduce the amount of HDD accesses used for "RAM"