JJG: For me the problem isn't the scaling issue in this context. That will need to be solved, and raising blocksize won't help. The idea that all transactions are stored forever is a big problem - storage bloat is IMO a much bigger issue than tx throughput, because the latter is temporary and the former is permanent. So where's that pruning, for example?
No, the issue is that the Core dev team have taken a stance where they are willing to alter economic incentives. Basically their argument, as far as I can tell, is that theirs is the conservative position - don't go around changing code in a rush. That's understandable, but which is more conservative - keeping a piece of code that was put in as a temporary fix, or allowing Bitcoin to run into an economic situation it has never been in before - actually hitting the TX cap?
I'll admit I've lost track of who advocates controlling the blockspace for what reason - I'm probably letting the 1MB-forever-gentlemen's-settlement-crowd influence my view of the Core team as a whole. That sentiment is the biggest sticking point for me, personally.
The thing is, though - it doesn't matter what the rationale is for forcing blocks to fill up. The end result is central economic planning.
For all the ideas you presented in your above post, isn't there going to be continued dialogue amongst bitcoin stakeholders, and we are not stuck in some kind of set and immutable system... .even though sometimes it seems as if changes are difficult to achieve?
In essence, if a sufficient number of people (stake holders) begin to believe that "storage bloat," for example is a problem, then there will likely be increased discussion of such issues, and then likely proposed solutions for such, and I have already heard various discussions about "storage bloat," so it is not a concept that is really lost upon people. Maybe storage bloat is one of those subjects that is used as a core team talking point because it seems that keeping blocksize limits lower helps to partly address the bloat issue.
Regarding bitcoin governance and/or the perception that the status quo is leading down a kind of centralized planning path, I guess that part of what I had been trying to say is that even though there continue to be claims and assertions that small groups have taken over blockchain decision-making, to me it seems that keeping the governance as is, and making changes to bitcoin fairly difficult to achieve remain decent tools that allow for less centralized planning in the long term, more likelihood that people will just develop their ideas around existing bitcoin systems and ruling of the ideas rather than people...
It seems considerably likely that there would be more predictability for markets with the whole bitcoin system if it remains more difficult to change, and if for example, there were a 75% consensus mechanism, bitcoin could be changed fairly easily and even possibly took over by some persuasive yet stealthily nefarious forces... .. accordingly, since bitcoin does not seem to be anywhere near broken technologically, it seems good that governance is difficult to change and consensus is also difficult to change to make it easy to make changes to our currently seemingly decentralized and disruptive investment.
So for example, if people come up with really good ideas regarding how to technologically improve bitcoin, then it is really likely to reach consensus because the idea is good and people are convinced that it is a good change (which seems to be the case with seg wit).
I think that part of the problems with both XT and Classic were that each of them attempted to accomplish too much (meaning including attempts at changing governance rather than focusing narrowly on technology). If each of those proposed changes had merely focused on technical fixes without attempting to change consensus (or governance), then there would have likely been a lot less resistance from a variety of current stake holders (frequently referred to as Core supporters).... Accordingly, the multi-faceted approach of both XT and Classic ended up removing credibility from their claims about bitcoin technological problems (if such problems actually really do exist to any degree near the level of voices shouting such "technological problems").