Yes, problem is a truly unlimited client would be dismissed out of hand. Even Mike Hearn (when he thought bitcoin unlimited meant no limit) made some comments to say that's a bad idea - let alone everyone else.
Now maybe he is wrong, but whether he is wrong or right doesn't matter in my view because bitcoin unlimited does not stop miners from generating blocks of a size they agree. As we expect miners to keep blocks small because of orphaning, yet above market demand because they want to increase the value of their holdings and hardware etc, having a limit above such demand is not any different than having no limit at all, except that it addresses the valid or invalid arguments about spam or big block attacks.
When things are going fine, there is no dispute etc, the choices that non-mining nodes make are irrelevant as they would simply be forking themselves off if they choose a size smaller than that chosen by miners. The non mining nodes would only come into play if there is a dispute, in my view, and in that case bitcoin unlimited gives the non mining nodes greater power than they currently have.
The only problem may be if miners say sorry we not increasing it anymore, but I'm sure miners like money and their value to increase so that's improbable as we have seen by their constant increase of the size in the past 7 years.
However, many are saying this is just theory etc, so, we probably want to create our own testnet and run it for quite some time with different parameters etc see how this would work in practice, if anything breaks etc. To make this more inclusive, perhaps we'd want to make it possible to connect to our testnet without having to download 70gb of data etc, so that anyone can just download a software and get it running on the test net straight away. We'd prob then want to advertise this on reddit, get many people to honestly choose a size and then try and break it etc and see how it goes.