Optimal fee strategies for miners

[Gavin Andresen]

The "microeconomics of a dynamic block size limit" thread didn't go where I wanted it to (analyzing microeconomics), so I'll try asking a different question:

If we start with reasonable assumptions about price/demand curves for transactions confirming "as soon as possible" to "sometime today" (maybe to "sometime this week"), and assume that miners could get together and agree to maximize profit, what would be the profit-maximizing dynamic block size limit?

Would it be blocks on average 50% full? 99% full? 1% full? If you change the assumptions about the demand curve, how much does that answer change?

Concrete toy example to show what I'm getting at (and apologies in advance to any real economists who might be reading this and cringing, I wasn't an econ major....):

Suppose there are three different groups of people with different fee/confirmation time preferences:
"A" : willing to pay up to $1 for transactions confirm in the next block with high probability
"B" : willing to pay up to $0.10 for transactions that confirm in the next 6 blocks
"C" : willing to pay up to $0.01 for transactions that confirm "sometime today"

Lets say there are ten times as many "B" people as "A", and 100 time as many "C". Everybody would be happy to pay less, and assume everybody has a smart wallet that adjusts fees based on past history, what transactions are waiting in the mempool (with what distribution of fees), etc.

How big should miners make their blocks to maximize total fees paid, given that the poisson distribution of block-finding time makes supply of block space inherently variable? Is the best strategy to keep blocks X% full, or maybe to sort the mempool by fee and then include the highest X% ?

"Include all transactions all the time" isn't the optimal strategy, because all of those "A" people who are willing to pay $1 would instead pay whatever the minimum spam fee is.

"Just include transactions that pay $1" isn't optimal either, because the miner loses all of the fees from "B" and "C".

(This is where I hope somebody tells me I'm being an idiot and if I just read chapter 11 of some economics textbook the answer would be obvious....)

 

[Bagatell]

This is where I hope somebody tells me I'm being an idiot

 

[freetrader]

> given that the poisson distribution of block-finding time makes supply of block space inherently variable

Ok, I'm not very educated on this subject, but I've been led to believe that the time to find a block practically does not depend on how many transactions the miner is trying to fit into it...

> how big should miners make their blocks to maximize total fees paid

If my assumption above is not incorrect, isn't the "best" alternative to mining empty blocks the "fit in as much of the highest-fee transactions as you possibly can" strategy (i.e. sort mempool by fee) ? This obviously runs into the big-blocks-more-likely-to-be-orphaned problem - so aren't you looking to maximise the block fees under the probability that the block is orphaned at the given size, making that probability the deciding factor?

Edit: would need to compute the highest-paying txns in terms of fee/size metric

 

[jonny1000]

Gavin

This is a great question. I do not think you are being an idiot nor is there a page in an economic textbook which should explain this.

You asked about the profit maximising strategy, in order to know this we would need also make some assumptions about costs. In particular the marginal costs, which means the addition to total cost of adding an extra transaction into your block. In a competitive environment micro economic theory suggests that miners will keep adding transactions to the block up to the point until the transaction fee equals the marginal cost.

The marginal cost would exclude things like rent, hardware, salaries or electricity. It would only include costs directly related to adding extra transactions in the block. This may be mostly orphan risk cost. The size of the block would therefore depend on fees and orphan risk. Please note this assumes a competitive environment.

I don't think the distribution of block finding time may matter, miners may try to maximise profit if they find a block. I could be wrong here. If this does matter it is certainly not standard economic theory. I think orphan risk costs may be impacted by the poisson process, but this is complex and I do not understand how the poisson process would impact orphan risk.

"Include all transactions all the time isn't the optimal strategy, because all of those "A" people who are willing to pay $1 would instead pay whatever the minimum spam fee is."

This is a very interesting point and its validity depends on the level of competition in the industry and the time horizons of the miners. Please see two examples below:

1. If a miner has a 0.1% market share and short time horizon you may be wrong here, this miner may only care about maximising revenue in the next block, they may never find another block again and from their point of view they could exit the industry tomorrow. Economists call this "perfect competition".

2. If a miner has a 20% market share and cares about the long term, your logic is likely to be correct. The miner may not include the C transactions otherwise they may not get A fees in the future. Economists call this "monopolistic" behaviour.

The industry will always contain a variety of miners with different market shares and different time horizons, these balances are likely to change in economic cycles, sometimes miners will act in line with "large blocker" assumptions and sometimes "small blocker" assumptions. This is why this issue is challenging and we must be open minded and act with a reasonable degree of caution.

Of course we would like the mining industry to be highly distributed and competitive, so in some ways we hope the "small blocker" include all transactions logic is most applicable.



Many thanks

 

[Georg Engelmann]

I don't think there's an "optimal strategy". It's all about balance: if miners include too many transactions people will pay lower fees, if they don't include enough transactions people will stop using Bitcoin.

 

[theZerg]

Gavin, I think that by "miners get together and agree to maximize profit" you mean a mining cartel or monopoly. These situations have been studied in economics.

You are correct in assuming that the price will eventually settle down to where everybody more or less pays the same thing for the same good (this is why we have first class seats in airplanes -- to create a slightly different product)

So basically you plot the demand curve (price is the vertical axis, quantity the horizontal). And for any point on the curve, your revenue is therefore the area of the square (i.e. price * quantity) below that curve. You can use calculus to maximize that volume (max of Q*R(Q)). You can see how if 2 people are willing to pay 10000 for the product and 100 people are willing to pay .1, it makes sense to produce very little volume.

Ok, next level of complexity is to subtract out your cost. So you plot a "cost" curve on the same graph where the vertical axis is price to produce, and horizontal is quantity. Typically the marginal cost of production (cost to produce 1 more doohickey) decreases as volumes increase. So now you use calculus to maximize the volume of Q*R(Q) - C(Q)

Maybe start here and click around: https://en.wikipedia.org/wiki/Profit_maximization


Where it gets interesting WRT bitcoin is to plot the demand curve to get into the NEXT transaction. So in this case we basically have multiple "products", and you could plot the demand of each separately, or pretend it is continuous and plot a 3d curve.

Anyway, it would probably make sense for miners to artificially create separate "products" based on time-to-confirm and see what demand emerges. To do this, the miner would simply delay every transaction that pays less than X by Y blocks. The advantage ofc is just like airplanes you manage to sell low volumes to price insensitive people and high volumes to everyone else, allowing you to essentially draw multiple squares under the demand curve.

You typically can't a-priori find out whether demand can be differentiated, or what the demand curve is. That's one reason companies have sales (to experiment with other points on the curve). In our case Core seems to think that most of the demand right now is very flexible. However, bumping against a supply limit will help tell us. If demand remains nearly the same and so the price rises dramatically the demand is considered "inflexible" -- which is what happened in the 1970's oil crunch. It wasn't that supply really dropped my much, it was more that almost everyone STILL had to drive to work no matter what the price of gas rose to.

 

[bitsko]

It's all a bit beyond me, but I enjoyed reading things I didn't fully understand about how the mail system offers different tiers of service through the same network. And I wondered if any of the conclusions the economists made would be applicable.

Related?:
The Economics of Bitcoin Transaction Fees
Including all transactions whatever the fee attached is the only subgame perfect Nash equilibrium of the game we described.
The Economics of Bitcoin Mining or, Bitcoin in the Presence of Adversaries 4.2:
Such an agreement can be modeled as a prisoner’s dilemma: the miners would benefit from cooperation but cannot agree not to defect

 

[Emperor Bob]

I assume an even, constant flow of new transactions, and that we're sticking with a single dynamically adjusted block size that varies over a long timescale (longer than the confirmation horizon of almost all users).

This becomes an optimization problem, and it actually depends mostly on the relative size of each group. A sample of outcomes:

At 50% full, transactions overflow the next block 40% of the time, overflow the hour about 15% of the time, and overflow the day essentially never.

You'd probably (if they want >60% chance of inclusion) get A and maybe B (if they want >85%) to outbid C users (The small size of both B and A users mean they'd never worry about outbidding each other). However since C bids as close to zero as possible, no one pays much at all.

At around 90% fullness, the odds of overflowing a full day are about 13%. So you'd expect C users to start bidding their fees up to $0.01 in order to get included. A and B would in turn also have to outbid the C users. The average would be slightly more than $0.01/tx.

To get A to have to outbid B, you'd have to get blocks filled with just B & A's transactions. That seems unlikely without first having a near infinite backlog.

Let's relax the assumptions a bit (I hope I'm not going to derail this conversation in yet another unforeseen way).

In your A, B, C example, with a co-operating mining cartel, you're best off having a mempool-to-block inclusion delay based on the fee, matching the preferences of the consumers (I believe that's what theZerg is explaining), with no actual block limits:

>= $1 gets you into the next block, >= $0.10 gets you in in 3 blocks, >= $0.01 gets you in in 72 blocks.

Obviously, this is not protocol enforceable. Do we have an alternative technique that gets some of this nice QoS pricing behaviour that can be enforced at the protocol level?

I'll suggest one: variable block sizes. Set some size. That's your base block size limit. Odd-height blocks are capped at that size. Even-height blocks are capped at twice that size, (height % 4) are twice *that*, and so on. You get one very large block per day on average. You can now tweak the numbers such that there's a daily backlog, to be cleared by just that block.

It seems a but silly, but I believe you could get much more revenue that way.

 

[jonny1000]

Gavin, I think that by "miners get together and agree to maximize profit" you mean a mining cartel or monopoly.

If a cartel was able to control supply like this, it is likely to also be able to censor transactions. We should try to ensure the industry is structured in such a way that any cartel breaks down, and we therefore have lower fees and larger/full blocks.

 

[Gavin Andresen]

RE: marginal costs: if it simplifies the analysis, assume zero marginal cost (no orphan cost for including another transaction).

RE: we can't a-priori know what the demand curve is: yes, but we should be able to make reasonable assumptions and get results for various changes in those assumptions. A reasonable assumption might be people are willing to pay at most 1% of transaction value in fees; that plus historical transaction data (e.g. "we see eleven times as many 11-cent transactions per day as 11-dollar transactions") could give a reasonable demand curve.

I'm hoping for less armchair speculation and more "I studied a similar problem for my PhD thesis in micro, here's a spreadsheet that calculates optimal strategy given blah blah blah...."

RE: "vary the size of blocks created to maximize revenue (e.g. one really big block per day)" : the random nature of block-finding does that automagically. Every once in a while miners will find several blocks more quickly than normal, which is equivalent to finding a bigger block-- low-fee transaction backlog will get cleared out. The opposite also happens, of course....

 

[Emperor Bob]

For the last point, though, the issue is that the Poisson process is most variable for short intervals, and least variable for long intervals. Meaning that there's either almost no fee pressure for people with time horizons > 2-3 hours, or there's a near-permanent backlog of priced out transactions.

Point taken about the hand-wavy nature of the discussion so far. I might be able to cook up a quick simulation to get a better grip on the numbers.

 

[Zangelbert Bingledack]

The two elephants in the room here are

1) Whether miners cartelize; I doubt they would as the incentive to break the cartel agreement is extreme

2) Efficiency of communication of bids/asks between miners and users

These two points are related, because dynamic two-way communication is more meaningful when miners are uncartelized and can't engage in price fixing, and efficient communication also makes the cartel more unstable.

Assuming perfect communication of asking prices for a given tx size from all miners creating the next block, the user's software can calculate the likelihood of next-block (etc.) inclusion for a given fee. Since the asks would change dynamically as miners weigh demand and orphan risk, so would the user's fee estimation. If miners honor their spot asking prices (i.e., never bump out your tx to make room for one with a juicier fee), it seems you would have a nice smooth curve.

However, if not all miners broadcasted asking prices continually, or if the user software wasn't equipped to receive and calculate the results for the user, or if miners were deceptive, or perhaps if something more complicated like smart-contract-based guarantees of inclusion for agreed-upon fee were used, the curves get more complicated.

Generally, though, the miners will keep including transactions, prioritized by fee density, until either

(a) cost to include the next-highest-priority tx exceeds the fee on that tx ("cost" meaning mainly orphan cost, under a high blocksize cap)

(b) block is full (under a low blocksize cap)
[doublepost=1451595815][/doublepost]

RE: marginal costs: if it simplifies the analysis, assume zero marginal cost (no orphan cost for including another transaction).

In that case, all blocks will be filled in order of maximum fee density (result: no blocksize cap is optimal), as there is no reason not to. Strategic signaling doesn't really work when the industry is not cartelized, and again cartelization can be avoided by smoother communication between miners and users. It may work under a scenario involving tx-inclusion contracts, but the range of possibilities that opens up would be nearly impossible to get a good handle on.

 

[Emperor Bob]

I think the reason we're talking about a miner cartel agreeing is because a protocol-enforced rule has much the same behaviour (it binds all miners), not because we actually expect miner cartels.

 

[Zangelbert Bingledack]

That sounds like a pretty good reason not to have such a protocol-enforced rule.

--

Re: Spreadsheets, we'll need a few more assumptions nailed down before that can be done. Though what is the motivation for doing this? Typically the specific number crunching is best left to miners in this division of labor. A problem I've had with Core's approach is they try to play economic central planner. That doesn't mean having some numbers to create a reasonable estimate of how things might play out isn't valuable, but the planner's paradigm can be hard to break out of.

 

[Emperor Bob]

Reading the initial post, and the post it refers to, here's the reason this is being talked about at all:

This was advice I had for lead developer of an altcoin who worries that no block size limit would mean a low-transaction-fee-death-spiral. Rather than trying to convince them they're wrong, I said this:

If you're concerned about fee pressure, you should implement a dynamic cap.

At every difficulty change:

Find average sigops/sighash per block over last 2016 blocks
Find average bytes per block

Maximum = some multiple of those percentages (or some minimum if blocks are empty). If you want a little fee pressure-- blocks on average half full-- choose 2.

If you want a lot of fee pressure, choose 1.2 (blocks on average 80% full).

See the charts at http://hashingit.com/analysis/34-bitcoin-traffic-bulletin to get an idea of how long the average transaction will have to wait at various percentages.

You should also change the default mining policy to "produce average blocks" (miners who care to influence the size up or down, or want to pick up extra fees can change it).

Then you're done, never have to touch the size limit again.

It would be spiffy to contrast that approach with the "flexcap" proposed by Mark Friedenbach, Blockstream and altcoin (Freicoin) founder. Seems to me a simple dynamic limit addresses the same (perhaps unfounded-- lets put that aside for a bit) concerns as flexcap but in a much simpler way. My objection to flexcap has always been that it is just a complicated way of setting a minimum transaction fee...

I think we should avoid derailing this thread too. The assumptions are what they are.

 

[Bloomie]

Silicon Valley is pitching in

 

[jonny1000]

RE: marginal costs: if it simplifies the analysis, assume zero marginal cost (no orphan cost for including another transaction).

Yes, I totally agree this is a very good assumption. I think this means there are circumstances where fees fall to near zero. (If the mining industry is highly distributed)

I'm hoping for less armchair speculation and more "I studied a similar problem for my PhD thesis in micro, here's a spreadsheet that calculates optimal strategy given blah blah blah...."

Yes, its good to listen to academics with models. At the same time its important to listen to people with industry experience and those who have witnessed various cycles. There is no perfect response to your question nor is there a perfect spreadsheet. I work as an investment anslyst in the resources mining industry, other analysts had very detailed technical models, however much of the assumptions broke down in 2015 as mining management teams had totally different incentives.

This is a very difficult issue which depends on peoples motivations and individual incentives, these change over time and cannot be modelled in a way that is always effective. Please appreciate that economics is a very different science to maths or computer science, less can be reliablely modelled and there are no right answers.

For whatever its worth, I think BIP100 is the most robust proposal from an economic perspective, as its likely to ensure the fee market functions well in a wide variety of potential scenarios.

 

[Brian Goss]

Are we assuming there are miners competing with each other? Or assuming there is just one mining entity?

 

[bitsko]

 

[theZerg]

John dont hijack the thread. Post a separate one.