Analysis of The Smoothing Pool
Analysis of The Smoothing Pool
Smoothing Pool
Ken Smith (@shtimseht) | Nextblock Solutions
August 2022
Abstract 2
Modeling Variables 3
Proposer Opportunities 3
Modeling Method 9
Model Results 11
SP Participant Performance. 12
Analysis of SP Performance 17
Contributing Works 23
Conclusion 24
Risks 25
Further Works. 25
General References 25
We accomplished the modeling using historical Ethereum proof-of-work mining block rewards to calculate future
Ethereum staking (beacon chain) proposer payment value (PPV1) rewards. We used an estimate of the number of beacon
chain validators to determine the likelihood that a given minipool would be selected to propose a block post-merge.
Many Monte Carlo tries were performed, predicting the profitability of a set of two cohorts: a collection of solitary
minipool(s) and the same set of minipools operated as participants in the SP. Profitability was defined as the amount of
PPV earned in a fixed time interval.
We generated a series of plots that displayed the performance of both cohorts over time. We then calculated an average
win-loss ratio over all the Monte Carlo trials to determine if joining the SP provided a performance advantage. Finally,
we repeated the simulation for four configurations of minipools (1, 3, 10, and 50) operated by a single NO.
We also compiled a series of heat maps looking at a combination of ratios of NO minipools to the number of SP
participants to determine how the performance varies based on the relative sizes of the two sets. We calculated an f
fraction representing a NO proportion of the minipool in the SP and plotted that against the performance advantage.
Tl;dr A NO participating in the SP is more likely to receive larger monthly ETH rewards than running solitarius
minipools. This depends on two prerequisites. First, the NO joins the SP only when their minipools will not
compose the majority of minipools in the SP. Second, the NO will not validate for an indefinite time.
1
PPV includes both priority fees and inclusion payments that are made payable to block proposers by block builders either via the feeRecipient address
or a specific transaction in the pre-built block.
However, this analysis will show that the SP does more than simply reduce the monthly variability of earned PPV. It also
has the added benefit of increasing the likelihood of participating minipools outperforming validators that decide not to
join the SP. We referred to this later set as solitarius minipools to distinguish them from solo node operators that operate
independent Ethereum validators that are not part of the RP protocol. Solitarius minipools are RP validators whose node
operators (NO) have chosen not to opt-in to the SP and remain in their solitary configuration.
Modeling Variables
To determine both the potential returns for a solitarius minipool versus an SP participating minipool, we need to model the
potential rewards that a minipool can receive. The value of potential rewards that any Ethereum validator, including RP
minipools, will receive post-merge depends on two factors. First is the number of block proposals that a validator is
randomly selected to produce. This number of expected blocks depends on two inputs: the length of time that the
minipool will be validating and the probability of being chosen to propose.
Second is the value of priority fees and inclusion payments made to the block proposer. Collectively we will refer to the
sum of priority fees and inclusion payment rewards as proposer payment value or PPV.
Proposer Opportunities
All validators who actively participate in consensus duties on the beacon chain are randomly selected to block propose.
The chance of being selected is a straight probability whose odds are determined by the total number of validators (n) on
the Ethereum network. At the writing of this report, the current n of Ethereum validators is 410,335.3 We will use n =
425,000 in our models as a reasonable estimate of the number of validators when the Merge is expected to take place.4
2
https://medium.com/rocket-pool/rocket-pool-the-merge-redstone-601d9efd6b4
3
https://beaconcha.in/ recorded on 7/23/2022.
4
Recent conversations among the core devs have suggested the week of September 19th, 2022, pending further testing on Goerli/Prater.
It is important to note that the frequency at which a minipool is expected to block-propose will decrease as the total
validator count (n) increases. It is anticipated that the number of validators on the Ethereum network will increase as the
Merge nears and continue to grow as the post-merge staking APR increases due to the inclusion of PPV for validating
duties. A larger number of n can be chosen to model the lower chance of receiving a proposal due to the continued
growth of the beacon chain.
Figure 1
Probability Mass Function of Block Proposer Opportunities in 28 days
In a given validating period, the number of times a team of 2 minipools will block-propose is expected to be twice that of
a single minipool. The odds of any given validator being randomly selected to propose in any slot is simply 1/n. The
probability of a validator working in a team of two that either of the validators will be selected to propose in a slot is 2/n.
Their odds are twice the single minipool odds that at least one of their two teamed minipools will be selected as the block
proposer. This comparison is shown in figure 3.
The astute reader will notice that although the odds of any minipool being selected in a larger set of minipools has
improved, the expected number of block proposals, when normalized to a per minipool basis, does not change. For
example, the most likely outcome over five years of validating with one minipool is 30 proposals, and with two minipools,
the expected outcome is 60 proposals which is still 30 proposals/minipool. We can observe later in our analysis that the
improvement in the SP performance is not attributable to an increase in the relative odds of being selected. Instead, it is
the fact that the more minipools operated, the more chances (proposals) are obtained in a finite validating period to
propose a winning lottery block.
To quantify PPV, we first need to understand the “maximum extractable value” (MEV). MEV refers to the total value that
can be extracted from the blockchain by altering the sequence of transactions and injecting bot-generated transactions to
take advantage of the mempool and EVM blockchain state. Projects like Flashbots MEV Explore5 have attempted to
5
https://explore.flashbots.net/
We discussed in more detail the composition and overall treads of MEV in a previous report, A Risk Analysis of Rocket
Pool Low Ether Bonded (LEB) Minipools.7 Readers are encouraged to read the section “Proposer Payments Value (PPV)”
if they are not already familiar with the subject. Although it is impossible to foresee the actual amount of PPV in the
future, we can predict by assuming that the MEV proposer payments obtained in recent transactions are probably
representative of proposer payments in the future.
To estimate the amount of PPV in a possible future block, we first need an accurate measurement of priority fees and other
inclusion tips made to the block proposer from recent block transactions for use as a surrogate dataset for future MEV
possibilities. We downloaded via the etherscan.io API 8 a record of the last 432,000 blocks worth of data that
corresponded from block 14,771,3999 to block 14,339,400.10 These block times varied due to the proof-of-work mining
protocol and spanned a total time frame of 67 days, 16 hours, 23 minutes, and 32 seconds. However, we will use this data
as a representation covering exactly 60 days as each post-merge as each block will be precisely 12 seconds in duration.
Two ETH (2E18 wei) were subtracted from the blockReward field to model only inclusion fees and coinbase payments.
Even after that correction, it is important to note that this dataset only contains the on-chain payments made to the miner.
It does not include any off-chain payments made for block ordering. As such, these payments are not included in the
forecasts made by the analytical techniques performed as part of this risk assessment.
The modified ehterscan.io dataset will be the basis for Monte Carlo models. The downloaded etherscan dataset was
plotted on a semi-log chart in figure 4. Let’s take a moment to understand the distribution and shape of this histogram. It
is important to note that there is a tremendously wide distribution of PPV per block. Figure 3, graphed on a semi-log
scale, shows an extremely long-tail distribution skewed to the right.
As visualized, most blocks have very little PPV, while some rare blocks in the long-tail distribution have a significant
amount of PPV. The most lucrative blocks where the PPV is greater than or equal to the NO deposit will be referred to as
lottery blocks. The top 1% of blocks (ordered from smallest PPV to largest) contain 45% of the PPV. This means that we
can expect that a single minipool in a fixed duration, even if years in length, is highly unlikely to be lucky enough to
access these lottery blocks of PPV. However, an SP that is x times larger than that minipool has x times more likely
chance of obtaining a lottery PPV block proposal.
This is because most of its value in the SP is obtained from a very small set of the luckiest minipools that receive
exceptionally large PPV blocks. As the modeling period lengthens, assuming all conditions are equal, all validators have
a random chance of being one of the lucky validators on the network. The expected performance of a single minipool
moves from the median toward the average.
6
Flashbots inspects only 10 DeFi protocols and tracks profit for single-tx MEV opportunities. It should be considered a lower bound of the total estimate
on MEV.
7
https://github.com/htimsk/LEBminipools/blob/main/report/Analysis%20of%20LEB%20Minipools.pdf
8
https://api.etherscan.io/api
9
Block mined May-14-2022 03:45:52 AM +UTC
10
Block mined Mar-07-2022 11:22:21 AM +UTC
Table 1
Statistical Analysis of blockRewards data form etherscan.io
The median is only 0.0561 ETH. A single minipool is best characterized as likely to receive a block proposal
represented by the median value of the distribution. Half the single minipools would be expected to receive a PPV above
this number, and half would be expected to be below this number. The median is the best value to represent the amount of
PPV that a single solitarius minipool would be expected to receive when it is presented with the opportunity to
block-propose.
The average PPV was 0.1711 ETH. A large enough set of minipools that collectively shares their PPV rewards is best
characterized as likely to receive a block proposer share represented by the mean value of the distribution. The mean
value is at the 85% percentile of all PPV block rewards in the historical dataset of proposer payments. The performance
enhancement when a solitarius minipool that was likely to receive a median (50%) reward is now likely to receive the
mean (85%) award. This large difference in percentiles establishes an upper bound to the performance enhancement
experienced by joining the SP.
Assuming that the blockchain events that lead to large MEV blocks are randomly distributed in time, the chance that an
individual wins a lottery block is improved only by the number of entries (block proposals) they have in that given
window. (i.e., The lottery - You only improve your odds by buying two tickets for the same drawing, not two tickets, one
for each week.) Since the chance of receiving a large PPV block is low, the more tries we enter in this defined period, the
more likely we will obtain a successful outcome and experience proposing a block with a high PPV award. Since SP
participants are in a pool with a higher likelihood of getting more blocks/minipool in that period, so they have better odds
of winning a lottery block. Because of the long-tail distribution of lottery proceeds, even after sharing it with all the
participants in the SP still, on average, earn more than the median solitarius minipool.
The caveat, of course, is that there is still a chance (albeit not likely) that a solitarius minipool will get a lottery block
proposal, and since they are not in the SP, they don't have to share it with anyone. A bit of a winner-keep-all treasure hunt
opportunity for those that want to press their luck and are content with likely sub-performance versus joining the SP.
Modeling Method
We performed a Monte Carlo analysis predicting 1,000 tries where each modeled validator was assigned a probability of
proposing and then randomly assigned one of the representative 432,000 historic blocks as their estimated PPV reward.
Statistical analysis and visual plots of the aggregate Monte Carlo tries were then performed to make general statements
about the probable returns that a typical (the median 50th percentile) NO would receive. We also computed the
probabilities for a more lucky validator (the 95th percentile).
1. Determine some period we wish to model our performance over a priori. This duration consisted of a validating
duration of 5 years and a periodic smaller duration of 28 days that we used as time points to measure the
minipools performance. We used five years as a reasonable period that an individual would commit capital assets
in a long-term investment. Twenty-eight days was chosen to match the RP claims period, which is the shortest
time an SP participant can receive payments from block proposals. The reader is encouraged to use the provided
python code and explore performance curves for longer or shorter investment durations of time.
3. For each predicted block proposal of a try, randomly choose one of the historically mined blocks from the
ehterscan.io dataset containing 432,000 of the most recent blocks. Use this selected modified blockRewards
value as the surrogate to predict PPV.
4. Use 3,000 as the size of the SP. A number equal to 3000 minus the number of NO minipool will be used to form
the SP cohort of minipools that do not belong to the NO (i.e., other people’s minipools in the SP). We chose the
size of 3,000 for the combined size of the SP as this number is about half the currently active number of RP
minipools. Perform steps 1 - 3 above for this set of smoothie minipools.
5. To form the alternative configuration (e.g., the NO joined the SP), we will add the performance of the NO’s
minipools to this set of smoothie minipools. Collectively the rewards by each participating SP minipool (the
NO minipools and the smoothies) will be summed and the average determined as the SP share of rewards for each
participation minipool. This way, the two case scenarios will have been created that use the same try outcomes of
the NO minipools. Case A where the NO had run solitarius minipools and case B where the NO had joined the
SP.
6. Record information about this try. The try number, the measurement period, the amount of PPV earned in the
measurement period, and the cumulative amount of PPV earned up to this measure interval from all the past
measurements. These values are recorded both for the solitarius minipools and the SP participants. The share of
PPV earned for each SP participant is calculated. Some running statistics about the number of times that the SP
PPV earned exceeds the earn of the solitarius case are also recorded.
The use of the Monte Carlo method will allow us to make some generalizations about the performance of both the
solitarius and SP case sets. The typical performance of each case is represented by the median value of the tries.
Model Results
In the narrative of this report we will display the plots using a single solitarius minipool for illustrative purposes. The
reader can find the other plot outputs for NO minipool sets of 3, 7, and 50 minipool sets in our GitHub repository
(https://github.com/htimsk/SPanalysis). The reader is also encouraged to download and use the python code to model any
combination of NO minipools and SP participants.
The lines of profitability shown on these charts only account for PPV payments. They do not include the consensus layer
rewards that a NO earns for performing validating duties.
Each of these upward lines represents a try when one of the solitarius minipool(s) was lucky and received a lottery block
as part of their proposal opportunities. All the line traces terminate after the five-year validating period. We can see
specific metrics about the Monte Carlo dataset, including the median, mean and standard deviation displayed in a text box.
Figure 6
Modeling runs showing cumulative ETH earned by solitarius minipools.
SP Participant Performance.
Next, we modeled the second case where the NO opted into the SP. To model this, we selected a number of an additional
set of smoothie minipools that, when added to the NO minipools, fill the SP to the desired 3,000 participating minipools
model parameter. We repeated the Monte Carlo modeling routine of determining the likelihood of each minipool being
selected to propose.
Figure 7
Modeling runs showing cumulative ETH earned by solitarius minipools.
Figure 7 shows the light blue line traces representing the tries modeling this SP. The value shown is the share of SP
earnings due to the NO. For example, if the NO had operated only one minipool, they would receive 1 / 3000 of the
earnings, 0.00034%. If the number of minipool operated by the NO is 50, then the NO would receive 50 / 3,000 or
0.017% of the SP earnings.
The dark blue line highlighted is the median thread as measured by the amount of PPV earned at the conclusion of the
5-year validating period. Because of the sizable averaging effect, the median (5.29 ETH) is nearly identical to the SP's
mean (5.28 ETH), with the standard deviation calculated at 0.23 ETH.
Of importance to note is that nearly all of the modeling tries for the SP outperformed (outearned) the median expected
performance of a solitarius minipool. Although a lottery-winning solitarius mini pool can outperform the median SP
participant, there were very few times in which the solitary case proposed a lottery block such that the solitarius case
earned more than the median SP try.
To better illustrate this fact, in figure 9, we have removed the SP threads and now show the median thread in dark blue.
This demonstrates that most solitarius minipools are predicted to perform less than the median performance number of the
SP. We have also overlaid the average performance of the SP with a dotted line, which is nearly identical to the trace of
the median thread.
Another perspective is to compare the PPV profit that a NO makes by participating in the SP vs. operating solitarius
minipools. To graph this for each Monte Carlo try, we take the NO’s share of the smoothing pool at the end of the
validating period and subtract the eared PPV from the corresponding cohort where the NO minipool(s) continued to
operate in a solitarius method. If the SP outperforms the solitarius configuration, the difference will be a positive value.
If the solitarius minipool was fortunate and proposed a lottery block, the difference will be a negative ETH value. You
can see from this graph that the SP was more profitable for most tries at the end of the validating period. For this example
of 1,000 trials, the SP outperformed the solitarius mini pools 82% of the time (819 tries). In only 181 cases did the
solitarius minipools end the validating period with more ETH in PPV than the SP. The median try (ranked from most
profitable to least) is shown in the dark olive color.
We can further view the propensity for the SP to outperform solitarius minipools by creating a “keyboard” chart that uses
a heatmap to illustrate the percent of times that the SP outperformed Solitarius minipools. The keys represent a per reward
block basis (every 28 days). Although random chances lead to some blocks having a higher level of performance versus
others, over the entire validating period, there is a strong positive correlation for the SP to outperform solitarius minipools.
The likelihood of the SP outperforming was 80%. In this model, the SP outperformed the solitarius minipool 5,199 times
out of 6500 total award periods.
In our GitHub repository (https://github.com/htimsk/SPanalysis), we have included the modeling plots for the other
minipool configurations of 3, 7, and 50 minipools.
Analysis of SP Performance
Based on these observations, we further expanded our research of the SP and prepared a series of heat maps that look at
the SP performance based on the fraction of the SP that a NO contributes. We will refer to this fraction as f, and it is
simply the NO minipools / total SP participants.
The performance advantage comes from the fact that as more members participate in the SP, there is a higher likelihood,
in a given period, that one of the participating minipools will propose a lottery block. If the NO’s fraction is small, it is
most likely that another participant's minipool will have been the validator to propose the lottery block. Because of the
long-tailed distribution of lottery blocks, the amount in the lottery far exceeds the average PPV amount not associated
Figure 12
Likelihood of SP outperforming Solitarius minipool(s) measured at the end of each reward period.
Figures 12 and 13 are similar to each other. Each displayed in a heatmap formation the performance increase of the SP.
Along the x-axis Are the number of mini pools operated by the NO. Along the y-axis is the amount of minipools
participating in the SP. The value found in each box is the relative performance increase of the SP over a solitarius
configuration. For example, a box that indicates 50% means that it is just as likely that a set of validators that operate in
the SP will perform equally as well as a set of solitarius minipools.
In figure 12, we can see that a clear pattern emerges. For combinations where f is less than 0.5 (for example, 6 out of 10
participating minipool are owned by a single NO), the observed performance is less than 50%. This means that we expect
the SP to underperform vs. solitarius minipools. In these cases, it is recommended that a NO not join the SP if their
f ≥ 0.5.
In cases where the NO f is less than 0.5, for example, 4 out of 10 minipools, we can see that the expected performance is
greater than 50%. The scale on the heat map was adjusted manually so that the green color would start to appear at a
2/3rds probability (66%) of outperformance. This was suggested as a confidence level as to when it might be preferred to
join the pool. The results show that the SP is expected to outperform so long as f < 0.5. This becomes most evident when
f < 0.1, as illustrated in the first column. The smaller the fraction, the higher the likelihood that participating in the SP
will outperform.
Figure 13 is a similar heatmap, but this time the performance is measured only at the end of the validating period. Here
the cumulative sum of each rewards claim period (28 days in our model) is carried forward. Only the final accumulated
sums of all PPV earned in the validation period are compared. We can still see a clear pattern that advantages SP
participation. The lower the f value, the higher likelihood the SP will earn a solitary configuration.
The increase in SP performance plotted against the fraction value ( f ) is displayed in figures 14 and 15. Both of these
plots can be considered derivatives of figures 12 and 13. In figure 14, we can see that as f decreases, the expected
performance increases. The maximum expected performance probability is about 80%. The colors of the individual data
points represent the column's from figure 12, which are the number of minipools that the NO has participated in the SP.
The blue dots clustered above the best-fit line are from the one minipool column in figure 13. Their separation from the
line is most likely attributable to a single minipool's probability of receiving 0 proposals in only 28 days.
Figure 17 plots the observed performance from the final earnings calculated at the end of the validating period (5 years in
our model). We again see a similar trend: the lower the f value, the higher the likelihood that the SP outperforms solitarius
minipools. Because of the extended measurement period of 5 years, all minipools sets are likely to propose a non-zero
amount of times. Because of that, all the data points cluster around the line of best fit.
Figure 16 displays the gain of proposals/minipool from participating in the SP divided by the proposals/minipool from the
remaining solitarius. Notice there is no pattern, and the observed gain or reduction is random. The results are just a few
percentage points above or below zero gains. This means there is no expected advantage in receiving more proposals per
minipool by participating in the SP.
Figure 17 is a derivative of figure 12 that shows no expected increase in the average number of proposals from
participating in the SP. There is no observed gain in the number of proposers/minipool.
Contributing Works
Already, derivatives of the above SP models have been shared by Valdorff. 11 They have expanded on the modeling results
detailed in this report into a single two-plots graphic. In Figure 18, their work is modified to match the same minipools
modeled in this report (1, 3, 7, and 50). The expected ETH rewards over a 5-year validating period are plotted on the
x-axis and are normalized on a per minipool basis.
In the top plot, we can see a distribution where the curve's height represents the likelihood of a minipool receiving that
value of ETH as PPV rewards. In the case of a single solitarius minipool (i.e., the blue line), we see a relatively flat curve
peaking at about 3 ETH traversing the range of 0 - 10 ETH. There are probability curves for each of the other solitarius
sets of minipools—all their curves peak at values less than 5 ETH.
Finally, in purple is the curve for a minipool participating in the SP. Here we can see a well-defined curve centered just
above 5 ETH rapidly dropping to zero within ±1 ETH of its peak. The curves illustrate the trade-off of a most certain
expected earning of 5 ETH from participating in the SP. Although each solitarius curve peaks before the SP curve, they
11
https://github.com/Valdorff/SPanalysis
The second plot shows the cumulative probability of receiving that amount of ETH or more over the 5-year validating
period. You can see that in the case of single-digit minipools, the probability begins to drop almost immediately as the
ETH value rises.
On the other hand, the SP continues to deliver a near certain likelihood of delivering at least 5 ETH per minipool over the
five-year validating period.
Figure 18
Per-minipool rewards for various setups after 5 years.
Conclusion
This modeling outputs the probabilities of the likely outcome. The outcome experienced may be that the solitarius case
may receive more proposals than if it had participated in the SP. Or that the solitarius case would have been lucky and
received lottery blocks of PPV. There is no way to be 100 percent certain of the outcome. This modeling technique
provides a method for a NO to understand and quantify the probability of the SP succeeding and failing. The NO then can
make a more infrared choice whether or not they wish to opt-in to the SP and, with moderate certainty, obtain a favorable
return. Or, remain in a solitarius configuration and have a more likely outcome of lower earnings but have a small, but not
impossible, chance of outperforming SP.
This modeling assumes that all participants in the SP act honestly. RP is developing a series of watchtower sentries to
monitor for cheating and a penalty schema designed to deter a NO from stealing the PPV. A two-strikes policy forgives
two occurrences of a PPV redirect before a penalty is assessed to the NO.
Suppose there is a significant amount of cheating by participants. In that case, this will adversely affect the SP's
performance, and its performance advantage over the solitarius minipool will be diminished. If the cheating is severe, it
may cause the SP to underperform relative to a solitarius configuration.
Further Works
Areas for further model development include:
● An expansion of this modeling could determine the number of selected lottery blocks that can be stolen to affect
SP performance adversely.
● Predictive modeling of the SP can be expanded to include the use of CADlabs Ethereum Economic Model.
● Perform a retrospective deconstruction of the Bayesian modeling results, we determine the contributing factors for
the observed performance advantage of the SP. The possible combinations of proposer selection and PPV
distribution can be modeled to understand what factors best created the observed effects in the model.
General References
1. Understanding the Validator lifecycle. Jim McDonald Jan 23, 2020.
2. Upgrading Ethereum Edition 0.1: Altair [WIP] by Ben Edgington
3. Github: Ethereum / consensus-specs / slash_validator
4. Github: Ethereum / consensus-specs / slashings
5. Github: Ethereum / consensus-specs / Rewards and penalties
6. Investopedia: Empirical Rule
7. Rewards and Penalties on Ethereum 2.0 [Phase 0] by James BeckMarch 2, 2020
8. The Ethereum 2.0 Annotated Specification by Ben Edgington
9. Validator Rewards in Practice - pintail.xyz
12
There is a current discussion in the Flashbots discord that the block builder, a third-party separate from the NO who assembles searcher bundles and
mempool transactions to compose the most valuable block ordering, will remit payment to the block proposer via a standard translation included in the
custom-built block. For this analysis, it does not matter how the proposer inclusion payment is made. It only matters that when it is made, it is directed
to the SP for fair distribution among its participants.