eth/downloader: introduce stricter block body throttling #31509
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…worst-case block size at the current gas limit
…r block cache memory to 1gb.
…ill allow for scheduling new block retrievals until the cumulative gas used of the in-flight retrievals is greater than a target threshold
…the initial fetch size
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, Geth calculates an exponentially-weighted moving average for the size of recently downloaded blocks, and uses this as a heuristic to figure out how many recently-downloaded blocks Geth is willing to keep in memory while they are pending addition to the chain. If we hit this number, we will throttle preventing subsequent requests for new blocks until the ones we have are imported into the chain.
Because this heuristic for expected average block size looks at recently downloaded blocks, we can have a situation where a prolonged lull in average block size drives the average down, causing Geth to increase the number of blocks it will simultaneously request. Then, if the newly-downloaded blocks are much larger than we expected, Geth can choke and go OOM.
In practice, this can't actually happen in a real-world setting: An attacker would have to be able to control the creation of a large sequence of hundreds of blocks.
However, we are seeing some OOMs in contrived attack scenarios, and this change intends to prevent those from occurring while hopefully not having a measurable effect on the sync speed.