5/30/24, 8:05 PM Web3.

py Patterns: Customization

Web3.py

Web3.py Patterns:
Customization
Marc Garreau
May 25, 2022 • 3 min read

If you're looking to make Web3.py do something outside of its native

functionality, you've got at least a few options: middleware, custom methods,
external modules, and custom providers. This post will walk through what each of
those are, when you might reach for them, and how to get started.

1. Middleware
What

Middleware allows you to add some behavior to existing methods prior to making
a request or after a result is received.

When

Reach for middleware when you want something to happen every time a certain
RPC call or set of calls are performed, e.g., logging, data visualization, data
conversion, etc.

How

A set of default middleware comes standard in Web3.py, along with more

optional middleware to pull in. However, If you need to write some custom
middleware, you have a couple of syntax options: functions or classes. The
function syntax is more typical for straightforward use cases.

def example_middleware(make_request, w3):

# do one-time setup operations here

def middleware(method, params):

# do pre-processing here

# perform the RPC request, getting the response

response = make_request(method, params)

# do post-processing here

# finally return the response

return response
return middleware

A middleware template utilizing the function syntax

Middleware is executed in a particular order, so the API allows you to add your
new middleware to the end of the list, inject , replace or remove a layer, or
clear the whole middleware stack.

2. Custom Methods
What

Arbitrary RPC methods may be added to existing modules.

When

Registering custom methods can be handy if you're working with a client with
nonstandard RPC commands or are testing some custom functionality within a
forked client.

Custom methods can also be used to overwrite existing methods, if you want to
apply your own request or result formatter.

How

The attach_methods function is available on every module and accepts a

dictionary with method names and corresponding Method :

https://snakecharmers.ethereum.org/web3-py-patterns-customizations/ 1/3
5/30/24, 8:05 PM Web3.py Patterns: Customization

from web3.method import Method

w3.eth.attach_methods({"create_access_list": Method("eth_createAccessList")})

w3.eth.create_access_list(...)

You may optionally include custom handlers for input munging, request, and
result formatters.

from web3.method import Method

w3.eth.attach_methods({
"example": Method(
"eth_example",
mungers=[...],
request_formatters=[...],
result_formatters=[...],
is_property=False,
),
})

w3.eth.example()

Adding a custom eth_example method to the Eth module

If you prefer, you can add a property instead of a method by setting

is_property to True .

3. External Modules
What

External modules offer still more flexibility by allowing you to import groups of
APIs under one banner. Think: plugins.

When

External modules may be useful for introducing an entire L2 API or several

nonstandard RPC methods supported by one client, e.g., Erigon-specific methods
like erigon_getHeaderByHash , erigon_getHeaderByNumber , and so on.

How

Modules need only be classes and can reference the parent Web3 instance.
Configure your external modules at the time of Web3 instantiation using the
external_modules keyword argument, or at any point via the attach_modules
method:

# add modules at instantiation:

w3 = Web3(
HTTPProvider(...),
external_modules={"example": ExampleModule}
)

# add modules after instantiation:

w3.attach_modules({"example": ExampleModule})

# invoking external modules:

w3.example.example_method()

More context, including a nested module example, available here.

4. Custom Providers
What

At its core, a provider defines how requests are performed.

When

Building a custom provider is only relevant for the rare occasions that you're
plugging into a custom testing framework, or something in that vein. If you're
just looking to connect to another EVM blockchain, sidechain, or rollup, typically
you can just configure one of the existing options: HTTPProvider, IPCProvider,
or WebsocketProvider.

How

Providers only require two methods, make_request and isConnected , and a

definition of middlewares .

from web3.providers.base import BaseProvider

class CustomProvider(BaseProvider):
middlewares = ()

https://snakecharmers.ethereum.org/web3-py-patterns-customizations/ 2/3
5/30/24, 8:05 PM Web3.py Patterns: Customization

def make_request(self, method, params):

return {"result": {"welp": "lol"}}

def isConnected(self):
print(True)

w3 = Web3(CustomProvider())

w3.eth.get_block("latest")
# AttributeDict({"welp": "lol"})

Wrapping up
The options above are ordered roughly from least to most flexibility they offer. In
practice, I expect middleware and external modules to get the most mileage,
particularly once trusted external modules become commonplace.

Deliberately not included in this post is monkey patching. If you've gone down
that path... is everything okay? Seriously though, open an issue if you have
another vector you'd like to customize that requires monkey patching today.

Want new posts in your inbox?

Enter your email Subscribe

web3.py Patterns: Bloom Filters

Have you ever queried an Ethereum block and wondered what that "logsBloom" was? Have you gone
looking for the most efficient way to find an event within a block? If you answered "yes" to either of
the above, then you're in for a good

Apr 24, 2024 4 min read

Snake Charmers © 2024

Powered by Ghost

https://snakecharmers.ethereum.org/web3-py-patterns-customizations/ 3/3