[python bindings] Incrementally rebuilding and debugging MLIR bindings

MLIR
,
1

Hey all,
I’m looking to get started with MLIR. My long-term goal is to familiarize myself with the code, and then use it in a personal project to improve my understanding of MLIR, and of compiler tech in general, especially in the context of machine learning code generation.

I’ve been following MLIR from the JAX side of things for a while, and wanted to start by working on the Python bindings. I noticed that some of the nanobind type casters were not yet noexcept as required by nanobind, so I opened PR #143866 to fix.

It’s not as easy as adding noexcept everywhere though, and my proposed fix ends in a std::bad_cast. So naturally, I want to start debugging to learn why, but due to my limited knowledge on debugging Python C++ extensions and lit/LLVM testing, I’m currently stuck.

I read the MLIR Python bindings doc, and managed to piece together a build command and a testing command from the snippets. But the debugging flow currently eludes me. Does anyone have resources / tips for debugging the Python bindings? I’m glad for any hints at this point.

PS: Ideally, in my mind, I would like iterate on the Python bindings by incrementally rebuilding the mlir wheel upon changes to the source, with a PEP517 build system (scikit-build-core comes to mind, since MLIR uses CMake). Is there potential interest in exploring this route?

Best regards,
Nico

2

I believe you should be able to just run ninja MLIRPythonModules for an incremental build of just the python modules.
But in general ninja check-mlir should be already pretty reasonable for incremental build/test here.

1 Like
3

This is doable; see mlir-wheels/pywasm/pyproject.toml at main · makslevental/mlir-wheels · GitHub for example (nevermind that it’s building a pyodide wheel - it is using scikit-build-core).

As @mehdi_amini said, ninja check-mlir-python, which will transitively perform ninja MLIRPythonModules (or whatever it’s called in Jax) will do all the correct things incrementally.

Does anyone have resources / tips for debugging the Python bindings?

Best advice I can give is come to the Discord where we have an MLIR and MLIR Python channel now and just ask there - someone will usually be able to help you out.