Prevent module reinitialisation with multi-phase init

[AA-Turner]

cc @nedbat

xref:

• 80e977c
• Use multi-phase initialisation (PEP 489) #1977

Changes written by a human ;-)

The ctracer module still uses static types (e.g. CTracerType), so is not isolated and may have undefined behaviour or crashes on use in subintepreters or after Py_Finalize cycles. Here, we raise an explicit ImportError for such cases.

Also, change the Py_GIL_DISABLED check to a version check for the Py_mod_gil slot (it is only the unstable API that is guarded behind Py_GIL_DISABLED), use PyMODINIT_FUNC for the init function, and use designated initialisers (I find them easier to read).

A

[nedbat]

Thanks, human!

[nedbat]

Hmm, turns out this causes a problem in the coverage measurement of coverage itself. If I comment out the re-import check everything seems to work, but I need to understand more.

[AA-Turner]

I added the check & error as multi-phase assumes that modules are safe for subinterpreters by default. The relevant documentation is https://docs.python.org/dev/howto/isolating-extensions.html. On a cursory examination I think CTracerType & CFileDispositionType would need to be converted to heap types. I don't know how coverage's self-measurement works though, and there might be a simpler fix there.

A

[nedbat]

This isn't about subinterpreters explicitly. The self-measurement involves deleting sys.modules["coverage"] and importing it again, which is when the failure happens. So it's all within one process. Would I still need to change the types? Or maybe just avoid initting them twice? There are also static interned strings which should only be interned once.

[AA-Turner]

Sorry for the crossed wires. I was referring to this note:

All modules created using multi-phase initialization are expected to support sub-interpreters, or otherwise explicitly signal a lack of support. This is usually achieved by isolation or blocking repeated initialization, as above. A module may also be limited to the main interpreter using the Py_mod_multiple_interpreters slot.

Per that same section, sharing Python objects between module instances would likely cause crashes or undefined behavior.. The side effect of the explicit error is that deleting the module from sys.modules and then re-importing it is expected to & will fail, as the import system attempts to re-initialise it in the same process. Clearly unwanted behaviour here, though.

Would importlib.reload work instead of deleting modules from sys.modules? That function has no effect for extension modules & so wouldn't casue issues. Alternatively, could the ctracer extension module be exempted from the deletion?

A

[nedbat]

I'm assuming that the warning is actually about sharing mutable Python objects between instances. In my case, they are types and interned strings.

This commit (8381ea7) seems good, but I welcome more experienced views. (Updated to use a mutex: 2d39c8c (Update: never mind, PyMutex_etc is new in 3.13))

[nedbat]

This is now released as part of coverage 7.9.0.