Skip to content

gh-91058: Add error suggestions to 'import from' import errors #98305

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Oct 25, 2022

Conversation

pablogsal
Copy link
Member

@pablogsal pablogsal commented Oct 15, 2022

@pablogsal pablogsal added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 17, 2022
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @pablogsal for commit 77f9de1 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 17, 2022
@pablogsal pablogsal added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 17, 2022
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @pablogsal for commit cd528b6 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 17, 2022
Signed-off-by: Pablo Galindo <[email protected]>
Copy link
Member

@isidentical isidentical left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks amazing (and a very useful error) @pablogsal. Just a quick question

return NULL;
}

PyObject* mod = PyImport_Import(mod_name);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering whether we could avoid "realy-importing" a module if it isn't already (e.g. whether it is currently cached or not)? (IIRC from x import y should put the x to the sys.modules, but might be missing something too).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyImport_Import goes through the cache if for whatever reason the module is already imported, no? (And indeed, the module is imported as part of raising the exception).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyImport_Import goes through the cache if for whatever reason the module is already imported, no? (And indeed, the module is imported as part of raising the exception).

Exactly, but consider the scenario where it isn't already imported and the module itself has some side effects (like it prints something at the top level). Importing it would then might produce unexpected side effects that only appear when the traceback is printed, so I thought whether we could limit this hint when the module is already cached.

if module not in sys.modules: return None # give up if it isn't already in the cache
module = PyImport_Import(mod_name)
...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but consider the scenario where it isn't already imported and the module itself has some side effects

Hummmm. But isn't this a pre-condition for this kind of import error? If from foo import x fails to import foo then there is no lookup failure and we will never arrive here. And to be able to look x in foo then foo needs to be in sys.modules. Maybe I am missing something ... :(

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's at least theoretically possible for the module to be removed by now, because some Python code may run between the time the error is thrown and the time this code runs. For example, in pythonrun.c we call print_exception_message() before print_exception_suggestions(), and that does a bunch of allocations and DECREFs that could trigger GC.

You could call https://docs.python.org/3/c-api/import.html#c.PyImport_GetModule instead to return the module only if it's already imported.

@pablogsal pablogsal added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 18, 2022
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @pablogsal for commit dee788b 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 18, 2022
Copy link
Member

@JelleZijlstra JelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, a few suggestions.

return NULL;
}

PyObject* mod = PyImport_Import(mod_name);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's at least theoretically possible for the module to be removed by now, because some Python code may run between the time the error is thrown and the time this code runs. For example, in pythonrun.c we call print_exception_message() before print_exception_suggestions(), and that does a bunch of allocations and DECREFs that could trigger GC.

You could call https://docs.python.org/3/c-api/import.html#c.PyImport_GetModule instead to return the module only if it's already imported.

pablogsal and others added 2 commits October 20, 2022 15:04
@pablogsal pablogsal merged commit 7cfbb49 into python:main Oct 25, 2022
@pablogsal pablogsal deleted the gh-91058 branch October 25, 2022 22:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants