BUG: re-importing `scipy` fails

[smurfix]

Describe your issue.

scipy.optimize._highspy._core is a C module. While it contains internal submodules, most notably simplex_constants, it's not a package by the strict definition of Python 3.12's importlib.

Importing directly from scipy.optimize._highspy._core.simplex_constants thus may fail.

Reproducing Code Example

* install cq-editor
* create a model
* render it
* render it again

Unfortunately this is somewhat non-deterministic, for whatever reason.

Error message

ModuleNotFoundError: no module named "scipy.optimize._highspy._core.simplex_constants"

SciPy/NumPy/Python version and system information

Python 3.12.10 | packaged by conda-forge | (main, Apr 10 2025, 22:21:13) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
<__, numpy.__version__, sys.version_info); scipy.show_config()
1.15.2 2.2.5 sys.version_info(major=3, minor=12, micro=10, releaselevel='final', serial=0)
Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: /opt/conda/include
    lib directory: /opt/conda/lib
    name: openblas
    openblas configuration: USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
      NO_LAPACK=0 NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP=0 PRESCOTT MAX_THREADS=128
    pc file directory: /opt/conda/lib/pkgconfig
    version: 0.3.21
  lapack:
    detection method: pkgconfig
    found: true
    include directory: /opt/conda/include
    lib directory: /opt/conda/lib
    name: openblas
    openblas configuration: USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
      NO_LAPACK=0 NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP=0 PRESCOTT MAX_THREADS=128
    pc file directory: /opt/conda/lib/pkgconfig
    version: 0.3.21
  pybind11:
    detection method: pkgconfig
    include directory: /opt/conda/include
    name: pybind11
    version: 2.13.6
Compilers:
  c:
    args: -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong,
      -fno-plt, -O2, -ffunction-sections, -pipe, -isystem, /opt/conda/include, -fdebug-prefix-map=/croot/scipy_1743153247252/work=/usr/local/src/conda/scipy-1.15.2,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix, -DNDEBUG, -D_FORTIFY_SOURCE=2,
      -O2, -isystem, /opt/conda/include
    commands: /croot/scipy_1743153247252/_build_env/bin/x86_64-conda-linux-gnu-cc
    linker: ld.bfd
    linker args: -Wl,-O2, -Wl,--sort-common, -Wl,--as-needed, -Wl,-z,relro, -Wl,-z,now,
      -Wl,--disable-new-dtags, -Wl,--gc-sections, -Wl,-rpath,/opt/conda/lib, -Wl,-rpath-link,/opt/conda/lib,
      -L/opt/conda/lib, -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong,
      -fno-plt, -O2, -ffunction-sections, -pipe, -isystem, /opt/conda/include, -fdebug-prefix-map=/croot/scipy_1743153247252/work=/usr/local/src/conda/scipy-1.15.2,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix, -DNDEBUG, -D_FORTIFY_SOURCE=2,
      -O2, -isystem, /opt/conda/include
    name: gcc
    version: 11.2.0
  c++:
    args: -fvisibility-inlines-hidden, -std=c++17, -fmessage-length=0, -march=nocona,
      -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong, -fno-plt,
      -O2, -ffunction-sections, -pipe, -isystem, /opt/conda/include, -fdebug-prefix-map=/croot/scipy_1743153247252/work=/usr/local/src/conda/scipy-1.15.2,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix, -DNDEBUG, -D_FORTIFY_SOURCE=2,
      -O2, -isystem, /opt/conda/include
    commands: /croot/scipy_1743153247252/_build_env/bin/x86_64-conda-linux-gnu-c++
    linker: ld.bfd
    linker args: -Wl,-O2, -Wl,--sort-common, -Wl,--as-needed, -Wl,-z,relro, -Wl,-z,now,
      -Wl,--disable-new-dtags, -Wl,--gc-sections, -Wl,-rpath,/opt/conda/lib, -Wl,-rpath-link,/opt/conda/lib,
      -L/opt/conda/lib, -fvisibility-inlines-hidden, -std=c++17, -fmessage-length=0,
      -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong,
      -fno-plt, -O2, -ffunction-sections, -pipe, -isystem, /opt/conda/include, -fdebug-prefix-map=/croot/scipy_1743153247252/work=/usr/local/src/conda/scipy-1.15.2,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix, -DNDEBUG, -D_FORTIFY_SOURCE=2,
      -O2, -isystem, /opt/conda/include
    name: gcc
    version: 11.2.0
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 3.0.11
  fortran:
    args: -fopenmp, -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong,
      -fno-plt, -O2, -ffunction-sections, -pipe, -isystem, /opt/conda/include, -fdebug-prefix-map=/croot/scipy_1743153247252/work=/usr/local/src/conda/scipy-1.15.2,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix
    commands: /croot/scipy_1743153247252/_build_env/bin/x86_64-conda-linux-gnu-gfortran
    linker: ld.bfd
    linker args: -Wl,-O2, -Wl,--sort-common, -Wl,--as-needed, -Wl,-z,relro, -Wl,-z,now,
      -Wl,--disable-new-dtags, -Wl,--gc-sections, -Wl,-rpath,/opt/conda/lib, -Wl,-rpath-link,/opt/conda/lib,
      -L/opt/conda/lib, -fopenmp, -march=nocona, -mtune=haswell, -ftree-vectorize,
      -fPIC, -fstack-protector-strong, -fno-plt, -O2, -ffunction-sections, -pipe,
      -isystem, /opt/conda/include, -fdebug-prefix-map=/croot/scipy_1743153247252/work=/usr/local/src/conda/scipy-1.15.2,
      -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix
    name: gcc
    version: 11.2.0
  pythran:
    include directory: ../../_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib/python3.12/site-packages/pythran
    version: 0.16.1
Machine Information:
  build:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
  cross-compiled: false
  host:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
Python Information:
  path: /opt/conda/bin/python
  version: '3.12'

[lucascolley]

Thanks for sending across a PR @smurfix !

This looks like what #22257 (comment) was reporting. cc @musicinmybrain @jaheer-hussain

While it contains internal submodules, most notably simplex_constants, it's not a package by the strict definition of Python 3.12's importlib.

For completeness, could you point to the upstream docs that show this?

[smurfix]

For completeness, could you point to the upstream docs that show this?

The error message says "… is not a package", which is defined (in the sources, Lib/importlib/_bootstrap.py::_find_and_load_unlocked) as having a __path__ attribute.

This makes sense, since if something doesn't have a file system location of its own you can't reload it.