BUG: Compilation of scipy 1.10.1 and 1.11.1 fails with Python 3.11

[lmmx]

Describe your issue.

I previously built a Docker image with Python 3.10, numpy 1.24.3, scipy 1.10.1, pandas 1.5.3.

Today I tried to build one identically but with different versions: Python 3.11 numpy 1.25.2, scipy 1.11.1, pandas 1.5.3

I can't say for sure yet which version change is at fault but I suspect it's not numpy, leaving either:

• Python 3.10 to 3.11 (which could be disproved by building a scipy 1.1.11 on Python 3.10)
• scipy 1.10 to 1.11 (which could be disproved by building a 1.10.1 on Python 3.11).

Edit I tried building version 1.10 again with Python 3.11 and got a different error but same Cython failure: I'll add a separate comment in thread

---

Similar to:

• BUG: <Compilation of scipy 1.11 falls with python3.12> #19022

Perhaps going back further to:

• BUG: Build failure with Cython 3.0.0b3 if scipy is already installed #18792

Reproducing Code Example

FROM mlupin/docker-lambda:python3.11-build AS build

USER root

WORKDIR /var/task

# https://towardsdatascience.com/how-to-shrink-numpy-scipy-pandas-and-matplotlib-for-your-data-product-4ec8d7e86ee4
ENV CFLAGS "-g0 -Wl,--strip-all -DNDEBUG -Os -I/usr/include:/usr/local/include -L/usr/lib64:/usr/local/lib64:/usr/lib:/usr/local/lib"

RUN yum install -y wget curl git nasm openblas-devel.x86_64 lapack-devel.x86_64 python-dev file-devel make Cython libgfortran10.x86_64 openssl-devel

# Download and install CMake
WORKDIR /tmp

ENV CMAKE_VERSION=3.26.4

# Download and install CMake
RUN wget https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}.tar.gz && \
    tar -xvzf cmake-${CMAKE_VERSION}.tar.gz && \
    cd cmake-${CMAKE_VERSION} && ./bootstrap && make -j4 && make install

# Clean up temporary files
RUN rm -rf /tmp/cmake-${CMAKE_VERSION} && \
    rm /tmp/cmake-${CMAKE_VERSION}.tar.gz

WORKDIR /var/task

RUN /usr/bin/cmake --version

# Upgrade GCC to version 8 for SciPy Meson build system
RUN wget https://ftp.gnu.org/gnu/gcc/gcc-8.4.0/gcc-8.4.0.tar.gz && \
    tar xf gcc-8.4.0.tar.gz && \
    rm gcc-8.4.0.tar.gz && \
    cd gcc-8.4.0 && \
    ./contrib/download_prerequisites && \
    mkdir build && \
    cd build && \
    ../configure --disable-multilib && \
    make -j$(nproc) && \
    make install && \
    cd / && \
    rm -rf gcc-8.4.0

# Set environment variables
ENV CC=/usr/local/bin/gcc
ENV CXX=/usr/local/bin/g++
ENV FC=/usr/local/bin/gfortran

# Verify GCC version
RUN gcc --version && \
    /usr/local/bin/gfortran --version

# ------------------- END OF SYSTEM BUILD DEPENDENCY SETUP ------------------

RUN mkdir -p /var/task/np_scipy_pd_layer/python && \
    mkdir -p /var/task/np_scipy_pd_layer/lib

# Install build dependencies for the wheels
RUN python3.11 -m pip install --upgrade pip && \
    python3.11 -m pip --version && \
    python3.11 -m pip install Cython pybind11 pythran

# ------------------- END OF PACKAGE BUILD DEPENDENCY SETUP ------------------

# Specify the version to use for numpy and scipy
ENV NUMPY_VERSION=1.25.2
ENV SCIPY_VERSION=1.11.1
ENV PANDAS_VERSION=1.5.3

# Download numpy and scipy source distributions
RUN python3.11 -m pip download --no-binary=:all: numpy==$NUMPY_VERSION

# Extract the numpy package and build the wheel
RUN ls && tar xzf numpy-$NUMPY_VERSION.tar.gz
RUN ls && cd numpy-$NUMPY_VERSION && python3.11 setup.py bdist_wheel build_ext -j 4

ENV BUILT_NUMPY_WHEEL=numpy-$NUMPY_VERSION/dist/numpy-$NUMPY_VERSION-*.whl

RUN ls $BUILT_NUMPY_WHEEL

# Don't install NumPy from the built wheel but use same version (it's a SciPy dependency)
RUN python3.11 -m pip install numpy==$NUMPY_VERSION
RUN python -c "import numpy"

# ------------------------ END OF NUMPY BUILD --------------------------------

# Extract the SciPy package and build the wheel
RUN git clone -b "v$SCIPY_VERSION" --depth 1 --recursive \
    https://github.com/scipy/scipy.git scipy-$SCIPY_VERSION && \
    cd scipy-$SCIPY_VERSION && \
    git submodule update --init

RUN cd scipy-$SCIPY_VERSION && python3.11 setup.py bdist_wheel build_ext -j 4

ENV BUILT_SCIPY_WHEEL=scipy-$SCIPY_VERSION/dist/SciPy-*.whl
RUN ls $BUILT_SCIPY_WHEEL

# ------------------------ END OF SCIPY BUILD --------------------------------

# Download pandas source from git tree (pip download failed due to bad metadata)
RUN git clone -b "v$PANDAS_VERSION" --depth 1 \
    https://github.com/pandas-dev/pandas.git pandas-$PANDAS_VERSION

# Extract the pandas package and build the wheel
RUN cd pandas-$PANDAS_VERSION && python setup.py bdist_wheel build_ext -j 4

ENV BUILT_PANDAS_WHEEL=pandas-$PANDAS_VERSION/dist/pandas-$PANDAS_VERSION-*.whl

RUN ls $BUILT_PANDAS_WHEEL

# ------------------------ END OF PANDAS BUILD --------------------------------

# Install the wheels with pip
# (Note: previously this used --compile but now we already did the wheel compilation)
RUN pip install --no-compile --no-cache-dir \
    -t /var/task/np_scipy_pd_layer/python \
    $BUILT_NUMPY_WHEEL \
    $BUILT_SCIPY_WHEEL \
    $BUILT_PANDAS_WHEEL && \
    ls /var/task/np_scipy_pd_layer/python

# -------------------- END OF PACKAGE INSTALLATION -----------------------------

# Clean up the sdists and wheels, uninstall non-built numpy after building SciPy wheel with it
RUN rm numpy-$NUMPY_VERSION.tar.gz && \
    rm -r numpy-$NUMPY_VERSION scipy-$SCIPY_VERSION pandas-$PANDAS_VERSION && \
    python3.11 -m pip uninstall numpy -y

RUN cp /usr/lib64/libblas.so.3.4.2 /var/task/np_scipy_pd_layer/lib/libblas.so.3 \
    && cp /usr/lib64/libgfortran.so.4.0.0 /var/task/np_scipy_pd_layer/lib/libgfortran.so.4 \
    && cp /usr/lib64/libgfortran.so.5.0.0 /var/task/np_scipy_pd_layer/lib/libgfortran.so.5 \
    && cp /usr/lib64/liblapack.so.3.4.2 /var/task/np_scipy_pd_layer/lib/liblapack.so.3 \
    && cp /usr/lib64/libquadmath.so.0.0.0 /var/task/np_scipy_pd_layer/lib/libquadmath.so.0 \
    && cd /var/task/np_scipy_pd_layer \
    && zip -r9 np_scipy_pd_layer.zip python  \
    && zip -r9 np_scipy_pd_layer.zip lib
    
FROM mlupin/docker-lambda:python3.11-build
 
COPY --from=build /var/task/np_scipy_pd_layer /opt
COPY --from=build /var/task /var/task

RUN PYTHONPATH=/opt/python python3 -c 'import numpy as np'
RUN PYTHONPATH=/opt/python python3 -c 'from scipy import spatial'
RUN PYTHONPATH=/opt/python python3 -c 'import pandas as pd'

Error message

louis 🚶 ~/dev/py-311-lm/layers/np_scipy_pd_layer/2 $ ./build_layer.sh                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
[+] Building 123.7s (23/35)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
 => [internal] load .dockerignore                                                                                                                        0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => => transferring context: 2B                                                                                                                          0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => [internal] load build definition from Dockerfile                                                                                                     0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => => transferring dockerfile: 5.42kB                                                                                                                   0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => [internal] load metadata for docker.io/mlupin/docker-lambda:python3.11-build                                                                         0.8s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [stage-1 1/6] FROM docker.io/mlupin/docker-lambda:python3.11-build@sha256:996db5091d804a6b2715c6acaa61e2b2ae1040120c907d1471b921e124907f64    0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [build  2/27] WORKDIR /var/task                                                                                                               0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [build  3/27] RUN yum install -y wget curl git nasm openblas-devel.x86_64 lapack-devel.x86_64 python-dev file-devel make Cython libgfortran1  0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [build  4/27] WORKDIR /tmp                                                                                                                    0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [build  5/27] RUN wget https://github.com/Kitware/CMake/releases/download/v3.26.4/cmake-3.26.4.tar.gz &&     tar -xvzf cmake-3.26.4.tar.gz &  0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [build  6/27] RUN rm -rf /tmp/cmake-3.26.4 &&     rm /tmp/cmake-3.26.4.tar.gz                                                                 0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [build  7/27] WORKDIR /var/task                                                                                                               0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [build  8/27] RUN /usr/bin/cmake --version                                                                                                    0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [build  9/27] RUN wget https://ftp.gnu.org/gnu/gcc/gcc-8.4.0/gcc-8.4.0.tar.gz &&     tar xf gcc-8.4.0.tar.gz &&     rm gcc-8.4.0.tar.gz &&    0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [build 10/27] RUN gcc --version &&     /usr/local/bin/gfortran --version                                                                      0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [build 11/27] RUN mkdir -p /var/task/np_scipy_pd_layer/python &&     mkdir -p /var/task/np_scipy_pd_layer/lib                                 0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [build 12/27] RUN python3.11 -m pip install --upgrade pip &&     python3.11 -m pip --version &&     python3.11 -m pip install Cython pybind1  0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [build 13/27] RUN python3.11 -m pip download --no-binary=:all: numpy==1.25.2                                                                  0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => CACHED [build 14/27] RUN ls && tar xzf numpy-1.25.2.tar.gz                                                                                           0.0s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => [build 15/27] RUN ls && cd numpy-1.25.2 && python3.11 setup.py bdist_wheel build_ext -j 4                                                           60.1s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => [build 16/27] RUN ls numpy-1.25.2/dist/numpy-1.25.2-*.whl                                                                                            0.4s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => [build 17/27] RUN python3.11 -m pip install numpy==1.25.2                                                                                            0.6s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => [build 18/27] RUN python -c "import numpy"                                                                                                           0.5s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => [build 19/27] RUN git clone -b "v1.11.1" --depth 1 --recursive     https://github.com/scipy/scipy.git scipy-1.11.1 &&     cd scipy-1.11.1 &&     g  40.5s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
 => ERROR [build 20/27] RUN cd scipy-1.11.1 && python3.11 setup.py bdist_wheel build_ext -j 4                                                           20.8s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
------                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
 > [build 20/27] RUN cd scipy-1.11.1 && python3.11 setup.py bdist_wheel build_ext -j 4:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
                                                                        
#0 20.18                                                                                                                                                      
#0 20.18 Error compiling Cython file:                                                                                                                         
#0 20.18 ------------------------------------------------------------                                                                                         
#0 20.18 ...                                                                                                                                                  
#0 20.18                                                                                                                                                      
#0 20.18 # The following cimport statement provides legacy ABI                                                                                                
#0 20.18 # support. Changing it causes an ABI forward-compatibility break                                                                                     
#0 20.18 # (gh-11793), so we currently leave it as is (no further cimport                                                                                     
#0 20.18 # statements should be used in this file).                                                                                                           
#0 20.18 from .cython_optimize._zeros cimport (                                                                                                               
#0 20.18 ^                                                                                                                                                    
#0 20.18 ------------------------------------------------------------                                                                                         
#0 20.18                                                                                                                                                      
#0 20.18 /var/task/scipy-1.11.1/scipy/optimize/cython_optimize.pxd:10:0: 'scipy/optimize/cython_optimize/cython_optimize/_zeros.pxd' not found
#0 20.31 warning: _hyp2f1.pxd:75:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                            
#0 20.31 warning: _hyp2f1.pxd:77:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                            
#0 20.31 warning: _hyp2f1.pxd:78:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                            
#0 20.45 warning: _hypergeometric.pxd:11:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                    
#0 20.45 warning: _hypergeometric.pxd:12:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                    
#0 20.48 Processing scipy/optimize/_trlib/_trlib.pyx                                                                                                          
#0 20.48 Traceback (most recent call last):                                                                                                                   
#0 20.48   File "/var/task/scipy-1.11.1/tools/cythonize.py", line 353, in <module>
#0 20.48     main()                                                                                                                                           
#0 20.48   File "/var/task/scipy-1.11.1/tools/cythonize.py", line 349, in main                                                                                
#0 20.48     find_process_files(root_dir)                                                                                                                     
#0 20.48   File "/var/task/scipy-1.11.1/tools/cythonize.py", line 337, in find_process_files
#0 20.48     for result in pool.imap_unordered(lambda args: process(*args), jobs):
#0 20.48   File "/var/lang/lib/python3.11/multiprocessing/pool.py", line 873, in next
#0 20.49     raise value                                                                                                                                      
#0 20.49   File "/var/lang/lib/python3.11/multiprocessing/pool.py", line 125, in worker
#0 20.49     result = (True, func(*args, **kwds))                                                                                                             
#0 20.49                     ^^^^^^^^^^^^^^^^^^^                                                                                                              
#0 20.49   File "/var/task/scipy-1.11.1/tools/cythonize.py", line 337, in <lambda>
#0 20.49     for result in pool.imap_unordered(lambda args: process(*args), jobs):
#0 20.49                                                    ^^^^^^^^^^^^^^                                                                                    
#0 20.49   File "/var/task/scipy-1.11.1/tools/cythonize.py", line 266, in process
#0 20.49     processor_function(fromfile, tofile, cwd=path)                                                                                                   
#0 20.49   File "/var/task/scipy-1.11.1/tools/cythonize.py", line 155, in process_tempita_pyx
#0 20.49     process_pyx(pyxfile, tofile, cwd)                                                                                                                
#0 20.49   File "/var/task/scipy-1.11.1/tools/cythonize.py", line 118, in process_pyx
#0 20.49     raise Exception('Cython system call failed:\n'                                                                                                   
#0 20.49 Exception: Cython system call failed:                                                                                                                
#0 20.49  cython --fast-fail -3 -o _zeros.c _zeros.pyx                                                                                                        
#0 20.51 warning: _digamma.pxd:22:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                           
#0 20.51 warning: _digamma.pxd:25:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                           
#0 20.51 warning: _digamma.pxd:27:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                           
#0 20.51 warning: _digamma.pxd:30:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                           
#0 20.51 warning: _digamma.pxd:32:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                           
#0 20.51 warning: _digamma.pxd:34:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                           
#0 20.51 warning: _digamma.pxd:36:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                           
#0 20.51 Cythonizing sources                                                                                                                                  
#0 20.51 Traceback (most recent call last):                                                                                                                   
#0 20.51   File "/var/task/scipy-1.11.1/setup.py", line 532, in <module>                                                                                      
#0 20.51     setup_package()                                                                                                                                  
#0 20.51   File "/var/task/scipy-1.11.1/setup.py", line 516, in setup_package                                                                                 
#0 20.51     generate_cython()                                                                                                                                
#0 20.51   File "/var/task/scipy-1.11.1/setup.py", line 265, in generate_cython
#0 20.51     raise RuntimeError("Running cythonize failed!")                                                                                                  
#0 20.51 RuntimeError: Running cythonize failed!                                                                                                              
#0 20.56 warning: _sici.pxd:22:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                              
#0 20.56 warning: _sici.pxd:23:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                              
#0 20.56 warning: _sici.pxd:24:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                              
#0 20.61 warning: _spence.pxd:20:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                            
#0 20.61 warning: _spence.pxd:21:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310                                                                            
------                                                                                                                                                        
Dockerfile:93                                                                                                                                                 
--------------------                                                                                                                                          
  91 |         git submodule update --init                                                                                                                    
  92 |                                                                                                                                                        
  93 | >>> RUN cd scipy-$SCIPY_VERSION && python3.11 setup.py bdist_wheel build_ext -j 4
  94 |                                                                                                                                                        
  95 |     ENV BUILT_SCIPY_WHEEL=scipy-$SCIPY_VERSION/dist/SciPy-*.whl                                                                                        
--------------------                                                                                                                                          
ERROR: failed to solve: process "/bin/sh -c cd scipy-$SCIPY_VERSION && python3.11 setup.py bdist_wheel build_ext -j 4" did not complete successfully: exit code: 1                                                                                                                                                           
d1ad2b90fddd3515eeee4bbb2ed478273263465448b8121ea83a401a62cf252d                                                                                              
Successfully copied 42.5MB to /home/louis/dev/py-311-lm/layers/np_scipy_pd_layer/2/.
layerbuild

SciPy/NumPy/Python version and system information

ENV NUMPY_VERSION=1.25.2
ENV SCIPY_VERSION=1.11.1
ENV PANDAS_VERSION=1.5.3

Python 3.11

Docker image of Python 3.11 built for deployment to AWS Lambda: https://hub.docker.com/layers/mlupin/docker-lambda/python3.11-build/images/sha256-7ec5685e7361cfbc09d11427cd949bf98396cdac19f56dff98ebd7daf0d4e773)

[lmmx]

With the same source but scipy version changed to 1.10.1, I get an error, so this is not just a scipy 1.11.1 bug: I suspect it's a Python 3.11 bug.

Click to show Dockerfile source for failing Python 3.11 build of scipy 1.10.1

FROM mlupin/docker-lambda:python3.11-build AS build

USER root

WORKDIR /var/task

# https://towardsdatascience.com/how-to-shrink-numpy-scipy-pandas-and-matplotlib-for-your-data-product-4ec8d7e86ee4
ENV CFLAGS "-g0 -Wl,--strip-all -DNDEBUG -Os -I/usr/include:/usr/local/include -L/usr/lib64:/usr/local/lib64:/usr/lib:/usr/local/lib"

RUN yum install -y wget curl git nasm openblas-devel.x86_64 lapack-devel.x86_64 python-dev file-devel make Cython libgfortran10.x86_64 openssl-devel

# Download and install CMake
WORKDIR /tmp

ENV CMAKE_VERSION=3.26.4

# Download and install CMake
RUN wget https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}.tar.gz && \
    tar -xvzf cmake-${CMAKE_VERSION}.tar.gz && \
    cd cmake-${CMAKE_VERSION} && ./bootstrap && make -j4 && make install

# Clean up temporary files
RUN rm -rf /tmp/cmake-${CMAKE_VERSION} && \
    rm /tmp/cmake-${CMAKE_VERSION}.tar.gz

WORKDIR /var/task

RUN /usr/bin/cmake --version

# Upgrade GCC to version 8 for SciPy Meson build system
RUN wget https://ftp.gnu.org/gnu/gcc/gcc-8.4.0/gcc-8.4.0.tar.gz && \
    tar xf gcc-8.4.0.tar.gz && \
    rm gcc-8.4.0.tar.gz && \
    cd gcc-8.4.0 && \
    ./contrib/download_prerequisites && \
    mkdir build && \
    cd build && \
    ../configure --disable-multilib && \
    make -j$(nproc) && \
    make install && \
    cd / && \
    rm -rf gcc-8.4.0

# Set environment variables
ENV CC=/usr/local/bin/gcc
ENV CXX=/usr/local/bin/g++
ENV FC=/usr/local/bin/gfortran

# Verify GCC version
RUN gcc --version && \
    /usr/local/bin/gfortran --version

# ------------------- END OF SYSTEM BUILD DEPENDENCY SETUP ------------------

RUN mkdir -p /var/task/np_scipy_pd_layer/python && \
    mkdir -p /var/task/np_scipy_pd_layer/lib

# Install build dependencies for the wheels
RUN python3.11 -m pip install --upgrade pip && \
    python3.11 -m pip --version && \
    python3.11 -m pip install Cython pybind11 pythran

# ------------------- END OF PACKAGE BUILD DEPENDENCY SETUP ------------------

# Specify the version to use for numpy and scipy
ENV NUMPY_VERSION=1.25.2
ENV SCIPY_VERSION=1.10.1
ENV PANDAS_VERSION=1.5.3

# Download numpy and scipy source distributions
RUN python3.11 -m pip download --no-binary=:all: numpy==$NUMPY_VERSION

# Extract the numpy package and build the wheel
RUN ls && tar xzf numpy-$NUMPY_VERSION.tar.gz
RUN ls && cd numpy-$NUMPY_VERSION && python3.11 setup.py bdist_wheel build_ext -j 4

ENV BUILT_NUMPY_WHEEL=numpy-$NUMPY_VERSION/dist/numpy-$NUMPY_VERSION-*.whl

RUN ls $BUILT_NUMPY_WHEEL

# Don't install NumPy from the built wheel but use same version (it's a SciPy dependency)
RUN python3.11 -m pip install numpy==$NUMPY_VERSION
RUN python3.11 -c "import numpy"

# ------------------------ END OF NUMPY BUILD --------------------------------

# Extract the SciPy package and build the wheel
RUN git clone -b "v$SCIPY_VERSION" --depth 1 --recursive \
    https://github.com/scipy/scipy.git scipy-$SCIPY_VERSION && \
    cd scipy-$SCIPY_VERSION && \
    git submodule update --init

RUN cd scipy-$SCIPY_VERSION && python3.11 setup.py bdist_wheel build_ext -j 4

ENV BUILT_SCIPY_WHEEL=scipy-$SCIPY_VERSION/dist/SciPy-*.whl
RUN ls $BUILT_SCIPY_WHEEL

# ------------------------ END OF SCIPY BUILD --------------------------------

# Download pandas source from git tree (pip download failed due to bad metadata)
RUN git clone -b "v$PANDAS_VERSION" --depth 1 \
    https://github.com/pandas-dev/pandas.git pandas-$PANDAS_VERSION

# Extract the pandas package and build the wheel
RUN cd pandas-$PANDAS_VERSION && python3.11 setup.py bdist_wheel build_ext -j 4

ENV BUILT_PANDAS_WHEEL=pandas-$PANDAS_VERSION/dist/pandas-$PANDAS_VERSION-*.whl

RUN ls $BUILT_PANDAS_WHEEL

# ------------------------ END OF PANDAS BUILD --------------------------------

# Install the wheels with pip
# (Note: previously this used --compile but now we already did the wheel compilation)
RUN python3.11 -m pip install --no-compile --no-cache-dir \
    -t /var/task/np_scipy_pd_layer/python \
    $BUILT_NUMPY_WHEEL \
    $BUILT_SCIPY_WHEEL \
    $BUILT_PANDAS_WHEEL && \
    ls /var/task/np_scipy_pd_layer/python

# -------------------- END OF PACKAGE INSTALLATION -----------------------------

# Clean up the sdists and wheels, uninstall non-built numpy after building SciPy wheel with it
RUN rm numpy-$NUMPY_VERSION.tar.gz && \
    rm -r numpy-$NUMPY_VERSION scipy-$SCIPY_VERSION pandas-$PANDAS_VERSION && \
    python3.11 -m pip uninstall numpy -y

RUN cp /usr/lib64/libblas.so.3.4.2 /var/task/np_scipy_pd_layer/lib/libblas.so.3 \
    && cp /usr/lib64/libgfortran.so.4.0.0 /var/task/np_scipy_pd_layer/lib/libgfortran.so.4 \
    && cp /usr/lib64/libgfortran.so.5.0.0 /var/task/np_scipy_pd_layer/lib/libgfortran.so.5 \
    && cp /usr/lib64/liblapack.so.3.4.2 /var/task/np_scipy_pd_layer/lib/liblapack.so.3 \
    && cp /usr/lib64/libquadmath.so.0.0.0 /var/task/np_scipy_pd_layer/lib/libquadmath.so.0 \
    && cd /var/task/np_scipy_pd_layer \
    && zip -r9 np_scipy_pd_layer.zip python  \
    && zip -r9 np_scipy_pd_layer.zip lib
    
FROM mlupin/docker-lambda:python3.11-build
 
COPY --from=build /var/task/np_scipy_pd_layer /opt
COPY --from=build /var/task /var/task

RUN PYTHONPATH=/opt/python python3 -c 'import numpy as np'
RUN PYTHONPATH=/opt/python python3 -c 'from scipy import spatial'
RUN PYTHONPATH=/opt/python python3 -c 'import pandas as pd'

#0 4.058 Processing scipy/linalg/_matfuncs_expm.pyx.in
#0 4.091 
#0 4.091 Error compiling Cython file:
#0 4.091 ------------------------------------------------------------
#0 4.091 ...
#0 4.091     if name == NULL:
#0 4.091         name_copy = name
#0 4.091     else:
#0 4.091         name_copy = strdup(name)
#0 4.091 
#0 4.091     capsule = PyCapsule_New(func, name_copy, &raw_capsule_destructor)
#0 4.091                                              ^
#0 4.091 ------------------------------------------------------------
#0 4.091 
#0 4.091 _ccallback_c.pyx:80:45: Cannot assign type 'void (*)(object) except *' to 'PyCapsule_Destructor'
#0 4.225 Processing scipy/linalg/_matfuncs_sqrtm_triu.pyx
#0 4.325 Processing scipy/linalg/_decomp_update.pyx.in
#0 4.390 Processing scipy/linalg/cython_lapack.pyx
#0 4.391 Traceback (most recent call last):
#0 4.391   File "/var/task/scipy-1.10.1/tools/cythonize.py", line 317, in <module>
#0 4.392     main()
#0 4.392   File "/var/task/scipy-1.10.1/tools/cythonize.py", line 313, in main
#0 4.392     find_process_files(root_dir)
#0 4.392   File "/var/task/scipy-1.10.1/tools/cythonize.py", line 302, in find_process_files
#0 4.393     for result in pool.imap_unordered(lambda args: process(*args), jobs):
#0 4.393   File "/var/lang/lib/python3.11/multiprocessing/pool.py", line 873, in next
#0 4.402     raise value
#0 4.403   File "/var/lang/lib/python3.11/multiprocessing/pool.py", line 125, in worker
#0 4.403     result = (True, func(*args, **kwds))
#0 4.403                     ^^^^^^^^^^^^^^^^^^^
#0 4.403   File "/var/task/scipy-1.10.1/tools/cythonize.py", line 302, in <lambda>
#0 4.404     for result in pool.imap_unordered(lambda args: process(*args), jobs):
#0 4.404                                                    ^^^^^^^^^^^^^^
#0 4.404   File "/var/task/scipy-1.10.1/tools/cythonize.py", line 236, in process
#0 4.404     processor_function(fromfile, tofile, cwd=path)
#0 4.405   File "/var/task/scipy-1.10.1/tools/cythonize.py", line 102, in process_pyx
#0 4.405     raise Exception('Cython failed')
#0 4.405 Exception: Cython failed
#0 4.415 Traceback (most recent call last):
#0 4.415   File "/var/task/scipy-1.10.1/setup.py", line 533, in <module>
#0 4.415 Cythonizing sources
#0 4.415     setup_package()
#0 4.415   File "/var/task/scipy-1.10.1/setup.py", line 517, in setup_package
#0 4.416     generate_cython()
#0 4.416   File "/var/task/scipy-1.10.1/setup.py", line 266, in generate_cython
#0 4.416     raise RuntimeError("Running cythonize failed!")
#0 4.416 RuntimeError: Running cythonize failed!
------
Dockerfile:93
--------------------
  91 |         git submodule update --init
  92 |     
  93 | >>> RUN cd scipy-$SCIPY_VERSION && python3.11 setup.py bdist_wheel build_ext -j 4
  94 |     
  95 |     ENV BUILT_SCIPY_WHEEL=scipy-$SCIPY_VERSION/dist/SciPy-*.whl
--------------------
ERROR: failed to solve: process "/bin/sh -c cd scipy-$SCIPY_VERSION && python3.11 setup.py bdist_wheel build_ext -j 4" did not complete successfully: exit code: 1

[tylerjereddy]

I have a few thoughts on what you're doing here, but my first one is if you really need bleeding edge Cython? Can you not use pip install "Cython<3.0.0"?

I can build SciPy from source and pass the full test suite locally with the maintenance/1.11.x branch and Cython 3.0.0. However, python setup.py bdist_wheel build_ext -j 4 does indeed fail with Cython 3.0.0, which is perhaps not hugely surprising considering our build system modernization/switch to focus on meson: https://scipy.github.io/devdocs/building/understanding_meson.html

Hopefully that information is helpful to you. Let me check with @rgommers re: priority on backporting support for Cython 3.0.0 with setup.py. My initial inclination is that it would be nice to just suggest folks start using meson in 1.11.x series, in cases where it is feasible to do so, and if using cython<3.0.0 helps in other from-source cases, that might be enough to get by. Our pyproject.toml should enforce the Cython upper bound for "conventional" wheel builds from source, but I see you are not doing that (I know there are reasons people have for not doing that).

[lmmx]

Ah yes it may well be Cython, I see version 3 came out mid July. I wasn't intentionally going "bleeding edge" no!

I ran our previous known-good build overnight and indeed that too now fails (where it previously ran) as it was last done mid-June pre-v3 release (Cython version was not pinned).

I will try to follow the guidance on meson in future.

As for reasons for not doing that now, originally the distutils build from source approach was to reduce packaged size for cloud deployment but I'm not sure if this is an issue now or merely requires more time investment to migrate.

Please feel free to close if Cython backport support is not a priority, thanks for all your work here.

[rgommers]

the distutils build from source approach was to reduce packaged size for cloud deployment

Do you do things like strip out tests and unused components to fit into AWS Lambda or some such thing? If so, I can suggest a replacement probably, and it'd be good to know if people still need that kind of thing.

Please feel free to close if Cython backport support is not a priority, thanks for all your work here.

That wasn't one simple fix, so I don't think we want to add Cython 3.0 support retroactively. So I'll close this.

Also, I'll open a PR to remove building with python setup.py install now. Two bug reports in a day is evidence that folks are still doing this instead of using pip/build, and it's basically degrading slowly and not robust. We're only keeping it around for conda-forge builds on Windows, no one else should be using it anymore.

[Kadelka]

No solution: pip doesn't work correctly for my system, even for matplotlib which I can compile with setup.py.
One problem seems to be
pkgutil.py
which in version 3.12 of python is missing the class "ImpImporter". Is there a workaround?

[rgommers]

I don't know how to answer that question. pip install scipy, or an equivalent like pip install . or python -m build, is the only supported build method. You are probably missing a non-Python dependency, or you have a broken environment. To say more, I'd need to see a full traceback.