Unable to build docker image with Dockerfile from docs

[raaymax]

I tried to create docker image with Dockerfile from docs but i was getting errors while build:

52.97       running build_ext
52.97       creating /tmp/pip-install-fk4h8ll_/pyarrow_01f9da22bdf240a7bab24a03236cc37f/build/temp.linux-aarch64-cpython-312
52.97       -- Running cmake for PyArrow
52.97       cmake -DCMAKE_INSTALL_PREFIX=/tmp/pip-install-fk4h8ll_/pyarrow_01f9da22bdf240a7bab24a03236cc37f/build/lib.linux-aarch64-cpython-312/pyarrow -DPYTHON_EXECUTABLE=/app/venv/bin/python3 -DPython3_EXECUTABLE=/app/venv/bin/python3 -DPYARROW_CXXFLAGS= -DPYARROW_BUILD_CUDA=off -DPYARROW_BUILD_SUBSTRAIT=off -DPYARROW_BUILD_FLIGHT=off -DPYARROW_BUILD_GANDIVA=off -DPYARROW_BUILD_ACERO=off -DPYARROW_BUILD_DATASET=off -DPYARROW_BUILD_ORC=off -DPYARROW_BUILD_PARQUET=off -DPYARROW_BUILD_PARQUET_ENCRYPTION=off -DPYARROW_BUILD_GCS=off -DPYARROW_BUILD_S3=off -DPYARROW_BUILD_HDFS=off -DPYARROW_BUNDLE_ARROW_CPP=off -DPYARROW_BUNDLE_CYTHON_CPP=off -DPYARROW_GENERATE_COVERAGE=off -DCMAKE_BUILD_TYPE=release /tmp/pip-install-fk4h8ll_/pyarrow_01f9da22bdf240a7bab24a03236cc37f
52.97       error: command 'cmake' failed: No such file or directory
52.97       [end of output]
52.97   
52.97   note: This error originates from a subprocess, and is likely not a problem with pip.
52.97   ERROR: Failed building wheel for pyarrow
52.97   Building wheel for websockets (pyproject.toml): started
53.06   Building wheel for websockets (pyproject.toml): finished with status 'done'
53.06   Created wheel for websockets: filename=websockets-10.4-cp312-cp312-linux_aarch64.whl size=95017 sha256=2fbe73b489f03d352eaeac2245efb4f27bb9bf4995588baf03ab3be1581bd645
53.06   Stored in directory: /root/.cache/pip/wheels/80/cf/6d/5d7e4c920cb41925a178b2d2621889c520d648bab487b1d7fd
53.06 Successfully built websockets
53.06 Failed to build pandas pyarrow
53.06 ERROR: Could not build wheels for pandas, pyarrow, which is required to install pyproject.toml-based projectsa

so I added:

RUN apt-get install build-essential cmake python3-dev -y  

but also with no luck:

364.8       -- Found Python3: /app/venv/bin/python3 (found version "3.12.2") found components: Interpreter Development.Module NumPy
364.8       -- Found Python3Alt: /app/venv/bin/python3
364.8       CMake Error at CMakeLists.txt:261 (find_package):
364.8         By not providing "FindArrow.cmake" in CMAKE_MODULE_PATH this project has
364.8         asked CMake to find a package configuration file provided by "Arrow", but
364.8         CMake did not find one.
364.8       
364.8         Could not find a package configuration file provided by "Arrow" with any of
364.8         the following names:
364.8       
364.8           ArrowConfig.cmake
364.8           arrow-config.cmake
364.8       
364.8         Add the installation prefix of "Arrow" to CMAKE_PREFIX_PATH or set
364.8         "Arrow_DIR" to a directory containing one of the above files.  If "Arrow"
364.8         provides a separate development package or SDK, be sure it has been
364.8         installed.
364.8       
364.8       
364.8       -- Configuring incomplete, errors occurred!
364.8       See also "/tmp/pip-install-yvmknp7o/pyarrow_8d1226c069b4498eada8f87b0c15edd3/build/temp.linux-aarch64-cpython-312/CMakeFiles/CMakeOutput.log".
364.8       error: command '/usr/bin/cmake' failed with exit code 1
364.8       [end of output]
364.8   
364.8   note: This error originates from a subprocess, and is likely not a problem with pip.
364.8   ERROR: Failed building wheel for pyarrow
364.8   Building wheel for websockets (pyproject.toml): started
365.0   Building wheel for websockets (pyproject.toml): finished with status 'done'
365.0   Created wheel for websockets: filename=websockets-10.4-cp312-cp312-linux_aarch64.whl size=106999 sha256=745f7e1a97183113e448135819f5c3e8b2a0d0198822621513f3ae926c3fc9b4
365.0   Stored in directory: /root/.cache/pip/wheels/80/cf/6d/5d7e4c920cb41925a178b2d2621889c520d648bab487b1d7fd
365.0 Successfully built pandas websockets
365.0 Failed to build pyarrow
365.0 ERROR: Could not build wheels for pyarrow, which is required to install pyproject.toml-based projects

Below my solution that worked for me:

[tool.poetry]
name = "project-name"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.9.2"
streamsync = {extras = ["ds"], version = "^0.3.0"}

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

and my Dockerfile:

FROM python:3.10-bullseye
RUN apt-get update -y && mkdir /app
RUN apt-get install build-essential cmake python3-dev -y
COPY . /app
WORKDIR /app
RUN pip3 install poetry
RUN poetry config virtualenvs.create false
RUN poetry install --only main
ENTRYPOINT [ "streamsync", "run" ]
EXPOSE 8080
CMD [ ".",  "--port", "8080", "--host", "0.0.0.0" ] 

[ramedina86]

Thanks for submitting, now that I see more detail I'd say this is just because we're using base image 3-slim, which now provides 3.11 which is incompatible with the pyarrow version we pinned.

So should probably be 3.10 for now. Can you try the 3.10 base image?

The next version of Streamsync will update to use newer versions of Pyarrow which have wheels for latest versions of Python

[raaymax]

Yup! It worked! Nice! I'll create PR for docs update.

[ramedina86]

Fixed in 0.4.0 update.