Installation via pip : a transitive dependency not installed in a very specific case

[tramora]

Description

When installing khiops-python via pip, it appears that the required transitive dependency "cffi" is not installed in a very specific case (and an exception is raised at runtime) :

Base image : datatools-virt-docker-stable.repos.tech.orange/spark-py:3.5.5-hadoop3.3.6-java17-scala2.13-python3.9
...
pip install --no-cache-dir 'https://github.com/KhiopsML/khiops-python/releases/download/10.3.0.0/khiops-10.3.0.0.tar.gz' "khiops[s3]"

The cffi package is missing

Successfully installed boto3-1.35.69 botocore-1.35.99 jmespath-1.0.1 joblib-1.4.2 khiops-10.3.0.0 numpy-2.0.2 pandas-2.2.3 pyopenssl-24.3.0 python-dateutil-2.9.0.post0 pytz-2025.2 s3transfer-0.10.4 scikit-learn-1.6.1 scipy-1.13.1 threadpoolctl-3.6.0 tzdata-2025.2 urllib3-1.26.20

In a successful installation (say with base image dockerproxy.repos.tech.orange/python:3.9-slim-bullseye), it seems fine

Successfully installed boto3-1.35.69 botocore-1.35.99 cffi-1.17.1 cryptography-44.0.2 jmespath-1.0.1 joblib-1.4.2 khiops-10.3.0.0 numpy-2.0.2 pandas-2.2.3 pycparser-2.22 pyopenssl-24.3.0 python-dateutil-2.9.0.post0 pytz-2025.2 s3transfer-0.10.4 scikit-learn-1.6.1 scipy-1.13.1 six-1.17.0 threadpoolctl-3.6.0 tzdata-2025.2 urllib3-1.26.20

We can see also the following packages are missing in the bogus case :

• cffi-1.17.1 (C Foreign Function Interface for Python)
• cryptography-44.0.2
• pycparser-2.22 (pycparser is a parser for the C language, written in pure Python)
• six-1.17.0 (Six is a Python 2 and 3 compatibility library)

[tramora]

There is a mystery with this base image datatools-virt-docker-stable.repos.tech.orange/spark-py:3.5.5-hadoop3.3.6-java17-scala2.13-python3.9 that prevents from installing all the required dependencies. I've tried upgrading pip (pip install -U ...) without any further improvement

[popescu-v]

Perhaps the native packages libssl3 and libffi8 are not installed or not up to date on the Docker image? Hence, perhaps doing apt-get update && apt-get install -y libssl3 libffi8 would improve the outcome?

[tramora]

Perhaps the native packages libssl3 and libffi8 are not installed or not up to date on the Docker image? Hence, perhaps doing apt-get update && apt-get install -y libssl3 libffi8 would improve the outcome?

I've tried this but libssl3 could not be installed because of a conflict and libffi8 is already the latest.

I found an interesting path :

In this datatool spark image, python3-cryptography 41.0.7-4ubuntu0.1 is already installed system-wide as an ubuntu package.
Thus, it is not installed any longer by pip.

When this package is removed before installing khiops-python, the 3 python modules are installed by pip as transitive dependencies with the correct versions : cryptography-44.0.2, cffi-1.17.1, pycparser-2.22.

To me it means that we have to ask for a minimal version of cryptography

[popescu-v]

Or perhaps using the Conda package in a Conda environment could provide a better insulation from the underlying system?

Or else, for a system-wide + Pip setup, force the installation, via Pip, of the correct cryptography package version, prior to Pip-installing Khiops?

[tramora]

@MathieuCoupe : could you please take into account the latest suggestions (instead of forcing the sole installation of cffi 1.17.1) and ask to close this issue if you are satisfied with the proposal ? At the project level we do not see what modification could handle such subtle specificity of the target environement.

[popescu-v]

A possible resolution path consists in pinning cryptography to >= 44.0.2 in setup.py in the extras-require "s3" key.

However, we need to investigate whether this constraint should also be enforced in the Conda package metadata, via an addition of cryptography >= 44.0.2 to the run_constrained requirement.