Skip to main content

PyPy builds on conda-forge

· 3 min read
Isuru Fernando

conda-forge now supports PyPy3.6 as the python interpreter in a conda environment

Supported platforms are,

  • Linux-x86_64 (glibc 2.12 or newer)
  • OSX-x86_64 (OSX 10.9 or newer)
  • Linux-aarch64 (glibc 2.17 or newer)
  • Linux-ppc64le (glibc 2.17 or newer)

How to use PyPy​

To use the PyPy builds you can do the following,

conda config --set channel_priority strict
conda create -n pypy pypy
conda activate pypy

If you don't have a conda installation already, you can use miniforge-pypy3 which gives you a conda installation powered by pypy itself.

However as of the writing of this post, not many conda packages can be installed into this environment, but noarch packages which do not depend on the python version nor the interpreter can be installed. For eg, mpmath is a noarch package without any dependencies.

conda install mpmath    # succeeds
conda install numpy     # fails as of March 10, 2020

UPDATE: numpy and scipy builds are working as of April 10, 2020.

All python C extensions needs to be rebuilt for the PyPy ABI. This is currently on the way and can be tracked at the status page.

python_abi Package​

As part of adding support for PyPy and to keep the older python builds working, a python_abi package was added. This defines the abi for the python package and any non-noarch python packages will have a dependency on this package. Older python downstream packages like numpy had their metadata patched to add a CPython ABI. You can ask for a specific python ABI.

conda install "python_abi=*=*_cp27mu"

If you are using python packages from packages other than defaults, you will be able to install python extensions built with CPython ABI into PyPy builds as their metadata have not been patched. Solution in this case is to hotfix the metadata which is available to only high volume conda channel or to rebuild those packages with the new python packages and mark the older ones as broken.

Using the newer packages will rerender the following requirement to add a dependency on python_abi. For eg,

requirements:
  host:
    - python 3.6
  run:
    - python

is rendered as,

requirements:
  host:
    - python 3.6.10 h9d8adfe_1009_cpython
  run:
    - python >=3.6,<3.7.0a0
    - python_abi 3.6 *_cp36m

whereas,

requirements:
  host:
    - python 3.6 *_73_pypy
  run:
    - python

is rendered as,

requirements:
  host:
    - python 3.6.9 0_73_pypy
  run:
    - python >=3.6,<3.7.0a0
    - python_abi 3.6 *_pypy36_pp73

Note that the PyPy ABI tag has pp73 at the end which indicates that the ABI is stable only for PyPy3.6 7.3.x series.

This opens up the possibility of adding debug builds of python and building extension packages using the Python Debug ABI.

Discussion on the PyPy builds can be found in the issue conda-forge/conda-forge.github.io#867.