Full installation instructions#

Installation instructions for the gmxapi Python package, built on GROMACS.

Command line examples assume the bash shell.

Regarding multiple GROMACS installations

Many GROMACS users switch between multiple GROMACS installations on the same computer using an HPC module system and/or a GMXRC configuration script. For the equivalent sort of environment switching with the gmxapi Python package, we recommend installing it in a different Python virtual environment for each GROMACS installation. Once built, a particular copy of the gmxapi Python package always refers to the same GROMACS installation.

Unprivileged pip install

The following documentation contains frequent references to the pip tool for installing Python packages. In some cases, an unprivileged user should use the --user command line flag to tell pip to install packages into the user site-packages directory rather than the default site-packages directory for the Python installation. This flag is not appropriate when running pip in a virtual environment (as recommended) and is omitted in this documentation. If you need the --user flag, you should modify the example commands to look something like pip install --upgrade somepackage --user

Python 3 executable names

These instructions use the executable names python and pip instead of python3 or pip3. Some Python installations require the 3 suffix, but it is usually not necessary if you have already activated a Python virtual environment (recommended).

Overview#

Typically, setting up the gmxapi Python package follows these three steps. If this overview is sufficient for your computing environment, you may disregard the rest of this document.

Install GROMACS#

Locate your GROMACS installation, or build and install. GROMACS 2022 or higher is recommended.

See also

GROMACS installation

The following assumes GROMACS is installed to /path/to/gromacs

Set up a Python virtual environment#

See also

Set up a Python virtual environment

Note

mpi4py may require additional arguments (compiler hints). See MPI requirements

python3 -m venv $HOME/myvenv
. $HOME/myvenv/bin/activate
python -m ensurepip --default-pip
pip install --upgrade pip setuptools wheel
pip install mpi4py

Install the gmxapi Python package#

. /path/to/gromacs/bin/GMXRC
pip install --no-cache-dir gmxapi

See also

Installing the Python package

Background#

gmxapi comes in three parts:

  • GROMACS gmxapi library for C++.

  • This Python package, supporting Python 3.7 and higher

  • MD restraint plugins and sample gmxapi client code

GROMACS requirements#

The Python package requires a GROMACS installation. Locate an existing GROMACS installation, or build and install GROMACS before proceeding.

Note

Note that gmxapi requires that GROMACS is configured with GMXAPI=ON and BUILD_SHARED_LIBS=ON. These are enabled by default in most cases. If these options were overridden for your GROMACS installation, you will see CMake errors when trying to build and install the gmxapi Python package or other client software.

If your installation has a GMXRC file, “source” the file as you normally would before using GROMACS. Otherwise, note the installation location so that you can provide it when building the gmxapi package.

Build system requirements#

gmxapi can be built for Python 3.7 and higher.

You will need a C++ 17 compatible compiler and a reasonably up-to-date version of CMake. Full gmxapi functionality may also require an MPI compiler (e.g. mpicc).

Important: To build a module that can be imported by Python, you need a Python installation that includes the Python headers. Unfortunately, it is not always obvious whether these headers are present or where to find them. The simplest answer is to just try to build the Python package using these instructions, and if gmxapi is unable to find the Python tools it needs, try a different Python installation or install the additional development packages.

On a Linux system, this may require installing packages such as python-dev and/or python3-dev. If you are building Python, either from scratch or with a tool like pyenv install (see wiki entry ), be sure to enable installation of the Python C library with the --enable-shared flag. Alternatively, various Python distributions provide a sufficient build environment while only requiring installation into a user home directory. (Some examples below.)

If you are using an HPC system with software available through modules you may be able to just module load a different Python installation and find one that works.

Python environment requirements#

gmxapi requires Python 3.7 or higher. Check your version with python3 --version or python --version.

Note

The following documentation assumes you do not need to use a trailing ‘3’ to access a Python 3 interpreter on your system. The default Python interpreter on your system may use python3 and pip3 instead of python and pip. You can check the version with python3 --version or python --version and pip --version.

To build and install, you need the Python packages for cmake, networkx, and setuptools (all available from PyPI with pip).

For full functionality, you should also have mpi4py and numpy. These requirements and version numbers are listed in requirements.txt.

The easiest way to make sure you have the requirements installed, first update pip, then use the requirements.txt file provided with the repository. File paths in this section are relative to the root directory of your local copy of the GROMACS source.

Confirm that pip is available, install pip if it is missing, or get instructions on how to install pip:

python -m ensurepip --default-pip

Install or upgrade required components:

python -m pip install --upgrade pip
pip install --upgrade setuptools wheel

“requirements” files in GROMACS source tree#

If you are building from source code in a local copy of the GROMACS source repository, a requirements.txt allows you to preinstall the Python requirements before installing the gmxapi package.

pip install -r python_packaging/gmxapi/requirements.txt

Documentation build requirements#

See Accessing gmxapi documentation

Testing requirements#

Note that the test suite is only available in the GROMACS source tree. (It is not part of the installed package.) Acquire the GROMACS sources with git or by downloading an archive, as documented elsewhere.

Testing is performed with pytest.

python_packaging/gmxapi/requirements.txt lists additional requirements for testing. With pip:

pip install -r python_packaging/gmxapi/requirements.txt

To test the full functionality also requires an MPI parallel environment. You will need the mpi4py Python package and an MPI launcher (such as mpiexec, mpirun, a launcher provided by your HPC queuing system, or whatever is provided by your favorite MPI package for your operating system).

MPI requirements#

For the ensemble simulations features, you will need an MPI installation.

On an HPC system, this means you will probably have to use module load to load a compatible set of MPI tools and compilers. Check your HPC documentation or try module avail to look for an openmpi, mpich, or mvapich module and matching compiler module. This may be as simple as:

module load gcc
module load mpicc

If you are using a GROMACS installation that is already available through module load, try to find a Python installation with the mpi4py package that is also available through module load. The module system will generally enforce toolchain compatibility between the loaded modules. If you module load mpi4py or a Python installation with mpi4py, you will probably want to use this version of the package in your venv. (See Set up a Python virtual environment) If you module load an MPI-enabled GROMACS installation, gmxapi will try to check mpi4py for compatibility.

Note that the compilers loaded might not be the first compilers discovered automatically by the build tools we will use below, so you may have to specify compilers on the command line for consistency. It may be necessary to require that GROMACS, gmxapi, and the sample code are built with the same compiler(s).

Note that strange errors have been known to occur when mpi4py is built with a different tool set than has been used to build Python and gmxapi. If the default compilers on your system are not sufficient for GROMACS or gmxapi, you may need to build, e.g., OpenMPI or MPICH, and/or build mpi4py with a specific MPI compiler wrapper. This can complicate building in environments such as Conda. You should be able to confirm that your MPI compiler wrapper is consistent with your GROMACS tool chain by comparing the output of mpicc --version with the compiler information reported by gmx --version.

Set the MPICC environment variable to the MPI compiler wrapper and forcibly reinstall mpi4py:

export MPICC=`which mpicc`
pip install --no-cache-dir --upgrade --no-binary ":all:" --force-reinstall mpi4py

If you have a different MPI C compiler wrapper, substitute it for mpicc above.

While gmxapi is configuring its build system during installation, it will try to confirm the compatibility of the mpi4py toolchain with that of the GROMACS installation. If they appear incompatible, you should see a CMake message that includes a guess at what you might try using for MPICC. (If using pip, consider using the --verbose option for more build output.)

Installing the Python package#

We recommend using Python’s pip package installer to automatically download, build, and install the latest version of the gmxapi package into a Python virtual environment, though it is also possible to install without a virtual environment. If installing without a virtual environment as an un-privileged user, you may need to use the --user option with pip install.

Install from source#

You can also install the gmxapi Python package from within a local copy of the GROMACS source repository. Assuming you have already obtained the GROMACS source code and you are in the root directory of the source tree, you will find the gmxapi Python package sources in the python_packaging/gmxapi directory.

cd python_packaging/gmxapi
pip install -r requirements.txt
pip install .

Offline install#

Recommended, first:

pip install --upgrade build pip setuptools wheel

You can use python -m build --skip-dependency-check to build a binary distribution archive (from the source distribution) for just the gmxapi package, but then you will have to manually satisfy (separate) dependencies in both the build and installation environments.

While you have internet access, you need to get access to the gmxapi source distribution and the package dependencies. You will also want the wheel and build packages in environments where the package(s) will be built. Only pip is necessary once a gmxapi wheel is built.

The following instructions are paraphrased from https://pip.pypa.io/en/stable/user_guide/#installing-from-local-packages

To build with internet access and then install without:

# Remove any locally cached (previously built) wheels.
pip cache remove gmxapi

# Download gmxapi and dependencies from pypi.
pip wheel --wheel-dir DIR gmxapi
# or, using package source from the GROMACS repository
cd python_packaging/gmxapi
pip wheel --wheel-dir DIR .

# Later, install.
pip install --no-index --find-links=DIR DIR/gmxapi*whl

To download packages and dependencies for later build and installation:

# if in the GROMACS source repository
cd python_packaging/gmxapi
# or download and expand the archive
pip download --destination-directory DIR gmxapi
tar xf DIR/gmxapi*
cd gmxapi*

# Pre-fetch dependencies to DIR
pip download --destination-directory DIR .

# Build and install from the source directory.
pip install --no-index --find-links=DIR .

Building a source archive#

A source archive for the gmxapi python package can be built from the GROMACS source repository using the Python build module.

Example:

pip install --upgrade setuptools build
cd python_packaging/gmxapi
python -m build --sdist

This command will create a dist directory containing a source distribution archive file. The file name has the form gmxapi-version.suffix, where version is the version from the package metadata, and suffix is an archive file extension determined by the local environment and the current packaging specifications.

The version information is derived from gmxapi.__version__ defined by the gmxapi.version module. Pending refinement under Issue 3851, the gmxapi version information is hard coded in the version.py. Make sure you have an up-to-date version of the sources and that the version information is appropriate before distributing a new release.

See also

Python documentation for creating a source distribution

Package maintainers may update the online repository by uploading a freshly built sdist with python -m twine upload dist/gmxapi-{version}.{suffix}. To update the repository at the PyPI test server, use python -m twine upload --repository testpypi dist/gmxapi-{version}.{suffix}.

Accessing gmxapi documentation#

Documentation for the Python classes and functions in the gmx module can be accessed in the usual ways, using pydoc from the command line or help() in an interactive Python session.

The complete documentation (which you are currently reading) can be browsed online or built from a copy of the GROMACS source repository.

Documentation is built from a combination of Python module documentation and static content, and requires a local copy of the GROMACS source repository.

Build with GROMACS#

To build the full gmxapi documentation with GROMACS, configure GROMACS with -DGMX_PYTHON_PACKAGE=ON and build the GROMACS documentation normally. This will first build the gmxapi Python package and install it to a temporary location in the build tree. Sphinx can then import the package to automatically extract Python docstrings.

Note that this is an entirely CMake-driven installation and Python dependencies will not be installed automatically. You can update your Python environment (before configuring with CMake) using the requirements.txt files provided in the python_packaging/ directory of the repository. Example:

pip install -r python_packaging/gmxapi/requirements.txt

Sometimes the build environment can choose a different Python interpreter than the one you intended. You can set the Python3_ROOT_DIR or CMAKE_PREFIX_PATH CMake variable to explicitly choose the Python installation or venv directory. See also CMake FindPython3.

If you use pyenv or pyenv-virtualenv to dynamically manage your Python version, you can help identify a particular version with pyenv version-name and the directory with pyenv prefix {version}. For example:

-DPython3_ROOT_DIR=$(pyenv prefix $(pyenv version-name))

Testing#

Note testing requirements above.

After installing the gmxapi Python package, you can run the Python test suite from the GROMACS source tree. Example:

# Assuming you are in the root directory of the repository:
pytest python_packaging/gmxapi/test/

Refer to python_packaging/README.md for more detailed information.

Troubleshooting#

ImportError at run time with dynamic linking error#

Symptom: Python fails with a weird ImportError citing something like dlopen:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen(/.../gmxapi/_gmxapi.so, 0x0002): Symbol not found:
__ZN12gmxapicompat11readTprFileERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
  Referenced from: /.../gmxapi/_gmxapi.so
  Expected in: /path/to/gromacs/lib/libgmxapi_mpi_d.0.3.1.dylib

Inconsistencies in the build and run time environments can cause dynamic linking problems at run time. This could occur if you reinstall GROMACS built with a different compiler, or if pip or CMake somehow get tricked into using the wrong compiler tool chain.

Refer to the gmxapi cmake hints for notes about compiler toolchains. Rebuild and reinstall the gmxapi Python package with --no-cache-dir and provide the gromacs-hints.cmake file for the GROMACS installation you intend to use.

AttributeError: module ‘enum’ has no attribute ‘IntFlag’#

If you had older versions of some of the dependencies installed, you might have picked up a transitive dependency on the enum34 package. Try:

pip uninstall -y enum34

and see if that fixes the problem. If not, try a fresh virtual environment (see above) to help narrow down the problem before you open an issue.

Errors regarding pybind11#

An error may occur in setup.py with output that contains something like the following:

  ModuleNotFoundError: No module named 'pybind11'
  Building wheel for gmxapi (pyproject.toml): finished with status 'error'
  ERROR: Failed building wheel for gmxapi
Failed to build gmxapi
ERROR: Could not build wheels for gmxapi, which is required to install pyproject.toml-based projects

The important information here is that pybind11 was not found.

Build dependencies aren’t always automatically installed. Even if you are using pip, you may have disabled automatic dependency fulfillment with an option like --no-build-isolation or --no-deps.

In any case, the problem should be resolved by explicitly installing the pybind11 Python package before attempting to build gmxapi:

pip install --upgrade pybind11

Couldn’t find the gmxapi support library?#

If you don’t want to “source” your GMXRC file, you can tell the package where to find a gmxapi compatible GROMACS installation with gmxapi_ROOT. E.g. gmxapi_ROOT=/path/to/gromacs pip install .

Before updating the gmxapi package it is generally a good idea to remove the previous installation and to start with a fresh build directory. You should be able to just pip uninstall gmxapi.

Do you see something like the following?

CMake Error at gmx/core/CMakeLists.txt:45 (find_package):
   Could not find a package configuration file provided by "gmxapi" with any
   of the following names:

     gmxapiConfig.cmake
     gmxapi-config.cmake

   Add the installation prefix of "gmxapi" to CMAKE_PREFIX_PATH or set
   "gmxapi_ROOT" to a directory containing one of the above files.  If "gmxapi"
   provides a separate development package or SDK, be sure it has been
   installed.

This could be because

  • GROMACS is not already installed

  • GROMACS was built without the CMake variable GMXAPI=ON

  • or if gmxapi_ROOT (or GROMACS_DIR) is not a path containing directories like bin and share.

If you are not a system administrator you are encouraged to install in a Python virtual environment, created with virtualenv or Conda. Otherwise, you will need to specify the --user flag to pip.

Two of the easiest problems to run into are incompatible compilers and incompatible Python. Try to make sure that you use the same C and C++ compilers for GROMACS, for the Python package, and for the sample plugin. These compilers should also correspond to the mpicc compiler wrapper used to compile mpi4py. In order to build the Python package, you will need the Python headers or development installation, which might not already be installed on the machine you are using. (If not, then you will get an error about missing Python.h at some point.) If you have multiple Python installations (or modules available on an HPC system), you could try one of the other Python installations, or you or a system administrator could install an appropriate Python dev package. Alternatively, you might try installing your own Anaconda or MiniConda in your home directory.

If an attempted installation fails with CMake errors about missing “gmxapi”, make sure that Gromacs is installed and can be found during installation. For instance,

gmxapi_ROOT=/Users/eric/gromacs pip install --verbose gmxapi

Pip and related Python package management tools can be a little too flexible and ambiguous sometimes. If things get really messed up, try explicitly uninstalling the gmxapi module and its dependencies, then do it again and repeat until pip can no longer find any version of any of the packages.

pip uninstall gmxapi
pip uninstall cmake
# ...

Successfully running the test suite is not essential to having a working gmxapi package. We are working to make the testing more robust, but right now the test suite is a bit delicate and may not work right, even though you have a successfully built the gmxapi package. If you want to troubleshoot, though, the main problems seem to be that automatic installation of required python packages may not work (requiring manual installations, such as with pip install somepackage) and ambiguities between python versions.

If you are working in a development branch of the repository, note that the upstream branch may be reset to main after a new release is tagged. In general, but particularly on the devel branch, when you do a git pull, you should use the --rebase flag.

If you fetch this repository and then see a git status like this:

$ git status
On branch devel
Your branch and 'origin/devel' have diverged,
and have 31 and 29 different commits each, respectively.

then gmxapi has probably entered a new development cycle. You can do git pull --rebase to update to the latest development branch.

If you do a git pull while in devel and get a bunch of unexpected merge conflicts, do git merge --abort; git pull --rebase and you should be back on track.

If you are developing code for gmxapi, this should be an indication to rebase your feature branches for the new development cycle.