Containers

GROMACS project infrastructure uses Docker containerization to isolate automated tasks. A number of images are maintained to provide a breadth of testing coverage.

Scripts and configuration files for building images are stored in the repository under admin/containers/ Images are (re)built manually by GROMACS project staff and pushed to DockerHub and GitLab. See https://hub.docker.com/u/gromacs and https://gitlab.com/gromacs/gromacs/container_registry

GitLab Container Registry

CI Pipelines use a GitLab container registry instead of pulling from Docker Hub.

Project members with role Developer or higher privilege can push images to the container registry.

Steps:

  1. Create a personal access token (docs) with write_registry and read_registry scopes. Save the hash!

  2. Authenticate from the command line with docker login registry.gitlab.com -u <user name> -p <hash>

  3. docker push registry.gitlab.com/gromacs/gromacs/<imagename>

Refer to buildall.sh in the master branch for the set of images currently built.

Within pipeline jobs, jobs specify a Docker image with the image property. For image naming convention, see utility.image_name(). Images from the GitLab registry are easily accessible with the same identifier as above. For portability, CI environment variables may be preferable for parts of the image identifier. Example:

some_job:
  image: ${CI_REGISTRY_IMAGE}/ci-<configuration>
  ...

For more granularity, consider equivalent expressions ${CI_REGISTRY}/${CI_PROJECT_PATH} or ${CI_REGISTRY}/${CI_PROJECT_NAMESPACE}/${CI_PROJECT_NAME} Ref: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html

Utilities

utility.py

A utility module to help manage the matrix of configurations for CI testing and build containers.

When called as a stand alone script, prints a Docker image name based on the command line arguments. The Docker image name is of the form used in the GROMACS CI pipeline jobs.

Example:

$ python3 -m utility --llvm --doxygen
gromacs/ci-ubuntu-20.04-llvm-7-docs

See also

buildall.sh

As a module, provides importable argument parser and docker image name generator.

Note that the parser is created with add_help=False to make it friendly as a parent parser, but this means that you must derive a new parser from it if you want to see the full generated command line help.

Example:

import utility.parser
# utility.parser does not support `-h` or `--help`
parser = argparse.ArgumentParser(
    description='GROMACS CI image creation script',
    parents=[utility.parser])
# ArgumentParser(add_help=True) is default, so parser supports `-h` and `--help`

See also

scripted_gmx_docker_builds.py

Authors:
utility.image_name(configuration: argparse.Namespace) → str

Generate docker image name.

Image names have the form ci-<slug>, where the configuration slug has the form:

<distro>-<version>-<compiler>-<major version>[-<gpusdk>-<version>][-<use case>]

This function also applies an appropriate Docker image repository prefix.

Parameters

configuration – Docker image configuration as described by the parsed arguments.

utility.parser = ArgumentParser(prog='sphinx-build', usage=None, description='GROMACS CI image slug options.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=False)

A parent parser for tools referencing image parameters.

This argparse parser is defined for convenience and may be used to partially initialize parsers for tools.

Warning

Do not modify this parser.

Instead, inherit from it with the parents argument to argparse.ArgumentParser

scripted_gmx_docker_builds.py

Building block based Dockerfile generation for CI testing images.

Generates a set of docker images used for running GROMACS CI on Gitlab. The images are prepared according to a selection of build configuration targets that hope to cover a broad enough scope of different possible systems, allowing us to check compiler types and versions, as well as libraries used for accelerators and parallel communication systems. Each combinations is described as an entry in the build_configs dictionary, with the script analysing the logic and adding build stages as needed.

Based on the example script provided by the NVidia HPCCM repository.

Reference:

NVidia HPC Container Maker

Authors:

Usage:

$ python3 scripted_gmx_docker_builds.py --help
$ python3 scripted_gmx_docker_builds.py --format docker > Dockerfile && docker build .
$ python3 scripted_gmx_docker_builds.py | docker build -

See also

buildall.sh