Collaboration diagram for Testing Utilities (testutils):

Description

Common helper classes and functions for writing tests using Google Test.

General documentation for the testing in GROMACS can be found in the developer guide. This page provides an overview of the actual API provided by the testutils module.

There are several distinct functionalities provided:

gmx::test::TestFileManager (in testfilemanager.h) provides functionality for locating test input files from the source directory and managing temporary files that need to be created during the test.
gmx::test::TestFileInputRedirector (in testfileredirector.h) provides functionality for capturing file existence checks in code that uses gmx::IFileInputRedirector.
gmx::test::TestFileOutputRedirector (in testfileredirector.h) provides functionality for capturing file output (including stdout) from code that uses gmx::IFileOutputRedirector, and checking that output against reference data.
gmx::test::InteractiveTestHelper (in interactivetest.h) provides a helper class for testing an interactive session that uses gmx::TextInputStream and gmx::TextOutputStream for prompting input and printing status messages.
GMX_TEST_OPTIONS macro provides facilities for adding custom command line options for the test binary.
testasserts.h provides several custom test assertions for better exception and floating-point handling than built-in Google Test assertions.
gmx::test::TestReferenceData and related classes (in refdata.h) provide utilities to write regression-style tests that check that the test produces the same results as an earlier run of the same test. See Using reference data in C++ tests for more details.
gmx::test::CommandLine and related classes (in cmdlinetest.h) provide utilities for constructing command line argument arrays for use in tests that invoke actual commands. The same header also provides gmx::test::CommandLineTestHelper and gmx::test::CommandLineTestBase for easily creating tests that execute essentially full command-line programs, including checking the output files produced.
gmx::test::StringTestBase provides a test fixture for tests that need to test long strings for correctness.
gmx::test::StdioTestHelper provides a helper class for tests that need to execute legacy code where stdin reading cannot be easily mocked.

Additionally, testinit.h and mpi_printer.h, and their corresponding source files, provide functionality that is not visible on the API level: they provide initialization routines for the above functionality, which are automatically called by the main() function provided in unittest_main.cpp.

mpi_printer.h provides a Google Test listener that is installed when the tests are compiled with MPI. This listener allows the test binary to be run on multiple MPI ranks, and synchronizes the execution and output from the test cases, as well as makes the test fail on even if an assertion fails only on one rank.

Assertions for exceptions
These macros replace `(ASSERT\|EXPECT)(_NO)?_THROW` from Google Test. They are used exactly like the Google Test ones, but also print details of any unexpected exceptions using GROMACS-specific routines. This makes it much easier to see at one glance what went wrong. See Google Test documentation for details on how to use the macros.
#define	GMX_TEST_THROW_(statement, expected_exception, fail)
	Internal implementation macro for exception assertations. More...

#define	GMX_TEST_NO_THROW_(statement, fail)
	Internal implementation macro for exception assertations. More...

#define	EXPECT_THROW_GMX(statement, expected_exception)
	Asserts that a statement throws a given exception.

#define	EXPECT_NO_THROW_GMX(statement)
	Asserts that a statement does not throw.

#define	ASSERT_THROW_GMX(statement, expected_exception)
	Asserts that a statement throws a given exception.

#define	ASSERT_NO_THROW_GMX(statement)
	Asserts that a statement does not throw.

#define	GMX_EXPECT_DEATH_IF_SUPPORTED(expr, msg)
	Wrapper around EXPECT_DEATH_IF_SUPPORTED gtest macro for thread safe execution. More...

#define	GMX_ASSERT_DEATH_IF_SUPPORTED(expr, msg)
	Wrapper around ASSERT_DEATH_IF_SUPPORTED gtest macro for thread safe execution. More...

Assertions for floating-point comparison
These routines extend `(EXPECT\|ASSERT)_(FLOAT\|DOUBLE)_EQ` and `(EXPECT\|ASSERT)_NEAR` from Google Test to provide more flexible assertions for floating-point values. See gmx::test::FloatingPointTolerance for the possible ways to specify the tolerance, and gmx::test::FloatingPointDifference for some additional details of the difference calculation.
template<typename FloatType >
static inline::testing::AssertionResult	gmx::test::assertEqualWithinTolerance (const char expr1, const char expr2, const char *, FloatType value1, FloatType value2, const FloatingPointTolerance &tolerance)
	Assertion predicate formatter for comparing two floating-point values. More...

#define	EXPECT_FLOAT_EQ_TOL(value1, value2, tolerance)
	Asserts that two single-precision values are within the given tolerance.

#define	EXPECT_DOUBLE_EQ_TOL(value1, value2, tolerance)
	Asserts that two double-precision values are within the given tolerance.

#define	EXPECT_REAL_EQ_TOL(value1, value2, tolerance)
	Asserts that two `real` values are within the given tolerance.

#define	ASSERT_FLOAT_EQ_TOL(value1, value2, tolerance)
	Asserts that two single-precision values are within the given tolerance.

#define	ASSERT_DOUBLE_EQ_TOL(value1, value2, tolerance)
	Asserts that two double-precision values are within the given tolerance.

#define	ASSERT_REAL_EQ_TOL(value1, value2, tolerance)
	Asserts that two `real` values are within the given tolerance.

#define	EXPECT_REAL_EQ(value1, value2) EXPECT_REAL_EQ_TOL(value1, value2, ::gmx::test::defaultRealTolerance())
	EXPECT_REAL_EQ_TOL with default tolerance.

#define	ASSERT_REAL_EQ(value1, value2) ASSERT_REAL_EQ_TOL(value1, value2, ::gmx::test::defaultRealTolerance())
	ASSERT_REAL_EQ_TOL with default tolerance.

Helper functions for computing floating-point differences
These routines are used to initialize FloatingPointDifference. They peek into some internal types from Google Test (gtest-internal.h), and duplicate some other functionality from there, but that is likely a better alternative than just copying all that code here.
template<typename FloatType >
FloatingPoint< FloatType >::Bits	gmx::test::anonymous_namespace{testasserts.cpp}::floatingPointToBiasedInteger (const FloatingPoint< FloatType > &value)
	Computes biased integer representation for a floating-point value. More...

template<typename FloatType >
uint64_t	gmx::test::anonymous_namespace{testasserts.cpp}::calculateUlpDifference (const FloatingPoint< FloatType > &value1, const FloatingPoint< FloatType > &value2)
	Computes the magnitude of the difference in ULPs between two numbers, treating also values of different sign.

template<typename FloatType >
void	gmx::test::anonymous_namespace{testasserts.cpp}::initDifference (FloatType raw1, FloatType raw2, double absoluteDifference, uint64_t ulpDifference, bool *bSignDifference)
	Helper to implement the constructors for FloatingPointDifference.

template<typename FloatType >
uint64_t	gmx::test::anonymous_namespace{testasserts.cpp}::relativeToleranceToUlp (FloatType tolerance)
	Converts a relative tolerance into an ULP difference.

Classes
class	gmx::test::CommandLine
	Helper class for tests that need to construct command lines. More...

class	gmx::test::CommandLineTestHelper
	Helper class for tests that construct command lines that need to reference existing files. More...

class	gmx::test::CommandLineTestBase
	Test fixture for tests that call a single command-line program with input/output files. More...

class	gmx::test::ConfMatch
	Match the contents as an gro file. More...

class	gmx::test::IFileMatcher
	Represents a file matcher, matching file contents against reference (or other) data. More...

class	gmx::test::IFileMatcherSettings
	Represents a factory for creating a file matcher. More...

class	gmx::test::TextFileMatch
	Use a ITextBlockMatcher for matching the contents. More...

class	gmx::test::NoContentsMatch
	Do not check the contents of the file. More...

class	gmx::test::InteractiveTestHelper
	Helper class for testing interactive sessions. More...

class	gmx::test::LoggerTestHelper
	Helper class for tests to check output written to a logger. More...

class	gmx::test::TestReferenceData
	Handles creation of and comparison to test reference data. More...

class	gmx::test::TestReferenceChecker
	Handles comparison to test reference data. More...

class	gmx::test::StdioTestHelper
	Helper class for tests where code reads directly from `stdin`. More...

class	gmx::test::StringTestBase
	Test fixture for tests that check string formatting. More...

class	gmx::test::FloatingPointDifference
	Computes and represents a floating-point difference value. More...

class	gmx::test::FloatingPointTolerance
	Specifies a floating-point comparison tolerance and checks whether a difference is within the tolerance. More...

class	gmx::test::TestException
	Exception class for reporting errors in tests. More...

class	gmx::test::TestFileManager
	Helper for tests that need input and output files. More...

class	gmx::test::TestFileInputRedirector
	In-memory implementation for IFileInputRedirector for tests. More...

class	gmx::test::TestFileOutputRedirector
	In-memory implementation of IFileOutputRedirector for tests. More...

class	gmx::test::TestOptionsProvider
	Provides additional options for the test executable. More...

class	gmx::test::ITextBlockMatcher
	Represents a text matcher, matching text stream contents against reference data. More...

class	gmx::test::ITextBlockMatcherSettings
	Represents a factory for creating a text matcher. More...

class	gmx::test::ExactTextMatch
	Use an exact text match (the contents should be exactly equal). More...

class	gmx::test::NoTextMatch
	Do not match the text (the contents are ignored). More...

class	gmx::test::FilteringExactTextMatch
	Use an exact text match after scrubbing lines of the text that match the supplied regular expressions. More...

class	gmx::test::XvgMatch
	Match the contents as an xvg file. More...

class	gmx::test::internal::TestReferenceDataImpl
	Private implementation class for TestReferenceData. More...

class	gmx::test::TestReferenceChecker::Impl
	Private implementation class for TestReferenceChecker. More...

class	gmx::test::anonymous_namespace{testinit.cpp}::TestProgramContext
	Custom program context for test binaries. More...

class	gmx::test::anonymous_namespace{testoptions.cpp}::TestOptionsRegistry
	Singleton registry for test options added with GMX_TEST_OPTIONS. More...

Macros
#define	GMX_MPI_TEST(RankRequirement)
	Declares that this test is an MPI-enabled unit test and expresses the conditions under which it can run. More...

#define	EXPECT_PLAIN(expr)
	Assert for predicates that return AssertionResult and produce a full failure message. More...

#define	ASSERT_PLAIN(expr)
	Assert for predicates that return AssertionResult and produce a full failure message. More...

#define	GMX_TEST_OPTIONS(name, options)
	Macro to add additional command-line options for the test binary. More...

Enumerations
enum	gmx::test::ReferenceDataMode : int { gmx::test::ReferenceDataMode::Compare, gmx::test::ReferenceDataMode::CreateMissing, gmx::test::ReferenceDataMode::UpdateChanged, gmx::test::ReferenceDataMode::UpdateAll, gmx::test::ReferenceDataMode::Count }
	Mode of operation for reference data handling. More...

Functions
int	gmx::test::getNumberOfTestMpiRanks ()
	Returns the number of MPI ranks to use for an MPI test. More...

bool	gmx::test::threadMpiTestRunner (std::function< void()> testBody)
	Helper function for GMX_MPI_TEST().

void	gmx::test::initReferenceData (IOptionsContainer *options)
	Initializes reference data handling. More...

static inline::testing::AssertionResult	gmx::test::plainAssertHelper (const char *, const ::testing::AssertionResult &expr)
	Helper method for `(EXPECT\|ASSERT)_PLAIN`. More...

void	gmx::test::checkTestNameLength (std::optional< std::string > testName=std::nullopt)
	Gives a GoogleTest assertion if the test's name is too long. More...

void	gmx::test::initTestUtils (const std::filesystem::path &dataPath, const std::filesystem::path &tempPath, bool usesMpi, bool usesHardwareDetection, bool registersDynamically, int argc, char **argv)
	Initializes the test utilities library. More...

void	gmx::test::finalizeTestUtils (bool usesHardwareDetection, bool registersDynamically)
	Finalizes the test utilities library. More...

void	gmx::test::registerTestOptions (const char name, TestOptionsProvider provider)
	Registers a test option provider with the test framework. More...

void	gmx::test::initTestOptions (IOptionsContainer *options)
	Initializes the options from all registered test providers. More...

void	gmx::test::initMPIOutput ()
	Customizes test output and test failure handling for MPI runs. More...

ReferenceDataEntry::EntryPointer	gmx::test::readReferenceDataFile (const std::string &path)
	Loads reference data from an XML file. More...

void	gmx::test::writeReferenceDataFile (const std::string &path, const ReferenceDataEntry &root)
	Saves reference data to an XML file. More...

template<typename FloatType >
FloatType	anonymous_namespace{testasserts_tests.cpp}::addUlps (FloatType value, int ulps)
	Helper to produce floating-point numbers with specified ULP difference. More...

static FloatingPointTolerance	ulpTolerance (uint64_t ulpDiff)
	Creates a tolerance that only allows a specified ULP difference. More...

FloatingPointTolerance	relativeToleranceAsFloatingPoint (double magnitude, double tolerance)
	Creates a tolerance that allows a difference in two compared values that is relative to the given magnitude. More...

FloatingPointTolerance	relativeToleranceAsPrecisionDependentFloatingPoint (double magnitude, float singleTolerance, double doubleTolerance)
	Creates a tolerance that allows a precision-dependent difference in two compared values that is relative to the given magnitude. More...

static FloatingPointTolerance	relativeToleranceAsPrecisionDependentUlp (double magnitude, uint64_t singleUlpDiff, uint64_t doubleUlpDiff)
	Creates a tolerance that allows a precision-dependent relative difference in a complex computation. More...

static FloatingPointTolerance	absoluteTolerance (double tolerance)
	Creates a tolerance that allows a specified absolute difference.

static FloatingPointTolerance	relativeToleranceAsUlp (double magnitude, uint64_t ulpDiff)
	Creates a tolerance that allows a relative difference in a complex computation. More...

static FloatingPointTolerance	defaultRealTolerance ()
	Returns the default tolerance for comparing `real` numbers.

static FloatingPointTolerance	defaultFloatTolerance ()
	Returns the default tolerance for comparing single-precision numbers when compared by GROMACS built in either precision mode. More...

Directories
directory	testutils
	Testing Utilities (testutils)

directory	tests
	Unit tests for Testing Utilities (testutils).

Files
file	cmdlinetest.cpp
	Implements classes from cmdlinetest.h.

file	conftest.cpp
	Implements routine to check the content of conf files.

file	filematchers.cpp
	Implements classes from filematchers.h.

file	cmdlinetest.h
	Declares utilities testing command-line programs.

file	conftest.h
	Declares function to add the content of a conf file to a checker.

file	filematchers.h
	Declares utility classes for testing file contents.

file	interactivetest.h
	Provides helper classes for testing interactive prompts.

file	loggertest.h
	Declares gmx::test::LoggerTestHelper.

file	mpitest.h
	Helper functions for MPI tests to make thread-MPI look like real MPI.

file	refdata.h
	Functionality for writing tests that can produce their own reference data.

file	setenv.h
	Helper functions to have identical behavior of setenv and unsetenv on Unix and Windows systems.

file	simulationdatabase.h
	Functionality for testing whether calls to mdrun produce the same energy and force quantities when they should do so.

file	stdiohelper.h
	Declares gmx::test::StdioTestHelper.

file	stringtest.h
	Declares gmx::test::StringTestBase.

file	test_device.h
	Describes test environment class which performs GPU device enumeration for unit tests.

file	test_hardware_environment.h
	Describes test environment class which performs hardware enumeration for unit tests.

file	testasserts.h
	Extra assertions for unit tests.

file	testexceptions.h
	Exception classes for errors in tests.

file	testfilemanager.h
	Declares gmx::test::TestFileManager.

file	testfileredirector.h
	Declares generic mock implementations for interfaces in fileredirector.h.

file	testinit.h
	Functions for initialing GROMACS unit test executables.

file	testmatchers.h
	Extra GoogleMock matchers for unit tests.

file	testoptions.h
	Functions for accessing test command-line options.

file	textblockmatchers.h
	Declares utility classes for testing multi-line strings against reference data.

file	tprfilegenerator.h
	Helper for generating reusuable TPR files for tests within the same test binary.

file	xvgtest.h
	Declares function to add the content of an xvg file to a checker.

file	interactivetest.cpp
	Implements classes from interactivetest.h.

file	loggertest.cpp
	Implements gmx::test::LoggerTestHelper.

file	mpi_printer.h
	Declares gmx::test::initMPIOutput().

file	mpitest.cpp
	Implements functions in mpitest.h.

file	refdata.cpp
	Implements classes and functions from refdata.h.

file	refdata_checkers.h
	Declares internal helper classes for the reference data framework to check reference data values of different types.

file	refdata_impl.h
	Declares internal data structures for the reference data framework.

file	refdata_xml.cpp
	Implements reference data XML persistence.

file	refdata_xml.h
	Declares functions for reference data XML persistence.

file	simulationdatabase.cpp
	Implements declarations from in simulationdatabase.h.

file	stdiohelper.cpp
	Implements classes in stdiohelper.h.

file	stringtest.cpp
	Implements gmx::test::StringTestBase.

file	test_device.cpp
	Implements test environment class which performs hardware enumeration for unit tests.

file	test_hardware_environment.cpp
	Implements test environment class which performs hardware enumeration for unit tests.

file	testasserts.cpp
	Implements floating-point comparison routines from testasserts.h.

file	testfilemanager.cpp
	Implements gmx::test::TestFileManager.

file	testfileredirector.cpp
	Implements classes from testfileredirector.h.

file	testinit.cpp
	Implements functions in testinit.h.

file	testmatchers.cpp
	Implements floating-point matchers from testmatchers.h for use with Google Mock.

file	testoptions.cpp
	Implements functions in testoptions.h.

file	interactivetest.cpp
	Self-tests for interactive test helpers.

file	mpitest.cpp
	Tests for infrastructure for running tests under MPI.

file	refdata_tests.cpp
	Tests utilities for test reference data.

file	testasserts_tests.cpp
	Tests GROMACS-specific test assertions.

file	xvgtest_tests.cpp
	Tests utilities for testing xvg files.

file	testutils_doc.h
	Dummy header for Testing Utilities (testutils) documentation.

file	textblockmatchers.cpp
	Implements classes from textblockmatchers.h.

file	tprfilegenerator.cpp
	Implements helper for generating reusuable TPR files for tests within the same test binary.

file	unittest_main.cpp
	main() for unit tests that use Testing Utilities (testutils).

file	xvgtest.cpp
	Implements routine to check the content of xvg files.

Macro Definition Documentation

#define ASSERT_PLAIN ( expr )

Assert for predicates that return AssertionResult and produce a full failure message.

expr should evaluate to AssertionResult, and on failure the message from the result is used as-is, unlike in ASSERT_TRUE().

#define EXPECT_PLAIN ( expr )

Assert for predicates that return AssertionResult and produce a full failure message.

expr should evaluate to AssertionResult, and on failure the message from the result is used as-is, unlike in EXPECT_TRUE().

#define GMX_ASSERT_DEATH_IF_SUPPORTED	(	expr,
		msg
	)

Wrapper around ASSERT_DEATH_IF_SUPPORTED gtest macro for thread safe execution.

Makes sure that tests that are supposed to trigger an assertion are only executed in a threadsafe environment when running under e.g. MPI.

#define GMX_EXPECT_DEATH_IF_SUPPORTED	(	expr,
		msg
	)

Wrapper around EXPECT_DEATH_IF_SUPPORTED gtest macro for thread safe execution.

Makes sure that tests that are supposed to trigger an assertion are only executed in a threadsafe environment, and not when when running under e.g. MPI.

#define GMX_MPI_TEST ( RankRequirement )

Value:

const int numRanks = ::gmx::test::getNumberOfTestMpiRanks();                              \
    if (!RankRequirement::conditionSatisfied(numRanks))                                       \
    {                                                                                         \
        GTEST_SKIP() << std::string("Test skipped because ") + RankRequirement::s_skipReason; \
        return;                                                                               \
    }                                                                                         \
    GMX_MPI_TEST_INNER;

Declares that this test is an MPI-enabled unit test and expresses the conditions under which it can run.

To write unit tests that run under MPI, you need to do a few things:

Put GMX_MPI_TEST(RankRequirement) as the first statement in your test body and either declare or use a suitable class as its argument to express what requirements exist on the number of MPI ranks for this test.
Declare your unit test in CMake with gmx_add_mpi_unit_test(). Note that all tests in the binary should fulfill the conditions above.

When you do the above, the following will happen:

The test will get compiled only if thread-MPI or real MPI is enabled.
The test will get executed only when the specified condition on the the number of ranks is satisfied.
If you are using real MPI, the whole test binary is run under MPI and test execution across the processes is synchronized (GMX_MPI_TEST() actually has no effect in this case, the synchronization is handled at a higher level).
If you are using thread-MPI, GMX_MPI_TEST() is required and it initializes thread-MPI with the specified number of threads and runs the rest of the test on each of the threads.

Parameters

[in] RankRequirement Class that expresses the necessary conditions on the number of MPI ranks for the test to continue. If run with unsupported number of ranks, the remainder of the test body is skipped, and the GTEST_SKIP() mechanism used to report the reason why the number of MPI ranks is unsuitable.

The RankRequirement class must have two static members; a static method bool conditionSatisfied(const int) that can be passed the number of ranks present at run time and return whether the test can run with that number of ranks, and a static const string s_skipReason describing the reason why the test cannot be run, when that is the case.

You need to be extra careful for variables in the test fixture, if you use one: when run under thread-MPI, these will be shared across all the ranks, while under real MPI, these are naturally different for each process. Local variables in the test body are private to each rank in both cases.

Currently, it is not possible to require the use of a single MPI rank, because that will lead to problems with (at least) thread-MPI, but such tests can be written as serial tests anyway.

#define GMX_TEST_NO_THROW_	(	statement,
		fail
	)

Value:

GTEST_AMBIGUOUS_ELSE_BLOCKER_                                              \
    if (::testing::AssertionResult gmx_ar = ::testing::AssertionSuccess())     \
    {                                                                          \
        try                                                                    \
        {                                                                      \
            GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement);         \
        }                                                                      \
        catch (std::exception const& ex)                                       \
        {                                                                      \
            gmx_ar << "Expected: " #statement " doesn't throw an exception.\n" \
                   << "  Actual: it throws.\n"                                 \
                   << "Exception details:\n"                                   \
                   << ::gmx::formatExceptionMessageToString(ex);               \
            goto GTEST_CONCAT_TOKEN_(gmx_label_testnothrow_, __LINE__);        \
        }                                                                      \
        catch (...)                                                            \
        {                                                                      \
            gmx_ar << "Expected: " #statement " doesn't throw an exception.\n" \
                   << "  Actual: it throws.";                                  \
            goto GTEST_CONCAT_TOKEN_(gmx_label_testnothrow_, __LINE__);        \
        }                                                                      \
    }                                                                          \
    else                                                                       \
        GTEST_CONCAT_TOKEN_(gmx_label_testnothrow_, __LINE__) : fail(gmx_ar.message())

Internal implementation macro for exception assertations.

Parameters

statement	Statements to execute.
fail	Function/macro to call on failure.

The implementation is copied and adjusted from include/gtest/internal/gtest-internal.h in Google Test 1.6.0.

#define GMX_TEST_OPTIONS	(	name,
		options
	)

Macro to add additional command-line options for the test binary.

Parameters

name	Unique name for the set of options.
options	Placeholder name for an gmx::Options object for adding options.

The macro should be followed by a block that adds the desired command-line options to options using gmx::Options::addOption(). Extensible Handling of Options (options) provides an overview of the options machinery.

name must be unique within the executable to which the options are added. If the macro is within an unnamed namespace, then it is sufficient that it is unique within the file.

Typical usage:

#include "gromacs/options/basicoptions.h"
#include "gromacs/options/ioptionscontainer.h"
#include "testutils/testoptions.h"
namespace gmx
{
namespace
{
bool g_optionValue = false;
//! \cond
GMX_TEST_OPTIONS(MyTestOptions, options)
{
    options->addOption(BooleanOption("flag").store(&g_optionValue)
                           .description("My description"));
}
//! \endcond
} // namespace
} // namespace gmx

\cond and \endcond statements are necessary around the macro to avoid Doxygen warnings.

One macro invocation per an added option, with more of the implementation details hidden inside the macro, could be nicer. But that requires more elaborate macro machinery, so it is probably not worth the effort and complexity.

#define GMX_TEST_THROW_	(	statement,
		expected_exception,
		fail
	)

Internal implementation macro for exception assertations.

Parameters

statement	Statements to execute.
expected_exception	Exception type that `statement` should throw.
fail	Function/macro to call on failure.

The implementation is copied and adjusted from include/gtest/internal/gtest-internal.h in Google Test 1.6.0.

Enumeration Type Documentation

enum gmx::test::ReferenceDataMode : int

strong

Mode of operation for reference data handling.

There should be no need to use this type outside the test utility module.

Enumerator
Compare	Compare to existing reference data. If reference data does not exist, or if the test results differ from those in the reference data, the test fails.
CreateMissing	Create missing reference data. If reference data does not exist for a test, that test behaves as if ReferenceDataMode::UpdateAll had been specified. Tests for which reference data exists, behave like with ReferenceDataMode::Compare.
UpdateChanged	Update reference data that does not pass comparison. Tests utilizing reference data should always pass in this mode unless there is an I/O error.
UpdateAll	Update reference data, overwriting old data. Tests utilizing reference data should always pass in this mode unless there is an I/O error.
Count	Marks the end of the enum.

Function Documentation

template<typename FloatType >

FloatType anonymous_namespace{testasserts_tests.cpp}::addUlps	(	FloatType	value,
		int	ulps
	)

Helper to produce floating-point numbers with specified ULP difference.

This doesn't work if the value would change sign.

template<typename FloatType >

static inline ::testing::AssertionResult gmx::test::assertEqualWithinTolerance	(	const char *	expr1,
		const char *	expr2,
		const char *	,
		FloatType	value1,
		FloatType	value2,
		const FloatingPointTolerance &	tolerance
	)

static

Assertion predicate formatter for comparing two floating-point values.

void gmx::test::checkTestNameLength ( std::optional< std::string > testName = std::nullopt )

Gives a GoogleTest assertion if the test's name is too long.

On Windows, MAX_PATH is 260 characters by default. Particularly when using parameterized tests it is convenient to generate descriptive names. However the reference data files for these tests can have names that get too long. For example, by default Visual Studio will put files in {user-folder}\source\repos\gromacs and we need to leave room for that, plus the path to the user's folder, plus a build folder, then e.g. src\gromacs\gmxpreprocess\tests\refdata and then the test data filename. So we limit the name of the test to help keep things working.

When we identify tests whose names go close to this limit (e.g. because MSVC builds fail), then call this function from them to help maintainers keep that length under control when working on other platforms.

If a test computes its own name for the purposes of reference data files (e.g. PME unit tests), pass that name in here.

static FloatingPointTolerance defaultFloatTolerance ( )

This permits a checker compiled with any GROMACS precision to compare equal or not in the same way.

void gmx::test::finalizeTestUtils	(	bool	usesHardwareDetection,
		bool	registersDynamically
	)

Finalizes the test utilities library.

Parameters

[in]	usesHardwareDetection	If hardwaredetection is enabled.
[in]	registersDynamically	Whether dynamical GoogleTest registration is used by this test binary

Does not throw. Terminates the program with a non-zero error code if an error occurs.

This function is automatically called by unittest_main.cpp.

template<typename FloatType >

FloatingPoint<FloatType>::Bits gmx::test::anonymous_namespace{testasserts.cpp}::floatingPointToBiasedInteger ( const FloatingPoint< FloatType > & value )

Computes biased integer representation for a floating-point value.

This moves the integer representation from a sign-and-magnitude representation to a biased representation where the 0x8000... represents zero, and the order of the integer values matches the order of the floating-point values.

int gmx::test::getNumberOfTestMpiRanks ( )

Returns the number of MPI ranks to use for an MPI test.

For thread-MPI builds, this will return the requested number of ranks even before the thread-MPI threads have been started.

void gmx::test::initMPIOutput ( )

Customizes test output and test failure handling for MPI runs.

Only one rank should report the test result. Errors detected on a subset of ranks need to be reported individually, and as an overall failure.

On non-MPI builds, does nothing.

void gmx::test::initReferenceData ( IOptionsContainer * options )

Initializes reference data handling.

Adds command-line options to options to set the reference data mode. By default, ReferenceDataMode::Compare is used, but --ref-data create or --ref-data update can be used to change it.

This function is automatically called by initTestUtils().

void gmx::test::initTestOptions ( IOptionsContainer * options )

Initializes the options from all registered test providers.

Parameters

options The options are added here.

This is called automatically by initTestUtils().

void gmx::test::initTestUtils	(	const std::filesystem::path &	dataPath,
		const std::filesystem::path &	tempPath,
		bool	usesMpi,
		bool	usesHardwareDetection,
		bool	registersDynamically,
		int *	argc,
		char ***	argv
	)

Initializes the test utilities library.

Does not throw. Terminates the program with a non-zero error code if an error occurs.

This function is automatically called by unittest_main.cpp.

Parameters

[in]	dataPath	Filepath to input files.
[in]	tempPath	Filepath to temporary files.
[in]	usesMpi	If the test is run with MPI or not.
[in]	usesHardwareDetection	If hardwaredetection is enabled.
[in]	registersDynamically	Whether dynamical GoogleTest registration is used by this test binary
[in]	argc	Number of cmdline options
[in]	argv	Cmdline options.

static inline ::testing::AssertionResult gmx::test::plainAssertHelper	(	const char *	,
		const ::testing::AssertionResult &	expr
	)

static

Helper method for (EXPECT|ASSERT)_PLAIN.

ReferenceDataEntry::EntryPointer gmx::test::readReferenceDataFile ( const std::string & path )

Loads reference data from an XML file.

Parameters

[in] path Path to the file from which the data is loaded.

Returns: Root entry for the reference data parsed from the file.

Exceptions

TestException if there is a problem reading the file.

void gmx::test::registerTestOptions	(	const char *	name,
		TestOptionsProvider *	provider
	)

Registers a test option provider with the test framework.

Parameters

[in]	name	Name of the options provider (for ordering).
[in]	provider	The provider to register.

Exceptions

std::bad_alloc if out of memory.

Typically not used directly in test code, but through the GMX_TEST_OPTIONS macro.

This gets called from constructors for global variables, so ideally it would not throw to avoid unhandled exceptions. But since this is only test code, it is not worth the effort to try to remove those rare exceptions (mutex failures and out-of-memory from STL).

FloatingPointTolerance relativeToleranceAsFloatingPoint	(	double	magnitude,
		double	tolerance
	)

Parameters

[in]	magnitude	Magnitude of the numbers the computation operates in.
[in]	tolerance	Relative tolerance permitted (e.g. 1e-4).

In addition to setting an relative tolerance for both precisions, this sets the absolute tolerance such that values close to zero (in general, smaller than magnitude) do not fail the check if they differ by less than tolerance evaluated at magnitude. This accounts for potential loss of precision for small values, and should be used when accuracy of values much less than magnitude do not matter for correctness.

FloatingPointTolerance relativeToleranceAsPrecisionDependentFloatingPoint	(	double	magnitude,
		float	singleTolerance,
		double	doubleTolerance
	)

Parameters

[in]	magnitude	Magnitude of the numbers the computation operates in.
[in]	singleTolerance	Relative tolerance permitted (e.g. 1e-4) in single precision.
[in]	doubleTolerance	Relative tolerance permitted (e.g. 1e-4) in double precision.

In addition to setting an relative tolerance for both precisions, this sets the absolute tolerance such that values close to zero (in general, smaller than magnitude) do not fail the check if they differ by less than tolerance evaluated at magnitude. This accounts for potential loss of precision for small values, and should be used when accuracy of values much less than magnitude do not matter for correctness.

static FloatingPointTolerance relativeToleranceAsPrecisionDependentUlp	(	double	magnitude,
		uint64_t	singleUlpDiff,
		uint64_t	doubleUlpDiff
	)

Parameters

[in]	magnitude	Magnitude of the numbers the computation operates in.
[in]	singleUlpDiff	Expected accuracy of single-precision computation (in ULPs).
[in]	doubleUlpDiff	Expected accuracy of double-precision computation (in ULPs).

This works as relativeToleranceAsUlp(), but allows setting the ULP difference separately for the different precisions. This supports cases where the double-precision calculation can acceptably has a higher ULP difference, but relaxing the single-precision tolerance would lead to an unnecessarily loose test.

static FloatingPointTolerance relativeToleranceAsUlp	(	double	magnitude,
		uint64_t	ulpDiff
	)

Parameters

[in]	magnitude	Magnitude of the numbers the computation operates in.
[in]	ulpDiff	Expected accuracy of the computation (in ULPs).

In addition to setting the ULP tolerance as ulpTolerance(), this sets the absolute tolerance such that values close to zero (in general, smaller than magnitude) do not fail the check if they differ by less than ulpDiff evaluated at magnitude. This accounts for potential loss of precision for small values, and should be used when accuracy of values much less than magnitude do not matter for correctness.

static FloatingPointTolerance ulpTolerance ( uint64_t ulpDiff )

The tolerance uses the given ULP value for both precisions, i.e., double precision will have much stricter tolerance.

void gmx::test::writeReferenceDataFile	(	const std::string &	path,
		const ReferenceDataEntry &	root
	)

Saves reference data to an XML file.

Parameters

[in]	path	Path to the file to which the data is saved.
[in]	root	Root entry for the reference data to write.

Exceptions

TestException if there is a problem writing the file.

Description

Assertions for exceptions

Assertions for floating-point comparison

Helper functions for computing floating-point differences

Classes

Macros

Enumerations

Functions

Directories

Files

Macro Definition Documentation

Enumeration Type Documentation

Function Documentation