Gromacs
5.1.1
|
Common helper classes and functions for writing tests using Google Test.
General documentation for the testing in GROMACS can be found in the developer guide. This page provides an overview of the actual API provided by the testutils
module.
There are several distinct functionalities provided:
stdin
reading etc. cannot be easily mocked.Additionally, testinit.h and mpi-printer.h, and their corresponding source files, provide functionality that is not visible on the API level: they provide initialization routines for the above functionality, which are automatically called by the main() function provided in unittest_main.cpp.
mpi-printer.h provides a Google Test listener that is installed when the tests are compiled with MPI. This listener allows the test binary to be run on multiple MPI ranks, and synchronizes the execution and output from the test cases, as well as makes the test fail on even if an assertion fails only on one rank.
Assertions for exceptions | |
These macros replace | |
#define | EXPECT_THROW_GMX(statement, expected_exception) |
Asserts that a statement throws a given exception. | |
#define | EXPECT_NO_THROW_GMX(statement) |
Asserts that a statement does not throw. | |
#define | ASSERT_THROW_GMX(statement, expected_exception) |
Asserts that a statement throws a given exception. | |
#define | ASSERT_NO_THROW_GMX(statement) |
Asserts that a statement does not throw. | |
Assertions for floating-point comparison | |
These routines extend See gmx::test::FloatingPointTolerance for the possible ways to specify the tolerance, and gmx::test::FloatingPointDifference for some additional details of the difference calculation. | |
#define | EXPECT_FLOAT_EQ_TOL(value1, value2, tolerance) |
Asserts that two single-precision values are within the given tolerance. | |
#define | EXPECT_DOUBLE_EQ_TOL(value1, value2, tolerance) |
Asserts that two double-precision values are within the given tolerance. | |
#define | EXPECT_REAL_EQ_TOL(value1, value2, tolerance) |
Asserts that two real values are within the given tolerance. | |
#define | ASSERT_FLOAT_EQ_TOL(value1, value2, tolerance) |
Asserts that two single-precision values are within the given tolerance. | |
#define | ASSERT_DOUBLE_EQ_TOL(value1, value2, tolerance) |
Asserts that two double-precision values are within the given tolerance. | |
#define | ASSERT_REAL_EQ_TOL(value1, value2, tolerance) |
Asserts that two real values are within the given tolerance. | |
Assertions for NULL comparison | |
These macros should be used instead of | |
#define | EXPECT_NULL(val) EXPECT_EQ((void *) NULL, val) |
Asserts that a pointer is null. More... | |
#define | ASSERT_NULL(val) ASSERT_EQ((void *) NULL, val) |
Asserts that a pointer is null. More... | |
Classes | |
class | gmx::test::CommandLine |
Helper class for tests that need to construct command lines. More... | |
class | gmx::test::CommandLineTestHelper |
Helper class for tests that construct command lines that need to reference existing files. More... | |
class | gmx::test::CommandLineTestBase |
Test fixture for tests that call a single command-line program with input/output files. More... | |
class | gmx::test::IntegrationTestFixture |
Test fixture for integration tests. More... | |
class | gmx::test::TestReferenceData |
Handles creation of and comparison to test reference data. More... | |
class | gmx::test::TestReferenceChecker |
Handles comparison to test reference data. More... | |
class | gmx::test::StringTestBase |
Test fixture for tests that check string formatting. More... | |
class | gmx::test::FloatingPointDifference |
Computes and represents a floating-point difference value. More... | |
class | gmx::test::FloatingPointTolerance |
Specifies a floating-point comparison tolerance and checks whether a difference is within the tolerance. More... | |
class | gmx::test::TestException |
Exception class for reporting errors in tests. More... | |
class | gmx::test::TestFileManager |
Helper for tests that need input and output files. More... | |
class | gmx::test::TestFileInputRedirector |
In-memory implementation for FileInputRedirectorInterface for tests. More... | |
class | gmx::test::TestOptionsProvider |
Provides additional options for the test executable. More... | |
Macros | |
#define | GMX_TEST_OPTIONS(name, options) |
Macro to add additional command-line options for the test binary. More... | |
Enumerations | |
enum | gmx::test::ReferenceDataMode { gmx::test::erefdataCompare, gmx::test::erefdataCreateMissing, gmx::test::erefdataUpdateAll } |
Mode of operation for reference data handling. More... | |
Functions | |
void | gmx::test::initReferenceData (Options *options) |
Initializes reference data handling. More... | |
gmx::test::FloatingPointDifference::FloatingPointDifference (float value1, float value2) | |
Initializes a single-precision difference. | |
gmx::test::FloatingPointDifference::FloatingPointDifference (double value1, double value2) | |
Initializes a double-precision difference. | |
bool | gmx::test::FloatingPointDifference::isNaN () const |
Whether one or both of the compared values were NaN. More... | |
double | gmx::test::FloatingPointDifference::asAbsolute () const |
Returns the difference as an absolute number (always non-negative). | |
gmx_uint64_t | gmx::test::FloatingPointDifference::asUlps () const |
Returns the difference as ULPs (always non-negative). More... | |
bool | gmx::test::FloatingPointDifference::signsDiffer () const |
Whether the compared values were of different sign. More... | |
bool | gmx::test::FloatingPointDifference::isDouble () const |
Whether the difference is between single- or double-precision numbers. | |
std::string | gmx::test::FloatingPointDifference::toString () const |
Formats the difference as a string for assertion failure messages. | |
gmx::test::FloatingPointTolerance::FloatingPointTolerance (float singleAbsoluteTolerance, double doubleAbsoluteTolerance, gmx_uint64_t singleUlpTolerance, gmx_uint64_t doubleUlpTolerance, bool bSignMustMatch) | |
Creates a tolerance with the specified values. More... | |
bool | gmx::test::FloatingPointTolerance::isWithin (const FloatingPointDifference &difference) const |
Checks whether a difference is within the specified tolerance. More... | |
std::string | gmx::test::FloatingPointTolerance::toString (const FloatingPointDifference &difference) const |
Formats the tolerance as a string for assertion failure messages. | |
void | gmx::test::registerTestOptions (const char *name, TestOptionsProvider *provider) |
Registers a test option provider with the test framework. More... | |
void | gmx::test::initTestOptions (Options *options) |
Initializes the options from all registered test providers. More... | |
static FloatingPointTolerance | ulpTolerance (gmx_uint64_t ulpDiff) |
Creates a tolerance that only allows a specified ULP difference. More... | |
FloatingPointTolerance | relativeToleranceAsFloatingPoint (double magnitude, double tolerance) |
Creates a tolerance that allows a difference in two compared values that is relative to the given magnitude. More... | |
static FloatingPointTolerance | relativeToleranceAsPrecisionDependentUlp (double magnitude, gmx_uint64_t singleUlpDiff, gmx_uint64_t doubleUlpDiff) |
Creates a tolerance that allows a precision-dependent relative difference in a complex computation. More... | |
static FloatingPointTolerance | absoluteTolerance (double tolerance) |
Creates a tolerance that allows a specified absolute difference. | |
static FloatingPointTolerance | relativeToleranceAsUlp (double magnitude, gmx_uint64_t ulpDiff) |
Creates a tolerance that allows a relative difference in a complex computation. More... | |
static FloatingPointTolerance | defaultRealTolerance () |
Returns the default tolerance for comparing real numbers. | |
Directories | |
directory | testutils |
Testing Utilities (testutils) | |
directory | tests |
Unit tests for Testing Utilities (testutils). | |
Files | |
file | cmdlinetest.h |
Declares utilities testing command-line programs. | |
file | integrationtests.h |
Declares test fixture for integration tests. | |
file | refdata.h |
Functionality for writing tests that can produce their own reference data. | |
file | stringtest.h |
Declares gmx::test::StringTestBase. | |
file | testasserts.h |
Extra assertions for unit tests. | |
file | testexceptions.h |
Exception classes for errors in tests. | |
file | testfilemanager.h |
Declares gmx::test::TestFileManager. | |
file | testfileredirector.h |
Declares generic mock implementations for interfaces in fileredirector.h. | |
file | testoptions.h |
Functions for accessing test command-line options. | |
#define ASSERT_NULL | ( | val | ) | ASSERT_EQ((void *) NULL, val) |
Asserts that a pointer is null.
Works exactly like ASSERT_EQ comparing with a null pointer.
#define EXPECT_NULL | ( | val | ) | EXPECT_EQ((void *) NULL, val) |
Asserts that a pointer is null.
Works exactly like EXPECT_EQ comparing with a null pointer.
#define GMX_TEST_OPTIONS | ( | name, | |
options | |||
) |
Macro to add additional command-line options for the test binary.
name | Unique name for the set of options. |
options | Placeholder name for an gmx::Options object for adding options. |
The macro should be followed by a block that adds the desired command-line options to options
using gmx::Options::addOption(). Extensible Handling of Options (options) provides an overview of the options machinery.
name
must be unique within the executable to which the options are added. If the macro is within an unnamed namespace, then it is sufficient that it is unique within the file.
Typical usage:
\cond
and \endcond
statements are necessary around the macro to avoid Doxygen warnings.
One macro invocation per an added option, with more of the implementation details hidden inside the macro, could be nicer. But that requires more elaborate macro machinery, so it is probably not worth the effort and complexity.
Mode of operation for reference data handling.
There should be no need to use this type outside the test utility module.
Enumerator | |
---|---|
erefdataCompare |
Compare to existing reference data. If reference data does not exist, or if the test results differ from those in the reference data, the test fails. |
erefdataCreateMissing |
Create missing reference data. If reference data does not exist for a test, that test behaves as if erefdataUpdateAll had been specified. Tests for which reference data exists, behave like with erefdataCompare. |
erefdataUpdateAll |
Update reference data, overwriting old data. Tests utilizing reference data should always pass in this mode unless there is an I/O error. |
|
inline |
Returns the difference as ULPs (always non-negative).
The ULPs are calculated for the type that corresponds to the constructor used to initialize the difference. The ULP difference between 0.0 and -0.0 is zero.
|
inline |
Creates a tolerance with the specified values.
[in] | singleAbsoluteTolerance | Allowed absolute difference in a single-precision number. |
[in] | doubleAbsoluteTolerance | Allowed absolute difference in a double-precision number. |
[in] | singleUlpTolerance | Allowed ULP difference in a single-precision number. |
[in] | doubleUlpTolerance | Allowed ULP difference in a double-precision number. |
[in] | bSignMustMatch | Whether sign mismatch fails the comparison. |
void gmx::test::initReferenceData | ( | Options * | options | ) |
Initializes reference data handling.
Adds command-line options to options
to set the reference data mode. By default, erefdataCompare is used, but --ref-data create
or --ref-data update
can be used to change it.
This function is automatically called by initTestUtils().
void gmx::test::initTestOptions | ( | Options * | options | ) |
Initializes the options from all registered test providers.
options | The options are added here. |
This is called automatically by initTestUtils().
bool gmx::test::FloatingPointDifference::isNaN | ( | ) | const |
Whether one or both of the compared values were NaN.
If this returns true
, other accessors return meaningless values.
bool gmx::test::FloatingPointTolerance::isWithin | ( | const FloatingPointDifference & | difference | ) | const |
Checks whether a difference is within the specified tolerance.
NaNs are always treated outside the tolerance.
void gmx::test::registerTestOptions | ( | const char * | name, |
TestOptionsProvider * | provider | ||
) |
Registers a test option provider with the test framework.
[in] | name | Name of the options provider (for ordering). |
[in] | provider | The provider to register. |
std::bad_alloc | if out of memory. |
tMPI::system_error | on mutex failures. |
Typically not used directly in test code, but through the GMX_TEST_OPTIONS macro.
This gets called from constructors for global variables, so ideally it would not throw to avoid unhandled exceptions. But since this is only test code, it is not worth the effort to try to remove those rare exceptions (mutex failures and out-of-memory from STL).
|
related |
Creates a tolerance that allows a difference in two compared values that is relative to the given magnitude.
[in] | magnitude | Magnitude of the numbers the computation operates in. |
[in] | tolerance | Relative tolerance permitted (e.g. 1e-4). |
In addition to setting an ULP tolerance equivalent to tolerance
for both precisions, this sets the absolute tolerance such that values close to zero (in general, smaller than magnitude
) do not fail the check if they differ by less than tolerance
evaluated at magnitude
. This accounts for potential loss of precision for small values, and should be used when accuracy of values much less than magnitude
do not matter for correctness.
The ULP tolerance for different precisions will be different to make them both match tolerance
.
|
related |
Creates a tolerance that allows a precision-dependent relative difference in a complex computation.
[in] | magnitude | Magnitude of the numbers the computation operates in. |
[in] | singleUlpDiff | Expected accuracy of single-precision computation (in ULPs). |
[in] | doubleUlpDiff | Expected accuracy of double-precision computation (in ULPs). |
This works as relativeToleranceAsUlp(), but allows setting the ULP difference separately for the different precisions. This supports cases where the double-precision calculation can acceptably has a higher ULP difference, but relaxing the single-precision tolerance would lead to an unnecessarily loose test.
|
related |
Creates a tolerance that allows a relative difference in a complex computation.
[in] | magnitude | Magnitude of the numbers the computation operates in. |
[in] | ulpDiff | Expected accuracy of the computation (in ULPs). |
In addition to setting the ULP tolerance as ulpTolerance(), this sets the absolute tolerance such that values close to zero (in general, smaller than magnitude
) do not fail the check if they differ by less than ulpDiff
evaluated at magnitude
. This accounts for potential loss of precision for small values, and should be used when accuracy of values much less than magnitude
do not matter for correctness.
|
inline |
Whether the compared values were of different sign.
0.0 and -0.0 are treated as positive and negative, respectively.
|
related |
Creates a tolerance that only allows a specified ULP difference.
The tolerance uses the given ULP value for both precisions, i.e., double precision will have much stricter tolerance.