A useful test distinguishes correct behavior from a plausible wrong answer.
For yaeos, that means testing mathematical contracts, units, array shapes,
phase balances, and failure handling—not only checking that an iteration
finished or a value is finite.
This guide starts with a small native test, then adds the independent checks needed for thermodynamic and cross-language work. Use workflow for prerequisites and build commands.
The native tests are standalone Fortran programs under test/, one file per
program (test_<feature>.f90). The project no longer uses test-drive suites or
a collector program. Every test is an independent executable, so a failure can
be run and debugged on its own.
The directories under test/ mirror src/. A test goes in the directory
that corresponds to the source it exercises. For example, tests of
src/equilibria/flash.f90 are in test/equilibria/, and tests of the cubic
mixing rules in src/models/residual_helmholtz/cubic/mixing_rules/ are in
test/models/residual_helmholtz/cubic/mixing_rules/. A test that exercises
several files goes in the directory of the routine it is mainly about.
Shared helpers are in the local package test/support/, which fpm.toml
lists as a dev-dependency. It holds testing_aux (assert, test_title, in
test/support/src/yaeos_testing.f90) and the model fixtures. It is a separate
package because fpm only lets a test program use modules from its own
directory or below. A helper module used by tests in a single directory can
stay next to them.
auto-tests is off: every test program has a [[test]] entry in fpm.toml,
generated by tools/dev/test_to_toml.py. Each test is named after its file
without the test_ prefix (test_flash.f90 is the target flash), so file
names must be unique across all directories. CI runs the script with
--check and fails if fpm.toml is out of date.
Python tests are under python/tests/. Some compare directly with native
Fortran calls by generating small programs in app/. Those tests need a
checkout, fpm, and a working native build as well as the Python package.
Save this complete program as test_my_feature.f90, in the test/
subdirectory that mirrors the source it tests (here a residual-model
calculation, so test/models/residual_helmholtz/), in a working development
branch. It follows the program-based pattern used by
test/models/residual_helmholtz/cubic/mixing_rules/test_huron_vidal.f90,
test/models/residual_helmholtz/cubic/mixing_rules/test_sddlc.f90, and
test/equilibria/test_flash_optional_iters.f90:
program test_my_feature
use yaeos, only: pr, ArModel, PengRobinson76
use testing_aux, only: assert, test_title
use, intrinsic :: ieee_arithmetic, only: ieee_is_finite
implicit none
class(ArModel), allocatable :: model
real(pr) :: n(2)
real(pr) :: specified_pressure
real(pr) :: calculated_pressure
real(pr) :: volume
real(pr) :: temperature
write(*, *) test_title("Volume/pressure regression")
model = PengRobinson76([190.564_pr, 305.32_pr], &
[45.99_pr, 48.72_pr], [0.01142_pr, 0.0995_pr])
n = [0.3_pr, 0.7_pr]
specified_pressure = 10.0_pr
temperature = 320.0_pr
call model%volume(n, specified_pressure, temperature, volume, &
root_type="stable")
call assert(ieee_is_finite(volume), "Volume must be finite")
call assert(volume > 0.0_pr, "Volume must be positive")
call model%pressure(n, volume, temperature, calculated_pressure)
call assert(ieee_is_finite(calculated_pressure), "Pressure must be finite")
call assert(abs(calculated_pressure - specified_pressure) < 1.0e-7_pr, &
"Volume reproduces the specified pressure")
end program test_my_feature
testing_aux is the repository’s test helper module, defined in
test/support/src/yaeos_testing.f90, not part of the installed public yaeos
API.
test_title formats a heading; assert(condition, message) reports a passing
check or terminates the program with error stop on failure. A failing
prerequisite therefore stops before later calculations use an invalid result.
Printing a warning without a nonzero exit status is not a failing test.
For a group of related checks, put private test subroutines after contains
and call them explicitly from the program body. There is no collector or
suite registry to update.
From the repository root, after saving the file in the directory that mirrors the source it tests:
python tools/dev/test_to_toml.py # adds the [[test]] entry to fpm.toml
fpm test --list
fpm test my_feature
The target name is the filename without test_, so use a descriptive and
unique test_*.f90 name. The script only registers files that contain a
program unit and its end program, and it refuses duplicate names. Commit
the fpm.toml change with the test. Run the focused target first, then
fpm test before review.
| Check | What it establishes | What it does not establish |
|---|---|---|
| Compare with independent data | Agreement for specified states and parameters | Accuracy everywhere |
| Numerical derivative check | Local agreement of derivatives and energy evaluation | That the energy equation is the intended physical model |
| Thermodynamic identity | Internal consistency of related quantities | Independence of shared implementation errors |
| Material balance and isofugacity | A candidate phase split satisfies its equations | Global stability against all other splits |
| Native/Python comparison | Agreement across wrapper paths | Independent verification of their shared native kernel |
| Regression reproducer | A particular failure remains fixed | Coverage of unrelated edge cases |
Use more than one kind of evidence for a new model or solver. Two routines that share the same incorrect derivative can agree perfectly.
For a scalar property , a central-difference estimate is
Choose steps with both an absolute and relative scale so that a zero input still gets a nonzero perturbation. Repeat at several step sizes. Small steps amplify cancellation; large steps introduce truncation error. Neither choosing the smallest possible number nor copying another test’s step blindly is a reliable validation strategy.
Test the amount derivatives required by ArModel and GeModel, not
derivatives of normalized mole fractions. For an equilibrium Jacobian in log
coordinates, perturb the packed log variable and include its chain-rule
factor. Avoid changing volume-root branches during a derivative check intended
for a single smooth branch.
Useful native helpers include numeric_ar_derivatives, ar_consistency,
numeric_ge_derivatives, ge_consistency, and individual_ar_calls under
src/consistency/. Read their interfaces and the cases that call them:
different outputs have different scalar, vector, and matrix shapes.
individual_ar_calls helps check optional-output combinations; it does not
replace a mathematical accuracy check.
For a derivative implementation, cover energy only, each derivative separately, all outputs together, mixed-output requests, and repeated calls at different states. AD implementations need these checks too: differentiation propagates the program’s operations, including mistakes in its primal expression.
For residual Helmholtz energy at fixed temperature,
Pressure and fugacity coefficients should stay unchanged under that same joint scaling. Scaling amounts without volume changes the density, so it is not the same test.
For an excess-Gibbs model, , whereas activity coefficients are unchanged. These checks help catch a missing total-amount factor or an inappropriate normalization.
At fixed component amounts, define and . The total heat capacities satisfy . Subtracting the ideal-gas difference gives
Here and . First derivatives of residual Helmholtz energy cannot simply be multiplied to obtain this identity.
This insertable test fragment assumes an initialized model and declared
n, V, and T; import pr and R from yaeos and assert from
testing_aux. Place the declarations before executable statements.
Choose a regular state where dPdV is nonzero:
real(pr) :: P
real(pr) :: dPdT
real(pr) :: dPdV
real(pr) :: Cp
real(pr) :: Cv
real(pr) :: expected_difference
call model%pressure(n, V, T, P, dPdT=dPdT, dPdV=dPdV)
call model%Cp_residual_vt(n, V, T, Cp=Cp)
call model%Cv_residual_vt(n, V, T, Cv=Cv)
expected_difference = -T * dPdT**2 / dPdV - sum(n) * R
call assert(abs(Cp - Cv - expected_difference) < 1.0e-8_pr, &
"Residual heat capacities satisfy the pressure-derivative identity")
This tests consistency of property relations and derivative paths. An independent derivative check is still needed to diagnose a shared model error. Near singular states, conditioning and cancellation can make this a poor choice of numerical test even when the identity is valid.
For a smooth excess-Gibbs model, the Hessian of is symmetric and
. ge_consistency checks
these relations and the link between energy and activity coefficients. The
matrix of first mole derivatives of is a Jacobian, not a
Hessian of .
Check dilute-gas limits where applicable: , , and residual energy tends to zero as density tends to zero at fixed amount and temperature. For activity models using pure-component reference states, check appropriate pure limits and parameter conventions. Use a sufficiently dilute finite state, not a singular zero-volume or zero-amount calculation.
For an accepted two-phase result, check:
z = (1-beta)*x + beta*y.For multiphase output, reconstruct the feed using all phase fractions,
including the reference phase. MPEquilibriumState%np counts main phases in
the current representation; read the corresponding routine’s contract rather
than assuming it is the total count.
Test ordinary cases, relevant boundaries, and a failure path. Keep near-critical conditioning tests distinct from ordinary accuracy tests. A converged root can be metastable, and a finite search that finds no destabilizing trial phase is not a proof of global stability.
Failure conventions differ. Some routines return a sentinel, some return an
early result or status field, and some terminate with error stop. Test the
specific contract; do not read uninitialized result arrays after a failed
call. To test an expected terminating failure, use a subprocess and check
its exit status from a harness. Do not add a program that always exits nonzero
as an ordinary passing fpm test target: fpm correctly treats that exit as
a test failure.
Python tests under python/tests/ include native-reference comparisons. Some
generate Fortran programs in app/, execute them with fpm, and parse their
output with helpers such as parse_vec and parse_mat. Inspect an existing
test for the exact protocol instead of inventing another incompatible one.
For an interface change, check:
A Python test that agrees with native Fortran validates the adaptation layer, not the physical model independently. Test an installed build after changes to the compiled wrapper; a stale extension can conceal or mimic a regression.
A practical floating-point comparison uses both scales:
Choose atol in the quantity’s units and rtol for its expected scale.
Near-zero consistency residuals need an absolute criterion. Properties spanning
orders of magnitude often need a relative criterion too. Do not compare
log-fugacity mismatch, temperature error, and material balance with a single
unexplained tolerance.
State the reason for a reference value and its tolerance in the test. Consider rounding in published parameters, conditioning, compiler differences, and finite-difference error. Do not loosen a tolerance until a failure disappears without understanding why.
CI runs on both an older and the newest Ubuntu image. The newest images use
x86-64-v3 (AVX2/FMA) builds of gfortran and BLAS/LAPACK, and fused
multiply-add changes the last bits of results. A limit of a few units in the
last place, such as 1e-15 on a thermodynamic identity, can then fail on one
machine and pass on another. Two measured examples set the scale:
1e-4) agree to about 1e-6 relative in every build;1e-15.The cubic EoS tests therefore use 1e-5 and 1e-13. Those limits sit well
above the rounding noise, and real derivative bugs give errors of order one.
To check a test against FMA rounding locally:
fpm test --profile release --flag "-g -ffree-line-length-0 -march=x86-64-v3" my_feature
Adding -ffp-contract=off to the same command disables FMA contraction. If
the failure then disappears, rounding is the cause and the tolerance is too
tight.
Run the affected target first, then the native suite. Rebuild the Python extension and run relevant Python tests for an interface change. Build the documentation and run or compile its complete examples separately: successful site generation does not execute the tutorial.
In the review summary, report which layers were tested, the revision and environment, and anything unavailable. Coverage is useful for finding unvisited branches, but it cannot establish that the equations, reference data, or assertions are correct.
CI measures line coverage from the release-profile test run and uploads it to
Codecov. The project target is 95%, and generated code such as the Tapenade
PC-SAFT derivatives counts toward it. To reproduce the measurement locally
(it needs gcovr):
fpm test --profile release --flag "-g -coverage -ffree-line-length-0"
bash ci/ci.sh coverage
ci/ci.sh coverage applies the same exclusions as CI: tests, examples,
app/, tools/, the legacy API and the vendored hyperdual module. Use
gcovr --html-details with those exclusions to see the uncovered lines. A new
test should check a result, not just execute lines.
Continue with workflow, numerical implementation, or the end-to-end change walkthrough.