Developer workflow

From a checkout to a reviewed change

This page explains how to build the project you are editing, rather than accidentally testing a different installed version. Read the architecture map first if you are unsure which layer a change belongs to.

There are three distinct outcomes to check: the native Fortran package, the Python package with its compiled extension, and the generated documentation. A successful build of one does not prove the others work.

1. Start from an identifiable revision

For a fresh checkout:

git clone https://github.com/ipqa-research/yaeos.git
cd yaeos
git switch -c feature/my-change
git status --short

For a fork-based contribution, use your fork’s clone URL and add the project repository as an upstream remote. Agree on the target branch before starting a larger change. The commands above create a new development branch; they do not reset or discard existing work.

Before editing an existing checkout, inspect git status and understand its modified and untracked files. A dirty tree can be perfectly ordinary work in progress. Do not clean, overwrite, or revert it merely to make a command run.

Record git rev-parse HEAD, compiler and fpm versions when diagnosing a numerical regression. A package version string alone may not identify the source of an uncommitted development checkout.

2. Understand the build boundaries

Outcome Entry point What gets produced
Native library, apps, examples, and tests Root fpm.toml Compiler-specific files under build/
Python package python/pyproject.toml and python/meson.build A compiled extension and installed Python modules
Fortran API and guide pages Root ford.md doc/ford_site/
Python API and tutorials python/docs/source/conf.py A Sphinx HTML tree in your selected output directory

Meson invokes fpm install internally to build a position-independent native library for the Python extension. That is a separate build from an ordinary root-level fpm build. Do not copy a random .mod file from one compiler or profile into another build: module files are compiler-specific artifacts.

Keep generated outputs out of commits. Check .gitignore for the current rules rather than assuming every file generated by a new tool is ignored.

3. Install the tools for the layer you need

Native Fortran work

You need a Fortran compiler, a C compiler for the Tapenade runtime support, fpm, Git for dependencies, and BLAS/LAPACK libraries. The root manifest links lapack and blas. On Debian/Ubuntu a typical system-library setup is:

sudo apt install gfortran gcc git libblas-dev liblapack-dev

Install fpm using its supported installation instructions, then check:

fpm --version
gfortran --version
gcc --version

The first build needs network access to fetch the Git dependencies declared in fpm.toml. Later builds normally reuse their downloaded copies under build/. Changing a dependency revision, compiler, profile, or compiler flags can require a different build or a rebuild. Diagnose fetch failures separately from compilation failures.

Python work

In addition to the native tools, the Python build uses meson-python, Meson, Ninja, NumPy/f2py, and a supported Python interpreter. Build isolation installs the requirements declared in python/pyproject.toml but does not install a system compiler or BLAS/LAPACK for you.

Check the actual compiler requirement. The current non-Windows branch of python/meson.build rejects a Fortran compiler version below 14.1 and uses GNU-specific flags, including heap trampoline support. This requirement belongs to this Python build, not to a generic Fortran language requirement or an equivalent minimum enforced by root fpm.toml. The Windows branch has different flags and checks; do not infer platform support solely from the absence of that version gate.

Documentation work

FORD needs a compatible Fortran preprocessor and Graphviz for the configured dependency graphs. Sphinx uses python/docs/requirements.txt; notebook conversion also needs Pandoc. Use a virtual environment for documentation packages instead of changing unrelated system Python installations.

4. Build the native package first

Run from the repository root:

fpm build
fpm run --example basics
fpm test mixing_base
fpm test

basics is a registered example in fpm.toml. mixing_base is the test program test/models/residual_helmholtz/cubic/mixing_rules/test_mixing_base.f90, registered in fpm.toml by tools/dev/test_to_toml.py. Every native test is a standalone program like this one, in a test/ subdirectory that mirrors src/, and the full fpm test runs all of them. Discover your checkout’s targets with fpm test --list and consult fpm test --help for your installed version.

Run build commands sequentially when they share the same build directory. Two concurrent fpm processes can overwrite intermediate files or make a failure harder to reproduce.

Debugging and performance builds are different checks

A debug build is useful for finding invalid array access or uninitialized state; a release build is useful for checking optimized behavior and timing:

fpm test --profile debug
fpm test --profile release

For additional compiler diagnostics, select flags supported by that compiler and document what you used. For example, a GNU debugging investigation might use --flag "-g -fcheck=all -fbacktrace". Bounds checks can expose existing defects as well as your new change. Do not turn them off merely to make a regression disappear.

Some existing test files contain long lines; GNU’s -ffree-line-length-none can be useful when reproducing a build with explicit flags. Do not hard-code this compiler-specific flag as if it applied to every toolchain.

A smaller test is a faster feedback loop

Start with the affected target, then run the larger suite before review. When changing a model kernel, test derivatives and a small property call before running an expensive phase envelope. A failing one-point correction is easier to inspect than the last point of a long curve.

The testing guide shows how to write a standalone test program with the existing assertion helper and how to choose numerical assertions.

5. Build the Python package separately

From the repository root, in a Python environment intended for development:

python -m pip install ./python
python -c "import yaeos; print(yaeos.__file__, yaeos.__version__)"

The printed path helps catch importing an old wheel or another environment. After changing compiled Fortran or wrapper signatures, rebuild/reinstall; a Python interpreter cannot pick up a newly changed shared library reliably just because its source file changed. Restart the interpreter after rebuilding.

The current Meson build locates the interface source relative to python/ and invokes fpm without an explicit project-directory argument. Its working directory and fpm’s upward manifest search therefore matter. Prefer the standard package build layout; do not assume an arbitrary external Meson build directory will work unchanged. See Python internals for the full pipeline.

From the python/ directory, run:

python -m pytest tests/ --import-mode=importlib

Some Python tests generate Fortran programs under root app/ and invoke fpm to compare the two language paths. They need a working checkout and native tools, not just an installed Python wheel. Do not run them concurrently against the same generated filenames. python/tox.ini documents the project’s test, style, coverage, and documentation environments; tox runs from python/.

Importing the package directly from source can also require installed package metadata: yaeos.__version__ uses importlib.metadata.version("yaeos"). Adding python/ to PYTHONPATH alone is not a substitute for an installation.

6. Build both documentation systems

From the repository root, FORD builds the native API and the Markdown guides:

ford ford.md

Open doc/ford_site/index.html. Review your rendered page, its equations, tables, code, images, and links. A successful process exit does not prove that a formula is correct or an example is runnable.

From the repository root, an explicit Sphinx output path avoids directory ambiguity:

sphinx-build -b html python/docs/source python/docs/build

Use -W for a warning-as-error check when the environment can import the documented package and all required extensions. Explain any existing warnings separately from warnings introduced by your changes. The Sphinx configuration disables notebook execution, so building the site does not validate notebook numerical outputs.

The documentation deployment workflow builds both systems and replaces FORD’s page/python-api/ output with the Sphinx HTML tree. Consequently a local FORD-only build shows the Python fallback page, not the complete Python API. Preserve that replacement path and test links in the assembled site when changing navigation.

Do not use a path to a personal compiler cache or temporary environment in a published example. Describe reproducible package-level commands, and identify any environment-specific command used for local debugging as such.

7. Diagnose failures in the right layer

Symptom First investigation
Dependency clone fails Network, Git URL/revision, and dependency cache
BLAS/LAPACK symbols are unresolved Installed libraries, link names, and search paths
A native call succeeds but Python fails Rebuilt extension, handle lookup, shapes, and option conversion
Python imports the wrong implementation Interpreter/environment and printed yaeos.__file__
Wrapper compiles but values are transposed Array ordering and native shape conventions
A regression occurs only in release mode Uninitialized values, aliasing, bounds, and undefined behavior
A Sphinx page omits an object Import diagnostics, installed metadata, and extension availability
FORD succeeds but a page is missing Front matter, ordered_subpage, filename, and page-relative links

If you need a clean build, inspect what is generated and what is user work before removing anything. fpm clean --help describes the installed version’s cleaning options. Do not put a broad deletion command into a debugging recipe without explaining exactly which artifacts it removes.

8. Prepare the change for review

Run git diff --check and inspect the diff yourself. Include implementation, tests, and documentation for one coherent change. If a failure predated your work, retain the evidence rather than silently changing an unrelated algorithm.

In the review description, include the affected layers, a minimal reproducer, the expected physical/numerical behavior, and the exact checks run. Distinguish native tests, Python tests, and documentation builds. A check that could not run is a limitation to report, not an implied success.

Read the actual .github/workflows/CI.yml, .github/workflows/docs.yml, and ci/ci.sh when comparing with CI. Workflow names, exclusions, or a commented command are not proof of test coverage. For example, current Python wheel testing and the nominal Python-API job do not execute identical commands.

Continue with writing tests, the C-facing boundary, or an end-to-end contribution.