3.4 KiB
Legendre transform tests
The harness compares the pinned upstream source with the current exoplasim/plasim/src/legmod.f90. The exoplasim-upstream-import Git tag identifies the imported upstream commit and ships with the project repository. No network fetch is needed.
Run from the repository root:
timeout 180 .venv/bin/python scripts/benchmark-transforms.py
timeout 180 .venv/bin/python scripts/benchmark-transforms.py --precision 4
The default precision is FP64. The harness tests T21, T42, and T85 with ten levels. It tests full and half-latitude partitions with checked and native compiler settings. Each executable covers deterministic nonzero coefficients, zero inputs, and nontrivial filters with a different planetary-vorticity value.
A small pumamod stub supplies the dimensions and state required by the complete Legendre module. This isolates the kernels from climate-model startup. Half-latitude tests exercise the distributed transform branch, not MPI communication.
The driver verifies that dv2uv restores its input vorticity array. It writes complete transform outputs and restored inputs as raw numeric arrays. The Python script compares shapes, finite values, bit patterns, maximum absolute error, and scaled error. Scaled error divides by the larger of one and the maximum absolute reference value. The acceptance threshold is 1e-12 for that scale.
Final acceptance matched upstream bit for bit in all 24 precision, resolution, partition, and build-mode combinations. Timing output covers repeated calls for at least 0.3 CPU seconds per kernel. These short synthetic timings locate opportunities and regressions. They do not establish a whole-model speedup.
--resolutions, --repeats, --timeout, --reference, and --work control the run. The default work directory is unique and lies below project tmp/. An explicit existing work directory may overwrite earlier harness results. Each run saves results.json, build logs, timings, and binary outputs.
Simulator benchmark
Use an existing runtime directory that contains namelists, planetary input files, and the desired restart. The generated build directory must already exist after local setup and model compilation.
timeout 1100 .venv/bin/python scripts/benchmark-simulator.py MODEL_DIRECTORY --steps 1440 --repeats 3
The script configures flags in disposable directories and builds pinned upstream and current sources separately. It resets every runtime copy to the same template. It explicitly sets run years, months, and days to zero so the requested timestep count controls the run. It rotates execution order, records CPU and wall time, and checks for model aborts.
The default variants are checked upstream, checked current, native-performance upstream, and native-performance current. Use --variants to select a subset. The numerical gate compares source variants with identical flags. Cross-flag comparisons are informational because compiler settings can change trajectories.
Results include output and restart hashes, per-field numeric differences, repeated-run timings, medians, flags, and source identity. Compilation, Python postprocessing, and setup time are outside the timed simulator section. The benchmark keeps outputs enabled.
Actual whole-model acceptance currently covers serial T21 with ten levels in FP64. It does not cover other resolutions, MPI, other planetary configurations, or long-run climate statistics.