Benchmark Suite

Benchmark Suite#

One command trains every registered problem across multiple epoch budgets and produces a complete analysis package — plots, CSV, Markdown, and reusable JSON.

CLI usage#

# Run all fast problems — smooth PDEs get [500, 1000, 2000, 5000] epochs;
# problems marked complex=True (ramp, toro3) get [2000, 5000, 15000, 40000]
python -m underPINN bench

# Select specific problems and a custom budget for the simple ones
python -m underPINN bench \
    --problems burgers wave helmholtz heat_steady ode_harmonic ramp toro3 \
    --epochs 500 1000 2000 5000 \
    --output outputs/bench

# Include slow problems (3-D pipe flow, viscous ramp NS — also complex=True)
python -m underPINN bench --all

# Override the complex-problem budget directly
python -m underPINN bench --all --complex-epochs 2000 5000 15000

# Regenerate plots from a previous run without re-training
python -m underPINN bench --from-json outputs/bench/results.json

Two-tier epoch budgets

Smooth PDEs (Burgers, wave, Helmholtz, …) use the default [500, 1000, 2000, 5000]. Problems marked complex=True on their evaluator — shocks, viscous SBLI, 3-D N-S (ramp, toro3, pipe_flow, ramp_ns) — automatically get a much larger [2000, 5000, 15000, 40000] budget, because they converge far more slowly and the shared small budget was under-training them. Both tiers are overridable via --epochs / --complex-epochs.

Programmatic usage#

from underPINN.benchmark_utils import BenchmarkRunner, generate_report

runner = BenchmarkRunner(
    problems      = ["burgers", "wave", "ode_exp", "helmholtz"],
    epoch_budgets = [500, 1000, 2000, 5000],
    seed          = 0, fast_only=True, verbose=True,
)
results = runner.run(out_dir="outputs/bench")
runner.save_json("outputs/bench/results.json")
generate_report(results, runner, out_dir="outputs/bench")

Outputs written to outputs/bench/#

File

Description

accuracy_vs_epochs.png

Log-log rel-L² vs epoch budget, one curve per problem

accuracy_summary_bar.png

Grouped bar chart of rel-L² at each epoch budget

wall_time_vs_epochs.png

Training time vs epoch budget

ms_per_epoch.png

Bar chart of training throughput per problem

loss_grid.png

Convergence curves for every problem

benchmark_results.csv

Full raw data table — importable into pandas

benchmark_summary.md

Markdown table, one row per problem at max epochs

results.json

Reusable for --from-json replays without re-training

Tip

results.json is a complete, self-contained snapshot of a benchmark run — check it into version control alongside a paper or report so figures can be regenerated byte-for-byte without re-running any GPU training.

Contents