Skip to content

Add high-accuracy test runner script for generating and comparing CAMB references#197

Open
cmbant wants to merge 5 commits into
masterfrom
codex/refactor-run_tests.py-for-high-accuracy-runs
Open

Add high-accuracy test runner script for generating and comparing CAMB references#197
cmbant wants to merge 5 commits into
masterfrom
codex/refactor-run_tests.py-for-high-accuracy-runs

Conversation

@cmbant

@cmbant cmbant commented Apr 22, 2026

Copy link
Copy Markdown
Owner

Motivation

  • Provide a standalone test driver to generate high-accuracy reference ini files and outputs and to compare default runs against those references.
  • Enable adjustable accuracy boosts and stability tuning for systematic high-precision validation of Fortran CAMB outputs.
  • Allow both module-based and command-line execution and support for overriding ini parameters across generated tests.

Description

  • Add fortran/tests/run_high_accuracy_tests.py, a new script that implements argument parsing and two main modes: generate_reference to create high-accuracy INI files and produce reference outputs, and compare to run standard tests and diff them against references using CAMB_test_files integration.
  • Implement high-accuracy parameter adjustments via flags such as --int_tol_boost, --l_sample_boost, --l_accuracy_boost, --lens_potential_accuracy, --lens_margin, --stability_factor, and --do_late_rad_truncation, applied in _apply_high_accuracy_settings before writing reference INIs.
  • Support overrides (--override KEY=VALUE), cleaning of target dirs, runner selection (--runner), and output customization; create reference INIs with make_reference_inis and execute them with run_reference_inis.
  • Add numeric comparison and summary logic in summarize_differences which reads numeric tables, computes per-file and overall RMS(relative) and worst errors, and prints top offenders.

Testing

  • No automated tests were added or executed as part of this change.

Codex Task

@cmbant

cmbant commented Apr 28, 2026

Copy link
Copy Markdown
Owner Author

Thanks, will try to have a look before too long.

Did self.assertAlmostEqual(pars.YHe, 0.246218, 5) give you a problem? Seems fine on the github runners.

The vscode config change also looks odd.

Does this method of configuring a BaseModel+ work with Cobaya?

@Auggie review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant