adding freertr to the suite - #2
Closed
mc36 wants to merge 1 commit into
Closed
Conversation
jopietsch
added a commit
that referenced
this pull request
Sep 3, 2026
… another A test may declare `repetitions: N`. The matrix repeats as a block rather than each cell in a row, so consecutive runs of one cell do not share a warm page cache and the same thermal state, and an interrupted batch holds one observation of everything rather than every observation of the first few cells. A repetition is part of a run's name, because every artifact a run writes is named from one stem and each is written with `os.replace` or `open(..., 'w')`. That stem was already losing evidence: it omitted `filter_test`, so the three policy cells of benchmarks/2026-filters.yaml wrote the same files, and the mid-run graphs were named from args.target alone. Both go through `bench_output_prefix()` now, and the run dicts record both dimensions. Cell ids say what a cell is, never when it ran, so resume survives a permuted execution order. A single-pass test says "no repetition" in the name and the id alike -- otherwise a completed batch that later gained repetitions matched its stored pass-1 ids and produced a CSV holding `bird` beside `bird #2`. Omitting the key also leaves single-pass ids in their pre-repetition shape, so an in-flight batch from an older build still resumes. `check_batch_test()` closes the gap beside `batch_repetitions()`: a missing axis reached expansion as a bare KeyError, which all of big-tests.yaml did. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
if you're interested in a java based, heavily multi-threaded implementation then here we go... :)
for now, it statically sets the java heap to 4gb, for (multiple) full tables, that could be low....
after running "bgperf2.py bench -t freertr" i got:
freertr,freertr,25.9.17,100,100,9900,10000,6,4,2,2,25.63,348,0.748,100,6.067,,2025-09-17,4,7.76GB,0,0,,,