Summary
When using @ParameterizedTest (with @ValueSource, @CsvSource, etc.), the Testomat Java reporter silently drops all results. The tests never appear in any run — neither as passed nor failed.
Environment
java-reporter-junit version: 0.11.6 (confirmed still present in latest release)- JUnit version: 5
- Maven Surefire: 3.5.3
Behaviour
Expected
Each parameterized test variant is reported to Testomat with its result (pass/fail), or at minimum the method-level result is aggregated and reported once per @TestId.
Actual
Tests annotated with @ParameterizedTest produce zero entries in Testomat runs. They have @TestId annotations, the IDs are registered in the Testomat project, but no run history is ever recorded.
Plain @Test methods on the same class, with the same setup, are reported correctly.
Reproduction
@TestId("abc12345")
@Tag("smoke")
@ParameterizedTest
@ValueSource(strings = {"EUR", "GBP"})
public void Transfer_Success(final String currency) {
// test body
}After running the test suite with TESTOMATIO env var set, the test with ID abc12345 shows no run history in Testomat.
Additional context
We confirmed this by cross-referencing JUnit XML surefire reports (which correctly record all parameterized results) against the Testomat run. A run with 158 total test cases — of which roughly half were @ParameterizedTest variants — showed only 72 results in Testomat, all of them plain @Test methods.
Related issue: shared @TestId on abstract base classes
A secondary issue: when @TestId is declared on a method in an abstract base class and multiple concrete subclasses inherit it, Testomat receives a result from each subclass under the same ID. The last result silently overwrites the earlier ones, which can produce false positives (a failing Consumer variant is hidden by a passing Corporate variant of the same test).
Summary
When using
@ParameterizedTest(with@ValueSource,@CsvSource, etc.), the Testomat Java reporter silently drops all results. The tests never appear in any run — neither as passed nor failed.Environment
java-reporter-junitversion:0.11.6(confirmed still present in latest release)Behaviour
Expected
Each parameterized test variant is reported to Testomat with its result (pass/fail), or at minimum the method-level result is aggregated and reported once per
@TestId.Actual
Tests annotated with
@ParameterizedTestproduce zero entries in Testomat runs. They have@TestIdannotations, the IDs are registered in the Testomat project, but no run history is ever recorded.Plain
@Testmethods on the same class, with the same setup, are reported correctly.Reproduction
After running the test suite with
TESTOMATIOenv var set, the test with IDabc12345shows no run history in Testomat.Additional context
We confirmed this by cross-referencing JUnit XML surefire reports (which correctly record all parameterized results) against the Testomat run. A run with 158 total test cases — of which roughly half were
@ParameterizedTestvariants — showed only 72 results in Testomat, all of them plain@Testmethods.Related issue: shared
@TestIdon abstract base classesA secondary issue: when
@TestIdis declared on a method in an abstract base class and multiple concrete subclasses inherit it, Testomat receives a result from each subclass under the same ID. The last result silently overwrites the earlier ones, which can produce false positives (a failing Consumer variant is hidden by a passing Corporate variant of the same test).