[ROCM-26203] Modify existing tests to work with pytest by eddieliao · Pull Request #4980 · ROCm/AMDMIGraphX

eddieliao · 2026-06-17T15:14:07Z

Motivation

Modifies some of our existing tests to work properly with pytest.

Technical Details

By default, pytest picks up any functions in the format of test_* or *_test and runs them as tests. This PR adds an underscore prefix to any functions that are meant to be helpers and not actual tests.

Changelog Category

Add a CHANGELOG.md entry for any option other than Not Applicable

- Added: New functionality.
- Changed: Changes to existing functionality.
- Removed: Functionality or support that has been removed. (Compared to a previous release)
- Optimized: Component performance that has been optimized or improved.
- Resolved Issues: Known issues from a previous version that have been resolved.
- Not Applicable: This PR is not to be included in the changelog.

Copilot

Pull request overview

This PR updates Python tests under test/py/ to behave correctly under pytest’s test discovery rules by preventing helper functions from being collected as tests.

Changes:

Rename helper functions from test_* to _test_* to avoid pytest collection.
Replace the old test_conv_relu(format) pattern (which pytest interprets as needing a fixture) with two explicit tests for msgpack/json serialization.
Update copyright headers to 2026 in the modified files.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
`test/py/test_save_load.py`	Renames the format-taking helper and adds explicit serialization tests (`msgpack`, `json`) to avoid pytest fixture/collection issues.
`test/py/test_array.py`	Renames `test_shape` helper to `_test_shape` and updates its call sites to prevent unintended pytest collection.

Comments suppressed due to low confidence (1)

test/py/test_save_load.py:31

The helper parameter name format shadows Python’s built-in format() and makes the call sites harder to read. Consider renaming the parameter to something more specific (e.g., serialization_format) and forwarding that into migraphx.save/load.

def _test_conv_relu(format):
    p1 = migraphx.parse_onnx("conv_relu_maxpool_test.onnx")
    print(p1)

    s1 = p1.get_output_shapes()[-1]

codecov · 2026-06-17T17:37:41Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #4980   +/-   ##
========================================
  Coverage    92.73%   92.73%           
========================================
  Files          592      594    +2     
  Lines        31289    31340   +51     
========================================
+ Hits         29015    29063   +48     
- Misses        2274     2277    +3

see 4 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

gh-app-migraphx-bot-pr-write · 2026-06-17T19:18:13Z

Test	Batch	New Rate (58fc8c)	Old Rate (f54ca3)*	Diff	Status
torchvision-resnet50	64	2,020.76	3,151.54	-35.88%	🔴
torchvision-resnet50_fp16	64	4,376.46	6,672.57	-34.41%	🔴
torchvision-densenet121	32	2,403.95	2,669.98	-9.96%	🔴
torchvision-densenet121_fp16	32	4,376.03	4,553.52	-3.90%	✅
torchvision-inceptionv3	32	1,791.62	1,794.69	-0.17%	✅
torchvision-inceptionv3_fp16	32	2,681.84	2,710.69	-1.06%	✅
cadene-inceptionv4	16	425.67	224.10	89.95%	🔆
cadene-resnext64x4	16	439.06	409.63	7.18%	🔆
slim-mobilenet	64	1,623.63	8,288.90	-80.41%	🔴
slim-nasnetalarge	64	nan	229.24	nan	❌
slim-resnet50v2	64	1,667.65	3,329.55	-49.91%	🔴
bert-mrpc-onnx	8	345.00	1,170.13	-70.52%	🔴
bert-mrpc-tf	1	80.80	482.99	-83.27%	🔴
pytorch-examples-wlang-gru	1	168.80	335.86	-49.74%	🔴
pytorch-examples-wlang-lstm	1	174.55	464.03	-62.38%	🔴
torchvision-resnet50_1	1	87.79	771.72	-88.62%	🔴
cadene-dpn92_1	1	150.00	451.76	-66.80%	🔴
cadene-resnext101_1	1	156.46	363.79	-56.99%	🔴
onnx-taau-downsample	1	330.76	400.76	-17.47%	🔴
dlrm-criteoterabyte	1	32.12	32.67	-1.69%	✅
dlrm-criteoterabyte_fp16	1	41.81	52.58	-20.48%	🔴
agentmodel	1	8,192.36	9,679.70	-15.37%	🔴
unet_fp16	2	34.93	57.23	-38.96%	🔴
resnet50v1_fp16	1	95.57	972.36	-90.17%	🔴
resnet50v1_int8	1	875.27	960.27	-8.85%	🔴
bert_base_cased_fp16	64	821.62	1,102.92	-25.50%	🔴
bert_large_uncased_fp16	32	169.24	347.52	-51.30%	🔴
bert_large_fp16	1	25.33	205.28	-87.66%	🔴
distilgpt2_fp16	16	878.03	2,094.45	-58.08%	🔴
yolov5s	1	68.96	568.20	-87.86%	🔴
tinyllama	1	14.74	46.01	-67.95%	🔴
vicuna-fastchat	1	9.27	44.04	-78.96%	🔴
whisper-tiny-encoder	1	132.61	419.49	-68.39%	🔴
whisper-tiny-decoder	1	205.43	420.06	-51.09%	🔴
llama2_7b	1	20.18	20.46	-1.39%	✅
qwen1.5-7b	1	2.31	23.67	-90.23%	🔴
phi3-3.8b	1	1.86	26.98	-93.12%	🔴
llama3-8b	1	8.38	21.83	-61.61%	🔴
whisper-large-encoder	1	10.31	10.32	-0.00%	✅
whisper-large-decoder	1	15.55	107.05	-85.47%	🔴
mistral-7b	1	14.75	23.86	-38.17%	🔴
FLUX.1-schnell	1	741.91	761.61	-2.59%	✅

Regressions detected 🔴

* No develop baseline was found for this PR's branch point; compared against the latest available develop run instead.

gh-app-migraphx-bot-pr-write · 2026-06-17T19:18:15Z

Test	Status	Result
bert-mrpc-onnx	✅	PASSED: MIGraphX meets tolerance
bert-mrpc-tf	❌	ERROR - check error output traceback Traceback (most recent call last): File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 377, in main() File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 313, in main import tensorflow as tf File "/usr/local/lib/python3.10/dist-packages/tensorflow/init.py", line 38, in from tensorflow.python.tools import module_util as _module_util File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/init.py", line 36, in from tensorflow.python import pywrap_tensorflow as _pywrap_tensorflow File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 26, in self_check.preload_check() File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/platform/self_check.py", line 63, in preload_check from tensorflow.python.platform import _pywrap_cpu_feature_guard ImportError: libamdhip64.so.6: cannot open shared object file: No such file or directory
pytorch-examples-wlang-gru	✅	PASSED: MIGraphX meets tolerance
pytorch-examples-wlang-lstm	✅	PASSED: MIGraphX meets tolerance
dlrm-criteoterabyte	✅	PASSED: MIGraphX meets tolerance
agentmodel	✅	PASSED: MIGraphX meets tolerance
unet	✅	PASSED: MIGraphX meets tolerance
resnet50v1	✅	PASSED: MIGraphX meets tolerance
bert_base_cased_fp16	✅	PASSED: MIGraphX meets tolerance
bert_large_uncased_fp16	🔴	FAILED: MIGraphX is not within tolerance - check verbose output
bert_large	✅	PASSED: MIGraphX meets tolerance
yolov5s	✅	PASSED: MIGraphX meets tolerance
tinyllama	✅	PASSED: MIGraphX meets tolerance
vicuna-fastchat	✅	PASSED: MIGraphX meets tolerance
whisper-tiny-encoder	✅	PASSED: MIGraphX meets tolerance
whisper-tiny-decoder	✅	PASSED: MIGraphX meets tolerance
distilgpt2_fp16	🔴	FAILED: MIGraphX is not within tolerance - check verbose output
llama2_7b	✅	PASSED: MIGraphX meets tolerance
qwen1.5-7b	✅	PASSED: MIGraphX meets tolerance
phi3-3.8b	✅	PASSED: MIGraphX meets tolerance
llama3-8b	✅	PASSED: MIGraphX meets tolerance
whisper-large-encoder	❌	ERROR - check error output traceback 2026-06-18 23:44:45.571550 [WARN] [/data/src/onnx/onnx_parser.cpp:282] Model has unbound symbolic dimension(s): batch_size, encoder_sequence_length, feature_size. These default to 1 and may cause unexpected behavior. Try setting `--dim-param @<name> <value>` or `--input-dim @<input> <dims>` if program compilation fails. Traceback (most recent call last): File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 377, in main() File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 224, in main model = migraphx.parse_onnx(model_name, default_dim_value=batch) RuntimeError: /data/src/include/migraphx/op/convolution.hpp:113: normalize_compute_shape: CONVOLUTION: mismatched channel numbers: input channels (1) != weights channels (80) * group (1)
whisper-large-decoder	✅	PASSED: MIGraphX meets tolerance
mistral-7b	✅	PASSED: MIGraphX meets tolerance
FLUX.1-schnell	✅	PASSED: MIGraphX meets tolerance

TedThemistokleous

LGTM on this one. Good work

ahsan-ca · 2026-06-18T19:19:26Z

 test_input()
 test_output()


Suggested change

if __name__ == "__main__":

test_input()

test_output()

I maybe wrong, but it seems without the if __name__ == "__main__": guard, the tests will execute twice, first during collection phase and then during execution phase.

Besides double execution, if the tests fail for any reason, pytest will fail at collection phase and it would mask the real-error/failure from the failing test, making it harder to debug.

That's what I originally thought too, but it seems like when we package the wheel it strips out any function calls. For example, here's what the original (failing) wheel run looked like:

((.venv) ) [root@efac41cc576b ROCM-26203]# pytest --pyargs migraphx_tests -v ============================================================================================================================ test session starts ============================================================================================================================= platform linux -- Python 3.12.13, pytest-9.1.0, pluggy-1.6.0 -- /code/ROCM-26203/.venv/bin/python3.12 cachedir: .pytest_cache rootdir: /code/ROCM-26203 collected 65 items .venv/lib/python3.12/site-packages/migraphx_tests/test_array.py::test_shape ERROR [ 1%] .venv/lib/python3.12/site-packages/migraphx_tests/test_array.py::test_input PASSED [ 3%] .venv/lib/python3.12/site-packages/migraphx_tests/test_array.py::test_output PASSED [ 4%] .venv/lib/python3.12/site-packages/migraphx_tests/test_autocast_fp8.py::test_autocast_fp8_1 PASSED [ 6%] .venv/lib/python3.12/site-packages/migraphx_tests/test_autocast_fp8.py::test_autocast_fp8_2 PASSED [ 7%] .venv/lib/python3.12/site-packages/migraphx_tests/test_cpu.py::test_conv_relu PASSED [ 9%] .venv/lib/python3.12/site-packages/migraphx_tests/test_cpu.py::test_add_scalar PASSED [ 10%] .venv/lib/python3.12/site-packages/migraphx_tests/test_cpu.py::test_module PASSED [ 12%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_no_debug_symbols_by_default PASSED [ 13%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_add_instruction_with_debug_symbols PASSED [ 15%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_add_instruction_empty_debug_symbols PASSED [ 16%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_add_instruction_with_mod_args_and_debug_symbols PASSED [ 18%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_add_literal_argument_with_debug_symbols PASSED [ 20%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_add_parameter_with_debug_symbols PASSED [ 21%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_add_return_with_debug_symbols PASSED [ 23%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_replace_return_with_debug_symbols PASSED [ 24%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_add_macro_with_debug_symbols_tags_all_added PASSED [ 26%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_add_macro_without_debug_symbols_is_unchanged PASSED [ 27%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_insert_macro_with_debug_symbols PASSED [ 29%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_iterate_instructions_debug_symbols PASSED [ 30%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_iterate_only_symbolized_instructions PASSED [ 32%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_parse_onnx_with_debug_symbols PASSED [ 33%] .venv/lib/python3.12/site-packages/migraphx_tests/test_debug_symbols.py::test_parse_onnx_without_debug_symbols PASSED [ 35%] .venv/lib/python3.12/site-packages/migraphx_tests/test_gpu.py::test_conv_relu PASSED [ 36%] .venv/lib/python3.12/site-packages/migraphx_tests/test_gpu.py::test_sub_uint64 PASSED [ 38%] .venv/lib/python3.12/site-packages/migraphx_tests/test_gpu.py::test_neg_int64 PASSED [ 40%] .venv/lib/python3.12/site-packages/migraphx_tests/test_gpu.py::test_nonzero PASSED [ 41%] .venv/lib/python3.12/site-packages/migraphx_tests/test_gpu.py::test_fp16_imagescaler PASSED [ 43%] .venv/lib/python3.12/site-packages/migraphx_tests/test_gpu.py::test_if_pl PASSED [ 44%] .venv/lib/python3.12/site-packages/migraphx_tests/test_gpu.py::test_dyn_batch PASSED [ 46%] .venv/lib/python3.12/site-packages/migraphx_tests/test_gpu_async.py::test_conv_relu PASSED [ 47%] .venv/lib/python3.12/site-packages/migraphx_tests/test_gpu_offload.py::test_conv_relu PASSED [ 49%] .venv/lib/python3.12/site-packages/migraphx_tests/test_instruction.py::test_instruction_shape PASSED [ 50%] .venv/lib/python3.12/site-packages/migraphx_tests/test_instruction.py::test_instruction_op PASSED [ 52%] .venv/lib/python3.12/site-packages/migraphx_tests/test_literal.py::test_add_fill PASSED [ 53%] .venv/lib/python3.12/site-packages/migraphx_tests/test_literal.py::test_add_create PASSED [ 55%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_macro_name_and_options PASSED [ 56%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_macro_no_options PASSED [ 58%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_add_macro_gemm PASSED [ 60%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_add_macro_gemm_transA PASSED [ 61%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_add_macro_gemm_with_params PASSED [ 63%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_add_macro_batchnorm PASSED [ 64%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_add_macro_batchnorm_with_epsilon PASSED [ 66%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_add_macro_convolution PASSED [ 67%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_add_macro_convolution_with_bias PASSED [ 69%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_add_macro_convolution_with_padding PASSED [ 70%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_add_macro_einsum_matmul PASSED [ 72%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_add_macro_einsum_transpose PASSED [ 73%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_add_macro_einsum_trace PASSED [ 75%] .venv/lib/python3.12/site-packages/migraphx_tests/test_macro.py::test_insert_macro PASSED [ 76%] .venv/lib/python3.12/site-packages/migraphx_tests/test_module_construct.py::test_add_op PASSED [ 78%] .venv/lib/python3.12/site-packages/migraphx_tests/test_module_construct.py::test_if_then_else PASSED [ 80%] .venv/lib/python3.12/site-packages/migraphx_tests/test_numpy.py::test_add_op PASSED [ 81%] .venv/lib/python3.12/site-packages/migraphx_tests/test_op.py::test_add_op PASSED [ 83%] .venv/lib/python3.12/site-packages/migraphx_tests/test_op.py::test_reduce_mean PASSED [ 84%] .venv/lib/python3.12/site-packages/migraphx_tests/test_save_load.py::test_conv_relu ERROR [ 86%] .venv/lib/python3.12/site-packages/migraphx_tests/test_save_load.py::test_save_load_buffer PASSED [ 87%] .venv/lib/python3.12/site-packages/migraphx_tests/test_save_load.py::test_load_save_arg PASSED [ 89%] .venv/lib/python3.12/site-packages/migraphx_tests/test_shape.py::test_create_shape PASSED [ 90%] .venv/lib/python3.12/site-packages/migraphx_tests/test_shape.py::test_create_shape_transposed PASSED [ 92%] .venv/lib/python3.12/site-packages/migraphx_tests/test_shape.py::test_create_shape_broadcast PASSED [ 93%] .venv/lib/python3.12/site-packages/migraphx_tests/test_shape.py::test_create_shape_type PASSED [ 95%] .venv/lib/python3.12/site-packages/migraphx_tests/test_shape.py::test_create_dyn_dims PASSED [ 96%] .venv/lib/python3.12/site-packages/migraphx_tests/test_shape.py::test_create_dyn_shape PASSED [ 98%] .venv/lib/python3.12/site-packages/migraphx_tests/test_shape.py::test_type_enum PASSED [100%]

test_array only runs test_input() and test_output() once. I can add the main guard if preferred, but then ideally it would be added for all the other files as well to maintain consistency.

Update: Just tested with a manual UFB run to package the wheel and it seems to pass all tests.

After discussion with Paul, it seems this requires a sync with QA team to ensure they are running tests consistently. I believe my comment would be resolved with that path.

Prefix internal functions with underscore

58fc8cc

eddieliao requested a review from Copilot June 17, 2026 15:14

eddieliao self-assigned this Jun 17, 2026

eddieliao added python Pull requests that update python code Changelog: Not Applicable This PR is not to be included in the Changelog. labels Jun 17, 2026

Copilot started reviewing on behalf of eddieliao June 17, 2026 15:15 View session

causten requested review from ahsan-ca and kentqian June 17, 2026 15:15

Copilot AI reviewed Jun 17, 2026

View reviewed changes

Comment thread test/py/test_array.py

eddieliao marked this pull request as ready for review June 17, 2026 15:40

eddieliao requested a review from causten as a code owner June 17, 2026 15:40

eddieliao requested a review from TedThemistokleous June 17, 2026 15:40

TedThemistokleous approved these changes Jun 18, 2026

View reviewed changes

TedThemistokleous added the Continous Integration Pull request updates parts of continous integration pipeline label Jun 18, 2026

ahsan-ca requested changes Jun 18, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ROCM-26203] Modify existing tests to work with pytest#4980

[ROCM-26203] Modify existing tests to work with pytest#4980
eddieliao wants to merge 1 commit into
developfrom
pytest_fixes

eddieliao commented Jun 17, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

codecov Bot commented Jun 17, 2026

Uh oh!

gh-app-migraphx-bot-pr-write Bot commented Jun 17, 2026 •

edited

Loading

Uh oh!

gh-app-migraphx-bot-pr-write Bot commented Jun 17, 2026 •

edited

Loading

Uh oh!

TedThemistokleous left a comment

Uh oh!

ahsan-ca Jun 18, 2026 •

edited

Loading

Uh oh!

eddieliao Jun 18, 2026 •

edited

Loading

Uh oh!

ahsan-ca Jun 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

eddieliao commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Technical Details

Changelog Category

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

codecov Bot commented Jun 17, 2026

Codecov Report

Uh oh!

gh-app-migraphx-bot-pr-write Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gh-app-migraphx-bot-pr-write Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TedThemistokleous left a comment

Choose a reason for hiding this comment

Uh oh!

ahsan-ca Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eddieliao Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ahsan-ca Jun 19, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

eddieliao commented Jun 17, 2026 •

edited

Loading

gh-app-migraphx-bot-pr-write Bot commented Jun 17, 2026 •

edited

Loading

gh-app-migraphx-bot-pr-write Bot commented Jun 17, 2026 •

edited

Loading

ahsan-ca Jun 18, 2026 •

edited

Loading

eddieliao Jun 18, 2026 •

edited

Loading