Skip to content

Handle benchmark MXR dump as successful compile#5021

Closed
ikalinic wants to merge 2 commits into
developfrom
ikalinic/benchmark-mxr-dump-no-error-logs
Closed

Handle benchmark MXR dump as successful compile#5021
ikalinic wants to merge 2 commits into
developfrom
ikalinic/benchmark-mxr-dump-no-error-logs

Conversation

@ikalinic

Copy link
Copy Markdown
Member

Motivation

Avoid error logs and nonzero driver execution for the expected migraphx-driver compile path that dumps benchmark MXR files via MIGRAPHX_GPU_DUMP_BENCHMARK_MXR.

Technical Details

Added a dedicated migraphx::benchmark_mxr_dumped exception and throw macro, used it for the benchmark MXR dump completion path, logged it as info in the pass manager, and swallowed it at driver command dispatch so the dump-only mode completes successfully.

Changelog Category

Add a CHANGELOG.md entry for any option other than Not Applicable

    • Added: New functionality.
    • Changed: Changes to existing functionality.
    • Removed: Functionality or support that has been removed. (Compared to a previous release)
    • Optimized: Component performance that has been optimized or improved.
    • Resolved Issues: Known issues from a previous version that have been resolved.
    • Not Applicable: This PR is not to be included in the changelog.

@ikalinic ikalinic requested a review from causten as a code owner June 30, 2026 09:58
@codecov

codecov Bot commented Jun 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/pass_manager.cpp 0.00% 3 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #5021      +/-   ##
===========================================
- Coverage    92.71%   92.71%   -0.01%     
===========================================
  Files          596      596              
  Lines        31733    31736       +3     
===========================================
  Hits         29421    29421              
- Misses        2312     2315       +3     
Files with missing lines Coverage Δ
src/include/migraphx/errors.hpp 100.00% <ø> (ø)
src/pass_manager.cpp 79.82% <0.00%> (-2.26%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gh-app-migraphx-bot-pr-write

gh-app-migraphx-bot-pr-write Bot commented Jun 30, 2026

Copy link
Copy Markdown
Test Batch New Rate (71ea83) Old Rate (9ad497) Diff Status
torchvision-resnet50 64 1,346.40 3,148.35 -57.23% 🔴
torchvision-resnet50_fp16 64 2,119.68 6,662.40 -68.18% 🔴
torchvision-densenet121 32 2,520.80 2,707.30 -6.89% 🔴
torchvision-densenet121_fp16 32 4,547.77 4,558.12 -0.23%
torchvision-inceptionv3 32 1,798.65 1,797.44 0.07%
torchvision-inceptionv3_fp16 32 2,820.38 2,824.56 -0.15%
cadene-inceptionv4 16 819.94 807.16 1.58%
cadene-resnext64x4 16 784.01 387.68 102.23% 🔆
slim-mobilenet 64 8,433.80 8,283.02 1.82%
slim-nasnetalarge 64 132.58 229.53 -42.24% 🔴
slim-resnet50v2 64 2,800.91 3,177.49 -11.85% 🔴
bert-mrpc-onnx 8 1,171.94 1,171.12 0.07%
bert-mrpc-tf 1 195.28 479.25 -59.25% 🔴
pytorch-examples-wlang-gru 1 330.73 327.85 0.88%
pytorch-examples-wlang-lstm 1 400.38 457.62 -12.51% 🔴
torchvision-resnet50_1 1 282.28 765.65 -63.13% 🔴
cadene-dpn92_1 1 440.64 447.53 -1.54%
cadene-resnext101_1 1 366.43 366.68 -0.07%
onnx-taau-downsample 1 400.73 401.14 -0.10%
dlrm-criteoterabyte 1 1.11 32.57 -96.59% 🔴
dlrm-criteoterabyte_fp16 1 52.65 52.61 0.07%
agentmodel 1 8,710.50 8,132.20 7.11% 🔆
unet_fp16 2 29.13 57.28 -49.14% 🔴
resnet50v1_fp16 1 964.74 932.12 3.50%
resnet50v1_int8 1 932.12 943.10 -1.16%
bert_base_cased_fp16 64 603.84 1,102.38 -45.22% 🔴
bert_large_uncased_fp16 32 347.39 347.33 0.02%
bert_large_fp16 1 207.46 205.67 0.87%
distilgpt2_fp16 16 2,089.81 2,094.34 -0.22%
yolov5s 1 591.91 597.91 -1.00%
tinyllama 1 45.37 45.98 -1.33%
vicuna-fastchat 1 23.40 44.11 -46.95% 🔴
whisper-tiny-encoder 1 421.45 420.33 0.27%
whisper-tiny-decoder 1 415.04 416.77 -0.41%
llama2_7b 1 20.57 20.50 0.33%
qwen1.5-7b 1 23.61 23.60 0.02%
phi3-3.8b 1 26.80 26.79 0.06%
llama3-8b 1 21.73 21.77 -0.19%
whisper-large-encoder 1 10.31 10.31 -0.03%
whisper-large-decoder 1 103.60 105.11 -1.43%
mistral-7b 1 23.76 23.78 -0.06%
FLUX.1-schnell 1 753.72 751.64 0.28%

Regressions detected 🔴

@gh-app-migraphx-bot-pr-write

Copy link
Copy Markdown
Test Status Result
bert-mrpc-onnx PASSED: MIGraphX meets tolerance
bert-mrpc-tf PASSED: MIGraphX meets tolerance
pytorch-examples-wlang-gru PASSED: MIGraphX meets tolerance
pytorch-examples-wlang-lstm PASSED: MIGraphX meets tolerance
dlrm-criteoterabyte PASSED: MIGraphX meets tolerance
agentmodel PASSED: MIGraphX meets tolerance
unet PASSED: MIGraphX meets tolerance
resnet50v1 PASSED: MIGraphX meets tolerance
bert_base_cased_fp16 PASSED: MIGraphX meets tolerance
bert_large_uncased_fp16 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
bert_large PASSED: MIGraphX meets tolerance
yolov5s PASSED: MIGraphX meets tolerance
tinyllama PASSED: MIGraphX meets tolerance
vicuna-fastchat PASSED: MIGraphX meets tolerance
whisper-tiny-encoder PASSED: MIGraphX meets tolerance
whisper-tiny-decoder PASSED: MIGraphX meets tolerance
distilgpt2_fp16 PASSED: MIGraphX meets tolerance
llama2_7b PASSED: MIGraphX meets tolerance
qwen1.5-7b PASSED: MIGraphX meets tolerance
phi3-3.8b PASSED: MIGraphX meets tolerance
llama3-8b PASSED: MIGraphX meets tolerance
whisper-large-encoder PASSED: MIGraphX meets tolerance
whisper-large-decoder PASSED: MIGraphX meets tolerance
mistral-7b PASSED: MIGraphX meets tolerance
FLUX.1-schnell PASSED: MIGraphX meets tolerance

@ahsan-ca ahsan-ca requested a review from pfultz2 June 30, 2026 14:14
* @brief Throw an exception with context information
*/
#define MIGRAPHX_THROW(...) throw migraphx::make_exception(MIGRAPHX_MAKE_SOURCE_CTX(), __VA_ARGS__)
#define MIGRAPHX_THROW_BENCHMARK_MXR_DUMPED(...) \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be added to the errors. This a gpu-specific error.

Comment thread src/driver/main.cpp
{
m.at(cmd)(ap, {args.begin() + 1, args.end()});
}
catch(const migraphx::benchmark_mxr_dumped& e)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont really like catching exceptions in the driver as it reduces stopping at the exception in the debugger.

This also gpu-specific and the driver is for all targets.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stop reading my mind. I think this would be fine though if we caught a specific execption this is trying to block out vs a catch-all here?

if(dump_mxr and is_root)
{
MIGRAPHX_THROW(
MIGRAPHX_THROW_BENCHMARK_MXR_DUMPED(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of throwing an exception we can just print a message with log::info and then exit(0).

Comment thread src/pass_manager.cpp
{
f();
}
catch(const benchmark_mxr_dumped& e)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pass_manager shouldn't be catching gpu-specific exceptions.

@TedThemistokleous TedThemistokleous left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the catch all - either catch the specific execption in the driver you're trying to avoid or make a clear log/warning and decide if you need to exit or can move forward

Comment thread src/driver/main.cpp
{
m.at(cmd)(ap, {args.begin() + 1, args.end()});
}
catch(const migraphx::benchmark_mxr_dumped& e)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stop reading my mind. I think this would be fine though if we caught a specific execption this is trying to block out vs a catch-all here?

@ikalinic

ikalinic commented Jul 1, 2026

Copy link
Copy Markdown
Member Author

Closing in favor of #5025

@ikalinic ikalinic closed this Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants