Skip to content

Sanitize benchmark mxr file name to use _ instead of invalid Windows characters#5024

Open
ahsan-ca wants to merge 5 commits into
developfrom
change-benchmark-mxr-filename
Open

Sanitize benchmark mxr file name to use _ instead of invalid Windows characters#5024
ahsan-ca wants to merge 5 commits into
developfrom
change-benchmark-mxr-filename

Conversation

@ahsan-ca

@ahsan-ca ahsan-ca commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Motivation

Windows has issues with filenames containing ::.
This changes mxr file names to use __ instead of ::.

Technical Details

Changelog Category

Add a CHANGELOG.md entry for any option other than Not Applicable

    • Added: New functionality.
    • Changed: Changes to existing functionality.
    • Removed: Functionality or support that has been removed. (Compared to a previous release)
    • Optimized: Component performance that has been optimized or improved.
    • Resolved Issues: Known issues from a previous version that have been resolved.
    • Not Applicable: This PR is not to be included in the changelog.

@ahsan-ca ahsan-ca requested review from kahmed10 and pfultz2 June 30, 2026 14:29
@ahsan-ca ahsan-ca self-assigned this Jun 30, 2026
@ahsan-ca ahsan-ca requested a review from causten as a code owner June 30, 2026 14:29
Copilot AI review requested due to automatic review settings June 30, 2026 14:29
@ahsan-ca ahsan-ca force-pushed the change-benchmark-mxr-filename branch from b5f11a8 to 442f654 Compare June 30, 2026 14:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates GPU benchmark .mxr file naming to avoid :: in operator names (improving filename portability, especially on Windows), and adds a VS Code workspace settings file.

Changes:

  • Sanitize preop.name() when generating benchmark .mxr filenames by replacing :: with _.
  • Add .vscode/settings.json with workspace settings (remote extension kind + disable CMake configure-on-open).

Reviewed changes

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

File Description
src/targets/gpu/compile_ops.cpp Sanitizes operator names before using them in benchmark .mxr filenames.
.vscode/settings.json Adds VS Code workspace settings.

Comment thread src/targets/gpu/compile_ops.cpp
Comment thread src/targets/gpu/compile_ops.cpp Outdated
@ahsan-ca ahsan-ca requested a review from ikalinic June 30, 2026 14:40
@ahsan-ca ahsan-ca force-pushed the change-benchmark-mxr-filename branch from 442f654 to 2726846 Compare June 30, 2026 14:41
@ahsan-ca ahsan-ca changed the title Change benchmark mxr file name to use _ instead of :: Change benchmark mxr file name to use __ instead of :: Jun 30, 2026
@ahsan-ca ahsan-ca force-pushed the change-benchmark-mxr-filename branch from 2726846 to ac65834 Compare June 30, 2026 14:49
@ahsan-ca ahsan-ca added the high priority A PR with high priority for review and merging. label Jun 30, 2026
Comment thread src/targets/gpu/compile_ops.cpp Outdated
auto problem_hash = std::hash<std::string>{}(to_string(config->problem));
auto mxr_file = mxr_dir / (preop.name() + "_" + std::to_string(i) + "_" +
// Sanitize the op name for use in a filename. Replace "::" with "__".
auto op_filename = replace_string(preop.name(), "::", "__");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pass_manager has a similar sanitize function, I wonder if we should reuse that somehow.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have made the changes for that. I had to pull the function from pass_manager out to be used for compile_ops. I have put it in filesystem.hpp, do you think it is better to keep it in filesystem.hpp or in stringutils.hpp?

@ahsan-ca ahsan-ca force-pushed the change-benchmark-mxr-filename branch from 985f007 to b13c749 Compare June 30, 2026 15:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comment on lines 27 to 31
#include <migraphx/config.hpp>
#include <algorithm>
#include <string>
#include <string_view>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pfultz2 wondering if we should move sanitize_filename() to stringutils.hpp for the above comment.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think fileutils.hpp is better suited for sanitize_filename() function.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made changes to move sanitize_filename() to fileutils.hpp. Re-requesting review after this change. @pfultz2

@ahsan-ca ahsan-ca changed the title Change benchmark mxr file name to use __ instead of :: Sanitize benchmark mxr file name to use _ instead of invalid Windows characters Jun 30, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Comment thread src/include/migraphx/fileutils.hpp
Comment thread src/pass_manager.cpp Outdated
Comment thread src/fileutils.cpp

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Comment thread src/include/migraphx/fileutils.hpp
Comment on lines +520 to 522
auto op_filename = sanitize_filename(preop.name());
auto mxr_file = mxr_dir / (op_filename + "_" + std::to_string(i) + "_" +
std::to_string(problem_hash) + ".mxr");
@codecov

codecov Bot commented Jun 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #5024   +/-   ##
========================================
  Coverage    92.73%   92.73%           
========================================
  Files          596      596           
  Lines        31813    31817    +4     
========================================
+ Hits         29500    29504    +4     
  Misses        2313     2313           
Files with missing lines Coverage Δ
src/fileutils.cpp 100.00% <100.00%> (ø)
src/include/migraphx/fileutils.hpp 100.00% <ø> (ø)
src/pass_manager.cpp 82.08% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gh-app-migraphx-bot-pr-write

Copy link
Copy Markdown
Test Batch New Rate (96d63f) Old Rate (9ad497)* Diff Status
torchvision-resnet50 64 3,154.82 3,148.35 0.21%
torchvision-resnet50_fp16 64 6,635.31 6,662.40 -0.41%
torchvision-densenet121 32 2,695.58 2,707.30 -0.43%
torchvision-densenet121_fp16 32 4,540.35 4,558.12 -0.39%
torchvision-inceptionv3 32 1,795.71 1,797.44 -0.10%
torchvision-inceptionv3_fp16 32 2,818.84 2,824.56 -0.20%
cadene-inceptionv4 16 822.01 807.16 1.84%
cadene-resnext64x4 16 782.67 387.68 101.88% 🔆
slim-mobilenet 64 8,385.05 8,283.02 1.23%
slim-nasnetalarge 64 228.93 229.53 -0.26%
slim-resnet50v2 64 3,161.07 3,177.49 -0.52%
bert-mrpc-onnx 8 1,171.75 1,171.12 0.05%
bert-mrpc-tf 1 494.13 479.25 3.11%
pytorch-examples-wlang-gru 1 323.60 327.85 -1.30%
pytorch-examples-wlang-lstm 1 454.98 457.62 -0.58%
torchvision-resnet50_1 1 771.37 765.65 0.75%
cadene-dpn92_1 1 451.85 447.53 0.97%
cadene-resnext101_1 1 366.72 366.68 0.01%
onnx-taau-downsample 1 398.50 401.14 -0.66%
dlrm-criteoterabyte 1 32.39 32.57 -0.53%
dlrm-criteoterabyte_fp16 1 51.80 52.61 -1.55%
agentmodel 1 9,340.99 8,132.20 14.86% 🔆
unet_fp16 2 56.92 57.28 -0.62%
resnet50v1_fp16 1 937.99 932.12 0.63%
resnet50v1_int8 1 932.30 943.10 -1.15%
bert_base_cased_fp16 64 1,097.49 1,102.38 -0.44%
bert_large_uncased_fp16 32 346.26 347.33 -0.31%
bert_large_fp16 1 203.71 205.67 -0.95%
distilgpt2_fp16 16 2,093.30 2,094.34 -0.05%
yolov5s 1 595.05 597.91 -0.48%
tinyllama 1 46.01 45.98 0.06%
vicuna-fastchat 1 44.15 44.11 0.10%
whisper-tiny-encoder 1 417.69 420.33 -0.63%
whisper-tiny-decoder 1 415.99 416.77 -0.19%
llama2_7b 1 20.34 20.50 -0.78%
qwen1.5-7b 1 23.61 23.60 0.02%
phi3-3.8b 1 26.85 26.79 0.23%
llama3-8b 1 21.74 21.77 -0.14%
whisper-large-encoder 1 10.28 10.31 -0.35%
whisper-large-decoder 1 104.21 105.11 -0.85%
mistral-7b 1 23.81 23.78 0.13%
FLUX.1-schnell 1 762.14 751.64 1.40%

Check flagged results 🔆

* No develop baseline was found for this PR's branch point; compared against the latest available develop run instead.

@gh-app-migraphx-bot-pr-write

Copy link
Copy Markdown
Test Status Result
bert-mrpc-onnx PASSED: MIGraphX meets tolerance
bert-mrpc-tf PASSED: MIGraphX meets tolerance
pytorch-examples-wlang-gru PASSED: MIGraphX meets tolerance
pytorch-examples-wlang-lstm PASSED: MIGraphX meets tolerance
dlrm-criteoterabyte PASSED: MIGraphX meets tolerance
agentmodel PASSED: MIGraphX meets tolerance
unet PASSED: MIGraphX meets tolerance
resnet50v1 PASSED: MIGraphX meets tolerance
bert_base_cased_fp16 PASSED: MIGraphX meets tolerance
bert_large_uncased_fp16 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
bert_large PASSED: MIGraphX meets tolerance
yolov5s PASSED: MIGraphX meets tolerance
tinyllama PASSED: MIGraphX meets tolerance
vicuna-fastchat PASSED: MIGraphX meets tolerance
whisper-tiny-encoder PASSED: MIGraphX meets tolerance
whisper-tiny-decoder PASSED: MIGraphX meets tolerance
distilgpt2_fp16 PASSED: MIGraphX meets tolerance
llama2_7b PASSED: MIGraphX meets tolerance
qwen1.5-7b PASSED: MIGraphX meets tolerance
phi3-3.8b PASSED: MIGraphX meets tolerance
llama3-8b PASSED: MIGraphX meets tolerance
whisper-large-encoder PASSED: MIGraphX meets tolerance
whisper-large-decoder PASSED: MIGraphX meets tolerance
mistral-7b PASSED: MIGraphX meets tolerance
FLUX.1-schnell PASSED: MIGraphX meets tolerance

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

high priority A PR with high priority for review and merging.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants