Skip to content

Align HTML report filename path semantics with TRX#8523

Merged
Evangelink merged 7 commits into
mainfrom
copilot/mtp-cli-analysis-report-filename-path-semantics
May 25, 2026
Merged

Align HTML report filename path semantics with TRX#8523
Evangelink merged 7 commits into
mainfrom
copilot/mtp-cli-analysis-report-filename-path-semantics

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 23, 2026

--report-html-filename and --report-trx-filename looked like sibling options but accepted different path shapes. HTML rejected nested paths while TRX accepted relative paths, absolute paths, and placeholders.

  • HTML filename validation

    • Allows bare filenames, relative paths, and absolute paths.
    • Rejects relative paths that escape the test results directory.
    • Keeps .html leaf extension validation.
  • HTML report output

    • Resolves standard artifact placeholders: {pname}, {pid}, {asm}, {tfm}, {time}.
    • Creates missing output directories.
    • Preserves absolute-path override behavior, matching TRX.
  • Help and tests

    • Updates help text to document the expanded contract.
    • Adds focused unit coverage for validation, placeholder resolution, sanitization, and directory handling.
    • Updates acceptance coverage for nested HTML report paths.

Example:

dotnet test --report-html --report-html-filename artifacts/run_{tfm}.html
dotnet test --report-html --report-html-filename /tmp/run.html

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copilot AI requested review from Copilot and removed request for Copilot May 23, 2026 16:04
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copilot AI requested review from Copilot and removed request for Copilot May 23, 2026 16:07
Copilot AI changed the title [WIP] Fix incompatible path semantics for report-html-filename and report-trx-filename Align HTML report filename path semantics with TRX May 23, 2026
Copilot AI requested a review from Evangelink May 23, 2026 16:09
@Evangelink Evangelink marked this pull request as ready for review May 25, 2026 09:58
Copilot AI review requested due to automatic review settings May 25, 2026 09:58
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Aligns --report-html-filename behavior with TRX report filename semantics in the Microsoft.Testing.Extensions.HtmlReport extension, expanding accepted path forms and adding placeholder resolution + write-time sanitization.

Changes:

  • Updated HTML report filename validation to accept relative/absolute paths while rejecting paths that can escape the results directory.
  • Added placeholder resolution ({pname}, {pid}, {asm}, {tfm}, {time}), leaf-name sanitization, and automatic creation of missing output directories when writing the HTML report.
  • Updated help/acceptance expectations and added unit + integration coverage for the new contract.
Show a summary per file
File Description
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/HtmlReportGeneratorCommandLineTests.cs Updates validation-focused unit tests to cover nested paths, absolute paths, traversal rejection, and drive-relative handling.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/HtmlReportEngineTests.cs Adds unit tests for relative/absolute output resolution, placeholder substitution, sanitization, and directory creation.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HtmlReportTests.cs Updates acceptance coverage to expect HTML reports generated under nested relative paths.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs Updates help output expectations to document new path/placeholder contract.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.*.xlf Regenerates localized strings for updated option description and validation messages.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/ExtensionResources.resx Updates option description and adjusts validation error wording to the new contract.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Microsoft.Testing.Extensions.HtmlReport.csproj Links in ArtifactNamingHelper to share placeholder resolution logic.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/HtmlReportGeneratorCommandLine.cs Implements updated validation rules (relative/absolute paths, traversal/escaping checks).
src/Platform/Microsoft.Testing.Extensions.HtmlReport/HtmlReportEngine.cs Implements placeholder resolution, leaf sanitization (TRX-aligned), and directory creation prior to writing.

Copilot's findings

Comments suppressed due to low confidence (1)

test/UnitTests/Microsoft.Testing.Extensions.UnitTests/HtmlReportGeneratorCommandLineTests.cs:66

  • Test name says the relative path "escapes" the results directory, but nested/../report.html doesn’t actually escape; it’s rejected because any .. segment is disallowed (matching TRX’s policy). Consider renaming the test/data row to reflect the actual rule (e.g., “contains parent-directory traversal segment”).
    [TestMethod]
    [DataRow("../report.html")]
    [DataRow("nested/../report.html")]
    public async Task IsInvalid_If_RelativePath_Escapes_TestResultsDirectory(string fileName)
    {
        var provider = new HtmlReportGeneratorCommandLine();
        Platform.Extensions.CommandLine.CommandLineOption option = provider.GetCommandLineOptions()
            .First(x => x.Name == HtmlReportGeneratorCommandLine.HtmlReportFileNameOptionName);
  • Files reviewed: 21/21 changed files
  • Comments generated: 2

- Drop the cross-platform IsDriveRelativePath heuristic from
  HtmlReportGeneratorCommandLine: on Unix ':' is a valid filename
  character and rejecting C:report.html there diverged from
  --report-trx-filename. Use Path.IsPathRooted alone (Windows-only in
  practice) to mirror TRX behavior.
- Split the nested-path data row that used '\\' into a Windows-only
  test so the assertion no longer implies cross-platform separator
  semantics that aren't implemented.
- Rename the '..' validation test to reflect the actual rule (any
  parent-directory segment is rejected; the path doesn't have to
  resolve outside the results directory).
- Gate the drive-relative-path test to Windows, matching the TRX
  unit test layout.
- Update the --info acceptance expectation for --report-html-filename
  to match the new option description (full path/placeholder
  contract), which was broken after the option description was
  expanded.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink
Copy link
Copy Markdown
Member

Addressed review feedback in 2816362 and fixed the broken --info acceptance test:

  • HtmlReportGeneratorCommandLine — dropped the cross-platform IsDriveRelativePath heuristic and now use Path.IsPathRooted only (matching TrxCommandLine). On Unix, C:report.html is a valid filename and is no longer rejected; on Windows, drive-relative paths like C:report.html are still rejected because Path.IsPathRooted returns true for them.
  • HtmlReportGeneratorCommandLineTests:
    • Split sub\report.html out of the cross-platform data row into a Windows-only test so the assertion no longer implies cross-platform separator semantics.
    • Renamed IsInvalid_If_RelativePath_Escapes_TestResultsDirectory to IsInvalid_If_RelativePath_Contains_ParentDirectorySegment to reflect the actual rule (any .. segment is rejected, even if it resolves inside the results dir).
    • Gated IsInvalid_If_HtmlFile_Uses_DriveRelativePath to Windows only (the C: form is a valid filename on Unix), matching the corresponding TRX test layout.
  • HelpInfoAllExtensionsTests — updated the --info expectation for --report-html-filename to include the new path/placeholder description, fixing the test failure on net10.0/net11.0.

Verified locally:

  • Microsoft.Testing.Extensions.UnitTests (net9.0): 15 HtmlReportGeneratorCommandLineTests pass.
  • Microsoft.Testing.Platform.Acceptance.IntegrationTests (net11.0): the 6 HelpInfoAllExtensionsTests.Help_*/Info_* tests pass.

Copilot AI review requested due to automatic review settings May 25, 2026 11:55
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 21/21 changed files
  • Comments generated: 1

Evangelink and others added 2 commits May 25, 2026 15:12
Splits the HTML filename validation errors so that the empty-argument
and empty-leaf cases use a dedicated 'must not be empty' message,
matching the TRX resource layout (TrxReportFileNameMustNotBeEmpty vs
TrxReportFileNameRelativePathMustStayUnderResultsDirectory). The
existing 'should not contain path' resource is renamed to
HtmlReportFileNameRelativePathMustStayUnderResultsDirectory so the
keys, wording, and intent stay aligned with TRX.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 25, 2026 13:25
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 21/21 changed files
  • Comments generated: 0 new

@Evangelink Evangelink merged commit 2d9cd1d into main May 25, 2026
26 checks passed
@Evangelink Evangelink deleted the copilot/mtp-cli-analysis-report-filename-path-semantics branch May 25, 2026 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[MTP CLI Analysis] report-html-filename and report-trx-filename expose incompatible path semantics

3 participants