Skip to content

dotnet test: truncate large failing-host output in summary (#52297)#55300

Open
Evangelink wants to merge 2 commits into
mainfrom
dev/amauryleve/sturdy-waffle
Open

dotnet test: truncate large failing-host output in summary (#52297)#55300
Evangelink wants to merge 2 commits into
mainfrom
dev/amauryleve/sturdy-waffle

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Fixes #52297

Problem

When a Microsoft.Testing.Platform test host exits with a failure, it often prints its entire command-line help (hundreds of lines — over 700 for xUnit) to standard output. The classic trigger is an invalid argument value, e.g. dotnet test --show-live-output true:

Option '--show-live-output' has invalid arguments: Invalid value 'true' (must be one of: 'on', 'off')
  Usage SharedAppHost.exe [option providers] [extension option providers]
...<700 more lines of help>...

dotnet test echoed all of it verbatim in the executable summary (AppendExecutableSummary), so the actual error — which appears at the top — was buried under a wall of noise that takes a lot of scrolling to find.

Fix

Truncate the standard output rendered in the summary: when it exceeds 40 lines, keep the first 30 lines (where the real error lives) and the last 10 lines (trailing diagnostics), and collapse the middle with a localized marker:

... output truncated, 660 lines omitted ...

Standard error is left untouched. Output that already fits within the budget is echoed unchanged.

Head+tail elision is used (rather than the existing live-stream tail) because it's robust across failure modes: it preserves the error at the top for the invalid-argument case and trailing output at the bottom for crashes/exceptions.

Changes

  • TerminalTestReporter.cs — new TruncateOutputForSummary helper, wired into AppendExecutableSummary.
  • CliCommandStrings.resx — new localized TestApplicationOutputTruncated string; all 13 .xlf files regenerated via /t:UpdateXlf.
  • TerminalTestReporterTests.cs — 3 tests: small output unchanged, large output head/tail/marker behavior, and end-to-end through AssemblyRunCompleted.

Verification

  • dotnet build of dotnet.csproj and dotnet.Tests.csproj: 0 warnings, 0 errors.
  • Truncation algorithm validated end-to-end against the test assertions (head preserved, tail preserved, middle dropped, omitted-count correct, small input unchanged).

When a Microsoft.Testing.Platform test host exits with a failure it often
prints its entire command-line help (hundreds of lines) to standard output -
for example when an argument has an invalid value. dotnet test echoed all of
it verbatim in the executable summary, burying the actual error (which appears
at the top) under a wall of noise.

Truncate the standard output shown in the summary to the first 30 and last 10
lines with a '... output truncated, N lines omitted ...' marker in between, so
the error stays easy to find while trailing diagnostics are preserved.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 75872aa1-648f-46d8-b49f-1b93ce271d0a
Copilot AI review requested due to automatic review settings July 15, 2026 14:26
@Evangelink
Evangelink requested a review from a team as a code owner July 15, 2026 14:26
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
2 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves the dotnet test terminal summary for Microsoft.Testing.Platform (MTP) runs by truncating excessively large standard output payloads (e.g., full test-host help dumps on invalid arguments) so the actionable error at the top remains visible.

Changes:

  • Add TerminalTestReporter.TruncateOutputForSummary and apply it when rendering failing-host stdout in AppendExecutableSummary.
  • Introduce a localized truncation marker string (TestApplicationOutputTruncated) and propagate it through regenerated .xlf files.
  • Add unit tests covering small-output passthrough, large-output head/tail truncation, and an end-to-end summary assertion via AssemblyRunCompleted.
Show a summary per file
File Description
test/dotnet.Tests/CommandTests/Test/TerminalTestReporterTests.cs Adds coverage for truncation behavior and summary rendering.
src/Cli/dotnet/Commands/Test/MTP/Terminal/TerminalTestReporter.cs Implements and wires stdout truncation into the failure summary output.
src/Cli/dotnet/Commands/CliCommandStrings.resx Adds the new localized truncation marker resource string.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.cs.xlf Regenerated localization entry for TestApplicationOutputTruncated.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.de.xlf Regenerated localization entry for TestApplicationOutputTruncated.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.es.xlf Regenerated localization entry for TestApplicationOutputTruncated.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.fr.xlf Regenerated localization entry for TestApplicationOutputTruncated.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.it.xlf Regenerated localization entry for TestApplicationOutputTruncated.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.ja.xlf Regenerated localization entry for TestApplicationOutputTruncated.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.ko.xlf Regenerated localization entry for TestApplicationOutputTruncated.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.pl.xlf Regenerated localization entry for TestApplicationOutputTruncated.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.pt-BR.xlf Regenerated localization entry for TestApplicationOutputTruncated.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.ru.xlf Regenerated localization entry for TestApplicationOutputTruncated.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.tr.xlf Regenerated localization entry for TestApplicationOutputTruncated.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.zh-Hans.xlf Regenerated localization entry for TestApplicationOutputTruncated.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.zh-Hant.xlf Regenerated localization entry for TestApplicationOutputTruncated.

Copilot's findings

  • Files reviewed: 16/16 changed files
  • Comments generated: 0

@Evangelink
Evangelink enabled auto-merge July 15, 2026 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dotnet test argument errors buried in noise

2 participants