Skip to content

Document MSTest 4.3 random test order and MTP 2.3 filter mutual exclusion#54724

Open
Evangelink wants to merge 3 commits into
mainfrom
dev/amauryleve/review-mtp-mstest-docs
Open

Document MSTest 4.3 random test order and MTP 2.3 filter mutual exclusion#54724
Evangelink wants to merge 3 commits into
mainfrom
dev/amauryleve/review-mtp-mstest-docs

Conversation

@Evangelink

@Evangelink Evangelink commented Jul 9, 2026

Copy link
Copy Markdown
Member

Summary

Follow-up review of MTP 2.3.0/2.3.1 and MSTest 4.3.0 feature coverage against the docs. The existing documentation (largely from #54187, #54480, and #54633) was already comprehensive; this PR fills the two remaining gaps found during the audit.

Changes

  • order-unit-tests.md — Added an "Order randomly" section to the MSTest zone covering the MSTest 4.3 RandomizeTestOrder and RandomTestOrderSeed runsettings options. Previously the article only documented OrderTestsByNameInClass (3.6), so the random-order feature wasn't discoverable from the dedicated ordering article (it was only in the Configure MSTest reference tables).
  • microsoft-testing-platform-cli-options.md — Documented the new --treenode-filter--filter-uid mutual exclusion introduced in MTP 2.3.0 (testfx#9458); added reciprocal notes on both option entries.

Audit result

Every other feature in the MTP 2.3.0/2.3.1 (Changelog-Platform.md) and MSTest 4.3.0 (Changelog.md) changelogs was verified as already documented:

  • Analyzers MSTEST0064–0071 (files, overview table, and TOC)
  • New assertions (AreSequenceEqual/AreNotSequenceEqual incl. SequenceOrder.InAnyOrder, AreEquivalent, ContainsAll/DoesNotContainAll, AreAllNotNull/AreAllDistinct/AreAllOfType, AddValueFormatter, Span<T>/Memory<T> HasCount overloads, structured messages)
  • Condition attributes ([MemberCondition], [ArchitectureCondition], [ExecutableCondition]), class-level [Retry], [AssemblyFixtureProvider], TestContext.Properties lifecycle flow
  • Experimental APIs (reflection source generator, ITestFilter/[TestFilterProviderAttribute], TestRun.Current)
  • MSBuild parallelization properties and random-order config
  • MTP 2.3 CLI/extension surface: --ansi, --progress, crash/hangdump -if-supported options, --list-tests json, --zero-tests-policy, HtmlReport/JUnitReport/CtrfReport/GitHubActionsReport/VideoRecorder/PackagedApp extensions, Azure DevOps publishing/attachments/coverage, Logging bridge, testconfig.json environmentVariables/IConfiguration, two-stage Ctrl+C, LLM-env behavior

MTP 2.3.1 is a single forward-compat bug fix with no user-facing doc need.

ai-usage: ai-assisted


Internal previews

📄 File 🔗 Preview link
docs/core/testing/microsoft-testing-platform-cli-options.md Microsoft.Testing.Platform (MTP) CLI options reference
docs/core/testing/order-unit-tests.md Order unit tests

Evangelink and others added 2 commits July 9, 2026 14:53
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 9, 2026 13:11
@dotnetrepoman dotnetrepoman Bot added this to the July 2026 milestone Jul 9, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR closes two remaining documentation gaps identified in an audit of MSTest 4.3 and Microsoft.Testing.Platform (MTP) 2.3 by updating existing reference articles.

Changes:

  • Added an MSTest-focused “Order randomly” section to document RandomizeTestOrder and RandomTestOrderSeed in the dedicated ordering article.
  • Documented --treenode-filter--filter-uid mutual exclusion in the MTP CLI options reference, with notes on both options.
Show a summary per file
File Description
docs/core/testing/order-unit-tests.md Adds MSTest 4.3 random ordering guidance to the test ordering article.
docs/core/testing/microsoft-testing-platform-cli-options.md Adds mutual exclusion notes between --filter-uid and --treenode-filter for MTP 2.3.0+.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 4

Comment thread docs/core/testing/order-unit-tests.md Outdated
Comment thread docs/core/testing/order-unit-tests.md Outdated
Comment thread docs/core/testing/microsoft-testing-platform-cli-options.md Outdated
Comment thread docs/core/testing/microsoft-testing-platform-cli-options.md Outdated
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink Evangelink enabled auto-merge (squash) July 9, 2026 17:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants