Skip to content

Document MSTest 4.3 random test order and MTP 2.3 filter mutual exclusion #54724

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Evangelink wants to merge 3 commits into
base: main
Choose a base branch
from
+23 −3
Open

Document MSTest 4.3 random test order and MTP 2.3 filter mutual exclusion #54724

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b964516
Document MSTest 4.3 random test order in order-unit-tests article
Evangelink Jul 8, 2026
4be3778
Document --treenode-filter and --filter-uid mutual exclusion (MTP 2.3.0)
Evangelink Jul 9, 2026
0ee9e81
Address review feedback: style and version scoping
Evangelink Jul 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions docs/core/testing/microsoft-testing-platform-cli-options.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Microsoft.Testing.Platform (MTP) CLI options reference
description: Find platform and extension command-line options for MTP in one place.
author: Evangelink
ms.author: amauryleve
ms.date: 06/16/2026
ms.date: 07/09/2026
ai-usage: ai-assisted
---

Expand Down Expand Up @@ -95,7 +95,7 @@ This article gives a central entry point for MTP command-line options.
Filters the tests to run by their test node UIDs. Accepts one or more UIDs.

> [!NOTE]
> This option is available in MTP starting with version 1.8.0.
> This option is available in MTP starting with version 1.8.0. Starting with MTP 2.3.0, you can't combine `--filter-uid` with `--treenode-filter`; specifying both fails command-line validation with the `InvalidCommandLine` exit code.

- **`--help`**

Expand Down Expand Up @@ -150,6 +150,9 @@ This article gives a central entry point for MTP command-line options.

Filters the tests to run by using a tree filter expression. Tree filters offer richer matching than `--filter` for advanced scenarios.

> [!NOTE]
> Starting with MTP 2.3.0, you can't combine `--treenode-filter` with `--filter-uid`; specifying both fails command-line validation with the `InvalidCommandLine` exit code.

- **`--zero-tests-policy`**

Controls whether a run that executes no tests because every test was skipped is treated as a failure. Valid values are `allow-skipped` (default) and `strict`. With `allow-skipped`, an all-skipped run succeeds; with `strict`, it fails with exit code 8 (the behavior before 2.3.0).
Expand Down
19 changes: 18 additions & 1 deletion docs/core/testing/order-unit-tests.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Order unit tests
description: Learn how to order unit tests with .NET Core.
ms.date: 02/20/2026
ms.date: 07/08/2026
ai-usage: ai-assisted
zone_pivot_groups: unit-testing-framework-set-one
---
Expand Down Expand Up @@ -49,6 +49,23 @@ Starting with MSTest 3.6, a new runsettings option lets you run tests by test na
</RunSettings>
```

## Order randomly

In MSTest 4.3 and later, run tests in a random order to surface hidden ordering dependencies between tests. To enable random order, set the `RandomizeTestOrder` setting to **true** in your runsettings file:

```xml
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>

<MSTest>
<RandomizeTestOrder>true</RandomizeTestOrder>
</MSTest>

</RunSettings>
```

To make the random order reproducible, set the `RandomTestOrderSeed` setting to an integer seed. MSTest then replays the same order on every run. When you leave the seed unset, MSTest generates a new seed for each run. Don't combine `RandomizeTestOrder` with `OrderTestsByNameInClass`, because MSTest applies only one ordering mode at a time. For more information, see [Configure MSTest](unit-testing-mstest-configure.md).

:::zone-end
:::zone pivot="xunit"

Expand Down
Loading