From b964516f6ab46bea31fe928b9ddb08cfa1bf463c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Amaury=20Lev=C3=A9?= Date: Wed, 8 Jul 2026 22:28:23 +0200 Subject: [PATCH 1/3] Document MSTest 4.3 random test order in order-unit-tests article Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/core/testing/order-unit-tests.md | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/docs/core/testing/order-unit-tests.md b/docs/core/testing/order-unit-tests.md index f3a0ce4a372b6..31e2bc5819e27 100644 --- a/docs/core/testing/order-unit-tests.md +++ b/docs/core/testing/order-unit-tests.md @@ -1,7 +1,7 @@ --- title: Order unit tests description: Learn how to order unit tests with .NET Core. -ms.date: 02/20/2026 +ms.date: 07/08/2026 ai-usage: ai-assisted zone_pivot_groups: unit-testing-framework-set-one --- @@ -49,6 +49,23 @@ Starting with MSTest 3.6, a new runsettings option lets you run tests by test na ``` +## Order randomly + +Starting with MSTest 4.3, to surface hidden ordering dependencies between tests, you can run tests in a random order. Set the `RandomizeTestOrder` setting to **true** in your runsettings file: + +```xml + + + + + true + + + +``` + +To make the random order reproducible across runs, set the `RandomTestOrderSeed` setting to an integer seed. When the seed is unset, a new seed is used for each run. You can't combine `RandomizeTestOrder` with `OrderTestsByNameInClass`. For more information, see [Configure MSTest](unit-testing-mstest-configure.md). + :::zone-end :::zone pivot="xunit" From 4be3778c8b627c6aeec6cc9c4adc923021d48ca0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Amaury=20Lev=C3=A9?= Date: Thu, 9 Jul 2026 14:54:58 +0200 Subject: [PATCH 2/3] Document --treenode-filter and --filter-uid mutual exclusion (MTP 2.3.0) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .../core/testing/microsoft-testing-platform-cli-options.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/core/testing/microsoft-testing-platform-cli-options.md b/docs/core/testing/microsoft-testing-platform-cli-options.md index 5d018601b41c4..b6bfd6fa2fe3a 100644 --- a/docs/core/testing/microsoft-testing-platform-cli-options.md +++ b/docs/core/testing/microsoft-testing-platform-cli-options.md @@ -3,7 +3,7 @@ title: Microsoft.Testing.Platform (MTP) CLI options reference description: Find platform and extension command-line options for MTP in one place. author: Evangelink ms.author: amauryleve -ms.date: 06/16/2026 +ms.date: 07/09/2026 ai-usage: ai-assisted --- @@ -95,7 +95,7 @@ This article gives a central entry point for MTP command-line options. Filters the tests to run by their test node UIDs. Accepts one or more UIDs. > [!NOTE] - > This option is available in MTP starting with version 1.8.0. + > This option is available in MTP starting with version 1.8.0. You can't combine `--filter-uid` with `--treenode-filter`. - **`--help`** @@ -150,6 +150,9 @@ This article gives a central entry point for MTP command-line options. Filters the tests to run by using a tree filter expression. Tree filters offer richer matching than `--filter` for advanced scenarios. + > [!NOTE] + > You can't combine `--treenode-filter` with `--filter-uid`. Starting with MTP 2.3.0, specifying both options fails command-line validation with the `InvalidCommandLine` exit code. + - **`--zero-tests-policy`** Controls whether a run that executes no tests because every test was skipped is treated as a failure. Valid values are `allow-skipped` (default) and `strict`. With `allow-skipped`, an all-skipped run succeeds; with `strict`, it fails with exit code 8 (the behavior before 2.3.0). From 0ee9e810e7324abebcf22d6fe026de9177cc81ac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Amaury=20Lev=C3=A9?= Date: Thu, 9 Jul 2026 18:46:59 +0200 Subject: [PATCH 3/3] Address review feedback: style and version scoping Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/core/testing/microsoft-testing-platform-cli-options.md | 4 ++-- docs/core/testing/order-unit-tests.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/core/testing/microsoft-testing-platform-cli-options.md b/docs/core/testing/microsoft-testing-platform-cli-options.md index b6bfd6fa2fe3a..6d01498c010e6 100644 --- a/docs/core/testing/microsoft-testing-platform-cli-options.md +++ b/docs/core/testing/microsoft-testing-platform-cli-options.md @@ -95,7 +95,7 @@ This article gives a central entry point for MTP command-line options. Filters the tests to run by their test node UIDs. Accepts one or more UIDs. > [!NOTE] - > This option is available in MTP starting with version 1.8.0. You can't combine `--filter-uid` with `--treenode-filter`. + > This option is available in MTP starting with version 1.8.0. Starting with MTP 2.3.0, you can't combine `--filter-uid` with `--treenode-filter`; specifying both fails command-line validation with the `InvalidCommandLine` exit code. - **`--help`** @@ -151,7 +151,7 @@ This article gives a central entry point for MTP command-line options. Filters the tests to run by using a tree filter expression. Tree filters offer richer matching than `--filter` for advanced scenarios. > [!NOTE] - > You can't combine `--treenode-filter` with `--filter-uid`. Starting with MTP 2.3.0, specifying both options fails command-line validation with the `InvalidCommandLine` exit code. + > Starting with MTP 2.3.0, you can't combine `--treenode-filter` with `--filter-uid`; specifying both fails command-line validation with the `InvalidCommandLine` exit code. - **`--zero-tests-policy`** diff --git a/docs/core/testing/order-unit-tests.md b/docs/core/testing/order-unit-tests.md index 31e2bc5819e27..007e95a18c0c7 100644 --- a/docs/core/testing/order-unit-tests.md +++ b/docs/core/testing/order-unit-tests.md @@ -51,7 +51,7 @@ Starting with MSTest 3.6, a new runsettings option lets you run tests by test na ## Order randomly -Starting with MSTest 4.3, to surface hidden ordering dependencies between tests, you can run tests in a random order. Set the `RandomizeTestOrder` setting to **true** in your runsettings file: +In MSTest 4.3 and later, run tests in a random order to surface hidden ordering dependencies between tests. To enable random order, set the `RandomizeTestOrder` setting to **true** in your runsettings file: ```xml @@ -64,7 +64,7 @@ Starting with MSTest 4.3, to surface hidden ordering dependencies between tests, ``` -To make the random order reproducible across runs, set the `RandomTestOrderSeed` setting to an integer seed. When the seed is unset, a new seed is used for each run. You can't combine `RandomizeTestOrder` with `OrderTestsByNameInClass`. For more information, see [Configure MSTest](unit-testing-mstest-configure.md). +To make the random order reproducible, set the `RandomTestOrderSeed` setting to an integer seed. MSTest then replays the same order on every run. When you leave the seed unset, MSTest generates a new seed for each run. Don't combine `RandomizeTestOrder` with `OrderTestsByNameInClass`, because MSTest applies only one ordering mode at a time. For more information, see [Configure MSTest](unit-testing-mstest-configure.md). :::zone-end :::zone pivot="xunit"