Add dotnet test environment variable parity#55325
Conversation
Pass -e values through opted-in MSBuild targets and preserve target modifications when launching MTP test applications. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5a61fea9-5cbe-4d4d-8740-d306bc0a13c0
|
Azure Pipelines: Successfully started running 2 pipeline(s). 1 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
This PR adds parity between dotnet test -e NAME=VALUE and dotnet run -e for MAUI-style (device) projects by flowing parsed environment variables through the Microsoft Testing Platform (MTP) pipeline. Environment variables can now be consumed during build/deploy/device-selection and ComputeRunArguments via @(RuntimeEnvironmentVariable) (for projects that opt in via RuntimeEnvironmentVariableSupport), while still allowing MSBuild targets to mutate the final set of variables applied when launching the test process.
Changes:
- Extend MTP build/test models to carry environment variables end-to-end, including preserving target-side modifications before process launch.
- Add out-of-process build injection for opted-in projects via a temporary
dotnet-test-env.props(matching thedotnet runapproach, but with a distinct filename). - Update specs and add focused integration/unit coverage to validate build/deploy/
ComputeRunArgumentspropagation and mutation.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| test/TestAssets/TestProjects/DotnetTestDevices/Program.cs | Logs runtime environment variables at execution to validate final process environment. |
| test/TestAssets/TestProjects/DotnetTestDevices/DotnetTestDevices.csproj | Opts into RuntimeEnvironmentVariableSupport and adds targets to log/mutate @(RuntimeEnvironmentVariable) during build/deploy/run-arg computation. |
| test/dotnet.Tests/CommandTests/Test/TestApplicationHandlerTests.cs | Updates tests to align with TestOptions and TestModule environment variable plumbing changes. |
| test/dotnet.Tests/CommandTests/Test/GivenDotnetTestSelectsDevice.cs | Adds integration coverage asserting env vars reach build/deploy/ComputeRunArguments and that target mutations affect the launched process. |
| test/dotnet.Tests/CommandTests/Run/GivenDotnetRunSelectsDevice.cs | Minor cleanup in an existing env-props assertion path. |
| test/dotnet.Tests/CommandTests/Run/EnvironmentVariablesToMSBuildTests.cs | Adds a unit test validating that generated props files contain well-formed RuntimeEnvironmentVariable items. |
| src/Cli/dotnet/Commands/Test/MTP/TestModulesFilterHandler.cs | Captures -e/--environment values and propagates them to TestModule instances created from module paths. |
| src/Cli/dotnet/Commands/Test/MTP/TestApplication.cs | Applies TestModule.EnvironmentVariables (including MSBuild target mutations) with precedence over launch profile variables. |
| src/Cli/dotnet/Commands/Test/MTP/SolutionAndProjectUtility.cs | Passes env vars into device selection and in-proc deploy/ComputeRunArguments, and reads back mutated items when supported. |
| src/Cli/dotnet/Commands/Test/MTP/Options.cs | Moves env vars from TestOptions into BuildOptions to reflect build/deploy ownership. |
| src/Cli/dotnet/Commands/Test/MTP/MSBuildUtility.cs | Injects env vars into out-of-proc project builds via a temporary props file when the project opts in. |
| src/Cli/dotnet/Commands/Test/MTP/Models.cs | Extends TestModule to include environment variables for the eventual launched process. |
| src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs | Wires env vars through build/device-list flows via BuildOptions. |
| src/Cli/dotnet/Commands/Run/RunCommand.cs | Updates props file creation to explicitly pass the dotnet-run-env.props filename. |
| src/Cli/dotnet/Commands/Run/EnvironmentVariablesToMSBuild.cs | Generalizes props generation to accept a props filename and improves path handling for intermediate output paths. |
| documentation/specs/dotnet-run-for-maui.md | Updates the MAUI command spec to document dotnet test env var parity and the build-time injection behavior/constraints. |
Evangelink
left a comment
Evangelink
left a comment
There was a problem hiding this comment.
Found two correctness issues: environment variables are not available during device discovery, and build-time injection silently displaces an existing CustomBeforeMicrosoftCommonProps import. I did not approve because both affect supported invocation paths.
Note
This review was generated with GitHub Copilot and validated with a local PR build and focused reproductions.
Evangelink
left a comment
Evangelink
left a comment
There was a problem hiding this comment.
Inline details for the two correctness issues described in my preceding review.
Note
This review was generated with GitHub Copilot and validated with a local PR build and focused reproductions.
| isInteractive, | ||
| msbuildArgs, | ||
| ImmutableDictionary<string, string>.Empty, | ||
| buildOptions.EnvironmentVariables, |
There was a problem hiding this comment.
These values never reach ComputeAvailableDevices. RunCommandSelector only consumes _environmentVariables in TryDeployToDevice; SelectDeviceForTfm calls TrySelectDevice and then disposes the selector. I reproduced this with an opted-in project whose BeforeTargets="ComputeAvailableDevices" target errors when @(RuntimeEnvironmentVariable) is empty: dotnet test --list-devices -e FOO=BAR still hit the error. Please add the items before TryComputeAvailableDevices (gated by the capability) and cover an environment-dependent device list.
There was a problem hiding this comment.
ComputeAvailableDevices doesn't need @(RuntimeEnvironmentVariable) items.
This target is for the Android/iOS workload to return a list of attached simulators/emulators and devices. Nothing would use the item group.
| buildOptions.EnvironmentVariables, | ||
| "dotnet-test-env.props", | ||
| project.GetPropertyValue(Constants.IntermediateOutputPath)); | ||
| parsedMSBuildArgs = EnvironmentVariablesToMSBuild.AddPropsFileToArgs(parsedMSBuildArgs, envPropsFile); |
There was a problem hiding this comment.
AddPropsFileToArgs replaces an existing CustomBeforeMicrosoftCommonProps value rather than composing with it. I reproduced dotnet test -e FOO=BAR -p:CustomBeforeMicrosoftCommonProps=custom.props on this asset: the command succeeded, but a marker target from custom.props never ran, so the user-provided import was silently dropped. This property is an MSBuild extension point; please preserve/import the original file as well as the generated props file and add a regression test.
There was a problem hiding this comment.
The dotnet run implementation has the same problem. I would like to avoid the temporary file entirely...
There is some discussion we had on this here:
I filed a tracking issue to follow up on this:
dotnet test -e NAME=VALUEalready passed variables to test processes, but MAUI-style projects could not consume them during build, device selection, deployment, orComputeRunArgumentslikedotnet runcan.This change carries parsed environment variables through the MTP project pipeline, exposes them as
@(RuntimeEnvironmentVariable)items for projects that declareRuntimeEnvironmentVariableSupport, and preserves target-side modifications when launching the test process. Out-of-process project builds use the same temporary props mechanism asdotnet run; solution builds intentionally skip build-time injection because capability opt-in cannot be applied independently through one global MSBuild property.The MAUI command specification and focused integration coverage are updated for build, deployment, run-argument mutation, and final process output.
Validation:
build.cmd /p:NativeAotSupported=false /p:PublishAot=false)dotnet runregression testsNativeAOT publishing was disabled because the validation machine does not have the Visual Studio C++ workload required by the finalizer project.