Source-only validation tests fail with NuGet `401 Unauthorized` in offline legs after PAT-less auth migration

[ellahathaway]

Summary

Source-only validation test legs are failing at NuGet restore (not on test assertions) with NU1301 ... 401 (Unauthorized) against the internal Azure DevOps feeds (dotnet10-internal, dotnet10-internal-transport). The tests never run — restore fails first.

Observed in dotnet-unified-build (internal) build 3021033 (main, commit 7e07d48).

Failing legs

All failing "Run Tests" steps are Offline legs:

• SB_CentOSStream10_Offline_CurrentSourceBuiltSdk_Validation_x64
• SB_CentOSStream10_Offline_PreviousSourceBuiltSdk_Validation_x64
• SB_Fedora43_Offline_CurrentSourceBuiltSdk_Validation_x64
• SB_CentOSStream10_Offline_MsftSdk_Validation_x64
• SB_Fedora43_Offline_MsftSdk_Validation_x64
• SB_AlmaLinux8_Offline_MsftSdk_Validation_x64

No Online leg''s tests failed.

Error

test/Microsoft.DotNet.SourceBuild.Tests/Microsoft.DotNet.SourceBuild.Tests.csproj : error NU1301:
  Unable to load the service index for source
  https://pkgs.dev.azure.com/dnceng/internal/_packaging/dotnet10-internal/nuget/v3/index.json.
  Response status code does not indicate success: 401 (Unauthorized).

56 errors total. Affected projects: Microsoft.DotNet.SourceBuild.Tests, Microsoft.DotNet.Tests, Microsoft.DotNet.Reproducibility.Tests, TestUtilities, Microsoft.DotNet.UnifiedBuild.Tasks.

Root cause

Two auth-refactoring changes landed in this build that migrated internal-feed auth from an embedded PAT to the PAT-less Azure Artifacts credential provider:

• dotnet/dotnet#7686 — removed the embedded PAT: & $setupScript $nugetConfig $env:Token -> & $setupScript $nugetConfig, and dropped env: Token: $(dn-bot-dnceng-artifact-feeds-rw).
• dotnet/dotnet#7688 — added NuGetAuthenticate@1.

Previously the PAT was written directly into each NuGet.config as cleartext credentials, so restore worked from any process. Now auth relies on the credential provider (env-var / session-token based), configured once at the job level.

For offline validation legs, the test restore is wrapped in sudo -E (eng/pipelines/templates/jobs/vmr-build.yml, the Run Tests step: customPreBuildArgs="$customPreBuildArgs sudo -E" when runOnline == False). Under sudo, $HOME changes to root''s home and the credential provider can''t locate its provisioned credentials -> 401. Online legs restore without sudo, so auth still works. The build step itself is unaffected because source-only builds don''t consume internal feeds — only the tests do.

Suggested fix

1. Preserve the credential-provider environment across the sudo boundary for the offline test restore (e.g. sudo --preserve-env=HOME,NUGET_PLUGIN_PATHS,NUGET_CREDENTIALPROVIDER_*,VSS_NUGET_*,SYSTEM_ACCESSTOKEN or a sudoers env_keep), or
2. Restore embedded credentials for the internal test NuGet.config specifically (targeted revert of #7686 for the test-config copy at vmr-build.yml).

Links

• Build: https://dev.azure.com/dnceng/internal/_build/results?buildId=3021033
• Related PRs: Fix internal feeds auth: Remove Token env variable from NuGet feeds setup dotnet#7686, Add NuGetAuthenticate task for authentication dotnet#7688

[ellahathaway]

@ViktorHofer - can you please take a look?