Skip to content

Proposal: capture assertion call-site (file/line/member) on AssertFailedException #8279

@Evangelink

Description

@Evangelink

Summary

Add optional caller-info properties to AssertFailedException (and AssertInconclusiveException), populated from [CallerFilePath] / [CallerLineNumber] / [CallerMemberName] parameters that every public Assert / CollectionAssert / StringAssert API would forward to the exception constructor.

This would let any consumer (the AzDO reporter, the terminal reporter, IDE test explorers, TRX writers, custom log enrichers, …) read the assertion's call site directly from the exception, instead of heuristically parsing Exception.StackTrace to skip framework frames.

This is a design suggestion — it should probably be promoted to its own RFC. It is intentionally not bundled with RFC 012; the two are complementary but solve different problems and have different API-surface implications.


Why

Today: stack-trace parsing is the only option

Every consumer that wants to point at the user's call site (the AzDO reporter is the most visible example — see #6925, #8278) has to:

  1. Read Exception.StackTrace as a string.
  2. Apply a regex to extract (file, line, method) per frame.
  3. Apply a heuristic to decide which frames are "framework" and should be skipped (the one this exception is reporting).
  4. Hope that the chosen frame is the user's intent — which fails for wrapper/helper methods, custom assertion combinators, or any case where the user's "true" call site is not the frame immediately above the Assert call.

This is brittle. It also leaks the framework's file layout into every consumer.

[StackTraceHidden] (#8277) helps but doesn't close the loop

On .NET 6+, marking the Assert APIs with [StackTraceHidden] removes framework frames from the stack — which means consumers can pick frame 0 and be done. But:

  • It doesn't help on netstandard2.0 / .NET Framework.
  • It doesn't help with wrapper/helper methods. If a team writes MyAssertHelpers.AssertOrderIsValid(order) that internally calls Assert.AreEqual(...), frame 0 is MyAssertHelpers.AssertOrderIsValid, not the test method that called the helper. Stack-trace inspection has no way to tell those apart from a "real" user frame.

Caller-info on the exception solves both

If the public Assert APIs accept [CallerFilePath] / [CallerLineNumber] / [CallerMemberName] and stash them on AssertFailedException, then:

  • The reporter just reads ex.CallerFilePath / ex.CallerLineNumber. No string parsing, no heuristics, no file-layout coupling.
  • A wrapper method MyAssertHelpers.AssertOrderIsValid(order) can forward its own caller info to the Assert call, so the exception carries the test's location rather than the helper's. That is a strict correctness improvement over any stack-walk approach.
  • Works on every TFM, including .NET Framework.
  • Works for every .NET language that supports the caller-info attributes — that's C#, F# (since F# 4.0), and VB. (Note: this is broader than RFC 012's call-site-expression line, which relies on CallerArgumentExpression and is therefore C#-only.)

How

Public API additions

On Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException (and AssertInconclusiveException):

public string? CallerFilePath { get; }
public int CallerLineNumber { get; }
public string? CallerMemberName { get; }

Plus a constructor (or constructor overload) that accepts them. Existing constructors continue to work — when caller info is not supplied, the properties are null / 0 and consumers fall back to the stack-trace heuristic.

Plumbing through every Assert API

Each public Assert API gains three optional parameters at the end of its parameter list:

public static void AreEqual<T>(
    T? expected,
    T? actual,
    string? message = null,
    [CallerArgumentExpression(nameof(expected))] string? expectedExpression = null,
    [CallerArgumentExpression(nameof(actual))]   string? actualExpression   = null,
    [CallerFilePath]   string? callerFilePath   = null,
    [CallerLineNumber] int     callerLineNumber = 0,
    [CallerMemberName] string? callerMemberName = null);

These are then forwarded into the AssertFailedException constructor when the assertion fails.

Because RFC 012 is already plumbing [CallerArgumentExpression] through every assertion API, the incremental cost of adding three more optional parameters per API is small — same files, same touchpoints, additional rows in PublicAPI.Unshipped.txt.

Consumer changes

AzureDevOpsReporter.GetErrorText (and equivalent code in the terminal reporter / TRX writer / etc.) becomes:

if (exception is AssertFailedException afex && afex.CallerFilePath is not null)
{
    // Use afex.CallerFilePath / afex.CallerLineNumber directly.
}
else
{
    // Existing stack-trace walking heuristic (fallback for older callers and non-MSTest exceptions).
}

Optional enhancement: a typed AssertionCallSite

Wrap the three properties into a single immutable value type (AssertionCallSite { FilePath, LineNumber, MemberName }) and expose AssertFailedException.CallSite. Slightly nicer API and forward-compatible if we ever want to add more fields (e.g., column number once [CallerColumnNumber] exists).


Impact

API surface

  • 2 exception types gain ~3 properties each + 1 new constructor overload each → 8 entries in PublicAPI.Unshipped.txt.
  • ~60 public assertion methods on Assert / CollectionAssert / StringAssert gain 3 optional parameters each. Source-compatible; the binary-compat story for adding trailing optional parameters is well understood and acceptable as long as we don't reorder existing ones.

Performance

  • Caller info attributes are compiled into constant strings at the call site — zero runtime cost on the success path.
  • On failure, three extra reference-typed fields on the exception. Negligible.

Risk / migration

  • Source compat: wrapper methods that already supply named arguments (Assert.AreEqual(expected: 1, actual: 2, message: "...")) keep working. Overload resolution is unaffected by trailing optional parameters with default values.
  • Binary compat: adding optional parameters to existing methods is a binary-breaking change in the strictest sense (signature changes), so this needs to ship in a major version bump or be implemented via new overloads. Standard MSTest practice for caller-info additions in the past has been to add overloads; we should follow the same path.
  • Wrapper methods: to get the best behavior, wrappers need to declare their own [CallerFilePath]/[CallerLineNumber]/[CallerMemberName] parameters and forward them. Wrappers that don't do this get the wrapper's location captured (i.e. the same as today's "frame 0 after [StackTraceHidden]" behavior — never worse).
  • F# / VB: caller-info attributes are supported in F# (since 4.0) and VB. Both languages benefit. Unlike [CallerArgumentExpression], there is no C#-only constraint here.

Why this is not in scope of RFC 012

RFC 012 standardizes the content of Exception.Message. This proposal extends the shape of Exception itself with structured metadata about where the assertion was invoked. They're additive and could happily ship in either order, but they involve different reviewers (message UX vs. public-API surface) and should be debated independently.


Open questions

  1. New constructor overloads vs. modifying existing ones (binary-compat policy)?
  2. Three properties vs. a single AssertionCallSite value type?
  3. Should we also capture caller info for Assert.Inconclusive / Assert.Fail (where there is no "operand")? Probably yes for parity.
  4. Promote to a full RFC?

Related: #6925, #8277, #8278.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/assertionAssert / StringAssert / CollectionAssert APIs.needs/designNeeds design / proposal before implementation.type/discussionOpen discussion / brainstorming.
    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions