Skip to content

Comments

support task and scorer errors#39

Merged
realark merged 2 commits intomainfrom
ark/eval-errors
Feb 19, 2026
Merged

support task and scorer errors#39
realark merged 2 commits intomainfrom
ark/eval-errors

Conversation

@realark
Copy link
Collaborator

@realark realark commented Feb 18, 2026 •
edited
Loading

  • tag the span as an error and attach the stack trace when the task function throws
  • allow scorers to implement custom scoring logic for errors (default is to score 0 using the scorer name)
  • also allow scorers to hook up a fallback scorer if the score method itself throws (default is also to score 0 using the scorer name)

@realark realark added the enhancement New feature or request label Feb 18, 2026
@realark realark marked this pull request as ready for review February 18, 2026 02:39
@realark realark merged commit d6f94ca into main Feb 19, 2026
1 check passed
@realark realark deleted the ark/eval-errors branch February 19, 2026 00:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@realark