fix(test): repair four unit tests drifting behind source and upstream APIs - #610
Draft
jariy17 wants to merge 1 commit into
Draft
fix(test): repair four unit tests drifting behind source and upstream APIs#610jariy17 wants to merge 1 commit into
jariy17 wants to merge 1 commit into
Conversation
… APIs TestAsyncMode multi-agent/bidi callbacks: both tests read the private registry._registered_callbacks, which now holds strands' _CallbackEntry wrapper objects rather than raw functions, so iscoroutinefunction() was always False. Switched to the public registry.get_callbacks_for(event), matching the sibling async tests that already pass. This also makes the bidi init 'not any(iscoroutinefunction(...))' assertion meaningful -- it previously passed only because the wrapper is never a coroutine fn. EvaluatorOutput validator: the error message claimed 'Either label, value, or errorCode must be set', but the validator only accepts label or errorCode and never inspects value. Corrected the message to match the documented and implemented behaviour. test_evaluation_with_empty_trajectory: run_evaluations() returns a single EvaluationReport, not a list, so reports[0] raised TypeError. Dropped the index and fixed the same stale pattern in the README, which would have handed users the same TypeError.
Contributor
✅ No Breaking Changes DetectedNo public API breaking changes found in this PR. |
Contributor
|
Claude Security Review: no high-confidence findings. (run) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the 4 pre-existing unit-test failures on
mainthat #609 does not cover. All four are tests (or a message/doc) that fell behind source and upstream changes; none are product bugs.1 + 2.
TestAsyncModemulti-agent and bidi callbackstest_agentcore_memory_session_manager.py:3726and:3756reached into the private registry dict:Upstream Strands now stores callbacks wrapped in a
_CallbackEntry:iscoroutinefunction()on the wrapper is alwaysFalse, so the assertion could never hold. The source is correct —session_manager.py:989-991does register async_offloadcallbacks.The fix is the accessor the passing sibling tests already use (lines 3636–3714): public
registry.get_callbacks_for(event), which unwraps.Side benefit: the bidi-init assertion at
:3751isnot any(iscoroutinefunction(...)). It was passing for the wrong reason — the wrapper is never a coroutine function, so it would have passed even if the callback were wrongly async. Against unwrapped callbacks it now actually tests what it claims.3.
EvaluatorOutputvalidator message — source fixThe message advertised a
valueescape hatch that does not exist:The class docstring (
models.py:63-65) and every caller insrc/agree thatlabelis required unlesserrorCodeis set, so the message was the defect, not the logic. Corrected to"label is required for success responses; ...", which is also what the test expected.4.
test_evaluation_with_empty_trajectoryrun_evaluations()returns a single report now, verified against the installed library:Dropped the
[0]. Also fixed the same stale pattern inREADME.md(2 occurrences) — the documentedreports = ...; report = reports[0]snippet would hand users the identicalTypeError.Testing
uv run pytest tests/on Python 3.10 → 2934 passed, 10 skipped, 4 xpassed, 0 failedTestAsyncModein isolation → 8 passedruff check src/ tests/→ all checks passed;ruff formatapplied (the shortened message now fits one line — verifiedmainwas format-clean beforehand, so that reflow is mine)Ordering
This branch is cut from
main, so on its own CI still hits the 2deepevalcollection errors that #609 fixes — verified with a cleanuv sync --dev:Kept as a draft for that reason. #609 then this turns
ci.ymlgreen; merge order the other way leaves 4 failures. Happy to rebase on #609 if you'd prefer to see this one green before review.Related: #608 (integration workflow), #609 (dev-group pin).