feat(eval): add --model-provider to imperative evaluator create/update

[jariy17]

What

Adds --model-provider (Bedrock default / OpenResponses) to the imperative agentcore eval evaluator llm-as-a-judge create and update commands, and makes update provider-safe.

Verification

• bun test .../sharedFlags.test.tsx .../core/evalEvaluatorProvider.test.tsx — 15 pass (provider resolution, arm construction, and the update-safety matrix: model-only Bedrock preserves tuning; instructions-only OpenResponses stays on the responses arm; provider switch requires a model; switch selects the correct arm without leaking the old tuning).
• Existing fixture-backed evaluator.test.tsx — 32 pass (Bedrock path unchanged).
• bun run typecheck + bun run lint:check clean; create/update --help list both providers.

Follow-ups

• OpenResponses lifecycle service fixtures for evaluator.test.tsx require recording against an account with OpenResponses access (bug-bash item).
• CDK synthesis of modelProvider: OpenResponses is handled by @aws/agentcore-cdk in the generated app; deploy-time verification in the bug-bash notes.

Bug-bash recording against the explore account to follow in a comment.

[jariy17]

Bug bash — imperative evaluator commands (live, sandbox account, us-west-2)

Recorded with the TUI harness against a live AgentCore account. All cases verified end-to-end; the created evaluators were deleted afterward (no residue).

#	Case	Result
1	create --model-provider OpenAI (unsupported)	❌ fails fast, before any AWS call: invalid --model-provider "OpenAI": expected Bedrock or OpenResponses
2	create --model-provider OpenResponses --model openai.gpt-5.4	✅ ACTIVE; service stores responsesEvaluatorModelConfig with maxOutputTokens: 4096, temperature: 0, no topP
3	instructions-only update on the OpenResponses evaluator	✅ stays on the responses arm — not silently converted to Bedrock, tuning preserved
4	update --model-provider Bedrock (no new model)	❌ rejected: changing provider requires a new --model (model ids are not portable across provider APIs)
5	provider switch with a new --model	✅ selects the target arm and does not leak the previous provider's tuning

Baseline Bedrock create/get/update/delete unchanged and green.

A screen recording of this run is archived internally (Artifactory); it is not linked here because this repository is public.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[codecov-commenter]

Codecov Report

❌ Patch coverage is 93.97590% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.25%. Comparing base (a23e472) to head (71af58e).
⚠️ Report is 1 commits behind head on refactor.

Files with missing lines	Patch %	Lines
...ers/eval/evaluator/llm-as-a-judge/create/index.tsx	70.58%	5 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##           refactor    #2299      +/-   ##
============================================
- Coverage     97.26%   97.25%   -0.02%     
============================================
  Files           610      610              
  Lines         40590    40656      +66     
============================================
+ Hits          39480    39540      +60     
- Misses         1110     1116       +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[notgitika]

Failing CI test

[notgitika]

The description mentioned this test file .../core/evalEvaluatorProvider.test.tsx which is not here was it meant to be included?

[notgitika]

can we reuse the provider-specific model validation from project add evaluator here and in the update path?

string min 1 means at this point " " would be valid. not a blocker imo

[nborges-aws]

+1

[nborges-aws]

Plus one on Gitika's comment on validation.

Also noticed the description mentions evalEvaluatorProvider.test.tsx under the testing header, but I don't see that file in the PR. Maybe it didn't get committed?

[jariy17]

I forgot to update the PR descriptions, ../core/evalEvaluatorProvider.test.tsx was proposely removed because the golden tests cover it.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[notgitika]

Thanks this is great!