test(vscuse): fix Copilot BizChat chat-input for new view and add F5 recovery#16367
Open
qfai wants to merge 1 commit into
Open
test(vscuse): fix Copilot BizChat chat-input for new view and add F5 recovery#16367qfai wants to merge 1 commit into
qfai wants to merge 1 commit into
Conversation
…recovery The M365 Copilot (BizChat) web view moved the chat input (~y=369 to ~y=487) and made the placeholder agent-specific. Update the input-click step in group__validation_DA_repair and group__validation_basic_ai_chat_bot_in_copilot to the new position with an OCR/placeholder-agnostic target and drop the stale dhash preconditions. Add up to 3 F5 refreshes before the agent-name assertion in group__debug_in_copilot_local and group__debug_in_copilot_local_none_da to recover from the intermittent 'This agent is not installed' state, and make the F5 steps optional (continue_on_error). Add a 90s retry window to the M365 sign-in assertion in group__login_m365_only so slower first-time sign-ins are not cut off.
Contributor
❌ VscUse Test Plan — Tests failureWhy these tests: Selected 128 impacted test plan(s) changed in this PR (no product code modified). Branch diff: Results: ✅ 61 passed · ❌ 67 failed (of 128 plans) Plans run:
ℹ️ How were these tests selected?GitHub Copilot (Claude Sonnet 4.6, high reasoning) analysed the PR title, description, and the diff between |
Contributor
E2E Test Selection — AI SelectedWhy these tests: All changed files are under packages/tests/vscuse/** which maps to rule 8 (vscuse UI tests), so no e2e tests are needed. Cases selected (0): Need to run more tests?Comment on this PR:
Then re-run the workflow. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #16367 +/- ##
=======================================
Coverage 87.61% 87.61%
=======================================
Files 643 643
Lines 33725 33725
Branches 7919 7919
=======================================
Hits 29547 29547
Misses 2369 2369
Partials 1809 1809 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Update the vscuse Copilot/BizChat test steps for the redesigned M365 Copilot (BizChat) web view, and make the intermittent "This agent is not installed" local‑debug flow resilient. JSON test‑asset changes only.
Changes
1. New BizChat view — chat input moved
The M365 Copilot web view was redesigned: the chat input moved down (~
y=369→ ~y=487) and the placeholder became agent‑specific (Message <agent name>). Updated the input‑click step to the new position, an OCR / placeholder‑agnostic target, and removed the staledhashpreconditions:group__validation_DA_repair.json(step_bee70d3d)group__validation_basic_ai_chat_bot_in_copilot.json(step_294018a0)Ref: OfficeDev/microsoft-365-agents-toolkit-test#15504
2. "Agent not installed" F5 recovery
Added up to 3 F5 refreshes before the agent‑name assertion and made the F5 steps optional (
continue_on_error), to recover from the intermittent "This agent is not installed" state:group__debug_in_copilot_local.jsongroup__debug_in_copilot_local_none_da.json3. Login robustness
Added a 90s retry window to the M365 sign‑in assertion so slower first‑time sign‑ins are not cut off:
group__login_m365_only.jsonValidation
Ran
Basic_Custom_Engine_Azure_OpenAI_js_Copilot_Local_Debuglocally against image20260715: 61/61 steps, 100%.+ Message vscuse_app_…localat (412, 489) and clicked it; the bot then responded successfully.