fix(tauri-plugin-webdriver): resolve pointerMove origin so click(options) hits the target by goosewobbler · Pull Request #433 · webdriverio/desktop-mobile

goosewobbler · 2026-06-19T02:01:54Z

Problem

element.click(options) (e.g. .click({}), .click({ button: 'right' })) silently misses its target on the embedded Tauri driver, while bare .click() works. Reported in #423.

.click() uses the elementClick endpoint (click_element → JS el.click(), coordinate-independent). .click(options) instead routes through the W3C Actions API: WebdriverIO sends a pointerMove with origin = the element reference and x/y = offsets (0,0 by default). The driver's PointerMove deserialized only x/y/duration and ignored origin, so serde dropped it and (0,0) was treated as absolute viewport coordinates → the pointer moved to the top-left corner and pointerDown/Up missed the element.

Fix

actions.rs: add origin to the pointerMove action and resolve it per the W3C Actions spec:
- "viewport" (or absent, the default) → x/y are absolute viewport coords (unchanged behaviour)
- "pointer" → relative to the current pointer position
- element ref { "element-6066-…": "<id>" } → offset from the element's in-view center
executor.rs: add Executor::get_element_center, returning the element's center in client/viewport coordinates via getBoundingClientRect() (scrolling it into view first). This is intentionally distinct from get_element_rect, which adds scroll offsets (page coords); pointer events dispatch on clientX/clientY, so viewport coords are required.

This also fixes moveTo, drag, and hover-with-offset — anything that routes through performActions with an origin — not just click.

Tests

New e2e/test/tauri/actions.spec.ts (the repro from [🐛 Bug]: Tauri plugin fails to execute click() correctly when arguments are passed #423): .click({}) and .click({ button: 'left' }) now increment the counter exactly like bare .click().

Validation

cargo check and cargo clippy --all-targets clean — no new warnings (the two incompatible_msrv notes are pre-existing in the cookie/OffsetDateTime path, unrelated to this change).
The e2e spec runs in CI (needs the built macOS fixture binary).

Fixes #423

🤖 Generated with Claude Code

github-actions · 2026-06-19T02:02:25Z

Standing release PR: #400 · 12 packages queued · open 67h 24m · ✅ ready to merge

Release Preview — 13 packages

Note: Labels on this PR are advisory in standing-pr mode. Bumps come from conventional commits in the standing PR; override by editing labels on the standing PR itself. Add release:immediate to bypass the standing PR and release this PR directly.

These changes will be added to the release PR (#400) when merged:

Changelog

Project-wide changes

Added

manage Metro lifecycle + zero-config Hermes setup (React Native service: manage Metro lifecycle + zero-config Hermes setup #406) (feat(react-native-service): manage Metro lifecycle + zero-config Hermes setup (#406) #426) (react-native-service)
zero-config execute/mock via auto VM-service setup (Flutter service: zero-config execute/mock via auto VM-service setup #405) (feat(flutter-service): zero-config execute/mock via auto VM-service setup (#405) #425) (flutter-service)
shared mobile setup automation (chromedriver-style zero-config DX) (Shared mobile setup automation in @wdio/native-mobile-core (chromedriver-style zero-config DX) #378) (feat(native-mobile-core): shared mobile setup automation (chromedriver-style zero-config DX) (#378) #424) (native-mobile-core)

Changed

narrow shared-package changes to affected services (ci(detect-changes): narrow shared-package changes to affected services #438) (detect-changes)
make iOS toolchain probes async (no event-loop block) (refactor(native-mobile-core): make iOS toolchain probes async (no event-loop block) #437) (native-mobile-core)

Fixed

reject unrecognised named pointerMove origins (tauri-plugin-webdriver)
honour capability-level launcher options (fix(native-mobile-core): honour capability-level launcher options #440) (native-mobile-core)
only clear primary-down state on a primary release (tauri-plugin-webdriver)
synthesize a click event on Actions-based clicks (tauri-plugin-webdriver)
persist pointer position across performActions calls (tauri-plugin-webdriver)
sync mocks via afterCommand so user command overrides survive (fix(tauri-service): sync mocks via afterCommand so user command overrides survive #434) (tauri-service) [🐛 Bug]: Tauri service clobbers user's command overrides for click etc #422
use instant scroll + floor for element center (tauri-plugin-webdriver)
resolve pointerMove origin so click(options) hits the target (tauri-plugin-webdriver) [🐛 Bug]: Tauri plugin fails to execute click() correctly when arguments are passed #423

Removed

tauri afterCommand mock-sync (fix(tauri-service): sync mocks via afterCommand so user command overrides survive #434) (revert: tauri afterCommand mock-sync (#434) #439)

@wdio/dioxus-service N/A → 1.1.0

Changed

Update version to 1.1.0

@wdio/electron-cdp-bridge wdio-electron-cdp-bridge@v10.0.0 → 10.1.0

Changed

Update version to 10.1.0

@wdio/electron-service wdio-electron-service@v10.0.0 → 10.1.0

Changed

Update version to 10.1.0

@wdio/flutter-service v1.0.0-next.0 → 1.1.0

Added

zero-config execute/mock via auto VM-service setup (Flutter service: zero-config execute/mock via auto VM-service setup #405) (feat(flutter-service): zero-config execute/mock via auto VM-service setup (#405) #425) (flutter-service)
shared mobile setup automation (chromedriver-style zero-config DX) (Shared mobile setup automation in @wdio/native-mobile-core (chromedriver-style zero-config DX) #378) (feat(native-mobile-core): shared mobile setup automation (chromedriver-style zero-config DX) (#378) #424) (native-mobile-core)

@wdio/native-cdp-bridge v0.1.0-next.0 → 0.2.0

Changed

Update version to 0.2.0

@wdio/native-mobile-core v1.0.0 → 1.1.0

@wdio/native-spy wdio-native-spy@v1.1.0 → 1.2.0

Changed

Update version to 1.2.0

@wdio/native-types wdio-native-types@v2.3.1 → 2.4.0

Added

manage Metro lifecycle + zero-config Hermes setup (React Native service: manage Metro lifecycle + zero-config Hermes setup #406) (feat(react-native-service): manage Metro lifecycle + zero-config Hermes setup (#406) #426) (react-native-service)
shared mobile setup automation (chromedriver-style zero-config DX) (Shared mobile setup automation in @wdio/native-mobile-core (chromedriver-style zero-config DX) #378) (feat(native-mobile-core): shared mobile setup automation (chromedriver-style zero-config DX) (#378) #424) (native-mobile-core)

@wdio/native-utils wdio-native-utils@v2.4.0 → 2.5.0

Changed

Update version to 2.5.0

@wdio/react-native-service v1.0.0-next.0 → 1.1.0

Added

manage Metro lifecycle + zero-config Hermes setup (React Native service: manage Metro lifecycle + zero-config Hermes setup #406) (feat(react-native-service): manage Metro lifecycle + zero-config Hermes setup (#406) #426) (react-native-service)
shared mobile setup automation (chromedriver-style zero-config DX) (Shared mobile setup automation in @wdio/native-mobile-core (chromedriver-style zero-config DX) #378) (feat(native-mobile-core): shared mobile setup automation (chromedriver-style zero-config DX) (#378) #424) (native-mobile-core)

Fixed

honour capability-level launcher options (fix(native-mobile-core): honour capability-level launcher options #440) (native-mobile-core)

@wdio/tauri-service wdio-tauri-service@v1.1.0 → 1.2.0

Fixed

sync mocks via afterCommand so user command overrides survive (fix(tauri-service): sync mocks via afterCommand so user command overrides survive #434) (tauri-service) [🐛 Bug]: Tauri service clobbers user's command overrides for click etc #422

Removed

tauri afterCommand mock-sync (fix(tauri-service): sync mocks via afterCommand so user command overrides survive #434) (revert: tauri afterCommand mock-sync (#434) #439)

dioxus-package-test-app v0.1.0 → 0.2.0

Changed

Update version to 0.2.0

tauri-plugin-wdio-webdriver tauri-plugin-wdio-webdriver@v1.1.0 → 1.2.0

Fixed

reject unrecognised named pointerMove origins (tauri-plugin-webdriver)
only clear primary-down state on a primary release (tauri-plugin-webdriver)
synthesize a click event on Actions-based clicks (tauri-plugin-webdriver)
persist pointer position across performActions calls (tauri-plugin-webdriver)
use instant scroll + floor for element center (tauri-plugin-webdriver)
resolve pointerMove origin so click(options) hits the target (tauri-plugin-webdriver) [🐛 Bug]: Tauri plugin fails to execute click() correctly when arguments are passed #423

After merge — predicted release

No version escalation — this PR's changes will be included in the queued release without affecting the projected versions.

Package	Standing PR	This PR	After merge
`@wdio/dioxus-service`	1.1.0	1.1.0	1.1.0
`@wdio/electron-cdp-bridge`	10.1.0	10.1.0	10.1.0
`@wdio/electron-service`	10.1.0	10.1.0	10.1.0
`@wdio/flutter-service`	1.1.0	1.1.0	1.1.0
`@wdio/native-cdp-bridge`	0.2.0	0.2.0	0.2.0
`@wdio/native-mobile-core`	1.1.0	1.1.0	1.1.0
`@wdio/native-spy`	1.2.0	1.2.0	1.2.0
`@wdio/native-types`	2.4.0	2.4.0	2.4.0
`@wdio/native-utils`	2.5.0	2.5.0	2.5.0
`@wdio/react-native-service`	1.1.0	1.1.0	1.1.0
`@wdio/tauri-service`	1.2.0	1.2.0	1.2.0
`dioxus-package-test-app`	0.2.0	0.2.0	0.2.0

Updated automatically by ReleaseKit

greptile-apps · 2026-06-19T02:04:56Z

Greptile Summary

This PR fixes element.click(options) silently missing its target on the embedded Tauri WebDriver by teaching the pointerMove handler to deserialize and resolve the W3C origin field — previously the field was silently dropped by serde, causing all pointer actions to land at viewport (0,0).

actions.rs: PointerAction::PointerMove gains an origin field (None/\"viewport\" → absolute coords, \"pointer\" → relative to current position, element ref → offset from element's in-view center); pointer position is now persisted across performActions calls; a synthesised click MouseEvent is emitted after a same-spot primary down+up so element click handlers fire correctly.
executor.rs: Adds get_element_center (scrolls the element into view with behavior: 'instant' before reading getBoundingClientRect, returning viewport-relative coordinates) and a PointerEventType::Click variant for explicit click dispatch.
session.rs: ActionState gains pointer_position to persist the pointer location across separate performActions calls, enabling correct origin: \"pointer\" resolution.

Confidence Score: 5/5

Safe to merge — the fix is narrowly scoped to origin resolution for pointerMove actions and correctly handles all three W3C origin variants.

The origin parsing, element-center calculation (with behavior: 'instant' scroll), click synthesis logic, and pointer-position persistence are all correctly implemented for the primary use case. The one edge case — primary_down_pos being local and therefore not bridging two separate performActions calls — does not affect element.click(options) (which sends all actions in one request) and is a pre-existing structural limitation rather than a regression.

No files require special attention; actions.rs has the most logic but it is well-structured and well-commented.

Important Files Changed

Filename	Overview
packages/tauri-plugin-webdriver/src/server/handlers/actions.rs	Adds `origin` deserialization for `pointerMove`, resolves viewport/pointer/element origins per W3C spec, persists pointer position across calls, and synthesises a `click` event after a same-spot down+up. Core logic is correct; `primary_down_pos` is not persisted between calls.
packages/tauri-plugin-webdriver/src/platform/executor.rs	Adds `PointerEventType::Click` variant and `get_element_center` helper; uses `behavior: 'instant'` in `scrollIntoView` so `getBoundingClientRect` reads settled viewport coordinates correctly.
packages/tauri-plugin-webdriver/src/webdriver/session.rs	Adds `pointer_position: (i32, i32)` to `ActionState` (defaults to `(0,0)` via `Default`), enabling cross-call `origin: "pointer"` resolution.
e2e/test/tauri/actions.spec.ts	New e2e spec reproducing issue #423: verifies `click({})` and `click({ button: 'left' })` hit the correct target by asserting counter increments.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant WD as WebdriverIO
    participant H as actions.rs (perform)
    participant S as SessionManager
    participant E as Executor
    participant JS as WebView JS

    WD->>H: "POST /session/:id/actions<br/>[pointerMove(origin=el, x=0, y=0), pointerDown, pointerUp]"
    H->>S: read pointer_position (persisted)
    H->>H: init pointer_state from persisted position

    Note over H: PointerMove action
    H->>S: read session elements → js_var for element ref
    H->>E: get_element_center(js_var)
    E->>JS: "scrollIntoView({behavior:'instant'}) + getBoundingClientRect()"
    JS-->>E: "{x: cx, y: cy}"
    E-->>H: (cx, cy)
    H->>H: "pointer_state = (cx+0, cy+0)"

    Note over H: PointerDown action
    H->>E: dispatch_pointer_event(Down, cx, cy, 0)
    E->>JS: dispatchEvent(mousedown)
    H->>H: "primary_down_pos = Some((cx, cy))"

    Note over H: PointerUp action
    H->>E: dispatch_pointer_event(Up, cx, cy, 0)
    E->>JS: dispatchEvent(mouseup)
    H->>H: "primary_down_pos == pointer_state → emit click"
    H->>E: dispatch_pointer_event(Click, cx, cy, 0)
    E->>JS: dispatchEvent(click)
    H->>H: "primary_down_pos = None"

    H->>S: "write pointer_position = (cx, cy)"
    H-->>WD: null response

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant WD as WebdriverIO
    participant H as actions.rs (perform)
    participant S as SessionManager
    participant E as Executor
    participant JS as WebView JS

    WD->>H: "POST /session/:id/actions<br/>[pointerMove(origin=el, x=0, y=0), pointerDown, pointerUp]"
    H->>S: read pointer_position (persisted)
    H->>H: init pointer_state from persisted position

    Note over H: PointerMove action
    H->>S: read session elements → js_var for element ref
    H->>E: get_element_center(js_var)
    E->>JS: "scrollIntoView({behavior:'instant'}) + getBoundingClientRect()"
    JS-->>E: "{x: cx, y: cy}"
    E-->>H: (cx, cy)
    H->>H: "pointer_state = (cx+0, cy+0)"

    Note over H: PointerDown action
    H->>E: dispatch_pointer_event(Down, cx, cy, 0)
    E->>JS: dispatchEvent(mousedown)
    H->>H: "primary_down_pos = Some((cx, cy))"

    Note over H: PointerUp action
    H->>E: dispatch_pointer_event(Up, cx, cy, 0)
    E->>JS: dispatchEvent(mouseup)
    H->>H: "primary_down_pos == pointer_state → emit click"
    H->>E: dispatch_pointer_event(Click, cx, cy, 0)
    E->>JS: dispatchEvent(click)
    H->>H: "primary_down_pos = None"

    H->>S: "write pointer_position = (cx, cy)"
    H-->>WD: null response

_{Reviews (7): Last reviewed commit: "fix(tauri-plugin-webdriver): reject unre..." | Re-trigger Greptile}

…ons) hits the target element.click(options) routes through the W3C Actions API instead of the elementClick endpoint. The embedded driver's PointerMove ignored the action `origin`, so WebdriverIO's element-relative move (origin=element, x/y=0) was treated as absolute viewport coordinates and landed at (0,0), missing the target. Resolve viewport/pointer/element origins and add Executor::get_element_center (client/viewport coords). Adds an e2e regression test (the repro from #423). Fixes #423 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…enter Address review feedback on get_element_center: scrollIntoView uses behavior 'instant' so getBoundingClientRect is not read mid-animation under scroll-behavior: smooth, and the center uses Math.floor to match the W3C "in-view center point" algorithm (avoids a 1px divergence at sub-pixel element boundaries). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ctions calls Address review feedback: pointer_state was a local reinitialized to (0, 0) at the start of every perform() call, so an origin: "pointer" move in a later performActions call computed from (0, 0) instead of the pointer's actual position. Persist it in the session's ActionState (read at the start of perform, written back at the end) and reset it on release, matching how pressed keys/buttons are tracked. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…d clicks element.click(options) routes through the W3C Actions API, which the embedded driver dispatched as bare mousedown/mouseup MouseEvents. Manually dispatched mouse events don't make the browser synthesize a click, so element click handlers never fired — .click({}) moved to the right spot (after the origin fix) but still didn't actually click. Emit a click event after a primary-button press + release on the same position (a click, not a drag). Real WebDriver providers (official/crabnebula) already do this; this brings the embedded driver in line, fixing the e2e on the embedded provider. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ry release Address review feedback: primary_down_pos was reset on every PointerUp regardless of button, so a non-primary release between a primary down and up (button 0 down -> button 1 up -> button 0 up) would drop the press state and suppress the synthesized click. Gate both the click synthesis and the reset on the primary button. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…igins The PointerMove origin match treated any named origin as viewport via a catch-all `Some(Origin::Named(_)) => (*x, *y)`. The W3C Actions spec defines only "viewport" and "pointer" as named origins, so give "viewport" its own arm and return invalid argument for anything else instead of silently falling through to viewport behaviour. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

goosewobbler added type:bug Something isn't working scope:tauri Tauri service and plugins priority:high High priority issue or PR labels Jun 19, 2026

greptile-apps Bot reviewed Jun 19, 2026

View reviewed changes

Comment thread packages/tauri-plugin-webdriver/src/platform/executor.rs Outdated

goosewobbler removed type:bug Something isn't working scope:tauri Tauri service and plugins priority:high High priority issue or PR labels Jun 19, 2026

goosewobbler force-pushed the fix/423-pointer-move-origin branch from eb067cd to 14f48f4 Compare June 19, 2026 02:21

goosewobbler and others added 5 commits June 19, 2026 08:48

goosewobbler merged commit ab145f8 into main Jun 19, 2026
167 of 169 checks passed

goosewobbler deleted the fix/423-pointer-move-origin branch June 19, 2026 23:21

This was referenced Jun 19, 2026

chore: release 0 package(s) #400

Closed

fix(electron-service,tauri-service): compose user command overrides instead of clobbering #441

Merged

chore(deps): bump actions/download-artifact from 7 to 8 #402

Open

chore: release 8 package(s) #456

Open

Uh oh!

Conversation

goosewobbler commented Jun 19, 2026

Problem

Fix

Tests

Validation

Uh oh!

github-actions Bot commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog

Added

Changed

Fixed

Removed

Changed

Changed

Changed

Added

Changed

Added

Changed

Fixed

Changed

Added

Changed

Added

Fixed

Fixed

Removed

Changed

Fixed

After merge — predicted release

Uh oh!

greptile-apps Bot commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Sequence Diagram

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions Bot commented Jun 19, 2026 •

edited

Loading

greptile-apps Bot commented Jun 19, 2026 •

edited

Loading