feat(core): implement Runner.runLive and LlmAgent live flow by tSte · Pull Request #445 · google/adk-js

tSte · 2026-06-19T15:33:00Z

Implements the bidirectional (live) orchestration that was left as a TODO after #409 (which added only the GeminiLlmConnection layer): runner.ts carried // TODO b/425992518: Implement runLive and LlmAgent.runLiveFlow threw not implemented.

Runner.runLive: opens the invocation, runs the before/on-event/after-run plugin hooks, drives agent.runLive, and persists events while skipping raw inline-audio blobs (shouldAppendLiveEvent).
LlmAgent.runLiveFlow: preprocess (same processors as runAsync) -> connect ->
parallel send loop + receive loop -> tool execution -> sub-agent transfer. Reconnects transparently on goAway / recoverable drops when a session resumption handle is available, skipping history replay on resume.
InvocationContext.liveRequestQueue / liveSessionResumptionHandle.
LiveRequestQueue.get(abortSignal) to release a parked read on teardown.
RunConfig.contextWindowCompression, forwarded to the live connect config.
Gemini.connect: model-version-aware live API version routing and stripping of the Vertex-only sessionResumption.transparent flag on the AI Studio backend.

Adds core/test/runner/run_live_test.ts (16 tests covering realtime/audio, transcription persistence, tool calls, resumption capture, goAway reconnect, externally supplied handles, activity signals, sub-agent transfer) and liveApiVersion / transparent-strip coverage in google_llm_test.ts.

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

Closes: Add support for real-time bidirectional audio streaming #239
Related: #issue_number

2. Or, if no issue exists, describe the change:

If applicable, please follow the issue templates to provide as much detail as
possible.

Problem:
A clear and concise description of what the problem is.

Solution:
A clear and concise description of what you want to happen and why you choose
this solution.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required
for all PRs that are not small documentation or typo fixes.

Unit Tests:

I have added or updated unit tests for my change.
All unit tests pass locally.

Please include a summary of passed npm test results.

Manual End-to-End (E2E) Tests:

Please provide instructions on how to manually test your changes, including any
necessary setup or configuration. Please provide logs or screenshots to help
reviewers better understand the fix.

Checklist

I have read the CONTRIBUTING.md document.
I have performed a self-review of my own code.
I have commented my code, particularly in hard-to-understand areas.
I have added tests that prove my fix is effective or that my feature works.
New and existing unit tests pass locally with my changes.
I have manually tested my changes end-to-end.
Any dependent changes have been merged and published in downstream modules.

Additional context

Add any other context or screenshots about the feature request here.

Implements the bidirectional (live) orchestration that was left as a TODO after google#409 (which added only the GeminiLlmConnection layer): runner.ts carried `// TODO b/425992518: Implement runLive` and LlmAgent.runLiveFlow threw `not implemented`. - Runner.runLive: opens the invocation, runs the before/on-event/after-run plugin hooks, drives agent.runLive, and persists events while skipping raw inline-audio blobs (shouldAppendLiveEvent). - LlmAgent.runLiveFlow: preprocess (same processors as runAsync) -> connect -> parallel send loop + receive loop -> tool execution -> sub-agent transfer. Reconnects transparently on goAway / recoverable drops when a session resumption handle is available, skipping history replay on resume. - InvocationContext.liveRequestQueue / liveSessionResumptionHandle. - LiveRequestQueue.get(abortSignal) to release a parked read on teardown. - RunConfig.contextWindowCompression, forwarded to the live connect config. - Gemini.connect: model-version-aware live API version routing and stripping of the Vertex-only sessionResumption.transparent flag on the AI Studio backend. Adds core/test/runner/run_live_test.ts (16 tests covering realtime/audio, transcription persistence, tool calls, resumption capture, goAway reconnect, externally supplied handles, activity signals, sub-agent transfer) and liveApiVersion / transparent-strip coverage in google_llm_test.ts.

kalenkevich

Thanks for the contribution!

I did some partial review, will continue tomorrow

kalenkevich · 2026-06-22T21:48:40Z

+    return true;
+  }
+  const inlineData = parts[0].inlineData;
+  if (!inlineData?.mimeType?.startsWith('audio/')) {


why only audio/?

what about video/ and image/?

Added audo/video/image & renamed the function

kalenkevich · 2026-06-22T21:49:11Z

+function shouldAppendLiveEvent(event: Event): boolean {
+  const parts = event.content?.parts;
+  if (!parts?.length) {
+    return true;


should return false

kalenkevich · 2026-06-22T21:50:32Z

+  if (!parts?.length) {
+    return true;
+  }
+  const inlineData = parts[0].inlineData;


why we are checking only the first part? We need to iterate on parts list and if at least one satisfy the criteria we should return true

Added .some((p) => {...}) for all parts

kalenkevich · 2026-06-22T21:51:15Z

+      runConfig.inputAudioTranscription ??= {};
+    }
+
+    const span = tracer.startSpan('invocation');


it should be runLive I assume

I kept this as 'invocation' intentionally. runAsync in this same class opens its top-level span with tracer.startSpan('invocation'), and Python ADK's run_async seems to do the same (correct me if I am wrong).

An 'invocation' seems to be the ADK convention for the one span that wraps a whole run, regardless of mode, correct? Naming only runLive's span differently would make the two entry points inconsistent in traces and break any span-name-based filtering that already assumes an 'invocation'. The agent/flow-level spans underneath still distinguish live from non-live work. Happy to rename if you'd prefer runLive here, just clarifying my thought process.

kalenkevich · 2026-06-22T21:51:43Z

+        ctx,
+        this,
+        async function* () {
+          const session = await this.sessionService.getSession({


getOrCreateSession

Applied & removed the if(!session) check.

kalenkevich · 2026-06-22T21:59:33Z

+      } else if (extractModelName(this.model).startsWith('gemini-2.5')) {
+        this._liveApiVersion = 'v1beta';


This is not how it is in python ADK https://github.com/google/adk-python/blob/main/src/google/adk/models/google_llm.py#L379-L388

Reflected to Python equivalent.

- runner.runLive: use getOrCreateSession instead of getSession + throw, matching Python ADK run_live. - shouldAppendLiveEvent -> isLiveModelMediaEventWithInlineData: iterate all parts and skip audio/video/image inline media (mirrors Python _is_live_model_media_event_with_inline_data); negate at the call site. - google_llm.liveApiVersion: drop the gemini-2.5 -> v1beta special case; Vertex -> v1beta1, AI Studio -> v1alpha, matching Python ADK. - Tests updated for the new behavior plus video/image and non-first-part media coverage.

tSte · 2026-06-23T09:10:08Z

Thanks for the contribution!

I did some partial review, will continue tomorrow

Thanks, I updated code and did successful re-test with our own agent.

tSte mentioned this pull request Jun 19, 2026

Release: v1.3.0 #407

Merged

tSte force-pushed the feat/run-live-flow branch from 69ef687 to 95f92f2 Compare June 19, 2026 16:01

kalenkevich reviewed Jun 22, 2026

View reviewed changes

tSte requested a review from kalenkevich June 24, 2026 11:04

		} else if (extractModelName(this.model).startsWith('gemini-2.5')) {
		this._liveApiVersion = 'v1beta';

Uh oh!

Conversation

tSte commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Link to Issue or Description of Change

Testing Plan

Checklist

Additional context

Uh oh!

kalenkevich left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tSte commented Jun 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

tSte commented Jun 19, 2026 •

edited

Loading