feat(agent): add push_notification to inject messages during multi-turn prompts#1858
Open
lazytanuki wants to merge 1 commit into
Open
feat(agent): add push_notification to inject messages during multi-turn prompts#1858lazytanuki wants to merge 1 commit into
push_notification to inject messages during multi-turn prompts#1858lazytanuki wants to merge 1 commit into
Conversation
…turn prompts Add a `pending_notifications` field to `Agent` that acts as a per-conversation message queue, and a `push_notification` method to enqueue messages from outside the prompt loop. The queued messages are consumed as `User` messages at the start of each loop iteration in both the non-streaming and streaming prompt paths, allowing external events (e.g. a background task or signal from a tool) to inject new input mid-conversation.
gold-silver-copper
added a commit
that referenced
this pull request
Jun 26, 2026
Reimplements the mid-run message-injection feature from #1858 on the new AgentRunner/AgentRun architecture (the original drove the now-replaced prompt_request loop). `AgentRunner::message_injector()` (also on PromptRequest/StreamingPromptRequest) returns a cloneable, Send `MessageInjector` bound to the run. `inject(..)` queues a message the driver folds into the conversation immediately before the next model call; in-flight tool calls in the current turn still finish first. If the run would otherwise finish with a message still queued, it is kept alive for one more turn to deliver it (bounded by the turn budget), after which `inject` returns `MessageInjectError::RunFinished`. An injected user message merges into a trailing user turn so injection never yields two consecutive user messages (which providers like Anthropic reject). The mechanism splits across the two layers: the sans-IO AgentRun owns folding (`inject_message`) and reopening a finished run (`reopen_for_injection`) as plain serializable transitions; the AgentRunner owns the async futures::mpsc transport drained before each step, so behavior is identical on the blocking and streaming drivers. Delivery mirrors pydantic-ai's AgentRun.enqueue and the "fold input between steps" point of Vercel AI SDK's prepareStep / LangGraph's Command(resume=..), without requiring a checkpointer. Adds AgentRun-level unit tests (fold/merge, reopen, budget bound, serde), blocking + streaming driver tests (delivery, keep-alive parity, post-run error), and the `agent_with_message_injection` example.
gold-silver-copper
added a commit
that referenced
this pull request
Jun 27, 2026
Lets external code inject a message into a running agent. A cloneable, Send MessageInjector (from AgentRunner::message_injector(), also on PromptRequest / StreamingPromptRequest) pushes a Message; the driver drains it before each model call and folds it into the sans-IO AgentRun via inject_message(), a serializable state transition. Behaves identically on the blocking and streaming drivers. Delivery is best-effort and in-flight: a message folds in before the next model call as its own user turn; if the run finishes first it is not delivered (logged), and a later inject returns MessageInjectError::RunFinished. Injection does not resurrect a finished run, only delivers user turns, and never displaces an output-mode reprompt or invalid-tool retry's feedback. The injected message is a separate user turn, never merged into a tool-result message (which OpenAI's converter drops). To keep consecutive user turns valid across every provider, a new shared providers::coalesce pass merges adjacent same-role USER turns on each provider's converted message list — applied across the in-core providers (anthropic, gemini, openai chat + responses, cohere, mistral, ollama, deepseek, azure, groq, together, llamafile, xai; openai-compat providers inherit it) and rig-bedrock. Required by AWS Bedrock's strictly- alternating Converse API; a no-op for already-alternating histories elsewhere. Parallel tool-result messages are never coalesced. This also normalizes RAG documents + prompt and hoisted-system gaps into single user turns. Reimplements #1858 on the AgentRunner architecture. Delivery model follows pydantic-ai's AgentRun.enqueue and the fold-between-steps point of Vercel AI SDK's prepareStep / LangGraph's Command(resume=..), without a checkpointer. Tests: AgentRun unit (push-separate delivery, fold order, ExecutingTools serde, reprompt/retry non-derailment, non-user drop), driver tests (inject-before, mid-tool-loop at the request level, multi-producer, RunFinished), an OpenAI- converter regression, per-provider coalesce + parallel-tool-results-preserved tests, and the agent_with_message_injection example.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi!
This PR adds a
push_notificationmethod toAgentthat allows injecting user messages into an already-running multi-turn prompt, similar to how OpenCode makes it possible to send input to an active agent mid-conversation.How it works
Agent::push_notification(conversation_id, message)enqueues a message into a per-conversation pending queue.Usermessages into the conversation history.Arc<Mutex<...>>, so cloning the agent shares the same queue, enabling cross-task patterns like spawning a background watcher that callspush_notificationwhen an external event occurs.Example
Disclosure
Thanks!