Skip to content

feat(agent): add push_notification to inject messages during multi-turn prompts#1858

Open
lazytanuki wants to merge 1 commit into
0xPlaygrounds:mainfrom
lazytanuki:agent-notification
Open

feat(agent): add push_notification to inject messages during multi-turn prompts#1858
lazytanuki wants to merge 1 commit into
0xPlaygrounds:mainfrom
lazytanuki:agent-notification

Conversation

@lazytanuki

Copy link
Copy Markdown

Hi!

This PR adds a push_notification method to Agent that allows injecting user messages into an already-running multi-turn prompt, similar to how OpenCode makes it possible to send input to an active agent mid-conversation.

How it works

  • Agent::push_notification(conversation_id, message) enqueues a message into a per-conversation pending queue.
  • At the start of each loop iteration (both streaming and non-streaming), the queue is drained and messages are injected as User messages into the conversation history.
  • The queue is stored behind Arc<Mutex<...>>, so cloning the agent shares the same queue, enabling cross-task patterns like spawning a background watcher that calls push_notification when an external event occurs.

Example

let agent = openai::Client::from_env()?
    .agent(openai::GPT_5_2)
    .conversation_id("my-conv")
    .toot(some_tool)
    .build();
let agent_clone = agent.clone();

let prompt_task = tokio::spawn(async move {
    // Start a long running multi-turn prompt.
    let resp = agent.prompt("Do this and that with your available tools.").await.unwrap();
    println!("{resp}");
})

let some_watcher_task = tokio::spawn(async move {
    // ... wait for some condition ...
    agent_clone.push_notification("my-conv", "Continue, but you should know that ...".to_string());
});

// Wait on both tasks...

Disclosure

  • Feature code was handwritten.
  • Tests were AI-generated then checked.

Thanks!

…turn prompts

Add a `pending_notifications` field to `Agent` that acts as a
per-conversation
message queue, and a `push_notification` method to enqueue messages from
outside the prompt loop.

The queued messages are consumed as `User` messages at the start of each
loop iteration in both the non-streaming and streaming prompt paths,
allowing
external events (e.g. a background task or signal from a tool) to inject
new input mid-conversation.
gold-silver-copper added a commit that referenced this pull request Jun 26, 2026
Reimplements the mid-run message-injection feature from #1858 on the new
AgentRunner/AgentRun architecture (the original drove the now-replaced
prompt_request loop).

`AgentRunner::message_injector()` (also on PromptRequest/StreamingPromptRequest)
returns a cloneable, Send `MessageInjector` bound to the run. `inject(..)` queues
a message the driver folds into the conversation immediately before the next
model call; in-flight tool calls in the current turn still finish first. If the
run would otherwise finish with a message still queued, it is kept alive for one
more turn to deliver it (bounded by the turn budget), after which `inject`
returns `MessageInjectError::RunFinished`. An injected user message merges into a
trailing user turn so injection never yields two consecutive user messages
(which providers like Anthropic reject).

The mechanism splits across the two layers: the sans-IO AgentRun owns folding
(`inject_message`) and reopening a finished run (`reopen_for_injection`) as plain
serializable transitions; the AgentRunner owns the async futures::mpsc transport
drained before each step, so behavior is identical on the blocking and streaming
drivers.

Delivery mirrors pydantic-ai's AgentRun.enqueue and the "fold input between
steps" point of Vercel AI SDK's prepareStep / LangGraph's Command(resume=..),
without requiring a checkpointer.

Adds AgentRun-level unit tests (fold/merge, reopen, budget bound, serde),
blocking + streaming driver tests (delivery, keep-alive parity, post-run error),
and the `agent_with_message_injection` example.
gold-silver-copper added a commit that referenced this pull request Jun 27, 2026
Lets external code inject a message into a running agent. A cloneable, Send
MessageInjector (from AgentRunner::message_injector(), also on PromptRequest /
StreamingPromptRequest) pushes a Message; the driver drains it before each model
call and folds it into the sans-IO AgentRun via inject_message(), a serializable
state transition. Behaves identically on the blocking and streaming drivers.
Delivery is best-effort and in-flight: a message folds in before the next model
call as its own user turn; if the run finishes first it is not delivered
(logged), and a later inject returns MessageInjectError::RunFinished. Injection
does not resurrect a finished run, only delivers user turns, and never displaces
an output-mode reprompt or invalid-tool retry's feedback.

The injected message is a separate user turn, never merged into a tool-result
message (which OpenAI's converter drops). To keep consecutive user turns valid
across every provider, a new shared providers::coalesce pass merges adjacent
same-role USER turns on each provider's converted message list — applied across
the in-core providers (anthropic, gemini, openai chat + responses, cohere,
mistral, ollama, deepseek, azure, groq, together, llamafile, xai; openai-compat
providers inherit it) and rig-bedrock. Required by AWS Bedrock's strictly-
alternating Converse API; a no-op for already-alternating histories elsewhere.
Parallel tool-result messages are never coalesced. This also normalizes RAG
documents + prompt and hoisted-system gaps into single user turns.

Reimplements #1858 on the AgentRunner architecture. Delivery model follows
pydantic-ai's AgentRun.enqueue and the fold-between-steps point of Vercel AI
SDK's prepareStep / LangGraph's Command(resume=..), without a checkpointer.

Tests: AgentRun unit (push-separate delivery, fold order, ExecutingTools serde,
reprompt/retry non-derailment, non-user drop), driver tests (inject-before,
mid-tool-loop at the request level, multi-producer, RunFinished), an OpenAI-
converter regression, per-provider coalesce + parallel-tool-results-preserved
tests, and the agent_with_message_injection example.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant