Skip to content

Commit 1fbc6e5

Browse files
committed
feat(workflows): selected outputs on sync runs + cross-run logs query augmentation
selectedOutputs was artificially stream-only: a sync run rejected it with a 400 and pointed at a second call (runs get) that resolves block ids only. The sync path already holds the full execution result, so selection now answers in the same response: blockOutputs keyed by the request's selector strings, names resolved against the exact state being run, absent paths omitted, failed and timed-out runs including the blocks that did run, values compacted like output. Route guard deleted, resume sends null, contract + openapi + cli-api + cli-docs regenerated, CLI --select-output works on a plain sync run (only --async still rejects it, locally, with the dialect hint). New agent-cli augmentation: logs query <workflowId> --block <name> [--field <path>] [--where <path>=<value>] [--status] [--trigger] [--limit] — one row per run across run history, recursive span walk, last match per run wins. The augmentation layer gains command-local flag parsing (--flag value / --flag=value / bare) to support it; positional-only commands are unchanged. Autopsy fix: the formatter neutral-path workaround from the display-frames commit dropped the caller's extension, so generate:openapi (JSON) aborted with a biome parse error while .ts generators stayed green. The extension now survives; regression test added and wired into test:generators. Claude-Session: https://claude.ai/code/session_01CgaxNAaeD3taGdghbXn17w
1 parent 5b8cb9e commit 1fbc6e5

23 files changed

Lines changed: 667 additions & 83 deletions

File tree

apps/docs/content/docs/cli/reference.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5157,7 +5157,7 @@ sim workflows run <workflowId> [options]
51575157
| `--input <json\|@file>` | No | Trigger input as JSON (JSON, or @path / @- to read a file or stdin). |
51585158
| `--async` | No | Queue the run and return immediately. |
51595159
| `--execution-timeout-seconds <value>` | No | Requested server-side timeout for an asynchronous run, in seconds. An upper bound, not the effective timeout: the run uses the smaller of this value and the plan's execution timeout, so requesting more than the plan allows silently yields the plan timeout. Rejected with `400` unless `async` is true. |
5160-
| `--select-output <value...>` | No | Return blockName.field values from the streamed result (e.g. agent_1.content), requires --follow; missing fields are omitted (space-separated, or @path / @- with one value per line; @@value for a literal leading @). |
5160+
| `--select-output <value...>` | No | Return blockName.field values (e.g. agent_1.content) — in blockOutputs on a sync run, or from the streamed result with --follow; missing fields are omitted. Not available with --async (space-separated, or @path / @- with one value per line; @@value for a literal leading @). |
51615161
| `--include-file-base64` | No | Inline eligible output files as base64 content. Rejected when `async` is true. |
51625162
| `--no-include-file-base64` | No | Send --include-file-base64 as false. |
51635163
| `--base64-max-bytes <value>` | No | Maximum total bytes of file content to inline as base64, lowering but never raising the server limit of 16 MiB. Rejected when `async` is true. |

apps/docs/content/docs/cli/workflows.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -532,7 +532,7 @@ sim workflows run <workflowId> [options]
532532
| `--input <json\|@file>` | No | Trigger input as JSON (JSON, or @path / @- to read a file or stdin). |
533533
| `--async` | No | Queue the run and return immediately. |
534534
| `--execution-timeout-seconds <value>` | No | Requested server-side timeout for an asynchronous run, in seconds. An upper bound, not the effective timeout: the run uses the smaller of this value and the plan's execution timeout, so requesting more than the plan allows silently yields the plan timeout. Rejected with `400` unless `async` is true. |
535-
| `--select-output <value...>` | No | Return blockName.field values from the streamed result (e.g. agent_1.content), requires --follow; missing fields are omitted (space-separated, or @path / @- with one value per line; @@value for a literal leading @). |
535+
| `--select-output <value...>` | No | Return blockName.field values (e.g. agent_1.content) — in blockOutputs on a sync run, or from the streamed result with --follow; missing fields are omitted. Not available with --async (space-separated, or @path / @- with one value per line; @@value for a literal leading @). |
536536
| `--include-file-base64` | No | Inline eligible output files as base64 content. Rejected when `async` is true. |
537537
| `--no-include-file-base64` | No | Send --include-file-base64 as false. |
538538
| `--base64-max-bytes <value>` | No | Maximum total bytes of file content to inline as base64, lowering but never raising the server limit of 16 MiB. Rejected when `async` is true. |

apps/docs/openapi-v2-workflows.json

Lines changed: 29 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5169,7 +5169,14 @@
51695169
},
51705170
"kind": {
51715171
"type": "string",
5172-
"enum": ["credential", "resource", "custom-tool", "mcp-tool", "skill"],
5172+
"enum": [
5173+
"credential",
5174+
"resource",
5175+
"custom-tool",
5176+
"mcp-tool",
5177+
"skill",
5178+
"block-output"
5179+
],
51735180
"description": "What kind of entity the reference was expected to name."
51745181
},
51755182
"reason": {
@@ -9094,6 +9101,23 @@
90949101
"output": {
90959102
"description": "Workflow output, including partial output on failure."
90969103
},
9104+
"blockOutputs": {
9105+
"anyOf": [
9106+
{
9107+
"type": "object",
9108+
"propertyNames": {
9109+
"type": "string"
9110+
},
9111+
"additionalProperties": {
9112+
"description": "Output value produced by one workflow block."
9113+
}
9114+
},
9115+
{
9116+
"type": "null"
9117+
}
9118+
],
9119+
"description": "Outputs of the blocks named by `selectedOutputs`, keyed by those selector strings, or null when none were requested. Selectors whose block did not run or whose path is absent are omitted; failed runs include the outputs of the blocks that did run."
9120+
},
90979121
"error": {
90989122
"anyOf": [
90999123
{
@@ -9121,7 +9145,7 @@
91219145
"minimum": 0
91229146
}
91239147
},
9124-
"required": ["runId", "workflowId", "status", "output", "error"],
9148+
"required": ["runId", "workflowId", "status", "output", "blockOutputs", "error"],
91259149
"additionalProperties": false,
91269150
"title": "Workflow run result",
91279151
"description": "Synchronous workflow run output and in-band execution status. Run failures are reported in band, not as HTTP errors — a run that exceeds its execution timeout returns HTTP 200 with `status: \"failed\"` and `error.code: \"TIMEOUT\"`, so branch on `status`."
@@ -9147,6 +9171,7 @@
91479171
"output": {
91489172
"result": "Ticket routed to Support"
91499173
},
9174+
"blockOutputs": null,
91509175
"error": null,
91519176
"startedAt": "2026-08-09T18:04:10.000Z",
91529177
"endedAt": "2026-08-09T18:04:11.000Z",
@@ -9305,7 +9330,7 @@
93059330
"type": "boolean"
93069331
},
93079332
"selectedOutputs": {
9308-
"description": "Block output references to include in a streamed response, as `blockId`, `blockId.path`, or `BlockName.path` (resolved against the live workflow). Requires `stream: true` — it shapes the streamed envelope only, so it is rejected on a sync request and when `async` is true. To narrow a finished run, pass `selectedOutputs` to the run resource instead.",
9333+
"description": "Block output references to include in the response, as `blockId`, `blockId.path`, or `BlockName.path` (resolved against the workflow state being run). On a sync request the named outputs come back in `blockOutputs`, keyed by these selector strings; on a stream they shape the streamed envelope. Selectors that resolve to no block or no value are omitted. Rejected when `async` is true — a queued run has produced nothing to select; narrow the finished run via the run resource instead.",
93099334
"maxItems": 100,
93109335
"type": "array",
93119336
"items": {
@@ -9883,6 +9908,7 @@
98839908
"output": {
98849909
"result": "Ticket routed to Support"
98859910
},
9911+
"blockOutputs": null,
98869912
"error": null,
98879913
"startedAt": "2026-08-09T18:04:10.000Z",
98889914
"endedAt": "2026-08-09T18:04:11.000Z",

apps/sim/app/api/v2/workflows/[workflowId]/execute/route.test.ts

Lines changed: 41 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -324,6 +324,7 @@ describe('POST /api/v2/workflows/[workflowId]/execute', () => {
324324
workflowId: 'workflow-1',
325325
status: 'completed',
326326
output: { result: 'done' },
327+
blockOutputs: null,
327328
error: null,
328329
durationMs: 42,
329330
})
@@ -590,12 +591,47 @@ describe('POST /api/v2/workflows/[workflowId]/execute', () => {
590591
expect(mockPreprocessExecution).not.toHaveBeenCalled()
591592
})
592593

593-
it('rejects selectedOutputs on a sync request rather than ignoring it', async () => {
594-
const res = await callExecute({ selectedOutputs: ['agent_1.content'] })
594+
it('returns blockOutputs for selectedOutputs on a sync request', async () => {
595+
const agentBlockId = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa'
596+
mockLoadDeployedWorkflowState.mockResolvedValue({
597+
blocks: { [agentBlockId]: { id: agentBlockId, name: 'Agent 1' } },
598+
edges: [],
599+
loops: {},
600+
parallels: {},
601+
variables: {},
602+
})
603+
mockExecuteWorkflowCore.mockResolvedValue({
604+
success: true,
605+
output: { result: 'done' },
606+
logs: [
607+
{
608+
blockId: agentBlockId,
609+
blockName: 'Agent 1',
610+
startedAt: 's',
611+
endedAt: 'e',
612+
durationMs: 5,
613+
success: true,
614+
output: { content: 'hi', tokens: { total: 7 } },
615+
},
616+
],
617+
metadata: {
618+
duration: 42,
619+
startTime: '2026-07-31T00:00:00.000Z',
620+
endTime: '2026-07-31T00:00:01.000Z',
621+
},
622+
})
595623

596-
expect(res.status).toBe(400)
597-
expect((await res.json()).error.message).toContain('selectedOutputs requires stream: true')
598-
expect(mockPreprocessExecution).not.toHaveBeenCalled()
624+
const res = await callExecute({
625+
input: {},
626+
selectedOutputs: ['Agent 1.content', 'Agent 1.absent', agentBlockId],
627+
})
628+
629+
expect(res.status).toBe(200)
630+
const body = await res.json()
631+
expect(body.data.blockOutputs).toEqual({
632+
'Agent 1.content': 'hi',
633+
[agentBlockId]: { content: 'hi', tokens: { total: 7 } },
634+
})
599635
})
600636

601637
it.each(['includeThinking', 'includeToolCalls'])(

apps/sim/app/api/v2/workflows/[workflowId]/execute/route.ts

Lines changed: 1 addition & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -256,19 +256,6 @@ export const POST = withRouteHandler(
256256
'Async execution does not support streaming or output-shaping options'
257257
)
258258
}
259-
/**
260-
* `selectedOutputs` shapes the streamed envelope only — the sync path
261-
* returns the workflow's own final output and never reads it. Accepting
262-
* it silently answered a full, unselected body to a caller who believed
263-
* they had narrowed it, so the option is refused where it does nothing
264-
* and the two paths that honour selection are named instead.
265-
*/
266-
if (body.selectedOutputs?.length && !body.stream) {
267-
return v2Error(
268-
'BAD_REQUEST',
269-
'selectedOutputs requires stream: true. For a completed run, request the run resource with ?selectedOutputs= instead.'
270-
)
271-
}
272259
const hasAgentStreamOptions = hasAgentStreamPolicy({
273260
includeThinking: body.includeThinking,
274261
includeToolCalls: body.includeToolCalls,
@@ -424,6 +411,7 @@ export const POST = withRouteHandler(
424411
workflowId: result.workflowId,
425412
status: result.status,
426413
output: result.output ?? null,
414+
blockOutputs: result.blockOutputs ?? null,
427415
error: result.error,
428416
startedAt: result.startedAt,
429417
endedAt: result.endedAt,

apps/sim/app/api/v2/workflows/[workflowId]/runs/[runId]/resume/route.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,8 @@ export const POST = withRouteHandler(
111111
workflowId,
112112
status: result.status as 'completed' | 'failed' | 'paused' | 'cancelled',
113113
output: result.output ?? null,
114+
// Resume has no request body to name selectors in, so selection never applies here.
115+
blockOutputs: null,
114116
error:
115117
typeof result.error === 'string'
116118
? classifyExecutionError(new Error(result.error))

apps/sim/lib/api/contracts/v2/openapi/workflows.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,7 @@ const RUN_RESULT_EXAMPLE = {
179179
workflowId: WORKFLOW_ID,
180180
status: 'completed',
181181
output: { result: 'Ticket routed to Support' },
182+
blockOutputs: null,
182183
error: null,
183184
startedAt: '2026-08-09T18:04:10.000Z',
184185
endedAt: '2026-08-09T18:04:11.000Z',

apps/sim/lib/api/contracts/v2/workflows.ts

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1293,7 +1293,7 @@ export const v2ExecuteWorkflowBodySchema = z
12931293
.max(100)
12941294
.optional()
12951295
.describe(
1296-
'Block output references to include in a streamed response, as `blockId`, `blockId.path`, or `BlockName.path` (resolved against the live workflow). Requires `stream: true` — it shapes the streamed envelope only, so it is rejected on a sync request and when `async` is true. To narrow a finished run, pass `selectedOutputs` to the run resource instead.'
1296+
'Block output references to include in the response, as `blockId`, `blockId.path`, or `BlockName.path` (resolved against the workflow state being run). On a sync request the named outputs come back in `blockOutputs`, keyed by these selector strings; on a stream they shape the streamed envelope. Selectors that resolve to no block or no value are omitted. Rejected when `async` is true — a queued run has produced nothing to select; narrow the finished run via the run resource instead.'
12971297
),
12981298
includeThinking: z
12991299
.boolean()
@@ -1365,6 +1365,12 @@ export const v2ExecuteWorkflowDataSchema = z
13651365
.enum(['completed', 'failed', 'paused', 'cancelled'])
13661366
.describe('Terminal or paused run status.'),
13671367
output: z.unknown().describe('Workflow output, including partial output on failure.'),
1368+
blockOutputs: z
1369+
.record(z.string(), z.unknown().describe('Output value produced by one workflow block.'))
1370+
.nullable()
1371+
.describe(
1372+
'Outputs of the blocks named by `selectedOutputs`, keyed by those selector strings, or null when none were requested. Selectors whose block did not run or whose path is absent are omitted; failed runs include the outputs of the blocks that did run.'
1373+
),
13681374
error: v2ExecutionErrorSchema
13691375
.nullable()
13701376
.describe('Structured execution failure, or null when none occurred.'),

apps/sim/lib/mothership/tools/cli-tool-display.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -224,6 +224,7 @@ export const CLI_TOOL_TITLES: Record<string, string> = {
224224
cli_workspaces_members: 'Listing workspace members',
225225
// Agent-only CLI augmentations
226226
cli_files_grep: 'Searching file contents',
227+
cli_logs_query: 'Querying run history',
227228
cli_workflow_blocks: 'Listing workflow blocks',
228229
cli_workflow_deps: 'Tracing block inputs',
229230
cli_workflow_edges: 'Reading workflow wiring',

apps/sim/lib/mothership/tools/handlers/agent-cli/agent-cli.test.ts

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -220,3 +220,130 @@ describe('workflow grep', () => {
220220
expect(result.stderr).toContain('Unexpected request')
221221
})
222222
})
223+
224+
describe('flag parsing', () => {
225+
it('collects --flag value, --flag=value, and bare flags without shifting positionals', () => {
226+
const match = matchAgentCliCommand([
227+
'logs',
228+
'query',
229+
'wf-1',
230+
'--block',
231+
'Router',
232+
'--limit=5',
233+
'--verbose',
234+
])
235+
expect(match?.rest).toEqual(['wf-1'])
236+
expect(match?.flags.get('block')).toBe('Router')
237+
expect(match?.flags.get('limit')).toBe('5')
238+
expect(match?.flags.get('verbose')).toBe(true)
239+
})
240+
})
241+
242+
describe('logs query', () => {
243+
const RUNS_PATH = '/api/v2/workflows/wf-1/runs'
244+
const runsResponse = {
245+
data: [
246+
{ runId: 'run-1', status: 'completed', startedAt: 't1' },
247+
{ runId: 'run-2', status: 'completed', startedAt: 't2' },
248+
{ runId: 'run-3', status: 'failed', startedAt: 't3' },
249+
],
250+
}
251+
const routedTrace = {
252+
data: {
253+
traceSpans: [
254+
{
255+
name: 'Start',
256+
children: [{ name: 'Router', status: 'success', output: { route: 'priority', n: 2 } }],
257+
},
258+
],
259+
},
260+
}
261+
const unroutedTrace = {
262+
data: { traceSpans: [{ name: 'Start', output: {} }] },
263+
}
264+
265+
it('emits one row per run with the block field dug from nested spans', async () => {
266+
const match = matchAgentCliCommand([
267+
'logs',
268+
'query',
269+
'wf-1',
270+
'--block',
271+
'Router',
272+
'--field',
273+
'output.route',
274+
])
275+
const result = await executeAgentCliCommand(
276+
match!,
277+
runtimeWith({
278+
[RUNS_PATH]: runsResponse,
279+
'/api/v2/logs/run-1': routedTrace,
280+
'/api/v2/logs/run-2': unroutedTrace,
281+
'/api/v2/logs/run-3': routedTrace,
282+
})
283+
)
284+
expect(result.exitCode).toBe(0)
285+
const report = JSON.parse(result.stdout)
286+
expect(report.runsScanned).toBe(3)
287+
expect(report.rows).toEqual([
288+
{
289+
runId: 'run-1',
290+
startedAt: 't1',
291+
runStatus: 'completed',
292+
hits: 1,
293+
blockStatus: 'success',
294+
value: 'priority',
295+
},
296+
{ runId: 'run-2', startedAt: 't2', runStatus: 'completed', hits: 0, value: null },
297+
{
298+
runId: 'run-3',
299+
startedAt: 't3',
300+
runStatus: 'failed',
301+
hits: 1,
302+
blockStatus: 'success',
303+
value: 'priority',
304+
},
305+
])
306+
})
307+
308+
it('filters rows with --where and reports unavailable traces instead of failing', async () => {
309+
const match = matchAgentCliCommand([
310+
'logs',
311+
'query',
312+
'wf-1',
313+
'--block',
314+
'Router',
315+
'--where',
316+
'output.route=priority',
317+
])
318+
const result = await executeAgentCliCommand(
319+
match!,
320+
runtimeWith({
321+
[RUNS_PATH]: runsResponse,
322+
'/api/v2/logs/run-1': routedTrace,
323+
'/api/v2/logs/run-2': unroutedTrace,
324+
})
325+
)
326+
expect(result.exitCode).toBe(0)
327+
const report = JSON.parse(result.stdout)
328+
expect(report.missingTrace).toBe(1)
329+
expect(report.rows).toEqual([
330+
{
331+
runId: 'run-1',
332+
startedAt: 't1',
333+
runStatus: 'completed',
334+
hits: 1,
335+
blockStatus: 'success',
336+
value: { route: 'priority', n: 2 },
337+
},
338+
{ runId: 'run-2', startedAt: 't2', runStatus: 'completed', hits: 0, value: null },
339+
{ runId: 'run-3', startedAt: 't3', runStatus: 'failed', note: 'trace unavailable' },
340+
])
341+
})
342+
343+
it('fails usefully without a workflow id or --block', async () => {
344+
const match = matchAgentCliCommand(['logs', 'query', 'wf-1'])
345+
const result = await executeAgentCliCommand(match!, runtimeWith({}))
346+
expect(result.exitCode).toBe(1)
347+
expect(result.stderr).toContain('--block')
348+
})
349+
})

0 commit comments

Comments
 (0)