Skip to content

Commit 4941c73

Browse files
feat(streaming): support nested workflow outputs (#7346)
* feat(streaming): support nested workflow outputs * fix(streaming): reset stale output drilldowns * fix(streaming): validate nested output selectors * fix(slack): use public output selectors * fix(streaming): scope nested outputs by workflow * fix(cli): align streaming selector help test
1 parent fd061f5 commit 4941c73

48 files changed

Lines changed: 1971 additions & 376 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

apps/docs/content/docs/cli/reference.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5157,7 +5157,7 @@ sim workflows run <workflowId> [options]
51575157
| `--input <json\|@file>` | No | Trigger input as JSON (JSON, or @path / @- to read a file or stdin). |
51585158
| `--async` | No | Queue the run and return immediately. |
51595159
| `--execution-timeout-seconds <value>` | No | Requested server-side timeout for an asynchronous run, in seconds. An upper bound, not the effective timeout: the run uses the smaller of this value and the plan's execution timeout, so requesting more than the plan allows silently yields the plan timeout. Rejected with `400` unless `async` is true. |
5160-
| `--select-output <value...>` | No | Return blockName.field values from the streamed result (e.g. agent_1.content), requires --follow; missing fields are omitted (space-separated, or @path / @- with one value per line; @@value for a literal leading @). |
5160+
| `--select-output <value...>` | No | Return streamed outputs as blockName.path or childWorkflowId.blockName.path; selecting a child workflow applies to every invocation, requires --follow (space-separated, or @path / @- with one value per line; @@value for a literal leading @). |
51615161
| `--include-file-base64` | No | Inline eligible output files as base64 content. Rejected when `async` is true. |
51625162
| `--no-include-file-base64` | No | Send --include-file-base64 as false. |
51635163
| `--base64-max-bytes <value>` | No | Maximum total bytes of file content to inline as base64, lowering but never raising the server limit of 16 MiB. Rejected when `async` is true. |

apps/docs/content/docs/cli/workflows.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -532,7 +532,7 @@ sim workflows run <workflowId> [options]
532532
| `--input <json\|@file>` | No | Trigger input as JSON (JSON, or @path / @- to read a file or stdin). |
533533
| `--async` | No | Queue the run and return immediately. |
534534
| `--execution-timeout-seconds <value>` | No | Requested server-side timeout for an asynchronous run, in seconds. An upper bound, not the effective timeout: the run uses the smaller of this value and the plan's execution timeout, so requesting more than the plan allows silently yields the plan timeout. Rejected with `400` unless `async` is true. |
535-
| `--select-output <value...>` | No | Return blockName.field values from the streamed result (e.g. agent_1.content), requires --follow; missing fields are omitted (space-separated, or @path / @- with one value per line; @@value for a literal leading @). |
535+
| `--select-output <value...>` | No | Return streamed outputs as blockName.path or childWorkflowId.blockName.path; selecting a child workflow applies to every invocation, requires --follow (space-separated, or @path / @- with one value per line; @@value for a literal leading @). |
536536
| `--include-file-base64` | No | Inline eligible output files as base64 content. Rejected when `async` is true. |
537537
| `--no-include-file-base64` | No | Send --include-file-base64 as false. |
538538
| `--base64-max-bytes <value>` | No | Maximum total bytes of file content to inline as base64, lowering but never raising the server limit of 16 MiB. Rejected when `async` is true. |

apps/docs/content/docs/integrations/slack.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1971,7 +1971,7 @@ Trigger from Slack events, interactions, and slash commands
19711971
| `manualChannelFilter` | string | No | Comma-separated channel IDs to restrict to. Set IDs directly here. |
19721972
| `threads` | string | No | Include thread replies, exclude them \(top-level only\), or fire only on thread replies. |
19731973
| `streamResponse` | boolean | No | Create a Slack agent session and stream selected workflow outputs into the conversation that started this run. Custom bots only. |
1974-
| `streamOutputs` | workflow-output-selector | No | Each selected block invocation creates its own Slack response. Agent outputs stream live; other outputs are sent when the block completes. |
1974+
| `streamOutputs` | workflow-output-selector | No | Use `&lt;blockName&gt;.&lt;outputPath&gt;` for this workflow or `&lt;childWorkflowId&gt;.&lt;blockName&gt;.&lt;outputPath&gt;` for a child workflow. Selecting a child workflow applies to every invocation of it. Agent outputs stream live; other outputs are sent when the block completes. |
19751975
| `streamTaskTitle` | string | No | Optional status Slack shows while each selected response is being produced. Leave empty to use Running. |
19761976
| `streamTaskDisplayMode` | string | No | Choose how Slack displays thinking and tool progress. |
19771977
| `streamIncludeThinking` | boolean | No | Show agent thinking as Slack task updates while the response is generated. |

apps/docs/openapi-v2-workflows.json

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8593,6 +8593,10 @@
85938593
"StoredChatDeploymentOutputConfig": {
85948594
"type": "object",
85958595
"properties": {
8596+
"workflowId": {
8597+
"description": "Child workflow containing the selected block. Omitted for the deployed workflow.",
8598+
"type": "string"
8599+
},
85968600
"blockId": {
85978601
"type": "string",
85988602
"description": "Block whose output the chat streams."
@@ -8902,6 +8906,11 @@
89028906
"ChatDeploymentOutputConfig": {
89038907
"type": "object",
89048908
"properties": {
8909+
"workflowId": {
8910+
"description": "Child workflow containing the selected block. Omit for the deployed workflow.",
8911+
"type": "string",
8912+
"minLength": 1
8913+
},
89058914
"blockId": {
89068915
"type": "string",
89078916
"minLength": 1,
@@ -9305,7 +9314,7 @@
93059314
"type": "boolean"
93069315
},
93079316
"selectedOutputs": {
9308-
"description": "Block output references to include in a streamed response, as `blockId`, `blockId.path`, or `BlockName.path` (resolved against the live workflow). Requires `stream: true` — it shapes the streamed envelope only, so it is rejected on a sync request and when `async` is true. To narrow a finished run, pass `selectedOutputs` to the run resource instead.",
9317+
"description": "Block output references to include in a streamed response. Use `<blockName>.<outputPath>` for the executed workflow or `<childWorkflowId>.<blockName>.<outputPath>` for a child workflow; block names are normalized workflow reference names. Selecting a child workflow applies to every invocation of it. Requires `stream: true` — it shapes the streamed envelope only, so it is rejected on a sync request and when `async` is true. To narrow a finished run, pass `selectedOutputs` to the run resource instead.",
93099318
"maxItems": 100,
93109319
"type": "array",
93119320
"items": {

apps/sim/app/(interfaces)/chat/hooks/use-chat-streaming.ts

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ import {
2121
isChatThinkingFrame,
2222
isChatToolFrame,
2323
} from '@/lib/workflows/streaming/agent-stream-protocol'
24+
import { scopeOutputBlockId } from '@/lib/workflows/streaming/output-selector'
2425
import type {
2526
ChatFile,
2627
ChatMessage,
@@ -70,7 +71,7 @@ function extractFilesFromData(
7071
}
7172

7273
export interface StreamingOptions {
73-
outputConfigs?: Array<{ blockId: string; path?: string }>
74+
outputConfigs?: Array<{ workflowId?: string; blockId: string; path?: string }>
7475
/**
7576
* Shared AbortController for fetch + SSE body reads. When provided (preferred),
7677
* Stop aborts the in-flight request server-side as well as the reader.
@@ -430,7 +431,10 @@ export function useChatStreaming() {
430431

431432
if (outputConfigs?.length && finalData.output) {
432433
for (const config of outputConfigs) {
433-
const blockOutputs = finalData.output[config.blockId]
434+
const outputBlockId = config.workflowId
435+
? scopeOutputBlockId(config.workflowId, config.blockId)
436+
: config.blockId
437+
const blockOutputs = finalData.output[outputBlockId]
434438
if (!blockOutputs) continue
435439

436440
const value = getOutputValue(blockOutputs, config.path)

apps/sim/app/api/chat/[identifier]/route.ts

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
1414
import { preprocessExecution } from '@/lib/execution/preprocessing'
1515
import { LoggingSession } from '@/lib/logs/execution/logging-session'
1616
import { ChatFiles } from '@/lib/uploads'
17+
import { formatOutputSelector } from '@/lib/workflows/streaming/output-selector'
1718
import { setChatAuthCookie, validateChatAuth } from '@/app/api/chat/utils'
1819
import { createErrorResponse, createSuccessResponse } from '@/app/api/workflows/utils'
1920

@@ -213,9 +214,11 @@ export const POST = withRouteHandler(
213214
const selectedOutputs: string[] = []
214215
if (deployment.outputConfigs && Array.isArray(deployment.outputConfigs)) {
215216
for (const config of deployment.outputConfigs) {
216-
const outputId = config.path
217-
? `${config.blockId}_${config.path}`
218-
: `${config.blockId}_content`
217+
const outputId = formatOutputSelector(
218+
config.blockId,
219+
config.path || 'content',
220+
config.workflowId
221+
)
219222
selectedOutputs.push(outputId)
220223
}
221224
}

apps/sim/app/api/v2/chat-deployments/utils.ts

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,9 +51,13 @@ function normalizeStoredOutputConfigs(raw: unknown): V2ChatDeploymentOutputConfi
5151
const configs: V2ChatDeploymentOutputConfig[] = []
5252
for (const entry of raw) {
5353
if (!entry || typeof entry !== 'object' || Array.isArray(entry)) continue
54-
const { blockId, path } = entry as Record<string, unknown>
54+
const { workflowId, blockId, path } = entry as Record<string, unknown>
5555
if (typeof blockId !== 'string' || blockId.length === 0) continue
56-
configs.push({ blockId, path: typeof path === 'string' ? path : '' })
56+
configs.push({
57+
...(typeof workflowId === 'string' && workflowId.length > 0 ? { workflowId } : {}),
58+
blockId,
59+
path: typeof path === 'string' ? path : '',
60+
})
5761
}
5862
return configs
5963
}

apps/sim/app/api/v2/workflows/[workflowId]/execute/route.test.ts

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -579,6 +579,31 @@ describe('POST /api/v2/workflows/[workflowId]/execute', () => {
579579
)
580580
})
581581

582+
it('maps malformed nested output selectors to an input failure', async () => {
583+
const result = await executeWorkflowService({
584+
workflowId: 'workflow-1',
585+
principal: { kind: 'personal_api_key', userId: 'actor-1', keyId: 'key-1' },
586+
userId: 'actor-1',
587+
input: {},
588+
triggerType: 'api',
589+
requestId: 'request-1',
590+
workflowRecord,
591+
selectedOutputs: ['workflow//agent.content'],
592+
mode: 'stream',
593+
requestHeaders: new Headers(),
594+
})
595+
596+
expect(result).toEqual({
597+
ok: false,
598+
failure: {
599+
kind: 'input',
600+
message: 'Invalid selectedOutputs: Invalid output selector: workflow//agent.content',
601+
statusCode: 400,
602+
},
603+
})
604+
expect(mockReleaseExecutionSlot).toHaveBeenCalledWith('execution-123')
605+
})
606+
582607
it('rejects async manual execution and conflicting mock input before dispatch', async () => {
583608
authenticatePersonalKey()
584609

apps/sim/app/api/workflows/[id]/execute/route.ts

Lines changed: 22 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@ import { workflow as workflowTable } from '@sim/db/schema'
44
import { createLogger } from '@sim/logger'
55
import { authorizeWorkflowByWorkspacePermission } from '@sim/platform-authz/workflow'
66
import { getErrorMessage, toError } from '@sim/utils/errors'
7-
import { generateId, isValidUuid } from '@sim/utils/id'
7+
import { generateId } from '@sim/utils/id'
8+
import type { BlockState } from '@sim/workflow-types/workflow'
89
import { eq } from 'drizzle-orm'
910
import { type NextRequest, NextResponse } from 'next/server'
1011
import {
@@ -141,6 +142,7 @@ import {
141142
forwardAgentStreamToExecutionEvents,
142143
shouldForwardAnswerTextFromSink,
143144
} from '@/lib/workflows/streaming/forward-agent-stream-events'
145+
import { resolveOutputSelectors } from '@/lib/workflows/streaming/resolve-output-selectors'
144146
import {
145147
agentStreamProtocolResponseHeaders,
146148
createStreamingResponse,
@@ -152,7 +154,6 @@ import {
152154
PublicApiNotAllowedError,
153155
validatePublicApiAllowed,
154156
} from '@/ee/access-control/utils/permission-check'
155-
import { normalizeName } from '@/executor/constants'
156157
import { ExecutionSnapshot } from '@/executor/execution/snapshot'
157158
import type {
158159
BlockCompletionCallbackData,
@@ -298,56 +299,13 @@ function payloadTooLargeResponse(message = 'Workflow execution response exceeds
298299
)
299300
}
300301

301-
function resolveOutputIds(
302+
async function resolveOutputIds(
302303
selectedOutputs: string[] | undefined,
303-
blocks: Record<string, any>
304-
): string[] | undefined {
305-
if (!selectedOutputs || selectedOutputs.length === 0) {
306-
return selectedOutputs
307-
}
308-
309-
return selectedOutputs.map((outputId) => {
310-
const underscoreIndex = outputId.indexOf('_')
311-
const dotIndex = outputId.indexOf('.')
312-
if (underscoreIndex > 0) {
313-
const maybeUuid = outputId.substring(0, underscoreIndex)
314-
if (isValidUuid(maybeUuid)) {
315-
return outputId
316-
}
317-
}
318-
319-
if (dotIndex > 0) {
320-
const maybeUuid = outputId.substring(0, dotIndex)
321-
if (isValidUuid(maybeUuid)) {
322-
return `${outputId.substring(0, dotIndex)}_${outputId.substring(dotIndex + 1)}`
323-
}
324-
}
325-
326-
if (isValidUuid(outputId)) {
327-
return outputId
328-
}
329-
330-
if (dotIndex === -1) {
331-
logger.warn(`Invalid output ID format (missing dot): ${outputId}`)
332-
return outputId
333-
}
334-
335-
const blockName = outputId.substring(0, dotIndex)
336-
const path = outputId.substring(dotIndex + 1)
337-
338-
const normalizedBlockName = normalizeName(blockName)
339-
const block = Object.values(blocks).find((b: any) => {
340-
return normalizeName(b.name || '') === normalizedBlockName
341-
})
342-
343-
if (!block) {
344-
logger.warn(`Block not found for name: ${blockName} (from output ID: ${outputId})`)
345-
return outputId
346-
}
347-
348-
const resolvedId = `${block.id}_${path}`
349-
logger.debug(`Resolved output ID: ${outputId} -> ${resolvedId}`)
350-
return resolvedId
304+
blocks: Record<string, BlockState>
305+
): Promise<string[] | undefined> {
306+
return resolveOutputSelectors({
307+
selectedOutputs,
308+
currentBlocks: blocks,
351309
})
352310
}
353311

@@ -1671,10 +1629,19 @@ async function handleExecutePost(
16711629
} else {
16721630
reqLogger.info('Using streaming API response')
16731631

1674-
const resolvedSelectedOutputs = resolveOutputIds(
1675-
selectedOutputs,
1676-
cachedWorkflowData?.blocks || {}
1677-
)
1632+
let resolvedSelectedOutputs: string[] | undefined
1633+
try {
1634+
resolvedSelectedOutputs = await resolveOutputIds(
1635+
selectedOutputs,
1636+
cachedWorkflowData?.blocks || {}
1637+
)
1638+
} catch (error) {
1639+
await releaseExecutionSlot(executionId)
1640+
return NextResponse.json(
1641+
{ error: `Invalid selectedOutputs: ${getErrorMessage(error)}` },
1642+
{ status: 400 }
1643+
)
1644+
}
16781645
const streamVariables = cachedWorkflowData?.variables ?? (workflow as any).variables
16791646
const streamWorkflow = {
16801647
id: workflow.id,

0 commit comments

Comments
 (0)