Skip to content

Commit 1903ddb

Browse files
author
Mark Pollack
committed
Framework smoke matrix: Micronaut agent and client cells
programs/micronaut now hosts the shared smoke agent and client on acp-micronaut (stdio, HTTP and WebSocket on the SDK listener; a client built from acp.client.*). fw-micronaut-agent-{stdio,http} replace micronaut-typescript-{stdio,http}: their steps are a subset of the smoke steps. The sample application keeps its own process tests.
1 parent d3b5e4e commit 1903ddb

16 files changed

Lines changed: 402 additions & 54 deletions

‎.github/workflows/cross-sdk.yml‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -49,15 +49,15 @@ jobs:
4949
PEER: ${{ github.event.inputs.peer || 'all' }}
5050
run: |
5151
# Per peer: its toolchain, its transports (cells x-java-<peer>-<t> and x-<peer>-java-<t>),
52-
# and the cache paths of its build. A peer's legs also run its micronaut-<peer>-<t> and
53-
# quarkus-<peer>-<t> scenarios (the acp-micronaut sample agent and the Quarkus-hosted agent
54-
# against that peer's client), where they exist.
55-
# java runs as one leg: the Java <-> Java cells, conformance, load.
52+
# and the cache paths of its build. A peer's legs also run its quarkus-<peer>-<t> scenarios
53+
# (the Quarkus-hosted agent against that peer's client, with 1 MB and 8 MB payloads), where
54+
# they exist. The framework smoke cells (fw-*, load-10-fw-*) run in framework-smoke.yml.
55+
# java runs as one leg: the Java <-> Java cells, conformance, load (not load-10-fw-*).
5656
# kotlin has no Streamable HTTP; its Gradle toolchain is JDK 21 while the SDK and the
5757
# Java programs stay on 17 (the last version listed is the default).
5858
peers="$(jq -c . <<'JSON'
5959
[
60-
{ "peer": "java", "java": "17", "only": "x-java-java-*,conf-*,load-*" },
60+
{ "peer": "java", "java": "17", "only": "x-java-java-*,conf-*,load-50,load-300,load-1000,load-shared-300" },
6161
{ "peer": "typescript", "sdk": "typescript-sdk", "java": "17", "node": true,
6262
"transports": ["stdio", "http", "ws"],
6363
"cache-paths": "integration-testing/.cache/peers/typescript-sdk@*\n~/.npm" },
@@ -75,11 +75,11 @@ jobs:
7575
)"
7676
picked="$(jq -c --arg p "$PEER" '[.[] | select($p == "all" or .peer == $p)]' <<< "$peers")"
7777
prepare="$(jq -c '{include: [.[] | select(.sdk)
78-
| . + {only: (["x-*-\(.peer)-*", "x-\(.peer)-*", "micronaut-\(.peer)-*", "quarkus-\(.peer)-*"] | join(","))}
78+
| . + {only: (["x-*-\(.peer)-*", "x-\(.peer)-*", "quarkus-\(.peer)-*"] | join(","))}
7979
| del(.transports)]}' <<< "$picked")"
8080
cells="$(jq -c '{include: [.[] as $x | ($x.transports // ["all"])[] as $t
8181
| $x + (if $t == "all" then {leg: $x.peer} else {leg: "\($x.peer)-\($t)", transport: $t,
82-
only: (["x-java-\($x.peer)-\($t)*", "x-\($x.peer)-java-\($t)*", "micronaut-\($x.peer)-\($t)*",
82+
only: (["x-java-\($x.peer)-\($t)*", "x-\($x.peer)-java-\($t)*",
8383
"quarkus-\($x.peer)-\($t)*"]
8484
| join(","))} end)
8585
| del(.transports)]}' <<< "$picked")"

‎integration-testing/README.md‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,6 @@ artifact.
105105
| `quarkus-typescript-http`, `quarkus-typescript-ws` | hand-written: the TypeScript client program against the Quarkus-hosted agent (`programs/quarkus`, an `@AcpAgent` bean served by `acp-quarkus` on the Quarkus HTTP server), running the catalogue steps that agent implements (initialize, sessions, echo chunks, stop reasons, -32601, 1 MB and 8 MB prompts and updates). Run in the `typescript-http`/`typescript-ws` legs of `cross-sdk.yml` |
106106
| `load-50`, `load-300`, `load-1000` | N Java clients on their own connections, each: initialize, session/new, then 10 (or 5) prompts streaming two updates each |
107107
| `load-shared-300` | 300 clients sharing one HttpClient, so one HTTP/2 connection |
108-
| `micronaut-typescript-stdio`, `micronaut-typescript-http` | hand-written: the TypeScript client against the `acp-micronaut` sample agent ([programs/micronaut](programs/micronaut), an annotated `@AcpAgent` bean served by Micronaut) over the steps a plain echo agent answers; over stdio, `stdio.eof-exit` checks that the Micronaut application exits when its input ends. They run in the TypeScript legs of `cross-sdk.yml` |
109108

110109
Over Streamable HTTP the cells also assert, through `matrix.json` facts, what the retired
111110
hand-written `interop-*` scenarios checked: the negotiated HTTP version on both sides (h2c between

integration-testing/configs/micronaut-typescript-http.json renamed to integration-testing/configs/fw-micronaut-agent-http.json

Lines changed: 24 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
2-
"name": "micronaut-typescript-http",
3-
"description": "typescript client -> micronaut-hosted java agent over http, 6 steps",
4-
"timeoutSec": 300,
2+
"name": "fw-micronaut-agent-http",
3+
"description": "typescript client -> micronaut-hosted java agent over http, 14 smoke steps (generated from smoke.json)",
4+
"timeoutSec": 370,
55
"peers": [
66
"typescript-sdk"
77
],
@@ -13,10 +13,12 @@
1313
"dir": "${ROOT}/programs/micronaut",
1414
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" ${ROOT}/programs/micronaut/launch/build.sh",
1515
"run": "${ROOT}/programs/micronaut/launch/agent.sh --transport http --port ${PORT}",
16-
"env": {},
16+
"env": {
17+
"TS_SDK": "${peer.typescript-sdk}"
18+
},
1719
"ready": {
1820
"line": "READY",
19-
"timeoutSec": 60
21+
"timeoutSec": 90
2022
}
2123
},
2224
{
@@ -27,33 +29,41 @@
2729
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" TS_SDK=\"${peer.typescript-sdk}\" ${ROOT}/programs/node/launch/build.sh",
2830
"run": "${ROOT}/programs/node/launch/client.sh --transport http --url http://127.0.0.1:${PORT}/acp",
2931
"env": {
30-
"STEPS": "init.initialize,session.new,update.agent_message_chunk,session.multi,error.method-not-found,conn.close",
32+
"STEPS": "init.initialize,init.agent-capabilities,init.agent-info,session.new,session.load,session.multi,update.agent_message_chunk,perm.selected,elicit.form,cancel.prompt,ext.client-request,error.method-not-found,http.reconnect,conn.close",
3133
"TS_SDK": "${peer.typescript-sdk}"
3234
},
33-
"timeoutSec": 240
35+
"timeoutSec": 190
3436
}
3537
],
3638
"assertions": {
39+
"requiredOutput": {
40+
"agent": [
41+
"STEP agent.perm.selected PASS",
42+
"STEP agent.elicit.form PASS",
43+
"STEP agent.cancel.prompt PASS"
44+
]
45+
},
3746
"forbiddenOutput": {
3847
"agent": [
3948
" ERROR ",
4049
" WARN "
4150
]
4251
},
52+
"requiredPatterns": {
53+
"client": [
54+
"STEP init\\.initialize PASS .*\"name\":\"interop-micronaut-agent\""
55+
]
56+
},
4357
"checks": [
44-
"client RESULT pass == 6",
58+
"client RESULT pass == 14",
4559
"client RESULT fail == 0"
4660
],
61+
"expectedFailures": [],
4762
"report": {
4863
"client": [
4964
"TIMEOUT"
5065
]
51-
},
52-
"requiredPatterns": {
53-
"client": [
54-
"STEP init\\.initialize PASS .*agentInfo=\\{\"name\":\"micronaut-echo-agent\",\"version\":\"1\\.0\\.0\""
55-
]
5666
}
5767
},
58-
"comment": "Hand-written, not generated: the acp-micronaut sample agent (an annotated @AcpAgent bean served by acp-micronaut, programs/micronaut) against the TypeScript client, over the catalogue steps a plain echo agent answers. stdio.eof-exit checks that the Micronaut application exits when its input ends. The initialize step requires the agentInfo the agent's @AcpAgent annotation declares."
68+
"comment": "GENERATED by GenConfigs.java from smoke.json, steps.json and matrix.json; do not edit, regenerate."
5969
}

integration-testing/configs/micronaut-typescript-stdio.json renamed to integration-testing/configs/fw-micronaut-agent-stdio.json

Lines changed: 20 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
2-
"name": "micronaut-typescript-stdio",
3-
"description": "typescript client -> micronaut-hosted java agent over stdio, 7 steps",
4-
"timeoutSec": 300,
2+
"name": "fw-micronaut-agent-stdio",
3+
"description": "typescript client -> micronaut-hosted java agent over stdio, 14 smoke steps (generated from smoke.json)",
4+
"timeoutSec": 370,
55
"peers": [
66
"typescript-sdk"
77
],
@@ -14,34 +14,42 @@
1414
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" TS_SDK=\"${peer.typescript-sdk}\" ${ROOT}/programs/node/launch/build.sh && IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" ${ROOT}/programs/micronaut/launch/build.sh",
1515
"run": "${ROOT}/programs/node/launch/client.sh --transport stdio",
1616
"env": {
17-
"STEPS": "init.initialize,session.new,update.agent_message_chunk,session.multi,error.method-not-found,stdio.eof-exit,conn.close",
17+
"STEPS": "init.initialize,init.agent-capabilities,init.agent-info,session.new,session.load,session.multi,update.agent_message_chunk,perm.selected,elicit.form,cancel.prompt,ext.client-request,error.method-not-found,stdio.eof-exit,conn.close",
1818
"AGENT_CMD": "${ROOT}/programs/micronaut/launch/agent.sh --transport stdio",
1919
"TS_SDK": "${peer.typescript-sdk}"
2020
},
21-
"timeoutSec": 240
21+
"timeoutSec": 190
2222
}
2323
],
2424
"assertions": {
25+
"requiredOutput": {
26+
"client": [
27+
"STEP agent.perm.selected PASS",
28+
"STEP agent.elicit.form PASS",
29+
"STEP agent.cancel.prompt PASS"
30+
]
31+
},
2532
"forbiddenOutput": {
2633
"client": [
2734
" ERROR ",
2835
" WARN "
2936
]
3037
},
38+
"requiredPatterns": {
39+
"client": [
40+
"STEP init\\.initialize PASS .*\"name\":\"interop-micronaut-agent\""
41+
]
42+
},
3143
"checks": [
32-
"client RESULT pass == 7",
44+
"client RESULT pass == 14",
3345
"client RESULT fail == 0"
3446
],
47+
"expectedFailures": [],
3548
"report": {
3649
"client": [
3750
"TIMEOUT"
3851
]
39-
},
40-
"requiredPatterns": {
41-
"client": [
42-
"STEP init\\.initialize PASS .*agentInfo=\\{\"name\":\"micronaut-echo-agent\",\"version\":\"1\\.0\\.0\""
43-
]
4452
}
4553
},
46-
"comment": "Hand-written, not generated: the acp-micronaut sample agent (an annotated @AcpAgent bean served by acp-micronaut, programs/micronaut) against the TypeScript client, over the catalogue steps a plain echo agent answers. stdio.eof-exit checks that the Micronaut application exits when its input ends. The initialize step requires the agentInfo the agent's @AcpAgent annotation declares."
54+
"comment": "GENERATED by GenConfigs.java from smoke.json, steps.json and matrix.json; do not edit, regenerate."
4755
}

‎integration-testing/configs/fw-micronaut-agent-ws.json‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
{
2+
"name": "fw-micronaut-agent-ws",
3+
"description": "typescript client -> micronaut-hosted java agent over ws, 13 smoke steps (generated from smoke.json)",
4+
"timeoutSec": 365,
5+
"peers": [
6+
"typescript-sdk"
7+
],
8+
"processes": [
9+
{
10+
"name": "agent",
11+
"language": "java",
12+
"role": "server",
13+
"dir": "${ROOT}/programs/micronaut",
14+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" ${ROOT}/programs/micronaut/launch/build.sh",
15+
"run": "${ROOT}/programs/micronaut/launch/agent.sh --transport ws --port ${PORT}",
16+
"env": {
17+
"TS_SDK": "${peer.typescript-sdk}"
18+
},
19+
"ready": {
20+
"line": "READY",
21+
"timeoutSec": 90
22+
}
23+
},
24+
{
25+
"name": "client",
26+
"language": "typescript",
27+
"role": "client",
28+
"dir": "${ROOT}/programs/node",
29+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" TS_SDK=\"${peer.typescript-sdk}\" ${ROOT}/programs/node/launch/build.sh",
30+
"run": "${ROOT}/programs/node/launch/client.sh --transport ws --url ws://127.0.0.1:${PORT}/acp",
31+
"env": {
32+
"STEPS": "init.initialize,init.agent-capabilities,init.agent-info,session.new,session.load,session.multi,update.agent_message_chunk,perm.selected,elicit.form,cancel.prompt,ext.client-request,error.method-not-found,conn.close",
33+
"TS_SDK": "${peer.typescript-sdk}"
34+
},
35+
"timeoutSec": 185
36+
}
37+
],
38+
"assertions": {
39+
"requiredOutput": {
40+
"agent": [
41+
"STEP agent.perm.selected PASS",
42+
"STEP agent.elicit.form PASS",
43+
"STEP agent.cancel.prompt PASS"
44+
]
45+
},
46+
"forbiddenOutput": {
47+
"agent": [
48+
" ERROR ",
49+
" WARN "
50+
]
51+
},
52+
"requiredPatterns": {
53+
"client": [
54+
"STEP init\\.initialize PASS .*\"name\":\"interop-micronaut-agent\""
55+
]
56+
},
57+
"checks": [
58+
"client RESULT pass == 13",
59+
"client RESULT fail == 0"
60+
],
61+
"expectedFailures": [],
62+
"report": {
63+
"client": [
64+
"TIMEOUT"
65+
]
66+
}
67+
},
68+
"comment": "GENERATED by GenConfigs.java from smoke.json, steps.json and matrix.json; do not edit, regenerate."
69+
}

‎integration-testing/configs/fw-micronaut-client-http.json‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
{
2+
"name": "fw-micronaut-client-http",
3+
"description": "micronaut-built java client -> typescript agent over http, 10 smoke steps (generated from smoke.json)",
4+
"timeoutSec": 350,
5+
"peers": [
6+
"typescript-sdk"
7+
],
8+
"processes": [
9+
{
10+
"name": "agent",
11+
"language": "typescript",
12+
"role": "server",
13+
"dir": "${ROOT}/programs/node",
14+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" TS_SDK=\"${peer.typescript-sdk}\" ${ROOT}/programs/node/launch/build.sh",
15+
"run": "${ROOT}/programs/node/launch/agent.sh --transport http --port ${PORT}",
16+
"env": {
17+
"TS_SDK": "${peer.typescript-sdk}"
18+
},
19+
"ready": {
20+
"line": "READY",
21+
"timeoutSec": 60
22+
}
23+
},
24+
{
25+
"name": "client",
26+
"language": "java",
27+
"role": "client",
28+
"dir": "${ROOT}/programs/micronaut",
29+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" ${ROOT}/programs/micronaut/launch/build.sh",
30+
"run": "${ROOT}/programs/micronaut/launch/client.sh --transport http --url http://127.0.0.1:${PORT}/acp",
31+
"env": {
32+
"STEPS": "init.initialize,init.agent-info,session.new,update.agent_message_chunk,perm.selected,fs.read,elicit.form,cancel.prompt,ext.client-request,conn.close",
33+
"TS_SDK": "${peer.typescript-sdk}"
34+
},
35+
"timeoutSec": 170
36+
}
37+
],
38+
"assertions": {
39+
"requiredOutput": {
40+
"agent": [
41+
"STEP agent.perm.selected PASS",
42+
"STEP agent.fs.read PASS",
43+
"STEP agent.elicit.form PASS",
44+
"STEP agent.cancel.prompt PASS"
45+
]
46+
},
47+
"checks": [
48+
"client RESULT pass == 10",
49+
"client RESULT fail == 0"
50+
],
51+
"expectedFailures": [],
52+
"report": {
53+
"client": [
54+
"TIMEOUT"
55+
]
56+
}
57+
},
58+
"comment": "GENERATED by GenConfigs.java from smoke.json, steps.json and matrix.json; do not edit, regenerate."
59+
}

‎integration-testing/configs/load-10-fw-micronaut.json‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
{
2+
"name": "load-10-fw-micronaut",
3+
"description": "10 connections x 10 prompts against the micronaut-hosted agent over http, two streamed updates each (generated from smoke.json)",
4+
"timeoutSec": 420,
5+
"processes": [
6+
{
7+
"name": "server",
8+
"language": "java",
9+
"role": "server",
10+
"dir": "${ROOT}/programs/micronaut",
11+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" ${ROOT}/programs/micronaut/launch/build.sh",
12+
"run": "${ROOT}/programs/micronaut/launch/agent.sh --transport http --port ${PORT}",
13+
"env": {
14+
"JAVA_OPTS": "-Xmx512m"
15+
},
16+
"ready": {
17+
"line": "READY",
18+
"timeoutSec": 90
19+
}
20+
},
21+
{
22+
"name": "clients",
23+
"language": "java",
24+
"role": "client",
25+
"dir": "${ROOT}/programs/java",
26+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" ${ROOT}/programs/java/launch/build.sh",
27+
"run": "java -Xmx1g -Dacp.client.level=INFO -cp target/classes:$(cat target/classpath.txt) load.LoadClients http://127.0.0.1:${PORT}/acp 10 10 perclient",
28+
"timeoutSec": 300
29+
}
30+
],
31+
"assertions": {
32+
"requiredOutput": {
33+
"clients": [
34+
"PEAK_REACHED",
35+
"CLOSED"
36+
]
37+
},
38+
"forbiddenOutput": {
39+
"clients": [
40+
"ERROR ",
41+
"CLOSE_ERROR"
42+
],
43+
"server": [
44+
" ERROR "
45+
]
46+
},
47+
"checks": [
48+
"clients RESULT errors == 0",
49+
"clients RESULT ok == expected",
50+
"clients RESULT expected == 100",
51+
"clients RESULT updates == 2*ok",
52+
"clients RESULT close_errors == 0"
53+
]
54+
},
55+
"comment": "GENERATED by GenConfigs.java from smoke.json, steps.json and matrix.json; do not edit, regenerate."
56+
}

‎integration-testing/programs/micronaut/launch/agent.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
#!/usr/bin/env bash
2-
# Starts the Micronaut-hosted agent: --transport stdio | --transport http|ws --port <port>.
2+
# Starts the Micronaut smoke agent: --transport stdio | --transport http|ws --port <port>.
33
# Launcher contract: integration-testing/README.md, "Contracts". Prints nothing itself: on stdio,
44
# stdout is the protocol stream (the application logs to stderr). JAVA_OPTS, if set, is passed on.
55
set -euo pipefail
66
here="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
77
# shellcheck disable=SC2086
8-
exec java ${JAVA_OPTS:-} -cp "$here/target/classes:$(cat "$here/target/classpath.txt")" interop.MicronautAgent "$@"
8+
exec java ${JAVA_OPTS:-} -cp "$here/target/classes:$(cat "$here/target/classpath.txt")" interop.micronaut.MicronautAgentApp "$@"

‎integration-testing/programs/micronaut/launch/build.sh‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,13 @@
11
#!/usr/bin/env bash
2-
# Builds the Micronaut-hosted agent against the SDK installed from this checkout.
3-
# Launcher contract: integration-testing/README.md, "Contracts". Env: MVNW, ACP_VERSION.
2+
# Builds the Micronaut smoke programs against the SDK installed from this checkout, with the root
3+
# pom's Micronaut core and logback versions. Launcher contract: integration-testing/README.md,
4+
# "Contracts". Env: MVNW, ACP_VERSION, IT_ROOT.
45
set -euo pipefail
56
cd "$(dirname "${BASH_SOURCE[0]}")/.."
7+
root_pom="${IT_ROOT:?IT_ROOT is not set}/../pom.xml"
8+
prop() { sed -n "s:.*<$1>\(.*\)</$1>.*:\1:p" "$root_pom" | head -1; }
9+
core="$(prop micronaut.core.version)" logback="$(prop logback.version)"
10+
[ -n "$core" ] && [ -n "$logback" ] || { echo "build.sh: no micronaut.core.version or logback.version in the root pom" >&2; exit 1; }
611
exec "${MVNW:?MVNW is not set}" -q -B -Dacp.version="${ACP_VERSION:?ACP_VERSION is not set}" \
12+
-Dmicronaut.core.version="$core" -Dlogback.version="$logback" \
713
compile dependency:build-classpath -Dmdep.outputFile=target/classpath.txt

0 commit comments

Comments
 (0)