Skip to content

Commit 5ca4e6d

Browse files
author
Mark Pollack
committed
Framework smoke matrix: Quarkus agent and client cells
programs/quarkus hosts the shared smoke agent and client on acp-quarkus. The transport and whether the agent is served are build-time settings, so build.sh packages three applications: HTTP and WebSocket, stdio, and the client. quarkus-typescript-{http,ws} keep their 1 MB and 8 MB payload steps, now against the shared agent.
1 parent 1903ddb commit 5ca4e6d

15 files changed

Lines changed: 493 additions & 100 deletions

‎integration-testing/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ artifact.
102102
| `x-<client>-<agent>-<transport>` | generated: the client program of one language against the agent program of another over one transport, running every stable catalogue step (`steps.json`). Pairs are Java<->Java and Java<->X for X in TypeScript, Rust, Python and Kotlin; transports are stdio, Streamable HTTP and WebSocket (Kotlin: stdio and WebSocket) |
103103
| `x-<client>-<agent>-<transport>-unstable` | the same pair running the `unstable` steps (`session.fork`); on demand only |
104104
| `conf-java-agent-<t>`, `conf-java-client-<t>`, `conf-java-client-catalogue-<t>` | the raw driver probing the Java agent, the Java client's raw mode, and the Java client running catalogue steps against the raw agent ([programs/raw/gen_conf.py](programs/raw/gen_conf.py)) |
105-
| `quarkus-typescript-http`, `quarkus-typescript-ws` | hand-written: the TypeScript client program against the Quarkus-hosted agent (`programs/quarkus`, an `@AcpAgent` bean served by `acp-quarkus` on the Quarkus HTTP server), running the catalogue steps that agent implements (initialize, sessions, echo chunks, stop reasons, -32601, 1 MB and 8 MB prompts and updates). Run in the `typescript-http`/`typescript-ws` legs of `cross-sdk.yml` |
105+
| `quarkus-typescript-http`, `quarkus-typescript-ws` | hand-written: the TypeScript client program against the Quarkus-hosted smoke agent (`programs/quarkus`: the shared `programs/framework` handlers as an `@AcpAgent` bean served by `acp-quarkus` on the Quarkus HTTP server), running the catalogue steps that agent implements (initialize, sessions, echo chunks, stop reasons, -32601, 1 MB and 8 MB prompts and updates). Run in the `typescript-http`/`typescript-ws` legs of `cross-sdk.yml` |
106106
| `load-50`, `load-300`, `load-1000` | N Java clients on their own connections, each: initialize, session/new, then 10 (or 5) prompts streaming two updates each |
107107
| `load-shared-300` | 300 clients sharing one HttpClient, so one HTTP/2 connection |
108108

‎integration-testing/configs/fw-quarkus-agent-http.json‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
{
2+
"name": "fw-quarkus-agent-http",
3+
"description": "typescript client -> quarkus-hosted java agent over http, 14 smoke steps (generated from smoke.json)",
4+
"timeoutSec": 370,
5+
"peers": [
6+
"typescript-sdk"
7+
],
8+
"processes": [
9+
{
10+
"name": "agent",
11+
"language": "java",
12+
"role": "server",
13+
"dir": "${ROOT}/programs/quarkus",
14+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" ${ROOT}/programs/quarkus/launch/build.sh",
15+
"run": "${ROOT}/programs/quarkus/launch/agent.sh --transport http --port ${PORT}",
16+
"env": {
17+
"TS_SDK": "${peer.typescript-sdk}"
18+
},
19+
"ready": {
20+
"line": "READY",
21+
"timeoutSec": 90
22+
}
23+
},
24+
{
25+
"name": "client",
26+
"language": "typescript",
27+
"role": "client",
28+
"dir": "${ROOT}/programs/node",
29+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" TS_SDK=\"${peer.typescript-sdk}\" ${ROOT}/programs/node/launch/build.sh",
30+
"run": "${ROOT}/programs/node/launch/client.sh --transport http --url http://127.0.0.1:${PORT}/acp",
31+
"env": {
32+
"STEPS": "init.initialize,init.agent-capabilities,init.agent-info,session.new,session.load,session.multi,update.agent_message_chunk,perm.selected,elicit.form,cancel.prompt,ext.client-request,error.method-not-found,http.reconnect,conn.close",
33+
"TS_SDK": "${peer.typescript-sdk}"
34+
},
35+
"timeoutSec": 190
36+
}
37+
],
38+
"assertions": {
39+
"requiredOutput": {
40+
"agent": [
41+
"STEP agent.perm.selected PASS",
42+
"STEP agent.elicit.form PASS",
43+
"STEP agent.cancel.prompt PASS"
44+
]
45+
},
46+
"forbiddenOutput": {
47+
"agent": [
48+
" ERROR ",
49+
" WARN "
50+
]
51+
},
52+
"requiredPatterns": {
53+
"client": [
54+
"STEP init\\.initialize PASS .*\"name\":\"interop-quarkus-agent\""
55+
]
56+
},
57+
"checks": [
58+
"client RESULT pass == 14",
59+
"client RESULT fail == 0"
60+
],
61+
"expectedFailures": [],
62+
"report": {
63+
"client": [
64+
"TIMEOUT"
65+
]
66+
}
67+
},
68+
"comment": "GENERATED by GenConfigs.java from smoke.json, steps.json and matrix.json; do not edit, regenerate."
69+
}

‎integration-testing/configs/fw-quarkus-agent-stdio.json‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
{
2+
"name": "fw-quarkus-agent-stdio",
3+
"description": "typescript client -> quarkus-hosted java agent over stdio, 14 smoke steps (generated from smoke.json)",
4+
"timeoutSec": 370,
5+
"peers": [
6+
"typescript-sdk"
7+
],
8+
"processes": [
9+
{
10+
"name": "client",
11+
"language": "typescript",
12+
"role": "client",
13+
"dir": "${ROOT}/programs/node",
14+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" TS_SDK=\"${peer.typescript-sdk}\" ${ROOT}/programs/node/launch/build.sh && IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" ${ROOT}/programs/quarkus/launch/build.sh",
15+
"run": "${ROOT}/programs/node/launch/client.sh --transport stdio",
16+
"env": {
17+
"STEPS": "init.initialize,init.agent-capabilities,init.agent-info,session.new,session.load,session.multi,update.agent_message_chunk,perm.selected,elicit.form,cancel.prompt,ext.client-request,error.method-not-found,stdio.eof-exit,conn.close",
18+
"AGENT_CMD": "${ROOT}/programs/quarkus/launch/agent.sh --transport stdio",
19+
"TS_SDK": "${peer.typescript-sdk}"
20+
},
21+
"timeoutSec": 190
22+
}
23+
],
24+
"assertions": {
25+
"requiredOutput": {
26+
"client": [
27+
"STEP agent.perm.selected PASS",
28+
"STEP agent.elicit.form PASS",
29+
"STEP agent.cancel.prompt PASS"
30+
]
31+
},
32+
"forbiddenOutput": {
33+
"client": [
34+
" ERROR ",
35+
" WARN "
36+
]
37+
},
38+
"requiredPatterns": {
39+
"client": [
40+
"STEP init\\.initialize PASS .*\"name\":\"interop-quarkus-agent\""
41+
]
42+
},
43+
"checks": [
44+
"client RESULT pass == 14",
45+
"client RESULT fail == 0"
46+
],
47+
"expectedFailures": [],
48+
"report": {
49+
"client": [
50+
"TIMEOUT"
51+
]
52+
}
53+
},
54+
"comment": "GENERATED by GenConfigs.java from smoke.json, steps.json and matrix.json; do not edit, regenerate."
55+
}

‎integration-testing/configs/fw-quarkus-agent-ws.json‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
{
2+
"name": "fw-quarkus-agent-ws",
3+
"description": "typescript client -> quarkus-hosted java agent over ws, 13 smoke steps (generated from smoke.json)",
4+
"timeoutSec": 365,
5+
"peers": [
6+
"typescript-sdk"
7+
],
8+
"processes": [
9+
{
10+
"name": "agent",
11+
"language": "java",
12+
"role": "server",
13+
"dir": "${ROOT}/programs/quarkus",
14+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" ${ROOT}/programs/quarkus/launch/build.sh",
15+
"run": "${ROOT}/programs/quarkus/launch/agent.sh --transport ws --port ${PORT}",
16+
"env": {
17+
"TS_SDK": "${peer.typescript-sdk}"
18+
},
19+
"ready": {
20+
"line": "READY",
21+
"timeoutSec": 90
22+
}
23+
},
24+
{
25+
"name": "client",
26+
"language": "typescript",
27+
"role": "client",
28+
"dir": "${ROOT}/programs/node",
29+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" TS_SDK=\"${peer.typescript-sdk}\" ${ROOT}/programs/node/launch/build.sh",
30+
"run": "${ROOT}/programs/node/launch/client.sh --transport ws --url ws://127.0.0.1:${PORT}/acp",
31+
"env": {
32+
"STEPS": "init.initialize,init.agent-capabilities,init.agent-info,session.new,session.load,session.multi,update.agent_message_chunk,perm.selected,elicit.form,cancel.prompt,ext.client-request,error.method-not-found,conn.close",
33+
"TS_SDK": "${peer.typescript-sdk}"
34+
},
35+
"timeoutSec": 185
36+
}
37+
],
38+
"assertions": {
39+
"requiredOutput": {
40+
"agent": [
41+
"STEP agent.perm.selected PASS",
42+
"STEP agent.elicit.form PASS",
43+
"STEP agent.cancel.prompt PASS"
44+
]
45+
},
46+
"forbiddenOutput": {
47+
"agent": [
48+
" ERROR ",
49+
" WARN "
50+
]
51+
},
52+
"requiredPatterns": {
53+
"client": [
54+
"STEP init\\.initialize PASS .*\"name\":\"interop-quarkus-agent\""
55+
]
56+
},
57+
"checks": [
58+
"client RESULT pass == 13",
59+
"client RESULT fail == 0"
60+
],
61+
"expectedFailures": [],
62+
"report": {
63+
"client": [
64+
"TIMEOUT"
65+
]
66+
}
67+
},
68+
"comment": "GENERATED by GenConfigs.java from smoke.json, steps.json and matrix.json; do not edit, regenerate."
69+
}

‎integration-testing/configs/fw-quarkus-client-http.json‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
{
2+
"name": "fw-quarkus-client-http",
3+
"description": "quarkus-built java client -> typescript agent over http, 10 smoke steps (generated from smoke.json)",
4+
"timeoutSec": 350,
5+
"peers": [
6+
"typescript-sdk"
7+
],
8+
"processes": [
9+
{
10+
"name": "agent",
11+
"language": "typescript",
12+
"role": "server",
13+
"dir": "${ROOT}/programs/node",
14+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" TS_SDK=\"${peer.typescript-sdk}\" ${ROOT}/programs/node/launch/build.sh",
15+
"run": "${ROOT}/programs/node/launch/agent.sh --transport http --port ${PORT}",
16+
"env": {
17+
"TS_SDK": "${peer.typescript-sdk}"
18+
},
19+
"ready": {
20+
"line": "READY",
21+
"timeoutSec": 60
22+
}
23+
},
24+
{
25+
"name": "client",
26+
"language": "java",
27+
"role": "client",
28+
"dir": "${ROOT}/programs/quarkus",
29+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" ${ROOT}/programs/quarkus/launch/build.sh",
30+
"run": "${ROOT}/programs/quarkus/launch/client.sh --transport http --url http://127.0.0.1:${PORT}/acp",
31+
"env": {
32+
"STEPS": "init.initialize,init.agent-info,session.new,update.agent_message_chunk,perm.selected,fs.read,elicit.form,cancel.prompt,ext.client-request,conn.close",
33+
"TS_SDK": "${peer.typescript-sdk}"
34+
},
35+
"timeoutSec": 170
36+
}
37+
],
38+
"assertions": {
39+
"requiredOutput": {
40+
"agent": [
41+
"STEP agent.perm.selected PASS",
42+
"STEP agent.fs.read PASS",
43+
"STEP agent.elicit.form PASS",
44+
"STEP agent.cancel.prompt PASS"
45+
]
46+
},
47+
"checks": [
48+
"client RESULT pass == 10",
49+
"client RESULT fail == 0"
50+
],
51+
"expectedFailures": [],
52+
"report": {
53+
"client": [
54+
"TIMEOUT"
55+
]
56+
}
57+
},
58+
"comment": "GENERATED by GenConfigs.java from smoke.json, steps.json and matrix.json; do not edit, regenerate."
59+
}

‎integration-testing/configs/load-10-fw-quarkus.json‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
{
2+
"name": "load-10-fw-quarkus",
3+
"description": "10 connections x 10 prompts against the quarkus-hosted agent over http, two streamed updates each (generated from smoke.json)",
4+
"timeoutSec": 420,
5+
"processes": [
6+
{
7+
"name": "server",
8+
"language": "java",
9+
"role": "server",
10+
"dir": "${ROOT}/programs/quarkus",
11+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" ${ROOT}/programs/quarkus/launch/build.sh",
12+
"run": "${ROOT}/programs/quarkus/launch/agent.sh --transport http --port ${PORT}",
13+
"env": {
14+
"JAVA_OPTS": "-Xmx512m"
15+
},
16+
"ready": {
17+
"line": "READY",
18+
"timeoutSec": 90
19+
}
20+
},
21+
{
22+
"name": "clients",
23+
"language": "java",
24+
"role": "client",
25+
"dir": "${ROOT}/programs/java",
26+
"build": "IT_ROOT=\"${ROOT}\" MVNW=\"${MVNW}\" ACP_VERSION=\"${ACP_VERSION}\" CACHE=\"${CACHE}\" ${ROOT}/programs/java/launch/build.sh",
27+
"run": "java -Xmx1g -Dacp.client.level=INFO -cp target/classes:$(cat target/classpath.txt) load.LoadClients http://127.0.0.1:${PORT}/acp 10 10 perclient",
28+
"timeoutSec": 300
29+
}
30+
],
31+
"assertions": {
32+
"requiredOutput": {
33+
"clients": [
34+
"PEAK_REACHED",
35+
"CLOSED"
36+
]
37+
},
38+
"forbiddenOutput": {
39+
"clients": [
40+
"ERROR ",
41+
"CLOSE_ERROR"
42+
],
43+
"server": [
44+
" ERROR "
45+
]
46+
},
47+
"checks": [
48+
"clients RESULT errors == 0",
49+
"clients RESULT ok == expected",
50+
"clients RESULT expected == 100",
51+
"clients RESULT updates == 2*ok",
52+
"clients RESULT close_errors == 0"
53+
]
54+
},
55+
"comment": "GENERATED by GenConfigs.java from smoke.json, steps.json and matrix.json; do not edit, regenerate."
56+
}

‎integration-testing/programs/quarkus/launch/agent.sh‎

Lines changed: 14 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,26 @@
11
#!/usr/bin/env bash
2-
# Starts the Quarkus-hosted interop agent: --transport http|ws --port <port> (one server serves
3-
# both). This program is built for HTTP only, so stdio is refused. Launcher contract:
4-
# integration-testing/README.md, "Contracts". JAVA_OPTS, if set, is passed to the JVM.
2+
# Starts the Quarkus smoke agent: --transport stdio | --transport http|ws --port <port> (one server
3+
# serves HTTP and WebSocket). Launcher contract: integration-testing/README.md, "Contracts".
4+
# Prints nothing itself: on stdio, stdout is the protocol stream. JAVA_OPTS, if set, is passed on.
55
set -euo pipefail
66
here="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
7+
usage() { echo "usage: agent.sh --transport stdio | --transport http|ws --port <port>" >&2; exit 2; }
78
transport="" port=""
89
while [ $# -gt 0 ]; do
910
case "$1" in
1011
--transport) transport="${2:-}"; shift 2 ;;
1112
--port) port="${2:-}"; shift 2 ;;
12-
*) echo "usage: agent.sh --transport http|ws --port <port>" >&2; exit 2 ;;
13+
*) usage ;;
1314
esac
1415
done
1516
case "$transport" in
16-
http|ws) [ -n "$port" ] || { echo "usage: agent.sh --transport http|ws --port <port>" >&2; exit 2; } ;;
17-
*) echo "agent.sh: the Quarkus agent serves http and ws only" >&2; exit 2 ;;
17+
stdio)
18+
# shellcheck disable=SC2086
19+
exec java ${JAVA_OPTS:-} -jar "$here/target/stdio/quarkus-run.jar" ;;
20+
http|ws)
21+
[ -n "$port" ] || usage
22+
# shellcheck disable=SC2086
23+
exec java ${JAVA_OPTS:-} -Dinterop.ready=true -Dquarkus.http.host=127.0.0.1 -Dquarkus.http.port="$port" \
24+
-jar "$here/target/quarkus-app/quarkus-run.jar" ;;
25+
*) usage ;;
1826
esac
19-
# shellcheck disable=SC2086
20-
exec java ${JAVA_OPTS:-} -Dquarkus.http.host=127.0.0.1 -Dquarkus.http.port="$port" \
21-
-jar "$here/target/quarkus-app/quarkus-run.jar"

‎integration-testing/programs/quarkus/launch/build.sh‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,19 @@
11
#!/usr/bin/env bash
2-
# Builds the Quarkus-hosted interop agent against the SDK installed from this checkout, with the
3-
# root pom's Quarkus version. Launcher contract: integration-testing/README.md, "Contracts".
4-
# Env: MVNW, ACP_VERSION, IT_ROOT.
2+
# Builds the Quarkus smoke programs against the SDK installed from this checkout, with the root
3+
# pom's Quarkus version. The agent transport and whether the agent is served are build-time
4+
# settings of acp-quarkus, so there are three packages:
5+
# target/quarkus-app the agent over HTTP and WebSocket (application.properties)
6+
# target/stdio the agent over stdio
7+
# target/client no agent served: the client program (launch/client.sh)
8+
# Launcher contract: integration-testing/README.md, "Contracts". Env: MVNW, ACP_VERSION, IT_ROOT.
59
set -euo pipefail
610
cd "$(dirname "${BASH_SOURCE[0]}")/.."
711
quarkus_version="$(sed -n 's:.*<quarkus.version>\(.*\)</quarkus.version>.*:\1:p' "${IT_ROOT:?IT_ROOT is not set}/../pom.xml" | head -1)"
812
[ -n "$quarkus_version" ] || { echo "build.sh: no <quarkus.version> in the root pom" >&2; exit 1; }
9-
exec "${MVNW:?MVNW is not set}" -q -B -Dacp.version="${ACP_VERSION:?ACP_VERSION is not set}" \
10-
-Dquarkus.version="$quarkus_version" package
13+
mvn_q() {
14+
"${MVNW:?MVNW is not set}" -q -B -Dacp.version="${ACP_VERSION:?ACP_VERSION is not set}" \
15+
-Dquarkus.version="$quarkus_version" "$@"
16+
}
17+
mvn_q package
18+
mvn_q package -Dquarkus.acp.agent.transport.type=stdio -Dquarkus.package.output-directory=stdio
19+
mvn_q package -Dquarkus.acp.agent.enabled=false -Dquarkus.package.output-directory=client

0 commit comments

Comments
 (0)