Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion infra/cassandra/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@ Container image that runs Apache Cassandra for NVCF, built on the official
metrics agent. To enable metrics on port 9500, supply a jar under `files/` and
pass both `--build-arg EXPORTER_JAR=files/<jar>` and
`--build-arg EXPORTER_JAVAAGENT=-javaagent:/opt/cassandra/lib/cassandra-exporter-agent.jar`.
Both args are required together; either alone runs without metrics.
Both args are required together; the build rejects either mismatched
combination. It replaces the jar's exact shaded Netty module set with
checksum-pinned artifacts through `scripts/repack-exporter-netty.sh` and fails
on an unexpected layout.

## Build

Expand All @@ -27,6 +30,9 @@ docker build -t nvcf-cassandra:dev infra/cassandra

# multi-arch
docker buildx build --platform linux/amd64,linux/arm64 -t <ref> infra/cassandra

# exporter dependency unit test (downloads checksum-pinned Netty jars)
infra/cassandra/scripts/repack-exporter-netty-test.sh
```

The `--platform=$BUILDPLATFORM` on the yq stage is intentional: it lets yq
Expand Down
56 changes: 52 additions & 4 deletions infra/cassandra/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,63 @@ RUN apk add --no-cache curl && \
printf '%s /tmp/yq\n' "${YQ_SHA256}" | sha256sum -c - && \
chmod +x /tmp/yq

FROM cassandra:5.0.9
# Cassandra 5.0.9 is still the latest supported 5.0 release, but its bundled
# Jackson and Netty versions can lag security-only patch releases. Download a
# checksum-pinned replacement set without adding build tools to the final image.
COPY java-libraries.lock /tmp/java-libraries.lock
RUN mkdir /tmp/java-libraries && \
while read -r checksum url; do \
case "${checksum}" in \#*|'') continue ;; esac; \
destination="/tmp/java-libraries/${url##*/}"; \
curl -fsSLo "${destination}" "${url}" && \
printf '%s %s\n' "${checksum}" "${destination}" | sha256sum -c - || exit 1; \
done < /tmp/java-libraries.lock

FROM --platform=$BUILDPLATFORM alpine:3.21 AS exporter-repacker
RUN apk add --no-cache curl unzip zip

# The release build supplies a compatible exporter agent. Replace the complete
# shaded Netty module set with checksum-pinned security updates while retaining
# the agent implementation, manifest, and unrelated dependencies.
ARG EXPORTER_JAR=files/.gitkeep
ARG EXPORTER_JAVAAGENT=""
COPY ${EXPORTER_JAR} /tmp/exporter-input.jar
COPY scripts/repack-exporter-netty.sh /usr/local/bin/repack-exporter-netty
RUN if [ -s /tmp/exporter-input.jar ] && [ -z "${EXPORTER_JAVAAGENT}" ]; then \
echo "EXPORTER_JAR and EXPORTER_JAVAAGENT must be set together" >&2; \
exit 1; \
fi && \
if [ ! -s /tmp/exporter-input.jar ] && [ -n "${EXPORTER_JAVAAGENT}" ]; then \
echo "EXPORTER_JAR and EXPORTER_JAVAAGENT must be set together" >&2; \
exit 1; \
fi && \
repack-exporter-netty /tmp/exporter-input.jar /tmp/cassandra-exporter-agent.jar

FROM cassandra:5.0.9@sha256:d35e159439b302146f964919904f84fd3c2cebf347272b8cb8c4368c1cf200e5

# Refresh the OpenSSL runtime packages independently of the upstream image
# rebuild cadence. Fail the build if the Debian repository cannot provide the
# first release containing the required fixes.
RUN apt-get update && \
apt-get install -y --no-install-recommends --only-upgrade \
libssl3t64 \
openssl-provider-legacy && \
dpkg --compare-versions "$(dpkg-query -W -f='${Version}' libssl3t64)" ge 3.5.7-1~deb13u2 && \
dpkg --compare-versions "$(dpkg-query -W -f='${Version}' openssl-provider-legacy)" ge 3.5.7-1~deb13u2 && \
rm -rf /var/lib/apt/lists/* && \
rm -f \
/opt/cassandra/lib/jackson-annotations-*.jar \
/opt/cassandra/lib/jackson-core-*.jar \
/opt/cassandra/lib/jackson-databind-*.jar \
/opt/cassandra/lib/jackson-datatype-jsr310-*.jar && \
find /opt/cassandra/lib -maxdepth 1 -type f -name 'netty-*-4.1.130.Final*.jar' -delete

# Metrics exporter agent. The OSS snapshot does not redistribute the jar;
# only files/.gitkeep ships, so the default here points at that placeholder
# and a bare `docker build` succeeds with the agent simply absent. Internal
# builds override this with --build-arg EXPORTER_JAR=files/<real-jar> (see
# Makefile / .gitlab-ci.yml). To build with the exporter locally, download
# the jar into files/ and pass --build-arg EXPORTER_JAR=files/<jar-name>.
ARG EXPORTER_JAR=files/.gitkeep

# The javaagent flag itself is also gated behind a build arg. Without a real
# jar, wiring -javaagent unconditionally would point the JVM at the empty
# .gitkeep placeholder and abort startup with "Error opening zip file or JAR
Expand All @@ -37,9 +84,10 @@ ARG EXPORTER_JAR=files/.gitkeep
# passes neither, so the image boots with no agent and no metrics.
ARG EXPORTER_JAVAAGENT=""

COPY ${EXPORTER_JAR} /opt/cassandra/lib/cassandra-exporter-agent.jar
COPY --from=exporter-repacker /tmp/cassandra-exporter-agent.jar /opt/cassandra/lib/cassandra-exporter-agent.jar
COPY scripts/cassandra-env.sh /etc/cassandra/cassandra-env.sh
COPY --from=yq-downloader /tmp/yq /usr/local/bin/yq
COPY --from=yq-downloader /tmp/java-libraries/*.jar /opt/cassandra/lib/

# load cassandra exporter prom metrics library
# The exporter agent reflects into JDK-internal com.sun.jmx classes to
Expand Down
4 changes: 3 additions & 1 deletion infra/cassandra/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,12 @@ docker build \
-t <your-registry>/<your-org>/nvcf-cassandra:<version> .
```

Supplying only `EXPORTER_JAR` without `EXPORTER_JAVAAGENT` copies the jar into the image but never loads it; the image still runs without metrics. Both build args must be set together to get a working exporter.
Both build args must be set together to get a working exporter. The build rejects either mismatched combination so an exporter jar cannot be silently omitted from the JVM or an empty placeholder loaded as a Java agent.

The jar must expose the same metrics interface the image's `JVM_EXTRA_OPTS` javaagent wiring expects (see `Dockerfile`). If you change the jar, update the `EXPORTER_JAR` build-arg accordingly; the source path and filename are entirely up to you.

The build replaces the exporter's complete shaded Netty module set with the checksum-pinned version declared in `scripts/repack-exporter-netty.sh`. It fails if the input jar's Netty module set changes, rather than leaving a partially updated dependency graph in the image. The exporter implementation, Java agent manifest, and unrelated shaded dependencies are preserved.

## Prerequisites

- Docker or another OCI-compatible builder (with `buildx` for multi-arch)
Expand Down
21 changes: 21 additions & 0 deletions infra/cassandra/java-libraries.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# SHA-256 and Maven Central URL for Cassandra runtime dependency overlays.
# Jackson annotations follows the upstream family's major.minor versioning.
53ca085f4a150f703f49e1aabd935bd03b43e1ea3d55d135438292af22cef56b https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.21/jackson-annotations-2.21.jar
4b40a06396f239f8de2da57419adde6e94e5edc18a2171d471ea05eeed4e5c2d https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.21.4/jackson-core-2.21.4.jar
3888e9e69ab66fbacaacc9aea0e9ffbf15368288e4aca468b024dba11c09fbf9 https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.21.4/jackson-databind-2.21.4.jar
d1ac4b98b70304e56448423589fde5e775b100889643ad1ead62cc7811633684 https://repo.maven.apache.org/maven2/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.21.4/jackson-datatype-jsr310-2.21.4.jar
75cb7db3492915953aa19c566e477fb371ae2b32aa12bc4fe85bd7a2a6b8f5e8 https://repo.maven.apache.org/maven2/io/netty/netty-all/4.1.136.Final/netty-all-4.1.136.Final.jar
c88f13fc41156fde5df918c7b240038dfbe97ac57576163f208630391789b5d5 https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.136.Final/netty-buffer-4.1.136.Final.jar
2de4fc13005c7740b46427a47ee04265a19779c147d53e583f441889b1148159 https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.136.Final/netty-codec-4.1.136.Final.jar
e2f73be7ec359b46583ad875521137000eff520bafd320e730846ec9974f3be6 https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.136.Final/netty-common-4.1.136.Final.jar
54bb1a59f46a3aefd117942e6acee09e555b756776bad731fc06159d89c2da28 https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.136.Final/netty-handler-4.1.136.Final.jar
47400551f0444dea33629ee0c52d4e30856b2ddf00b12adf1881902957e74cf2 https://repo.maven.apache.org/maven2/io/netty/netty-handler-proxy/4.1.136.Final/netty-handler-proxy-4.1.136.Final.jar
d9830d3e64fd5448a67a54deaf8ff1be40d05db1b862f98c90399a947958f249 https://repo.maven.apache.org/maven2/io/netty/netty-handler-ssl-ocsp/4.1.136.Final/netty-handler-ssl-ocsp-4.1.136.Final.jar
e64972fec474f5b9bd086b738154d190a923e82c4923ac550aff4f5ea9e98600 https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.136.Final/netty-resolver-4.1.136.Final.jar
b92881ea925721ed42fee5122a42b8bce4b84da737dbc3ca2d84dfaca52c28b6 https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.136.Final/netty-transport-4.1.136.Final.jar
f6a0b631b98667f131daf4ca07a9cae1072d58e259dcbc0f8c6a053d843449c6 https://repo.maven.apache.org/maven2/io/netty/netty-transport-classes-epoll/4.1.136.Final/netty-transport-classes-epoll-4.1.136.Final.jar
4ed7ee5cb8c611e879d0d6621e8a1b4255e7ba5a31e5f7828d635682ef244a20 https://repo.maven.apache.org/maven2/io/netty/netty-transport-classes-kqueue/4.1.136.Final/netty-transport-classes-kqueue-4.1.136.Final.jar
41a30e3e096091695ef7b7cf16c85802bbc05d4be94643ecc5e8a8d9759acb77 https://repo.maven.apache.org/maven2/io/netty/netty-transport-native-epoll/4.1.136.Final/netty-transport-native-epoll-4.1.136.Final.jar
f264cf4bcc1e97ad148cd4f0d8777eacf404cd4ed80830de3f8a2359f60edc38 https://repo.maven.apache.org/maven2/io/netty/netty-transport-native-epoll/4.1.136.Final/netty-transport-native-epoll-4.1.136.Final-linux-aarch_64.jar
c3956f90241582bbbad5612c3159cffeb7c3533324759f3c3bd778ca9b176cae https://repo.maven.apache.org/maven2/io/netty/netty-transport-native-epoll/4.1.136.Final/netty-transport-native-epoll-4.1.136.Final-linux-x86_64.jar
7e014c9b13defd9d254d4e5a5edd8ab6dde17533e1152f99699a6b28b2967a8a https://repo.maven.apache.org/maven2/io/netty/netty-transport-native-unix-common/4.1.136.Final/netty-transport-native-unix-common-4.1.136.Final.jar
133 changes: 133 additions & 0 deletions infra/cassandra/scripts/repack-exporter-netty-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
#!/bin/sh
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

set -eu

script_dir=$(CDPATH='' cd -- "$(dirname -- "$0")" && pwd)
work_dir=$(mktemp -d)
trap 'rm -rf "${work_dir}"' EXIT HUP INT TERM

input_dir=${work_dir}/input
mkdir -p \
"${input_dir}/META-INF/maven/com.zegelin.cassandra-exporter/agent" \
"${input_dir}/META-INF/maven/com.zegelin.cassandra-exporter/common" \
"${input_dir}/com/zegelin/cassandra/exporter" \
"${input_dir}/io/netty/legacy"

printf '%s\n' \
'Manifest-Version: 1.0' \
'Premain-Class: com.zegelin.cassandra.exporter.Agent' \
> "${input_dir}/META-INF/MANIFEST.MF"
printf '%s\n' 'stale signature metadata' > "${input_dir}/META-INF/TEST.SF"
printf '%s\n' 'stale signature block' > "${input_dir}/META-INF/TEST.RSA"
printf '%s\n' 'stale alternate signature' > "${input_dir}/META-INF/SIG-TEST"
printf '%s\n' 'preserve this exporter payload' \
> "${input_dir}/com/zegelin/cassandra/exporter/Agent.class"
printf '%s\n' 'remove this old Netty payload' \
> "${input_dir}/io/netty/legacy/Old.class"

for project in agent common; do
printf '%s\n' \
'<project>' \
' <dependency>' \
' <groupId>io.netty</groupId>' \
' <version>4.1.135.Final</version>' \
' </dependency>' \
'</project>' \
> "${input_dir}/META-INF/maven/com.zegelin.cassandra-exporter/${project}/pom.xml"
done

for artifact in \
netty-buffer \
netty-codec \
netty-codec-http \
netty-common \
netty-handler \
netty-resolver \
netty-transport \
netty-transport-native-unix-common; do
metadata_dir=${input_dir}/META-INF/maven/io.netty/${artifact}
mkdir -p "${metadata_dir}"
printf '%s\n' "artifactId=${artifact}" 'groupId=io.netty' 'version=4.1.135.Final' \
> "${metadata_dir}/pom.properties"
done

input_jar=${work_dir}/input.jar
output_jar=${work_dir}/output.jar
(
cd "${input_dir}"
find . -type f -print | LC_ALL=C sort | zip -Xq "${input_jar}" -@
)

missing_metadata_dir=${work_dir}/missing-metadata
cp -R "${input_dir}" "${missing_metadata_dir}"
rm -rf "${missing_metadata_dir}/META-INF/maven/com.zegelin.cassandra-exporter"
missing_metadata_jar=${work_dir}/missing-metadata.jar
(
cd "${missing_metadata_dir}"
find . -type f -print | LC_ALL=C sort | zip -Xq "${missing_metadata_jar}" -@
)
if "${script_dir}/repack-exporter-netty.sh" \
"${missing_metadata_jar}" "${work_dir}/unexpected-output.jar" \
> "${work_dir}/missing-metadata.stdout" 2> "${work_dir}/missing-metadata.stderr"; then
echo 'repack unexpectedly accepted exporter input without Maven metadata' >&2
exit 1
fi
grep -q 'exporter Maven metadata is missing' "${work_dir}/missing-metadata.stderr"

missing_pom_dir=${work_dir}/missing-pom
cp -R "${input_dir}" "${missing_pom_dir}"
rm -rf "${missing_pom_dir}/META-INF/maven/com.zegelin.cassandra-exporter"
mkdir -p "${missing_pom_dir}/META-INF/maven/com.zegelin.cassandra-exporter"
printf '%s\n' 'metadata directory without a POM' \
> "${missing_pom_dir}/META-INF/maven/com.zegelin.cassandra-exporter/README"
missing_pom_jar=${work_dir}/missing-pom.jar
(
cd "${missing_pom_dir}"
find . -type f -print | LC_ALL=C sort | zip -Xq "${missing_pom_jar}" -@
)
if "${script_dir}/repack-exporter-netty.sh" \
"${missing_pom_jar}" "${work_dir}/unexpected-pom-output.jar" \
> "${work_dir}/missing-pom.stdout" 2> "${work_dir}/missing-pom.stderr"; then
echo 'repack unexpectedly accepted exporter metadata without a POM' >&2
exit 1
fi
grep -q 'exporter Maven metadata contains no pom.xml files' "${work_dir}/missing-pom.stderr"

"${script_dir}/repack-exporter-netty.sh" "${input_jar}" "${output_jar}"

original_payload=$(unzip -p "${input_jar}" com/zegelin/cassandra/exporter/Agent.class)
repacked_payload=$(unzip -p "${output_jar}" com/zegelin/cassandra/exporter/Agent.class)
[ "${original_payload}" = "${repacked_payload}" ]

original_manifest=$(unzip -p "${input_jar}" META-INF/MANIFEST.MF)
repacked_manifest=$(unzip -p "${output_jar}" META-INF/MANIFEST.MF)
[ "${original_manifest}" = "${repacked_manifest}" ]

if unzip -Z1 "${output_jar}" |
grep -Eq '^META-INF/[^/]+\.(SF|RSA|DSA|EC)$|^META-INF/SIG-'; then
echo 'invalidated JAR signature metadata remains in the repacked jar' >&2
exit 1
fi

if unzip -Z1 "${output_jar}" | grep -q '^io/netty/legacy/Old.class$'; then
echo 'old Netty class remains in the repacked jar' >&2
exit 1
fi
unzip -Z1 "${output_jar}" | grep -q '^io/netty/handler/codec/http/HttpRequest.class$'

module_count=$(unzip -Z1 "${output_jar}" |
sed -n 's#^META-INF/maven/io\.netty/[^/]*/pom\.properties$#module#p' |
wc -l | tr -d ' ')
[ "${module_count}" = 8 ]

for properties in $(unzip -Z1 "${output_jar}" | grep '^META-INF/maven/io.netty/.*/pom.properties$'); do
[ "$(unzip -p "${output_jar}" "${properties}" | sed -n 's/^version=//p')" = '4.1.137.Final' ]
done

if unzip -p "${output_jar}" | grep -q '4.1.135.Final'; then
echo 'old Netty version remains in the repacked jar' >&2
exit 1
fi
echo 'repack-exporter-netty: PASS'
Loading
Loading