Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion infra/cassandra/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@ Container image that runs Apache Cassandra for NVCF, built on the official
metrics agent. To enable metrics on port 9500, supply a jar under `files/` and
pass both `--build-arg EXPORTER_JAR=files/<jar>` and
`--build-arg EXPORTER_JAVAAGENT=-javaagent:/opt/cassandra/lib/cassandra-exporter-agent.jar`.
Both args are required together; either alone runs without metrics.
Both args are required together; the build rejects either mismatched
combination. It replaces the jar's exact shaded Netty module set with
checksum-pinned artifacts through `scripts/repack-exporter-netty.sh` and fails
on an unexpected layout.

## Build

Expand All @@ -27,6 +30,9 @@ docker build -t nvcf-cassandra:dev infra/cassandra

# multi-arch
docker buildx build --platform linux/amd64,linux/arm64 -t <ref> infra/cassandra

# exporter dependency unit test (downloads checksum-pinned Netty jars)
infra/cassandra/scripts/repack-exporter-netty-test.sh
```

The `--platform=$BUILDPLATFORM` on the yq stage is intentional: it lets yq
Expand Down
24 changes: 21 additions & 3 deletions infra/cassandra/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,26 @@ RUN mkdir /tmp/java-libraries && \
printf '%s %s\n' "${checksum}" "${destination}" | sha256sum -c - || exit 1; \
done < /tmp/java-libraries.lock

FROM --platform=$BUILDPLATFORM alpine:3.21 AS exporter-repacker
RUN apk add --no-cache curl unzip zip

# The release build supplies a compatible exporter agent. Replace the complete
# shaded Netty module set with checksum-pinned security updates while retaining
# the agent implementation, manifest, and unrelated dependencies.
ARG EXPORTER_JAR=files/.gitkeep
ARG EXPORTER_JAVAAGENT=""
COPY ${EXPORTER_JAR} /tmp/exporter-input.jar
COPY scripts/repack-exporter-netty.sh /usr/local/bin/repack-exporter-netty
RUN if [ -s /tmp/exporter-input.jar ] && [ -z "${EXPORTER_JAVAAGENT}" ]; then \
echo "EXPORTER_JAR and EXPORTER_JAVAAGENT must be set together" >&2; \
exit 1; \
fi && \
if [ ! -s /tmp/exporter-input.jar ] && [ -n "${EXPORTER_JAVAAGENT}" ]; then \
echo "EXPORTER_JAR and EXPORTER_JAVAAGENT must be set together" >&2; \
exit 1; \
fi && \
repack-exporter-netty /tmp/exporter-input.jar /tmp/cassandra-exporter-agent.jar

FROM cassandra:5.0.9@sha256:d35e159439b302146f964919904f84fd3c2cebf347272b8cb8c4368c1cf200e5

# Refresh the OpenSSL runtime packages independently of the upstream image
Expand All @@ -56,8 +76,6 @@ RUN apt-get update && \
# builds override this with --build-arg EXPORTER_JAR=files/<real-jar> (see
# Makefile / .gitlab-ci.yml). To build with the exporter locally, download
# the jar into files/ and pass --build-arg EXPORTER_JAR=files/<jar-name>.
ARG EXPORTER_JAR=files/.gitkeep

# The javaagent flag itself is also gated behind a build arg. Without a real
# jar, wiring -javaagent unconditionally would point the JVM at the empty
# .gitkeep placeholder and abort startup with "Error opening zip file or JAR
Expand All @@ -66,7 +84,7 @@ ARG EXPORTER_JAR=files/.gitkeep
# passes neither, so the image boots with no agent and no metrics.
ARG EXPORTER_JAVAAGENT=""

COPY ${EXPORTER_JAR} /opt/cassandra/lib/cassandra-exporter-agent.jar
COPY --from=exporter-repacker /tmp/cassandra-exporter-agent.jar /opt/cassandra/lib/cassandra-exporter-agent.jar
COPY scripts/cassandra-env.sh /etc/cassandra/cassandra-env.sh
COPY --from=yq-downloader /tmp/yq /usr/local/bin/yq
COPY --from=yq-downloader /tmp/java-libraries/*.jar /opt/cassandra/lib/
Expand Down
4 changes: 3 additions & 1 deletion infra/cassandra/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,12 @@ docker build \
-t <your-registry>/<your-org>/nvcf-cassandra:<version> .
```

Supplying only `EXPORTER_JAR` without `EXPORTER_JAVAAGENT` copies the jar into the image but never loads it; the image still runs without metrics. Both build args must be set together to get a working exporter.
Both build args must be set together to get a working exporter. The build rejects either mismatched combination so an exporter jar cannot be silently omitted from the JVM or an empty placeholder loaded as a Java agent.

The jar must expose the same metrics interface the image's `JVM_EXTRA_OPTS` javaagent wiring expects (see `Dockerfile`). If you change the jar, update the `EXPORTER_JAR` build-arg accordingly; the source path and filename are entirely up to you.

The build replaces the exporter's complete shaded Netty module set with the checksum-pinned version declared in `scripts/repack-exporter-netty.sh`. It fails if the input jar's Netty module set changes, rather than leaving a partially updated dependency graph in the image. The exporter implementation, Java agent manifest, and unrelated shaded dependencies are preserved.

## Prerequisites

- Docker or another OCI-compatible builder (with `buildx` for multi-arch)
Expand Down
89 changes: 89 additions & 0 deletions infra/cassandra/scripts/repack-exporter-netty-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
#!/bin/sh
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

set -eu

script_dir=$(CDPATH='' cd -- "$(dirname -- "$0")" && pwd)
work_dir=$(mktemp -d)
trap 'rm -rf "${work_dir}"' EXIT HUP INT TERM

input_dir=${work_dir}/input
mkdir -p \
"${input_dir}/META-INF/maven/com.zegelin.cassandra-exporter/agent" \
"${input_dir}/META-INF/maven/com.zegelin.cassandra-exporter/common" \
"${input_dir}/com/zegelin/cassandra/exporter" \
"${input_dir}/io/netty/legacy"

printf '%s\n' \
'Manifest-Version: 1.0' \
'Premain-Class: com.zegelin.cassandra.exporter.Agent' \
> "${input_dir}/META-INF/MANIFEST.MF"
printf '%s\n' 'preserve this exporter payload' \
> "${input_dir}/com/zegelin/cassandra/exporter/Agent.class"
printf '%s\n' 'remove this old Netty payload' \
> "${input_dir}/io/netty/legacy/Old.class"

for project in agent common; do
printf '%s\n' \
'<project>' \
' <dependency>' \
' <groupId>io.netty</groupId>' \
' <version>4.1.135.Final</version>' \
' </dependency>' \
'</project>' \
> "${input_dir}/META-INF/maven/com.zegelin.cassandra-exporter/${project}/pom.xml"
done

for artifact in \
netty-buffer \
netty-codec \
netty-codec-http \
netty-common \
netty-handler \
netty-resolver \
netty-transport \
netty-transport-native-unix-common; do
metadata_dir=${input_dir}/META-INF/maven/io.netty/${artifact}
mkdir -p "${metadata_dir}"
printf '%s\n' "artifactId=${artifact}" 'groupId=io.netty' 'version=4.1.135.Final' \
> "${metadata_dir}/pom.properties"
done

input_jar=${work_dir}/input.jar
output_jar=${work_dir}/output.jar
(
cd "${input_dir}"
find . -type f -print | LC_ALL=C sort | zip -Xq "${input_jar}" -@
)

"${script_dir}/repack-exporter-netty.sh" "${input_jar}" "${output_jar}"

original_payload=$(unzip -p "${input_jar}" com/zegelin/cassandra/exporter/Agent.class)
repacked_payload=$(unzip -p "${output_jar}" com/zegelin/cassandra/exporter/Agent.class)
[ "${original_payload}" = "${repacked_payload}" ]

original_manifest=$(unzip -p "${input_jar}" META-INF/MANIFEST.MF)
repacked_manifest=$(unzip -p "${output_jar}" META-INF/MANIFEST.MF)
[ "${original_manifest}" = "${repacked_manifest}" ]

if unzip -Z1 "${output_jar}" | grep -q '^io/netty/legacy/Old.class$'; then
echo 'old Netty class remains in the repacked jar' >&2
exit 1
fi
unzip -Z1 "${output_jar}" | grep -q '^io/netty/handler/codec/http/HttpRequest.class$'

module_count=$(unzip -Z1 "${output_jar}" |
sed -n 's#^META-INF/maven/io\.netty/[^/]*/pom\.properties$#module#p' |
wc -l | tr -d ' ')
[ "${module_count}" = 8 ]

for properties in $(unzip -Z1 "${output_jar}" | grep '^META-INF/maven/io.netty/.*/pom.properties$'); do
[ "$(unzip -p "${output_jar}" "${properties}" | sed -n 's/^version=//p')" = '4.1.137.Final' ]
done

if unzip -p "${output_jar}" | grep -q '4.1.135.Final'; then
echo 'old Netty version remains in the repacked jar' >&2
exit 1
fi
echo 'repack-exporter-netty: PASS'
180 changes: 180 additions & 0 deletions infra/cassandra/scripts/repack-exporter-netty.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
#!/bin/sh
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

set -eu

NETTY_VERSION=4.1.137.Final
MAVEN_CENTRAL_URL=https://repo1.maven.org/maven2/io/netty
NETTY_ARTIFACTS='netty-buffer f474b14c7734f15e0540394cb6f39d67777b7581a42919e4ac89d253d4efd929
netty-codec 9987b6a660b0a6b1f0d791485dae33180b3d1c63687c006fe6d3fd025e9e3798
netty-codec-http 0535bb5a736472bef5c948d15eb273c4ab9f796656fc7c5d6b982ad92bddbd49
netty-common d31926b01adcc07af86f5e27b81b6d6c115df17d366e835d1fc3f5a1924e7e52
netty-handler d0e4c6ee4779f59f6ab2fb5d388e4f57147c82270164b37945764bb9bda96a44
netty-resolver b4cf2aeedd9fc7c8c439bbfe574f63cfe5b83392e88bbc07ca0e8424b7cff955
netty-transport 6251adc2a2921572382732a2db188d4f4f2251fd6ebb49c5d44bbf33d6bfb1a7
netty-transport-native-unix-common 8056e7637f9948f953314894cf995ab664711b66c73c4cd5b593ae84f0b1048c'

usage() {
echo "usage: $0 <input-exporter.jar> <output-exporter.jar>" >&2
exit 2
}

[ "$#" -eq 2 ] || usage
input=$1
output=$2

for tool in awk basename cp curl dirname find grep mkdir mktemp mv rm sed sha256sum sort touch unzip zip; do
command -v "${tool}" >/dev/null 2>&1 || {
echo "missing required command: ${tool}" >&2
exit 1
}
done

[ -f "${input}" ] || {
echo "exporter input does not exist: ${input}" >&2
exit 1
}

# The public build uses an empty placeholder when no exporter is supplied.
if [ ! -s "${input}" ]; then
cp "${input}" "${output}"
exit 0
fi

unzip -tq "${input}" >/dev/null

expected_modules=$(printf '%s\n' "${NETTY_ARTIFACTS}" | awk 'NF { print $1 }' | sort)
input_modules=$(
unzip -Z1 "${input}" |
sed -n 's#^META-INF/maven/io\.netty/\([^/]*\)/pom\.properties$#\1#p' |
sort -u
)
[ "${input_modules}" = "${expected_modules}" ] || {
echo "exporter Netty module set changed; refusing a partial replacement" >&2
echo "expected modules:" >&2
printf '%s\n' "${expected_modules}" >&2
echo "input modules:" >&2
printf '%s\n' "${input_modules}" >&2
exit 1
}

work_dir=$(mktemp -d)
trap 'rm -rf "${work_dir}"' EXIT HUP INT TERM
payload_dir=${work_dir}/payload
downloads_dir=${work_dir}/downloads
mkdir -p "${payload_dir}" "${downloads_dir}"
unzip -q "${input}" -d "${payload_dir}"

# Remove the complete shaded Netty payload and its dependency metadata. The
# module-set check above makes this fail closed if the exporter gains another
# Netty module that is not pinned here.
rm -rf \
"${payload_dir}/io/netty" \
"${payload_dir}/META-INF/maven/io.netty" \
"${payload_dir}/META-INF/native-image/io.netty"
rm -f \
"${payload_dir}/META-INF/io.netty.versions.properties" \
"${payload_dir}/META-INF/services/reactor.blockhound.integration.BlockHoundIntegration"

printf '%s\n' "${NETTY_ARTIFACTS}" |
while read -r artifact checksum; do
[ -n "${artifact}" ] || continue
jar=${downloads_dir}/${artifact}-${NETTY_VERSION}.jar
curl -fsSLo "${jar}" \
"${MAVEN_CENTRAL_URL}/${artifact}/${NETTY_VERSION}/${artifact}-${NETTY_VERSION}.jar"
printf '%s %s\n' "${checksum}" "${jar}" | sha256sum -c -

# Only copy Netty-owned entries. This keeps the exporter implementation,
# manifest, and all unrelated shaded dependencies byte-for-byte intact.
unzip -oq "${jar}" \
'io/netty/*' \
'META-INF/maven/io.netty/*' \
-d "${payload_dir}"
if unzip -Z1 "${jar}" | grep -q '^META-INF/native-image/io.netty/'; then
unzip -oq "${jar}" 'META-INF/native-image/io.netty/*' -d "${payload_dir}"
fi
if unzip -Z1 "${jar}" |
grep -q '^META-INF/services/reactor.blockhound.integration.BlockHoundIntegration$'; then
unzip -oq "${jar}" \
'META-INF/services/reactor.blockhound.integration.BlockHoundIntegration' \
-d "${payload_dir}"
fi
done

# The exporter POMs are scanner-visible provenance. Update only their Netty
# version declarations so they describe the dependency payload just inserted.
find "${payload_dir}/META-INF/maven/com.zegelin.cassandra-exporter" -name pom.xml -type f |
while read -r pom; do
if ! awk -v version="${NETTY_VERSION}" '
/<dependency>/ { in_dependency = 1; dependency = "" }
in_dependency {
dependency = dependency $0 ORS
if (/<\/dependency>/) {
if (dependency ~ /<groupId>io[.]netty<\/groupId>/) {
if (dependency ~ /<version>4[.]1[.][0-9][0-9]*[.]Final<\/version>/) {
sub(/<version>4[.]1[.][0-9][0-9]*[.]Final<\/version>/,
"<version>" version "</version>", dependency)
changed = 1
}
}
printf "%s", dependency
in_dependency = 0
next
}
next
}
{ print }
END {
if (in_dependency || !changed) {
exit 2
}
}
' "${pom}" > "${pom}.updated"; then
rm -f "${pom}.updated"
echo "unexpected exporter POM dependency layout: ${pom}" >&2
exit 1
fi
mv "${pom}.updated" "${pom}"
done

output_modules=$(
find "${payload_dir}/META-INF/maven/io.netty" -name pom.properties -type f |
sed 's#.*/META-INF/maven/io\.netty/\([^/]*\)/pom\.properties$#\1#' |
sort -u
)
[ "${output_modules}" = "${expected_modules}" ] || {
echo "replacement Netty module set is incomplete" >&2
exit 1
}

for properties in "${payload_dir}"/META-INF/maven/io.netty/*/pom.properties; do
grep_version=$(sed -n 's/^version=//p' "${properties}")
[ "${grep_version}" = "${NETTY_VERSION}" ] || {
echo "unexpected Netty version in ${properties}: ${grep_version}" >&2
exit 1
}
done

# Netty's per-module files share this name. Combine them so Version.identify()
# can report every module from the repacked fat jar.
versions_file=${payload_dir}/META-INF/io.netty.versions.properties
: > "${versions_file}"
printf '%s\n' "${NETTY_ARTIFACTS}" |
while read -r artifact checksum; do
[ -n "${artifact}" ] || continue
unzip -p "${downloads_dir}/${artifact}-${NETTY_VERSION}.jar" \
META-INF/io.netty.versions.properties >> "${versions_file}"
done

mkdir -p "$(dirname "${output}")"
output_abs=$(cd "$(dirname "${output}")" && pwd)/$(basename "${output}")
rm -f "${output_abs}"
# ZIP records DOS timestamps. Normalize them so identical inputs and pinned
# dependencies produce an identical repacked jar on every build.
find "${payload_dir}" -type f -exec touch -t 198001010000 {} +
(
cd "${payload_dir}"
find . -type f -print | LC_ALL=C sort | zip -Xq "${output_abs}" -@
)
unzip -tq "${output}" >/dev/null