Skip to content

[DO NOT MERGE] Test RAFT 3052 #1431

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
divyegala wants to merge 6 commits into
base: main
Choose a base branch
from
Open

[DO NOT MERGE] Test RAFT 3052 #1431

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ec7f82a
[DO NOT MERGE] Test RAFT 3052
divyegala Jun 12, 2026
13b0fec
Merge branch 'main' into test-raft-3052
divyegala Jun 12, 2026
fef8de5
properly prepend
divyegala Jun 24, 2026
13eb3e3
Merge remote-tracking branch 'upstream/main' into test-raft-3052
divyegala Jun 24, 2026
4b10d25
Merge remote-tracking branch 'origin/test-raft-3052' into test-raft-3052
divyegala Jun 24, 2026
9db5917
Merge branch 'main' into test-raft-3052
divyegala Jun 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ci/build_cpp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ RAPIDS_PACKAGE_VERSION=$(rapids-generate-version)
export RAPIDS_PACKAGE_VERSION

# populates `RATTLER_CHANNELS` array and `RATTLER_ARGS` array
source ./ci/use_conda_packages_from_prs.sh
source rapids-rattler-channel-string


Expand Down
1 change: 1 addition & 0 deletions ci/build_docs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ rapids-logger "Downloading artifacts from previous jobs"
CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcuopt cuopt --cuda "$RAPIDS_CUDA_VERSION")")
PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_python cuopt cuopt --py "$RAPIDS_PY_VERSION" --cuda "$RAPIDS_CUDA_VERSION")")

source ./ci/use_conda_packages_from_prs.sh
rapids-logger "Generating conda environment YAML"

rapids-dependency-file-generator \
Expand Down
1 change: 1 addition & 0 deletions ci/build_python.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ for package_name in cuopt cuopt_server; do
done

# populates `RATTLER_CHANNELS` array and `RATTLER_ARGS` array
source ./ci/use_conda_packages_from_prs.sh
source rapids-rattler-channel-string

# Override `rapids-rattler-channelstring` while cuOpt is not on the standard RAPIDS release cycle
Expand Down
1 change: 1 addition & 0 deletions ci/build_wheel_cuopt.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")"
RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")"
LIBCUOPT_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_cpp libcuopt cuopt --cuda "$RAPIDS_CUDA_VERSION")")

source ./ci/use_wheels_from_prs.sh
echo "libcuopt-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo ${LIBCUOPT_WHEELHOUSE}/libcuopt_*.whl)" >> "${PIP_CONSTRAINT}"

EXCLUDE_ARGS=(
Expand Down
2 changes: 2 additions & 0 deletions ci/build_wheel_libcuopt.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,8 @@ rapids-dependency-file-generator \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION};cuda_suffixed=true" \
| tee /tmp/requirements-build.txt

source ./ci/use_wheels_from_prs.sh

rapids-logger "Installing build requirements"
rapids-pip-retry install \
-v \
Expand Down
4 changes: 4 additions & 0 deletions ci/test_cpp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,15 @@ rapids-logger "Configuring conda strict channel priority"
conda config --set channel_priority strict

CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcuopt cuopt --cuda "$RAPIDS_CUDA_VERSION")")
source ./ci/use_conda_packages_from_prs.sh

rapids-logger "Generate C++ testing dependencies"
source ./ci/use_conda_packages_from_prs.sh
rapids-dependency-file-generator \
--output conda \
--file-key test_cpp \
--prepend-channel "${LIBRAFT_CHANNEL}" \
--prepend-channel "${RAFT_CHANNEL}" \
--prepend-channel "${CPP_CHANNEL}" \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch)" | tee env.yaml

Expand Down
4 changes: 4 additions & 0 deletions ci/test_cpp_memcheck.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,16 @@ set -euo pipefail
rapids-logger "Configuring conda strict channel priority"
conda config --set channel_priority strict

source ./ci/use_conda_packages_from_prs.sh
RAPIDS_VERSION="$(rapids-version)"

rapids-logger "Generate C++ testing dependencies"
source ./ci/use_conda_packages_from_prs.sh
rapids-dependency-file-generator \
--output conda \
--file-key test_cpp \
--prepend-channel "${LIBRAFT_CHANNEL}" \
--prepend-channel "${RAFT_CHANNEL}" \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch)" | tee env.yaml

rapids-mamba-retry env create --yes -f env.yaml -n test
Expand Down
4 changes: 4 additions & 0 deletions ci/test_notebooks.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,17 @@ rapids-logger "Downloading artifacts from previous jobs"
CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcuopt cuopt --cuda "$RAPIDS_CUDA_VERSION")")
PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_python cuopt cuopt --py "$RAPIDS_PY_VERSION" --cuda "$RAPIDS_CUDA_VERSION")")

source ./ci/use_conda_packages_from_prs.sh
rapids-logger "Generate notebook testing dependencies"

ENV_YAML_DIR="$(mktemp -d)"

source ./ci/use_conda_packages_from_prs.sh

@coderabbitai coderabbitai Bot Jun 24, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Remove duplicate sourcing of use_conda_packages_from_prs.sh.

The script is already sourced at Line 17; sourcing again at Line 22 re-runs artifact/channel side effects (including conda config --system --add channels) unnecessarily. Keep a single source call.

🧰 Tools
🪛 Shellcheck (0.11.0)

[info] 22-22: Not following: ./ci/use_conda_packages_from_prs.sh was not specified as input (see shellcheck -x).

(SC1091)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/test_notebooks.sh` at line 22, The test_notebooks.sh script sources
use_conda_packages_from_prs.sh twice, which repeats its side effects
unnecessarily. Remove the duplicate source call and keep only one sourcing point
in the script so the conda channel/artifact setup runs once, referencing the
source invocation in test_notebooks.sh and the use_conda_packages_from_prs.sh
helper.

rapids-dependency-file-generator \
--output conda \
--file-key test_notebooks \
--prepend-channel "${LIBRAFT_CHANNEL}" \
--prepend-channel "${RAFT_CHANNEL}" \
--prepend-channel "${CPP_CHANNEL}" \
--prepend-channel "${PYTHON_CHANNEL}" \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" | tee "${ENV_YAML_DIR}/env.yaml"
Expand Down
3 changes: 3 additions & 0 deletions ci/test_python.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,13 @@ rapids-logger "Downloading artifacts from previous jobs"
CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcuopt cuopt --cuda "$RAPIDS_CUDA_VERSION")")
PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_python cuopt cuopt --py "$RAPIDS_PY_VERSION" --cuda "$RAPIDS_CUDA_VERSION")")

source ./ci/use_conda_packages_from_prs.sh
rapids-logger "Generate Python testing dependencies"
rapids-dependency-file-generator \
--output conda \
--file-key test_python \
--prepend-channel "${LIBRAFT_CHANNEL}" \
--prepend-channel "${RAFT_CHANNEL}" \
--prepend-channel "${CPP_CHANNEL}" \
--prepend-channel "${PYTHON_CHANNEL}" \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" | tee env.yaml
Expand Down
1 change: 1 addition & 0 deletions ci/test_self_hosted_service.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ LIBCUOPT_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_
CUOPT_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_python cuopt cuopt --py "$RAPIDS_PY_VERSION" --cuda "$RAPIDS_CUDA_VERSION")")
CUOPT_SERVER_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_python cuopt-server cuopt --pure --arch any --cuda "$RAPIDS_CUDA_VERSION")")

source ./ci/use_wheels_from_prs.sh
# generate constraints (possibly pinning to oldest support versions of dependencies)
rapids-generate-pip-constraints test_python "${PIP_CONSTRAINT}"

Expand Down
1 change: 1 addition & 0 deletions ci/test_wheel_cuopt.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ LIBCUOPT_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_
CUOPT_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_python cuopt cuopt --py "$RAPIDS_PY_VERSION" --cuda "$RAPIDS_CUDA_VERSION")")
CUOPT_SH_CLIENT_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_python cuopt-sh-client cuopt --pure --arch any)")

source ./ci/use_wheels_from_prs.sh
# update pip constraints.txt to ensure all future 'pip install' (including those in ci/thirdparty-testing)
# use these wheels for cuopt packages
cat > "${PIP_CONSTRAINT}" <<EOF
Expand Down
1 change: 1 addition & 0 deletions ci/test_wheel_cuopt_server.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ CUOPT_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_pyt
CUOPT_SERVER_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_python cuopt-server cuopt --pure --arch any --cuda "$RAPIDS_CUDA_VERSION")")
CUOPT_SH_CLIENT_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_python cuopt-sh-client cuopt --pure --arch any)")

source ./ci/use_wheels_from_prs.sh
# generate constraints (possibly pinning to oldest support versions of dependencies)
rapids-generate-pip-constraints test_python "${PIP_CONSTRAINT}"

Expand Down
30 changes: 30 additions & 0 deletions ci/use_conda_packages_from_prs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION.
# SPDX-License-Identifier: Apache-2.0

# download CI artifacts
Comment on lines +1 to +5

@coderabbitai coderabbitai Bot Jun 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add strict shell options in this new CI helper.

This new script lacks set -euo pipefail, so standalone invocation can miss failures or unset-variable mistakes.

💡 Proposed fix 
 #!/bin/bash
 # SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION.
 # SPDX-License-Identifier: Apache-2.0
+
+set -euo pipefail

As per coding guidelines, new scripts under ci/**/*.sh should follow set -euo pipefail hygiene. Based on learnings, this repository prefers relying on set -u unbound-variable behavior instead of custom guards.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/use_conda_packages_from_prs.sh` around lines 1 - 5, Add strict shell
options to the script by inserting the line enabling "set -euo pipefail"
immediately after the existing shebang (#!/bin/bash) so the script exits on
errors, treats unset variables as failures, and propagates failures through
pipes; do not add custom guards—rely on set -u for unbound-variable behavior as
preferred by the repo.

Sources: Coding guidelines, Learnings

LIBRAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 cpp conda)
RAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 python conda)
Comment on lines +6 to +7

@coderabbitai coderabbitai Bot Jun 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Avoid hard-coding a single upstream PR artifact ID.

Line 6 and Line 7 pin artifact retrieval to raft PR 3052, which makes these jobs validate against one fixed external PR and can fail when that artifact is unavailable. Parameterize the PR ID from CI input.

💡 Proposed fix 
- LIBRAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 cpp conda)
- RAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 python conda)
+ RAFT_PR_ID="${RAFT_PR_ID}"
+ LIBRAFT_CHANNEL=$(rapids-get-pr-artifact raft "${RAFT_PR_ID}" cpp conda)
+ RAFT_CHANNEL=$(rapids-get-pr-artifact raft "${RAFT_PR_ID}" python conda)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
LIBRAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 cpp conda)
RAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 python conda)
RAFT_PR_ID="${RAFT_PR_ID}"
LIBRAFT_CHANNEL=$(rapids-get-pr-artifact raft "${RAFT_PR_ID}" cpp conda)
RAFT_CHANNEL=$(rapids-get-pr-artifact raft "${RAFT_PR_ID}" python conda)
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/use_conda_packages_from_prs.sh` around lines 6 - 7, The script currently
hard-codes raft PR id 3052 in the rapids-get-pr-artifact calls for
LIBRAFT_CHANNEL and RAFT_CHANNEL; change these to read the PR id from a CI
input/env var (e.g., RAFT_PR or INPUT_RAFT_PR) and use that variable in the
rapids-get-pr-artifact invocations, providing a sensible fallback or failing
fast with a clear error if the variable is missing; update the references to
LIBRAFT_CHANNEL and RAFT_CHANNEL so they call rapids-get-pr-artifact with the
parameterized PR id instead of the literal 3052.


# For `rattler` builds:
#
# Add these channels to the array checked by 'rapids-rattler-channel-string'.
# This ensures that when conda packages are built with strict channel priority enabled,
# the locally-downloaded packages will be preferred to remote packages (e.g. nightlies).
#
RAPIDS_PREPENDED_CONDA_CHANNELS=(
"${LIBRAFT_CHANNEL}"
"${RAFT_CHANNEL}"
)
export RAPIDS_PREPENDED_CONDA_CHANNELS

# For tests and `conda-build` builds:
#
# Add these channels to the system-wide conda configuration.
# This results in PREPENDING them to conda's channel list, so
# these packages should be found first if strict channel priority is enabled.
#
for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}"
do
conda config --system --add channels "${_channel}"
done
18 changes: 18 additions & 0 deletions ci/use_wheels_from_prs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION.
# SPDX-License-Identifier: Apache-2.0

# initialize PIP_CONSTRAINT
source rapids-init-pip

RAPIDS_PY_CUDA_SUFFIX=$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")
Comment on lines +1 to +8

@coderabbitai coderabbitai Bot Jun 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add strict mode in the helper script entry.

Line 1 introduces a standalone CI helper, but set -euo pipefail is not enabled. If this file is sourced by a non-strict caller, failed artifact lookup or constraint generation can continue silently.

Suggested patch 
 #!/bin/bash
 # SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION.
 # SPDX-License-Identifier: Apache-2.0
 
+set -euo pipefail
+
 # initialize PIP_CONSTRAINT
 source rapids-init-pip

As per coding guidelines, CI shell scripts should keep set -euo pipefail hygiene, and based on learnings this repo prefers relying on Bash’s built-in unbound-variable handling via set -u.

🧰 Tools
🪛 Shellcheck (0.11.0)

[info] 6-6: Not following: rapids-init-pip was not specified as input (see shellcheck -x).

(SC1091)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/use_wheels_from_prs.sh` around lines 1 - 8, This script lacks Bash
strict-mode so failures can be masked; add a strict-mode declaration by
inserting set -euo pipefail (and optionally IFS=$'\n\t' if desired) immediately
after the shebang so any failure in sourcing rapids-init-pip, the
rapids-wheel-ctk-name-gen call, or use of RAPIDS_PY_CUDA_SUFFIX will abort and
surface errors; ensure the declaration appears before the source rapids-init-pip
and before any use of RAPIDS_CUDA_VERSION/RAPIDS_PY_CUDA_SUFFIX.

Sources: Coding guidelines, Learnings


# download wheels, store the directories holding them in variables
LIBRAFT_WHEELHOUSE=$(rapids-get-pr-artifact raft 3052 cpp wheel)
PYLIBRAFT_WHEELHOUSE=$(rapids-get-pr-artifact raft 3052 python wheel --pkg_name pylibraft)

# write a pip constraints file saying e.g. "whenever you encounter a requirement for 'librmm-cu12', use this wheel"
cat > "${PIP_CONSTRAINT}" <<EOF
libraft-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBRAFT_WHEELHOUSE}"/libraft_*.whl)
raft-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${PYLIBRAFT_WHEELHOUSE}"/pylibraft_*.whl)
EOF
Loading