Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions tests/bdd/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,9 @@ logic into `dsl/`.
namespace, and intended Gateway parent plus the shared context and timeout.
The step requires `Accepted=True` and `ResolvedRefs=True` for that parent but
does not allowlist route kinds or duplicate Gateway API validation.
- File-mutating steps (`I copy the file`, `I update yaml file`,
`I prepare self-managed secrets file`, `I substitute a block`)
- File-mutating steps (`I copy the file`, `I write yaml file`,
`I update yaml file`, `I prepare self-managed secrets file`,
`I substitute a block`)
snapshot the destination through `Suite.Ledger` before the first write.
Suite teardown restores every snapshotted path.
- `Given command has succeeded:` keys on the fully resolved command
Expand Down
3 changes: 2 additions & 1 deletion tests/bdd/PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ refactor in every consumer; that is a feature.
| Step | Notes |
|------|-------|
| `And I copy the file {string} to {string}` | Both paths are repo-relative. |
| `And I write yaml file {string} with values:` (two-column table of dotted-path and value) | Creates a new YAML file from the visible table. The destination must not already exist; the step fails instead of overwriting so an authored file is never silently replaced. Parent directories are created. Path syntax and `${VAR}` expansion match `I update yaml file`. Boolean literals and collection literals such as `[]` are written as native YAML types, not quoted strings, because Helm treats the string `"false"` as truthy. Numbers stay as strings. The destination is ledger-backed and removed at teardown. |
| `And I update yaml file {string} with keys:` (two-column table of dotted-path and value) | Path supports dotted notation and `[n]` indices (e.g. `global.imagePullSecrets[0].name`). Missing intermediate maps and missing list indices are upserted: writing `global.imagePullSecrets[0].name` against a file that has neither `global.imagePullSecrets` nor any list entry creates both. Existing scalars at intermediate positions cause the step to fail rather than silently overwrite a non-map. Value cells expand `${VAR}` from `os.Environ`. |
| `And I prepare Helmfile environment {string} for stack {string} from fixture {string} with values:` (two-column table of dotted-path and value) | Validates the stack and environment names, derives `deploy/stacks/<stack>/environments/<environment>.yaml` from the absolute repository root, copies the explicit fixture, and applies the visible values table with the same YAML update and `${VAR}` interpolation behavior. Supported stacks are `self-managed`, `observability`, and `nvcf-compute-plane`. The destination is ledger-backed. |
| `And I prepare self-managed secrets file {string} from template {string} using the current NGC registry credential` | The destination and template are explicit repo-relative paths with `${VAR}` interpolation. Replaces the template's registry credential placeholder with base64 of the current `$oauthtoken:<NGC_API_KEY>` credential and writes the destination with mode `0600`. The destination is ledger-backed, and secret material never enters Gherkin, command logs, or failure messages. |
Expand Down Expand Up @@ -273,7 +274,7 @@ contract verified in `src/clis/nvcf-cli/cmd/`):
## File restoration

Every step that writes into a path under the repo working tree
(`I copy the file ... to ...`, `I update yaml file ...`,
(`I copy the file ... to ...`, `I write yaml file ...`, `I update yaml file ...`,
`I prepare self-managed secrets file ...`, `I substitute a block ...`)
registers that path with the runner's
restoration ledger:
Expand Down
50 changes: 50 additions & 0 deletions tests/bdd/dsl/yamledit.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,32 @@ const (
MatchSubset
)

// RenderYAMLFromKeys builds a YAML document from the supplied
// dotted-path/value pairs and returns its serialized bytes. It is the
// pure counterpart of UpdateYAMLKeys for a file that does not exist
// yet: the caller owns the destination path, the existence check, and
// the write. Path syntax matches UpdateYAMLKeys. Value cells run
// through Interpolate and then decodeTypedValue so booleans and
// collection literals reach Helm as native YAML types rather than
// quoted strings.
func RenderYAMLFromKeys(keys [][2]string) ([]byte, error) {
rootMap := map[string]any{}
for _, kv := range keys {
segments, err := parsePath(kv[0])
if err != nil {
return nil, fmt.Errorf("render yaml: %w", err)
}
if err := setNested(rootMap, segments, decodeTypedValue(Interpolate(kv[1]))); err != nil {
return nil, fmt.Errorf("render yaml: %w", err)
}
}
body, err := yaml.Marshal(rootMap)
if err != nil {
return nil, fmt.Errorf("render yaml: marshal: %w", err)
}
return body, nil
}

// UpdateYAMLKeys reads the YAML file at path, applies each (dotted-path,
// value) pair as an upsert, and writes the file back. Path syntax uses
// "." between segments and "[n]" for list indices; missing intermediate
Expand Down Expand Up @@ -210,6 +236,30 @@ func SubstituteFileBlock(path, spec string) error {
return SubstituteFile(path, oldBlock, newBlock)
}

// decodeTypedValue converts the YAML-significant literals that Helm
// evaluates differently when quoted. Booleans are decoded because
// Go templates treat the string "false" as truthy. Collection
// literals like "[]" are decoded so Helm sees an empty list instead
// of a non-empty string. Numbers are left as strings: Helm coerces
// them in template expressions, and eagerly parsing "1.0" as a float
// would lose the trailing zero on round-trip.
func decodeTypedValue(s string) any {
switch s {
case "true":
return true
case "false":
return false
}
var decoded any
if err := yaml.Unmarshal([]byte(s), &decoded); err == nil {
switch decoded.(type) {
case []any, map[string]any:
return decoded
}
}
return s
}

// readYAMLAny reads path and unmarshals into a generic any value.
// An empty document parses to nil.
func readYAMLAny(path string) (any, error) {
Expand Down
101 changes: 101 additions & 0 deletions tests/bdd/dsl/yamledit_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,107 @@ func TestSubstituteFileBlockRejectsMissingOldBlock(t *testing.T) {
}
}

// TestRenderYAMLFromKeysBuildsNestedDocument verifies that
// RenderYAMLFromKeys produces a nested YAML structure from dotted-path
// key/value pairs without touching the filesystem.
func TestRenderYAMLFromKeysBuildsNestedDocument(t *testing.T) {
keys := [][2]string{
{"llmRequestRouter.fullnameOverride", "llm-request-router-region-b"},
{"llmRequestRouter.replicaCount", "2"},
{"llmRequestRouter.workload.kind", "StatefulSet"},
}
body, err := RenderYAMLFromKeys(keys)
if err != nil {
t.Fatalf("render: %v", err)
}

out := string(body)
for _, want := range []string{
"fullnameOverride: llm-request-router-region-b",
"replicaCount: \"2\"",
"kind: StatefulSet",
} {
if !strings.Contains(out, want) {
t.Fatalf("output missing %q:\n%s", want, out)
}
}
}

// TestRenderYAMLFromKeysPreservesBoolsAndCollections verifies that
// booleans and collection literals are decoded to native YAML types
// while numbers remain as quoted strings.
func TestRenderYAMLFromKeysPreservesBoolsAndCollections(t *testing.T) {
keys := [][2]string{
{"router.enabled", "true"},
{"router.pki.enabled", "false"},
{"router.replicaCount", "2"},
{"router.discovery.remoteWatchUrls", "[]"},
{"router.name", "region-b"},
}
body, err := RenderYAMLFromKeys(keys)
if err != nil {
t.Fatalf("render: %v", err)
}
out := string(body)

for _, want := range []string{
"enabled: true",
"enabled: false",
"remoteWatchUrls: []",
"name: region-b",
} {
if !strings.Contains(out, want) {
t.Fatalf("output missing %q:\n%s", want, out)
}
}
for _, unwanted := range []string{
`enabled: "true"`,
`enabled: "false"`,
`remoteWatchUrls: "[]"`,
} {
if strings.Contains(out, unwanted) {
t.Fatalf("value emitted as quoted string %q:\n%s", unwanted, out)
}
}
// Numbers stay as quoted strings; Helm coerces them in templates.
if !strings.Contains(out, `replicaCount: "2"`) {
t.Fatalf("replicaCount should remain a quoted string:\n%s", out)
}
}

// TestRenderYAMLFromKeysRejectsInvalidPath confirms that a malformed
// dotted path surfaces as an error instead of a partial document.
func TestRenderYAMLFromKeysRejectsInvalidPath(t *testing.T) {
keys := [][2]string{
{"router.name", "region-b"},
{"router..enabled", "true"},
}
body, err := RenderYAMLFromKeys(keys)
if err == nil || !strings.Contains(err.Error(), "empty segment") {
t.Fatalf("err = %v, want invalid-path error", err)
}
if body != nil {
t.Fatalf("body should be nil on error, got:\n%s", body)
}
}

// TestRenderYAMLFromKeysInterpolatesValues confirms that ${VAR}
// references in value cells are expanded before serialization.
func TestRenderYAMLFromKeysInterpolatesValues(t *testing.T) {
t.Setenv("BDD_TEST_HOST", "region-b.example.invalid")

keys := [][2]string{
{"service.host", "${BDD_TEST_HOST}"},
}
body, err := RenderYAMLFromKeys(keys)
if err != nil {
t.Fatalf("render: %v", err)
}
if !strings.Contains(string(body), "host: region-b.example.invalid") {
t.Fatalf("interpolation failed:\n%s", body)
}
}

func TestParsePathInvalidShapes(t *testing.T) {
bads := []string{
"a..b",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,12 +70,180 @@ Feature: Register an LLM worker securely with routers in two local regions
When I run command "kubectl --context k3d-ncp-local-cp rollout status deployment/llm-request-router -n nvcf --timeout=10m"
Then the command exit code should be 0

# This script still hides the Region B release values, routes, alias
# endpoints, and rollout waits. Replacing it with visible DSL steps is
# tracked in https://github.com/NVIDIA/nvcf/issues/1391.
When I run command "tests/bdd/scripts/install-llm-region-b.sh"
# Region B: write visible override values for a second LLM request
# router with a distinct StatefulSet identity.
Given I write yaml file "tests/bdd/out/region-b-values.yaml" with values:
| llmRequestRouter.fullnameOverride | llm-request-router-region-b |
| llmRequestRouter.replicaCount | 2 |
| llmRequestRouter.workload.kind | StatefulSet |
| llmRequestRouter.service.headlessName | llm-request-router-region-b-headless |
| llmRequestRouter.kubernetes.advertisedHostnameTemplate | {pod_name}.llm-request-router-region-b-headless.nvcf.svc.cluster.local |
| llmRequestRouter.discovery.remoteWatchUrls | [] |
| llmRequestRouter.backendRouter.enabled | true |
| llmRequestRouter.backendRouter.pylonGrpcDialAddress | https://region-b-watch.nvcf.svc.cluster.local:50071 |
| llmRequestRouter.backendRouter.pylonReverseTunnelDialAddress | region-b-watch.nvcf.svc.cluster.local:50072 |
| llmRequestRouter.backendRouter.image.pullPolicy | IfNotPresent |
| llmRequestRouter.serviceAccount.create | false |
| llmRequestRouter.serviceAccount.name | llm-request-router |
| llmRequestRouter.pki.enabled | false |
| llmRequestRouter.certificate.enabled | false |
| llmRequestRouter.tls.mode | existingSecret |
| llmRequestRouter.tls.secretName | stargate-quic-tls |
| llmRequestRouter.image.pullPolicy | IfNotPresent |

# Export Region A base values so Region B inherits image tags and
# shared config; then install Region B with the visible overrides.
When I successfully run command:
"""
/bin/bash -c 'helm --kube-context k3d-ncp-local-cp get values llm-request-router --namespace nvcf --output json | jq "{llmRequestRouter: .llmRequestRouter}" > ${REPO_ROOT}/tests/bdd/out/region-a-base-values.json'
"""
When I run command:
"""
helm --kube-context k3d-ncp-local-cp upgrade --install llm-request-router-region-b ${REPO_ROOT}/deploy/helm/llm-request-router/llm-request-router --namespace nvcf --values ${REPO_ROOT}/tests/bdd/out/region-a-base-values.json --values ${REPO_ROOT}/tests/bdd/out/region-b-values.yaml --wait --timeout 10m
"""
Then the command exit code should be 0

# Region B gateway resources: GRPCRoute, BackendTrafficPolicy,
# and cross-namespace ReferenceGrant.
When I successfully run command:
"""
kubectl --context k3d-ncp-local-cp apply -f - <<'YAML'
apiVersion: gateway.networking.k8s.io/v1
kind: GRPCRoute
metadata:
name: llm-worker-region-b-grpc
namespace: envoy-gateway-system
spec:
parentRefs:
- name: grpc-gw
namespace: envoy-gateway-system
sectionName: llm-grpc
hostnames:
- "region-b-watch.nvcf.svc.cluster.local"
- "*.llm-request-router-region-b-headless.nvcf.svc.cluster.local"
rules:
- backendRefs:
- name: llm-request-router-region-b-backend-router
namespace: nvcf
port: 50071
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
name: llm-worker-region-b-grpc-streams
namespace: envoy-gateway-system
spec:
targetRefs:
- group: gateway.networking.k8s.io
kind: GRPCRoute
name: llm-worker-region-b-grpc
timeout:
http:
requestTimeout: 0s
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
name: allow-llm-worker-region-b-route
namespace: nvcf
spec:
from:
- group: gateway.networking.k8s.io
kind: GRPCRoute
namespace: envoy-gateway-system
to:
- group: ""
kind: Service
name: llm-request-router-region-b-backend-router
YAML
"""

# Discover the control-plane endpoint IP for the region-b-watch alias.
When I run command "kubectl --context k3d-ncp-local-compute-1 get endpoints llm-request-router --namespace nvcf --output jsonpath={.subsets[0].addresses[0].ip}"
Then the command exit code should be 0
And I export command output to environment variable "CONTROL_PLANE_IP"

# Apply the region-b-watch Service and Endpoints alias in both
# clusters so each cluster can reach the Region B backend-router
# by name. The discovered control-plane IP is interpolated by the
# DSL so no external templating tool is needed.
When I successfully run command:
"""
kubectl --context k3d-ncp-local-cp apply -f - <<'YAML'
apiVersion: v1
kind: Service
metadata:
name: region-b-watch
namespace: nvcf
spec:
ports:
- name: llm-grpc
port: 50071
targetPort: llm-grpc
protocol: TCP
- name: llm-quic
port: 50072
targetPort: llm-quic
protocol: UDP
---
apiVersion: v1
kind: Endpoints
metadata:
name: region-b-watch
namespace: nvcf
subsets:
- addresses:
- ip: ${CONTROL_PLANE_IP}
ports:
- name: llm-grpc
port: 50071
protocol: TCP
- name: llm-quic
port: 50072
protocol: UDP
YAML
"""
When I successfully run command:
"""
kubectl --context k3d-ncp-local-compute-1 apply -f - <<'YAML'
apiVersion: v1
kind: Service
metadata:
name: region-b-watch
namespace: nvcf
spec:
ports:
- name: llm-grpc
port: 50071
targetPort: llm-grpc
protocol: TCP
- name: llm-quic
port: 50072
targetPort: llm-quic
protocol: UDP
---
apiVersion: v1
kind: Endpoints
metadata:
name: region-b-watch
namespace: nvcf
subsets:
- addresses:
- ip: ${CONTROL_PLANE_IP}
ports:
- name: llm-grpc
port: 50071
protocol: TCP
- name: llm-quic
port: 50072
protocol: UDP
YAML
"""

# Wait for Region B workloads to be ready.
When I successfully run command "kubectl --context k3d-ncp-local-cp rollout status statefulset/llm-request-router-region-b --namespace nvcf --timeout=10m"
And deployment "llm-request-router-region-b-backend-router" in namespace "nvcf" using context "k3d-ncp-local-cp" should complete rollout within "10m"

# The initial region advertises an explicit HTTPS recursive seed while
# retaining every concrete Deployment pod identity. Three distinct
# <replicaset-hash>-<pod-suffix> identities prove per-pod discovery
Expand Down
Loading