feat: RAG service API key file management

[tsivaprasad]

Summary

This PR adds RAGServiceKeysResource, a resource that stores provider API keys (OpenAI, Anthropic, etc.) as files on the host filesystem and mounts them into the RAG container in read-only mode. The keys are not exposed via Docker Swarm configs or API responses.

Changes

• Add RAGServiceKeysResource (swarm.rag_service_keys) — manages the {DataDir}/services/{instanceID}/keys/ directory: Create writes key files at mode 0600 in a 0700 directory, Update rewrites them for key rotation, Delete removes the directory via os.RemoveAll
• Add extractRAGAPIKeys helper that extracts API keys from a parsed RAGServiceConfig into a {pipeline_name}_embedding.key / {pipeline_name}_rag.key filename map
• Extend ServiceContainerSpecOptions with KeysPath; when non-empty a read-only bind mount to /app/keys is added to the container spec
• Wire RAGServiceKeysResource into generateRAGInstanceResources after RAGServiceUserRole; RAG config is now parsed at resource-generation time
• Register the new resource type in RegisterResourceTypes

Testing

1. Covered unit tests
2. Manual Verification

• Created cluster
• Created DB using below config:

{
  "id": "storefront",
  "spec": {
    "database_name": "storefront",
    "database_users": [
      {
        "username": "admin",
        "password": "password",
        "db_owner": true,
        "attributes": ["SUPERUSER", "LOGIN"]
      }
    ],
    "port": 0,
    "nodes": [
      { "name": "n1", "host_ids": ["host-1"] }
    ],
    "services": [
      {
        "service_id": "rag",
        "service_type": "rag",
        "version": "latest",
        "host_ids": ["host-1"],
        "port": 0,
        "config": {
          "defaults": {
            "token_budget": 2000,
            "top_n": 10
          },
          "pipelines": [
            {
              "name": "default",
              "description": "Main RAG pipeline",
              "tables": [
                {
                  "table": "documents_content_chunks",
                  "text_column": "content",
                  "vector_column": "embedding"
                }
              ],
              "embedding_llm": {
                "provider": "openai",
                "model": "text-embedding-3-small",
                "api_key": "sk-proj-AAA"
              },
              "rag_llm": {
                "provider": "anthropic",
                "model": "claude-sonnet-4-5",
                "api_key": "sk-ant-XXX"
              },
              "token_budget": 4000,
              "top_n": 15,
              "search": {
                "hybrid_enabled": true,
                "vector_weight": 0.7
              }
            }
          ]
        }
      }
    ]
  }
}

• No api_keys in response
• Confirmed that the key files exists at path

# pwd
/host-1/services/storefront-rag-host-1/keys
# ls -la
total 8
drwx------ 4 root root 128 Mar 17 16:48 .
drwx------ 3 root root  96 Mar 17 16:48 ..
-rw------- 1 root root 164 Mar 17 16:48 default_embedding.key
-rw------- 1 root root 108 Mar 17 16:48 default_rag.key

Checklist

• Tests added

Notes for Reviewers

• RAGServiceKeysResource owns the keys/ subdirectory entirely (no separate DirResource), since no other resource shares it
• The KeysPath field on ServiceContainerSpecOptions is currently unused by MCP (zero value skips the mount) — no behaviour change for existing MCP services
• Key files are written as root-owned (UID 0); the RAG container UID and ownership will be addressed when container deployment is wired up in the next PR

PLAT-490

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 09f162c4-cda5-4425-80c2-8774ce29026f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The changes introduce a new RAGServiceKeysResource to manage RAG API keys on the host filesystem. The orchestrator now parses RAG service configs, generates key resources that write authentication files to a configurable host path, and integrates them into the resource orchestration chain. Container specifications now support mounting these key directories read-only.

Changes

Cohort / File(s)	Summary
RAG Keys Resource Implementation server/internal/orchestrator/swarm/rag_service_keys_resource.go	New resource type managing RAG API key lifecycle (Create, Update, Delete, Refresh), extracting keys from RAG service configs by pipeline name convention, and writing files with restricted permissions to a host directory.
Resource Tests server/internal/orchestrator/swarm/rag_service_keys_resource_test.go	Comprehensive unit tests covering resource metadata, executor binding, key extraction from single and multi-pipeline configs, refresh validation, and integration with the orchestrator's resource generation pipeline.
Orchestrator Integration server/internal/orchestrator/swarm/orchestrator.go, server/internal/orchestrator/swarm/resources.go, server/internal/orchestrator/swarm/rag_service_user_role_test.go	Wires RAGServiceKeysResource into the orchestration chain following RAG user role creation, registers the new resource type in the registry, and updates existing tests to validate the presence of the keys resource alongside user role resources.
Container Configuration server/internal/orchestrator/swarm/service_spec.go	Adds optional KeysPath field to ServiceContainerSpecOptions for read-only mounting of RAG key directories at /app/keys within containers alongside existing data mounts.

Poem

🐰 Keys now neatly filed away,
On host paths, safe and snug they stay,
Pipelines dance with embedding pride,
In mounted volumes, keys reside,
RAG secrets organized, hooray! 🔑

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 15.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main feature being added: RAG service API key file management, which aligns with the primary objective of the changeset.
Description check	✅ Passed	The PR description comprehensively covers all required template sections: Summary, Changes, Testing (with manual verification steps and output), and a filled Checklist confirming tests were added, with an issue link included.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch PLAT-490-rag-service-api-key-file-management

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Tip

CodeRabbit can approve the review once all CodeRabbit's comments are resolved.

Enable the reviews.request_changes_workflow setting to automatically approve the review once all CodeRabbit's comments are resolved.

[tsivaprasad]

@coderabbitai review

[coderabbitai]

@tsivaprasad Sure! I'll review the changes in this PR now.

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@server/internal/orchestrator/swarm/rag_service_keys_resource.go`:
- Around line 121-127: extractRAGAPIKeys currently uses pipeline names directly
to build filenames (e.g., keys[p.Name+"_embedding.key"]) and writeKeyFiles
writes them to disk, allowing path traversal via crafted pipeline names; fix by
sanitizing/validating pipeline names before using them in filenames (e.g.,
reject names containing path separators or non-alphanumeric chars, or replace
unsafe chars with underscores) and build file paths with filepath.Join then
verify the cleaned path is inside KeysDir (ensure filepath.Clean(joinedPath) has
KeysDir as a prefix) before writing; apply the same hardening wherever p.Name is
used to form filenames in extractRAGAPIKeys and writeKeyFiles.
- Around line 76-80: The Refresh loop in Refresh is only checking
os.IsNotExist(err) and ignores other stat errors; update the loop over r.Keys
(using r.KeysDir and filepath.Join) so that after calling os.Stat you: if err ==
nil continue; if os.IsNotExist(err) return resource.ErrNotFound; otherwise
return or wrap the encountered err (preserving context such as the key name) so
permission/I/O errors are surfaced instead of treated as present. Ensure you
reference the os.Stat call and return the real error for non-NotExist cases.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d7d38e6a-f745-4663-993e-e43256518cb2

📥 Commits

Reviewing files that changed from the base of the PR and between 4392b87 and 26eec5e.

📒 Files selected for processing (6)

• server/internal/orchestrator/swarm/orchestrator.go
• server/internal/orchestrator/swarm/rag_service_keys_resource.go
• server/internal/orchestrator/swarm/rag_service_keys_resource_test.go
• server/internal/orchestrator/swarm/rag_service_user_role_test.go
• server/internal/orchestrator/swarm/resources.go
• server/internal/orchestrator/swarm/service_spec.go