Skip to content

Fail fast on missing inference credentials - #1163

Open
qianl-nv wants to merge 1 commit into
mainfrom
qianl/fix/inference-credential-preflight
Open

Fail fast on missing inference credentials#1163
qianl-nv wants to merge 1 commit into
mainfrom
qianl/fix/inference-credential-preflight

Conversation

@qianl-nv

@qianl-nv qianl-nv commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

Validate inference credentials before simulation

Detailed description

  • Prevent missing API keys from starting Isaac Sim unnecessarily
  • Reuse credential resolution in the inference backend and CLI preflight
  • Cover environment, explicit-key, and recovery-path behavior

Fix for v0.3 VDR

Check endpoint credentials before full mode starts Isaac Sim so
configuration errors avoid GPU startup and unclean simulator teardown.

Signed-off-by: Qian Lin <qianl@nvidia.com>
@qianl-nv
qianl-nv marked this pull request as ready for review August 31, 2026 06:49
@greptile-apps

greptile-apps Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR centralizes endpoint credential resolution and adds a credential preflight before full-mode Isaac Sim startup.

  • Adds require_inference_api_key and reuses it during backend construction.
  • Runs credential validation before entering the simulator context.
  • Adds coverage for environment, explicit-key, and missing-key behavior.

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking robustness issue when the CLI runs under optimized Python.

The normal CLI path correctly validates credentials before simulator startup, but optimized execution strips the assertion and allows the expensive startup to proceed before authentication fails downstream.

Files Needing Attention: isaaclab_arena/agentic_environment_generation/inference_backend.py

Important Files Changed

Filename Overview
isaaclab_arena/agentic_environment_generation/inference_backend.py Extracts shared endpoint/key resolution, but the new preflight contract remains dependent on an optimization-sensitive assertion.
isaaclab_arena_examples/agentic_environment_generation/cli_runner.py Adds pre-simulator credential validation only to full mode, whose downstream path always performs inference.
isaaclab_arena/tests/test_inference_backend.py Covers environment-key resolution, explicit-key precedence, and recovery guidance for missing credentials.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Start full-mode CLI] --> B[Resolve endpoint and API key]
  B --> C{Credential present?}
  C -- No --> D[Exit before simulator startup]
  C -- Yes --> E[Enter SimulationAppContext]
  E --> F[Construct inference backend]
  F --> G[Generate and build environment]
Loading

Reviews (1): Last reviewed commit: "Fail fast on missing inference credentia..." | Re-trigger Greptile

Comment on lines +103 to +107
assert resolved_api_key, (
f"API key required for the {inference_endpoint.name!r} inference endpoint: set "
f"{inference_endpoint.api_key_env_var} or pass api_key. Select another endpoint with "
f"{INFERENCE_ENDPOINT_ENV_VAR}."
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Optimization strips credential preflight

If the CLI runs under python -O or with PYTHONOPTIMIZE enabled, Python removes this assertion, so full mode enters SimulationAppContext with no API key and fails during downstream client authentication after Isaac Sim has started.

Suggested change
assert resolved_api_key, (
f"API key required for the {inference_endpoint.name!r} inference endpoint: set "
f"{inference_endpoint.api_key_env_var} or pass api_key. Select another endpoint with "
f"{INFERENCE_ENDPOINT_ENV_VAR}."
)
if not resolved_api_key:
raise AssertionError(
f"API key required for the {inference_endpoint.name!r} inference endpoint: set "
f"{inference_endpoint.api_key_env_var} or pass api_key. Select another endpoint with "
f"{INFERENCE_ENDPOINT_ENV_VAR}."
)

Knowledge Base Used:

return 0

# Validate inference endpoint credentials before full mode starts Isaac Sim.
require_inference_api_key(args_cli.inference_endpoint)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Could full mode just resolve before starting Isaac Sim?

A missing key is only one of the resolve-time failures that currently pays for a full Isaac Sim startup — five lines below, an invalid agent spec also bails out with return 1 after the sim is already up. Since --mode resolve runs resolve_env_spec with no sim at all, could full mode do the same and enter SimulationAppContext only once it has a spec path? That would cover every resolve failure and this preflight (and the new helper's second caller) could go away:

env_graph_spec_path = resolve_env_spec(args_cli)
if env_graph_spec_path is None:
    return 1
with SimulationAppContext(args_cli):
    build_env_and_run_policy(env_graph_spec_path, args_cli)
return 0

Or is there an import-ordering reason the resolve step has to run inside the context here?

@arena-review-bot

Copy link
Copy Markdown
Contributor

🤖 Isaac Lab-Arena Review Bot

Summary

Extracts the endpoint + API-key resolution out of InferenceBackend.__init__ into a reusable require_inference_api_key() and calls it as a preflight in the CLI runner so --mode full fails before Isaac Sim starts. The extraction is clean and the backend keeps identical behavior; my one question is whether the preflight is treating a symptom rather than the cause.

Design, Boundaries & Scope

--mode full starts Isaac Sim before running the agent, so every resolve-time failure — not just a missing key — burns a sim startup. The invalid-spec path a few lines down (return 1 on env_graph_spec_path is None) still does exactly what this PR is trying to avoid. --mode resolve already runs resolve_env_spec with no sim, so hoisting the resolve step above SimulationAppContext in full mode would cover all of these cases at once and make the preflight unnecessary. Raised as a question inline — there may be an import-ordering reason it sits inside the context.

Findings

🟡 Warning: isaaclab_arena_examples/agentic_environment_generation/cli_runner.py:384 — credential preflight covers one failure mode; hoisting the resolve step out of the sim context would cover all of them. (inline)

Test Coverage

Three new unit tests on require_inference_api_key (env key, explicit-key precedence, error message content); they run in phase 1 and the autouse clean_endpoint_env fixture correctly clears all three key vars so the negative test does not depend on the developer's environment. Worth noting the behavior this PR actually changes — full mode aborting before SimulationAppContext — has no test, though cli_runner.py has no existing test harness so that is understandable.

Verdict

Minor fixes needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant