[AlejandroMorgante]

Add DatabricksAgentInvokeOperator and DatabricksAgentHook to invoke agents hosted with DurableAgentServer on Databricks Apps. Supports service principal OAuth, background submission, synchronous polling and deferrable waiting, preserving the full invocation response in XCom.

The hook implements BaseManagedAgentHook for Common AI and toolset use, with a default session per invocation. Request, state and error handling were checked against the runtime and templates in Databricks' databricks-ai-bridge repository, tag databricks-agentbricks-v0.4.0. Stored failures are distinguished from transient HTTP errors, and interruptions nested in the output are returned by the operator and rejected by Common AI.

Invocation IDs remain stable across retries and clears with unchanged input and session. Stored failures stop task retries and require a new ID to run again. GET timeouts use the minimum of the hook's HTTP timeout and the remaining budget; synchronous POST requests retain the full remaining budget. Deadline checks reject late results, but network activity and OAuth refresh can delay timeout reporting. Background hook methods accept the app URL per call.

Includes documentation, focused unit tests and a deterministic system-test fixture requiring no model endpoint.

Validation:

• After the GET timeout and per-call URL fixes, 174 hook, operator and trigger tests passed in Breeze, plus five local timeout-budget probes.
• After adding an async HTTP-options regression test and strengthening the decreasing polling-budget test, the affected modules passed again: 120 hook tests and 46 operator tests.
• The system test was rerun successfully against a real Databricks workspace with OAuth, covering synchronous, Common AI and deferrable invocation, before the final test-only changes.
• The Databricks documentation build passed after clarifying timeout guarantees. Ruff and commit hooks passed for the final test changes. Broader checks are left to CI.

Earlier validation included the full Databricks suite (1087 passed, 12 skipped), 142 agent tests with the minimum tenacity==8.3.0, five local checks against the Databricks 0.4.0 runtime with simulated OAuth, targeted mypy, static/manual checks, spellcheck and the dependency lower-bound checker. Those broader and minimum-dependency checks have not been repeated for the latest changes.

See the fixture README for deployment, configuration and cleanup. The fixture uses an in-memory store and does not test recovery across app restarts. With the app and configuration prepared:

SYSTEM_TESTS_ENV_ID=your-unique-id \
BREEZE_INIT_COMMAND='set -a; . /files/databricks-agent-system-test.env; set +a' \
breeze testing system-tests \
  --backend sqlite \
  --forward-credentials \
  --test-timeout 2400 \
  providers/databricks/tests/system/databricks/example_databricks_agent.py \
  -q

---

Was generative AI tooling used to co-author this PR?

• Yes — Codex (GPT-6)

Generated-by: Codex (GPT-6) following the guidelines

[AlejandroMorgante]

@SameerMesiah97 could you please help me with a review when you have some time? Thanks! 🙏

[AlejandroMorgante]

End-to-end validation

Validated commit 68fc3d9b60d8c6f24ef4e074d6bf052d09fd5ff2 against a real Databricks App running the deterministic agent fixture.

Both synchronous and deferrable invocations completed successfully. Verification tasks confirmed the response content, original input, and session ID. The deferrable task resumed successfully after the trigger fired.

Environment: Airflow 3.3.0, Python 3.12, PostgreSQL, and LocalExecutor. The test loaded the provider source from the PR commit. No model endpoint was used.

Successful Airflow run

Synchronous invocation

Deferrable invocation

Agent input and output

Databricks App

Databricks invocation logs

The temporary OAuth secret was revoked and the Databricks App stopped after validation.