Skip to content

feat(es): route Elasticsearch through Kibana's Console proxy - #489

Closed
dnovitski wants to merge 1 commit into
elastic:mainfrom
dnovitski:feat/es-via-kibana
Closed

feat(es): route Elasticsearch through Kibana's Console proxy#489
dnovitski wants to merge 1 commit into
elastic:mainfrom
dnovitski:feat/es-via-kibana

Conversation

@dnovitski

Copy link
Copy Markdown

Closes #488

Problem

Deployments that publish only Kibana and keep Elasticsearch unreachable from clients
cannot use any elastic es … command — getEsClient() requires an elasticsearch block
with a directly reachable url, so the CLI stops at:

missing_config: No Elasticsearch connection configured in the active context.

Kibana can already reach the cluster on the caller's behalf through its Console proxy,
the same route Dev Tools uses.

Approach

An elasticsearch block may now declare via: kibana instead of a url:

contexts:
  proxied:
    kibana:
      url: https://kibana.example.internal
      auth:
        api_key: $(keychain:elastic-cli/proxied:kibana.auth.api_key)
    elasticsearch:
      via: kibana

Because the proxy returns the Elasticsearch response body verbatim and reports the true
Elasticsearch status in a header, this fits entirely behind the transport contract: every
generated ES command and the bulk/scroll/msearch/watch helpers work unchanged.
There is deliberately no search-specific code path.

  • EsTransport is extracted into src/lib/es-transport.ts so a transport can depend on the
    contract without importing the direct client (avoiding an import cycle). es-client.ts
    re-exports it, so existing imports are untouched. The six consuming sites already named
    their variable transport, making this a type-only change.
  • EsConsoleProxyClient folds the ES path and querystring into the proxy's path
    parameter, sends the required x-elastic-internal-origin header, and reconstructs the
    real ES status from x-console-proxy-status-code.
  • Failure layers stay distinguishable: a non-2xx from Kibana is an EsConnectionError
    (the ES call never happened), while a proxied 4xx/5xx becomes the usual
    EsResponseError, so status codes, --json payloads, and exit codes are identical to a
    direct connection.
  • elastic status reports the route (green (5 nodes) via Kibana) and probes the cluster
    through the proxy, reusing the same URL/header builders so the two cannot drift.
  • --es-via is added to config context add/edit via the existing PLAIN_FIELDS table.

Config validation

via is mutually exclusive with both url and auth, and requires a kibana block in the
same context. These are enforced rather than ignored on purpose: config objects strip
unknown keys rather than failing, so { via: kibana, url: … } would otherwise silently
drop one of the two.

Notable finding

Both Kibana versions reject the Console proxy unless the caller sends
x-elastic-internal-origin: Kibana, and the rejection is misleading:

400 {"message":"uri [/api/console/proxy] with method [post] exists but is not available
     with the current configuration"}

That reads as "the route is disabled on this deployment" when the header is simply missing.
The error path in this PR adds an explicit hint for it, and the reasoning is captured in
comments so it is not lost.

Verification

npm test (build + unit + license), npm run test:lint, scripts/check-spdx and
npm run test:notice all pass. Coverage is 96.6% lines / 90.4% branches / 93.5% functions
overall; es-console-proxy-client.ts is at 100% lines / 100% functions.

Tested end-to-end against Kibana 9.2.1 and 8.18.2 with a genuinely firewalled cluster
(direct :9200 unreachable):

Case Result
status ✓ green (5 nodes) via Kibana
es cluster health full health document
es cat indices text passthrough; ES security_exception surfaced verbatim
es search --input-file identical hits.total to querying via Kibana directly
search on a missing index transport_error, status_code: 404, exit 1
--dry-run validates, no network call
direct-ES context unchanged

Unit tests cover adversarial paths (spaces, *, ../, ?, #, empty), both auth types,
NDJSON bulk bodies, HEAD semantics, the header-absent fallback, and RequestInit
(method, redirect: 'error', headers).

Open questions (from #488)

  1. Is via: kibana the shape you want, or would you prefer it nested (e.g. under a
    transport: key), or z.enum to leave room for future routes?
  2. Should a via-Kibana context expose ELASTIC_ES_* to extensions? This PR omits them,
    since there is no Elasticsearch endpoint to pass on.
  3. Do you want a functional test? test/functional/es/ is codegen output today and
    test/functional/kb/ is hand-written; a test here needs a live Kibana.

Happy to reshape any of the above.

Deployments that publish only Kibana and keep Elasticsearch unreachable could not
use any `elastic es` command: getEsClient() requires a directly reachable url.

An `elasticsearch` block may now declare `via: kibana` instead of a url, in which
case requests are forwarded by Kibana's Console proxy using the context's Kibana
credentials. Because the proxy returns the Elasticsearch body verbatim, this sits
behind the transport contract and every generated ES command and helper works
unchanged -- there is no search-specific path.

- extract the EsTransport contract into es-transport.ts so a transport can depend
  on it without importing the direct client; EsClient re-exports it for consumers
- add EsConsoleProxyClient, which folds the ES path and querystring into the
  proxy's `path` parameter, sends the required x-elastic-internal-origin header,
  and reconstructs the real ES status from x-console-proxy-status-code
- distinguish the two failure layers: a non-2xx from Kibana is an
  EsConnectionError (the ES call never happened) with a hint for the misleading
  "not available with the current configuration" reply, while a proxied 4xx/5xx
  becomes the usual EsResponseError, preserving status codes and exit codes
- reject `via` together with `url` or `auth`, and require a `kibana` block to
  route through, since config objects strip unknown keys rather than failing
- report the route in `elastic status` (`green (5 nodes) via Kibana`) and probe
  the cluster through the proxy
- omit ELASTIC_ES_* from extension env for such a context: there is no endpoint
- add `--es-via` to `config context add`/`edit`

Verified end-to-end against Kibana 9.2.1 and 8.18.2 with a firewalled cluster:
status, cluster health, cat (text passthrough), search, ES 404 mapping, and
--dry-run. Both majors require the internal-origin header; without it Kibana
answers 400 with wording that reads as though the route were disabled.
@margaretjgu

Copy link
Copy Markdown
Member

Thanks for pr! Left a comment on the issue instead to better understand the context to see if this change is truly needed

@dnovitski

Copy link
Copy Markdown
Author

Closing this: the premise it rested on was wrong. Elasticsearch is reachable here after all — details and the corrected reproduction are in #488 (comment). Thanks @margaretjgu for pushing back rather than taking it at face value.

Leaving the branch on my fork in case the convenience angle (one endpoint, no CA distribution) ever looks worth pursuing, but that is your call and a much weaker case than what I originally claimed.

@dnovitski dnovitski closed this Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Elasticsearch access through Kibana's Console proxy (ES unreachable, Kibana-only deployments)

2 participants