feat(es): route Elasticsearch through Kibana's Console proxy - #489
Closed
dnovitski wants to merge 1 commit into
Closed
feat(es): route Elasticsearch through Kibana's Console proxy#489dnovitski wants to merge 1 commit into
dnovitski wants to merge 1 commit into
Conversation
Deployments that publish only Kibana and keep Elasticsearch unreachable could not use any `elastic es` command: getEsClient() requires a directly reachable url. An `elasticsearch` block may now declare `via: kibana` instead of a url, in which case requests are forwarded by Kibana's Console proxy using the context's Kibana credentials. Because the proxy returns the Elasticsearch body verbatim, this sits behind the transport contract and every generated ES command and helper works unchanged -- there is no search-specific path. - extract the EsTransport contract into es-transport.ts so a transport can depend on it without importing the direct client; EsClient re-exports it for consumers - add EsConsoleProxyClient, which folds the ES path and querystring into the proxy's `path` parameter, sends the required x-elastic-internal-origin header, and reconstructs the real ES status from x-console-proxy-status-code - distinguish the two failure layers: a non-2xx from Kibana is an EsConnectionError (the ES call never happened) with a hint for the misleading "not available with the current configuration" reply, while a proxied 4xx/5xx becomes the usual EsResponseError, preserving status codes and exit codes - reject `via` together with `url` or `auth`, and require a `kibana` block to route through, since config objects strip unknown keys rather than failing - report the route in `elastic status` (`green (5 nodes) via Kibana`) and probe the cluster through the proxy - omit ELASTIC_ES_* from extension env for such a context: there is no endpoint - add `--es-via` to `config context add`/`edit` Verified end-to-end against Kibana 9.2.1 and 8.18.2 with a firewalled cluster: status, cluster health, cat (text passthrough), search, ES 404 mapping, and --dry-run. Both majors require the internal-origin header; without it Kibana answers 400 with wording that reads as though the route were disabled.
Member
|
Thanks for pr! Left a comment on the issue instead to better understand the context to see if this change is truly needed |
Author
|
Closing this: the premise it rested on was wrong. Elasticsearch is reachable here after all — details and the corrected reproduction are in #488 (comment). Thanks @margaretjgu for pushing back rather than taking it at face value. Leaving the branch on my fork in case the convenience angle (one endpoint, no CA distribution) ever looks worth pursuing, but that is your call and a much weaker case than what I originally claimed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #488
Problem
Deployments that publish only Kibana and keep Elasticsearch unreachable from clients
cannot use any
elastic es …command —getEsClient()requires anelasticsearchblockwith a directly reachable url, so the CLI stops at:
Kibana can already reach the cluster on the caller's behalf through its Console proxy,
the same route Dev Tools uses.
Approach
An
elasticsearchblock may now declarevia: kibanainstead of a url:Because the proxy returns the Elasticsearch response body verbatim and reports the true
Elasticsearch status in a header, this fits entirely behind the transport contract: every
generated ES command and the
bulk/scroll/msearch/watchhelpers work unchanged.There is deliberately no search-specific code path.
EsTransportis extracted intosrc/lib/es-transport.tsso a transport can depend on thecontract without importing the direct client (avoiding an import cycle).
es-client.tsre-exports it, so existing imports are untouched. The six consuming sites already named
their variable
transport, making this a type-only change.EsConsoleProxyClientfolds the ES path and querystring into the proxy'spathparameter, sends the required
x-elastic-internal-originheader, and reconstructs thereal ES status from
x-console-proxy-status-code.EsConnectionError(the ES call never happened), while a proxied 4xx/5xx becomes the usual
EsResponseError, so status codes,--jsonpayloads, and exit codes are identical to adirect connection.
elastic statusreports the route (green (5 nodes) via Kibana) and probes the clusterthrough the proxy, reusing the same URL/header builders so the two cannot drift.
--es-viais added toconfig context add/editvia the existingPLAIN_FIELDStable.Config validation
viais mutually exclusive with bothurlandauth, and requires akibanablock in thesame context. These are enforced rather than ignored on purpose: config objects strip
unknown keys rather than failing, so
{ via: kibana, url: … }would otherwise silentlydrop one of the two.
Notable finding
Both Kibana versions reject the Console proxy unless the caller sends
x-elastic-internal-origin: Kibana, and the rejection is misleading:That reads as "the route is disabled on this deployment" when the header is simply missing.
The error path in this PR adds an explicit hint for it, and the reasoning is captured in
comments so it is not lost.
Verification
npm test(build + unit + license),npm run test:lint,scripts/check-spdxandnpm run test:noticeall pass. Coverage is 96.6% lines / 90.4% branches / 93.5% functionsoverall;
es-console-proxy-client.tsis at 100% lines / 100% functions.Tested end-to-end against Kibana 9.2.1 and 8.18.2 with a genuinely firewalled cluster
(direct
:9200unreachable):status✓ green (5 nodes) via Kibanaes cluster healthes cat indicessecurity_exceptionsurfaced verbatimes search --input-filehits.totalto querying via Kibana directlytransport_error,status_code: 404, exit 1--dry-runUnit tests cover adversarial paths (spaces,
*,../,?,#, empty), both auth types,NDJSON bulk bodies,
HEADsemantics, the header-absent fallback, andRequestInit(
method,redirect: 'error', headers).Open questions (from #488)
via: kibanathe shape you want, or would you prefer it nested (e.g. under atransport:key), orz.enumto leave room for future routes?ELASTIC_ES_*to extensions? This PR omits them,since there is no Elasticsearch endpoint to pass on.
test/functional/es/is codegen output today andtest/functional/kb/is hand-written; a test here needs a live Kibana.Happy to reshape any of the above.