Skip to content

out_opensearch,in_opensearch: rebuild the client when only the port, scheme or path of a host changes - #202

Merged
Watson1978 merged 1 commit into
fluent:mainfrom
lbarrosby:fix-reuse-client-port-scheme
Nov 10, 2026
Merged

Watson1978 merged 1 commit into
fluent:mainfrom
lbarrosby:fix-reuse-client-port-scheme

Conversation

@lbarrosby

Copy link
Copy Markdown
Contributor

is_existing_connection (in both out_opensearch and in_opensearch) is meant to drop the cached client when the hosts change, but the port check never runs:

if !host[i][:host].eql? @current_config[i][:host] || host[i][:port] != @current_config[i][:port]

Without parentheses this parses as host.eql?(current_host || port_changed), so only the host name is compared. When a host placeholder expands to the same name on another port (for example hosts ${tag} with tags os.internal:9200 and os.internal:9201), the chunk is sent through the client built for the first port. A change of scheme or path is not detected either.

This change compares the whole host hash instead. The client from opensearch-ruby normalizes the hashes it receives (port.to_i, path.chomp('/')), and @current_config used to share them through a shallow clone, so it now keeps copies to compare against what get_connection_options returns next time.

Tests added:

  • test_out_opensearch.rb: test_client_is_rebuilt_when_only_a_part_other_than_the_host_name_changes, with port, scheme and path cases. The same host returns the same client, and the changed one returns a new client that points to the new port, scheme or path.
  • test_in_opensearch.rb: test_client_is_rebuilt_when_only_the_port_changes.

All four fail on main and pass with the change. bundle exec rake test: 464 tests, 0 failures, 1 error. The error is test_out_opensearch_data_stream.rb test_time_placeholder, which also fails on main here: it stubs foo_<local date> while the plugin asks for foo_<UTC date>, and they differ near midnight in my time zone.

(check all that apply)

  • tests added
  • tests passing
  • README updated (if needed)
  • README Table of Contents updated (if needed)
  • History.md and version in gemspec are untouched
  • backward compatible

is_existing_connection compared the port inside the argument of eql?
(`a.eql? b || c` parses as `a.eql?(b || c)`), so a host that only
changed its port kept the client of the previous one. A host
placeholder that expands to the same name on another port sent the
chunk to the old port. Compare the whole host hash instead, and keep
copies of the hashes because the client normalizes the ones it gets.

Signed-off-by: Lucas Barros <lucasbarros.kam@gmail.com>

@Watson1978 Watson1978 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix. I checked it locally.

  • The four new tests fail on main and pass on this PR. The full test suite passes.
  • When a host placeholder expands to the same name on another port, chunks now go to the right port. On main, they all go to the first one.
  • When the host settings do not change, the same client is still reused. I tried many setups, including hosts URLs with a trailing slash, path, credentials, IPv6 and an AWS endpoint.

LGTM.

@Watson1978
Watson1978 merged commit ccd1e8a into fluent:main Nov 10, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants