Skip to content

Modify the config.py to allow using env vars#2109

Open
Leaf-Lin wants to merge 1 commit into
elastic:masterfrom
Leaf-Lin:feature/allow_env_var_in_rally_ini
Open

Modify the config.py to allow using env vars#2109
Leaf-Lin wants to merge 1 commit into
elastic:masterfrom
Leaf-Lin:feature/allow_env_var_in_rally_ini

Conversation

@Leaf-Lin

Copy link
Copy Markdown

A small change to allow the datastore.password and any other values from rally.ini to take from environmental variable.

@Leaf-Lin Leaf-Lin requested a review from a team as a code owner April 24, 2026 10:04
Copilot AI review requested due to automatic review settings April 24, 2026 10:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates Rally’s configuration loading to support environment-variable-based substitution in rally.ini, enabling values like datastore.password to be sourced from env vars at load time.

Changes:

  • Switches config template substitution to include all os.environ variables.
  • Updates imports to use os directly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread esrally/config.py
with open(self.location, encoding="utf-8") as src:
contents = src.read()
contents = Template(contents).substitute(CONFIG_DIR=self.config_dir)
contents = Template(contents).safe_substitute({"CONFIG_DIR": self.config_dir, **os.environ})
Comment thread esrally/config.py
with open(self.location, encoding="utf-8") as src:
contents = src.read()
contents = Template(contents).substitute(CONFIG_DIR=self.config_dir)
contents = Template(contents).safe_substitute({"CONFIG_DIR": self.config_dir, **os.environ})
Comment thread esrally/config.py
Comment on lines +64 to 65
contents = Template(contents).safe_substitute({"CONFIG_DIR": self.config_dir, **os.environ})
config.read_string(contents, source=self.location)
@pquentin

Copy link
Copy Markdown
Member

Hey @Leaf-Lin, is #1674 enough for your needs?

@Leaf-Lin

Leaf-Lin commented Apr 24, 2026 via email

Copy link
Copy Markdown
Author

@pquentin

Copy link
Copy Markdown
Member

buildkite test this please

(sorry, using this PR to test an unrelated webhook change)

@pquentin

Copy link
Copy Markdown
Member

buildkite test this please

@gbanasiak gbanasiak left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this logical extension.

  • Any specific reason for switching from substitute() to safe_substitute()? I think throwing an error if any of the identifiers is missing makes sense, so I would keep substitute().
  • Regarding the order, I think CONFIG_DIR followed by env vars makes sense, i.e. we want to be able to override the defaults.
  • Existing Rally env vars begin with RALLY_ (e.g. RALLY_HOME or the mentioned RALLY_REPORTING_DATASTORE_PASSWORD). This change surfaces CONFIG_DIR in a sense. To keep some consistency I would rename CONFIG_DIR to RALLY_CONFIG_DIR (please update esrally/resources/rally.ini).
  • Please update docs/configuration.rst mentioning the new feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants