Implement Orion as Hydra Sweeper

[Delaunay]

Motivation

Allow users to use Orion as a sweeper.

Oríon is an asynchronous framework for black-box function optimization.
Its purpose is to serve as a meta-optimizer for machine learning models and training, as well as a flexible experimentation platform for large scale asynchronous optimization procedures.
Core design value is the minimum disruption of a researcher’s workflow. It allows fast and efficient tuning, providing minimum simple non-intrusive (not even necessary!) helper client interface for a user’s script.

Effortless to adopt, deeply customizable

• Natively asynchronous, thus resilient and easy to parallelize
• Offers the latest established hyperparameter algorithms
• Elegant and rich search-space definitions
• Comprehensive configuration system with smart defaults
• Transparent persistence in local or remote database
• Integrate seamlessly your own hyper-optimization algorithms

Have you read the Contributing Guidelines on pull requests?

Yes

Test Plan

Unit tested

[facebook-github-bot]

Hi @Delaunay!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

[lgtm-com]

This pull request introduces 1 alert when merging 2bc3e89 into f5a447f - view on LGTM.com

new alerts:

• 1 for Unused import

[Delaunay]

CLA as signed today not sure how long it takes to propagate

[facebook-github-bot]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

[Delaunay]

Hello,

@pixelb What would be the next steps to get this merged into the main branch ?

Thanks,

[pixelb]

We're considering having a contrib/ dir to serve as a staging area for various new launchers / sweepers.
Your implementation looks good, and well integrated.
We'll do the necessary to merge it to contrib/.

Thank you for your contribution!

[bouthilx]

@pixelb Thanks for the feedback! What will be the difference of support/usage for sweepers in contrib compared to actual sweepers (Ax, Nevergrad & Optuna)? Will they be included in the documentation as well?

[jieru-hu]

hi @Delaunay thanks very much for you contribution.

We've created contrib here for plugin contributions. We've also included a README there.
Could you update this PR and move this plugin to the contrib folder?

pls let us know if you have any question on this. thanks again.

[omry]

@Delaunay, @bouthilx:

Thank you for this contribution, and apologies for the very long delay here.

At a glance, the plugin looks good and appears to have been integrated in the expected way, though I have not done a deep review. I also want to acknowledge that the previous maintainer direction in 2022 was to move this into contrib/.

Since then, Hydra maintenance capacity has been limited, and one recurring difficulty has been keeping official and in-repo plugins up to date with their upstream dependencies. Even when plugin authors do most of the implementation work, continued maintenance, testing, releases, and CI coverage still create ongoing cost for the Hydra maintainers.

For that reason, I do not think we should add new fully supported plugins to this repository by default. More generally, there is no fundamental reason Hydra plugins need to live in the Hydra repo. The main benefit is discoverability, and I think we can provide that without taking ownership of the plugin code.

If you, or someone else from the Orion team, still want to pursue this, I think the better path is:

1. Host the plugin in the Orion repo, or in a standalone repo if that is more appropriate. It can depend on Hydra as an external dependency. If Hydra does not expose everything needed for plugin testing or development, we can discuss improving that.

2. Publish it as a community package, for example hydra-contrib-orion-sweeper, so users can distinguish it from officially maintained Hydra plugins.

3. We will create a community plugins section in the Hydra website, where this plugin can be listed and documented.

Let me know what you think. To keep the PR queue actionable, I plan to close this PR in 3 months if there is no response. The offer above stands regardless; if you want to pursue the external/community-plugin path later, please comment here.