Support rolling cluster ID migration - #6852
Conversation
guilload
left a comment
guilload
left a comment
There was a problem hiding this comment.
Approved but would be good to fix the nits.
| #[derive(Clone)] | ||
| pub struct Cluster { | ||
| cluster_id: String, | ||
| extra_cluster_ids: Vec<String>, |
There was a problem hiding this comment.
| extra_cluster_ids: Vec<String>, | |
| extra_cluster_ids: HashSet<String>, |
| &self.cluster_id | ||
| } | ||
|
|
||
| pub(crate) fn accepts_cluster_id(&self, cluster_id: &str) -> bool { |
There was a problem hiding this comment.
This is used once as !accepts_cluster_id so we might as well write:
fn rejects_cluster_id(&self, cluster_id: &str) {
self.cluster_id != cluster_id && !self.extra_cluster_ids.contains(cluster_id)
}| } | ||
|
|
||
| #[cfg(any(test, feature = "testsuite"))] | ||
| #[allow(clippy::too_many_arguments)] |
| fn validate(node_config: &NodeConfig) -> anyhow::Result<()> { | ||
| validate_identifier("cluster", &node_config.cluster_id)?; | ||
| for cluster_id in &node_config.extra_cluster_ids { | ||
| validate_identifier("cluster", cluster_id)?; |
There was a problem hiding this comment.
| validate_identifier("cluster", cluster_id)?; | |
| validate_identifier("extra cluster", cluster_id)?; |
| QW_PEER_SEEDS, | ||
| QW_REST_LISTEN_PORT, | ||
| QW_DISABLE_DOCS_CLUSTERING | ||
| QW_DISABLE_DOCS_CLUSTERING, |
There was a problem hiding this comment.
Keep env vars sorted (including QW_DISABLE_DOCS_CLUSTERING).
| } | ||
|
|
||
| #[cfg(any(test, feature = "testsuite"))] | ||
| impl<'a> ClusterTestBuilder<'a> { |
| } | ||
|
|
||
| #[cfg(any(test, feature = "testsuite"))] | ||
| impl<'a> ClusterTestBuilder<'a> { |
There was a problem hiding this comment.
TestClusterBuilder
(regular code suggestion won't work on this line for some reason...)
1c30409 to
ad37a2f
Compare
ad37a2f to
17150fc
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ad37a2fecf
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| | --- | --- | --- | --- | | ||
| | `version` | Config file version. `0.7` is the only available value with a retro compatibility on `0.5` and `0.4`. | | | | ||
| | `cluster_id` | Unique identifier of the cluster the node will be joining. Clusters sharing the same network should use distinct cluster IDs.| `QW_CLUSTER_ID` | `quickwit-default-cluster` | | ||
| | `extra_cluster_ids` | Allows nodes to communicate with peers using another cluster ID. This can be used to update `cluster_id` without downtime. The environment variable accepts a comma-separated list of IDs. | `QW_EXTRA_CLUSTER_IDS` | `[]` | |
There was a problem hiding this comment.
A rolling deployment that changes a node to cluster_id: new with extra_cluster_ids: [old] while its peers still have only the old configuration is asymmetric: those old peers have not accepted new, so they reject the new node's cluster-state requests (rejects_cluster_id checks the receiver's extra IDs) and do not join its new-ID gossip. This partitions membership/control-plane visibility during the rollout, contrary to the claimed no-downtime migration; document that all nodes must first be rolled with cluster_id: old and extra_cluster_ids: [new], then switch primary IDs while retaining the old ID temporarily.
AGENTS.md reference: AGENTS.md:L23-L24
Useful? React with 👍 / 👎.
Expose extra_cluster_ids and QW_EXTRA_CLUSTER_IDS, use native Chitchat support, and allow configured IDs during gRPC cluster-state exchange. Pin the merged Chitchat HashSet change and cover configuration, membership, ingestion, and search. Implemented with OpenAI gpt-6-astra.
17150fc to
a6f6a32
Compare
2 participants
Summary
extra_cluster_idsandQW_EXTRA_CLUSTER_IDS.a3a34b721053b0353fa5ce1890a2800794976eabValidation
AI assistance
Implemented with OpenAI
gpt-6-astra.