Skip to content

fix: Compute PCA variances and OLS TSS with a two-pass scheme - #628

Open
KFilippopolitis wants to merge 1 commit into
masterfrom
claude/musing-hamilton-eddfae
Open

KFilippopolitis wants to merge 1 commit into
masterfrom
claude/musing-hamilton-eddfae

Conversation

@KFilippopolitis

Copy link
Copy Markdown
Collaborator

Summary

PCA, PCA with transformations, and the OLS total sum of squares computed variance-like quantities from aggregated raw moments:

  • PCA: variances = (sum(x^2) - n * mean^2) / (n - 1) (in FederatedPCA.fit and duplicated in the pca_with_transformations wrapper).
  • OLS: tss = sum(y^2) - 2 * mean * sum(y) + n * mean^2.

Both cancel catastrophically when the mean is large relative to the spread (e.g. values around 1e8 with unit standard deviation). This PR switches them to the same two-pass scheme already used for Pearson, describe, and the t-tests:

  1. Aggregate n and sums to get the global mean.
  2. Aggregate sums of squared deviations from that mean.

Changes

  • exaflow/algorithms/federated/decomposition/pca.py: new global_mean_std(agg_client, X) helper (round 1 sums [n, column sums] in one call, round 2 sums centered squares). FederatedPCA.fit uses it. The negative-variance clip is removed because a sum of squares cannot be negative.
  • exaflow/algorithms/exareme3/decomposition/pca_with_transformations.py: reuses the helper instead of its own raw-moment copy. The n <= 1 fallback is unchanged.
  • exaflow/algorithms/federated/linear_model/ols.py: drops the sum(y^2) aggregation and aggregates sum((y - mean_y)^2) after the residual sums.
  • Every worker issues the same aggregation calls in the same order, with no data-dependent branches. Aggregation call counts: PCA 4 to 3, wrapper standardization 3 to 2, OLS unchanged.
  • Docs: PCA.md, PCAWithTransformation.md, LinearRegression.md describe the two-round flow. federated/docs/ols.md no longer lists y'y as aggregated (the commit hook also reformatted that file's Markdown).

Tests

New large-offset regression tests (OFFSET = 1e8, 3 simulated workers via _simulate_federated_execution, rtol 1e-6):

  • test_pca.py: test_global_mean_std_large_offset, test_pca_large_offset, test_pca_with_transformations_large_offset.
  • test_ols.py: test_ols_tss_large_offset (TSS, R², F against statsmodels).

Against the old code, the PCA eigenvalue test fails (up to 158% relative error) and the OLS TSS test fails (42% relative error).

uv run pytest -q tests/standalone_tests/federated_algorithms/decomposition tests/standalone_tests/federated_algorithms/linear_model

128 passed. Other OLS/PCA consumers (two-way ANOVA, cross-validation, pipeline, linear regression wrapper) also pass.

Risk

  • No API, request, or response shape change. Results change only by becoming more accurate; differences on normally scaled data are at floating-point noise level.
  • No privacy, deployment, or proto impact.
  • Rollback: revert this commit.

🤖 Generated with Claude Code

PCA (and PCA with transformations) derived sample variances from aggregated
sum(x) and sum(x^2), and OLS derived the total sum of squares from sum(y) and
sum(y^2). Both cancel catastrophically when the mean is large relative to the
spread (e.g. values around 1e8 with unit standard deviation).

Aggregate n and sums first to get the global mean, then aggregate sums of
squared deviations from that mean. PCA and the PCA-with-transformations
wrapper share a new global_mean_std helper. Every worker issues the same
aggregation calls in the same order.

Add large-offset regression tests and describe the two-round flow in the
PCA, PCA with transformations and linear regression docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant