Skip to content

Reactive-limit outer loop, new physical checks, continuous batching (+ its adjoint) - #21

Open
BDonnot wants to merge 11 commits into
mainfrom
ccr-84198c79-i1fcdj
Open

Reactive-limit outer loop, new physical checks, continuous batching (+ its adjoint)#21
BDonnot wants to merge 11 commits into
mainfrom
ccr-84198c79-i1fcdj

Conversation

@BDonnot

@BDonnot BDonnot commented Oct 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Six feature commits on top of 5eea0c5 (the same as dev_outerloops), plus a merge of main, two test fixes, a regression test and a benchmark fix. Labelled gpu-ci, so the Modal GPU suite (build.yml → gpu-tests) runs on this PR.

What it adds

  • Reactive-limit outer loop (ContingencyAnalysisGPU(reactive_limits_outer_loop=True), opt-in). This is one pass of OpenLoadFlow's ReactiveLimits. Every contingency whose physical checks flag a switch is re-solved once, with the switch applied, in a second, smaller ScenarioSweepGPU batch. The switches it handles:

    • PV→PQ at the summed limit.
    • Release of a local frozen regulator back to PV.
    • A VoltageControl group of generators that saturates in full: each controller is held at its limit.
    • Release of a frozen remote regulator, through lightsim2grid's held controllers.

    The second pass starts from the first-pass voltages (outer_loop_warm_start). get_outer_loop_status() reports what happened to each contingency.

  • Remote voltage control check: LOW_VOLTAGE_REMOTE_CONTROL / HIGH_VOLTAGE_REMOTE_CONTROL on GENERATOR, ported from lightsim2grid's RemoteVoltageControlCheck.hpp and routed through the release plan.

  • Overshoot of a capped slack unit in the redistribute_slack pre-pass. Port of lightsim2grid's distribute_with_overshoot.

  • HVDC frozen at its AC-emulation limit: the HVDC_AC_EMULATION_RELEASE record, raised when the droop asks for less than the frozen value.

  • Continuous batching (scheduling="continuous") on the three batch facades. A row leaves its slot as soon as it has converged, diverged or used up its nb_iter budget, and the slot is refilled from the queue. Each row's outcome is in get_row_status() / get_row_iterations(). Chunk mode stays numerically identical to the former code.

  • Adjoint through a continuous run (BatchPowerFlow.from_lsgrid(scheduling="continuous")). The backward rebuilds each row's J from its converged V, S rows at a time.

See each commit message for details and validation figures.

Merge of main (3e5d0b1)

16 files conflicted. The merge commit message lists each resolution. The ones that change behaviour:

  • BatchDriverConfig now carries scheduling. With main's driver persistence:
    • In continuous mode, all three sessions take the cold path, as ScenarioSweepSession already did.
    • A reused driver gets nb_iter_per_round and tol patched in, like nb_iter.
  • InjectionBatch::set_rows (main's hot path) now goes through the same gen_v upload as initialize. This fixes a bug present on main: a reused injection-sweep driver kept the map from gen_v columns to VoltageControl groups that it built the first time.
    • A remote regulator's column added afterwards was ignored.
    • A new set of columns was mapped with the old map: values could land in the wrong group, or be read past the end of the map.
    • Regression test in a3dc9d3, see below.
  • solve_JT_batch: main's single-chunk guard applies to a chunked forward only.
  • Held controllers: they are wired in make_single_system_buffers, so AcPfGPU.resolve() pins them too and restarts them at their frozen output.

Test fixes after the first GPU runs (88601b5, 44f11ce)

  • test_failed_forward_invalidates_a_pending_alias_backward (from main). This branch's BatchPowerFlow sets the capacity back to n_scen on every chunked forward, so the test can no longer force a multi-chunk forward through batch_size. The failing run now goes to the session directly, with keep_final_jacobian still on. It checks the same run_counter path.

  • test_remote_voltage_control_violations. Two changes in lightsim2grid master moved its references:

    • Realistic-voltage range. init_from_pypowsybl now reads the installed OpenLoadFlow's defaults (0.51 / 1.96 pu after the margin), so the test sets the former OLF range on the grid explicitly. That range only feeds the checks, never the power flow.
    • Unconverged batch rows. With B6-G regulating remotely, lightsim2grid master's ContingencyAnalysisCPP leaves 6 of IEEE 14's 20 N-1 rows unconverged after one iteration. The grid stays connected, and lightsim2grid's single solve of each one converges. Every GPU row is now also checked against that single solve, and against ContingencyAnalysisCPP where it converged. The GPU record on the failing row equals the single solve's (14.430316 kV).

    That ContingencyAnalysisCPP behaviour is reported upstream as ContingencyAnalysisCPP: connected N-1 rows fail after 1 iteration with a remote voltage regulator (depends on nb_thread) lightsim2grid#240. Which rows fail there depends on nb_thread.

Regression test and benchmark fix (a3dc9d3, d1a7391)

  • test_is_hot_gen_v_drives_remote_regulators (test_batch_driver_reuse.py) uses case14 with gen 3 regulating bus 9. It runs hot set_gen_v updates on a reused InjectionSweepGPU driver:

    • a remote column brought in after a first build without gen_v;
    • then more columns, then none of the group's.

    Each step must keep the driver, match a fresh session, and hold |V| at the regulated bus on the row's target.

  • benchmarks/contingency_analysis.py stopped with a NameError (ac_Ybus) on any grid with more branches than nb_max_cont_cpu. The GPU-only contingencies are now drawn from the branches the CPU does not simulate, so none runs twice.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NHgKiD6QFuLy63bRfEV9wW

BDonnot added 6 commits October 1, 2026 13:46
@BDonnot
Add an opt-in reactive-limit outer loop to ContingencyAnalysisGPU
7acb18c 
reactive_limits_outer_loop (off by default) runs one pass of OpenLoadFlow's
ReactiveLimits outer loop: after the batch, the contingencies whose physical
checks report a switch the loop handles are re-solved once, with the
switches applied, in a second, smaller batch -- a ScenarioSweepGPU built
lazily from the same grid, one row per flagged contingency. Their voltages,
residuals and violations replace the first pass'; get_outer_loop_status()
says what happened to each contingency (NO_SWITCH, RECOMPUTED, UNSUPPORTED,
LEFT_OUT by the batch rule on a small last chunk, DIVERGED).

Handled switches:
- LOW_Q / HIGH_Q on a bus its own machines hold: PQ at the summed limit
  (ScenarioSweepSession::set_pv_pq_switches, the bus joining the reserved
  switchable set on that row only);
- the release of a machine frozen at a reactive limit regulating its own
  bus: PV again at its target (Q row pinned, |V| re-seeded);
- LOW_Q / HIGH_Q on every controller bus of a voltage-control group of
  generators: each controller held at its limit, by value, through the
  group's voltage row and sharing rows (set_vc_controller_pins);
- the release of a frozen REMOTE regulator, through lightsim2grid's held
  controllers (set_hold_frozen_regulators): the second pass is built from a
  held copy of the grid, every row pins the held controllers at their frozen
  output (offset out of the controller mismatch) and a row releases one
  (set_vc_controller_releases). Stranding is now decided per row on the
  controllers the row leaves free.
Anything else makes the contingency UNSUPPORTED. The second pass checks with
the new labels through per-row skip masks of the reactive and release plans.

The second pass starts each contingency from its first-pass voltages
(outer_loop_warm_start, default on; ScenarioSweepSession::set_v_init_from_ptr,
a device-to-device copy, a masked bus starting from the base case), and
outer_loop_nb_iter sets its Newton iterations.

InjectionSweepSession refuses a grid holding frozen regulators. Pinned
against one-off lightsim2grid solves of the grid rebuilt with the switches,
and against lightsim2grid's own analysis of a grid whose base-case slack it
capped (LSGrid.cap_slack_at_active_limits).

Assisted-by: Claude Code (claude-opus-5-5)
Signed-off-by: Benjamin Donnot <benjamin.donnot@rte-france.com>
@BDonnot
Report a remote voltage controller at an unrealistic voltage
4a0593e 
Port lightsim2grid's remote voltage control check
(RemoteVoltageControlCheck.hpp) to the batch sessions: a generator
regulating a remote bus whose own bus leaves the realistic range set with
LSGrid.set_remote_voltage_control_vm_range, which OpenLoadFlow's robust
remote voltage control switches to PQ, is reported as
LOW_VOLTAGE_REMOTE_CONTROL / HIGH_VOLTAGE_REMOTE_CONTROL (13 / 14) on the
GENERATOR, value and limit in kV.

Routed through the PQ -> PV release plan, like the standby SVC check: its
`standby` column becomes the entry's check code (0 release, 1 standby SVC,
2 remote control). The bridge appends, from lightsim2grid's own
build_remote_voltage_control_plan (behind __has_include), one GENERATOR
entry per finite bound with the generator's OWN bus as reg_bus_solver (the
bus checked) and the remote one as gen_bus_solver (only its mask). The
kernel masks them by a generator contingency, never moves them by a row's
targets, and gives them two groups of their own
(N_GEN_PV_RELEASE_VIOLATION_GROUPS 4 -> 6). The reactive-limit outer loop
maps generators to release entries only, and treats a row carrying such a
record as another loop's (unsupported), as for a standby SVC.

Tested in test_remote_voltage_control_violations.py against lightsim2grid's
single solve and ContingencyAnalysisCPP, N case and rows.

Assisted-by: Claude Code (claude-opus-5-5)
Signed-off-by: Benjamin Donnot <benjamin.donnot@rte-france.com>
@BDonnot
Keep a unit capped beyond its limit until the slack shift uses it up
d2c26b0 
Port of lightsim2grid's overshoot of a "can participate in the slack" unit
(LSGrid::set_gen_can_participate_slack_overshoot): OpenLoadFlow shares the
slack from the raw set-points, so a unit it capped well beyond its max_p
only leaves it once the common shift of a later imbalance has used that
up.

- SlackRedistributionData carries an optional overshoot_mw per unit (empty:
  all 0), filled by the bridge from GenInfo / StorageInfo
  can_participate_slack_overshoot_mw for a pre-pass-only unit (behind
  LS2G_HAS_CAN_PARTICIPATE_SLACK_OVERSHOOT), and in array mode from the
  tuple's optional last entry.
- slack_redistribution::distribute dispatches, when a unit carries one, to
  the port of lightsim2grid's distribute_with_overshoot (one common shift,
  p_k = clamp(v_k + shift * w_k), bisection); the round-based algorithm is
  unchanged otherwise.

Tested in test_can_participate_slack.py: the data carries the overshoot and
the GPU contingency analysis gives lightsim2grid's voltages.

Assisted-by: Claude Code (claude-opus-5-5)
Signed-off-by: Benjamin Donnot <benjamin.donnot@rte-france.com>
@BDonnot
Report an hvdc line frozen at its AC-emulation limit when its droop a…
4efbbe7 
…sks for less

The bridge reads lightsim2grid's frozen lines (set_hvdc_ac_emulation_frozen)
into the base state; check_hvdc_p_violations_kernel checks them in a second
group (HVDC_AC_EMULATION_RELEASE), each hvdc record now carrying its type.

Assisted-by: Claude Code (claude-opus-5-5)
Signed-off-by: Benjamin Donnot <benjamin.donnot@rte-france.com>
@BDonnot
Add continuous batching to the batch solves (scheduling="continuous")
063b78e 
Chunked, every row runs nb_iter iterations and pays for the hardest one.
scheduling="continuous" keeps batch_size slots: every nb_iter_per_round
iterations (default 1) each row is checked, and one that converged
(||F||inf < tol, per unit), diverged or used its nb_iter budget leaves at
once, its slot refilled from the queue. Part A of
docs/dev_notes/continuous_batching.md, on the three batch facades.

Both schedules go through one load path: the sources' per-row streams
(Ybus patches, masks, vm_reseed, dP) are kept one segment per active row
and gathered into slot-tagged streams for the slots being loaded, so the
per-entry kernels are unchanged; the post-solve kernels take each slot's
active row. Chunk mode is numerically the former code (golden dumps of the
whole suite; voltages within 3e-12 on four French snapshots). Both report
each row's status and iteration count (RowStatus).

Four French snapshots, 5000 N-1 each: 13.2 s continuous against 16.1 s
chunked at nb_iter 4 and 31.8 s at nb_iter 8, voltages within 8e-9.

Assisted-by: Claude Code (claude-opus-5-5)
Signed-off-by: Benjamin Donnot <benjamin.donnot@rte-france.com>
@BDonnot
Differentiate through a continuous batch run (BatchPowerFlow)
e0f1ad1 
A continuous run has no buffer holding every row's Jacobian: rows leave
their slot at different rounds. Instead of storing each row's J, patched
Ybus and V at eviction (n_rows x (nnz_J + 2 nnz_Y), the dev note's plan),
the backward rebuilds them: a row's J depends only on its converged V,
its patched Ybus, its slack weights and its masked / pinned rows, and the
run already keeps V while the batch source holds the rest.

BatchPfDriver::_solve_JT_rebuilt takes the active rows S at a time,
reloads them through the source's own load_slots, overlays their
converged V (finite entries: a masked bus never left its loaded value),
refills J with nr_fill_J_at_current_V and factorizes / solves J^T chunk
by chunk. The forward does nothing extra and no memory is kept beyond V.
keep_final_jacobian has no effect there; J / Ybus snapshots are refused.

BatchPowerFlow.from_lsgrid gains scheduling, batch_size,
nb_iter_per_round and tol; its snapshot in continuous mode is V alone.
Every continuous forward is still cold (A.6 item 2), so a backward must
precede the next forward and gradcheck cannot run there.

Gradients equal the chunked ones (S = 1, 3, n; trips, gen_v, generator
contingencies, masked and dropped islands, redistribute_slack) and the
central differences. case1354pegase, 1024 rows: same gradients to 5e-15,
backward 148 ms against 119 ms chunked (13 ms of it rebuilding J, most of
the rest the J^T analysis a cold forward makes the backward repeat).

Assisted-by: Claude Code (claude-opus-5-5)
Signed-off-by: Benjamin Donnot <benjamin.donnot@rte-france.com>
@BDonnot BDonnot added the gpu-ci Run the GPU test suite on Modal (build.yml gpu-tests job) label Oct 14, 2026 — with Claude
@BDonnot
Merge main into ccr-84198c79-i1fcdj
3e5d0b1 
Brings in the CI (build.yml, the Modal GPU job), the driver persistence of
the contingency analysis / injection sweep sessions, AcPfGPU.resolve() and
the outer-loop detection review fixes. Resolutions:

- BatchDriverConfig (now shared) carries the branch's `scheduling`; the
  three sessions take the cold path in continuous mode, as
  ScenarioSweepSession already did, and patch nb_iter_per_round / tol on a
  reused driver like nb_iter.
- ContingencyAnalysisSession::run(): main's hot / cold split, the held
  controller pins and the continuous slot count in the cold branch.
- InjectionBatch::set_rows() goes through the same gen_v upload as
  initialize() (_upload_gen_v): it also refreshes the VoltageControl
  columns and the per-slot scratch the branch's load_slots() reads, and
  clears the columns when the new override drives none.
- solve_JT_batch: main's single-chunk guard applies to a chunked forward
  only; a continuous one rebuilds J S rows at a time.
- make_single_system_buffers() wires the held controllers (ctor and
  resolve()); resolve() restarts their Q at the frozen output, as the ctor.
- physical_checks::before_solve: main's PhysicalSetupKey path, with the
  per-row skip masks synced next to the targets; the scenario sweep clears
  their dirty flags only on a run with the checks on.
- The three violation codes: both sides agree (13 / 14 / 15); the branch
  computes them, so the "not computed yet" notes go.

Assisted-by: Claude Code
Claude-Session: https://claude.ai/code/session_01NHgKiD6QFuLy63bRfEV9wW
Signed-off-by: Benjamin Donnot <benjamin.donnot@rte-france.com>
@BDonnot
Fix the two GPU tests the merge with main left red
88601b5 
- test_remote_voltage_control_violations: init_from_pypowsybl now reads the
  installed OpenLoadFlow's realistic-voltage defaults (0.5 / 2.0 pu with
  recent ones, 0.51 / 1.96 after the margin) instead of its 0.8 / 1.2
  fallback, so B6-G at 14.2 kV (own bus 1.20 pu) no longer left the range
  and lightsim2grid's own reference reported nothing. The test now sets
  the former OLF range on the grid itself (it only feeds the checks, never
  the powerflow); lightsim2grid master then gives the 0 / 1 records again.
- test_failed_forward_invalidates_a_pending_alias_backward (from main):
  this branch's BatchPowerFlow sets the capacity back to n_scen on every
  chunked forward, so a forward can no longer be forced into several
  chunks through batch_size. The failing run now goes to the session
  directly, keep_final_jacobian still on: same run_counter path.

Assisted-by: Claude Code
Claude-Session: https://claude.ai/code/session_01NHgKiD6QFuLy63bRfEV9wW
Signed-off-by: Benjamin Donnot <benjamin.donnot@rte-france.com>
@BDonnot
Check the remote control records against lightsim2grid's single solve…
44f11ce 
…s too

lightsim2grid master's ContingencyAnalysisCPP leaves 6 of IEEE 14's 20 N-1
rows unconverged after one iteration once B6-G regulates a remote bus,
whatever the start and the init mode, although the grid stays connected
and lightsim2grid's own single solve of each converges (on plain IEEE 14
every row converges). Branch 5 is one of them, so its reference row was
empty while the GPU reported the record that single solve gives
(14.430316 kV, equal to every printed digit).

Every row is now compared with lightsim2grid's single solve of its
contingency, and with ContingencyAnalysisCPP where that converged.

Assisted-by: Claude Code
Claude-Session: https://claude.ai/code/session_01NHgKiD6QFuLy63bRfEV9wW
Signed-off-by: Benjamin Donnot <benjamin.donnot@rte-france.com>
@BDonnot BDonnot mentioned this pull request Oct 14, 2026
Open
@BDonnot
Pin InjectionSweepGPU's gen_v hot path on a remote regulator
a3dc9d3 
The fix came with the merge of main (InjectionBatch::set_rows going
through _upload_gen_v): main's set_rows re-uploaded the gen_v values of a
reused driver but not the map from each active column to the
VoltageControl group it drives, nor the per-slot scratch, both built once
by initialize(). So a hot run
- after a first build without gen_v ignored a remote regulator's column
  (the group kept the grid's v_set, |V| at the regulated bus with it);
- with another set of active columns applied the old map to them: a
  value written into the wrong group, and past the end of the map when
  the new override had more columns.

test_is_hot_gen_v_drives_remote_regulators covers both on case14 with
gen 3 regulating bus 9: each hot run must take the reused driver, match
a fresh session, and hold |V| at the regulated bus on its row's target.

Assisted-by: Claude Code
Claude-Session: https://claude.ai/code/session_01NHgKiD6QFuLy63bRfEV9wW
Signed-off-by: Benjamin Donnot <benjamin.donnot@rte-france.com>
@BDonnot
Fix the contingency-analysis benchmark on grids with many branches
d1a7391 
On any grid with more branches than nb_max_cont_cpu (1000 by default)
the contingency selection read an undefined `ac_Ybus` and stopped with a
NameError. The GPU-only contingencies are now the branches the CPU does
not simulate: all of them when they fit in the GPU budget, a random
subset otherwise (it used to draw from every branch, the CPU ones
included, so some contingencies ran twice).

Checked on case1354pegase (1991 branches) with 200 CPU / 500 GPU
contingencies: the old script stops at the NameError, the new one selects
500 distinct contingencies and runs the CPU reference.

Assisted-by: Claude Code
Claude-Session: https://claude.ai/code/session_01NHgKiD6QFuLy63bRfEV9wW
Signed-off-by: Benjamin Donnot <benjamin.donnot@rte-france.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

gpu-ci Run the GPU test suite on Modal (build.yml gpu-tests job)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@BDonnot