Skip to content

feat: integrate CV-space runtime, trainable channels, and optimization workflows - #154

Merged
chaoming0625 merged 20 commits into
mainfrom
integration/optim-cv-runtime
Sep 8, 2026
Merged

feat: integrate CV-space runtime, trainable channels, and optimization workflows#154
chaoming0625 merged 20 commits into
mainfrom
integration/optim-cv-runtime

Conversation

@TLDSZ

@TLDSZ TLDSZ commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Description

This PR establishes a differentiable cell optimization workflow around a CV-space density runtime. It reduces redundant mechanism state storage, introduces
Cell-owned trainable channel parameters, and adds gradient, fitting, and profiling tools. It also fixes point-local current handling and defines consistent
fixed-step current-clamp sampling.

CV-space runtime and numerical correctness

Commits: 0bf3725, f3be24a, 22c52ee

  • Store painted Channel/Ion parameters and dynamic states on the CV axis. For example, a five-CV morphology with eleven electrical points now stores
    density mechanism states as (population, 5) instead of (population, 11).
  • Preserve the point-space workspace needed by the staggered/DHS voltage solve. Point mechanisms use their corresponding point voltages, including the
    boundary-current contributions handled by that solver.
  • Update density layouts, bindings, current assembly, and recording resolution to follow the new storage model.
  • Preserve state dtype when precision settings change, preventing solver intermediates and current assembly from changing the established state
    representation.

Initial trainable channel parameter API

Commit: 3767c2d

  • Introduce schema-backed parameter bindings for IL, Na_HH1952, and K_HH1952.
  • Support direct parameters, shared scale factors, and callable parameterizations through braincell.trainable.parameter, scale, and parameterized.
  • Support row, population, cv, and all grouping while retaining the original optimizer-facing ParamState objects.
  • Add Cell-owned root management, transactional binding registration, differentiable parameter materialization, and atomic parameter value updates.
  • Preserve trainable roots across dynamic-state resets and reject ordinary writes that would overwrite an established binding.

Fixed-step current clamps and recording

Commit: 76da0ce

  • Sample each current clamp once at t + 0.5 * dt and hold the resulting current throughout the main integration step.
  • Cache both individual clamp contributions and their point-space accumulation so integration and recording consume the same applied input.
  • Add ClampView selection and recording, plus observe.clamp_current() with summed or per-clamp output.
  • Use fixed-shape recording indices for supported Cell.run() schedules under JIT and differentiation.
  • Handle piecewise current boundaries consistently, including cumulative floating-point durations.

Gradient engines and correctness experiments

Commits: 5569582, ef5c766

  • Add experimental full-state exact RTRL and BPTT tooling with a shared rollout/loss interface.
  • Include parameter-dependent initialization in forward sensitivity propagation.
  • Add prefix-gradient, directional finite-difference, and multi-CV cross-compartment sensitivity checks.
  • Separate gradient correctness experiments from performance studies and expose sensitivity, learning-signal, and gradient diagnostics.

Parameter fitting, initialization, and stimulus design

Commits: bcfe5c7, 7328850, dea5ce4, 7d78275

  • Add composable fitting components for datasets, model configuration, losses, optimizers, search, training, diagnostics, and reporting.
  • Compare RTRL and BPTT training from matched initial parameters using the same objective and update configuration.
  • Add DC-protocol datasets and hybrid parameter initialization workflows, including the optional optimization dependency.
  • Add stimulus-design and identifiability experiments covering local sensitivities, global parameter ensembles, and robust optimal experimental design.

Scaling and runtime profiling

Commits: 7ff9627, e542f13, d04188b

  • Add RTRL/BPTT scaling studies and report generation across model and rollout configurations.
  • Extend GPU trace parsing and capture tooling.
  • Add BrainCell runtime ablation benchmarks to investigate the cost of individual execution components.
  • Keep benchmark configuration and reproduction instructions alongside the experimental tools.

SWC morphology export

Commit: 88c20fd

  • Add Morphology.to_swc() for morphologies with complete point geometry.
  • Validate attachment geometry and representability before writing.
  • Support reversed child orientation, radius discontinuities, supported soma attachments, suffix handling, and atomic destination replacement.
  • Add round-trip checks against bundled morphology fixtures.

Documentation and experiment organization

Commits: 94a16b2, 17993a5, 17b82ab, 9989d01, 3126211

  • Consolidate optimization API, architecture, implementation plans, derivations, and experiment references.
  • Move and refresh the project TODO under docs/design/.
  • Organize experimental work by gradient correctness, scaling, fitting, initialization, stimulus design, and training comparison.
  • Add the one-CV comparison example, refresh clamp results, and exclude generated parameter-learning artifacts from version control.

How Has This Been Tested

Validation was performed on an isolated snapshot of this branch at 22c52ee.

Environment: Python 3.11.4, JAX 0.8.0, CPU.

The following focused regression suite passed: 144 passed.

python -m pytest -q \
  braincell/_compute/layouts_test.py \
  braincell/trainable \
  braincell/quad/_staggered_test.py \
  braincell/_multi_compartment/clamps_test.py \
  braincell/network/recording_test.py \
  braincell/io/swc/writer_test.py \
  examples/experimental/optim/gradients_test.py \
  examples/profiling/simulator_compare/braincell_ablation_test.py

These checks cover CV layouts, parameter ownership and materialization, staggered integration, clamp sampling, recording, SWC export, gradient engine
behavior, and runtime ablation tooling.

Additional checks:

  • Ruff 0.16.2 lint passed for braincell.
  • Ruff formatting checks passed for all 55 changed package Python files.
  • git diff --check passed.

Historical CPU and A100 gradient, scaling, and training measurements are documented separately in docs/design/optim/references/bptt-rtrl-experimental- results.md.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • Code follows the code style of this project.
  • Changes follow the CONTRIBUTING guidelines.
  • Update necessary documentation accordingly.
  • Lint and tests pass locally with the changes.
  • Check issues and pull requests first. You don't want to duplicate effort.

Other information

Compatibility considerations:

  • Code that directly inspects painted density mechanism runtime arrays must use the CV axis rather than the electrical-point axis.
  • Current clamps now use main-step midpoint sampling. Their recordings report the actual midpoint sampling times.
  • Differentiable Cell.run() recording requires start=0 and a duration divisible by each recording period.
  • Exact point-voltage handling described above applies to the staggered/DHS path; explicit-solver boundary input handling remains a separate follow-up.

The gradient engines, fitting workflows, and benchmark tools remain experimental implementations under examples/experimental and examples/profiling.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, we are unable to review this pull request

The GitHub API does not allow us to fetch diffs exceeding 20000 lines

@chaoming0625
chaoming0625 merged commit 46e50bc into main Sep 8, 2026
9 checks passed
@chaoming0625
chaoming0625 deleted the integration/optim-cv-runtime branch September 8, 2026 02:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants