Skip to content

Commit 999212f

Browse files
authored
Release GNN statistics before the next message-passing hop (#1038)
> Release GNN statistics after aggregation so the previous hop’s tensor does not overlap the next allocation. Model arithmetic is unchanged. > > Pretrained RelBench inference: 12 cases on an NVIDIA L4, PyTorch 2.14; all predictions bitwise identical. > > | Largest classification case | Before | After | > |---|---:|---:| > | Transient prediction peak | 223.8 MB | 160.2 MB (−28.4%) | > | Total allocated peak | 543.3 MB | 479.8 MB (−11.7%) | > > Latency was unchanged. A separate two-hop GPU Inductor check preserved old/new outputs exactly; compiled transient allocation was unchanged at 7.07 MB because the compiler already eliminated the overlap.
1 parent b386d1b commit 999212f

1 file changed

Lines changed: 2 additions & 0 deletions

File tree

‎sdm/models/kumo/relational/invariant_gnn.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,8 @@ def forward(
109109
stats.flatten(1),
110110
self.aggregation_lin.weight.T,
111111
)
112+
# Release before allocating the next hop's statistics.
113+
del stats
112114

113115
if i == num_hops - 1:
114116
start = graph.start_node_offsets[readout_table]

0 commit comments

Comments
 (0)