perf: DH-23096: Multi Column Sort Experiments#8236
Open
cpwright wants to merge 14 commits into
Open
Conversation
Contributor
No docs changes detected for 30a072b |
Add GenerateTimsortKernels (./gradlew generateTimsortKernels), a JavaPoet generator that emits all 70 timsort kernels into io.deephaven.engine.table.impl.sort.timsort2, functionally identical to the replicated kernels in sort.timsort and implementing the same Long/Int/ByteSortKernel interfaces. Kernels are modeled as a list of key columns plus an optional permutation payload column, so the generator can be extended to sort multiple key columns of any type simultaneously. Copy the existing timsort test suite to a timsort2 test package (96/96 pass) and the JMH sort kernel benchmarks to a timsort2 benchmark package, with a SortKernelComparisonBenchmark runner (sortKernelComparisonBenchmark task) that compares the two implementations; results are at parity (geomean 0.996).
…ort()
Extend GenerateTimsortKernels to emit kernels that sort multiple key columns
of any type, comparing each column in turn (lazily: later columns are only
read when earlier columns tie). Three kernel families are generated into
sort.timsort2.multi:
- {Kinds}MultiColumnTimsortKernel: 8x8 ascending pairs of engine column
kinds, moving the values and row keys as the single-column kernels do.
- {Kinds}IndirectMultiColumnTimsortKernel: the same pairs, but the timsort
permutes only a parallel int position chunk; column values never move and
the permuted row keys are assembled in a single linear pass at the end.
- {Kind}IndirectTimsortKernel: single-column indirect variants.
SortHelpers tries a generated kernel (via the MultiColumnSortKernel interface
and the generated dispatchers) for multi-column sorts when
QueryTable.useMultiColumnSortKernel is set, choosing the indirect variant
when QueryTable.useIndirectMultiColumnSortKernel is also set (which likewise
covers single-column ascending sorts); comparators, data indexes, descending
columns, three or more columns, and boolean columns fall back to the existing
one-column-at-a-time pipeline. Both properties default to false.
TestMultiColumnSort verifies all paths produce identical stable permutations
across every column-type pair, with nulls, duplicates, and floating point
special values. MultiColumnSortBenchmark (engine-table jmh source set)
compares the pipeline and both kernel families: the indirect kernel is
fastest in 16 of 18 two-column cells (geomean 0.75x pipeline, best 0.58x,
worst 1.06x); for single columns it is ~1.2-1.7x faster on Object columns
but 1.2-1.4x slower on mid/high-cardinality primitives, where the direct
kernels' sequential merge reads win.
…by default Add MultiColumnTimsortKernelFactory (engine-table main, following the typed aggregation hasher pattern): it owns the indirect kernel emitter and the dispatch policy. Sorts of Object columns (either direction) and sorts of multiple columns use indirect timsort kernels that compare each column in turn while permuting only a parallel chunk of positions, assembling the permuted row keys in a single linear pass at the end. Kernels for the common shapes — a single Object column and every all-ascending pair of column types — are pregenerated by GenerateTimsortKernels through the factory's emitter; other shapes (three or more sort columns, descending multi-column components) are compiled on demand with the QueryCompiler and cached. Single-column sorts of primitive types keep the existing direct kernels, which their sequential merge reads make faster; boolean chunks and comparator/data-index sorts also use the existing path. The behavior is controlled by QueryTable.useGeneratedSortKernels (default true), replacing the two experimental properties. The experimental direct (value-moving) multi-column kernels and their emitter and dispatcher are removed. Benchmarks (4M rows): two-column sorts run at geomean 0.75x of the run-finding pipeline (best 0.58x, worst 1.05x); three-column sorts through the compiled kernels at geomean 0.78x. TestMultiColumnSort verifies flag-on and flag-off produce identical stable permutations across all type pairs, single columns in both directions, and compiled-kernel shapes including a four-column mixed-direction sort; QueryTableSortTest, MultiColumnSortTest, and TestSort pass with the new default enabled.
cpwright
force-pushed
the
nightly/cpw/multi-column-sort
branch
from
July 14, 2026 15:51
3a0ee8b to
dc4e9f6
Compare
Pregenerate only the single-column timsort kernels: the existing direct (value-moving) kernels, and indirect single-column kernels for every engine column type in both directions, selected by the generated IndirectTimsortDispatcher. The pregenerated two-column kernels are removed; MultiColumnTimsortKernelFactory now compiles every multi-column shape on demand with the QueryCompiler and caches it, so multi-column sorts never fall back to the one-column-at-a-time pipeline. The compiled kernels support any combination of sort directions and per-column Comparators (ComparatorSortColumn): a comparator column's doComparison delegates to a comparator held by the kernel context, with descending handled by reversing the comparator at context creation. Ties under a comparator are broken by the subsequent sort columns; for comparators that do not respect equality this differs from the previous pipeline, which only applied later columns within runs of equal values. Single-column comparator sorts continue to use ComparatorLongTimsortKernel. Rework the kernel emitters (both the factory's and the single-column replicator's) to use idiomatic JavaPoet throughout: beginControlFlow / nextControlFlow / addStatement / addComment with default output style, replacing the raw text blocks and named-argument templates; the labeled gallop loops and permute-conditional statements are now explicit builder branches. Benchmarks (4M rows, kernel/pipeline): two-column geomean 0.73 (best 0.58, worst 1.07), three-column geomean 0.82, single Object column 0.57-0.86; runtime-compiled kernels match the previously pregenerated classes within noise. TestMultiColumnSort covers the compiled comparator, descending, and three-plus-column shapes; QueryTableSortTest, MultiColumnSortTest, and TestSort pass with the default enabled.
…ted ones Generate the timsort kernels directly into io.deephaven.engine.table.impl.sort.timsort, replacing the text-replicated kernels (and the hand-written char sources they were replicated from) with the functionally identical JavaPoet-generated kernels; the timsort2 staging package is removed. Class names are unchanged, so kernel consumers are unaffected. The multi-column machinery — MultiColumnSortKernel factory, IndirectTimsortDispatcher, the single-column indirect kernels, and on-demand compilation of multi-column kernels — moves to sort.timsort.multi. ReplicateSortKernel no longer replicates the timsort kernels; it retains the megamerge, find-runs, partition, and permute replication, the LongSortKernel to Int/ByteSortKernel interface replication, and the radix kernels. The shared comparison-fixup helpers used by the SSA, SSM, and dup-compact replicators are kept. ReplicateSortKernelTests is unchanged and the existing timsort test suite now validates the generated kernels; the duplicate timsort2 test and benchmark copies are removed. The generated null-aware kernels name their contexts consistently (NullAwareCharLongSortKernelContext); the replication had left them with the plain-char context names, and MegaMergeTestUtils is updated accordingly.
Sorts of at least QueryTable.parallelSortMinimumSize rows (default one million; zero or negative disables parallelism) now parallelize using the operation initializer's thread pool: - SortHelpers.makeAndFillValues splits the value read across tasks, each filling its own slice of the single shared values chunk from the corresponding row positions with its own fill context. - ParallelIndirectSort splits the positions chunk into segments of at least QueryTable.parallelSortSegmentSize rows (default 256K), at most the operation initializer's parallelism factor, sorts them independently, and merges adjacent sorted runs pairwise, submitting each merge as soon as both of its inputs complete. The permuted row keys are then assembled in a parallel linear pass. MultiColumnSortKernel gains sortPositions and mergePositions primitives for this (defaults throw; the generated kernel contexts override them), and the contexts' whole-chunk buffers are now lazily allocated so the per-task contexts are pure scratch. The parallel sort covers the paths served by the indirect kernels: all multi-column sorts and single-column Object sorts. Single-column primitive sorts continue to use the direct kernels serially. The off-thread work of each scheduler is folded into the enclosing operation's performance entry; note that reading an OperationInitializerJobScheduler's accumulated performance waits for its jobs and may be consumed only once, so each phase uses its own scheduler. Benchmarks (4M rows, cardinality 100K, 24 cores): multi-column in-memory sorts speed up by a geomean of 5.3x (best 6.6x, Sym,I1 1449ms to 220ms); cold-parquet sorts by about 2x at the previous 4-segment default. The MultiColumnSortBenchmark gains parallelMinSize and storage parameters, the latter including parquet and parquet-cold (reopening the table each iteration) modes. TestMultiColumnSort verifies serial and parallel sorts produce identical results across column shapes, comparator sorts, the one-column-at-a-time pipeline, edge sizes down to two rows (which also fixed the fill segment count to never exceed the fill size), odd merge-tree shapes, and an exact two-segment split.
Extend LongSortKernel (and its Int and Byte replicas) with ranged sort and adjacent-run merge operations, with defaults that throw for kernels that do not support them; the generated permute-kernel contexts implement them by exposing the kernels' existing timSort and merge internals. This covers every direct kernel, including ComparatorLongTimsortKernel, so single-column comparator sorts parallelize as well; the boolean radix kernel keeps the defaults and continues to sort serially. Rename ParallelIndirectSort to ParallelTimsort and generalize it: a shared pairwise merge tree drives both sortIndirect (positions chunk plus a final parallel row-key gather, for the multi-column kernels) and sortDirect (which sorts the values and row-key chunks in place with the direct kernels, no positions or gather). Single-column sorts of primitive and comparator columns that meet the parallel threshold now use sortDirect with a per-task kernel context; Object columns continue through the indirect kernels, which parallelize internally. Sorting segments with the direct kernels rather than routing primitives through the indirect kernels keeps each column shape on its fastest kernel: benchmarks (4M rows, 24 cores) show single-column speedups of 5.9x to 7.2x (int 384ms to 65ms, double 507ms to 77ms, String 1234ms to 171ms), with the int case about 35% faster than the indirect route would have provided.
…poisoned contexts The sort listener of a refreshing table runs on an update graph thread, whose ExecutionContext has no QueryCompiler. For a blink table the initial sort is empty and never touches a kernel, so the listener was the first to need the multi-column kernel and the on-demand compile failed with ExecutionContextRegistrationException (caught by BlinkTableOperationsTest#testSortMultipleColumns). SortOperation now resolves the kernel for its shape when it is constructed, on a thread that can compile, via MultiColumnTimsortKernelFactory.prepareKernel; the compile-and-cache step is factored out of makeCompiledContext so cycle-time sorts always hit the cache. The preparation does not consider the data index, because the listener path sorts without one. Relatedly, PoisonedOperationInitializer.canParallelize() throws rather than answering false, so a listener-cycle sort above the parallel threshold would have failed in SortHelpers' segment sizing. The fill and sort segment counts now treat a poisoned operation initializer as non-parallelizable and sort serially.
cpwright
changed the title
cpwright
added
NoDocumentationNeeded
ReleaseNotesNeeded
…ulti package to indirect Single-column Object sorts with a comparator now use the indirect kernels instead of the direct ComparatorLongTimsortKernel, so every Object sort takes the same permute-positions-only path. The comparator kernel is pregenerated (ComparatorIndirectTimsortKernel, one kernel comparing in the ascending sense; descending sorts reverse the comparator when the context is created) and IndirectTimsortDispatcher takes the comparators array and selects it, keeping the invariant that every single-column shape is checked-in generated code and only multi-column shapes compile on demand. Rename the io.deephaven.engine.table.impl.sort.timsort.multi package to sort.timsort.indirect, and MultiColumnTimsortKernelFactory to IndirectTimsortKernelFactory, to describe the kernels rather than one of their uses. The generator's package constant follows the rename (an earlier sed missed the concatenated ".multi" literal, which had briefly regenerated the old directory), and runtime-compiled kernels now land under sort.timsort.indirect.gen.
…rnel generators Add the boolean Configuration property QueryTable.parallelSort (default true) as an explicit switch for sort parallelization; when false every sort runs on the calling thread, regardless of the size thresholds. Replace the text-based javapoet ClassName constants in GenerateTimsortKernels and IndirectTimsortKernelFactory with references to the actual classes (e.g. ClassName.get(IntChunk.class)), including the name-concatenated cases: ChunkFamily and ColumnType take their chunk and comparison classes as constructor arguments, and the permute sort kernel interfaces resolve through a ChunkFamily mapping. References to generated kernel classes remain name-based, since the generator must run before they exist. The replicator needs util-annotations at runtime now that it references VisibleForTesting.class. Regenerating every kernel with the class-based generator reproduces the checked-in files exactly. Fix TestMultiColumnSort to disable query memoization: each A/B check sorts the same table with the same columns twice, so with memoization on the second sort returned the first result and the comparisons were vacuous. The suite passes with the comparisons live. Also pin that the parallelSort switch produces correct results when the size thresholds would otherwise parallelize, and format the ColumnType enum one constant per line.
Sorts initiated from listeners on update graph threads block on a future, so the sort, fill, and gather tasks must not run on the update graph's own threads: concurrent sorts could block every update thread, and the tasks that would complete their futures could then never be scheduled. Comment the scheduler choice at each creation site, including why the operation initializer pool is deadlock-free (its workers decline canParallelize, so worker-initiated sorts run serially, and the merge tree's tasks never block) and that a continuation-based sort listener would be the better long-term structure.
…rt tests Sort operations now resolve (compiling on demand) their multi-column sort kernel when they are constructed, which requires an ExecutionContext with a QueryCompiler. PyIcebergTestUtils sorted its EXPECTED_DATA table in a static initializer with no context, and once that initializer threw, the poisoned class failed every test that referenced it — including tests that had opened a context of their own. The initializer now builds the table under TestExecutionContext.createForUnitTests(). PyIceberg2Test, PyIceberg3aTest, and PyIceberg4bTest also sort in their test bodies without an engine context, passing only when another test had already populated the kernel cache; they now use the same EngineCleanup setup as their sibling tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.