Skip to content

Zero-copy anticompaction via verbatim compression-chunk copy - #5015

Open
clohfink wants to merge 1 commit into
apache:cassandra-4.1from
clohfink:zero-copy-anticompaction
Open

Zero-copy anticompaction via verbatim compression-chunk copy#5015
clohfink wants to merge 1 commit into
apache:cassandra-4.1from
clohfink:zero-copy-anticompaction

Conversation

@clohfink

Copy link
Copy Markdown
Contributor

Note on PR size: 78% documentation and tests

Anticompact an sstable by splitting it copying its compression chunks verbatim and rebuilding every other component from an Index.db-only pass -- instead of rewriting every row, when its full/transient/unrepaired partitions form contiguous token runs. Interleaved ranges, which is what vnodes produce, fall back to the existing rewrite. Off by default
(zero_copy_anticompaction_enabled).

Where the filesystem can share extents (xfs -m reflink=1, btrfs) the child's Data.db is reflinked from the parent's rather than copied, so a split writes no data blocks and needs no additional disk space.

Because a verbatim copy has no CompactionController, this path retains the droppable tombstones and shadowed data a rewrite would have purged, and the children inherit the parent's per-sstable statistics rather than recomputing them. Retention only, never data loss.

A child that does not begin on a chunk boundary carries a dead prefix of unindexed bytes, so Scrubber and Verifier now seek to the first index position rather than requiring it to be zero, MmappedRegions seeds segment placement at the first chunk's offset rather than at 0, and CassandraOutgoingFile.contained measures the live span so such a child stays eligible for entire-sstable streaming.

New: ZeroCopySSTableSplitter, AntiCompactionRunPlanner, Reflink.
Config: zero_copy_anticompaction_enabled, zero_copy_split_reflink_enabled,
zero_copy_split_digest_enabled. Metric: BytesZeroCopyAnticompaction.

Anticompact an sstable by splitting it -- copying its compression chunks
verbatim and rebuilding every other component from an Index.db-only pass --
instead of rewriting every row, when its full/transient/unrepaired partitions
form contiguous token runs. Interleaved ranges, which is what vnodes produce,
fall back to the existing rewrite. Off by default
(zero_copy_anticompaction_enabled).

Where the filesystem can share extents (xfs -m reflink=1, btrfs) the child's
Data.db is reflinked from the parent's rather than copied, so a split writes no
data blocks and needs no additional disk space.

Because a verbatim copy has no CompactionController, this path retains the
droppable tombstones and shadowed data a rewrite would have purged, and the
children inherit the parent's per-sstable statistics rather than recomputing
them. Retention only, never data loss.

A child that does not begin on a chunk boundary carries a dead prefix of
unindexed bytes, so Scrubber and Verifier now seek to the first index position
rather than requiring it to be zero, MmappedRegions seeds segment placement at
the first chunk's offset rather than at 0, and CassandraOutgoingFile.contained
measures the live span so such a child stays eligible for entire-sstable
streaming.

New: ZeroCopySSTableSplitter, AntiCompactionRunPlanner, Reflink.
Config: zero_copy_anticompaction_enabled, zero_copy_split_reflink_enabled,
zero_copy_split_digest_enabled. Metric: BytesZeroCopyAnticompaction.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant