Skip to content

Inline Redis collection TTL metadata#997

Open
bootjp wants to merge 23 commits into
mainfrom
design/ttl-inline-value-metadata
Open

Inline Redis collection TTL metadata#997
bootjp wants to merge 23 commits into
mainfrom
design/ttl-inline-value-metadata

Conversation

@bootjp

@bootjp bootjp commented Jul 7, 2026

Copy link
Copy Markdown
Owner

Summary

  • embed collection TTL metadata with collection values instead of requiring sidecar anchors
  • add inline TTL support for Redis HyperLogLog values
  • add migration for legacy TTL anchors and mark the design implemented

Design doc

  • docs/design/2026_04_17_implemented_ttl_inline_value.md

Validation

  • go test ./adapter ./internal/backup -run 'TestRedis.*TTL|TestRedis.*HLL|TestRedisCollection|TestTTL|TestInline|TestEncodeRedis|TestDecodeRedis' -count=1
  • golangci-lint --config=.golangci.yaml run ./adapter ./internal/backup --timeout=5m
  • git verify-commit for all branch commits

Author: bootjp

Summary by CodeRabbit

  • 新機能
    • 文字列、HLL、リスト、ハッシュ、セット、ZSET、ストリームのTTLをインラインで保持できるようになりました。
    • レガシーTTL参照のフォールバック無効化を設定できるようになりました。
  • バグ修正
    • 期限切れの再作成時に、古い内容やTTLが復活しないよう改善しました。
    • 期限切れ後の再投入(例:ストリーム)でもTTLが誤って復活しないよう調整しました。
    • MULTI/EXECや移行後のTTL反映を正確化しました。
  • テスト
    • TTL更新・移行・再作成・バックアップ/復元を広範に検証するテストを追加・拡充しました。

@coderabbitai

coderabbitai Bot commented Jul 7, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@bootjp, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 34 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 7f8d8a8f-fddb-4f6c-afb0-6de8dc496df3

📥 Commits

Reviewing files that changed from the base of the PR and between 3b39744 and 74f6102.

📒 Files selected for processing (5)
  • adapter/redis_collection_ttl_test.go
  • adapter/redis_compat_commands_stream_test.go
  • adapter/redis_stream_cmds.go
  • adapter/redis_ttl_inline_migrator.go
  • adapter/redis_txn_test.go
📝 Walkthrough

Walkthrough

RedisのTTLをコレクションメタデータとHLL値へインライン化し、型別のTTL読み書き、トランザクション、レガシーデータ移行、デルタコンパクション、期限切れ再作成、バックアップ復元を更新しました。

Changes

TTLインライン化と互換性

Layer / File(s) Summary
メタデータ形式とバックアップ復元
store/*_helpers.go, internal/backup/*
コレクションメタデータとHLL値にExpireAtを追加し、旧形式の読み取りとインライン形式のバックアップ復元に対応しました。
型別TTL読み書きと再作成
adapter/redis_collection_ttl.go, adapter/redis_compat_types.go, adapter/redis_expire_cmds.go, adapter/redis_set_cmds.go, adapter/redis_stream_cmds.go
インラインTTLを優先して読み取り、型別の期限更新、HLL保存、期限切れコレクションのクリーンアップを実装しました。
トランザクション、Lua、コンパクション
adapter/redis_txn.go, adapter/redis_lua_context.go, adapter/redis_delta_compactor.go
MULTI/EXECとLuaでTTL付きメタを生成し、デルタ畳み込み時にExpireAtを保持します。
レガシーTTL移行
adapter/redis_ttl_inline_migrator.go, adapter/redis_hash_cmds.go, adapter/redis_zset_cmds.go
TTLインデックスとレガシーブロブを走査して、インラインメタへ移行します。
検証と設計更新
adapter/*_test.go, internal/backup/*_test.go, docs/design/*
インラインTTL、移行、フォールバック、デルタ、ストリーム、バックアップ、トランザクションを検証します。

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 28.65% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 変更の中心であるRedisコレクションのTTLメタデータをインライン化する内容を正確に要約しており、簡潔で分かりやすいです。

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements inline TTL for Redis collections (list, hash, set, zset, stream) and HLL payloads by embedding expiration timestamps directly in their metadata anchors, supported by a background migrator and an option to disable the legacy secondary index fallback. The code review identified several critical issues: first, applyPositiveExpire does not update collection metadata with the new TTL, leading to desynchronization; second, the background migrator batches too many keys in a single transaction, risking frequent write conflicts; and third, both hasExpired and decodeHLLSetValueAt bypass the disableLegacyTTLReadFallback gate by unconditionally calling legacyIndexTTLAt. Additionally, the migrator should be made more robust to handle legacy TTL updates on already-migrated keys.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread adapter/redis_txn.go Outdated
Comment thread adapter/redis_ttl_inline_migrator.go Outdated
Comment on lines +177 to +181
if err := c.dispatchCompaction(ctx, readTS, elems); err != nil {
c.logger.WarnContext(ctx, "ttl inline migrator: batch dispatch failed",
"type", h.typeName, "error", err)
return err
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

migrateTTLInlineHandler accumulates migration elements for up to 512 different user keys (ttlInlineMigrationTickScanLimit) and dispatches them all in a single Raft transaction via c.dispatchCompaction. In a production environment, a single transaction containing up to 1024 elements (migration writes + TTL index syncs) across hundreds of distinct keys is highly susceptible to write conflicts. If a concurrent client write touches even one of these keys during the transaction, the entire batch of 512 keys will fail with ErrWriteConflict and abort. This leads to wasted CPU/network resources, high transaction retry overhead, and can severely slow down or stall the background migration. Recommendation: Batch the migration dispatches into much smaller groups (e.g., 16 or 32 keys per transaction), or dispatch them individually/in very small batches so that a write conflict on a single hot key does not roll back the migration progress of the entire scanned window.

Comment thread adapter/redis_compat_types.go Outdated
Comment on lines +458 to +460
if err == nil && !found {
ttl, err = r.legacyIndexTTLAt(ctx, userKey, readTS)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

In hasExpired, when nonStringOnly is true and collectionTTLAt misses (found == false), legacyIndexTTLAt is called unconditionally. This completely bypasses the r.disableLegacyTTLReadFallback gate. Even after the operator disables the legacy fallback to optimize hot-path reads, any non-string check (such as EXISTS on a non-existent key or a key without a TTL) will still perform an expensive, redundant point lookup on the !redis|ttl|<key> index. Recommendation: Wrap the legacyIndexTTLAt call in a check for !r.disableLegacyTTLReadFallback.

Suggested change
if err == nil && !found {
ttl, err = r.legacyIndexTTLAt(ctx, userKey, readTS)
}
if err == nil && !found && !r.disableLegacyTTLReadFallback {
ttl, err = r.legacyIndexTTLAt(ctx, userKey, readTS)
}

Comment thread adapter/redis_ttl_inline_migrator.go Outdated
Comment on lines +145 to +150
if ttlMs == 0 {
ttlMs, err = legacyTTLMillisAt(ctx, c.st, userKey, readTS)
if err != nil {
return nil, err
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In simpleTTLInlineMigrationHandler, the migrator only queries the legacy TTL index via legacyTTLMillisAt if ttlMs == 0 (i.e., the metadata key is in the legacy format or has no inline TTL). If a collection key has already been migrated to the inline format (so ttlMs > 0), but a subsequent EXPIRE command executed inside a MULTI/EXEC transaction or a Lua script updates the legacy TTL index without updating the metadata key, the migrator will completely skip reading the legacy TTL index. As a result, the metadata key will remain stuck with the stale TTL, and ttlAt (which prioritizes inline metadata) will return the stale TTL. While the primary fix should be ensuring that transactions and Lua scripts update the metadata key directly, the migrator could also be made more robust by always syncing the metadata TTL if the legacy TTL index has a different, newer value during the migration window.

Comment thread adapter/redis_compat_helpers.go Outdated
Comment on lines +612 to +617
if !embedded {
ttl, err = r.legacyIndexTTLAt(ctx, key, readTS)
if err != nil {
return redisSetValue{}, err
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In decodeHLLSetValueAt, if the HLL payload is in the legacy format (embedded == false), legacyIndexTTLAt is called unconditionally. This ignores the r.disableLegacyTTLReadFallback gate, meaning we will still perform a secondary index lookup for legacy HLL keys even after the fallback has been disabled. Recommendation: Respect r.disableLegacyTTLReadFallback before calling legacyIndexTTLAt.

Suggested change
if !embedded {
ttl, err = r.legacyIndexTTLAt(ctx, key, readTS)
if err != nil {
return redisSetValue{}, err
}
}
if !embedded && !r.disableLegacyTTLReadFallback {
ttl, err = r.legacyIndexTTLAt(ctx, key, readTS)
if err != nil {
return redisSetValue{}, err
}
}

@bootjp

bootjp commented Jul 7, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 50d34a5e09

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread adapter/redis_collection_ttl.go
Comment thread adapter/redis_txn.go Outdated
Comment thread adapter/redis_collection_ttl.go
Comment thread adapter/redis_ttl_inline_migrator.go Outdated
@bootjp

bootjp commented Jul 8, 2026

Copy link
Copy Markdown
Owner Author

Handled the latest TTL-inline review blockers.

  • EXPIRE/PEXPIRE now falls back correctly for legacy hash/set/zset collection blobs instead of returning 0 before migration.
  • Delta-heavy list/hash metadata no longer rejects TTL updates solely because full delta folding is truncated; base TTL or the scan index still advances.
  • MULTI/EXEC staged list and zset creates now write inline TTL metadata in the same transaction.
  • Lua staged collection creates now full-commit new list/zset keys so inline TTL metadata is present.
  • The TTL-inline migrator no longer skips legitimate list user keys beginning with d|; real list meta delta rows are distinguished by value length.

Validation:

  • golangci-lint --config=.golangci.yaml run --fix
  • go test ./adapter -run "TestRedisCollectionExpireHandlesLegacyBlobs|TestRedisCollectionExpireAllowsDeltaHeavyCollections|TestRedisMultiExecStagedCollectionCreateWritesInlineTTL|TestRedisLuaStagedCollectionCreateWritesInlineTTL|TestRedisCollectionExpireWritesInlineMetaTTL|TestDeltaCompactor_TTLInlineMigratesLegacyString|TestDeltaCompactor_TTLInlineMigratesLegacyHLL|TestDeltaCompactor_TTLInlineMigratesLegacyHashMeta|TestDeltaCompactor_TTLInlineMigratesListUserKeyStartingWithDeltaPrefix|TestDeltaCompactor_TTLInlineRefreshesStaleInlineHashMeta|TestDeltaCompactor_TTLInlineMigrationDispatchesSmallBatches|TestRedis_MultiExec_DelThenRPushRecreatesList|TestRedis_MultiExec_ExpireSetsVisibleTTL|TestRedis_MultiExec_SetEXStoresTTL" -count=1 -timeout=300s
  • go test ./adapter -run "^$" -count=1 -timeout=120s

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 607ee9c82d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread adapter/redis_compat_types.go
Comment thread adapter/redis_delta_compactor.go
Comment thread adapter/redis_ttl_inline_migrator.go Outdated
Comment thread adapter/redis_ttl_inline_migrator.go
@bootjp

bootjp commented Jul 8, 2026

Copy link
Copy Markdown
Owner Author

Handled the latest TTL-inline review blockers.

  • Logical Redis backup now strips the HLL inline TTL envelope before writing hll/*.bin and deduplicates the matching hll_ttl.jsonl entry when the scan-index TTL row also appears.
  • TTL-inline migration now runs even on nodes that lead only non-default shard groups, while still filtering candidate keys through IsLeaderForKey before dispatch.
  • Empty Redis keys are treated as valid migration keys; nil remains the invalid extractor result.
  • TTL-index migration now covers base-less collection keys, including delta-only wide-column collections and legacy hash/set/zset blobs, and leaves base-meta keys to the existing base-meta migrator to avoid overlapping stale-read updates.

Validation:

  • go test ./internal/backup -count=1 -timeout=240s
  • go test ./adapter -run "TestDeltaCompactor_TTLInline|TestRedisCollectionExpire|TestRedisMultiExecStagedCollectionCreateWritesInlineTTL|TestRedisLuaStagedCollectionCreateWritesInlineTTL|TestRedis_MultiExec_DelThenRPushRecreatesList|TestRedis_MultiExec_ExpireSetsVisibleTTL|TestRedis_MultiExec_SetEXStoresTTL|TestRedisHLL" -count=1 -timeout=300s
  • golangci-lint --config=.golangci.yaml run ./adapter/... ./internal/backup/... --timeout=5m
  • git diff --check
  • git verify-commit HEAD

Note: go test ./adapter -count=1 -timeout=300s reached the package-level timeout after the focused coverage and lint above had passed.

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9b26fd2a95

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread adapter/redis_compat_types.go Outdated
@bootjp

bootjp commented Jul 8, 2026

Copy link
Copy Markdown
Owner Author

Fixed the latest HLL TTL precedence finding.

Changes:

  • make ttlAt consult HLL TTL ownership before collection inline TTL
  • treat a new-format HLL anchor without embedded TTL as authoritative no-TTL, so stale collection metadata cannot expire a recreated HLL
  • keep legacy HLL payloads eligible for the legacy !redis|ttl| fallback
  • add regression coverage for stale expired collection metadata plus a fresh no-TTL HLL anchor

Caller audit:

  • hllTTLAt is only called by ttlAt
  • ttlAt callers cover Redis TTL/PTTL/Lua and the non-string guard path through hasExpired; HLL still enters the normal ttlAt path because rawKeyTypeAt classifies HLL anchors as string-family keys

Validation:

  • go test ./adapter -run 'TestRedisHLLTTLAtIgnoresStaleCollectionTTL|TestRedisHLLTTLAtReadsEmbeddedAfterScanIndexDeleted|TestRedisDispatchHLLExpireWritesInlineAnchorAndScanIndex|TestRedisLoadHLLAtTreatsExpiredInlineTTLAsEmpty|TestRedisLoadHLLAtTreatsExpiredLegacyTTLAsEmpty|TestRedisLoadHLLAtSkipsLegacyTTLWhenFallbackDisabled|TestRedisHasExpiredSkipsLegacyTTLWhenFallbackDisabled|TestRedisTTLAt_LegacyFallbackCanBeDisabled|TestDeltaCompactor_TTLInlineMigratesLegacyHLL' -count=1 -timeout=240s
  • go test ./adapter -run 'TestRedis(HLL|LoadHLL|CollectionExpire|_MultiExec_ExpireSetsVisibleTTL|_ExpiredKey_BecomesInvisible|_HGET_TTLExpired|_HEXISTS_TTLExpired|_SISMEMBER_TTLExpired)|TestLua_(HGET|HEXISTS|ZSCORE|ZRANGEBYSCORE)_TTLExpired|TestDeltaCompactor_TTLInline' -count=1 -timeout=300s
  • golangci-lint --config=.golangci.yaml run ./adapter/... --timeout=5m
  • git diff --check
  • git verify-commit HEAD

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cd45d8ccfb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread adapter/redis_txn.go Outdated
Comment thread internal/backup/redis_string.go
Comment thread adapter/redis_collection_ttl.go Outdated
Comment thread internal/backup/redis_hash.go Outdated
Comment thread adapter/redis_collection_ttl.go Outdated
@bootjp

bootjp commented Jul 8, 2026

Copy link
Copy Markdown
Owner Author

Handled the latest TTL-inline review blockers.

  • MULTI/EXEC SET followed by EXPIRE now treats the staged string as the TTL owner instead of trying the HLL rewrite path.
  • Current-format no-TTL collection metadata is authoritative, so stale scan-index TTL rows no longer leak into TTL/PTTL or expiry checks.
  • Logical backup now treats current-format HLL/string anchors and inline collection metadata as owning TTL, including no-TTL anchors, so stale scan-index rows are ignored.
  • Delta-only hash/set/zset EXPIRE keeps the scan-index update and signals urgent compaction when the delta scan is truncated.

Validation:

  • go test ./adapter -run 'TestRedis_MultiExec_SetThenExpireNewString|TestRedis_MultiExec_SetEXStoresTTL|TestRedis_MultiExec_ExpireSetsVisibleTTL|TestRedisCollectionExpire|TestRedisTTLAtInlineNoTTLCollectionMetaSkipsLegacyFallback|TestRedisHLLTTLAt|TestRedisHasExpiredSkipsLegacyTTLWhenFallbackDisabled|TestRedisTTLAt_LegacyFallbackCanBeDisabled|TestDeltaCompactor_TTLInline' -count=1 -timeout=300s\n- go test ./internal/backup -count=1 -timeout=240s\n- golangci-lint --config=.golangci.yaml run ./adapter/... ./internal/backup/... --timeout=5m\n- git diff --check\n- git verify-commit HEAD\n\n@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b1b3251649

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread adapter/redis_ttl_inline_migrator.go Outdated
Comment on lines +155 to +156
if legacyTTL != 0 && legacyTTL != ttlMs {
ttlMs = legacyTTL

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve authoritative inline collection TTLs

When this handler scans an already-current hash/set/zset meta value, ttlMs came from the inline metadata and is authoritative, but this fallback unconditionally re-imports any stale !redis|ttl| row. For a current 16-byte meta with ExpireAt=0 plus an old scan-index TTL, the migrator rewrites the collection with that stale TTL and the key can later expire even though the inline anchor says it is persistent. Only consult the legacy TTL when pair.Value is the legacy 8-byte meta format.

Useful? React with 👍 / 👎.

Comment thread adapter/redis_ttl_inline_migrator.go Outdated
Comment on lines +461 to +462
if legacyTTL != 0 && legacyTTL != meta.ExpireAt {
meta.ExpireAt = legacyTTL

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not resurrect stale TTLs for current list meta

For a current-format list meta row, meta.ExpireAt is already the authoritative TTL, including 0 for no TTL; this unconditional legacy fallback will copy a stale !redis|ttl| value back into the inline meta. A list that was made persistent but still has an old scan-index row will be rewritten as expiring by the migrator. Gate this fallback on the legacy 24-byte list meta format; the stream migrator below repeats the same pattern and needs the same guard for 24-byte stream meta.

Useful? React with 👍 / 👎.

Comment thread adapter/redis_ttl_inline_migrator.go
Comment thread adapter/redis_txn.go Outdated
@bootjp

bootjp commented Jul 8, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@bootjp

bootjp commented Jul 8, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@bootjp

bootjp commented Jul 8, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@bootjp

bootjp commented Jul 10, 2026

Copy link
Copy Markdown
Owner Author

Handled the latest inline TTL metadata blockers on dfea1eb590c4e8291cf09246204282f01596863f.

Changes:

  • XADD now preserves stream ExpireAt when rewriting stream metadata, including first writes that clean up a legacy stream blob.
  • Legacy hash/set/zset migration and hash legacy rewrite now copy the existing !redis|ttl| timestamp into the new inline metadata.
  • MULTI/EXEC hash create/recreate followed by EXPIRE now emits staged hash metadata with the staged field count and TTL instead of rebuilding from the pre-transaction base.
  • Exact set creation now treats a physical HLL anchor as type-protecting while keeping TTL-filtered HLL existence for logical HLL reads.

Caller audit:

  • hllExistsAt remains the logical HLL existence helper; set WRONGTYPE protection now uses hllAnchorExistsAt directly.
  • buildHashLegacyRewriteElems callers were updated to pass the effective legacy/staged TTL.
  • streamWriteBase callers keep the loaded StreamMeta, so XADD/XTRIM retain ExpireAt through metadata rewrites.

Validation:

  • go test ./adapter -run 'TestRedis(MultiExecStagedCollectionCreateWritesInlineTTL|XAddPreservesStreamInlineTTL|LegacyCollectionMigrationPreservesLegacyTTL|_MultiExec_ExpireRecreated(Hash|List)KeepsStagedMeta|_SAddRejectsExpiredHLLPayload)|TestRedisTxnHashLegacyRewrite(PreservesTTL|WritesFence)|TestRedisSetLegacyMigrationWritesFenceWithoutLenDelta' -count=1 -timeout=240s
  • go test ./adapter -run 'TestRedis(CollectionExpire|TTLAt|HLLTTLAt|LoadHLL|HLL|HasExpiredSkipsLegacyTTLWhenFallbackDisabled|TTLAt_LegacyFallbackCanBeDisabled|MultiExecStagedCollectionCreateWritesInlineTTL|LuaStagedCollectionCreateWritesInlineTTL|_MultiExec_ExpireRecreated(Hash|List)KeepsStagedMeta|_MultiExec_HSetRecreatesExpiredHash|_SAddRejectsExpiredHLLPayload)|TestDeltaCompactor_TTLInline|TestRedisTxnHashLegacyRewrite|TestRedisSetLegacyMigrationWritesFenceWithoutLenDelta' -count=1 -timeout=300s
  • go test -race ./adapter -run 'TestRedis(MultiExecStagedCollectionCreateWritesInlineTTL|XAddPreservesStreamInlineTTL|LegacyCollectionMigrationPreservesLegacyTTL|_MultiExec_ExpireRecreated(Hash|List)KeepsStagedMeta|_SAddRejectsExpiredHLLPayload)|TestRedisTxnHashLegacyRewrite(PreservesTTL|WritesFence)' -count=1 -timeout=240s
  • golangci-lint run ./adapter --timeout=5m
  • go test ./... -run TestNonexistent -count=0 -timeout=300s
  • git diff --check
  • git verify-commit HEAD

@codex review
@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown

@bootjp Thanks for the update. I'll kick off a fresh review covering the stream ExpireAt preservation in XADD/XTRIM metadata rewrites, the legacy hash/set/zset TTL migration copying the existing !redis|ttl| timestamp, the MULTI/EXEC hash create/recreate+EXPIRE staged metadata path, and the HLL anchor type-protection for exact set creation, at commit dfea1eb590c4e8291cf09246204282f01596863f.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dfea1eb590

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread adapter/redis_stream_cmds.go
Comment thread adapter/redis_set_cmds.go Outdated
}
}
value, err := r.loadSetAt(ctx, hllKind, key, readTS)
value, _, err := r.loadHLLPayloadAt(ctx, key, readTS)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep PFCOUNT from reading expired HLL anchors

For an HLL whose inline TTL has expired but whose physical !redis|hll| row has not been swept yet, the preceding TTL-filtered keyTypeAt returns redisTypeNone, but this unfiltered load still decodes the anchor and unions its members. In that window PFCOUNT key returns the stale cardinality instead of treating the key as absent, so this should use the TTL-filtered HLL load/existence path.

Useful? React with 👍 / 👎.

Comment thread adapter/redis_ttl_inline_migrator.go Outdated
Comment on lines +151 to +152
if ttlMs == 0 {
ttlMs, err = legacyTTLMillisAt(ctx, c.st, userKey, readTS)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Gate legacy TTL migration on legacy meta width

This fallback runs whenever the decoded ExpireAt is zero, including current inline metadata that explicitly means “no TTL”. If such a key has a stale !redis|ttl| scan-index entry, the migrator copies that stale timestamp back into the authoritative inline meta (the list/stream handlers use the same pattern), so persistent collections can become expiring after a compaction pass; only consult legacyTTLMillisAt for legacy-width metadata.

Useful? React with 👍 / 👎.

Comment thread adapter/redis_ttl_inline_migrator.go Outdated
Comment on lines +332 to +333
case redisTypeNone, redisTypeString, redisTypeList, redisTypeStream:
return nil, nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Migrate legacy stream TTLs before disabling fallback

When the TTL-index migrator encounters a legacy !redis|stream| blob with no new !stream|meta| anchor, this branch skips it entirely, unlike the hash/set/zset legacy blob cases below. After WithRedisLegacyTTLReadFallback(false) is enabled following migration, those legacy streams no longer consult !redis|ttl|, so an expired legacy stream is treated as live until some later write happens to clean it up.

Useful? React with 👍 / 👎.

@bootjp

bootjp commented Jul 10, 2026

Copy link
Copy Markdown
Owner Author

Handled the latest TTL-inline review blockers on 62e29c2c4962ed272dcdac8fa3ca622d10b5abfd.

Changes:

  • XADD now clears stale wide-column stream rows and resets stream metadata when recreating a logically expired stream, so stale entries and ExpireAt do not leak into the new stream.
  • PFCOUNT now uses TTL-filtered HLL existence and payload loading, so expired HLL anchors no longer contribute to counts while still preserving wrong-type protection for live non-HLL keys.
  • TTL-inline migration only consults the legacy TTL index when the scanned metadata is actually legacy-width, preventing stale scan-index TTL rows from being applied to current no-TTL metadata.
  • TTL-index migration now rewrites legacy stream blobs into stream metadata/entry rows before legacy fallback can be disabled.

Caller audit:

  • xaddTxn is only called from XADD; XTRIM keeps the existing streamTypeForWrite path.
  • hllExistsAt remains the logical HLL existence helper; exact set type protection still uses physical HLL anchor checks.
  • simpleTTLInlineMigrationHandler, migrateListTTLInlineElems, migrateStreamTTLInlineElems, and legacyCollectionTTLInlineElems callers remain the TTL-inline migrator registration/index path only.

Validation:

  • go test ./adapter -run 'TestRedis_(SAddRejectsExpiredHLLPayload|StreamXAddRecreatesExpiredStreamWithoutStaleEntriesOrTTL)$|TestDeltaCompactor_TTLInline(MigratesLegacyStreamTTL|KeepsCurrentNoTTLMetaFromStaleLegacyIndex|DoesNotOverwriteCurrentMetaFromStaleLegacyIndex|MigratesListUserKeyStartingWithDeltaPrefix|MigratesEmptyCollectionKeys)$' -count=1 -timeout=240s\n- go test ./adapter -run 'TestRedis_(HLL|PF|SAddRejectsExpiredHLLPayload|Stream)|TestRedisLoadHLLAt|TestRedisTTLAt|TestDeltaCompactor_TTLInline' -count=1 -timeout=300s\n- golangci-lint run ./adapter --timeout=5m\n- git diff --check\n- git verify-commit HEAD\n\nCurrent unresolved non-outdated review threads: 0.\n\n@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 62e29c2c49

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread adapter/redis_stream_cmds.go
Comment thread adapter/redis_zset_cmds.go
Comment thread adapter/redis_set_cmds.go
Comment thread internal/backup/redis_string.go
Comment thread internal/backup/redis_hash.go
@bootjp

bootjp commented Jul 10, 2026

Copy link
Copy Markdown
Owner Author

Handled the latest TTL-inline restore and recreate blockers on aceb441fd7d05137414f8b550e301fe044b598ee.

Changes:

  • legacy hash/set/zset/stream recreate paths now drop expired legacy !redis|ttl| values instead of carrying them into new inline metadata
  • the same expired legacy TTL guard is applied to hash for the matching legacy recreate surface
  • Redis logical restore now recreates HLL inline TTL envelopes when hll_ttl.jsonl is present
  • Redis logical restore now writes inline TTL metadata for hash/set/list/zset/stream while still preserving the scan-index TTL row
  • added direct byte-level restore tests so the inline metadata/envelope is verified without relying on decode round-trips

Validation:

  • go test ./adapter -run 'TestRedisLegacyCollectionMigration(PreservesLegacyTTL|ClearsExpiredLegacyTTL)|TestRedisLegacyStreamRecreateClearsExpiredLegacyTTL|TestRedis_StreamXAddRecreatesExpiredStreamWithoutStaleEntriesOrTTL|TestDeltaCompactor_TTLInlineMigratesLegacyCollectionBlobsFromTTLIndex' -count=1 -timeout=240s
  • go test ./internal/backup -run 'TestRedisEncode(HLLTTL|CollectionTTLs|HashRoundTrip|SetRoundTrip|ListRoundTrip|ZSetRoundTrip|StreamRoundTrip)' -count=1 -timeout=240s
  • go test ./adapter ./internal/backup -run 'TestRedis.*TTL|TestDeltaCompactor_TTLInline|TestRedisEncode.*TTL|TestRedisEncodeCollectionTTLsAreInlineInMeta|TestRedisDB_.*TTL' -count=1 -timeout=300s
  • go test ./internal/backup -count=1 -timeout=300s
  • go test ./internal/backup -run 'TestRedisEncodeCollectionTTLsAreInlineInMeta' -count=1 -timeout=120s
  • git diff --check
  • git verify-commit HEAD

Note: go test ./adapter ./internal/backup -count=1 -timeout=360s timed out in the adapter package after the focused coverage above had passed; internal/backup passed in that run.

Current unresolved non-outdated review threads: 0.

@codex review

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
adapter/redis_lua_context.go (1)

3178-3186: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

commit の ctx を下流の store 読み取りまで伝播してください

nonStringTTLElemsdirtyPositiveTTLOnLogicallyAbsentStartcontext.Background()collectionExpireElems / keyTypeAt を呼んでいるため、redisDispatchTimeout の期限が効きません。commit() で作った ctx を引数で渡し、commitPlanForKey 内の keyTypeAt も合わせて置き換えてください。

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@adapter/redis_lua_context.go` around lines 3178 - 3186, commit() で生成した ctx
が下流のストア読み取りまで伝播するよう修正してください。nonStringTTLElems と
dirtyPositiveTTLOnLogicallyAbsentStart に ctx を引数として追加し、内部の collectionExpireElems
および keyTypeAt 呼び出しで context.Background() を使わず渡された ctx を使用してください。commitPlanForKey
内の keyTypeAt 呼び出しも同じ ctx に置き換え、各呼び出し元を更新して redisDispatchTimeout が適用されるようにしてください。
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@adapter/redis_collection_ttl.go`:
- Around line 359-377: In simpleMetaExpireScanErr, invoke
r.triggerUrgentCompaction(typeName, key) in the ErrDeltaScanTruncated branch
even when exists is true, before returning the metadata update, so delta folding
is triggered consistently with the other collection types.

---

Nitpick comments:
In `@adapter/redis_lua_context.go`:
- Around line 3178-3186: commit() で生成した ctx
が下流のストア読み取りまで伝播するよう修正してください。nonStringTTLElems と
dirtyPositiveTTLOnLogicallyAbsentStart に ctx を引数として追加し、内部の collectionExpireElems
および keyTypeAt 呼び出しで context.Background() を使わず渡された ctx を使用してください。commitPlanForKey
内の keyTypeAt 呼び出しも同じ ctx に置き換え、各呼び出し元を更新して redisDispatchTimeout が適用されるようにしてください。
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ab049491-393b-4d1e-8af2-31bf37780eb7

📥 Commits

Reviewing files that changed from the base of the PR and between cd58354 and aceb441.

📒 Files selected for processing (41)
  • adapter/redis.go
  • adapter/redis_collection_ttl.go
  • adapter/redis_collection_ttl_test.go
  • adapter/redis_compat_commands_stream_test.go
  • adapter/redis_compat_helpers.go
  • adapter/redis_compat_types.go
  • adapter/redis_delta_compactor.go
  • adapter/redis_delta_compactor_test.go
  • adapter/redis_expire_cmds.go
  • adapter/redis_hash_cmds.go
  • adapter/redis_hll_ttl_test.go
  • adapter/redis_lua_context.go
  • adapter/redis_proxy_leader.go
  • adapter/redis_retry_test.go
  • adapter/redis_set_cmds.go
  • adapter/redis_storage_codec_test.go
  • adapter/redis_stream_cmds.go
  • adapter/redis_ttl_compat_test.go
  • adapter/redis_ttl_inline_migrator.go
  • adapter/redis_txn.go
  • adapter/redis_txn_test.go
  • adapter/redis_zset_cmds.go
  • docs/design/2026_04_17_implemented_ttl_inline_value.md
  • internal/backup/encode_redis.go
  • internal/backup/encode_redis_coll.go
  • internal/backup/encode_redis_coll_test.go
  • internal/backup/encode_redis_test.go
  • internal/backup/redis_hash.go
  • internal/backup/redis_hash_test.go
  • internal/backup/redis_list.go
  • internal/backup/redis_meta.go
  • internal/backup/redis_set.go
  • internal/backup/redis_stream.go
  • internal/backup/redis_string.go
  • internal/backup/redis_string_test.go
  • internal/backup/redis_zset.go
  • store/hash_helpers.go
  • store/list_helpers.go
  • store/set_helpers.go
  • store/stream_helpers.go
  • store/zset_helpers.go

Comment thread adapter/redis_collection_ttl.go

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aceb441fd7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread adapter/redis_lua_context.go Outdated
keyBytes := []byte(key)
elems := []*kv.Elem[kv.OP]{}
if preserveExisting {
metaElems, ok, err := c.server.collectionExpireElems(context.Background(), keyBytes, c.startTS, typ, ttlMillis(ttl))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve ZSET delta when updating Lua TTL

When a Lua script mutates a ZSET through the delta path and also changes its TTL while exactly store.MaxDeltaScanLimit existing ZSetMetaDeltaKey rows are present, zsetDeltaMetaElems can compact those old deltas together with the script's current lenDelta into a new base meta earlier in the same transaction. This later collectionExpireElems call rebuilds the meta again from c.startTS without the current lenDelta and is appended after the ZSET elems, so it overwrites the compacted meta with a stale cardinality while the member rows have already changed; ZCARD/empty detection then observe the wrong length until a later repair.

Useful? React with 👍 / 👎.

Comment thread store/hash_helpers.go
Comment thread adapter/redis_collection_ttl.go Outdated
Comment on lines +375 to +376
r.triggerUrgentCompaction(typeName, key)
return nil, true, nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Don’t acknowledge EXPIRE without inline metadata

When a hash/set/zset exists only as delta rows and has more than MaxDeltaScanLimit deltas, this branch reports success to EXPIRE while emitting no inline metadata update, so only the !redis|ttl| scan row is written. The urgent compactor that runs afterward folds those deltas into a current 16-byte meta with ExpireAt=0, and the TTL migrator then skips the scan row because a base meta exists, leaving the key permanently unexpired (and even the fallback path is shadowed by the zero inline TTL).

Useful? React with 👍 / 👎.

@bootjp

bootjp commented Jul 10, 2026

Copy link
Copy Markdown
Owner Author

Addressed the latest inline TTL review feedback.

Changes:

  • Trigger urgent compaction for truncated simple-meta EXPIRE even when base metadata exists.
  • Avoid acknowledging delta-only truncated collection EXPIRE without an inline metadata update.
  • Preserve legacy hash TTL when HDEL rewrites a non-empty legacy hash into wide-column metadata.
  • Keep Lua ZSET delta compaction cardinality and final TTL in the same metadata write, and avoid a later snapshot-based TTL rewrite from overwriting it.
  • Prevent MULTI/EXEC from writing a collection TTL scan index when the matching inline metadata update could not be emitted.

Verification:

  • go test ./adapter -run 'TestRedis(CollectionExpireAllowsDeltaHeavyCollections|CollectionExpireTriggersCompactionForDeltaOnlyTruncatedCollections|HDelLegacyRewritePreservesLegacyTTL|LuaZSetDeltaTTLCompactionPreservesCardinality|LuaStagedCollectionCreateWritesInlineTTL|MultiExecStagedCollectionCreateWritesInlineTTL)|TestRedisTxnHashLegacyRewritePreservesTTL' -count=1 -timeout=300s\n- go test ./adapter -run 'TestRedisTxn|TestRedis_MultiExec|TestRedisMultiExecStagedCollectionCreateWritesInlineTTL' -count=1 -timeout=300s\n- golangci-lint run ./adapter --timeout=5m\n- git diff --check\n\n@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 66d868cdee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread adapter/redis_collection_ttl.go Outdated
if exists {
return listMetaTTLUpdateElem(key, meta, expireAtMs)
}
return nil, true, nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not acknowledge truncated list EXPIRE without meta

For a list that only has ListMetaDeltaKey rows and more than MaxDeltaScanLimit deltas, this branch returns ok=true with no metadata update, so standalone EXPIRE and MULTI/EXEC will still write only !redis|ttl| and report success. The urgent compactor then materializes a 32-byte list meta with ExpireAt=0, and the inline TTL migrator treats that as authoritative and can delete the scan-index TTL, so the list silently loses the expiry after compaction/migration.

Useful? React with 👍 / 👎.

Comment thread adapter/redis_lua_context.go
Comment thread adapter/redis_txn.go
Comment thread adapter/redis_txn.go
Comment thread adapter/redis_txn.go
@bootjp

bootjp commented Jul 10, 2026

Copy link
Copy Markdown
Owner Author

Addressed latest-head TTL review findings:

  • Delta-only truncated collection EXPIRE now returns ErrDeltaScanTruncated after triggering urgent compaction instead of acknowledging success without an inline TTL anchor. This covers list/hash/set/zset collection metadata paths.
  • SET replacement now clears pending collection TTL rebuild state for the same key, so an earlier collection EXPIRE in EXEC cannot append stale collection metadata after the string replacement.
  • HLL dirty TTL working entries are skipped when the same user key is replaced by SET, so the replacement cannot reintroduce a stale !redis|hll| anchor.

Caller audit:

  • collectionExpireElems callers are standalone EXPIRE, MULTI/EXEC collection TTL rebuild, and Lua non-string TTL sync. All now propagate the truncated-delta error rather than reporting a successful TTL write without metadata.
  • Existing base-meta truncated cases still update the existing base metadata TTL and request urgent compaction.

Validation:

  • go test ./adapter -run 'TestRedisCollectionExpireTriggersCompactionForDeltaOnlyTruncatedCollections|TestRedisTxnSetAfter(CollectionExpire|HLLExpire)' -count=1 -timeout=240s\n- go test ./adapter -run 'TestRedis(CollectionExpire|TxnSetAfter|MultiExec_Expire|MultiExec_HSet|SAddRejectsExpiredHLL|LuaZSetDeltaTTLCompaction)' -count=1 -timeout=300s\n- golangci-lint run ./adapter --timeout=5m\n- git diff --check\n- git verify-commit HEAD\n\n@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 30c54f9f61

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread adapter/redis_compat_types.go Outdated
Comment on lines +428 to +431
if embedded || newFormat {
return ttl, true, nil
}
return nil, false, nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Stop falling through from legacy HLLs to collection TTLs

When an HLL anchor is still in the legacy payload format, this returns found=false, so ttlAt continues into collectionTTLAt before consulting the legacy !redis|ttl| index. In the same stale-metadata scenario covered for new HLL envelopes (a leftover hash/list/set/zset meta row for the same user key), a pre-migration HLL can be reported expired or assigned the collection's TTL even though decodeHLLSetValueAt would otherwise use the HLL's own legacy TTL/no-TTL state. Treat the presence of the HLL anchor as authoritative here, falling back to the legacy TTL index for old HLL payloads instead of letting collection metadata win.

Useful? React with 👍 / 👎.

Comment thread adapter/redis_txn.go

bootjp commented Jul 10, 2026

Copy link
Copy Markdown
Owner Author

Fixed the current test (ubuntu-latest) failure in TestRedis_StreamXReadLatencyIsConstant.

Changes:

  • Seed the 10k stream-lat entries directly in the test store using the wide-column stream metadata/entry layout.
  • Keep the measured path on XREAD, while removing the setup loop of thousands of XADD writes that could hit transient OCC write conflicts under -race.

Validation:

  • go test -race ./adapter -run '^TestRedis_StreamXReadLatencyIsConstant$' -count=1 -timeout=8m
  • go test ./adapter -run '^$' -count=1 -timeout=120s
  • git diff --check
  • commit hook: golangci-lint --config=.golangci.yaml run --fix
  • git verify-commit HEAD

Current head: 248ae1045d2117435d1cc130b51e8e4a0642cbce

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 248ae1045d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread adapter/redis_txn.go
Comment thread adapter/redis_hash_cmds.go
Comment thread adapter/redis_ttl_inline_migrator.go Outdated
Comment on lines +476 to +480
elems = append(elems, &kv.Elem[kv.OP]{
Op: kv.Put,
Key: store.StreamEntryKey(userKey, parsed.ms, parsed.seq),
Value: entryValue,
})

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep legacy stream blobs discarded during TTL migration

For a pre-wide-column stream that still has a !redis|ttl| row, the TTL-inline migrator now materializes every legacy stream entry into !stream|entry| rows. That contradicts the stream migration contract in the read/write path, where legacy stream blobs are ignored by loadStreamAt and deleted on the next write, so a background TTL migration can make previously discarded legacy entries visible again to XLEN/XRANGE/XREAD.

Useful? React with 👍 / 👎.

@bootjp

bootjp commented Jul 11, 2026

Copy link
Copy Markdown
Owner Author

Addressed the current-head TTL-inline review blockers.

Changes:

  • Expired legacy hash/set/zset blobs now migrate by deleting the stale legacy blob and TTL sidecar instead of materializing persistent inline collection metadata.
  • Transactional list/hash/zset create + PEXPIRE paths now keep the TTL scan index while relying on the staged inline metadata write.
  • Legacy stream TTL-inline migration now deletes the discarded legacy blob and writes empty stream metadata with the inline TTL, preserving the read/write contract that old stream blob entries stay invisible.

Caller audit:

  • buildHashLegacyMigrationElems callers: HSET/HINCRBY migration paths now combine expired-blob cleanup with fresh field writes; persistHashTxn now treats expired legacy hash rewrites as cleanup only, matching expired-key semantics.
  • buildSetLegacyMigrationElems and buildZSetLegacyMigrationElems are only used by set/zset wide-column migration paths, which now discard expired legacy members instead of reviving them.
  • legacyStreamTTLInlineElems is only called by the TTL-inline migrator, so the stream visibility change is confined to background migration.

Verification:

  • go test ./adapter -run 'TestRedisLegacyCollectionMigrationPreservesLegacyTTL|TestRedisLegacyCollectionMigrationCleansExpiredLegacyBlob|TestRedisHSetExpiredLegacyHashRecreatesWithoutStaleFields|TestRedisTxnCollectionCreateExpireKeepsTTLIndex|TestDeltaCompactor_TTLInlineMigratesLegacyStreamTTL' -count=1 -timeout=240s\n- go test ./adapter -run '^$' -count=1 -timeout=120s\n- go test ./adapter -count=1 -timeout=900s\n- golangci-lint --config=.golangci.yaml run ./adapter/... --timeout=5m\n- git diff --check\n- git verify-commit HEAD\n\n@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 751f0547c3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread adapter/redis_ttl_inline_migrator.go
Comment thread adapter/redis_collection_ttl.go
Comment thread adapter/redis_hash_cmds.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
adapter/redis_lua_context.go (1)

3393-3407: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

フルコミット時に ListMeta.Tail を設定してください。

要素はシーケンス 0..len-1 に保存されますが、Tail がゼロのため、永続化された範囲とメタデータが不整合になります。

修正案
 	keyBytes := []byte(key)
 	elems := make([]*kv.Elem[kv.OP], 0, len(st.values)+listFullCommitExtraElems)
 	for seq, value := range st.values {
 		elems = append(elems, &kv.Elem[kv.OP]{
 			Op:    kv.Put,
 			Key:   listItemKey(keyBytes, int64(seq)),
 			Value: []byte(value),
 		})
 	}
-	meta, err := store.MarshalListMeta(store.ListMeta{Len: int64(len(st.values)), ExpireAt: ttlMillis(ttl)})
+	listLen := int64(len(st.values))
+	meta, err := store.MarshalListMeta(store.ListMeta{
+		Head:     0,
+		Tail:     listLen,
+		Len:      listLen,
+		ExpireAt: ttlMillis(ttl),
+	})
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@adapter/redis_lua_context.go` around lines 3393 - 3407, フルコミット時に生成する ListMeta
の Tail が未設定です。要素をシーケンス 0..len-1 で保存している処理と対応するよう、MarshalListMeta に渡す
store.ListMeta で Tail を st.values の末尾シーケンス(空の場合も適切に扱う)として設定してください。対象はこの elems
構築処理と store.MarshalListMeta 呼び出しです。
🧹 Nitpick comments (1)
adapter/redis_zset_cmds.go (1)

617-617: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

既存定数を使って //nolint:mnd を削除してください。

zsetElemsPerAdded が同じ最大3要素を表しているため、リテラルと抑制コメントは不要です。

修正案
-	elems := make([]*kv.Elem[kv.OP], 0, len(base.cleanupElems)+len(base.migrationElems)+len(pairs)*3+setWideColOverhead) //nolint:mnd // 3 ops per pair
+	elems := make([]*kv.Elem[kv.OP], 0, len(base.cleanupElems)+len(base.migrationElems)+len(pairs)*zsetElemsPerAdded+setWideColOverhead)
-	elems := make([]*kv.Elem[kv.OP], 0, len(base.cleanupElems)+len(base.migrationElems)+3) //nolint:mnd // del old score + put member + put score index
+	elems := make([]*kv.Elem[kv.OP], 0, len(base.cleanupElems)+len(base.migrationElems)+zsetElemsPerAdded)

As per coding guidelines, 「Avoid using //nolint directives; refactor instead in Go code」.

Also applies to: 746-746

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@adapter/redis_zset_cmds.go` at line 617, Replace the literal multiplier 3 and
its //nolint:mnd suppression in the capacity calculations involving
cleanupElems, migrationElems, and pairs with the existing zsetElemsPerAdded
constant; apply the same change to the corresponding calculation near the second
referenced location.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@adapter/redis_lua_context.go`:
- Around line 3393-3407: フルコミット時に生成する ListMeta の Tail が未設定です。要素をシーケンス 0..len-1
で保存している処理と対応するよう、MarshalListMeta に渡す store.ListMeta で Tail を st.values
の末尾シーケンス(空の場合も適切に扱う)として設定してください。対象はこの elems 構築処理と store.MarshalListMeta 呼び出しです。

---

Nitpick comments:
In `@adapter/redis_zset_cmds.go`:
- Line 617: Replace the literal multiplier 3 and its //nolint:mnd suppression in
the capacity calculations involving cleanupElems, migrationElems, and pairs with
the existing zsetElemsPerAdded constant; apply the same change to the
corresponding calculation near the second referenced location.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3d0e73b8-c757-4d6f-9201-19a09c95233b

📥 Commits

Reviewing files that changed from the base of the PR and between 751f054 and 3b39744.

📒 Files selected for processing (11)
  • adapter/redis_collection_ttl.go
  • adapter/redis_collection_ttl_test.go
  • adapter/redis_compat_helpers.go
  • adapter/redis_compat_types.go
  • adapter/redis_hash_cmds.go
  • adapter/redis_hll_ttl_test.go
  • adapter/redis_lists.go
  • adapter/redis_lua_context.go
  • adapter/redis_set_cmds.go
  • adapter/redis_ttl_inline_migrator.go
  • adapter/redis_zset_cmds.go
💤 Files with no reviewable changes (1)
  • adapter/redis_compat_helpers.go
🚧 Files skipped from review as they are similar to previous changes (5)
  • adapter/redis_hll_ttl_test.go
  • adapter/redis_compat_types.go
  • adapter/redis_set_cmds.go
  • adapter/redis_collection_ttl.go
  • adapter/redis_ttl_inline_migrator.go

@bootjp

bootjp commented Jul 11, 2026

Copy link
Copy Markdown
Owner Author

Addressed the latest stream TTL cleanup gap on the current head.

Changes:

  • Expired inline stream recreation now deletes the stale !redis|ttl|<key> scan-index row in the same cleanup batch.
  • Expired legacy stream recreation now treats the legacy TTL sidecar as expired state, drops the sidecar, and writes the recreated stream with no inherited ExpireAt.
  • Removed the now-unused TTL helper after switching stream recreation to the expired-aware migration helper.
  • Added regression coverage for both inline-expired XADD recreation and expired legacy stream cleanup elems.

Caller audit:

  • streamCleanupForExpiredRecreate is only called by xaddTxn; it already deletes stale stream rows and now also deletes the TTL scan-index row.
  • streamWriteBase is used by XADD/XTRIM stream write paths; live legacy TTLs are still preserved, while expired legacy TTLs now append Del redisTTLKey(key) and reset ExpireAt to 0.
  • legacyTTLMillisForMigrationAt remains shared by hash/set/zset/stream migration/recreate paths and now covers the stream expired branch directly.

Validation:

  • go test ./adapter -run 'TestRedis_StreamXAddRecreatesExpiredStreamWithoutStaleEntriesOrTTL|TestRedisLegacyStreamRecreateClearsExpiredLegacyTTL|TestRedisXAddPreservesStreamInlineTTL' -count=1 -timeout=300s
  • go test ./adapter -run 'TestRedis(XAddPreservesStreamInlineTTL|LegacyStreamRecreateClearsExpiredLegacyTTL|_StreamXAddRecreatesExpiredStreamWithoutStaleEntriesOrTTL|_SAddRejectsExpiredHLLPayload|ExpiredInlineCollectionRecreateCleansPhysicalRows|LegacyCollectionMigrationCleansExpiredLegacyBlob)|TestRedis(HSetExpiredLegacyHashRecreatesWithoutStaleFields|HIncrByExpiredLegacyHashStartsFromZero)|TestRedisTxn(CollectionExpireDeltaOnlyTruncatedReturnsError|SetAfterCollectionExpireSkipsInlineTTLRebuild|CollectionCreateExpireKeepsTTLIndex|SetAfterHLLExpireSkipsDirtyHLLAnchor)|TestRedisLuaNonStringTTLDeltaOnlyTruncatedReturnsError|TestDeltaCompactor_TTLInline' -count=1 -timeout=300s
  • go test ./internal/backup -count=1 -timeout=240s
  • golangci-lint --config=.golangci.yaml run ./adapter/... ./internal/backup/... --timeout=5m
  • git diff --check

Current CI for 74f6102d6009d43238b7a726e186455b3862b1ee has started and is still in progress.

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 74f6102d60

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@bootjp

bootjp commented Jul 11, 2026

Copy link
Copy Markdown
Owner Author

Resolved the remaining addressed review threads on the current head after rechecking the implementation and focused tests.

Verification:

  • go test ./adapter -run 'TestRedisTxn(SetAfterCollectionExpireSkipsInlineTTLRebuild|SetAfterHLLExpireSkipsDirtyHLLAnchor|CollectionExpireReturnsDeltaScanTruncated|StagedStringWinsOverDeletionOnlyCollectionStates)|TestRedisHIncrByExpiredLegacyHashStartsFromZero|TestRedisLua.*DeltaOnly|TestRedisLuaStagedCollectionCreateWritesInlineTTL|TestRedisTTLInlineMigratorCleansExpiredLegacyTTLIndex|TestRedisExpiredLegacy|TestRedisXAddRecreatesExpiredStreamWithoutStaleEntriesOrTTL' -count=1 -timeout=300s

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 74f6102d60

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant