Helper files for flatKV cache by cody-littley · Pull Request #3056 · sei-protocol/sei-chain

cody-littley · 2026-03-11T17:42:47Z

Describe your changes and provide context

These are helper files for the FlatKV cache. Not general purpose utilities, but also not the core cache implementation.

Testing performed to validate your change

Tested in main branch (unit tests, benchmark, etc)

github-actions · 2026-03-11T17:43:47Z

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
`✅ passed`	`✅ passed`	`✅ passed`	`✅ passed`	Mar 12, 2026, 9:20 PM

github-actions · 2026-03-11T17:43:47Z

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
`✅ passed`	`✅ passed`	`✅ passed`	`✅ passed`	Mar 11, 2026, 5:43 PM

codecov · 2026-03-11T17:47:04Z

Codecov Report

❌ Patch coverage is 97.36842% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.37%. Comparing base (904de28) to head (e404540).
⚠️ Report is 7 commits behind head on main.

Files with missing lines	Patch %	Lines
sei-db/db_engine/pebbledb/pebblecache/cache.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3056      +/-   ##
==========================================
+ Coverage   58.27%   58.37%   +0.10%     
==========================================
  Files        2076     2083       +7     
  Lines      171480   171083     -397     
==========================================
- Hits        99933    99873      -60     
+ Misses      62619    62277     -342     
- Partials     8928     8933       +5

Flag	Coverage Δ
sei-chain-pr	`97.36% <97.36%> (?)`
sei-db	`70.41% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
sei-db/db_engine/pebbledb/pebblecache/lru_queue.go	`100.00% <100.00%> (ø)`
...ei-db/db_engine/pebbledb/pebblecache/noop_cache.go	`100.00% <100.00%> (ø)`
...db/db_engine/pebbledb/pebblecache/shard_manager.go	`100.00% <100.00%> (ø)`
sei-db/db_engine/types/types.go	`100.00% <100.00%> (ø)`
sei-db/db_engine/pebbledb/pebblecache/cache.go	`0.00% <0.00%> (ø)`

... and 253 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

blindchaser · 2026-03-11T19:59:01Z

sei-db/db_engine/pebbledb/pebblecache/cache.go

+	IsDelete bool
+}
+
+// Cache describes a cache kapable of being used by a FlatKV store.


nit: capable

blindchaser · 2026-03-11T19:59:43Z

sei-db/db_engine/pebbledb/pebblecache/lru_queue.go

+	lru.totalSize += size
+}
+
+// Signal that an entry has been interated with, moving it to the to the back of the queue


typo: interacted, double to the

blindchaser · 2026-03-11T20:00:26Z

sei-db/db_engine/pebbledb/pebblecache/shard_manager.go

+
+// A utility for assigning keys to shard indices.
+type shardManager struct {
+	// A random seed that makmes it hard for an attacker to predict the shard index and to skew the distribution.


typo: makes

blindchaser · 2026-03-11T20:03:15Z

sei-db/db_engine/pebbledb/pebblecache/shard_manager.go

+
+// Shard returns a shard index in [0, numShards).
+// addr should be the raw address bytes (e.g., 20-byte ETH address).
+func (s *shardManager) Shard(addr []byte) uint64 {


suggest adding unit testings for shard_manager.go

blindchaser · 2026-03-11T20:51:23Z

sei-db/db_engine/pebbledb/pebblecache/shard_manager.go

+func (s *shardManager) Shard(addr []byte) uint64 {
+	h := s.pool.Get().(*maphash.Hash)
+	h.SetSeed(s.seed)
+	h.Reset()


curious how does the h.SetSeed(s.seed) then h.Resest() next line work work?

h.Reset() doesn't remove the seed, it just removes the accumulated digest. But digging deeper into the implementation, it looks like h.SetSeed() also removes any accumulated digest, so this call was redundant. Removed.

blindchaser · 2026-03-11T20:53:35Z

sei-db/db_engine/pebbledb/pebblecache/shard_manager.go

+
+// Creates a new Sharder. Number of shards must be a power of two and greater than 0.
+func NewShardManager(numShards uint64) (*shardManager, error) {
+	if numShards <= 0 || (numShards&(numShards-1)) != 0 {


nit, numShards == 0 for uint64

blindchaser · 2026-03-11T20:57:01Z

sei-db/db_engine/pebbledb/pebblecache/lru_queue.go

+}
+
+// Create a new LRU queue.
+func NewLRUQueue() *lruQueue {


Exported constructor returns unexported type lru_queue.go, shard_manager.go

NewLRUQueue() and NewShardManager() are exported but return unexported types. Since both are internal, how about unexport the constructors: newLRUQueue, newShardManager.

constructors are no longer exported

blindchaser · 2026-03-11T20:59:35Z

sei-db/db_engine/pebbledb/pebblecache/lru_queue.go

+	key []byte,
+	// the size of the key + value
+	size int,
+) {


nit: Push() accepts negative values

changed type to uint64

blindchaser · 2026-03-11T21:00:30Z

sei-db/db_engine/pebbledb/pebblecache/noop_cache.go

+		if err != nil {
+			keys[k] = types.BatchGetResult{Error: err}
+		} else {
+			keys[k] = types.BatchGetResult{Value: val, Found: found}


looks the error result is silently swallow here

Made it so that if any of the get calls fail, then this entire method returns an error. Should simplify caller logic (they won't have to scan the entire map before using the results).

blindchaser · 2026-03-11T21:03:30Z

sei-db/db_engine/pebbledb/pebblecache/cache.go

+	// The key to update.
+	Key []byte
+	// The value to set. If nil, the key will be deleted.
+	Value []byte


in the comment, when the value is nil, the key will be deleted. with this, do we still ned IsDelete ?

Good point, simplified

cody-littley · 2026-03-13T16:39:11Z

Is superseded by #3072. All fixes for comments in this PR have been carried over into that new branch.

Helper files for flatKV cache

111459f

cody-littley requested review from blindchaser and yzang2019 March 11, 2026 17:42

cody-littley self-assigned this Mar 11, 2026

cody-littley added the non-app-hash-breaking label Mar 11, 2026

add missing struct

d40395f

blindchaser reviewed Mar 11, 2026

View reviewed changes

Cody Littley and others added 4 commits March 12, 2026 08:10

Merge branch 'main' into cjl/cache-auxilery

c8e85d2

made suggested changes

ed7e4b6

fix tests

5c46647

Merge branch 'main' into cjl/cache-auxilery

e404540

cody-littley mentioned this pull request Mar 13, 2026

Helper files for the flatKV cache implementation #3072

Open

cody-littley closed this Mar 13, 2026

Conversation

cody-littley commented Mar 11, 2026

Describe your changes and provide context

Testing performed to validate your change

Uh oh!

github-actions bot commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 11, 2026

Uh oh!

codecov bot commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cody-littley commented Mar 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions bot commented Mar 11, 2026 •

edited

Loading

codecov bot commented Mar 11, 2026 •

edited

Loading