Skip to content

fix(chat-store): unify message kind classification - #205

Merged
jlucaso1 merged 15 commits into
mainfrom
fm/oc-195-message-kinds-r1
Sep 24, 2026
Merged

jlucaso1 merged 15 commits into
mainfrom
fm/oc-195-message-kinds-r1

Conversation

@jlucaso1

@jlucaso1 jlucaso1 commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Share wrapper normalization and message classification across inbound, outgoing, edits, session poll extraction, and storage secret preservation.
  • Classify poll creations V5/V6, album parents, and products; expose MessageKind::of and typed kinds.
  • Retain poll secrets, while leaving V5/V6 non-votable until their extended ballot semantics are supported.
  • Repair recoverable persisted unknown kinds on prepare, only from valid non-tombstoned proto rows; update chat preview kind only when the head timestamp is unambiguous. Repeated repair is idempotent.

Resolves #195. Reviewed current main and #200; neither changes the classifier/storage evidence relevant to this fix.

Local validation

  • cargo fmt --all -- --check passed.
  • Focused chat-store tests passed: poll_album_and_product_kinds_share_wrapper_normalization, wrapped_and_new_poll_kinds_are_classified_on_outgoing_write, prepare_repairs_recoverable_unknown_kind_idempotently, and v5_v6_poll_secrets_survive_storage_compaction.
  • Focused session hydration test passed: cargo test -p oxidezap-session --lib a_stored_poll_creation_hydrates_as_a_votable_poll.
  • Focused clippy did not complete cleanly: it reported existing clippy::nonminimal_bool in unchanged crates/chat-store/src/queries.rs:489; the workspace CI clippy result is still needed.

No live WhatsApp account or E2E receive/restart/vote test was available or run. Previously stripped secrets cannot be reconstructed; V5/V6 are retained/classified but remain non-votable in this change.


Summary by cubic

Unifies message-kind classification across incoming, outgoing, edited, session-rendered, and stored messages, and repairs recoverable unknown kinds already persisted in the database.

  • Shares one normalization and poll-detection pass so wrappers (group mentions, associated children, V4 polls, edited payloads) classify identically across the store and the session's live and hydrated paths, including media, quote, and text extraction.
  • Adds album and product kinds, classifies V5/V6 polls, and exposes MessageKind::of for callers.
  • Keeps V5/V6 poll secrets through storage compaction; V5/V6 polls stay non-votable until their extended ballot semantics are supported, and poll listing now paginates past them so unsupported variants don't consume the requested result count.
  • prepare reclassifies recoverable unknown rows from valid stored protos in bounded id pages, guarded by a versioned marker stored in the new chat_store_meta table, and recomputes the chat preview; rerunning is harmless.

Written for commit 61cd3b4. Summary will update on new commits.

Review in cubic

@coderabbitai

coderabbitai Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 13 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: d7b9e755-ac5a-4067-a70f-5f2bc8ce3a93

📥 Commits

Reviewing files that changed from the base of the PR and between 1a7479a and 61cd3b4.

📒 Files selected for processing (14)
  • crates/chat-store/migrations/2026-09-24-000001_message_kind_repair/down.sql
  • crates/chat-store/migrations/2026-09-24-000001_message_kind_repair/up.sql
  • crates/chat-store/src/lib.rs
  • crates/chat-store/src/materialize.rs
  • crates/chat-store/src/queries.rs
  • crates/chat-store/src/storage_proto.rs
  • crates/chat-store/src/store/mod.rs
  • crates/chat-store/src/types.rs
  • crates/chat-store/tests/outgoing.rs
  • crates/chat-store/tests/polls.rs
  • crates/session/src/whatsapp/convert.rs
  • crates/session/src/whatsapp/mod.rs
  • crates/session/src/whatsapp/ops.rs
  • crates/session/src/whatsapp/tests.rs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-24T15:52:37.312799Z 343656c Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 36baa66acc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/chat-store/src/store/mod.rs Outdated
Comment thread crates/chat-store/src/store/mod.rs Outdated
Comment thread crates/chat-store/src/store/mod.rs Outdated
@jlucaso1

Copy link
Copy Markdown
Member Author

Fixed the CI clippy needless_update findings in the two test fixtures (commit fefa109). Focused local chat-store clippy passes with -D clippy::nonminimal_bool because that unrelated warning is already present in unchanged queries.rs; CI on this updated commit is rerunning.

@jlucaso1

Copy link
Copy Markdown
Member Author

Addressed all three Codex repair comments in commit a31520f: repair now runs once per classifier version via a transactional marker, uses bounded 256-row pages, restores recoverable normalized text/captions as well as kinds, and recomputes each affected chat preview from the actual ordered message rows (preserving activity time). Tombstones and undecodable protos remain untouched. Focused test now checks restored text/preview, retained activity timestamp, tombstone non-resurrection, and idempotence.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ad88205748

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/chat-store/src/materialize.rs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a080514c5a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/chat-store/src/materialize.rs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c842210255

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/chat-store/src/materialize.rs
@jlucaso1

Copy link
Copy Markdown
Member Author

Fixed the latest CI compile error in 2e65c7a: restored the MessageExt trait import still needed by text_content/get_caption after routing the live handler through shared normalization. cargo check -p oxidezap-session --lib now passes locally; CI is rerunning for the pushed head.

@jlucaso1

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2e65c7a037

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/session/src/whatsapp/mod.rs
@jlucaso1

Copy link
Copy Markdown
Member Author

Addressed the final Codex finding in 8??: live text and captions now come from the already-normalized base_msg, not the outer wrapper. cargo check -p oxidezap-session --lib and git diff --check pass. CI is rerunning on the new head.

@jlucaso1

Copy link
Copy Markdown
Member Author

The live wrapper text/caption change is in commit a214f3c (not the placeholder hash in my previous comment).

@jlucaso1

Copy link
Copy Markdown
Member Author

Reconciled main into this branch with a normal merge (no force push). The only content conflict was crates/chat-store/src/store/mod.rs in the migration downgrade test; I combined both branches’ reverse-order downgrade assertions (identity queue → message-kind metadata → identity state → preference provenance). The two branches also had a migration-version collision: main adds 2026-09-24-000000_message_identity_repair_state, so I moved this PR’s migration to 2026-09-24-000001_message_kind_repair. Local cargo fmt --all -- --check, cargo test -p oxidezap-chat-store --all-features, and git diff --cached --check passed. Merge commit: 49c42bf. Hosted CI/Pages have restarted; I’ll monitor them. No merge performed.

@jlucaso1

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 49c42bfcbf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@jlucaso1

Copy link
Copy Markdown
Member Author

A newer main commit (#201, a738c24) landed while the first reconciliation was building, so I merged that base too. The only conflict from this second merge was crates/chat-store/src/queries.rs: I combined #201’s typed MessageRow query with this PR’s keyset cursor filter in poll_messages_page. The second merge is 192268e; PR API now reports base a738c24, mergeable=true, and no merge has been performed. After this reconciliation, cargo test -p oxidezap-chat-store --all-features passed (including #201 regression tests); formatting and diff checks passed. Monitoring checks on the current head.

@jlucaso1

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 192268e98a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/chat-store/src/materialize.rs
Comment thread crates/chat-store/src/materialize.rs
@jlucaso1

Copy link
Copy Markdown
Member Author

Addressed both new Codex findings in 343656c: history hydration now normalizes nested wrappers before deriving media/polls/quotes, and the live fallback emits the same [poll], [album], or [product] labels as hydration. Added focused regressions for nested media+quote hydration and fallback labels. Latest main 1a7479a is merged into the branch; the conflict was confined to the migration downgrade test in crates/chat-store/src/store/mod.rs, where I preserved the reverse-order assertions for #202’s chat-name migration and #195/#204 migrations. Targeted downgrade test passes locally; current hosted checks are rerunning. No merge.

@jlucaso1

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 343656cce5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@jlucaso1

Copy link
Copy Markdown
Member Author

The first hosted Check after the wrapper-hydration fix caught one -D warnings issue: an obsolete MessageExt import in convert.rs. Removed it in 61cd3b4; formatting/diff checks pass. New hosted CI/Pages are running on the latest head. The earlier failure was only the unused import; no behavior change.

@jlucaso1

Copy link
Copy Markdown
Member Author

Final reconciliation status: current base is main at 1a7479a; latest pushed head is 61cd3b4 (61cd3b46a2531eb3ba28bf8897dfec0941ecc29a). All three normal merges are included; the actual conflict files were crates/chat-store/src/store/mod.rs (twice, migration-downgrade test sequencing) and crates/chat-store/src/queries.rs (combined #201’s typed row query with this PR’s poll-page cursor). The downgrade sequence now includes #202 chat-name provenance, then #201/#204 identity queue, #195 classifier metadata, identity state, and preference provenance. The message-kind migration was renumbered to 2026-09-24-000001_message_kind_repair to avoid #204’s 000000 migration-version collision. Local full chat-store tests passed after #201; after #202 the targeted downgrade test passed. Latest hosted CI and Pages are green (10 pass, 2 skipped); all 10 review threads are resolved, and Codex found no major issues on 343656c. PR remains open, cleanly mergeable, and not merged.

@jlucaso1
jlucaso1 merged commit 096c039 into main Sep 24, 2026
12 checks passed
@jlucaso1
jlucaso1 deleted the fm/oc-195-message-kinds-r1 branch September 24, 2026 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chat-store: message_kind files poll creations v5/v6, album parents and products as unknown, and is crate-private

1 participant