feat(search): include _rowid in lance_fts and lance_hybrid_search results

[jackylee-ch]

lance_fts and lance_hybrid_search did not surface Lance's row id, so hits could not be joined back to the dataset. This adds _rowid (UInt64) next to the existing _score/_distance pseudo-columns.

The FTS schema and stream now enable with_row_id() and project _rowid; the hybrid path emits the row ids it already computes while ranking. The C++ binds read the Arrow schema dynamically, so no change is needed there.

Testing

GEN=ninja make test_release — search_functions.test covers _rowid on both functions (fails without the projection, passes with it).

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[jackylee-ch]

Thanks! The direct-path _rowid here is ready. Carrying the same contract through the namespace (query_table) path needs _rowid added to the bind schema in src/lance_search.cpp, which an open PR is currently modifying — so I'm holding that piece to avoid a conflicting change and will extend this PR to cover namespace FTS/vector once that lands. Happy to split it out if you'd rather take the direct-path part first.

[lance-gatekeeper]

⚠️ Gate recommendation: approve with a non-blocking risk.

Your sequencing note prompted me to recalibrate the earlier namespace requirement. This patch can be accepted independently: the direct FTS and hybrid paths return row IDs that join back correctly, and the namespace omission predates this diff.

The non-blocking limitation is that namespace-backed FTS still cannot expose _rowid. I’m withdrawing the requirement to include that extension before merge; namespace support can follow separately.