Revisit nulls in blocking_key: treat as missing data, not isolated block

[nickteff]

Problem

Null in blocking_key is missing data—we don’t know which block that row belongs to. Current behavior: nulls are put in one isolated block (nulls only match nulls). That is likely the wrong direction: we’re forcing “unknown” rows to match only other “unknown” rows, when in reality they could match rows in any block.

Goal

Decide and implement a better way to handle rows with null in the blocking key(s). Open question: what is the right semantics and how do we avoid both wrong results and a comparison explosion?

Possible directions (to be decided)

• Cross-block: Rows with null in blocking_key are compared against every block (or against all other rows). Finds more potential matches but can be expensive (many comparisons).
• Per-block inclusion: For each non-null block, include rows with null blocking key as candidates in that block (nulls can match within each block). Same idea as cross-block but structured per block; cost depends on number of blocks and null count.
• Exclude from blocking: Rows with null blocking key are not blocked—they are matched against the full other side (or a subset). Expensive if there are many nulls.
• Parameter: Expose a choice (e.g. blocking_key_nulls="isolate" | "cross_block" | "exclude") so users can pick semantics and pay the cost they accept. Default TBD (could stay "isolate" for backward compatibility until we’re confident in another default).
• Other: There may be a better approach (e.g. only compare null-block rows to a sample of other blocks, or something else).

Tasks

• Design: Discuss and decide the desired semantics for nulls in blocking_key (and document trade-offs: correctness vs cost). Consider backward compatibility if we change the default.
• Implement: Change block construction / matching so null rows are handled according to the chosen behavior. Update tests (current test expects nulls in one block; adjust or add tests for the new behavior).
• Document: README and docstrings: describe how nulls in blocking key(s) are handled and when to expect higher cost.

Reference

TECH_REVIEW_AND_STRESS_TEST_PLAN.md §1.4 (Nulls in blocking_key). Current implementation: e.g. _paired_blocks_by_key treats block_val is None as one block; test_match_blocking_key_nulls_form_one_block in test_core.py.