Skip to content

Extract the per-chromosome dynamic-SQL IEJoin machinery into a shared helper #217

Description

@conradbzura

Description

The DuckDB per-chromosome dynamic-SQL machinery — build a string_agg over the distinct chromosomes into a SET VARIABLE __giql_..._<token> = COALESCE((...), '<empty schema>'), then read it back via SELECT ... FROM query(getvariable(...)) AS <alias> — currently lives privately inside src/giql/expanders/intersects_duckdb.py. It routes a column-to-column INTERSECTS join through DuckDB's IE_JOIN by removing the low-cardinality chrom equality per partition.

Two more operators need the same partitioning to reach IE_JOIN: DISJOIN's breakpoint join (#216) and NEAREST's k-nearest sweep. Rather than duplicate the scaffolding, extract the target-agnostic pieces into a shared helper.

Motivation

Avoid three copies of the SET VARIABLE/string_agg/getvariable scaffolding. The operator-specific parts (the per-chromosome SQL template, the chromosome-partition source, the empty-schema fallback, and the outer projection) stay with each expander; only the shared scaffolding is centralized.

Expected Outcome

A small shared utility (e.g. giql/expanders/_per_chrom.py) exposing: the chrom string-literal interpolation constant, a collision-resistant session-variable name generator, the SET VARIABLE ... = COALESCE((SELECT string_agg(<template>, ' UNION ALL ') FROM (<partition>)), '<empty>') builder, and the query(getvariable('<var>')) AS <alias> relation reference. intersects_duckdb.py is refactored to use it with byte-identical emitted SQL (all existing tests stay green — this is a pure refactor with no behavior change). DISJOIN (#216) and NEAREST then reuse the same helper.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

refactorCode restructuring without behavior change

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions