Skip to content

Fix rsrc static missmatch in snowflake - #502

Open
thoffmann-sf wants to merge 4 commits into
mainfrom
fix_rsrc_static_missmatch_in_snowflake
Open

thoffmann-sf wants to merge 4 commits into
mainfrom
fix_rsrc_static_missmatch_in_snowflake

Conversation

@thoffmann-sf

@thoffmann-sf thoffmann-sf commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Description

Fixed RSRC static missmatch as snowflake does not support wildcard operators in = instead it has to use the keyword like
When the rsrc_static is defined as rsrc_static = '%Test123%' snowflake trests it as string literal in the where clause of src_new_x
` src_new_1 AS (

    SELECT
        hk_1,
        hk_2_hub,
        hk_3_hub,
        ldts,
        rsrc
    FROM stage src
    
    WHERE 
    
      (src.rsrc= '%Test123%' AND src.ldts> (
        SELECT MAX(max_ldts)
        FROM max_ldts_per_rsrc_static_in_target
        WHERE rsrc_static = '%Test123%' AND max_ldts != TO_TIMESTAMP('8888-12-31T23:59:59', 'YYYY-MM-DDTHH24:MI:SS')
        ))

)`

Therefore not resolving the right entries in a delta load this is fixed by replacing the src.rsrc = with an src.rsrc like
Which I've done in the Hub, Link and nh_link the rec_track_satellite does not need this yet as it's still in the old format like hubs, links and nh_links used to be in

Fixes #(issue)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce.

I've testes the entities cause after a full DV refresh the next daily load produced relationship errors between sat's and links/hubs. I've implemented the changes locally and ran the entities again and the newest delta was successfully loaded

  • Tests you ran

Test Configuration:

  • datavault4dbt-Version: 2.1.0
  • dbt-Version: _cloud, version number newest

Checklist:

  • I have performed a self-review of my code

Fixed RSRC Static missmatch as snowflake does not support wildcard operators in = instead it has to use the keyword like
Fixed RSRC static missmatch as snowflake does not support wildcard operators in = instead it has to use the keyword like
Fixed RSRC static missmatch as snowflake does not support wildcard operators in = instead it has to use the keyword like
Copilot AI lite review requested due to automatic review settings September 10, 2026 09:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Documented * wildcards remain incompatible with SQL LIKE, requiring normalization or contract updates and regression coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Updates Snowflake rsrc_static filtering to use LIKE instead of equality checks.

Changes:

  • Modified hub filtering.
  • Modified link filtering.
  • Modified non-historized link filtering.
File summaries
File Description
macros/tables/snowflake/hub.sql Updates hub source filtering.
macros/tables/snowflake/link.sql Updates link source filtering.
macros/tables/snowflake/nh_link.sql Updates non-historized link filtering.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread macros/tables/snowflake/hub.sql
Comment thread macros/tables/snowflake/link.sql
Comment thread macros/tables/snowflake/nh_link.sql
Adding documentation to the rsrs_static page for the Snowflake special case
Copilot AI review requested due to automatic review settings September 15, 2026 08:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Snowflake _ wildcard handling and inconsistent documented examples must be addressed.

Get a fresh assessment by requesting another Copilot review.

Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 4
  • Review effort level: Lite

WHERE
{% for rsrc_static in rsrc_statics %}
(src.{{ src_rsrc }} = '{{ rsrc_static }}' AND src.{{ src_ldts }} > (
(src.{{ src_rsrc }} like '{{ rsrc_static }}' AND src.{{ src_ldts }} > (
WHERE
{% for rsrc_static in rsrc_statics %}
(src.{{ src_rsrc }} = '{{ rsrc_static }}' AND src.{{ src_ldts }} > (
(src.{{ src_rsrc }} like '{{ rsrc_static }}' AND src.{{ src_ldts }} > (
WHERE
{% for rsrc_static in rsrc_statics %}
(src.{{ src_rsrc }} = '{{ rsrc_static }}' AND src.{{ src_ldts }} > (
(src.{{ src_rsrc }} like '{{ rsrc_static }}' AND src.{{ src_ldts }} > (
Comment on lines +68 to +71
Therefore, when configuring rsrc_static for Snowflake:

- Use `%` as the wildcard character instead of `*`.
- Wildcard-based rsrc_static values are evaluated using the `LIKE` operator.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants