Describe the bug
Expression simplification rewrites col ~ '.*' to col IS NOT NULL, which loses SQL NULL semantics. For a NULL input, NULL ~ '.*' is NULL (three-valued logic), but NULL IS NOT NULL is false. So in a projection context the rewrite returns false where the correct answer is NULL.
The !~ (RegexNotMatch) branch of the same rule is already NULL-aware (col IS NULL AND NULL); only the ~ (RegexMatch) branch drops the NULL. Same class as #24246, but a different rule (regex .*).
To Reproduce
SELECT s, s ~ '.*' AS m
FROM (VALUES (CAST(NULL AS VARCHAR)), ('x')) t(s);
Actual:
+---+-------+
| s | m |
+---+-------+
| | false | <- wrong, should be NULL
| x | true |
+---+-------+
Expected (what NULL semantics require):
+---+------+
| s | m |
+---+------+
| | NULL |
| x | true |
+---+------+
Expected behavior
col ~ '.*' is true for a non-NULL string and NULL for a NULL input, matching NestedLoopJoin-free evaluation.
Additional context
Root cause: simplify_regex_expr in datafusion/optimizer/src/simplify_expressions/regex.rs rewrites the ~ '.*' case to left.is_not_null(). In a WHERE filter this is fine (both FALSE and NULL reject the row, which is why existing SLT filter tests didn't catch it), but it is wrong in a general/projection context. Fix + regression test coming.
Describe the bug
Expression simplification rewrites
col ~ '.*'tocol IS NOT NULL, which loses SQL NULL semantics. For a NULL input,NULL ~ '.*'isNULL(three-valued logic), butNULL IS NOT NULLisfalse. So in a projection context the rewrite returnsfalsewhere the correct answer isNULL.The
!~(RegexNotMatch) branch of the same rule is already NULL-aware (col IS NULL AND NULL); only the~(RegexMatch) branch drops the NULL. Same class as #24246, but a different rule (regex.*).To Reproduce
Actual:
Expected (what NULL semantics require):
Expected behavior
col ~ '.*'istruefor a non-NULL string andNULLfor a NULL input, matchingNestedLoopJoin-free evaluation.Additional context
Root cause:
simplify_regex_exprindatafusion/optimizer/src/simplify_expressions/regex.rsrewrites the~ '.*'case toleft.is_not_null(). In a WHERE filter this is fine (both FALSE and NULL reject the row, which is why existing SLT filter tests didn't catch it), but it is wrong in a general/projection context. Fix + regression test coming.