Avoid per-row pattern allocation in regexp_is_match for repeated patterns

[yongster]

Is your feature request related to a problem or challenge?

regexp_is_match already caches compiled Regex values in a HashMap<String, Regex>, but it builds an owned String for every row before consulting that cache:

• no flags: pattern.to_string()
• with flags: format!("(?{flag}){pattern}")

Cache misses also get and then entry, hashing the key twice. For a low-cardinality pattern column (one or a few patterns applied to many rows) Regex compilation is already avoided, but the per-row allocation and copy remain. On short strings that fixed cost dominates.

The no-flags path is also boxed as Box<dyn Iterator<Item = Option<String>>>, which is unnecessary.

regexp_is_match_scalar already compiles once and does not have this overhead. The array/array kernel should be able to get much closer to that for repeated patterns.

Describe the solution you'd like

A focused change to regexp_is_match only:

• No-flags path: HashMap<&str, Regex> so a cache hit allocates nothing.
• Flags path: borrowed key (&str, Option<&str>); build (?flags)pattern only on Vacant.
• Use HashMap::entry for a single lookup.
• Specialize the no-flags path instead of a boxed iterator.
• Do not change regexp_match or the scalar kernel in the same PR (regexp_match is dominated by result construction; a borrowed-key prototype did not help there).
• Tests for repeated patterns, repeated flags, null/empty patterns, and Utf8 / LargeUtf8 / Utf8View.

Independent remeasurement on arm64 macOS against 381eea177, 65,536 one-byte values, one repeated pattern. Prototype output matched the current kernel.

Case	Current	Borrowed-key prototype	Speedup
repeated pattern, no flags	1.906 ms	1.033 ms	1.85×
repeated pattern + flags "i"	2.980 ms	1.556 ms	1.91×
regexp_is_match_scalar (same pattern)	250 µs	—	current array path is 7.6× slower than scalar

Describe alternatives you've considered

• Interning owned String keys. That still allocates on the first insert of each distinct pattern, and hits still have to construct the lookup String.
• A small linear cache instead of HashMap. Fine for tiny cardinality, but HashMap<&str, Regex> is already the natural fit and matches the current design.
• Folding regexp_match into the same PR. Local checks showed no benefit; keep that kernel separate.

Additional context

#5235 already removed per-row Regex clones. #5246 added a scalar path for regexp_match. Neither addresses the per-row String allocated before the cache lookup in regexp_is_match.

The in-tree regexp_kernels bench only covers regexp_match, not regexp_is_match. A PR should add a repeated-pattern regexp_is_match bench.

[adriangb]

A note on an adjacent cost in the same module, in case it is useful context for this work. It is not a request to widen the scope of this issue.

The scalar path avoids the per-row work described here, but it compiles the pattern once per call. A caller that applies one literal pattern to a long scan calls the kernel once per batch, so the same pattern is compiled again for every batch.

I measured the regex crate directly (release build, arm64 macOS, 8192 rows of ~60 characters, 2000 compiles per pattern), to compare compilation with matching for one batch:

pattern	compile	match 8192 rows	compile share
abc	1.55 µs	90.6 µs	1.7%
^[a-z]+@[a-z]+\.[a-z]{2,}$	6.58 µs	98.9 µs	6.2%
(?i)error|warn(ing)?	62.0 µs	232.5 µs	21.1%
\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?Z?	249.5 µs	103.7 µs	70.7%

For the last pattern, compilation is more expensive than the matching it enables.

The interesting part is that arrow-string already contains the object that solves this. Predicate in predicate.rs analyses a pattern once, holds the compiled Regex, and evaluates a whole array (evaluate_array). The LIKE family uses it. The regexp module has no equivalent, and Predicate is pub(crate).

A public prepared type for the regexp kernels would let a caller compile once and reuse across batches:

pub struct RegexPredicate { /* opaque */ }

impl RegexPredicate {
    pub fn try_new(pattern: &str, flags: Option<&str>) -> Result<Self, ArrowError>;
    pub fn is_match(&self, array: &dyn Array) -> Result<BooleanArray, ArrowError>;
    pub fn matches(&self, array: &dyn Array) -> Result<ArrayRef, ArrowError>;
}

regexp_is_match_scalar and regexp_match then become thin wrappers, so existing callers do not change. regex stays a private dependency, because the compiled value is hidden inside an arrow type. RowConverter is a precedent for a public, reusable object of this kind.

It also improves error reporting: the compile failure occurs in try_new, at a time the caller chooses, so a caller with a literal pattern can report an invalid pattern before execution starts instead of during it.

Happy to open a separate issue for that if it is wanted, and to keep it out of the way of this one.