Repository navigation
Conversation
sqlparser parses `expr [NOT] RLIKE pattern` and `expr [NOT] REGEXP pattern` into `Expr::RLike`, but `SqlToRel` had no arm for it and refused the query with "Unsupported ast node in sqltorel". Plan it as a call to `regexp_like`, resolved through the function registry the way an explicit `regexp_like(...)` call is, and wrap it in `NOT` when negated. A session that overrides `regexp_like` gets its override for all 3 spellings; a session without it gets a planning error naming the function. Explicit calls and `RLIKE`/`REGEXP` share a new `plan_scalar_udf` helper, so registered `ExprPlanner::plan_scalar` hooks see every spelling.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
No existing issue. This PR states the problem and a reproduction below.
Rationale for this change
sqlparser parses
expr [NOT] RLIKE patternandexpr [NOT] REGEXP patternfor every dialect, but DataFusion refuses both:Expected:
true. DataFusion already has the semantics asregexp_like(expr, pattern). Spark, Hive and MySQL users write these operators directly.What changes are included in this PR?
SqlToRelplansexpr [NOT] RLIKE|REGEXP patternas[NOT] regexp_like(expr, pattern).regexp_likeresolves throughContextProvider::get_function_meta, the same lookup as an explicit call. A session that overridesregexp_likegets its override for all 3 spellings. A session without it gets a planning error namingregexp_like.plan_scalar_udfhelper. RegisteredExprPlanner::plan_scalarhooks therefore seeRLIKE,REGEXPand their negations, just as they seeregexp_like(...). The helper keeps a planner's result distinct from the default call, so a planner's rewrite is returned as is and only the default call gets the alias for a function called by another name.Operator::RegexMatch. The unparser rendersRegexMatchas~for every dialect, while a function call can be renamed per dialect. The built-in'ssimplifystill rewrites it to~for execution.RLIKEandNOT RLIKEin the operators guide.What is the testing strategy for this PR?
regexp/regexp_like.slt:RLIKE,REGEXP,NOT RLIKEandNOT REGEXPover the shared test data, a NULL subject, a NULL pattern, a column pattern, and row-for-row agreement withregexp_like(...).sql_integration.rs:select_rlike_plans_as_regexp_like: registry resolution and plan shape.select_rlike_without_regexp_like: the planning error.select_rlike_with_expr_planner: anExprPlannerthat rewritesregexp_liketo~applies to the explicit call,RLIKE,REGEXP,NOT RLIKEandNOT REGEXP.select_expr_planner_rewrite_to_other_function: anExprPlannerthat rewritesregexp_liketo another scalar function keeps that function's output name, for the explicit call and forRLIKE, with no alias back toregexp_like(...).main.Are there any user-facing changes?
Yes:
RLIKEandREGEXPnow plan instead of erroring. Documented in the operators guide.ExprPlanner::plan_scalarnow also sees these operators. No public API change.