Skip to content

SNOW-3930192: Recognize any scheme:// in split_statements, not just file:// - #3001

Merged
sfc-gh-fpawlowski merged 1 commit into
mainfrom
SNOW-3930192-split-statements-truncates-urls-with-scheme
Aug 18, 2026
Merged

sfc-gh-fpawlowski merged 1 commit into
mainfrom
SNOW-3930192-split-statements-truncates-urls-with-scheme

Conversation

@sfc-gh-fpawlowski

Copy link
Copy Markdown
Contributor
  1. What GitHub issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

    SNOW-3930192 — follow-up to SNOW-3772985 (SNOW-3772985: Recognize // line comments in split_statements #2944), which shipped in 4.7.2.

  2. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am modifying authorization mechanisms
    • I am adding new credentials
    • I am modifying OCSP code
    • I am adding a new dependency
  3. Please describe how your code solves the related issue.

The bug

4.7.2 (SNOW-3772985, 57faef8) added // to INLINE_COMMENT_PREFIXES so // line comments are recognized alongside --. To keep file:// stage paths working, it guarded both the line-comment and block-comment branches with a literal "file://" check.

That guard only covers file://, so every other URL scheme containing :// is truncated at the first //:

from io import StringIO
from snowflake.connector.util_text import split_statements

for url in ["http://example.com/path", "https://example.com/path",
            "s3://bucket/key", "snow://db.schema.stage/path"]:
    result, _ = next(split_statements(StringIO(url), remove_comments=True))
    print(f"{url!r:45}  =>  {result!r}")

On 4.7.2:

'http://example.com/path'          =>  'http:'
'https://example.com/path'         =>  'https:'
's3://bucket/key'                  =>  's3:'
'snow://db.schema.stage/path'      =>  'snow:'
'file:///path/to/file'             =>  'file:///path/to/file'   # only this one protected

The fix

Both guards now test whether the current token contains ://, instead of matching one hardcoded scheme:

elif (
    line[col:].startswith(INLINE_COMMENT_PREFIXES)
    and "://" not in line[col0 : col + 3]
):
...
elif line[col:].startswith("/*") and "://" not in line[col0 : col + 3]:

Three decisions the tests pin down, each verified by running the suite against the alternative:

  • The check is scoped to the current token (col0), not to the single character before the marker. file:///path contains two overlapping // pairs, and the second is preceded by /, not :. A one-character lookback still truncates it to file:/ — that variant fails the existing test_file_with_slash_star and test_space_before_put.

  • The /* block-comment guard needs the same fix, not just the // one. It carries the identical "file://"-only check. Once // is fixed for other schemes that branch becomes reachable and misreads globs like s3://bucket/*.csv as a block-comment start (yielding put s3://bucket). test_scheme_url_with_glob_not_treated_as_block_comment covers exactly this: it is the one test that separates a complete fix from a //-only fix.

  • A substring check rather than a scheme-shaped regex (^[A-Za-z][A-Za-z0-9+.-]*://). The regex rejects scheme-like tokens that are not valid RFC 3986 schemes, so my_scheme://bucket/*.csv would still be truncated to my_scheme:. The substring check leaves it intact and is identical across both guards.

Requiring whitespace before the marker was also considered and rejected: it breaks the existing test_remove_comments_end_of_line (L_SHIPINSTRUCT-- this is a comment), since -- and // start a comment with no preceding whitespace per SQL comment semantics.

Tests

test/unit/test_split_statement.py gains two parametrized tests (9 cases). Measured behaviour of the suite against each code version:

Code version Result
This PR 39/39 pass
4.7.2 (buggy) 9 new cases fail
//-guard-only half-fix exactly the 3 glob cases fail

Genuine comments are unaffected — select 1 -- c, select 1 // c, no-space word//comment, x::int//comment, and comments whose text contains a URL (-- see http://a.com) are all still stripped.

Existing coverage is left untouched; the test-file diff is purely additive.

🤖 Generated with Claude Code

…ile://

4.7.2 (SNOW-3772985) added `//` to INLINE_COMMENT_PREFIXES so `//` line
comments are recognized alongside `--`. To keep `file://` stage paths
working, it guarded both the line-comment and block-comment branches with
a literal `"file://"` check.

That guard only covered `file://`, so every other URL scheme containing
`://` was truncated at the first `//`:

    'http://example.com/path'     => 'http:'
    's3://bucket/key'             => 's3:'
    'snow://db.schema.stage/path' => 'snow:'

Both guards now test whether the current token contains `://` instead of
matching one hardcoded scheme.

Two details the tests pin down:

- The check is scoped to the current token (`col0`), not to the single
  character before the marker. `file:///path` contains two overlapping
  `//` pairs, and the second is preceded by `/`, not `:`, so a one-char
  lookback still truncates it to `file:/` (this breaks the existing
  test_file_with_slash_star and test_space_before_put).

- The `/*` guard needs the same fix, not just the `//` one. Once `//` is
  fixed for other schemes, that branch becomes reachable and misreads
  globs like `s3://bucket/*.csv` as a block-comment start.

A substring check is used rather than a scheme-shaped regex so that
scheme-like tokens that are not valid RFC 3986 schemes (e.g.
`my_scheme://bucket/*.csv`) are also left intact.

Requiring whitespace before the marker was considered and rejected: it
breaks the existing test_remove_comments_end_of_line, since `--` and `//`
start a comment with no preceding whitespace per SQL comment semantics.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@sfc-gh-fpawlowski
sfc-gh-fpawlowski marked this pull request as ready for review August 12, 2026 12:33
@sfc-gh-fpawlowski
sfc-gh-fpawlowski requested a review from a team as a code owner August 12, 2026 12:33
@codecov-commenter

codecov-commenter commented Aug 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.09%. Comparing base (869bcf9) to head (5a02b2e).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3001      +/-   ##
==========================================
+ Coverage   65.06%   65.09%   +0.02%     
==========================================
  Files         123      123              
  Lines       17452    17452              
  Branches     2398     2398              
==========================================
+ Hits        11356    11361       +5     
+ Misses       5655     5651       -4     
+ Partials      441      440       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sfc-gh-fpawlowski
sfc-gh-fpawlowski merged commit 8bd19ee into main Aug 18, 2026
53 of 56 checks passed
@sfc-gh-fpawlowski
sfc-gh-fpawlowski deleted the SNOW-3930192-split-statements-truncates-urls-with-scheme branch August 18, 2026 08:00
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants