Skip to content

globset: add fast path for case-insensitive literal/basename patterns - #3488

Open
dhmztr wants to merge 1 commit into
BurntSushi:masterfrom
dhmztr:fix/globset-case-insensitive-literal-perf
Open

globset: add fast path for case-insensitive literal/basename patterns#3488
dhmztr wants to merge 1 commit into
BurntSushi:masterfrom
dhmztr:fix/globset-case-insensitive-literal-perf

Conversation

@dhmztr

@dhmztr dhmztr commented Jul 23, 2026

Copy link
Copy Markdown

Summary

Addresses #1086: case-insensitive glob patterns that are pure literals (no wildcards) — e.g. a bare filename with .case_insensitive(true) — were always falling through to full regex matching, even though there's no wildcard to justify it. On a large pattern set this is orders of magnitude slower than the literal fast path already used for case-sensitive literals (see the benchmark in the issue).

Scope note: this covers the Literal and BasenameLiteral strategies, which is what the issue's own ignore::overrides benchmark exercises. ext()/prefix()/suffix() are intentionally left as-is (still falling back to regex for case-insensitive patterns) — ExtensionStrategy and the prefix/suffix MultiStrategyBuilder have no case-insensitive comparison mode, and wiring case-insensitive literals into them without that support would silently produce wrong matches (e.g. a case-insensitive *.TXT pattern failing to match a real photo.TXT file). Extending those is a reasonable follow-up but is a larger, separate change — didn't want to bundle a correctness risk into a perf fix.

Changes

  • literal() / basename_literal() no longer bail out early on case_insensitive; they ASCII-lowercase the extracted literal and let it flow into the existing Literal/BasenameLiteral match strategies instead of forcing MatchStrategy::Regex.
  • LiteralStrategy / BasenameLiteralStrategy gained a case-insensitive mode: literals are lowercased on insert, lookups lowercase the candidate path/basename before hashing.
  • GlobSet::new() now builds separate case-sensitive and case-insensitive instances of each strategy and routes each pattern into the right one.
  • Lowercasing is ASCII-only (to_ascii_lowercase, not to_lowercase) to match this crate's actual case-insensitive semantics — regex-syntax is built here without the unicode-case feature, so even the pre-existing (?i) regex fallback can't fold non-ASCII case pairs (confirmed directly: (?i) on a pattern with a non-ASCII literal fails to compile at all in this build with UnicodeCaseUnavailable). Using Unicode-aware lowercasing in the new fast path would have made it behave differently from the regex path it replaces for the same logical pattern.

Test plan

  • cargo test -p globset — 293 passed, 0 failed (added case-insensitive literal/basename correctness tests, plus fixed two pre-existing tests that encoded the old bail-out-to-None behavior as expected)
  • cargo test -p ignore (direct consumer) — 182 unit + 5 integration + 7 doctests, all pass
  • cargo fmt -p globset -- --check — clean
  • cargo clippy -p globset --all-targets -- -D warnings — no new warnings introduced by this diff (24 pre-existing warnings elsewhere in the crate, unrelated to this change, left untouched)
  • Full workspace build not runnable in this environment (local rustc 1.95, workspace ripgrep/grep-* crates pin 1.96 — unrelated to this change; globset itself only requires 1.88 and builds/tests fine)

Case-insensitive glob patterns that are pure literals (no wildcards),
e.g. "foo.txt" or "**/FOO" with case_insensitive(true), previously
fell through to full regex matching for every candidate path, even
though the pattern has no wildcards at all. On large pattern sets this
is orders of magnitude slower than the literal fast path already used
for case-sensitive literals (see benchmark in BurntSushi#1086).

- literal()/basename_literal() no longer bail out early on
  case-insensitive patterns; they now ASCII-lowercase the extracted
  literal and let it flow into the existing Literal/BasenameLiteral
  match strategies.
- LiteralStrategy/BasenameLiteralStrategy gained a case-insensitive
  mode: literals are lowercased on insert, and lookups lowercase the
  candidate path/basename before hashing.
- GlobSet::new() now builds separate case-sensitive and
  case-insensitive instances of each strategy, routing each pattern
  into the right one.
- ext()/prefix()/suffix() are intentionally left untouched (still
  bail out on case-insensitive patterns) since their strategies
  (ExtensionStrategy, prefix/suffix MultiStrategyBuilder) don't have
  a case-insensitive comparison mode; wiring case-insensitive
  literals into them without that support would silently produce
  wrong matches for real files with different-case extensions.
- ASCII-only lowercasing (not Unicode `to_lowercase()`) to match the
  existing behavior of this crate's regex fallback, which is built
  without the regex-syntax `unicode-case` feature and cannot fold
  non-ASCII case pairs either (confirmed: even the pre-existing
  `(?i)` regex path fails to compile for non-ASCII case-insensitive
  literals in this build configuration).

Addresses the literal and basename-literal cases from BurntSushi#1086, which
cover the reported `ignore::overrides` benchmark scenario. Extension,
prefix, and suffix case-insensitive patterns still fall back to regex
and are left for a follow-up, so this does not fully close BurntSushi#1086.
Copilot AI review requested due to automatic review settings July 23, 2026 23:22

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves globset performance for case-insensitive glob patterns that are pure literals (full-path literals and basename-only literals) by avoiding the regex-based fallback and using dedicated literal match strategies instead.

Changes:

  • Route case-insensitive literal and basename-literal patterns into dedicated LiteralStrategy / BasenameLiteralStrategy instances.
  • Add case-insensitive support to LiteralStrategy and BasenameLiteralStrategy by ASCII-lowercasing stored keys and lowercasing candidates on lookup.
  • Update Glob literal/basename-literal extraction to support case-insensitive literals (ASCII-only) and adjust tests accordingly.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
crates/globset/src/lib.rs Splits literal/basename-literal strategies into case-sensitive and case-insensitive variants and wires routing in GlobSet::new().
crates/globset/src/glob.rs Enables extracting literals/basename-literals for case-insensitive globs (ASCII lowercased) and updates matching/tests.
Comments suppressed due to low confidence (2)

crates/globset/src/lib.rs:771

  • matches_into also allocates a lowercased Vec for every case-insensitive candidate lookup. Similar to is_match, it can avoid allocating for already-lowercase candidates by attempting get with the original bytes first and only lowercasing on a miss.
        let hits = if self.1 {
            self.0.get(&candidate.path.to_ascii_lowercase())
        } else {
            self.0.get(candidate.path.as_bytes())
        };

crates/globset/src/lib.rs:823

  • BasenameLiteralStrategy::matches_into allocates a lowercased Vec on every case-insensitive lookup. As with is_match, you can avoid allocations for already-lowercase basenames by attempting get with the original bytes first and only lowercasing on miss.
        let hits = if self.1 {
            self.0.get(&candidate.basename.to_ascii_lowercase())
        } else {
            self.0.get(candidate.basename.as_bytes())
        };

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/globset/src/lib.rs
Comment on lines 749 to +753
fn is_match(&self, candidate: &Candidate<'_>) -> bool {
self.0.contains_key(candidate.path.as_bytes())
if self.1 {
self.0.contains_key(&candidate.path.to_ascii_lowercase())
} else {
self.0.contains_key(candidate.path.as_bytes())
Comment thread crates/globset/src/lib.rs
Comment on lines 795 to +799
fn is_match(&self, candidate: &Candidate<'_>) -> bool {
if candidate.basename.is_empty() {
return false;
}
self.0.contains_key(candidate.basename.as_bytes())
if self.1 {
Comment thread crates/globset/src/lib.rs
Comment on lines +485 to +489
let case_insensitive = p.is_case_insensitive();
match MatchStrategy::new(p) {
MatchStrategy::Literal(lit) => {
lits.add(i, lit);
if case_insensitive {
lits_ci.add(i, lit);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants