Fix quadratic key disambiguation in lock files

[fzakaria]

Motivation

I am working on a new idea 💡 and I ran into this performance problem when evaluating many Nix flakes.

LockFile::toJSON() gives every node a unique key, disambiguating collisions by appending _2, _3, … The search restarts at 2 on every collision:

if (!keys.insert(key).second) {
    for (int n = 2;; ++n) {
        auto k = fmt("%s_%d", key, n);
        if (keys.insert(k).second) { key = k; break; }
    }
}

So the k-th node sharing a base name costs k iterations, making the pass O(n²) in the number of colliding nodes.

This is easy to hit in practice. A flake that depends on many other flakes accumulates one systems / flake-compat / nixpkgs node per dependency, and they all share a base name. In a 4000-input flake, 3999 nodes are named systems_2 … systems_4000, costing ~8M fmt() calls.

The cost is not confined to writing the file. flake.cc passes the serialized lock to call-flake.nix on every evaluation: nix eval, nix flake metadata and nix flake lock are all O(n²) for this name determination

Context

I hit this building a flake with a very large number of inputs. Evaluating an output that touches no input at all took 11.5s, which was surprising given inputs are lazy.

The fix

Track the highest suffix already handed out per base key and resume from there instead of rescanning from 2. The keys.insert check is retained, so a name that legitimately appears in the lock file (an input actually called systems_2) is still skipped and the resulting keys are unchanged.

Benchmarks

A flake with n inputs that each pull in their own systems, timing nix eval of an output touching no input:

inputs	nodes	before	after	speedup
500	1,000	0.64s	0.15s	4.3x
1,000	2,000	0.88s	0.16s	5.5x
2,000	4,000	3.05s	0.28s	10.9x
4,000	8,000	11.49s	0.54s	21.3x

The before column is quadratic (2x inputs → ~3.8x time); the after column is linear.

Reproducer:

python3 -c '
REV="11707dc2f618dd54ca8739b309ec4fc024de578b"
n=4000
L=["{"]
for i in range(n): L.append(f"  inputs.f{i}.url = \"github:numtide/flake-utils/{REV}\";")
L.append("  outputs = { self, ... }@inputs: { trivial = \"hi\"; };")
L.append("}")
open("flake.nix","w").write("\n".join(L))'
git init -q . && git add -N flake.nix && nix flake lock
time nix eval .#trivial

Correctness

Lock files produced before and after are byte-identical, verified on:

• the synthetic cases above (500/2000/4000 inputs, ~3.9MB lock)
• an adversarial lock containing pre-existing systems_2, systems_3 and systems_5 inputs interleaved with 41 auto-disambiguated systems nodes
• an existing real-world flake's flake.lock, regenerated and compared

Priorities

Add 👍 to pull requests you find important.