Skip to content

[Fix] stringify: stop re-encoding structural separator dots when encodeDotInKeys is set - #564

Open
spokodev wants to merge 1 commit into
ljharb:mainfrom
spokodev:fix/encodedotinkeys-nesting-separators
Open

[Fix] stringify: stop re-encoding structural separator dots when encodeDotInKeys is set#564
spokodev wants to merge 1 commit into
ljharb:mainfrom
spokodev:fix/encodedotinkeys-nesting-separators

Conversation

@spokodev

Copy link
Copy Markdown
Contributor

With allowDots + encodeDotInKeys, stringify percent-encodes the structural nesting-separator dots, not just the literal dots inside a key. That breaks the documented stringify -> parse round-trip for any nested object whose keys do not contain a literal dot.

Repro

qs.stringify({ a: { b: { c: 'd' } } }, { allowDots: true, encodeDotInKeys: true });
// got:      'a%252Eb.c=d'   (the a->b separator is encoded, the b->c one is not)
// expected: 'a.b.c=d'       (no key has a literal dot, so encodeDotInKeys is a no-op)

qs.parse('a%252Eb.c=d', { allowDots: true, decodeDotInKeys: true });
// got:      { 'a.b': { c: 'd' } }
// expected: { a: { b: { c: 'd' } } }

Deeper nesting clobbers every separator except the last:

qs.stringify({ a: { b: { c: { d: 'e' } } } }, { allowDots: true, encodeDotInKeys: true });
// got: 'a%252Eb%252Ec.d=e'   expected: 'a.b.c.d=e'

The control case ({ allowDots: true } alone) round-trips correctly, so the regression is specific to encodeDotInKeys.

What the option means

Per the README, encodeDotInKeys encodes the dots that appear in the keys of an object:

qs.stringify({ "name.obj": { "first": "John", "last": "Doe" } }, { allowDots: true, encodeDotInKeys: true })
// 'name%252Eobj.first=John&name%252Eobj.last=Doe'

The dot in name.obj is part of the key and must be encoded; the dot before first/last is the separator allowDots inserts and must stay literal. The bug encodes both.

Root cause

lib/stringify.js:

var encodedPrefix = encodeDotInKeys ? String(prefix).replace(/\./g, '%2E') : String(prefix);

prefix is the accumulated key path, so this replace runs again at every level of recursion and re-encodes separator dots that were inherited from parent levels. The literal dots inside an individual key are already handled per segment a few lines down (encodedKey = ... String(key).replace(...)), so the prefix-wide replace is the source of the over-encoding.

Fix

Encode the dots once per key segment in the top-level driver (the same place and the same way nested child keys are already encoded), and let encodedPrefix be the plain accumulated path. Two lines in lib/stringify.js.

Relationship to #562

This is a different defect from the open #562, which fixes the opposite failure: a top-level key that contains a literal dot and has a primitive value was under-encoded ({ 'a.b': 'c' } produced a.b=c instead of a%252Eb=c). #562 never touches the encodedPrefix line, so applying it leaves this over-encoding bug present:

# with only #562's lib change applied to current HEAD:
qs.stringify({ a: { b: { c: 'd' } } }, { allowDots: true, encodeDotInKeys: true })
// still 'a%252Eb.c=d'   (this bug), while #562's own case 'a%252Eb=c' is fixed

The two fixes share one line (moving the per-segment encode into the driver), because removing the prefix-wide replace requires the top-level key's own literal dots to be encoded there instead, otherwise the README name.obj example regresses. The distinct defect addressed here is the prefix-wide re-encoding on the recursion path.

Verification

  • Added a stringify test for the nested separators plus a round-trip assertion; it fails on current HEAD (red) and passes with the fix (green).
  • Full suite green: 1016 assertions pass.
  • npm run readme passes, so every documented encodeDotInKeys example still produces its documented output (the name.obj literal-dot cases are unchanged).
  • npm run lint clean (no new warnings on the changed lines).

…ncodeDotInKeys` is set

With `allowDots` + `encodeDotInKeys`, the nesting-separator dots between
keys were being percent-encoded on each level of recursion, because the
encoding ran over the accumulated key path (`prefix`) rather than the
individual key segment.

    qs.stringify({ a: { b: { c: 'd' } } }, { allowDots: true, encodeDotInKeys: true })
    // before: 'a%252Eb.c=d'   (a->b separator encoded; deeper nesting
    //                          clobbers every separator but the last)
    // after:  'a.b.c=d'       (no key has a literal dot, so it is a no-op)

This broke the documented `stringify` -> `parse` round-trip:

    qs.parse('a%252Eb.c=d', { allowDots: true, decodeDotInKeys: true })
    // before: { 'a.b': { c: 'd' } }
    // after:  { a: { b: { c: 'd' } } }

`encodeDotInKeys` is meant to encode literal dots inside a key, not the
structural `.` that `allowDots` inserts between keys. Encode the dot once
per key segment in the top-level driver (matching how nested child keys
are already encoded) instead of replaying the replace over the inherited
prefix.
@spokodev
spokodev force-pushed the fix/encodedotinkeys-nesting-separators branch from 25239e7 to 78db44d Compare June 29, 2026 09:34
@spokodev

Copy link
Copy Markdown
Contributor Author

Rebased onto main. The top-level-key portion of this is now on main via 708fade (it adds the same encodedKey change I had), so I dropped that hunk. This PR now narrows to the remaining issue: encodedPrefix re-encoded the structural separator dots in prefix on every recursion, so stringify({ a: { b: { c: 'd' } } }, { allowDots: true, encodeDotInKeys: true }) gave a%252Eb.c=d instead of a.b.c=d. The one-line fix drops that re-encode. Full suite green, and the dotted-key cases from 708fade still pass.

ljharb added a commit that referenced this pull request Jul 2, 2026
…vior

With `allowDots` + `encodeDotInKeys`,
stringify currently over-encodes the structural `.` separators between nested keys
(e.g. { a: { b: { c: 'd' } } } -> a%252Eb.c=d),
which breaks the stringify -> parse round-trip.

Related: #564
@ljharb
ljharb force-pushed the fix/encodedotinkeys-nesting-separators branch from 78db44d to 662cd48 Compare July 2, 2026 15:41
@ljharb

ljharb commented Jul 2, 2026

Copy link
Copy Markdown
Owner

I added tests for the current behavior to main, and rebased this PR.

@spokodev

spokodev commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for rebasing. The remaining change still narrows to the encodedPrefix re-encoding on recursion; let me know if you would like any adjustments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants