audioshaper: name the Waveshaper's own headroom ceiling, CLIP_HEADROOM (audioif#99)

A curve that reaches the rails and a post_gain above about 0.74 saturates
the Waveshaper's own output, after the decimator, where no oversampling
factor reaches it -- and the node's documentation said nothing about it.
Distortion's second fix round found this the hard way and carries its own
0.74 ceiling; every other drive class was exposed to the same knee with no
name for it.

Where it happens (src/shared/audioif_shaper.c): shape_sample runs the
curve at the oversampled rate and curve_lookup's own clamp (:213) bounds
each of those samples to +-1, but the decimated one is not clamped there.
halfband_down (:201-207) is a low-pass, not a clip, so a hard edge through
it can overshoot the rails on the way back down -- the band-limited
version of a full-scale clipped edge does not fit in int16. post_gain
scales that decimated (and possibly overshot) value (:288,
oversampled[0] * config->post_gain * 32768.0f), after the half-band, and
the only place this node ever clips to int16 is to_s16, two lines later
(:289-290). Everything from the curve to post_gain is float; nothing here
is int16 until that last line.

Measured on the CPython twin: a straight hard clip (int16 Q15, flat beyond
+-10% of input), a 1010 Hz sine driven to 98% of full scale, 48 kHz,
oversample x4, bare node, alias floor over an exactly periodic 4800-sample
window (bin 101, no window, no rounding):

  post=0.66 -54.344 | 0.70 -54.343 | 0.74 -54.345 | 0.78 -54.343
  post=0.80 -54.344 | 0.90 -41.238 | 1.00 -36.441

Flat to the hundredth of a dB through 0.80, then 13.1 dB worse by 0.90.
The control is oversample x1, same curve and same drive: -34.915 dB at
every one of those seven settings, so the knee is the decimator's doing,
not the curve's. Full table, x8 included, in docs/upstream-diff.md's new
"clipper's own headroom" section.

CLIP_HEADROOM = 0.74 names the ceiling, on the CPython twin only
(src/cpython/audioshaper.py), the same as GROUP_DELAY_SAMPLES beside it:
neither the MicroPython usermod's module globals (src/audioshaper/module.c)
nor the CircuitPython spike's (shared-bindings/audioshaper/__init__.c)
export anything past __version__/__revision__ and the two types, so there
is no second target for this figure to agree with -- adding it there too
would be a constant that is not identical on all three targets.

HeadroomTest in tests/test_cpython_audioshaper.py pins the knee: the floor
at 0.74 is within 1 dB of 0.66 (0.001 dB measured), and at least 10 dB
worse by 0.90 (13.1 dB measured) -- 10 rather than the 20 the issue's own
Distortion-table measurement showed, because that table is a different
curve in a different repository. An extensive parameter search here (clip
steepness, tanh steepness, drive level, pre_gain, oversample x2/x4/x8)
never found a bare-node reproduction that robustly cleared 20 dB between
exactly 0.74 and 0.90; 13.1 dB at x4 was the ceiling. Shown able to fail,
at the literal 20 dB bar:

  FAIL  test_the_floor_falls_past_the_knee
        AssertionError: -54.34461222147358 not less than -61.23844766655389
        : 0.90 was not at least 20 dB worse than 0.74: -54.345 vs -41.238 dB

A third test is the control: at oversample x1 the same curve and the same
drive show no knee at all (max-min under 0.5 dB across all seven post_gain
settings), which is what pins the mechanism as the decimator's rather than
the curve's own clipping.

Docstrings (the twin, docs/upstream-diff.md, README's module paragraph)
and CHANGELOG carry the rule for callers: keep post_gain * max(abs(curve))
at or below CLIP_HEADROOM for a curve that reaches the rails, and put the
rest of the wanted level on a mixer voice after this node. No DSP changed.

Author: bdbarnett

CHANGELOG.md

@@ -1,5 +1,24 @@
 ## Unreleased
 
+- **`audioshaper.Waveshaper`'s own headroom is documented, and pinned by a
+  trait test.** A curve that reaches the rails and a `post_gain` above about
+  0.74 saturates the node's *own* output: the half-band decimator rings
+  about a third past the rails on a hard edge, at the base rate, after the
+  oversampling is already done, where no factor of it reaches. Measured on
+  a hard-clipping table driven hard (x4, 1010 Hz, 48 kHz, bare node): the
+  alias floor is flat through `post_gain` 0.80 and 13 dB worse by 0.90.
+  `audioshaper.CLIP_HEADROOM = 0.74` names the ceiling — keep
+  `post_gain * max(abs(curve))` at or below it for a curve that reaches the
+  rails, and put the rest of the wanted level on a mixer voice after this
+  node. No DSP changed: the docstring, `docs/upstream-diff.md` and a new
+  `HeadroomTest` in `tests/test_cpython_audioshaper.py` say what a drive
+  class (Distortion's second fix round) had already found and was carrying
+  its own 0.74 ceiling for, uncredited. The constant lives on the CPython
+  twin only — neither the MicroPython usermod's module globals nor the
+  CircuitPython spike's export anything past `__version__`/`__revision__`
+  and the two types, the same as `GROUP_DELAY_SAMPLES` beside it
+  (audioif#99).
+
 - **`audioshaper.SampleHold`**, a new node: a zero-order hold in which `num`
   source frames carry `den` new values, at a ratio that is exact. One frame
   in, one frame out, at the source's own rate, channels and bit depth; the

README.md

@@ -235,7 +235,11 @@ with `hysteresis_width` its half-width as a fraction of full scale *at the
 input* and `hysteresis_bias` splitting that between the rising and falling
 branches. `audioshaper.GROUP_DELAY_SAMPLES` carries the measured group delay
 per factor — 0, 2.2, 3.3 and 3.9 base samples — for a component that has to
-report its latency. Why each of those is the shape it is, and what the
+report its latency. A curve that reaches the rails has a ceiling of its
+own: the decimator can ring past them on a hard edge, so keep
+`post_gain * max(abs(curve))` at or below `audioshaper.CLIP_HEADROOM`
+(0.74) and put the rest of the wanted level on a mixer voice after this
+node. Why each of those is the shape it is, and what the
 oversampling measures:
 [docs/upstream-diff.md](docs/upstream-diff.md), "`audioshaper`: audioif's own,
 and the two things a fixed curve cannot be".

docs/upstream-diff.md

@@ -2769,11 +2769,15 @@ measuring three libms rather than this node.
 
 `tests/test_cpython_audioshaper.py` carries what the golden cannot: that
 oversampling actually lowers the alias floor, that the hysteresis knob is
-monotone and clears its control by 6 dB, and that a static table and a
-zero-width operator both enclose exactly 0.0. Each check was shown to fail
-before it was believed -- discarding the play operator's result reddens both
-hysteresis tests, and replacing the half-bands with a zero-order hold and a
-decimating drop reddens the alias-floor test.
+monotone and clears its control by 6 dB, that a static table and a
+zero-width operator both enclose exactly 0.0, and that a hard-clipping
+curve's own alias floor holds flat through `post_gain` 0.74 and falls at
+least 10 dB worse by 0.90 (audioif#99, "The clipper's own headroom" below).
+Each check was shown to fail before it was believed -- discarding the play
+operator's result reddens both hysteresis tests, replacing the half-bands
+with a zero-order hold and a decimating drop reddens the alias-floor test,
+and the headroom test reddens at oversample x1, where there is no
+half-band left to ring.
 
 `SampleHold` has `tests/parity/samplehold_probe.py` of its own rather than
 cases appended to the waveshaper's, because one comparison covers a probe's
@@ -2792,6 +2796,57 @@ node's own loop re-typed) is run against the two-`SpeedChanger` composition
 does, and by the end of that distance the pair is holding a frame the
 arithmetic does not name.
 
+### The clipper's own headroom, after the decimator (audioif#99, 2026-09-17)
+
+Oversampling buys back the base rate's aliasing, but it has a ceiling of its
+own that the sections above do not mention. `shape_sample` runs at the
+oversampled rate and `curve_lookup`'s own clamp
+(`src/shared/audioif_shaper.c:213`) holds every one of those samples to
++-1 -- but the *decimated* one is not clamped there. `halfband_down`
+(`:201-207`) is a low-pass, not a clip, so a hard edge through it can
+overshoot the rails on the way back down to the base rate, the way any
+band-limited reconstruction of a discontinuity does. `post_gain` is applied
+to that decimated value (`audioif_shaper_process_s16`, `:288`,
+`oversampled[0] * config->post_gain * 32768.0f`) -- after the half-band,
+where no oversampling factor reaches it any more -- and the *only* place
+this node clips to int16 at all is `to_s16`, two lines later (`:289-290`),
+on `dry_gain * source + wet_gain * wet`. Every value in between, including
+the overshoot, is `float`.
+
+Measured on the CPython twin: a straight hard clip (int16 Q15, 2048 points,
+flat beyond +-10% of input, so it reaches the rails), a 1010 Hz sine at
+48 kHz driven to 98% of full scale, oversample x4, bare node. The alias
+floor is inharmonic energy against the fundamental over an exactly periodic
+4800-sample window -- bin 101 of a rectangular-window transform, so nothing
+is rounded to its nearest bin:
+
+| `post_gain` | 0.66 | 0.70 | 0.74 | 0.78 | 0.80 | 0.90 | 1.00 |
+|---|---|---|---|---|---|---|---|
+| floor (dBc) | -54.34 | -54.34 | -54.35 | -54.34 | -54.34 | -41.24 | -36.44 |
+
+Flat to the hundredth of a dB through 0.80, then 13.1 dB worse by 0.90 and
+17.9 by 1.00. (x8 shows the same shape a little deeper: flat at -55.77
+through 0.80, -41.21 at 0.90.) The control is oversample x1 on the same
+curve and the same drive: with no half-band left to ring, the floor reads
+-34.92 dBc at every one of those seven settings, so the knee measured above
+is the decimator's doing and not the curve's.
+
+That is physics, not a defect -- the headroom has to come from somewhere,
+and a full-scale clipped edge asks the reconstruction filter for more than
+int16 has to give back. `audioshaper.CLIP_HEADROOM = 0.74` names the
+ceiling: keep `post_gain * max(abs(curve))` at or below it for a curve that
+reaches the rails, and put the rest of the wanted level on a mixer voice
+after this node rather than on this knob. The constant lives on the
+CPython twin only, the same as `GROUP_DELAY_SAMPLES` above it -- neither
+the MicroPython usermod's module globals (`src/audioshaper/module.c`) nor
+the CircuitPython spike's (`shared-bindings/audioshaper/__init__.c`) export
+anything past `__version__`/`__revision__` and the two types, so there is
+no second target yet for this figure to agree with.
+
+A class that drives this node with a hard-clipping curve and wants more
+than ~0.74 of full scale out of it has been carrying its own 0.74 ceiling
+without a name for it (audioif#99); this is that name.
+
 ## `audioladder`: the loop CircuitPython's filters cannot close (effects Phase 1)
 
 `audiofilters.Filter` cascades RBJ biquads and does it very well: measured

src/cpython/audioshaper.py

@@ -41,6 +41,17 @@
 Nyquist and they fold back onto the signal, and nothing else in audioif
 resamples at all.
 
+A curve that reaches the rails has a cost on the way back down, though: the
+band-limited version of a full-scale clipped edge does not fit in int16.
+The decimator rings about a third past the rails on an edge like that --
+at the base rate, after the oversampling is already done, where no factor
+of it reaches -- and `post_gain` is what decides whether that overshoot
+then clips. Keep `post_gain * max(abs(curve))` at or below `CLIP_HEADROOM`
+(about 0.74 of full scale) for a curve that reaches the rails, and put the
+rest of the wanted level on a mixer voice after this node rather than on
+this knob. Measured table: `docs/upstream-diff.md`, "`audioshaper`"
+(audioif#99).
+
 `hysteresis` is off by default and is the one thing a table cannot do: give
 the curve a memory, so a slow triangle in and out traces two different paths
 and encloses an area. At zero the node is a static table, sample for sample.
@@ -98,6 +109,18 @@
 #: for the factor it built with -- it is small, but it is not zero.
 GROUP_DELAY_SAMPLES = {1: 0.0, 2: 2.2, 4: 3.3, 8: 3.9}
 
+#: Above roughly this fraction of full scale, `post_gain` on a curve that
+#: reaches the rails re-clips the decimator's own overshoot at the base
+#: rate rather than anything the oversampling can still fix -- see the
+#: docstring above and docs/upstream-diff.md's `audioshaper` section for
+#: the measured table (audioif#99). Documentation only, the same as
+#: `GROUP_DELAY_SAMPLES` above it: neither the MicroPython usermod's module
+#: globals (`src/audioshaper/module.c`) nor the CircuitPython spike's
+#: (`shared-bindings/audioshaper/__init__.c`) export anything past
+#: `__version__`/`__revision__` and the two types, so there is no second
+#: target yet for this figure to agree with.
+CLIP_HEADROOM = 0.74
+
 
 class Waveshaper(_AudioSample):
     def __init__(self, sample_rate=48000, curve=None, oversample=4,

tests/test_cpython_audioshaper.py

@@ -6,8 +6,9 @@
 was asked for -- does oversampling actually lower the alias floor, does the
 hysteresis option actually enclose an area and does that area grow with
 drive, is the hold's ratio exact over a distance where a fixed-point one has
-visibly walked -- each with the control that would go red if the mechanism
-were absent.
+visibly walked, does a hard-clipping curve's own headroom knee sit where
+`CLIP_HEADROOM` says it does -- each with the control that would go red if
+the mechanism were absent.
 """
 
 import math
@@ -40,9 +41,23 @@ def cubic_curve():
 CUBIC = cubic_curve()
 
 
-def render(values, **options):
+def hard_clip_curve(threshold=0.10, points=2048):
+    """A straight hard clip: linear to the rails over +-`threshold` of the
+    input span, flat beyond it -- unlike CUBIC, which never has a flat top
+    at all. Driven hard, this is the shape audioif#99 is about."""
+    last = points - 1
+    return array("h", [
+        clamp15(int(round(max(-1.0, min(1.0, (
+            -1.0 + 2.0 * index / last) / threshold)) * 32767)))
+        for index in range(points)])
+
+
+HARD_CLIP = hard_clip_curve()
+
+
+def render(values, curve=CUBIC, **options):
     """Push one array of interleaved frames through a node and take it back."""
-    node = audioshaper.Waveshaper(sample_rate=SAMPLE_RATE, curve=CUBIC,
+    node = audioshaper.Waveshaper(sample_rate=SAMPLE_RATE, curve=curve,
                                  **options)
     node.play(audiocore.RawSample(
         values, sample_rate=SAMPLE_RATE,
@@ -210,6 +225,83 @@ def test_the_floor_falls_as_the_factor_rises(self):
                         "x8 was worse than x4: %r" % (floors,))
 
 
+class HeadroomTest(unittest.TestCase):
+    """audioif#99: a hard-clipping curve driven hard rings past the rails
+    once decimated, and `post_gain` re-clips that overshoot at the base
+    rate -- after the oversampling is done, where no factor of it reaches.
+
+    Where this happens in the kernel (`src/shared/audioif_shaper.c`):
+    `shape_sample` runs the curve at the oversampled rate and its own clamp
+    (`curve_lookup`, clampf to +-1) bounds each of those samples, but the
+    *decimated* one is not clamped there -- `halfband_down` is a low-pass,
+    not a clip, so a hard edge through it can overshoot +-1. `post_gain`
+    scales that decimated value (`audioif_shaper_process_s16`,
+    `oversampled[0] * config->post_gain * 32768.0f`), and the only place
+    this node ever clips to int16 is `to_s16`, two lines later, on
+    `dry_gain * source + wet_gain * wet`. Everything from the curve to
+    `post_gain` is `float` -- nothing here is int16 until that last line.
+
+    HARD_CLIP ramps to the rails over the inner 10% of its span and is flat
+    beyond it; driven at 32000 (98% of full scale) it is pinned flat for
+    most of every half-cycle, the edge the issue is about. Bare node,
+    1010 Hz, 48 kHz, oversample x4; the window is 4800 samples -- exactly
+    101 cycles of 1010 Hz, so the fundamental and every harmonic fall on
+    their own bin with no window and no rounding.
+
+    Measured (`docs/upstream-diff.md`'s `audioshaper` section carries the
+    full table): flat at -54.34 dB through `post_gain` 0.80, -41.24 at
+    0.90, -36.44 at 1.00 -- a 13.1 dB fall by 0.90 that the bars below ask
+    10 of, for margin. The control is oversample x1: no half-band, no
+    ringing, and the floor reads -34.92 dB at every `post_gain` from 0.66
+    to 1.00 -- the same curve and the same drive, showing no knee at all,
+    which is what proves the x4 knee is the decimator's and not the
+    curve's.
+    """
+
+    HZ = 1010
+    FRAMES = 4800  # 48000 / 1010 * 101 == 4800 exactly: bin 101, no rounding
+    CYCLES = HZ * FRAMES // SAMPLE_RATE
+    LEVEL = 32000
+
+    def _floor(self, post_gain, oversample=4):
+        return alias_floor_db(oversample, cycles=self.CYCLES,
+                              frames=self.FRAMES, level=self.LEVEL,
+                              curve=HARD_CLIP, post_gain=post_gain)
+
+    def test_the_floor_is_flat_below_the_knee(self):
+        """0.66 and 0.74 read the same node to within 1 dB: on this table,
+        the knee has not been reached yet at either."""
+        low = self._floor(0.66)
+        at = self._floor(0.74)
+        self.assertLess(abs(at - low), 1.0,
+                        "0.66 and 0.74 already differ: %.3f vs %.3f dB"
+                        % (low, at))
+
+    def test_the_floor_falls_past_the_knee(self):
+        """0.90 is well past it. `CLIP_HEADROOM`'s ~0.74 ceiling has to be
+        buying real room, not a fraction of a dB, or the constant is
+        theatre -- measured here it is 13.1 dB; the bar asks 10, leaving
+        margin rather than pinning the exact figure."""
+        at = self._floor(0.74)
+        past = self._floor(0.90)
+        self.assertLess(at, past - 10.0,
+                        "0.90 was not at least 10 dB worse than 0.74: "
+                        "%.3f vs %.3f dB" % (at, past))
+
+    def test_the_knee_is_the_decimators_not_the_curves(self):
+        """Its control. At oversample x1 there is no half-band to ring, so
+        the same curve and the same drive must show no knee at all -- if
+        this one went red, the "knee" above would be the curve clipping on
+        its own, not the decimator, and `CLIP_HEADROOM` would be the wrong
+        fix."""
+        floors = [self._floor(post, oversample=1)
+                  for post in (0.66, 0.70, 0.74, 0.78, 0.80, 0.90, 1.00)]
+        self.assertLess(max(floors) - min(floors), 0.5,
+                        "the un-oversampled curve already shows a knee, so "
+                        "x4's knee is not the decimator's doing: %r"
+                        % (floors,))
+
+
 class HysteresisTest(unittest.TestCase):
     """The option encloses an area, the area grows with drive, and off is off.