Skip to content

AwsTemplateProcessor: short placeholder replacement corrupts later literal % characters in chunked response #477

Description

@Xylopyrographer

Platform

ESP32 (ESP32-S3)

IDE / Tooling

PlatformIO (Arduino framework), pioarduino platform-espressif32

Library version

ESPAsyncWebServer 3.12.1 (ESP32Async/ESPAsyncWebServer @^3.12.1), AsyncTCP ^3.5.0

What happened?

When a %VAR% placeholder handled by a AwsTemplateProcessor is replaced with a value that is
much shorter than the placeholder text itself, the streaming/chunked response gets corrupted
for the remainder of the file. Specifically, any later literal % character in the response
body (e.g. a CSS percentage like width: 100%; or border-radius: 50%;) gets misinterpreted as
the start of a new template token, and the surrounding text is spliced/garbled in the actual
bytes sent to the client.

This only manifests when:

  1. A registered placeholder's replacement text is significantly shorter than the placeholder
    itself (e.g. %MATRIX_WIDTH% (14 chars) replaced with "8" (1 char)).
  2. There is at least one literal % character later in the same response body that is not part
    of an actual template token.

I confirmed this by inspecting the raw served bytes with curl -s http://<device-ip>/ | grep -n '...'
and observed garbled output such as:

width: 100MATRIX_WIDTHMATRIX_WIDTHMATRIX_HEIGHT;

where the literal 100% had been spliced with leftover token-scanning state from an earlier,
much-shorter-than-expected replacement.

Workaround used: avoid mixing %VAR% placeholders with literal % characters (e.g. CSS
percentages) in the same streamed response — confine placeholders to a section with no
competing literal %, and do the CSS-relevant work in JS at runtime instead.

Minimal Reproducible Example (MRE)

#include <Arduino.h>
#include <WiFi.h>
#include <ESPAsyncWebServer.h>

AsyncWebServer server(80);

const char INDEX_HTML[] PROGMEM = R"HTML(
<!DOCTYPE html>
<html>
<head>
<style>
  /* literal '%' characters here trigger the corruption once a short
     placeholder has been substituted earlier in the stream */
  .box { width: 100%; border-radius: 50%; }
</style>
</head>
<body>
<script>
  const W = %MATRIX_WIDTH%, H = %MATRIX_HEIGHT%; // placeholder replaced with "8" (short)
</script>
</body>
</html>
)HTML";

void setup() {
  Serial.begin(115200);
  WiFi.begin("SSID", "PASSWORD");
  while (WiFi.status() != WL_CONNECTED) delay(200);

  server.on("/", HTTP_GET, [](AsyncWebServerRequest *request) {
    request->send_P(200, "text/html", INDEX_HTML, [](const String &var) -> String {
      if (var == "MATRIX_WIDTH") return String(8);
      if (var == "MATRIX_HEIGHT") return String(8);
      return String();
    });
  });

  server.begin();
}

void loop() {}

Fetching / and inspecting the raw bytes (curl -s http://<device-ip>/) shows the CSS <style>
block's literal % characters corrupted/garbled, even though they appear before the
<script> block containing the actual placeholders — the corruption appears to stem from
internal buffer/state used by the chunked template-scanning logic that isn't fully reset when
a replacement is shorter than the token it replaces.

Expected behavior

Literal % characters in the response body that are not part of a recognized %VAR% token
should always be passed through unmodified, regardless of the length of prior replacements
elsewhere in the same response.

Workaround (for anyone hitting this)

Do not mix %VAR% template placeholders with literal % characters (e.g. CSS percentage
values) in the same response body. Keep all placeholders in a script/text section free of
competing literal %, and apply percentage-based styling via JavaScript at runtime instead.

I confirm that:

  • I have read the documentation.
  • I have searched for similar discussions.
  • I have searched for similar issues.
  • I have looked at the examples.
  • I have upgraded to the latest version of ESPAsyncWebServer (and AsyncTCP for ESP32).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions