Skip to content

Establish reliable runtime regression tests for the alpha #383

Closed

Description

@jderochervlk

Part of #382 — v1.0.0 alpha readiness.

Problem

The review at ebe9cca601fb85e6064c786367e546ded9b07d05 found 42 ReScript test fixtures, but the runner executes only seven modules. Only Classify and SafeFetch contain assertions; the other five runtime modules are mostly no-throw/logging smoke tests. The remaining 35 fixtures are compile-only, not browser tests.

The generated-output synchronization check cannot detect drift: it runs git ls-files --modified against generated JavaScript that is ignored and untracked.

Sources: tests/index.js:14-L29, tests/index.js:65-L80, .gitignore:20-L22.

Scope

  • Explicitly distinguish compile-only fixtures, executable runtime tests, and documentation examples; make missing runtime-test registration detectable.
  • Retain useful compile coverage and existing SafeFetch assertions.
  • Replace logging-only success checks with assertions on values and behavior for Headers, Request, Response, URLSearchParams, and URL.
  • Remove the ineffective generated-output guard, or replace it with intentionally maintained golden baselines if generated JavaScript is a supported contract.
  • Add a small real-browser runner for regression tests contributed by the binding-fix issues. Chromium coverage is sufficient for this alpha target; a full browser-engine matrix belongs to later hardening.
  • Add focused expected-failure consumer fixtures for unsafe type widening and nullable/type-contract changes. Match the intended diagnostic, not any compilation failure.

Acceptance criteria

  • A deliberately failing runtime assertion causes the documented test command and CI to fail.
  • Runtime test discovery/registration has a check; a new runtime fixture cannot silently become compile-only.
  • Existing Node-compatible smoke tests assert meaningful results.
  • The misleading Git synchronization check is removed or replaced by a working regression mechanism.
  • A real-browser smoke test runs in CI, and the linked binding fixes can add native API regressions to it.
  • Compile-only fixtures remain supported; do not execute all existing examples indiscriminately. For example, the current Fetch example supplies a body with the default GET method.
  • Test/contributor documentation describes the actual commands and categories.

Boundaries

No requirement to adopt a particular test framework, achieve an arbitrary coverage percentage, or certify every browser API before alpha. Node Fetch tests and DOM emulation alone must not be described as real-browser validation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions