Skip to content

API consistency findings surfaced by TS contract tests #1202

Closed as not planned
Closed as not planned

Description

@msluszniak

The new API contract tests added in #1018 (branch @ms/api-tests-scaffold) surfaced several inconsistencies across the public TypeScript surface. They are currently encoded as explicit exception sets in the test files so the suite passes today; this issue tracks the cleanup work.

Modules that don't extend BaseModule

BaseModule provides the shared delete() / forwardET() / getInputShape() plumbing and the generateFromFrame JSI binding hook. The following modules skip it and re-implement (or omit) parts of that contract:

  • OCRModule
  • VerticalOCRModule
  • LLMModule
  • SpeechToTextModule
  • TextToSpeechModule
  • TokenizerModule

Listed in SKIPS_BASE_MODULE in __tests__/api/moduleContracts.test.ts.

Modules without a static from* factory

Every other module uses Module.fromModelName(...) / Module.fromCustomModel(...) as the construction entry point:

  • ExecutorchModule — constructs blank and exposes an instance load() method instead.
  • TokenizerModule — no factory at all.

Listed in SKIPS_STATIC_FACTORY.

Hook naming inconsistency

Every hook drops the Module suffix from its module class name (ClassificationModule → useClassification, LLMModule → useLLM) except:

  • ExecutorchModule → useExecutorchModule (keeps the suffix).

Listed in HOOK_NAME_EXCEPTIONS.

Informational: parameterized registry accessor

models.ocr.craft({ language }) requires a language argument, while every other accessor in the registry is either fully parameterless or takes only optional ModelOpts. This is intentional (OCR pipelines are picked per language), but worth flagging because the test walker has to special-case it. Listed in PARAMETERIZED_ACCESSORS in __tests__/api/modelRegistry.test.ts.

How to act on this

Each bullet above corresponds to an entry in one of the exception sets. As an inconsistency is fixed at the source, the matching entry can be deleted and the contract test will start enforcing the rule for that module.

Refs #1018.

Activity

  1. added
    testIssue and PR related to tests or testing infrastructure
    on May 29, 2026
  2. added theissue type on May 29, 2026
  3. msluszniak commented on May 29, 2026

    @msluszniak
    MemberAuthor

    Update: deeper tests on branch @ms/api-tests-scaffold surface two more inconsistencies:

    useTextToSpeech takes two positional args

    Every other public useXxx hook takes a single object argument, e.g.:

    useClassification({ model, preventLoad });

    useTextToSpeech is the lone outlier:

    useTextToSpeech(model, { preventLoad });

    Tracked in __tests__/api/hookPropsContract.test.ts as a documented opt-out (useTextToSpeech is excluded from the single-arg contract assertion). Normalising the signature to useTextToSpeech({ model, preventLoad }) would be a breaking change but would let the contract test enforce the rule across the whole hook surface.

    TokenizerModule has no delete() method

    BaseModule.delete() calls this.nativeModule.unload() to release native resources. TokenizerModule doesn't extend BaseModule (already noted in the original list) and doesn't declare its own delete(), so the native tokenizer handle allocated in load() is never released — a latent memory leak per loaded tokenizer.

    Tracked in __tests__/api/modulePrototype.test.ts via SKIPS_DELETE (TokenizerModule is the sole entry). Adding a delete() that unloads this.nativeModule is the minimal fix.

  4. msluszniak commented on May 29, 2026

    @msluszniak
    MemberAuthor

    Two more inconsistencies surfaced while writing moduleConstruction.test.ts and moduleHookSignatureAlignment.test.ts:

    TextToImageModule.forward is exposed as useTextToImage().generate

    The module's primary inference method is forward(input, imageSize?, numSteps?, seed?) and the hook return advertises it as generate(...) with the same signature. Every other module that has both a forward on the prototype and a forward field on the hook return uses the same name; TTI is the outlier.

    moduleHookSignatureAlignment.test.ts documents the rename as textToImage_forward_to_generate so the signatures still get cross-checked.

    VisionModule.runOnFrame is a throwing getter

    Not a public-API bug, but worth flagging for anyone introspecting the module surface (e.g. doc generators, IDE tooling): accessing VisionModule.prototype.runOnFrame without a loaded nativeModule throws RnExecutorchError(ModuleNotLoaded). modulePrototype.test.ts works around this by using Object.getOwnPropertyDescriptor instead of direct property access.

  5. self-assigned this
    on May 29, 2026
  6. msluszniak commented on Aug 12, 2026

    @msluszniak
    MemberAuthor

    This issue is stale as we reconstructed an API on rne-rewrite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

testIssue and PR related to tests or testing infrastructure

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions