Skip to content

[RNE Rewrite] Proper way to handle dynamic input shapes #1323

Closed

Description

@barhanc

cc: @msluszniak @benITo47

This issue starts a discussion about the proper way we should handle dynamic input shapes. The current solution is a temporary one that fills the missing ExecuTorch runtime feature with adhoc solution, but seeing the changes required for OCR (#1322) makes it clear to me that we need a much more robust solution. I quickly came up with the following idea and would like your input on this:

  1. The SymbolicShape utility should be mirrored in both C++ and TS.
  2. On both sides the validation should allow for checking:
    • static vs range dim (both symbolic and concrete, so validating e.g. shape {1, "H", RangeDim{.min=0, .max="W", .step=4}} (C++ does it but only for concrete RangeDim, TS doesn't have RangeDims)
    • validating an enumeration of shapes, so in C++ it should be possible to write fromJs(..., {{1, "H", 64}, {1, "H", 128}} (TS has this)
    • (optionally) it should also allow cartesian product of dims as a shortcut to explicit enumeration, so e.g. {1, {64,128}, {64, 128}} <=> {{1, 64, 64}, {1, 64, 128}, ...} but this might introduce very severe difficulties for companion method export and symbolic -> concrete shape decay, so I wouldn't actually implement that
  3. We need a differentiation between SymbolicShape and ConcreteShape (in both TS and C++).
    type Symbol = number | string;
    type SymbolicRangeDim = readonly {min: Symbol, max: Symbol, step?: Symbol}
    type SymbolicShape = readonly (Symbol | SymbolicRangeDim)[]
    
    type RangeDim = readonly {min: number, max: number, step?: number}
    type Shape = readonly (number | RangeDim)[]
    and similar distinction on C++ side.
    • C++ fromJs would take an enumeration of SymbolicShape or ConcreteShape to validate the input tensor.
    • C++ in model.cpp the dynamicInputShapes would contain only ConcreteShape for each dynamic method (or even perhaps we could build map inputShapes for all methods (not only dynamic) to unify the C++ API)
    • getMethodMeta would return ConcreteShape for each method input tensor meta (I don't know what about the output tensor meta in the case of dynamic outputs)
    • In TS, validateModelSchema would now check expected SymbolicShape against returned ConcreteShape by model.getMethodMeta and would return a type-safe ConcreteShape so that non-null assertion operator ! wouldn't be needed.
  4. The Python export contract should only require one additional companion method get_dynamic_dims_<method_name>, however I'm not sure how to encode both range dims and enumerated shapes here. Adding auxiliary methods on top of the get_dynamic_.... doesn't seem like a robust solution.

Activity

  1. self-assigned this
    on Jul 20, 2026
  2. msluszniak commented on Jul 20, 2026

    @msluszniak
    Member

    I agree that we need to deal with the input shapes in a more robust way. I read your suggestions and I agree with them. For now, I have one question regarding 3. point, how about the shapes that are neither purely symbolic nor purely numeric? About cartesian product I'm agaist these changes, it will over complicate this imo. Validation in both ts and cpp sounda like a must. About 4, maybe we should pass some disctionaries instead of arrays of size 3 and handle it it this way with named keys?

  3. added theissue type on Jul 22, 2026
  4. added
    ideaNew idea to enhance the library, suggestion, etc.
    on Jul 23, 2026
  5. added this to the v0.10.0 milestone on Jul 23, 2026
  6. added a commit that references this issue on Aug 3, 2026
    b38a61f
  7. barhanc commented on Aug 3, 2026

    @barhanc
    MemberAuthor

    Closed as completed in #1327

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

ideaNew idea to enhance the library, suggestion, etc.

Type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions