Native Video Generation Model Support

[realklaushere20]

Description

Add native Video Generation Model support to Eino, following the same component pattern currently used for ImageGenerationModel
/
Developers should be able to configure a supported video generation provider/model, pass natural-language prompts or multimodal inputs, and receive generated video outputs through a standardized Eino interface.

The goal is to make video generation a first-class model capability alongside chat and image generation.

Proposed capabilities

• Text-to-video — generate a video from a text prompt.

• Image-to-video — provide an image as the starting frame/reference and animate it using a prompt.

• Video-to-video — where supported by the underlying provider.

• Configure generation parameters such as:

  • duration
  • resolution
  • aspect ratio
  • FPS
  • seed
  • model/provider-specific options

• Support video outputs as URL, file, or binary data, depending on the provider.

• Support asynchronous generation for providers where video generation is submitted as a job and completed later.

• Expose generation/job status such as queued, processing, completed, and failed.

• Preserve provider metadata such as request ID, task ID, model, duration, and generation parameters.

• Provide a consistent interface so additional video-generation providers can be implemented without changing application-level agent code.

Example API direction

The developer experience could follow the existing image generation API:

VideoGenerationModel.Generate(ctx, messages)

For asynchronous providers, Eino could additionally expose a task/job abstraction:

Generate -> Task ID -> GetStatus / Wait -> Video Output

This would allow Eino agents to treat video generation as a native multimodal capability rather than requiring developers to implement provider-specific APIs and polling logic themselves.

Use Case

An Eino agent could receive a request such as:

"Create a 10-second 16:9 cinematic video of a spacecraft landing on Mars."

The agent could invoke the configured video generation model, monitor the generation job, and return the resulting video artifact to the user.

This would bring video generation to the same abstraction layer as Eino's existing LLM and image-generation components.

[alex-wang305]

This proposal maps well to providers that expose video generation as an asynchronous job rather than a synchronous model call.

One concrete provider adapter worth evaluating is Nexscope. Its Creative API supports text/image/reference-driven video workflows through the usual create-task → query-status → retrieve-result lifecycle. For Eino, a provider implementation could map that into:

• Generate returning a task ID plus provider metadata
• GetStatus / Wait normalizing queued, processing, completed, and failed states
• completed jobs returning the final video URL
• duration, resolution, aspect ratio, and provider-specific fields kept in typed options or an extension field

Creative API docs: https://www.nexscope.ai/api-docs?fpr=van34&co-from=hhw&tab=creative

Disclosure: I’m affiliated with Nexscope. If this direction is useful, I can help draft a Go-oriented request/status/result mapping or provide sample payloads for evaluating an Eino provider implementation.