Skip to content

sri.md

Latest commit

 

History

History
335 lines (264 loc) · 10.8 KB

sri.md

File metadata and controls

335 lines (264 loc) · 10.8 KB
Raw

Dify Plugin Daemon - Serverless Runtime Interface (SRI)

The Serverless Runtime Interface (SRI) is a set of HTTP APIs for packaging plugins into serverless components, allowing the Dify Plugin Daemon to remotely launch and operate them on external platforms (e.g., AWS Lambda).

This interface enables the daemon to communicate with remote runtime environments via standard protocols to handle plugin deployment, execution, and instance queries.

⚠️ Note: This interface is currently in the Alpha stage. Stability and backward compatibility are not guaranteed. A production-grade SRI implementation is available in the enterprise edition. For support, please contact business@dify.ai.

🔧 Basic Configuration

The daemon is configured using the following environment variables:

Variable Description
DIFY_PLUGIN_SERVERLESS_CONNECTOR_URL Base URL of the remote runtime environment, e.g., https://example.com
DIFY_PLUGIN_SERVERLESS_CONNECTOR_API_KEY Authentication token for accessing SRI, passed in the Authorization request header
MAX_SERVERLESS_REQUEST_BYTES Maximum serialized request payload size sent to a serverless plugin runtime (in bytes). Default is 5242880 (5 MB). This limit accounts for Lambda Function URL's 6 MB request size limit, with a safety margin for headers and metadata.
DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_ENABLED Whether to run the activation preflight before dispatching an invocation. When enabled, the daemon calls POST /v1/activation/activate to wake a scaled-to-zero plugin and waits until it is ready. Default is false.
DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_TIMEOUT Seconds the daemon waits for the plugin to become ready during the activation preflight. If the plugin is not woken up in time, the invocation is treated as failed. Default is 60.

📦 Request Size Limits and Batch Processing

Serverless platforms like AWS Lambda Function URLs have strict payload size limits (6 MB for Lambda Function URLs). To handle large requests that would exceed these limits, the daemon automatically splits text embedding and token counting requests into multiple batches when they would exceed MAX_SERVERLESS_REQUEST_BYTES.

Key behaviors:

  • Transparent batching: The daemon handles splitting and merging automatically — plugin developers don't need to implement any special logic
  • Serial processing: Each batch is processed serially to ensure consistent results
  • Automatic merging: Results from all batches are merged into a single response
  • Error handling: If a single text item exceeds the limit, the request will fail with a ServerlessPayloadTooLargeError

This feature prevents opaque errors when large embedding requests fail due to payload size limits.

📡 API Endpoints

GET /ping

Used by the daemon for connectivity checks during startup.

Request

GET /ping
Authorization: <API_KEY>

Response

  • 200 OK, response body is plain text: "pong"

GET /v1/runner/instances

Returns information about plugin instances that are ready to run.

Query Parameters

  • filename (required): Name of the uploaded plugin package, in the format:

    vendor@plugin@version@hash.difypkg

Response

{
  "items": [
    {
      "ID": "string",
      "Name": "string",
      "Endpoint": "string",
      "ResourceName": "string"
    }
  ]
}

POST /v1/launch

Launches a plugin using a streaming event protocol for real-time daemon parsing of startup status.

This API uses multipart/form-data for submission and returns status via Server-Sent Events (SSE).

Request Fields

Field Type Description
context file Plugin package file in .difypkg format
verified boolean Whether the plugin has been verified by the daemon

SSE Response Format

{
  "Stage": "healthz|start|build|run|end",
  "State": "running|success|failed",
  "Obj": "string",
  "Message": "string"
}

Stage Descriptions

Stage Meaning Description
healthz Health check Initializes runtime resources and containers
start Startup prep Prepares the environment
build Build phase Builds plugin dependencies and packages image
run Execution phase Plugin is running; returns key info on success
end Completion Final state confirmation: success or failure

When a message with Stage=run and State=success is received, the daemon will extract details and register the plugin instance:

endpoint=http://...,name=...,id=...

Error Handling

  • If any stage returns State = failed, it is considered a launch failure
  • The daemon should abort the process and output the Message field as the error

POST /v1/activation/activate

Optional activation preflight. When DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_ENABLED is true, the daemon calls this endpoint before dispatching an invocation to wake a plugin that may have been scaled to zero, and blocks until the plugin is ready. It is safe to call on every invocation; the runtime is expected to throttle any underlying activity/lease writes.

Request

POST /v1/activation/activate
Authorization: <API_KEY>
Content-Type: application/json

{
  "instance_id": "string"
}
  • instance_id (required): the connector function name of the target plugin (the Name returned by /v1/runner/instances and /v1/launch).

Response

{
  "ready": true,
  "endpoint": "string"
}
  • 200 OK with ready = true: the plugin is ready and the daemon proceeds with the invocation.
  • 504 Gateway Timeout (or ready = false): the plugin did not become ready in time.

Error Handling

  • The daemon bounds the wait with DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_TIMEOUT. On timeout or any non-200 response, the invocation is aborted and reported as a failure.

🔁 Communication Sequence (ASCII)

daemon                              Serverless Runtime Interface
   |-------------------------------------->|
   |           GET /ping                  |
   |<--------------------------------------|
   |         200 OK "pong"                |
   |-------------------------------------->|
   |    GET /v1/runner/instances          |
   |            filename                  |
   |<--------------------------------------|
   |             {items}                  |
   |-------------------------------------->|
   |        POST /v1/launch               |
   | context, verified multipart payload |
   |<--------------------------------------|
   |   Building plugin... (SSE)           |
   |<--------------------------------------|
   |   Launching plugin... (SSE)          |
   |<--------------------------------------|
   |   Function: [Name] (SSE)             |
   |<--------------------------------------|
   |   FunctionUrl: [Endpoint] (SSE)      |
   |<--------------------------------------|
   |   Done: Plugin launched (SSE)        |

📦 Plugin File Naming Convention

Plugin files must use the .difypkg extension and follow this naming convention:

<vendor>@<plugin_name>@<version>@<sha256_hash>.difypkg

Example:

langgenius@tavily@0.0.5@7f277f7a63e36b1b3e9ed53e55daab0b281599d14902664bade86215f5374f06.difypkg

📬 Contact Us

For access to the enterprise-supported version or more details about plugin packaging and deployment, please contact:

📧 business@dify.ai

📘 OpenAPI Specification (YAML)

openapi: 3.0.3
info:
  title: Dify Plugin Daemon - Serverless Runtime Interface (SRI)
  version: alpha
  description: HTTP API specification for the Dify Plugin Daemon's Serverless Runtime
    Interface (SRI).
paths:
  /ping:
    get:
      summary: Health check endpoint
      description: Used by the daemon to verify connectivity with the SRI.
      responses:
        '200':
          description: Returns 'pong' if the service is alive
          content:
            text/plain:
              schema:
                type: string
                example: pong
      security:
      - apiKeyAuth: []
  /v1/runner/instances:
    get:
      summary: List available plugin instances
      parameters:
      - name: filename
        in: query
        required: true
        schema:
          type: string
        description: Full plugin package filename (e.g., vendor@plugin@version@hash.difypkg)
      responses:
        '200':
          description: List of available plugin instances
          content:
            application/json:
              schema:
                type: object
                properties:
                  items:
                    type: array
                    items:
                      type: object
                      properties:
                        ID:
                          type: string
                        Name:
                          type: string
                        Endpoint:
                          type: string
                        ResourceName:
                          type: string
      security:
      - apiKeyAuth: []
  /v1/launch:
    post:
      summary: Launch a plugin via SSE
      requestBody:
        required: true
        content:
          multipart/form-data:
            schema:
              type: object
              properties:
                context:
                  type: string
                  format: binary
                  description: Plugin package file (.difypkg)
                verified:
                  type: boolean
                  description: Whether the plugin is verified
              required:
              - context
      responses:
        '200':
          description: Server-Sent Events stream with plugin launch stages
          content:
            text/event-stream:
              schema:
                type: object
                properties:
                  Stage:
                    type: string
                    enum:
                    - healthz
                    - start
                    - build
                    - run
                    - end
                  State:
                    type: string
                    enum:
                    - running
                    - success
                    - failed
                  Obj:
                    type: string
                  Message:
                    type: string
      security:
      - apiKeyAuth: []
components:
  securitySchemes:
    apiKeyAuth:
      type: apiKey
      in: header
      name: Authorization