Skip to content

README.md

Latest commit

 

History

History
123 lines (101 loc) · 4.76 KB

README.md

File metadata and controls

123 lines (101 loc) · 4.76 KB
Raw

How to Support XSched on a New Platform

Step 1: Implement interception code

XSched provides code generation tools for interception boilerplate. The steps below use opencl as an example.

1. Setup directory structure

python3 tools/autogen/setup_template.py ./platforms/example --platform opencl
  • --platform: the name of the platform, which will be used as the directory name.

2. Gather platform headers

For single-header platforms, copy the header to platforms/example/hal/include/xsched/opencl/hal.

For platforms with multiple headers, use tools/autogen/merge_headers.py to merge them into a single header.

python3 tools/autogen/merge_headers.py \
    -d /usr/include/CL/ \
    -o platforms/example/hal/include/xsched/opencl/hal/cl.h \
    -e *.hpp -e cl_dx* -e cl_d3d* -e cl_icd.h -e cl_layer.h -e cl_va_api_media_sharing_intel.h
  • -d(--directory): Directory containing header files to merge
  • -o(--output): Output file path (default is "./merged.h")
  • -e(--exclude): Pattern to exclude header files (can be used multiple times, supports glob patterns like *.hpp)
  • -I(--include-dir): Additional include directories to search for headers (can be used multiple times, e.g., -I /usr/include)

3. Generate interception code (driver.h & intercept.cpp)

python3 tools/autogen/gen.py \
    --source platforms/example/hal/include/xsched/opencl/hal/cl.h \
    --platform opencl \
    --prefix cl \
    --lib /usr/lib/x86_64-linux-gnu/libOpenCL.so \
    --driver platforms/example/hal/include/xsched/opencl/hal/driver.h \
    --intercept platforms/example/shim/src/intercept.cpp
  • -s(--source): Path to the header source
  • -I(--include): Path to the additional include directory
  • --platform: Platform name
  • --prefix: Prefix for the function names
  • --lib: Driver library file
  • --driver: Output driver header file
  • --intercept: Output intercept source file

Step 2: Define HwQueue & HwCommand abstraction

XSched provides the base preempt::HwQueue class in hw_queue.h. A new platform queue should inherit from it and implement the interfaces below.

Preemption Level Interface Description
Level-1 GetDevice() Get device type of HwQueue
GetHandle() Get HwQueue handle
SupportDynamicLevel() Whether the platform supports multiple preemption levels
GetMaxSupportedLevel() The max supported preemption level
Launch(HwCommand) Launch a HwCommand by calling HwCommand->Enqueue()
Synchronize() Wait for all commands in the HwQueue to complete
Level-2 Deactivate() Deactivate the HwQueue to prevent all its commands from being selected for execution
Reactivate() Reactivate the HwQueue to allow all its commands to be selected for execution
Level-3 Interrupt() Interrupt the running command of the HwQueue
Restore() Restore the interrupted command of the HwQueue

Note that only the Level-1 interfaces are mandatory for supporting a new XPU, while the Level-2 and Level-3 interfaces are optional since they necessitate additional hardware capabilities.

XSched also provides the base preempt::HwCommand class. See hw_command.h for the interface details.

Step 3: Finish XShim Lib

Finish functions that need extra handling in platforms/example/shim/src/shim.cpp, such as XCreateCommandQueueWithProperties(), XCreateKernel(), XEnqueueNativeKernel(), etc. The main objectives are:

  1. Create XQueue synchronously when creating hardware queue.
  2. Encapsulate the launch kernel as HwCommand and submit it to XQueue.
  3. Ensure correctness by extra kernel arguments lifecycle management and synchronization handling.

Then change function calling path in platforms/example/shim/src/intercept.cpp to call XShim functions instead of original driver API. For example, change Driver::CreateKernel to XCreateKernel().