Skip to content

Execution‐Driven Mode (EDM) Support for Olympia

Jump to bottom
Aditya Jha edited this page Jun 15, 2026 · 18 revisions

What EDM is

Execution-Driven Mode (EDM) for a performance model allows a performance model to execute a workload/application/binary directly on the performance simulator without the need for an intermediary format like a trace.

The advantages include

  1. Compile a workload, run directly on the performance model
  2. No need to generate an intermediate format like STF or JSON
  3. Observe impacts of speculative execution (for out-of-order designs) like wrong-path instruction cache loads, data cache loads, etc
  4. Observe impacts of interrupt/exception handling
  5. Observe impacts of shared memory contentions in multithreaded/multicore simulations

The disadvantages include

  1. Slower simulation time
  2. The simulator must be functionally correct

Why EDM Matters

Trace-driven simulation is fast and deterministic, but it cannot capture several important performance effects:

  1. Speculative Execution: When a branch is mispredicted, the processor fetches and executes instructions from the wrong path. These instructions pollute the instruction cache and data cache before the misprediction is detected. Traces only contain the correct path, so this effect is invisible.

  2. Cache Pollution: Wrong-path instructions bring data into the cache that wouldn't be there in a trace-driven simulation. This affects the hit/miss behavior of subsequent correct-path instructions.

  3. Dynamic Behavior: Real programs have dynamic behavior based on runtime conditions, interrupts, and exceptions. Traces are static snapshots of one execution.

  4. Multi-core Effects: In multi-core systems, threads interact through shared memory and caches. EDM can model these interactions more accurately.

EDM enables Olympia to model these real-world effects, providing more accurate performance predictions for out-of-order processors with speculative execution.

Adding EDM Support to Olympia

Initial discussion and issues related to this project:

Requirements are documented in Issue 14

There are two parts to EDM support for EDM:

  • The functional "driver" or functional model that executes instructions and provides an instruction stream
  • The performance simulator that receives the dynamic stream of instructions and make decisions based on that stream
    • Olympia

The main requirement is to ensure Olympia is extensible or agnostic to the functional driver.

Nature of the Implementation : Execute At Fetch

The edm support follows an Execute at Fetch paradigm. In this model, the functional backend (pegasus or whisper) acts as an oracle. When Olympia fetches an instruction, the functional model executes it immediately through its internal pipeline and determine the outcome, such as the register values, memory updates and the next program counter.

However, this execution is non-binding:

  • Deferred Update of State: The functional model processes the instructions but it does not commit it and make it permanent to its state at the time of fetch (from Olympia). Meaning when Olympia fetches, the functional model executes it determines everything related to it but does not commit the instruction.
  • Pending Events: Instructions are held in a pending state until the Olympia's Reorder Buffer does not explicitly commits them.
  • Explicit Commitment: The events are committed inside both olympia and the functional model only when olympia commits it inside its own pipeline and instructs the backend to also commit it.
  • Speculative Nature: This separation between the executor and the one directing the execution allows us to steer the functional model down towards speculative "wrong paths". If a mispredictions occurs; Olympia can trigger a flush to revert the functional model to a specific checkpoint and resume from the correct path. This allows for a more real world cycle prediction for Olympia.

Building with EDM Support

EDM support can be build using the following CMake flag and is off by default :

cmake -DEDM_ENABLED=ON -DDEFAULT_BACKEND=pegasus -DSPARTA_SEARCH_DIR=/usr/local ..

DEFAULT_BACKEND accepts pegasus, whisper or all. When EDM_ENABLED=ON the preprocessor symbol EDM_ENABLED is defined throughout the build, and the edm static library is build and linked into instgen.

Pegasus is fetched automatically via FetchContent at the pinned tag specified in the CMakeLists.txt at : core/edm/CMakeLists.txt

Runtime Configuration

In your arch file (e.g. arches/small_core.yaml), set:

top.cpu.core0:
    fetch.params.backend: "pegasus"
    fetch.params.backend_config_file: "config/pegasus.yaml"

The backend config file (e.g. config/pegasus.yaml) is backend specific:

ilimit: 10000
params : {top.core0.params.isa: "rv64imafdcbv_zicsr_zifencei_zbkx"}
pegasus_loggers: []
db_file: "pegasus-cosim-work.db"
snapshot_threshold: 1000

The workload, (elf file) is detected automatically via file extension (.elf) - no other flag is needed:

./olympia --workload coremark.elf

Implementation Details:

In order to implement this backend agnostic architecture, the following files and classes have been defined:

  • EDMInterface.hpp : This class will contain the contract. This is a pure virtual interface that defines the mandatory methods (step, commit, flush ) that any functional backend must implement.

  • EDMTypes.hpp : This contains the backend agnostic datastructures like RegAccess, MemAccess and InstructionInfo used to pass data from the backend to Olympia. This makes it such that when Olympia talks to the backend it would not know if the instruction is a Event ( for pegasus ) or an InstrPac (for whisper). We will translate these datatype into the InstructionInfo and then the InstructionInfo into Inst (olympia)

    Event is the class that stores the instruction in pegasus across its pipeline, similarly, InstrPac is for Whisper and Inst is for Olympia

  • EDMInstGenerator : This class extends Olympia's base InstGenerator. It would be defined inside the core/InstGeneartor.hpp/.cpp in Olympia, alongside the already existing TraceInstGenerator (for stf traces) and JSONInstGenerator ( for json ). The EDMInstGenerator class would implement and override the three functions : getNextInst, isDone and reset.

Diagrams

Authored by Komal Kumavat

graph TB
    subgraph "Layer 1: Olympia Pipeline"
        F[Fetch]
        D[Decode]
        R[Rename]
        Dis[Dispatch]
        E[Execute]
        ROB[ROB]
    end
    
    subgraph "Layer 2: Instruction Generation"
        IG[InstGenerator]
        TIG[TraceInstGenerator]
        JIG[JSONInstGenerator]
        EIG[EDMInstGenerator]
    end
    
    subgraph "Layer 3: EDM Integration NEW"
        API[EDMInterface]
        PA[PegasusAdapter]
        WA[WhisperAdapter]
    end
    
    subgraph "Layer 4: Functional Models"
        PEG[Pegasus]
        WHI[Whisper]
    end
    
    F --> IG
    IG --> TIG
    IG --> JIG
    IG --> EIG
    EIG --> API
    API --> PA
    API --> WA
    PA --> PEG
    WA --> WHI
    
    style EIG fill:#2e7d32
    style API fill:#2e7d32
    style PA fill:#2e7d32
    style WA fill:#2e7d32
Loading

Execution Flow

Normal Execution

sequenceDiagram
    participant F as Fetch
    participant E as EDMInstGenerator
    participant I as EDMInterface
    participant P as Pegasus
    
    F->>E: getNextInst()
    E->>I: step()
    I->>P: step(core, hart)
    P-->>I: EventAccessor
    I-->>E: EDMInstructionInfo
    
    alt Is Branch
        E->>E: saveCheckpoint_()
    end
    
    E->>E: buildInst_()
    E-->>F: InstPtr
Loading

Speculative Execution

sequenceDiagram
    participant F as Fetch
    participant E as EDMInstGenerator
    participant I as EDMInterface
    participant P as Pegasus
    participant BP as BranchPredictor
    
    F->>E: getNextInst()
    E->>I: step()
    I->>P: step()
    P-->>E: branch (next_pc=A, alt_pc=B)
    E->>E: saveCheckpoint_(correct=A)
    E-->>F: branch InstPtr
    
    F->>BP: predict()
    BP-->>F: taken → B
    
    F->>E: getNextInst()
    E->>I: stepWithOverridePc(B)
    I->>P: step(core, hart, B)
    P->>P: Execute at B (speculative)
    P-->>E: speculative inst
    E-->>F: InstPtr
    
    Note over F: Branch resolves - WRONG
    
    F->>E: reset(branch_inst)
    E->>I: flush(checkpoint)
    I->>P: flush(accessor)
    P->>P: Rewind to checkpoint
Loading

High-Level Execution Flow

This flow chart shows the complete execution cycle from Olympia's perspective, illustrating when functional models and branch predictors are invoked:

flowchart LR
    Start([Start]) --> Fetch[Fetch:<br/>getNextInst]
    
    Fetch --> FuncExec[Functional Model<br/>Executes Instruction]
    FuncExec --> CheckBranch{Branch?}
    
    CheckBranch -->|Yes| SaveCP[Save Checkpoint<br/>ISS UID + PC]
    CheckBranch -->|No| BuildInst[Build Inst]
    
    SaveCP --> BranchPred[Branch<br/>Predictor]
    BranchPred -->|Sequential| BuildInst
    BranchPred -->|Taken| OverridePC[Override PC]
    OverridePC --> BuildInst
    
    BuildInst --> Pipeline[Pipeline:<br/>Decode→Rename<br/>→Dispatch→Execute]
    
    Pipeline --> ROB{ROB<br/>Commit?}
    
    ROB -->|Branch<br/>Mispredicted| Flush[Flush<br/>Pipeline]
    ROB -->|Correct| Commit[Commit to<br/>Func Model]
    
    Flush --> Rewind[Rewind Func Model<br/>to Checkpoint]
    Rewind --> Fetch
    
    Commit --> Prune[Prune Old<br/>Checkpoints]
    Prune --> Done{Done?}
    
    Done -->|No| Fetch
    Done -->|Yes| End([End])
    
    style SaveCP fill:#ffeb3b
    style BranchPred fill:#9c27b0,color:#fff
    style Flush fill:#f44336,color:#fff
    style Commit fill:#4caf50,color:#fff
    style FuncExec fill:#2196f3,color:#fff
Loading

API and Data structure design:

File Structure

File Purpose
core/edm/EDMInterface.hpp Pure virtual Interface all backends must implement
core/edm/EDMType.hpp Backend Agnostic data structures ( will hold the EDMInstructionBase class)
core/edm/EDMFactory.hpp/.cpp Factory + registry for backend creation by name
core/edm/adapters/Pegasus/Pegasus.hpp/.cpp Pegasus backend adapter
core/InstGenerator.hpp/.cpp EDMInstGenerator lives here alongside other implementation like TraceInstGenerator and JSONInstGenerator
core/Inst.hpp/.cpp EDM_ENABLED gated notify hooks (notifyRetire, notifyFlush, etc.)
core/ROB.cpp Calls notifyRetire / notifyFlush on Instructions
core/lsu/LSU.cpp calls notifyStoreCommit / notifyStoreDrop on store instructions

EDMInterface :

The contract every backend must fulfill:

class EDMInterface {
public:
    virtual ~EDMInterface() = default;
    virtual bool isFinished(CoreId core_id, HartId hart_id) const = 0;
    virtual Addr peekNextPc(CoreId core_id, HartId hart_id) const = 0;
    virtual InstructionInfo step(CoreId core_id, HartId hart_id) = 0;
    virtual InstructionInfo stepWithOverridePc(CoreId core_id, HartId hart_id, Addr override_pc) = 0;
    virtual void commitInstruction(CoreId core_id, HartId hart_id, uint64_t iss_uid) = 0;
    virtual void commitStoreWrite(CoreId core_id, HartId hart_id, uint64_t iss_uid) = 0;
    virtual void dropStoreWrite(CoreId core_id, HartId hart_id, uint64_t iss_uid) = 0;
    virtual void flush(CoreId core_id, HartId hart_id, const EDMCheckpoint & checkpoint) = 0;
};

Adapter

namespace olympia::edm
{
    // PegasusEDMAdapter
    //
    //   implements the  EDMInterface using PegasusCoSim as the ISS.
    // Constructed by createEDMBackend("pegasus", workload, params).
    //
    // Owns cosim_ (PegasusCoSim).
    // The ELF binary is loaded inside the PegasusCoSim constructor:
    //   PegasusCoSim(ilimit, workload, params, db_file, snapshot)
    //   → processParameter("top.extension.sim.workloads", workload)
    //   By the time the constructor returns, the binary is loaded and
    //   the ISS PC is at the ELF entry point.
    //
    // Key design point — EventAccessor storage:
    //   cosim_->flush() requires an EventAccessor object, not just a
    //   UID. So this adapter maintains:
    //     pending_branch_events_: iss_uid → EventAccessor
    //   populated in step() / stepWithOverridePc() for every branch
    //   instruction. Entries removed in commitInstruction() and flush().
    //
    // Method → PegasusCoSim mapping:
    //
    //  isFinished()
    //      cosim_->isSimulationFinished(CORE_ID, HART_ID)
    //      reads PegasusState::SimState::sim_stopped
    //
    //  peekNextPc()
    //      cosim_->getPc(CORE_ID, HART_ID)
    //      reads PegasusState::getPc() directly — no execution
    //
    //  step()
    //      cosim_->step(CORE_ID, HART_ID)
    //      runs ActionGroup loop: fetch→translate→decode→execute
    //      returns EventAccessor wrapping completed Event
    //      → eventToInfo_(accessor) translates to InstructionInfo
    //      if branch: stores accessor in pending_branch_events_[iss_uid]
    //
    //  stepWithOverridePc(override_pc)
    //      cosim_->step(CORE_ID, HART_ID, override_pc)
    //      internally: setPc(override_pc) then step()
    //      → same eventToInfo_ path as step()
    //
    //  commitInstruction(iss_uid)
    //      look up accessor = pending_branch_events_[iss_uid]
    //      cosim_->commit(accessor)
    //      → evt_pipeline->commitUpTo(euid)
    //      erase from pending_branch_events_
    //
    //  flush(checkpoint)
    //      look up accessor = pending_branch_events_[checkpoint.iss_uid]
    //      cosim_->flush(accessor, flush_younger_only=true)
    //      → evt_pipeline->flush(euid, true, observer, state)
    //      erase all pending_branch_events_ entries with uid >= checkpoint
    //
    //  commitStoreWrite / dropStoreWrite

    class PegasusEDMAdapter : public EDMInterface
    {
      public:
        // workload   — path to ELF binary (passed to PegasusCoSim)
        // ilimit     — instruction count limit (UINT64_MAX = unlimited)
        // params     — extra Pegasus sim params (processParameter calls)
        // db_file    — SimDB output path ("" = no db)
        // snapshot   — CherryPick checkpointer snapshot threshold
        PegasusEDMAdapter(
            const std::string & workload,
            uint64_t ilimit = std::numeric_limits<uint64_t>::max(),
            const std::map<std::string, std::string> & params = {},
            const std::string & db_file = "",
            size_t snapshot_threshold = 1000);


        bool isFinished() const override;
        Addr peekNextPc() const override;

        // Calls cosim_->step(CORE_ID, HART_ID).
        // Translates EventAccessor → InstructionInfo via eventToInfo_.
        // If result is branch, stores EventAccessor in
        // pending_branch_events_[info.iss_uid].
        InstructionInfo step(CoreId core_id, HartId hart_id) override;

        // Calls cosim_->step(CORE_ID, HART_ID, override_pc).
        // Same post-processing as step().
        InstructionInfo stepWithOverridePc(CoreId core_id, HartId hart_id, Addr override_pc) override;

        // Calls cosim_->commit(accessor).
        // Looks up accessor from pending_branch_events_[iss_uid].
        // Removes entry after commit.
        void commitInstruction(CoreId core_id, HartId hart_id uint64_t iss_uid) override;

        void commitStoreWrite(CoreId core_id, HartId hart_id , uint64_t iss_uid) override;
        void dropStoreWrite(CoreId core_id, HartId hartId, uint64_t iss_uid) override;

        // Calls cosim_->flush(accessor, flush_younger_only=true).
        // Looks up accessor from pending_branch_events_[checkpoint.iss_uid].
        // Clears all pending_branch_events_ entries younger than
        // checkpoint.iss_uid.
        void flush(CoreId, core_id, HartId hart_id, const EDMCheckpoint & checkpoint) override;

        // Calls cosim_->getPc(CORE_ID, HART_ID) then setPc.
        void setPc(CoreId core_id, HartId hart_id, Addr pc) override;

      private:
        InstructionInfo eventToInfo_(
            const pegasus::cosim::EventAccessor & accessor);
            
        // These functions translates the registers from Pegasus to something
        // compatible with Olympia
         static std::vector<RegAccess> convertRegAccesses_(
            const std::vector<pegasus::cosim::Event::RegReadAccess> & src);
        static std::vector<RegAccess> convertRegAccesses_(
            const std::vector<pegasus::cosim::Event::RegWriteAccess> & src);
        static std::vector<MemAccess> convertMemAccesses_(
            const std::vector<pegasus::cosim::Event::MemReadAccess> & src);
        static std::vector<MemAccess> convertMemAccesses_(
            const std::vector<pegasus::cosim::Event::MemWriteAccess> & src);
        

        std::unique_ptr<pegasus::cosim::PegasusCoSim> cosim_;

        // Maps iss_uid (Event::getEuid()) → EventAccessor for every
        // branch instruction currently in flight (fetched but not yet
        // committed or flushed).
        // Entries added in step() when info.is_branch == true.
        // Entries removed in commitInstruction() and flush().
        // flush() requires the EventAccessor object directly —
        // this map is the bridge between our iss_uid-based API
        // and Pegasus's EventAccessor-based flush() call.
        std::unordered_map<uint64_t,
                           pegasus::cosim::EventAccessor> pending_branch_events_;
    };

}

Adding a New Backend

  1. Implement the adapter : Create a class ( e.g., WhisperAdapter ) under core/edm/adapters/Whisper that inherits from EDMInterface. Its constructor must accept (const std::string& config_file, const std::string& filename).

  2. Register it : Inside the static lambda in EDMBackendFactory::create() in EDMFactory.cpp, add:

EDMBackendFactory::registerBackend(
       "whisper",
       [](const std::string & cfg, const std::string & fn) {
           return std::make_unique<WhisperAdapter>(cfg, fn);
       });

  1. Guard with a feature flag : wrap the include and registration in #ifdef WHISPER_AVAILABLE.

  2. Update CMakeLists.txt : use FetchContent or ExternalProject or however you wish to point and attach your backend to Olympia.

  3. Select a runtime : in your arch YAML set fetch.params.backend: "whisper" and "fetch.params.backend_config_file: "config/whisper.yaml"`.

Yaml Configuration ( Planned - future implementation )

Below is a proposed configuration that would direct and help in testing the speculative execution of Olympia:

rules:
  ## Lets consider that we know that we want to divert to model going to a different pc instead of a particular one 
  ## then we have this - type = at_pc :  0x10048 -> divert it to the pc 0x10200.
  - trigger:
      type: at_pc
      pc: 0x10048        # hex PC 
    action:
      type: redirect_to_pc
      target_pc: 0x10200 # where our pc will be diverted to 
    fire_limit: 1        # only fire once


  # After this n number of brnaches, when you encounter the next beq or any branch condition 
  # take the wrong path - and run for 20 wrong path or how many are present in the binary
  # and then flush everything
  - trigger:
      type: after_n_insts
      n: 500
    action:
      type: take_wrong_path
      wrong_path_limit: 20
    fire_limit: 3        # a limit of 3

  # every time we hit this PC, flip to wrong path indefinitely
  # until the pipeline flushes naturally
  - trigger:
      type: at_pc
      pc: 0x10084
    action:
      type: take_wrong_path
      wrong_path_limit: 0  # no auto-flush
    fire_limit: 0           # fire every time we hit this PC

Walk through the 6 interface methods and map them to the backend:

step()

  • Pegasus: cosim_->step() runs the ActionGroup loop, fires the CoSimObserver, creates an Event, assigns EUID, returns EventAccessor. We then translate the Event to Inst via the eventToInst_()
  • Whisper: fetch() + execute() are two seperate calls inside whisper. We call both - which are boolean signifying success or failure and then we call getInstructionPacket(hartIx, tag), which returns the InstrPac and then we call packetToInst_()

stepWithOverridePc()

  • Pegasus: : setPc() first needs to be called, then we will step().
  • Whisper: : The same fetch() call is to be used, it has some sanity checks for tags but it allows us to fetch() at any opcode.

commitInstruction()

  • Pegasus: cosim_->commit(EventAccessor&) - commits everything upto the EUID, moves event from uncommitted_evts_buffer_ to committed_evts_buffer_
  • Whisper: retire() + drainStore(). We need two steps because non store packets are removed from hartPacketMaps at retire. Store packets stay until drainStore() writes to memory via pokeMemory().

flush()

  • Pegasus: This call needs EventAccessor object, not just the EUID - hence we are keeping a mapping of pending_branch_events_ map. It calls CoSimEventPipeline::flush() which reverts the state from to the EventAccessor, then calls reload_event() which restores registers using the CherryPickPointer checkpoints.
  • Whisper: Takes just the tag, iterates hartPacketMaps in reverse undoes register renaming via hartRegProducers, erases from both hartPacketMaps and hartStoreMaps.

peekNextPc()

This method is just for the speculative execution - to see which pc are we going to encounter next, without using the step and creating any changes to the state.

  • Pegasus: cosim_->getPc() reads PegasusState::getPc() directly. This creates zero side effects to the state.
  • Whisper: nextPc can be obtained from the previous IntrPac.

isFinished()

  • Pegasus: isSimulatorFinished() is the call to be used here.
  • Whisper: We could keep the track of the nextPc() inside the different InstrPac

How everything maps to the current state:

The EDMInstrGenerator inherit from InstGenerator which has the following methods to override: getNextInst(), isDone and reset

Here is the three could be implemented :

getNextInst()

InstrPtr EDMInstGenerator::getNextInt(...)
{
    Addr next_pc = edm_->peekNextPc(core_id, hart_id);

    SteeringDecision decision = evaluateRules_(next_pc);

    InstructionInfo info;
    switch(decision.action) {
        case STEP_NORMAL:
            info = edm_->step(core_id, hart_id);
            break;
        
        case STEP_WITH_OVERRIDE:
            info = edm_->stepWithOverridePc(
                core_id, hart_id, decision.override_pc
            );
            break; 
    }

    if (info.is_branch) 
    {
        saveCheckpoint_(info)
    }

    return buildInst_(info);
}

getNextInst() uses: peekNextPc(), step, stepWithOverride

isDone()

bool EDMInstGenerator::isDone() const 
{
    return edm_->isFinished(core_id, hart_id);
}

This only uses the isFinished

reset()

void EDMIntGenerator::reset(const InstrPtr& flush_inst)
{
    auto check_point = get_insruction(checkpoint_queue_);
    // We get the instruction

    edm_->flush(core_id, hart_id, check_point);
    checkpoint_queue_.erase(it, checkpoint_queue_end());
}

This only uses the flush

commitInstructions

The communication on the status of the instructions via the backend and olympia is done via the notify method that exist in the Inst object itself. The Inst Object knows the backend and calls the backend methods when required.

Testing

For the purpose of testing we aim to have coverage of the Drystone and the Coremark Tests. If time permits we also aim to have the two backends run simultaneously at the same time, and compare the results and the memory behaviors between them. In addition to Dhrystone and CoreMark, we will use targeted microbenchmarks (e.g., branches over loads) to cause mispredictions and verify the correctness of the instruction stream recovery using a simulator-to-simulator 'bridge' API

RISC-V Performance Model

Clone this wiki locally