Clean Prefetcher implementation

[pm-ju]

Resolves #142

@klingaard @arupc @rajatbhatia1 can you review this one

[klingaard]

Can you create an arch that has the big core with the prefetcher enabled? Would be interesting to compare reports.

[pm-ju]

this is the results I got after comparing `big_core` vs `big_core_with_prefetcher` using `dhry_riscv.zstf` (2M instructions) I got after comparing @klingaard also I also made changes to `DCache.cpp` to handle prefetch requests that have no `InstPtr` (they're synthetic requests from the prefetch engine). Can you review those?

[rajatbhatia1]

Please comment on what 2 and 64 mean here

[pm-ju]

I have added a comment for better clarity.

[rajatbhatia1]

This is extremely well written implementation of the prefetcher.

I have pointed a few places that may need improvement.

[rajatbhatia1]

We have tested that stride prefetcher is generating prefetches. However we should also check that correct prefetch addresses are generated

[rajatbhatia1]

What interface is used to restore prefetch_credits_?

[pm-ju]

I have added restorePrefetchCredit() as a public method on Prefetcher. DCache calls it when a prefetch request completes (hit or miss resolved), which increments prefetcher_credits_ and re-schedules generation if the engine has pending prefetches.

[rajatbhatia1]

Please add assertion on table size and cache line size to be positive

[rajatbhatia1]

I think we should limit the confidence to a max value, otherwise there is a risk of overflow.

[pm-ju]

@rajatbhatia1 I have done the changes you recommended. Can you check it if they are correct?

[rajatbhatia1]

Here is an example of how credit mechanism is used in Olympia

On Producer side (see Dispatcher.cpp/hpp)

1. Create a data type to hold the credits
   uint32_t unit_credits_ = 0;

2. Only sends a packet when credit > 0

bool canAccept() const {
            // The dispatcher must be have enough credits in the
            // execution pipe AND still has bandwidth
            return (unit_credits_ != 0) && (num_can_dispatch_ != 0);
        }

3. Consume (decrement) credit when sending the packet

out_inst_->send(inst);
 --unit_credits_;

4. Have a port to receive the credits from the Receiver
   in_credits->registerConsumerHandler(CREATE_SPARTA_HANDLER_WITH_DATA(Dispatcher, receiveCredits_, uint32_t));

5. Have a function that increments the credits when received on the port

void Dispatcher::receiveCredits_(const uint32_t & credits) {
        unit_credits_ += credits;
        ILOG(name_ << " got " << credits << " credits, total: " << unit_credits_);

        dispatch_->scheduleDispatchSession();
    }

On Receiver side (e.g. see IssueQueue.hpp/cpp)

1. Declare port to send credits

sparta::DataOutPort<uint32_t> out_scheduler_credits_{&unit_port_set_,
                                                             "out_scheduler_credits"};

2. On startup (setupIssueQueue_)
   out_scheduler_credits_.send(scheduler_size_);

3. When instructions vacate the queue (popIssueQueue_)
   out_scheduler_credits_.send(1, 0); // send credit back to dispatch, we now have more room in IQ

4. On Flush (flushInst_)

++credits_to_send (for each flushed element in issue queue)
if (credits_to_send)
        {
            out_scheduler_credits_.send(credits_to_send, 0);

            ILOG("Flush " << credits_to_send << " instructions in issue queue!");
        }

[Ma-gi-cian]

Hi @pm-ju,

Thanks for the work on the prefetcher and the communication earlier today.
We would like to keep this PR clean and follow the refactoring that is being introduced in the #298 . Could you organize the files in this manner :

core/prefetcher/
├── CMakeLists.txt
├── Prefetcher.cpp
├── Prefetcher.hpp
├── PrefetcherIF.hpp
└── engines/
    ├── NextLinePrefetchEngine.cpp
    ├── NextLinePrefetchEngine.hpp
    ├── StridePrefetchEngine.cpp
    └── StridePrefetchEngine.hpp

Important Notes:

• Please do not move any files that already existed in the repository before this Pr (like lsu, test, CPUFactories.hpp, CPUTopology.cpp, MemoryAccessInfo.hpp)
• The test/core/prefetcher directory can stay as it is.
• Only reorganize the new prefetcher files you added in this PR.

Could you also point me to where the DCache is restoring the credits of the prefetcher, I was not able to find it. Or is it still a work in progress.

Let me know if you run into any issue with restructuring or anything else.

Thanks again for your contribution.

[pm-ju]

Hey, I've pushed the latest changes which include both the file reorganization and the complete refactoring of the Prefetcher credit system to use a decoupled, port-based Sparta signaling architecture.

Here is a summary of the structural changes implemented:

1. File Reorganization:

• Moved all prefetcher-related files into core/prefetcher/ and the engines into core/prefetcher/engines/.

• Updated all CMake configurations and #include paths across the project to reflect the new structure.

2. Prefetcher Credit System Refactoring

• Decoupled Architecture: Replaced the direct C++ function call (restorePrefetchCredit()) with a standard Sparta port-based mechanism.

• Port Implementation: Added sparta::DataInPort<uint32_t> in_prefetcher_credits_ to Prefetcher and sparta::DataOutPort<uint32_t> out_prefetch_credits_ to DCache.

• Topology: Updated CPUTopology.cpp to bind the new ports.

3. DCache Arbitration & Deadlock Prevention

• Realized that the Prefetcher and LSU were both bound to the same DCache input port (in_lsu_lookup_req_), causing collisions.

• Added a dedicated in_prefetcher_req_ port to DCache.

• Refactored DCache::arbitrateL2LsuReq_ into a multi-cycle arbitrator to safely multiplex requests between L2 Refill, LSU, and the Prefetcher without silently dropping packets.

• Implemented credit returns across all exit paths (L1 hit, MSHR full, L2 refill return, and L1 refill deallocation).

Current Status & Request for Help: Despite these structural improvements, the CI tests are still aborting (Subprocess aborted). I've spent some time investigating the test failures but I haven't been able to pinpoint the exact sequence of events that's continuing to trigger the aborts.

Could I get some extra time to investigate this via the CI logs, or would someone from the team be able to shed some light on what might be triggering the remaining aborts? Any guidance or pointers on debugging this specific failure would be greatly appreciated!

[Ma-gi-cian]

The above changes should clear all your prefetcher and regression tests, for example here is the __ olympia_arch_big_core_with_prefetcher_test__ .

jha@MAGICIAN:~/workspace/opensource/riscv-perf-model/release/test/sim$ ctest -R olympia_arch_big_core_with_prefetcher_test --verbose
UpdateCTestConfiguration  from :/home/jha/workspace/opensource/riscv-perf-model/release/test/sim/DartConfiguration.tcl
UpdateCTestConfiguration  from :/home/jha/workspace/opensource/riscv-perf-model/release/test/sim/DartConfiguration.tcl
Test project /home/jha/workspace/opensource/riscv-perf-model/release/test/sim
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 14
    Start 14: olympia_arch_big_core_with_prefetcher_test

14: Test command: /home/jha/workspace/opensource/riscv-perf-model/release/olympia "-i500K" "--workload" "traces/dhry_riscv.zstf" "--arch" "big_core_with_prefetcher"
14: Working Directory: /home/jha/workspace/opensource/riscv-perf-model/release/test/sim
14: Test timeout computed to be: 10000000
14: # Name:     Olympia RISC-V Perf Model
14: # Cmdline:  /home/jha/workspace/opensource/riscv-perf-model/release/olympia -i500K --workload traces/dhry_riscv.zstf --arch big_core_with_prefetcher
14: # Exe:      /home/jha/workspace/opensource/riscv-perf-model/release/olympia
14: # SimulatorVersion: v0.1.0
14: # Repro:    Git SHA: 66d7060
14: # Start:    Sunday Sun Apr  5 15:32:53 2026
14: # Elapsed:  0.002476s
14: # Sparta Version: map_v2.1.15
14:   [PARAMETER INCLUDE NOTE] : Including "arches/big_core.yaml"
14:   [PARAMETER INCLUDE NOTE] : Including "arches/medium_core.yaml"
14:   [PARAMETER INCLUDE NOTE] : Including "arches/small_core.yaml"
14:   [in] Arch Config: ArchCfg Node "" <- file: "/home/jha/workspace/opensource/riscv-perf-model/arches/big_core_with_prefetcher.yaml"
14:
14: Setting up Simulation Content...
14: Resources:
14:   cpu
14: Building tree...
14: Configuring tree...
14: Finalizing tree...
14: Inst Allocator: 1238 Inst objects allocated/created
14: Inst Allocator: 1238 Inst objects allocated/created
14: Inst Allocator: 0 Inst objects allocated/created
14: Inst Allocator: 0 Inst objects allocated/created
14: Inst Allocator: 1238 Inst objects allocated/created
14:     NOTE: unread optional unbound parameter: "top.cpu.core0.extension.core_extensions.exe_pipe_rename" from: "". value: "[[exe0,sys_pipe],[exe1,alu1_pipe],[exe2,alu2_pipe],[exe3,alu3_pipe],[exe4,alu4_pipe],[exe5,alu5_pipe],[exe6,fpu0_pipe],[exe7,fpu1_pipe],[exe8,br0_pipe],[exe9,br1_pipe],[exe10,vint_pipe]]". Path exists in tree up to: "top.cpu.core0"
14:     NOTE: unread optional unbound parameter: "top.cpu.core0.extension.core_extensions.issue_queue_to_pipe_map" from: "". value: "[[0,1],[2,3],[4,5],[6,7],[8,9],[10]]". Path exists in tree up to: "top.cpu.core0"
14:     NOTE: unread optional unbound parameter: "top.cpu.core0.extension.core_extensions.pipelines" from: "". value: "[[sys],[int,div],[int,mul],[int,mul,i2f,cmov],[int],[int,vset],[float,faddsub,fmac],[float,f2i],[br],[br],[vint,vdiv,vmul,vfixed,vmask,vmv,v2s,vfloat,vfdiv,vfmul,vpermute,vload,vstore]]". Path exists in tree up to: "top.cpu.core0"
14: Preparing to run...
14: Meta-Parameters:
14:   architecture: big_core_with_prefetcher
14:   is_final_config: false
14: Non-default model parameters: 17
14: Running...
14: olympia: STF file input detected
14: Running Complete
14:   Simulation Performance      : wall(31.7200), system(0.0600), user(32.6200)
14:   Scheduler Tick Rate  (KTPS): 18.1644  (1k ticks per second)
14:   Scheduler Event Rate (KEPS): 645.636 KEPS (1k events per second)
14:   Scheduler Events Fired: 21060649
14: Run Successful!
14: Saving reports...
14: Inst Allocator: 1238 Inst objects allocated/created
14: Inst Allocator: 1238 Inst objects allocated/created
14: Inst Allocator: 316 Inst objects allocated/created
1/1 Test #14: olympia_arch_big_core_with_prefetcher_test ...   Passed   32.20 sec

The following tests passed:
        olympia_arch_big_core_with_prefetcher_test

100% tests passed, 0 tests failed out of 1

Total Test time (real) =  32.21 sec
jha@MAGICIAN:~/workspace/opensource/riscv-perf-model/release/test/sim$

[Ma-gi-cian]

The Dcache test is still failing, you can investigate it via going into the test/core/dcache folder and running the following command :

jha@MAGICIAN:~/workspace/opensource/riscv-perf-model/release/test/core/dcache$ ctest --verbose
UpdateCTestConfiguration  from :/home/jha/workspace/opensource/riscv-perf-model/release/test/core/dcache/DartConfiguration.tcl
UpdateCTestConfiguration  from :/home/jha/workspace/opensource/riscv-perf-model/release/test/core/dcache/DartConfiguration.tcl
Test project /home/jha/workspace/opensource/riscv-perf-model/release/test/core/dcache
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
    Start 1: Dcache_test_arbitrate

1: Test command: /home/jha/workspace/opensource/riscv-perf-model/release/test/core/dcache/Dcache_test "arbitrate.out" "-c" "test_arches/1_src_Dcache.yaml" "--input-file" "next_lvl_cache_refill.json"
1: Working Directory: /home/jha/workspace/opensource/riscv-perf-model/release/test/core/dcache
1: Test timeout computed to be: 10000000
1:   [in] Configuration: Node "" <- file: "test_arches/1_src_Dcache.yaml"
1: 
1: Setting up Simulation Content...
1: Resources:
1:   
1: Building tree...
1: Configuring tree...
1: Finalizing tree...
1: Inst Allocator: 1238 Inst objects allocated/created
1: Inst Allocator: 1238 Inst objects allocated/created
1: Inst Allocator: 0 Inst objects allocated/created
1: Inst Allocator: 0 Inst objects allocated/created
1: Inst Allocator: 1238 Inst objects allocated/created
1: olympia: JSON file input detected
1: Preparing to run...
1: Meta-Parameters:
1:   architecture: NONE
1:   is_final_config: false
1: Non-default model parameters: 5
1: Running...
1: Running Complete
1:   *** Simulation Performance cannot be measured -- no user time detected. Did the simulator run long enough?
1:   Scheduler Events Fired: 41
1: Run Successful!
1: Saving reports...
1: File comparison test between "arbitrate.out" and "expected_output/arbitrate.out.EXPECTED" FAILED on line 143 in file /home/jha/workspace/opensource/riscv-perf-model/test/core/dcache/Dcache_test.cpp
1:   Exception: Files differed at pos 460 (line 2, col 58) with chars: 'A' != 'R'
1: 
1: Inst Allocator: 1238 Inst objects allocated/created
1: Inst Allocator: 1238 Inst objects allocated/created
1: Inst Allocator: 2 Inst objects allocated/created
1: 
1: 1 ERROR(S) found during test.
1: 
1/1 Test #1: Dcache_test_arbitrate ............***Failed    0.19 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.19 sec

The following tests FAILED:
          1 - Dcache_test_arbitrate (Failed)
Errors while running CTest
Output from these tests are in: /home/jha/workspace/opensource/riscv-perf-model/release/test/core/dcache/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.

But this is just a comparison between a new arbitrate.out and expected_output/arbitrate.out.EXPECTED. You can see the diff via this :

jha@MAGICIAN:~/workspace/opensource/riscv-perf-model/release/test/core/dcache$ diff arbitrate.out expected_output/arbitrate.out.EXPECTED
6,7c6,7
< #Start:    Sunday Sun Apr  5 15:36:49 2026
< #Elapsed:  0.022949s
---
> #Start:    Saturday Sat Oct 19 15:35:23 2024
> #Elapsed:  0.002073s
10c10
< {0000000000 00000000 top.dcache info} arbitrateL2LsuReq_: Arbitrating LSU request memptr: deadbeef uid:0 BEFORE_FETCH 0 pid:1 uopid:0 'lw     5,3' 
---
> {0000000000 00000000 top.dcache info} arbitrateL2LsuReq_: Received LSU request memptr: deadbeef uid:0 BEFORE_FETCH 0 pid:1 uopid:0 'lw        5,3' 
30c30
< {0000000007 00000007 top.dcache info} arbitrateL2LsuReq_: Arbitrating L2 Refill request memptr: deadbeef uid:0 BEFORE_FETCH 0 pid:1 uopid:0 'lw       5,3' 
---
> {0000000007 00000007 top.dcache info} arbitrateL2LsuReq_: Received Refill request memptr: deadbeef uid:0 BEFORE_FETCH 0 pid:1 uopid:0 'lw     5,3' 
37c37
< {0000000008 00000008 top.dcache info} arbitrateL2LsuReq_: Arbitrating LSU request memptr: deedbeef uid:1 BEFORE_FETCH 0 pid:2 uopid:0 'lw     5,3' 
---
> {0000000008 00000008 top.dcache info} arbitrateL2LsuReq_: Received LSU request memptr: deedbeef uid:1 BEFORE_FETCH 0 pid:2 uopid:0 'lw        5,3' 
63c63
< {0000000015 00000015 top.dcache info} arbitrateL2LsuReq_: Arbitrating L2 Refill request memptr: deedbeef uid:1 BEFORE_FETCH 0 pid:2 uopid:0 'lw       5,3' 
---
> {0000000015 00000015 top.dcache info} arbitrateL2LsuReq_: Received Refill request memptr: deedbeef uid:1 BEFORE_FETCH 0 pid:2 uopid:0 'lw     5,3' 
jha@MAGICIAN:~/workspace/opensource/riscv-perf-model/release/test/core/dcache$ 

You would have to do cp arbitrate.out expected_output/arbitrate.out.EXPECTED . But I would wait for the maintainers to review it. This is like a comparison between the output and a golden.out file.

[Ma-gi-cian]

The tests should pass and here is the diff for your reference.

        Start 123: ICache_test_single_access
143/153 Test #125: ICache_test_random .......................................................   Passed    0.20 sec
        Start 147: UNIT_TMPL_test
144/153 Test #123: ICache_test_single_access ................................................   Passed    0.03 sec
        Start 124: ICache_test_simple
145/153 Test #147: UNIT_TMPL_test ...........................................................   Passed    0.03 sec
        Start 150: Prefetcher_test_nextline
146/153 Test #107: L2Cache_test_hit_case ....................................................   Passed    0.25 sec
        Start 151: Prefetcher_test_stride
147/153 Test #124: ICache_test_simple .......................................................   Passed    0.03 sec
        Start 152: Prefetcher_test_edge_cases
148/153 Test #150: Prefetcher_test_nextline .................................................   Passed    0.02 sec
149/153 Test #151: Prefetcher_test_stride ...................................................   Passed    0.01 sec
150/153 Test #152: Prefetcher_test_edge_cases ...............................................   Passed    0.01 sec
151/153 Test #149: UNIT_TMPL_json_test ......................................................   Passed    0.17 sec
152/153 Test #108: Rename_test_Run_Small ....................................................   Passed    0.78 sec
153/153 Test  #24: olympia_arch_big_core_1_custom_core ......................................   Passed   48.38 sec

99% tests passed, 1 tests failed out of 153

Total Test time (real) = 1347.03 sec

The following tests FAILED:
        126 - Dcache_test_arbitrate (Failed)
Errors while running CTest
Output from these tests are in: /home/jha/workspace/opensource/riscv-perf-model/release/test/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
make[3]: *** [test/CMakeFiles/regress.dir/build.make:70: regress] Error 8
make[2]: *** [CMakeFiles/Makefile2:2319: test/CMakeFiles/regress.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:2326: test/CMakeFiles/regress.dir/rule] Error 2
make: *** [Makefile:878: regress] Error 2
jha@MAGICIAN:~/workspace/opensource/riscv-perf-model/release$ git diff
diff --git a/core/prefetcher/Prefetcher.cpp b/core/prefetcher/Prefetcher.cpp
index cb3697e..b22b9ad 100644
--- a/core/prefetcher/Prefetcher.cpp
+++ b/core/prefetcher/Prefetcher.cpp
@@ -59,7 +59,10 @@ namespace olympia
     {
         // Queue incoming buffer
         req_queue_.push(mem_access_info_ptr);
-        ev_handle_incoming_req_.schedule(sparta::Clock::Cycle(0));
+        if(!ev_handle_incoming_req_.isScheduled())
+        {
+            ev_handle_incoming_req_.schedule(sparta::Clock::Cycle(0));
+        }
     }

     //! \brief Override handleMemoryAccess to use credit-based flow control
@@ -70,7 +73,7 @@ namespace olympia
         if (getPrefetchEngine()->handleMemoryAccess(access))
         {
             // Don't send prefetches immediately — schedule credit-based generation
-            if (prefetcher_credits_ > 0)
+            if (prefetcher_credits_ > 0 && !ev_gen_prefetch_.isScheduled())
             {
                 ev_gen_prefetch_.schedule(sparta::Clock::Cycle(0));
             }
@@ -91,9 +94,9 @@ namespace olympia
             handleMemoryAccess(access);
         }

-        if (!req_queue_.empty())
+        if (!req_queue_.empty() && !ev_handle_incoming_req_.isScheduled())
         {
-            ev_handle_incoming_req_.schedule(sparta::Clock::Cycle(1));
+            ev_handle_incoming_req_.schedule(sparta::Clock::Cycle(0));
         }
         return;
     }
jha@MAGICIAN:~/workspace/opensource/riscv-perf-model/release$

Again, thanks for your work

[klingaard]

Thanks for jumping in, Aditya! Pritish, when you "deleted" the files, did you use git mv? If not, you've lost history on those files (unless you know where to look).

[pm-ju]

Thanks @Ma-gi-cian for that commit, that sorted out the problem and sorry I am having my exams so couldn't give much time to this and @klingaard I had moved those files not by command but by manual work as something is wrong with my repo and if I use git commands it may pollute the whole branch like my last PR.

[Ma-gi-cian]

@rajatbhatia1 please hold off merging this, although the tests are passing, the credit mechanism still has some bugs. The next_line prefetcher is working - it fetches aggressively but it works - ( the ipc in this went down ) but the stride prefetcher is not being run.

Diff between normal big_core and big_core with stride prefetcher

      Report "top.cpu.core0.prefetcher"                                               Report "top.cpu.core0.prefetcher"
          cnt_req_rcvd = 376812                                                           cnt_req_rcvd = 376812
          cnt_prefetch_sent = 0                                                           cnt_prefetch_sent = 0

Maybe it is something to do with the confidence or something will update after fixing it - will also put a diff of the report. Thanks

[Ma-gi-cian]

I have found a couple of issues for the stride prefetcher. The next line works

1. Counters in the base class PrefetcherIF.hpp are stuck at zero. The base class has the functions PrefetcherIF::handleMemoryAccess and the rest that holds the code for incrementing cnt_req_rcvd_ and cnt_prefetch_snd_. However, Prefetcher.cpp override handleMemoryAccess with the credit based version that does not update these variables. They are always at zero.

cnt_req_rcvd    = 0
cnt_prefetch_sent = 0

This is with the current stride prefetcher.

I got around this via making these variables protected in the PrefetcherIF.hpp and incrementing them manually in the Prefetcher.cpp file

2. The StridePrefetchEngine never schedules a prefetch. I believe it is partly because the conditions for generating the prefetch are very strict but this is the output that I got after making the protected changes :

 Report "top.cpu.core0.prefetcher"
          cnt_req_rcvd = 376812                                                           
          cnt_prefetch_sent = 0                                                           

If anyone could look at this and provide with some guidance would be great. Thanks.

[klingaard]

@pm-ju can you address the issues that @Ma-gi-cian points out?

[pm-ju]

Hey @Ma-gi-cian I have fixed those issue you were facing. Can you check ?