AIA: DTS riscv,guest-index-bits mismatch with GEILEN breaks SMP boot

[matthewgui]

IMSIC Device Tree Configuration: Matching riscv,guest-index-bits with Hardware GEILEN

1. Overview

When using the RISC-V AIA (Advanced Interrupt Architecture) interrupt controller, the riscv,guest-index-bits property in the Device Tree must strictly match the hardware GEILEN (Guest External Interrupt Length) parameter. Misconfiguration will cause SMP multi-core system boot failures.

This document aims to explain:

• How the hardware GEILEN parameter affects IMSIC address alignment
• How the software guest-index-bits property affects IMSIC address calculation
• The root cause why these two must match

---

2. Hardware Perspective: GEILEN and Address Alignment

2.1 GEILEN Parameter Definition

GEILEN (Guest External Interrupt Length) is a hardware-implemented parameter that specifies the number of virtual interrupt files supported per CPU core.

• GEILEN = 3 means each core has 3 virtual interrupt files (VS0, VS1, VS2)
• GEILEN = 4 means each core has 4 virtual interrupt files (VS0, VS1, VS2, VS3)

2.2 Address Alignment Rules

Each IMSIC interrupt file is 4KB in size (IMSIC_MMIO_PAGE_SZ).

All interrupt files (physical + virtual) for a core are arranged contiguously in the physical address space, and the entire block is aligned to 2^(ceil(log2(GEILEN + 1))).

Specifically, the IMSIC address block size per core is:

align_size = 2^(ceil(log2(GEILEN + 1))) * 4KB

2.3 Address Layout Examples

Example 1: GEILEN = 3 (3 virtual interrupt files)

• Total files per core: 1 (S-mode) + 3 (VS-mode) = 4
• Alignment requirement: 2^(ceil(log2(4))) = 4 → 4 × 4KB = 16KB alignment
• Actual usage: 4 × 4KB = 16KB (no padding)

Core0: [S | VS0 | VS1 | VS2]   ← 16KB, no padding
Core1: [S | VS0 | VS1 | VS2]   ← 16KB, no padding
Core2: [S | VS0 | VS1 | VS2]   ← 16KB, no padding
...

When GEILEN + 1 is a power of two, addresses are tightly packed with no padding.

Example 2: GEILEN = 4 (4 virtual interrupt files)

• Total files per core: 1 (S-mode) + 4 (VS-mode) = 5
• Alignment requirement: 2^(ceil(log2(5))) = 8 → 8 × 4KB = 32KB alignment
• Actual usage: 5 × 4KB = 20KB, 12KB padding

Core0: [S | VS0 | VS1 | VS2 | VS3 | PAD | PAD | PAD]  ← 32KB
Core1: [S | VS0 | VS1 | VS2 | VS3 | PAD | PAD | PAD]  ← 32KB
Core2: [S | VS0 | VS1 | VS2 | VS3 | PAD | PAD | PAD]  ← 32KB
...

When GEILEN + 1 is not a power of two, padding (PAD) regions exist in the hardware address space.

2.4 Hardware Address Layout Summary

GEILEN	Virtual Files	Total Files per Core	Alignment Multiplier	Actual Usage	Padding
0	0	1	1	4KB	None
1	1	2	2	8KB	None
2	2	3	4	12KB	4KB
3	3	4	4	16KB	None
4	4	5	8	20KB	12KB
5	5	6	8	24KB	8KB
6	6	7	8	28KB	4KB
7	7	8	8	32KB	None

---

3. Software Perspective: guest-index-bits and Address Calculation

3.1 Property Definition

The riscv,guest-index-bits property in the Device Tree specifies the bit width used to encode the Guest index in the MSI target address.

This value must equal ceil(log2(GEILEN + 1)), which is the alignment multiplier for each core's IMSIC address block.

Important Note: This property is only used for S-mode MSI address configuration. M-mode MSI addresses do not require this property.

3.2 Linux Kernel Address Calculation Logic

In the Linux kernel's drivers/irqchip/irq-riscv-imsic.c, each CPU's IMSIC address is calculated as follows:

/*
 * Calculate IMSIC offset for each core
 * i: current CPU index
 * guest_index_bits: value configured in DTS
 */
reloff = i * BIT(global->guest_index_bits) * IMSIC_MMIO_PAGE_SZ;

Key Code Analysis

/* Offset per core = core_index × 2^guest_index_bits × 4KB */
reloff = i * BIT(global->guest_index_bits) * IMSIC_MMIO_PAGE_SZ;

/* Locate which MMIO region this core belongs to */
for (j = 0; nr_mmios; j++) {
    if (reloff < resource_size(&mmios[j])) {
        index = j;
        break;
    }
    /* Align to guest_index_bits boundary when crossing regions */
    reloff -= ALIGN(resource_size(&mmios[j]),
                    BIT(global->guest_index_bits) * IMSIC_MMIO_PAGE_SZ);
}

/* Final MSI address = base + offset */
local->msi_pa = mmios[index].start + reloff;

3.3 Why They Must Match

The Linux kernel uses guest_index_bits to calculate the MSI address offset per core:

Stride per core = 2^(guest_index_bits) × 4KB

The actual hardware layout has the same stride per core:

Stride per core = 2^(ceil(log2(GEILEN + 1))) × 4KB

If guest_index_bits ≠ ceil(log2(GEILEN + 1)):

• The software-calculated address offset does not match the actual hardware layout
• The kernel may access MMIO addresses that fall into padding regions or beyond hardware range
• Interrupts cannot be delivered correctly, ultimately causing system boot failure

---

4. Configuration Requirements and Examples

4.1 Configuration Rule

riscv,guest-index-bits = ceil(log2(GEILEN + 1))

4.2 Correct Configuration Examples

GEILEN	Virtual Files	guest-index-bits	Description
0	0	0	2^0 = 1 alignment
1	1	1	2^1 = 2 alignment
2	2	2	2^2 = 4 alignment
3	3	2	2^2 = 4 alignment
4	4	3	2^3 = 8 alignment
5	5	3	2^3 = 8 alignment
6	6	3	2^3 = 8 alignment
7	7	3	2^3 = 8 alignment

4.3 Device Tree Configuration Example

  imsic_s: interrupt-controller@4f200000 {
    compatible = "riscv,imsics";
    msi-controller;
    /* 4*4KB × 8 cores = 128KB, MSI device region supports up to 16 cores */
    reg = <0x00000000 0x4f200000 0x00000000 0x40000>;  
    interrupts-extended = < &cpu0_intc 9 &cpu1_intc 9
                            &cpu2_intc 9 &cpu3_intc 9
                            &cpu4_intc 9 &cpu5_intc 9
                            &cpu6_intc 9 &cpu7_intc 9>;
    interrupt-controller;
    #interrupt-cells = <0>;
    riscv,num-ids = <0xff>;
    /* Hardware GEILEN = 3 → ceil(log2(4)) = 2 */
    riscv,guest-index-bits = <2>; 
  };

---

5. Frequently Asked Questions

5.1 Nuclei Evalsoc AIA Features

Currently, only the Nuclei 1000 series supports the AIA controller. Key features of the Evalsoc AIA on the 1000 series:

1. Supports up to 16-core SMP auto-detection. If the actual number of CPU cores is less than the number configured in the DTS, Linux can auto-detect the actual core count via IPI during bringup secondary cpu without hanging.

2. Interrupt files are not grouped, so the riscv,group-index-bits property does not need to be configured in the DTS, or can be set to 0.

3. The most important IMSIC parameters in the DTS are riscv,num-ids (number of interrupt IDs) and riscv,guest-index-bits (alignment for S-mode MSI configuration). For other IMSIC properties, refer to the Linux kernel documentation: Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml.

5.2 Nuclei QEMU Runtime Notes

1. To run the Nuclei Linux SDK (with H-extension and AIA-extension support) on Nuclei QEMU, the minimum QEMU version required is 2025.12.10.

2. In the Nuclei Linux SDK branch dev_nuclei_6.12_v3_hypervisor_aia, the QEMU runtime parameters for Hypervisor and AIA are configured in linux_sdk/conf/evalsoc/build.mk. Note that the parameter aia-guests=4 represents the hardware GEILEN value.

---

6. Summary

• Hardware GEILEN determines the number of virtual interrupt files per core and affects the physical alignment of IMSIC addresses.
• Software guest-index-bits determines the stride used by the kernel to calculate each core's MSI address.
• Both must strictly match (guest_index_bits = ceil(log2(GEILEN + 1))); otherwise, the system will fail to boot or interrupt functionality will be broken.
• Always consult the hardware specification when configuring to ensure the Device Tree parameters are consistent with the hardware implementation.