Skip to content

target: add NVIDIA H20 support - #4

Open
Fucov wants to merge 1 commit into
RightNow-AI:mainfrom
Fucov:feat/h20-target
Open

target: add NVIDIA H20 support#4
Fucov wants to merge 1 commit into
RightNow-AI:mainfrom
Fucov:feat/h20-target

Conversation

@Fucov

@Fucov Fucov commented Sep 10, 2026

Copy link
Copy Markdown

Summary

Add NVIDIA H20 as a first-class GpuTarget and validate the existing Hopper sm_90 AutoMegaKernel path on real H20 hardware.

This is intentionally a minimal hardware-retargeting change: H20 reuses the existing SM90 CUDA implementation and requires no H20-specific scheduler, VM, ABI, or kernel branch.

H20 target

Validated on an NVIDIA H20 96GB system:

  • Compute capability: 9.0 (sm_90)
  • SMs: 78
  • CUDA-visible memory: ~95.1 GiB
  • L2 cache: 60 MiB
  • Shared memory / SM: 233472 B
  • Maximum opt-in shared memory / block: 232448 B
  • Nominal HBM bandwidth used by the target: 4000 GB/s

Fresh eval/peak_bandwidth.py measurements:

run 1: 3714.1 GB/s
run 2: 3712.3 GB/s
run 3: 3710.7 GB/s

The target records the median result:

measured_bw_gbs = 3712.3

which is approximately 92.8% of the nominal bandwidth.

Validation

H20 target registration and CUDA execution:

amk doctor                              PASS
SM90 CUDA JIT build                     PASS
ReferenceVM vs eager                    PASS
CUDA megakernel correctness             PASS
multi-token generation vs eager         PASS

The 8-token generation test reports:

matches_eager = true

Correctness-gated single-stream decode baselines:

toy:
  eager   2.0876 ms/token
  AMK     0.3653 ms/token
  speedup 5.72x

llama:
  eager   3.3003 ms/token
  AMK     0.6349 ms/token
  speedup 5.20x

The H20/IR/ABI targeted regression suite passes:

25 passed

including the frozen validator and ABI regression tests.

Existing upstream test status

The full test suite on the H20 host reports:

115 passed, 2 failed

The two failures are:

tests/test_cuda_fp16.py::test_cuda_attention_head_dim_256_static
tests/test_search.py::test_search_rejects_invalid_configs_without_crashing

Both failures were reproduced unchanged on pristine upstream/main at commit a514bbc on the same machine, before applying the H20 target change.

Similarly, ruff check . currently reports five violations in the untouched instructions/reference.py; this PR does not modify that file.

These pre-existing failures are intentionally not changed here in order to keep the H20 hardware-target contribution focused.

Scope

This PR only adds:

  • the NVIDIA H20 GpuTarget
  • H20 target regression coverage

It intentionally excludes unrelated model-importer, scheduler, attention, search-test, and local benchmark-artifact changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant