Add AVX-512 Ice Lake decoder and retarget the library to .NET 10.

Port simdutf's Ice Lake base64 kernel (VPERMI2B lookup, VPCOMPRESSB
whitespace compression, masked 48-byte stores) using the .NET 10
AVX-512 VBMI/VBMI2 intrinsics. Dispatch prefers AVX-512 VBMI2, then
AVX2, SSSE3, or scalar. Benchmarks on a Xeon Gold 6548N reach 11.3 GB/s
versus 4.7 GB/s for System.Buffers.Text.Base64.DecodeFromUtf8.

Author: lemire

.github/workflows/docs.yml

@@ -27,10 +27,10 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v4
 
-      - name: Setup .NET 9.0
+      - name: Setup .NET 10.0
         uses: actions/setup-dotnet@v4
         with:
-          dotnet-version: '9.0.x'
+          dotnet-version: '10.0.x'
 
       - name: Install DocFX
         run: dotnet tool update -g docfx

.github/workflows/dotnet.yml

@@ -13,11 +13,10 @@ jobs:
     steps:
     - uses: actions/checkout@v4
 
-    - name: Setup .NET 9.0 (preview)
+    - name: Setup .NET 10.0
       uses: actions/setup-dotnet@v4
       with:
-        dotnet-version: 9.0.x
-        dotnet-quality: preview
+        dotnet-version: 10.0.x
     - name: Restore dependencies
       run: dotnet restore
     - name: Build

README.md

@@ -36,7 +36,8 @@ fully reproducible.
 |:----------------|:------------------------|:-------------------|:-------------------|
 | Apple M2 processor (ARM, 3.5 Ghz)   | 10                     | 3.8               | 2.6 x |
 | AWS Graviton 3 (ARM, 2.6 GHz)   | 5.1 | 2.0 | 2.6 x |
-| Intel Ice Lake (2.0 GHz)  | 7.6                     | 3.4              | 2.2 x |
+| Intel Xeon Gold 6548N (AVX-512, 2.8 GHz) | 11.3 | 4.7 | 2.4 x |
+| Intel Ice Lake (AVX2, 2.0 GHz)  | 7.6                     | 3.4              | 2.2 x |
 | AMD EPYC 7R32 (Zen 2, 2.8 GHz)    |  6.9       | 3.0 | 2.3 x |
 
 ## Results (SimdBase64 vs. string .NET functions)
@@ -59,17 +60,20 @@ byte[] newBytes = SimdBase64.Base64.FromBase64String(s);
 | processor and base freq.      | SimdBase64 (GB/s) | .NET speed (GB/s) | speed up |
 |:----------------|:------------------------|:-------------------|:-------------------|
 | Apple M2 processor (ARM, 3.5 Ghz)   | 4.0                     | 1.1              | 3.6 x |
+| Intel Xeon Gold 6548N (AVX-512, 2.8 GHz) | 2.1 | 0.71 | 2.9 x |
 | Intel Ice Lake (2.0 GHz)  | 2.5                      | 0.65             | 3.8 x |
 
 ## AVX-512
 
-As for .NET 9, the support for AVX-512 remains incomplete in C#. In particular, important
-VBMI2 instructions are missing. Hence, we are not using AVX-512 under x64 systems at this time.
-However, as soon as .NET offers the necessary support, we will update our results.
+On .NET 10, we use AVX-512 VBMI / VBMI2 when the CPU supports them (Ice Lake and later,
+including the Xeon Gold 6548N numbers above). The kernel is a C# port of the
+[simdutf Ice Lake decoder](https://github.com/simdutf/simdutf): a 64-byte
+`VPERMI2B` lookup, `VPCOMPRESSB` to strip white space, and a masked 48-byte store.
+On older x64 CPUs the library still dispatches to AVX2 or SSSE3.
 
 ## Requirements
 
-We require .NET 9 or better: https://dotnet.microsoft.com/en-us/download/dotnet/9.0
+We require .NET 10 or better: https://dotnet.microsoft.com/en-us/download/dotnet/10.0
 
 ## Usage
 
@@ -177,6 +181,7 @@ You can convert an integer to a hex string like so: `$"0x{MyVariable:X}"`.
 ## Performance tips
 
 - Be careful: `Vector128.Shuffle` is not the same as `Ssse3.Shuffle` nor is  `Vector256.Shuffle` the same as `Avx2.Shuffle`. Prefer the latter.
+- Likewise `Vector512.Shuffle` is a full 64-byte permute; `Avx512BW.Shuffle` is lane-wise `VPSHUFB`. For the Ice Lake kernel use `Avx512Vbmi.PermuteVar64x8` / `PermuteVar64x8x2`.
 - Similarly `Vector128.Shuffle` is not the same as `AdvSimd.Arm64.VectorTableLookup`, use the latter.
 - `stackalloc` arrays should probably not be used in class instances.
 - In C#, `struct` might be preferable to `class` instances as it makes it clear that the data is thread local.

benchmark/Benchmark.cs

@@ -321,6 +321,24 @@ public unsafe void RunSSEDecodingBenchmarkWithAllocUTF16(string[] data, int[] le
             }
         }
 
+        public unsafe void RunAVX512DecodingBenchmarkUTF8(string[] data, int[] lengths)
+        {
+            for (int i = 0; i < FileContent.Length; i++)
+            {
+                byte[] base64 = input[i];
+                byte[] dataoutput = output[i];
+                int bytesConsumed = 0;
+                int bytesWritten = 0;
+                SimdBase64.AVX512.Base64.DecodeFromBase64AVX512(base64.AsSpan(), dataoutput, out bytesConsumed, out bytesWritten, false);
+                if (bytesWritten != lengths[i])
+                {
+                    Console.WriteLine($"Error: {bytesWritten} != {lengths[i]}");
+#pragma warning disable CA2201
+                    throw new Exception("Error");
+                }
+            }
+        }
+
         public unsafe void RunAVX2DecodingBenchmarkUTF8(string[] data, int[] lengths)
         {
             for (int i = 0; i < FileContent.Length; i++)
@@ -620,6 +638,11 @@ public unsafe void AVX2DecodingRealDataUTF8()
             RunAVX2DecodingBenchmarkUTF8(FileContent, DecodedLengths);
         }
 
+        public unsafe void AVX512DecodingRealDataUTF8()
+        {
+            RunAVX512DecodingBenchmarkUTF8(FileContent, DecodedLengths);
+        }
+
         [Benchmark]
         [BenchmarkCategory("default")]
         public unsafe void SimdBase64DecodingRealDataUTF8()

benchmark/benchmark.csproj

@@ -2,7 +2,7 @@
 
   <PropertyGroup>
     <OutputType>Exe</OutputType>
-    <TargetFramework>net9.0</TargetFramework>
+    <TargetFramework>net10.0</TargetFramework>
     <ImplicitUsings>enable</ImplicitUsings>
     <Nullable>enable</Nullable>
     <AllowUnsafeBlocks>true</AllowUnsafeBlocks> 

docs/articles/benchmarks.md

@@ -29,7 +29,8 @@ decoder. SimdBase64 is **1.7×–2.6×** faster on realistic inputs of a few kil
 |:------------------------------------|:-----------------:|:-----------:|:--------:|
 | Apple M2 (ARM, 3.5 GHz)             | 10  | 3.8 | 2.6× |
 | AWS Graviton 3 (ARM, 2.6 GHz)       | 5.1 | 2.0 | 2.6× |
-| Intel Ice Lake (2.0 GHz)            | 7.6 | 3.4 | 2.2× |
+| Intel Xeon Gold 6548N (AVX-512, 2.8 GHz) | 11.3 | 4.7 | 2.4× |
+| Intel Ice Lake (AVX2, 2.0 GHz)      | 7.6 | 3.4 | 2.2× |
 | AMD EPYC 7R32 (Zen 2, 2.8 GHz)      | 6.9 | 3.0 | 2.3× |
 
 ## vs. `Convert.FromBase64String`
@@ -40,6 +41,7 @@ The .NET runtime does **not** accelerate `Convert.FromBase64String`. Replacing i
 | processor and base freq.            | SimdBase64 (GB/s) | .NET (GB/s) | speed-up |
 |:------------------------------------|:-----------------:|:-----------:|:--------:|
 | Apple M2 (ARM, 3.5 GHz)             | 4.0 | 1.1  | 3.6× |
+| Intel Xeon Gold 6548N (AVX-512, 2.8 GHz) | 2.1 | 0.71 | 2.9× |
 | Intel Ice Lake (2.0 GHz)            | 2.5 | 0.65 | 3.8× |
 
 > Hardware, runtime version and input all affect these numbers. Treat the tables as

docs/articles/contributing.md

@@ -91,6 +91,8 @@ A few hard-won tips when working on the SIMD kernels:
 
 - `Vector128.Shuffle` is **not** the same as `Ssse3.Shuffle`, nor is `Vector256.Shuffle`
   the same as `Avx2.Shuffle`. Prefer the architecture-specific intrinsics.
+- `Vector512.Shuffle` is a full 64-byte permute; `Avx512BW.Shuffle` is lane-wise `VPSHUFB`.
+  The Ice Lake kernel uses `Avx512Vbmi.PermuteVar64x8` / `PermuteVar64x8x2`.
 - Likewise, `Vector128.Shuffle` differs from `AdvSimd.Arm64.VectorTableLookup`; use the latter on ARM.
 - Avoid `stackalloc` arrays in class instances.
 - Prefer `struct` over `class` to make thread-local data explicit.

docs/articles/getting-started.md

@@ -1,11 +1,11 @@
 # Getting started
 
 SimdBase64 is a small, dependency-free C# library that decodes base64 with SIMD
-instructions. It targets **.NET 9** (or better) and runs on x64 and ARM64.
+instructions. It targets **.NET 10** (or better) and runs on x64 and ARM64.
 
 ## Requirements
 
-- [.NET 9 SDK](https://dotnet.microsoft.com/en-us/download/dotnet/9.0) or newer.
+- [.NET 10 SDK](https://dotnet.microsoft.com/en-us/download/dotnet/10.0) or newer.
 - A 64-bit x64 or ARM64 CPU for the SIMD kernels (a portable scalar fallback covers everything else).
 
 ## Build & reference
@@ -94,7 +94,8 @@ byte[] bytes = SimdBase64.Base64.FromBase64String(s);
 ## Choosing a specific kernel
 
 `DecodeFromBase64` dispatches to the fastest kernel your CPU supports. The architecture-specific
-implementations live in nested namespaces (`SimdBase64.Arm`, `SimdBase64.AVX2`, `SimdBase64.SSE`,
-`SimdBase64.Scalar`) and can be called directly — useful for testing or pinning behaviour.
+implementations live in nested namespaces (`SimdBase64.Arm`, `SimdBase64.AVX512`,
+`SimdBase64.AVX2`, `SimdBase64.SSE`, `SimdBase64.Scalar`) and can be called directly — useful
+for testing or pinning behaviour.
 
 Continue to [How it works](how-it-works.md) or jump to the [API reference](xref:SimdBase64.Base64).

docs/articles/how-it-works.md

@@ -35,25 +35,27 @@ At a high level, each vectorized block:
 A single public method picks the best kernel for the host CPU, in priority order:
 
 ```text
-ARM64 NEON  →  AVX2  →  SSE4.2 / SSSE3  →  scalar fallback
+ARM64 NEON  →  AVX-512 VBMI2  →  AVX2  →  SSSE3  →  scalar fallback
 ```
 
-This means you write one call and automatically get NEON on an Apple M-series laptop, AVX2 on a
-current x64 server, and a correct scalar implementation everywhere else.
+This means you write one call and automatically get NEON on an Apple M-series laptop, AVX-512
+on Ice Lake and later x64 servers, AVX2 on older x64, and a correct scalar implementation
+everywhere else.
 
 | Back-end | Vector width | Typical hardware |
 |----------|--------------|------------------|
+| AVX-512 VBMI2 | 512-bit | Ice Lake, Sapphire Rapids, Emerald Rapids, Zen 4+ |
 | AVX2 | 256-bit | Most current x64 |
 | SSE4.2 / SSSE3 | 128-bit | Older x64 |
 | ARM64 NEON | 128-bit | Apple Silicon, AWS Graviton, Snapdragon |
 | Scalar | — | Portable fallback |
 
-## What about AVX-512?
+## AVX-512
 
-As of .NET 9, the C# support for AVX-512 is still incomplete — in particular the VBMI2
-instructions this algorithm relies on are missing. So SimdBase64 does **not** use AVX-512 under
-x64 at this time. As soon as the runtime exposes the necessary intrinsics, we will add a kernel
-and update the benchmarks.
+On .NET 10 the VBMI / VBMI2 intrinsics (`PermuteVar64x8x2`, `Compress`) are available, so we
+ship an Ice Lake kernel ported from [simdutf](https://github.com/simdutf/simdutf). It processes
+64 input bytes per iteration, compresses white space with `VPCOMPRESSB`, and writes exactly 48
+decoded bytes with a masked store.
 
 ## Why an `OperationStatus`, not a `bool`?
 

docs/docfx.json

@@ -13,7 +13,7 @@
       "memberLayout": "separatePages",
       "enumSortOrder": "declaringOrder",
       "properties": {
-        "TargetFramework": "net9.0"
+        "TargetFramework": "net10.0"
       }
     }
   ],