Skip to content

Latest commit

 

History

History
468 lines (339 loc) · 19.7 KB

File metadata and controls

468 lines (339 loc) · 19.7 KB

Tuning your GPU — a step-by-step guide

This guide walks through measuring the best producerOpenCL settings for your hardware with the TuneConfiguration command, and (optionally) contributing the result back so other people with similar hardware get a useful starting point.

Every command is spelled out and explained. You do not need to know Java or Maven — for the tuning part you only need the released jar. The contribution part additionally needs Python 3 and a git clone.

Time required: about 15 minutes of setup, then 15–60 minutes of unattended measuring per GPU.

What this is not: the tuner does not read, build or modify your address database. It builds a synthetic filter of the size you tell it to and throws it away afterwards.


Table of contents

  1. What the tuner actually measures
  2. Before you start
  3. Step 1 — find your devices
  4. Step 2 — write a tuning config
  5. Step 3 — run the tuner and capture the log
  6. Step 4 — read the report
  7. Step 5 — apply the result
  8. Tuning a second GPU
  9. Contributing your numbers
  10. Troubleshooting

1. What the tuner actually measures

TuneConfiguration runs your pipeline once for every combination ("arm") of two settings:

Setting What it controls
batchSizeInBits How many candidate keys one GPU launch computes: 2^batchSizeInBits. 20 means 1 048 576 keys per launch.
keysPerWorkItem How many keys each GPU thread computes within that launch.

For each arm it measures the net end-to-end throughput — candidate keys per second through the whole pipeline, not just kernel time — and prints a table of every arm plus the winner.

The best combination is device-specific and not guessable. On the same laptop, one GPU peaked at batchSizeInBits=23 and another at 20. Vendor, driver, memory type and compute-unit count all move it.


2. Before you start

You need:

  • The runnable jar. bitcoinaddressfinder-<version>-jar-with-dependencies.jar, from the releases page or built with mvn package -P assembly -DskipTests.
  • A Java 21+ runtime. Check with java -version.
  • Working GPU drivers with OpenCL. Step 1 verifies this.
  • The launcher scripts from examples/run_TuneConfiguration.bat (Windows) or run_TuneConfiguration.sh (Linux/macOS).

A bare java -jar works. The fat jar's manifest carries the Add-Opens the JVM needs before it will let the LMDB layer reach into java.nio internals, so it is self-sufficient. The launcher scripts remain useful for what they also set — the heap (-Xmx8g) and the Logback configuration — so prefer them if you have not tuned those yourself. Older releases did require the launcher: if you see InaccessibleObjectException, you are on a jar built before this became automatic.

Put the jar, the launcher and your config file in the same directory. The launcher refers to the jar by name, so if your version differs, open it in a text editor and fix the filename on the bitcoinaddressfinder-…jar line.


3. Step 1 — find your devices

The tuner targets one device at a time, identified by two numbers: platformIndex (which OpenCL driver) and deviceIndex (which device under that driver). Ask the tool what it sees.

examples/config_OpenCLInfo.json already contains everything this needs — the whole file is two lines:

{
  "command": "OpenCLInfo"
}

Run it with the supplied launcher (run_OpenCLInfo.bat on Windows, run_OpenCLInfo.sh elsewhere), or directly:

java -jar bitcoinaddressfinder-1.8.0-jar-with-dependencies.jar config_OpenCLInfo.json

No database is involved here, so nothing beyond the plain command is needed.

You get one block per device. The two numbers you need are printed with it, along with a heuristic starting point:

--- Info for OpenCL device: NVIDIA RTX 500 Ada Generation Laptop GPU ---
CL_DEVICE_NAME:                        NVIDIA RTX 500 Ada Generation Laptop GPU
CL_DEVICE_MAX_COMPUTE_UNITS:           16
CL_DEVICE_MAX_MEM_ALLOC_SIZE:          1023 MByte
SUGGESTED START CONFIG (heuristic from the info above; sweep keysPerWorkItem to confirm):
    producerOpenCL.batchSizeInBits = 21
    producerOpenCL.keysPerWorkItem = 256

Write down, for each GPU you want to tune, its platform index and device index. Platforms are numbered from 0 in the order printed; devices are numbered from 0 within each platform.

Two GPUs from different vendors are usually on different platforms. An NVIDIA card and an Intel integrated GPU are typically platformIndex 0 / deviceIndex 0 and platformIndex 1 / deviceIndex 0not device 0 and 1 of the same platform. Two cards from the same vendor usually are device 0 and 1 of one platform. Reading it off the output beats guessing.

If this command lists no devices, stop here and fix your OpenCL installation — nothing below will work. See Troubleshooting.


4. Step 2 — write a tuning config

Start from examples/config_TuneConfiguration.json and change four things. Copy it to config_TuneConfiguration_gpu0.json (name it after the device — you will make one per GPU).

4.1 Point it at your device

In the producerOpenCL block:

"platformIndex": 0,
"deviceIndex": 0,

Use the numbers from Step 1.

4.2 Widen keysPerWorkItemCandidates — this one matters

The shipped default is:

"keysPerWorkItemCandidates": [1, 4, 16, 64, 256],

On modern GPUs the best value is frequently above 256, and a sweep that stops at 256 will report 256 as the winner simply because it never tried anything larger. Use:

"keysPerWorkItemCandidates": [16, 64, 256, 512, 1024, 2048],

Dropping 1 and 4 costs you nothing — they are far too small to win on any GPU — and pays for the larger values without lengthening the run.

You no longer have to catch this yourself. If the winner lands on the largest value swept, the tuner keeps measuring past your list on its own — doubling keysPerWorkItem, or adding a bit to batchSizeInBits — until an extra arm stops improving. It costs one arm per step instead of a second 20-minute run. The report and the log say when it happened.

It stops for two different reasons, and the report distinguishes them:

  • The optimum was inside your range. The extra arm did not beat the winner. Nothing to do.
  • batchSizeInBits=24 was reached. That is the framework maximum (BIT_COUNT_FOR_MAX_CHUNKS_ARRAY), not a truncated sweep — there is nothing above it to try.

Set extendSweepWhenWinnerIsAtTheEdge: false to measure exactly the candidates you listed and nothing more, or raise maxExtensionArms (default 4) to let it look further.

4.3 Set targetDatabaseEntries to your real database size

"targetDatabaseEntries": 141045995,

State the size of the database you intend to scan against, not one you happen to have. The tuner builds a synthetic filter of exactly that size — the filter's contents do not affect timing, but its size does (GPU memory occupancy and how many candidates cross the PCIe bus).

The value is a count of addresses. Two reference points, both from the README's database section: the Light DB is 141045995 and the Full DB 1472947953 as of the 2026-07-20 publication. They grow each time the databases are republished, so check there rather than trusting a number copied from a guide. If your database is already imported, the address count is printed at startup by any Find run (Binary Fuse16 filter: ready (141045995 addresses, …)).

4.4 Point the consumer at your database (optional but recommended)

"lmdbConfigurationReadOnly": {
  "lmdbDirectory": "lmdb",
  ...
}

This is used only to measure what one database lookup costs on your storage — the number the FUSE_8 vs FUSE_16 recommendation hinges on, and one that ranges from 4 µs to 293 µs depending on whether the data is in the page cache. The grid sweep itself never touches the database.

Leave the directory pointing at a real LMDB if you have one. If you do not, the tuner still runs and falls back to a documented estimate — it says so in the report.

4.5 Leave these alone

Setting Default Why
secondsPerArm 20 Below ~5 s the ranking of neighbouring arms stops being reproducible.
warmupSecondsPerArm 5 Covers kernel compilation and GPU clock ramp-up, which would otherwise drag the first arm down.
sweepFilterTypes false Setting it true forces a second full filter build (~150 s at the Light tier, ~26 min at the Full tier) to measure something that is derived accurately anyway.

5. Step 3 — run the tuner and capture the log

The report is printed to the log, not written to a file. Capture it, or you will watch a 20-minute measurement scroll out of your terminal buffer.

Windows — open run_TuneConfiguration.bat in a text editor. Two edits:

  1. Change the config filename on the last line to your file:
    config_TuneConfiguration_gpu0.json
  2. Uncomment the redirect on the final line by deleting the leading rem :
    >> log_TuneConfiguration_gpu0.txt 2>&1

Then run it:

run_TuneConfiguration.bat

Linux / macOS — same two edits in run_TuneConfiguration.sh (delete the leading # on the redirect line), then:

chmod +x run_TuneConfiguration.sh
./run_TuneConfiguration.sh

Or redirect without editing the script:

./run_TuneConfiguration.sh > log_TuneConfiguration_gpu0.txt 2>&1

2>&1 means "send error output to the same file as normal output" — without it, errors go to the screen and are lost from the log.

How long it takes

run time = number of arms × (warmupSecondsPerArm + secondsPerArm)

With 7 batch sizes × 6 keysPerWorkItem values = 42 arms × 25 s ≈ 18 minutes, plus a one-time filter build of roughly 44 s per 100 M entries.

Leave the machine otherwise idle. The tuner measures throughput; anything else using the GPU, the CPU or memory bandwidth during a run lands directly in the numbers.

What files this creates

File Created by Contents
log_TuneConfiguration_gpu0.txt your redirect The whole run, including the report. This is the file to keep.
(nothing else) The tuner writes no files of its own. The recommended config is printed inside the log, not saved separately.

6. Step 4 — read the report

Find the block delimited by ########## BEGIN TuneConfiguration report ##########.

Arms (all figures MEASURED on this machine):
  batchSizeInBits  keysPerWorkItem        candidates/s addresses checked/s
  23               64                    64,588,860.65       1,006,766.88
  23               256                   94,791,059.02       1,473,710.16
  ...
Winner (MEASURED): batchSizeInBits=23 keysPerWorkItem=256 at 94,791,059.02 candidates/s.

Filter choice - total = probe + fpr x verification:
  verification cost   12.61 us   MEASURED on this database
  FUSE_8              probe 76.70 ns, fpr 0.003874   DOCUMENTED constants -> total 125.55 ns
  FUSE_16             probe 80.40 ns, fpr 0.000016   DOCUMENTED constants -> total 80.60 ns
  Recommended: FUSE_16   (DERIVED, not measured)

Paste-ready configuration:
{ ... }

Every figure is labelled MEASURED (on your machine, just now) or DOCUMENTED / ESTIMATED (a published constant that does not vary by machine), so you always know which is which.

Three things to check before trusting the winner:

  1. Did the sweep extend itself? If the winner landed on the largest value you listed, the tuner continued past it automatically and the extra arms appear in the table. A line stating why it stopped tells you whether the optimum was inside your range or the framework maximum was reached.
  2. Any FAILED: arms? Those are combinations your driver rejected, usually because the output buffer for that batchSizeInBits exceeds the device's maximum allocation. Harmless — the sweep records them and moves on.
  3. Any arms reading 0.00 candidates/s? Not a failure: that arm is so slow that not one batch completed inside the 20 s measurement window. It happens at large batchSizeInBits combined with tiny keysPerWorkItem. Ignore those arms, or raise secondsPerArm if you specifically want a number for them.

In release 1.7.0 the report arrives as one very long line with | between what should be line breaks. That is the log pattern's CRLF guard folding the block. To read it, paste it into an editor and replace | with a newline. Later releases emit the report one line per record and need no such treatment.


7. Step 5 — apply the result

The report ends with a paste-ready configuration — a complete Find config carrying the winning values, with everything the sweep did not vary (device indices, consumer settings, key producer) carried through unchanged. Copy it into your config_Find.json and run it.

If you prefer to edit your existing config by hand, copy exactly three values into your producerOpenCL block:

"batchSizeInBits": 23,
"keysPerWorkItem": 256,
"gpuFilterType": "FUSE_16"

8. More than one GPU

Nothing to do — every GPU is measured by default. Leave producerOpenCL empty and the tuner enumerates the machine's GPUs, sweeps each one in turn, and reports a winner per device. The paste-ready configuration then contains one entry per device, each carrying its own measured values.

"producerOpenCL": []

Listing entries explicitly remains the override: name one device and exactly that device is swept. If you do, and the machine has more GPUs than you listed, the log says so rather than leaving you to notice.

Two things worth knowing:

  • Devices are measured one at a time, on purpose. Two GPUs running at once contend for host memory bandwidth, and an integrated GPU — which has no memory of its own and shares system RAM with the host — loses the most. Tuning them simultaneously would measure that interference instead of the devices.

    How much it costs varies by a lot, so do not plan around a single figure. Two measured pairs:

    integrated GPU alone alongside the discrete card loss
    Intel Arc Pro (laptop, next to an RTX 500 Ada) 16.3 M keys/s 5.5 M keys/s ~3×
    AMD gfx1036 (desktop, next to an RX 7900 XTX) 3.12 M keys/s 2.07 M keys/s ~1.5×

    Same phenomenon, twice the severity on the laptop. Platform, memory bandwidth headroom and thermal coupling all move it, so measure your own pair rather than assuming either number.

  • Expect roughly N × the run time. The tuner states it up front. Only the sweep repeats — the filter is built once and reused across devices.

After a multi-device run, check the Keys per producer field of the statistics line during a real Find run to see the actual split between your devices.

9. Contributing your numbers

Results are hardware-dependent, so the project keeps them per machine rather than averaging them away. Two GPUs are currently registered, both high-end desktop parts — laptop, integrated and mid-range hardware is exactly what is missing.

This part needs a git clone of the repository and Python 3 (standard library only, nothing to install).

9.1 Register your machine

python docs/measurements/register_machine.py --set storage="Samsung 990 PRO 4TB NVMe"

This detects your CPU, cache sizes, RAM, GPU, OS and JDK and writes an entry into docs/measurements/machines.json. It prints the machine_id it generated, e.g. ryzen75800h-63g-win11. Re-running updates the same entry instead of adding a duplicate.

Useful variants:

python docs/measurements/register_machine.py --dry-run        # show the entry, write nothing
python docs/measurements/register_machine.py --id my-laptop   # choose the id yourself
python docs/measurements/register_machine.py --set cpu.l3_mb=32   # fill in a field it missed

Anything it cannot detect is left null. Two fields are worth checking by eye:

  • cpu.l3_mb — the filter-selection plots annotate the L3 boundary from it, and detection fails on some platforms.

  • gpu — a list, because a laptop with a discrete card also has an integrated one and this tool runs on both. Compare it against what OpenCLInfo reported in step 1: the script reads the operating system's device list, which need not agree. Correct it with a comma-separated value if a device is missing:

    python docs/measurements/register_machine.py --set gpu="RTX 500 Ada,Arc Pro Graphics"

9.2 Send the numbers

Open an issue or a discussion and attach:

What Why it is needed
log_TuneConfiguration_<gpu>.txt, one per GPU The arms table is the actual data; the log also records driver version, device properties and the exact settings used.
The machine_id and the machines.json entry register_machine.py printed Every measurement row references a machine; without it a number cannot be interpreted.
Whether the sweep hit the edge (see step 4) A truncated sweep reports a lower bound, not a peak, and must be labelled as such.

If you would rather open a pull request directly, append one row per arm to a docs/measurements/tuner_<machine_id>_<gpu>.csv using the column layout of tuner_ryzen9800x3d_gfx1100.csv:

machine_id,gpu,date,batch_size_in_bits,keys_per_work_item,candidates_per_second,addresses_checked_per_second,seconds_per_arm,warmup_seconds,verification_micros_measured,winner,kernel

Then regenerate the plots and the tables in performance.md:

python docs/measurements/plot.py

Never retype a number into prose — the CSVs are the single source of truth and the tables are generated from them between <!-- BEGIN GENERATED:… --> markers.


10. Troubleshooting

OpenCLInfo lists no devices. Install your vendor's OpenCL runtime — the GPU driver alone is sometimes not enough. On Linux, install the ICD package (nvidia-opencl-icd, intel-opencl-icd, mesa-opencl-icd) and verify with clinfo -l, which enumerates platforms and devices in the same order this tool does.

CL_OUT_OF_RESOURCES during the run. The per-launch buffer for that batchSizeInBits is larger than the device allows. The sweep records the arm as failed and continues, so this is not fatal during tuning. If it happens on every arm, start from the SUGGESTED START CONFIG values that OpenCLInfo printed.

InaccessibleObjectException at startup. Your jar predates the manifest that carries the required Add-Opens (added in 1.8.0). Use the launcher script, which passes the same flags on the command line, or rebuild from a current checkout.

Numbers differ noticeably between two runs of the same config. Something else was using the machine. Close other GPU workloads, then re-run. Laptops additionally throttle on temperature — a run started on a cold machine and one started on a hot machine will not agree.

The winner changes every time I run it. If several arms are within a few percent of each other, any of them is a fine choice — pick the one with the smaller batchSizeInBits, which reaches steady state faster and shuts down more responsively.