Skip to content

ARSITEKTUR.md

Latest commit

 

History

History
535 lines (378 loc) · 18.8 KB

ARSITEKTUR.md

File metadata and controls

535 lines (378 loc) · 18.8 KB
Raw

ARSITEKTUR.md — Cara Kerja Internal hash256-miner-c

Deep-dive ke kode untuk yang ingin paham, modifikasi, atau extend miner-nya.

Daftar isi

  1. Overview komponen
  2. Keccak-256 vs SHA-256
  3. AVX-512 SIMD: 8-way Keccak
  4. C miner: threading & cancellation
  5. Orchestrator: pre-flight checks
  6. Monitor: read-only dashboard
  7. Trade-off design
  8. Roadmap & extensions

Overview komponen

flowchart LR
    ENV[(.env)]
    STATS[(stats.json)]
    LOG[(miner.log)]
    O[orchestrator.js]
    M["bin/miner<br/>(C, AVX-512 SIMD)"]
    RPC((Ethereum RPC))
    SRV[monitor/server.js]
    HTML[dashboard HTML/JS]

    ENV -.read on start.-> O
    O -- "spawn subprocess<br/>per challenge" --> M
    M -- "stdout: FOUND nonce hash" --> O
    M -- "stderr: hashrate, init" --> O
    O -- "POST mine(nonce)" --> RPC
    RPC -- "challenge, gas, receipt" --> O
    O -- "write every 10s + on event" --> STATS
    O -- "append" --> LOG
    SRV -.read.-> STATS
    SRV -.read.-> LOG
    SRV -- "GET /api/network" --> RPC
    HTML -- "fetch JSON every 3s" --> SRV
Loading

3 process yang jalan:

Process Bahasa Tugas
bin/miner C (pthreads + AVX-512) Search nonce yang menghasilkan keccak256(challenge || nonce) < target
orchestrator.js Node.js (ethers.js) Fetch challenge dari kontrak, spawn miner, submit mine(nonce) tx
monitor/server.js Node.js (express) Web dashboard read-only

Komunikasi antar-process:

  • C miner ↔ orchestrator: pipe stdin/stdout (orchestrator parse FOUND <nonce> <hash> line dari stdout)
  • orchestrator ↔ monitor: tidak langsung; orchestrator tulis stats.json, monitor baca file itu
  • orchestrator ↔ kontrak: HTTP RPC via ethers.js JsonRpcProvider

Keccak-256 vs SHA-256

Walau nama repo hash256-miner-c, hashing sebenarnya pakai Keccak-256, bukan SHA-256. Ini penting karena:

Aspek SHA-256 (FIPS 180-4) Keccak-256 (Ethereum)
Designed NSA, 2001 Bertoni, Daemen, Peeters, Van Assche, 2008
Standard FIPS 180-4 NIST SHA-3 finalist (versi padding berbeda)
Construction Merkle-Damgård + ARX (Add/Rotate/XOR) Sponge + permutation Keccak-f[1600]
Hardware accel SHA-NI (Intel/AMD), SHA extension ARMv8 TIDAK ada hardware accel di CPU komersil
Software bottleneck ~80-120 MH/s/core dengan SHA-NI ~25-150 MH/s/core dengan SIMD
Pakai di Bitcoin, banyak chain non-EVM Ethereum (semua opcode hash, address derivation, dst)

Kenapa Ethereum pakai Keccak (bukan SHA-3 standar)?

NIST tweak Keccak sebelum jadi SHA-3 (ganti padding 0x06 → 0x06...01 di FIPS 202). Ethereum freeze versi pre-tweak pas Keccak masih draft. Hasilnya:

Keccak-256("")      = c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470
SHA-3-256("")       = a7ffc6f8bf1ed76651c14756a061d662f580ff4de43b49fa82d80a4b80f8434a
                     ^^ DIFFERENT — different padding

Implementasi kontrak hash256.org pakai opcode EVM KECCAK256 = persis Keccak-256 pre-NIST-tweak. Code C kita harus match itu byte-for-byte.

Verification: tests/test_keccak.c cross-check vs ethers.solidityPackedKeccak256 (yang juga pakai pre-tweak Keccak) untuk sample fixed input.

AVX-512 SIMD: 8-way Keccak

Sponge construction

Keccak-256 = sponge dengan rate 1088 bits + capacity 512 bits, output 256 bits. Untuk input pendek (32 + 32 = 64 bytes seperti challenge || nonce), cukup 1 absorb round + squeeze.

State Keccak = 5 × 5 lanes × 64 bits = 200 bytes = uint64_t state[25].

Permutation Keccak-f[1600] = 24 round × { θ, ρ, π, χ, ι } = banyak XOR + rotate + AND + XOR-NOT.

Scalar reference (src/keccak_scalar.c)

Implement straightforward Keccak-f[1600] untuk 1 input pakai uint64_t state[25]. Throughput: ~10-25 MH/s di 1 core (sangat slow, untuk verifikasi saja).

AVX-512 8-way (src/keccak_avx512.c)

Trick: alih-alih 1 state per call, simpan 8 state secara paralel di __m512i state[25]. Setiap __m512i = 8 lanes × 64-bit (lihat operasi sebagai SIMD vector).

Setiap operasi Keccak (XOR, rotate, AND-NOT) dilakukan ke 8 state sekaligus dalam 1 instruksi:

// Skalar (1 state):
state[i] = state[i] ^ state[j];

// AVX-512 8-way (8 states):
state[i] = _mm512_xor_si512(state[i], state[j]);   // 1 instruksi, 8 XOR sekaligus

Throughput per core: ~110-140 MH/s (vs scalar ~10-25 MH/s = ~10x speedup per core).

Kenapa 8-way (bukan 4 / 16 / 32)?

  • AVX-512 register ZMM = 512 bits = 8 × 64-bit. Native lane count untuk Keccak (yang state-nya u64).
  • AVX-256 = 4 lanes / 64-bit, throughput ~half
  • AVX-1024 / AVX-2048: tidak ada

Hashing per call = 8 nonces yang di-test paralel. Total per cycle CPU ~8 lanes × 8 ops/instr × ~3-4 instr/round × 24 round = ~500-700 ops.

Loop structure (keccak256_avx512_search)

int keccak256_avx512_search(
    uint64_t challenge_le[4],      /* fixed challenge as 4 u64 lanes */
    uint64_t target_be[4],         /* difficulty target, BE */
    uint64_t nonce_base,           /* starting nonce */
    int batch_size,                /* how many 8-nonce batches to try */
    uint64_t *out_nonce,
    uint8_t out_hash[32],
    uint64_t *out_hashes_done
) {
    for (int b = 0; b < batch_size; b++) {
        /* 1. Build 8 inputs: challenge || (nonce_base + lane_i) */
        __m512i nonces = _mm512_add_epi64(nonce_vec_base,
                                           _mm512_setr_epi64(0,1,2,3,4,5,6,7));

        /* 2. Initialize 8 sponge states */
        __m512i state[25] = { ... };
        absorb_8way(state, challenge_le, nonces);

        /* 3. Run Keccak-f[1600] permutation, 24 rounds, 8-way */
        keccak_f_8way(state);

        /* 4. Extract first 256 bits of squeeze (4 u64 lanes per state)
              Compare with target. If lane_i hash < target, it's a hit. */
        for (int lane = 0; lane < 8; lane++) {
            uint64_t hash[4] = extract_lane(state, lane);
            if (compare_be(hash, target_be) < 0) {
                *out_nonce = nonce_base + b * 8 + lane;
                memcpy(out_hash, hash, 32);
                return 1;
            }
        }
        nonce_base += 8;
    }
    *out_hashes_done = batch_size * 8;
    return 0;  /* no hit in this batch */
}

Verifikasi correctness

Self-test (tests/test_keccak.c) random 100 input, hitung dengan kedua path, require byte-equal:

for (int i = 0; i < 100; i++) {
    uint8_t challenge[32] = random_bytes();
    uint64_t nonce       = random_u64();

    uint8_t hash_scalar[32], hash_simd[32];
    keccak256_scalar(challenge, nonce, hash_scalar);
    keccak256_avx512(challenge, nonce, hash_simd);

    if (memcmp(hash_scalar, hash_simd, 32) != 0) {
        printf("MISMATCH at vector %d\n", i);
        exit(1);
    }
}

Plus cross-check vs ethers.js solidityPackedKeccak256 untuk fixed test vector — proves we match Ethereum semantics.

C miner: threading & cancellation

Untuk maksimalkan throughput multi-core, kita run 4 worker thread, masing-masing search non-overlapping nonce range.

flowchart TD
    Main["main thread"]
    Main -- "pthread_create x N" --> W1["worker 1<br/>nonce_base + 0×stride"]
    Main -- "pthread_create x N" --> W2["worker 2<br/>nonce_base + 1×stride"]
    Main -- "pthread_create x N" --> W3["worker 3<br/>nonce_base + 2×stride"]
    Main -- "pthread_create x N" --> W4["worker 4<br/>nonce_base + 3×stride"]
    Main -- "pthread_create" --> R["reporter thread<br/>print hashrate every 2s"]

    W1 -- "atomic flag<br/>g_found = 1" --> SHARED((g_found, g_shutdown))
    W2 -- "atomic flag<br/>g_found = 1" --> SHARED
    W3 --> SHARED
    W4 --> SHARED

    W1 -- "first hit prints<br/>FOUND <nonce> <hash>" --> STDOUT[stdout]
Loading

Nonce stride calculation

Setiap thread punya starting nonce yang berjarak 2^58 antar-thread (big_stride = 1ULL << 58). Stride sangat besar supaya tidak ada overlap di realistic mining time:

  • 4 thread × 400 MH/s combined ≈ search ~10^9 nonces/second
  • 1 thread butuh 2^58 / 10^9 ≈ 9 juta detik = 100+ hari untuk crash ke nonce thread berikutnya

Praktis, miner solve dalam menit, jauh sebelum overlap.

Cooperative cancellation

while (!atomic_load(&g_shutdown) && !atomic_load(&g_found)) {
    int hit = keccak256_avx512_search(challenge, target, nonce_base, BATCH_SIZE, ...);
    if (hit) {
        /* claim find via CAS */
        int expected = 0;
        if (atomic_compare_exchange_strong(&g_found, &expected, 1)) {
            printf("FOUND %llu 0x%s\n", found_nonce, hash_hex);
            fflush(stdout);
        }
        return NULL;
    }
    nonce_base += BATCH_SIZE * 8;
}

g_found = atomic int yang di-set ke 1 oleh thread pertama yang ketemu. Worker lain check di awal loop iter berikutnya, jadi exit dalam <100µs.

g_shutdown = atomic int yang di-set saat SIGINT/SIGTERM atau orchestrator kill miner. Same fast exit.

Race in find handoff

Bug halus: kalau 2 thread find di batch yang sama (sangat jarang, tapi mungkin), siapa yang printf?

Solusi: atomic_compare_exchange_strong(&g_found, &expected_0, 1) — cuma 1 thread yang sukses CAS, yang itu printf. Yang gagal CAS langsung return tanpa print (their nonce dibuang, tidak masalah karena yang utama adalah valid nonce di stdout).

Verifikasi pre-print

Ada paranoid check: setelah AVX-512 path bilang "hit", verify ulang pakai scalar Keccak. Ini protect dari potential SIMD bug:

keccak256_from_lanes(challenge_le, found_nonce, verify_hash);
if (memcmp(hash, verify_hash, 32) != 0) {
    fprintf(stderr, "[miner] WARN: SIMD/scalar mismatch...\n");
    nonce_base = found_nonce + 1;
    continue;  /* skip past, continue search */
}

In practice, never fires (test_keccak.c sudah verify). But cheap insurance.

Orchestrator: pre-flight checks

Submit mine(nonce) ke contract = bayar gas (~$0.05-0.50 tergantung base fee). Kalau tx revert → tetap bayar gas tapi tidak dapat reward. Untuk minimize ETH waste, orchestrator run 5 pre-flight checks sebelum kirim tx beneran:

flowchart TD
    F[FOUND nonce hash<br/>dari miner stdout]
    F --> P1{1. Re-fetch<br/>challenge?}
    P1 -- "challenge rotated" --> S1[SKIP<br/>save gas]
    P1 -- "challenge same" --> P2{2. epochBlocksLeft<br/>>= 3?}
    P2 -- "blocks < 3" --> S2[SKIP<br/>race risk]
    P2 -- "OK" --> P3{3. hash still<br/>< difficulty?}
    P3 -- "diff raised" --> S3[SKIP<br/>obsolete]
    P3 -- "OK" --> P4{4. base fee <br/>MAX_GAS_PRICE_GWEI?}
    P4 -- "gas spike" --> S4[SKIP<br/>too expensive]
    P4 -- "OK" --> P5{5. eth_call<br/>simulation?}
    P5 -- "would revert" --> S5[SKIP<br/>guaranteed fail]
    P5 -- "OK" --> SUBMIT[SUBMIT mine tx]
    SUBMIT --> RECEIPT{receipt status}
    RECEIPT -- "1 (success)" --> CONFIRMED[CONFIRMED<br/>+100 HASH]
    RECEIPT -- "0 (revert)" --> REVERT[REVERTED<br/>-gas, no reward]
Loading

1. Challenge re-fetch

Antara waktu miner solve sampai orchestrator submit, challenge bisa rotasi (kalau orang lain submit duluan di blok sebelum kita). Submit dengan stale challenge = guaranteed revert.

const freshChallenge = await contract.getChallenge(wallet.address);
if (freshChallenge !== foundContext.challenge) {
    skip("challenge rotated");
}

Cost: 1 RPC call (~50-200ms latency).

2. Epoch proximity

epochBlocksLeft < 3 artinya difficulty akan adjust dalam <36 detik (~3 blok × 12s). High risk: tx kita land setelah adjust → difficulty baru → hash kita sudah obsolete.

const blocksLeft = Number(freshState.epochBlocksLeft_);
if (blocksLeft < 3) skip(`only ${blocksLeft} blocks until epoch`);

3. Difficulty re-check

Edge case: dalam 3 blok terakhir, difficulty bisa raise. Re-verify hash kita masih < difficulty current.

if (BigInt(foundContext.hash) >= freshState.difficulty) {
    skip("difficulty raised, our hash now obsolete");
}

4. Gas price ceiling

Kalau base fee melebihi MAX_GAS_PRICE_GWEI (default 20 gwei), skip. Logic: better hilang 1 reward 100 HASH daripada bayar $5+ gas.

const baseFeeGwei = Number(ethers.formatUnits(feeData.gasPrice || 0n, "gwei"));
if (baseFeeGwei > MAX_GAS_PRICE_GWEI) skip(`gas ${baseFeeGwei} > MAX ${MAX_GAS_PRICE_GWEI}`);

5. Static call simulation

eth_call execute tx secara virtual di node tanpa state change, tanpa gas. Kalau revert di simulasi → revert beneran juga = pasti gagal.

try {
    await contract.mine.staticCall(nonce);
    /* OK, lanjut submit */
} catch (err) {
    skip(`staticCall would revert: ${err.shortMessage}`);
}

Powerful: catch race conditions yang 1-4 miss (e.g., orang lain juga submit ke nonce sama dalam 1 blok).

Hasil di production

Dari miner-firstrun.log 1 sesi:

attempts=2 ok=2 revert=0 skipped=0

Pre-flight extremely effective — most "stale solutions" caught BEFORE submitting, ETH terjaga.

Monitor: read-only dashboard

Monitor adalah Node.js + Express server yang tidak punya state sendiri. Read dari:

  • orchestrator/stats.json — atomic snapshot ditulis tiap 10s + on event oleh orchestrator
  • orchestrator/miner.log — full log file
  • Ethereum RPC — live contract state (cache 15s untuk hemat call)
flowchart LR
    O[orchestrator.js<br/>writes stats.json + log] --> JSON[(stats.json)]
    O --> LOG[(miner.log)]
    SRV[server.js<br/>port 7878]
    SRV -- "GET /api/stats" --> JSON
    SRV -- "GET /api/log" --> LOG
    SRV -- "GET /api/network" --> RPC((RPC))
    HTML["public/index.html<br/>(vanilla JS, fetch every 3s)"]
    HTML --> SRV
    USER[User browser] --> HTML
Loading

Endpoints

Endpoint Hit cost Cache Returns
GET /api/health trivial none uptime + flags
GET /api/stats read 1 file none stats.json content
GET /api/log?lines=200 read tail file none last N lines
GET /api/network 5 RPC calls 15s block, era, reward, difficulty, gas

Why decoupled (file-based, not pipe)?

Kalau monitor crash, orchestrator tetap jalan — tidak ada dependency. Restart monitor = re-read file, instant resync.

Trade-off: stats.json bisa stale 0-10 detik. For mining where solve time = minutes, OK.

Trade-off design

1. CPU only, no GPU

Pro: Simple, portable, cocok dengan economics current. AVX-512 cukup untuk solve avg 11 menit di difficulty saat ini.

Con: Kalau difficulty raise 100x, GPU jadi worth-it. Roadmap di GPU-PORTING.md.

2. Single wallet (no proxy / multi-wallet)

Pro: Simple. 1 wallet → 1 challenge dari kontrak. Pre-flight checks hindari double-submit.

Con: Kalau wallet kena banned (kontrak tidak punya banlist saat ini, tapi possible), satu titik failure. Workaround: setup multiple instances dengan wallet beda di VPS terpisah.

3. Node.js orchestrator (bukan Rust/Go)

Pro: ethers.js mature, banyak example, gampang debug, package ecosystem solid.

Con: Slightly slower startup, JIT warmup. Tapi orchestrator I/O-bound, bukan CPU-bound.

Alternatif kalau mau Rust: pakai ethers-rs atau alloy-rs. Boilerplate ~3x lebih, gain perf negligible (orchestrator bukan bottleneck).

4. State as JSON file (bukan SQLite/Redis)

Pro: Human-readable, no migration, no daemon dependency, gampang inspect.

Con: Race kalau monitor + orchestrator coba write sama-sama. Solusi: monitor read-only, hanya orchestrator yang write.

5. C for hashing (bukan Rust/asm)

Pro: Most direct path ke compiler intrinsics (_mm512_*). Banyak referensi (XKCP). Gcc punya --param=max-inline-insns-single yang cukup untuk inline Keccak-f.

Con: Manual memory mgmt (minimal di kasus ini), no formal verification.

Rust bisa pakai core::arch::x86_64::* intrinsics — sama-sama _mm512_*. Performance equivalent. Tapi sudah ada XKCP reference C, no benefit redo in Rust.

6. Compile dengan -march=native

Pro: Compiler tune untuk CPU lokal — best perf.

Con: Binary tidak portable. Kalau build di Skylake-X, run di Zen 4 → mungkin slower (pakai instr Skylake, bukan Zen-optimal).

Workaround untuk binary distribution: build dengan -march=skylake-avx512 (lowest common AVX-512 baseline), accept ~5-10% loss vs native.

Roadmap & extensions

Add proxy / multi-RPC failover

Saat ini single RPC. Kalau publicnode down, miner stuck.

Tambah ke orchestrator.js:

const RPC_URLS = [
    "https://ethereum-rpc.publicnode.com",
    "https://eth.llamarpc.com",
    "https://rpc.ankr.com/eth",
];

async function tryRpcs(fn) {
    for (const url of RPC_URLS) {
        const p = new ethers.JsonRpcProvider(url);
        try { return await fn(p); } catch (e) { logErr(`RPC ${url} failed`); }
    }
    throw new Error("all RPCs down");
}

Add Telegram alert

Kalau miner crash / ETH < 0.001 / wallet balance > N (auto-withdraw target):

const TG_TOKEN = process.env.TELEGRAM_BOT_TOKEN;
const TG_CHAT  = process.env.TELEGRAM_CHAT_ID;
async function alert(msg) {
    if (!TG_TOKEN) return;
    await fetch(`https://api.telegram.org/bot${TG_TOKEN}/sendMessage`, {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ chat_id: TG_CHAT, text: msg }),
    });
}

Add multi-wallet farm

Run N orchestrator instance dengan wallet berbeda, shared mining pool:

  • 1 daemon pool.js yang dispatch challenge ke pool of miner workers
  • N orchestrator hanya submit; tidak spawn miner

Big rewrite. Worth it kalau punya 10+ wallets.

Add GPU support

Lihat GPU-PORTING.md untuk full roadmap (CUDA atau OpenCL backend).

Add MEV protection (Flashbots / mev-share)

Saat ini mine tx submit ke public mempool — bisa di-front-run oleh searcher kalau strategy mereka catch bahwa hash kita valid.

Workaround: submit via Flashbots Protect RPC. Same code, ganti URL ke https://rpc.flashbots.net.

Auto-difficulty / hashrate tuner

Sekarang THREADS fixed. Bisa auto-detect jumlah core via:

const os = require("os");
const THREADS = parseInt(process.env.THREADS || os.cpus().length, 10);

Atau adaptive: kalau miner tidak find dalam 30 menit, tambah thread (tapi udah max-out di physical core, useless above that).

Kalau mau dive code

Urut bacaan ngerti yang paling cepat:

  1. src/util.h — hex encoding, BE/LE conversion (utility, baca dulu)
  2. src/keccak_scalar.c — Keccak-f[1600] reference, ~150 baris
  3. src/keccak_avx512.h — AVX-512 macro definitions, ~50 baris
  4. src/keccak_avx512.c — AVX-512 8-way implementation, ~400 baris
  5. src/miner.c — main, threading, FOUND output
  6. orchestrator/orchestrator.js — main loop + spawn miner + submit tx
  7. orchestrator/preflight.js — config validation
  8. monitor/server.js — REST endpoints

Extra reading:

  • XKCP reference: https://github.com/XKCP/XKCP
  • "Cryptographic Sponge Functions" paper (Bertoni et al)
  • Ethereum yellow paper Appendix C (Keccak as KECCAK256 opcode)