feat(ds4v): dflash vision serving runtime with bounded IQ85 conversion - #722
Open
marcelormendes wants to merge 85 commits into
Open
feat(ds4v): dflash vision serving runtime with bounded IQ85 conversion#722marcelormendes wants to merge 85 commits into
marcelormendes wants to merge 85 commits into
Conversation
CPU-only tool turning abliterated Vision-Exp safetensors into ROCmFPX MIX GGUF. Down experts to qtype 105 with embedded P4MIXv1 codebooks. Gate and up experts to qtype 106 with a split GUMIXs1 sidecar. Vision, aligner, image and bias_vl tensors pass through losslessly. Calibration needs an imatrix file or explicit absmax-only. Includes block codec roundtrip plus layout plus rejection unit test.
This reverts commit 8e28cfb.
This reverts commit 4f388d1.
…evidence Adds the operator handoff for the DeepSeek V4 Flash Vision work: the continuation status authority, the DS4V vision plan it supersedes, the four decision logs for the DS4V-1/2/3 and continuation passes, and the IQ85 candidate evidence summary describing conversion output, trials and the unmet quality/speed gates. No code changes.
Reconciles the vision image-prompt work with upstream's paged-serving refactor. Conflict resolutions: - ggml: keep both extension op sets (four DS4V vision ops plus GGML_OP_DS4_MOE_COMBINE); GGML_OP_COUNT 105 -> 110 in the enum and both ggml.c tables. - deepseek4 budget: compute_ds4_hybrid_budget_info now takes both with_vision and paged, charges paged KV on the primary target and still reserves the vision scratch allowance through vision::remaining_expert_budget. - attention: build_mla_attention_lane_core takes upstream's prepared lane and out_attn_context alongside the image span view; the contiguous wrapper forwards image spans so HEAD callers keep working. - snapshot: upstream moved the snapshot helpers into deepseek4_snapshot.cpp (HEAD's inline copies were byte-identical to the merge base), so the duplicated definitions were dropped and release_image_scratch was kept.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
38 issues found and verified against the latest diff
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="server/tools/ds4v_image_prepare/verify.py">
<violation number="1" location="server/tools/ds4v_image_prepare/verify.py:27">
P1: Running this verifier with Python optimization (`-O` or `PYTHONOPTIMIZE`) removes the hash, count, and probe-status checks, allowing invalid fixtures or a failing probe to produce a `PASS` verdict. Replace these assertions with explicit exceptions or return-code checks.</violation>
<violation number="2" location="server/tools/ds4v_image_prepare/verify.py:30">
P2: If the native probe hangs, this wrapper waits forever instead of failing the qualification. Add a bounded timeout and convert `TimeoutExpired` into a nonzero verification result.</violation>
<violation number="3" location="server/tools/ds4v_image_prepare/verify.py:34">
P2: When a rerun fails after an earlier success, the old `verdict.json` remains `PASS` and can be mistaken for the current result. Remove or invalidate the existing verdict before validation, then publish the success artifact only after all checks pass.</violation>
</file>
<file name="server/tools/ds4v_image_prompt/verify_fixtures.py">
<violation number="1" location="server/tools/ds4v_image_prompt/verify_fixtures.py:20">
P1: When this verifier runs with `python -O` or `-OO`, Python removes the hash, count, and probe-status assertions, so invalid fixtures or a failing probe can still produce `PASS`. Replace these assertions with explicit exceptions or return-code checks.</violation>
<violation number="2" location="server/tools/ds4v_image_prompt/verify_fixtures.py:27">
P2: On a fresh checkout, the documented verification reaches this line and fails because the output parent directory does not exist. Create `a.output.parent` before writing the verdict.</violation>
</file>
<file name="server/tools/ds4v_vision/compare.py">
<violation number="1" location="server/tools/ds4v_vision/compare.py:22">
P1: When the reference manifest is empty or omits one required image, this loop skips that case and the comparator still exits 0. Validate the expected `corn` and `carrots` image set before iterating.</violation>
<violation number="2" location="server/tools/ds4v_vision/compare.py:25">
P2: When this verifier runs with Python optimization enabled, the fixture hashes and shape check disappear because they are `assert` statements. Replace these gate validations with explicit failure checks so an invalid reference cannot return a successful qualification.</violation>
</file>
<file name="server/CMakeLists.txt">
<violation number="1" location="server/CMakeLists.txt:383">
P2: On Windows, the default build enters this branch and fails compiling `ds4_mix_converter` because its source is POSIX-only. Gate this target on non-Windows or port the converter to the Win32 file APIs.</violation>
</file>
<file name="server/tools/ds4v_image_policy/verify_fixtures.py">
<violation number="1" location="server/tools/ds4v_image_policy/verify_fixtures.py:16">
P2: This verifier is only manually invoked, so routing or raw-mask regressions can ship while the configured CTest and CI suites remain green. Register it with the existing test framework or add a CI job that stages its required fixtures and model.</violation>
<violation number="2" location="server/tools/ds4v_image_policy/verify_fixtures.py:28">
P2: When this verifier runs with `python -O`, it skips all source, fixture, routing, and mask validation yet can still write `verdict: PASS`. Replace verification assertions with explicit exceptions, or reject optimized mode before running the checks.</violation>
</file>
<file name="server/tools/ds4v_vision/linear_source.cpp">
<violation number="1" location="server/tools/ds4v_vision/linear_source.cpp:75">
P2: On HIP, this assertion requires the unbiased lane to dispatch the fused vision operation, producing two fused nodes and two Lt launches. Keep the unbiased lane on the ordinary product/cast path and assert one fused operation and one launch for the biased lane.</violation>
</file>
<file name="server/tools/ds4_bf16_affine/CMakeLists.txt">
<violation number="1" location="server/tools/ds4_bf16_affine/CMakeLists.txt:7">
P2: `ggml/src/libggml-cpu.so.0` is the wrong path, so the EXISTS check always fails and the build aborts with FATAL_ERROR before it ever links. ggml-cpu is defined in the `ggml/src/ggml-cpu` subdirectory, so its built library lives at `${GGML_BUILD}/ggml/src/ggml-cpu/libggml-cpu.so.0` (ggml-base is at `ggml/src/` and ggml-hip at `ggml/src/ggml-hip/`, which are correct). Change the ggml-cpu entry to `ggml/src/ggml-cpu/libggml-cpu.so.0`.</violation>
</file>
<file name="server/tools/ds4v_image_integration/test.cpp">
<violation number="1" location="server/tools/ds4v_image_integration/test.cpp:216">
P2: This integration test is not part of the repository’s normal build or CI test gate, so regressions in these invariants can ship with every standard `ctest` run still passing. Add the coverage to the established server test target, or explicitly wire this standalone project into CI.</violation>
</file>
<file name="server/tools/ds4v_image_assembly/test.cpp">
<violation number="1" location="server/tools/ds4v_image_assembly/test.cpp:23">
P2: This coverage is absent from the normal `DFLASH27B_TESTS`/`test_server_unit` CTest run because this file creates a separate executable. Move these cases into the existing unit-test framework so standard server CI executes them.</violation>
</file>
<file name="server/tools/ds4v_vision/reference_math.py">
<violation number="1" location="server/tools/ds4v_vision/reference_math.py:58">
P2: When `OUTPUT_DIR` aliases `NATIVE`, the script overwrites each native tensor before reading it, so `native_vs_math` reports an artificial exact match. Read both inputs before writing outputs, or reject overlapping directories.</violation>
<violation number="2" location="server/tools/ds4v_vision/reference_math.py:59">
P2: If a reference fixture is stale or corrupted, this script accepts it as the baseline because it validates only source hashes. Verify each patch and reference file against the manifest before loading them, as `compare.py` does.</violation>
</file>
<file name="server/tools/ds4_mix_converter/ds4_mix_converter.cpp">
<violation number="1" location="server/tools/ds4_mix_converter/ds4_mix_converter.cpp:626">
P2: `--validate-input-only` cannot validate a source directory unless the caller also supplies an unused imatrix or `--absmax-only`. Exempt this mode from the mutually-exclusive calibration requirement for the non-IQ85 recipe.</violation>
<violation number="2" location="server/tools/ds4_mix_converter/ds4_mix_converter.cpp:903">
P2: When a global embedding or output weight is FP8, the converter fails on its companion `.scale` tensor after successfully pairing it in `mapped_source_spec`. Skip global FP8 scale entries during enumeration, as the layer path already does.</violation>
</file>
<file name="server/tools/ds4v_image_prompt/test.cpp">
<violation number="1" location="server/tools/ds4v_image_prompt/test.cpp:19">
P2: This test never runs in the repository’s normal `server` build or CTest suite, so regressions in `prepare_image_prompt` can ship while CI remains green. Integrate these cases with the existing server unit-test target and registration, or explicitly wire this target into the parent build rather than relying on a separately configured tool project.
(Based on your team's feedback about integrating new tests with the existing framework.)</violation>
</file>
<file name="server/tools/ds4v_vision/loader_tests.py">
<violation number="1" location="server/tools/ds4v_vision/loader_tests.py:25">
P2: The malformed-loader suite is not part of the normal CTest run, so CI can pass without executing these 16 rejection checks. Register this harness with CTest or move the cases into the existing test suite.</violation>
</file>
<file name="server/tools/ds4v_image_prompt/fixtures.cpp">
<violation number="1" location="server/tools/ds4v_image_prompt/fixtures.cpp:46">
P2: The added source-fixture coverage is not run by CTest, so layout regressions can ship while the registered suite remains green. Integrate these cases into the registered test target or register this executable conditionally with its fixture directory.
(Based on your team's feedback about extending tests through the existing test framework).</violation>
</file>
<file name="server/tools/ds4v_preprocess_probe/generate_reference_fixtures.py">
<violation number="1" location="server/tools/ds4v_preprocess_probe/generate_reference_fixtures.py:78">
P2: When `--output` names an existing directory, this recursively deletes the entire directory, including files unrelated to these fixtures. Refuse non-empty output paths or delete only files created by this generator.</violation>
<violation number="2" location="server/tools/ds4v_preprocess_probe/generate_reference_fixtures.py:135">
P2: When Python runs with `-O`, this assertion and the other assertions in `main()` disappear, so the generator can emit fixtures without checking source and rebuilt tensors match. Replace validation assertions with explicit exceptions that remain active in optimized runs.</violation>
</file>
<file name="server/tools/ds4_mix_converter/CMakeLists.txt">
<violation number="1" location="server/tools/ds4_mix_converter/CMakeLists.txt:17">
P2: When the build host has no network access, configuration fails even if `nlohmann_json` is already installed because this file never checks for a system package. Prefer `find_package(nlohmann_json CONFIG QUIET)` and retain the pinned FetchContent block only as a fallback.</violation>
<violation number="2" location="server/tools/ds4_mix_converter/CMakeLists.txt:45">
P2: The new fitter and IQ tests are invisible to the repository’s standard test command because this separate CMake project is not part of the parent test registration. Move these cases into the existing CppUnit suite or wire their targets into the parent CTest discovery so CI runs them.</violation>
</file>
<file name="server/tools/ds4_mix_converter/prove_iq85.py">
<violation number="1" location="server/tools/ds4_mix_converter/prove_iq85.py:44">
P2: When shard payloads or config/tokenizer files change without changing the index, this manifest still claims the same source. Record checksums for every source file consumed by the converter, or write a complete source checksum manifest.</violation>
<violation number="2" location="server/tools/ds4_mix_converter/prove_iq85.py:63">
P2: When a lane exceeds `--timeout`, the manifest omits its exit status and duration, and the wrapper exits with Python's generic failure status. Catch `TimeoutExpired`, record `exit_code=124`, save the lane, and terminate with status 124.</violation>
</file>
<file name="server/tools/ds4v_vision/reference_stages.py">
<violation number="1" location="server/tools/ds4v_vision/reference_stages.py:30">
P2: When this tool runs with Python optimization enabled, the source-integrity and fixture-reproduction checks disappear, so it can produce stage metrics from an unverified parent implementation. Replace these assertions with explicit conditional checks that raise an exception.</violation>
</file>
<file name="server/tools/ds4v_image_input/test.cpp">
<violation number="1" location="server/tools/ds4v_image_input/test.cpp:19">
P2: Adding `main()` here creates a second image-input test executable outside the server’s normal CppUnit/CTest suite. Move these cases into `test_server_unit` and remove the standalone runner so the coverage is discovered and maintained with the existing suite.
(Based on your team's feedback about extending tests through the existing test framework.) [2bd18e9d-595d-43f5-9a23-697c6928cbc2]</violation>
</file>
<file name="server/src/deepseek4/deepseek4_image_policy.h">
<violation number="1" location="server/src/deepseek4/deepseek4_image_policy.h:26">
P2: The interval comment gives `image_end` inclusive semantics, but `raw_key_visible` and the production caller use an exclusive end. Document the range as `[IMAGE_START, IMAGE_END)` so callers do not pass an inclusive endpoint and lose the final image key.</violation>
</file>
<file name="server/tools/ds4v_vision/probe.cpp">
<violation number="1" location="server/tools/ds4v_vision/probe.cpp:71">
P2: When a probe is rerun with an existing output directory, this silently overwrites qualification artifacts and can leave stale stage snapshots mixed with the new run. Require a fresh output directory, matching the other vision qualification tools, before writing any files.</violation>
</file>
<file name="server/tools/ds4v_vision/linear_rounding.cpp">
<violation number="1" location="server/tools/ds4v_vision/linear_rounding.cpp:49">
P2: `ds4v_linear_rounding` is never run by CTest, so this self-contained rounding regression check can silently regress. Register it with CTest or fold its case into `ds4v_linear_contract`.
(Based on your team's feedback about extending tests through the existing test framework.)</violation>
</file>
<file name="decisions-ds4v-continuation.tsv">
<violation number="1" location="decisions-ds4v-continuation.tsv:21">
P2: Spaces are stripped between words and numbers throughout this decision log, so factual records are hard to read and ambiguous: 'All129 tensors', 'complete2filemanifestpublishedbeforeexit0', 'staybyteidentical', 'nowpass but featuresstillfail', 'Implementation scopedPASS; targettowerexit3', 'test PID exited;8016not touched', 'PID3396669gone/KFDempty'. The PR frames this TSV as the record for future gating iterations, so restore the intended spacing so PID values, tensor counts, and exit codes are unambiguous.</violation>
</file>
<file name="server/tools/ds4v_image_prepare/compose.cpp">
<violation number="1" location="server/tools/ds4v_image_prepare/compose.cpp:46">
P2: When a custom Jinja template preserves the marker count but reorders or relocates markers, this check passes. `prepare_image_prompt` then pairs extraction-order images with rendered-order markers, attaching pixels to the wrong prompt position. Preserve per-image identity through rendering or reject templates whose marker sequence changes.</violation>
</file>
<file name="server/tools/ds4v_vision/norm_contract.cpp">
<violation number="1" location="server/tools/ds4v_vision/norm_contract.cpp:17">
P2: When core dumps are enabled, each intentionally aborted child can write a dump, so this test may create ten core files and exhaust CI disk or leave large artifacts. Disable core dumps in the child before exercising rejected inputs, as `attention_contract.cpp` does.</violation>
</file>
<file name="server/tools/ds4v_vision/linear_contract.cpp">
<violation number="1" location="server/tools/ds4v_vision/linear_contract.cpp:13">
P2: Each invalid-input child can emit a core dump, and this test launches 21 such aborting children. Disable core dumps in the child before constructing invalid tensors, as `attention_contract.cpp` does.</violation>
</file>
<file name="server/deps/llama.cpp/ggml/rocmfpx/rocmfpx.c">
<violation number="1" location="server/deps/llama.cpp/ggml/rocmfpx/rocmfpx.c:1357">
P2: When a finite input is large enough for `delta * delta` to overflow, every candidate scores `INFINITY` and the MIX encoder rejects the row. Accumulate or clamp each error term before comparing candidates so finite inputs follow the format's saturation behavior.</violation>
</file>
<file name="docs/ds4v-uncensored-vision-plan.md">
<violation number="1" location="docs/ds4v-uncensored-vision-plan.md:3">
P2: The plan's deliverable is a model explicitly designed to comply with harmful requests, and Appendix C's only mitigation is 'keep them local and never serve them publicly,' yet DS4V-3's verification exercises the model through a persistent OpenAI-compatible /v1/chat/completions endpoint and the broader PR wires serving-runtime integration for it. Nothing in the plan enforces the 'never serve publicly' constraint, so a misconfigured deployment would expose the uncensored model. Make the public-exposure restriction an explicit, checked gate (e.g., bind to loopback / auth required, and a lane that asserts it) rather than a prose note.</violation>
</file>
Tip: cubic used a learning from your PR history. Let your coding agent read cubic learnings directly with the cubic MCP.
Re-trigger cubic
| verified={} | ||
| for name,sha in expected.items(): | ||
| if name.split('/')[0] in ('corn','carrots'): | ||
| assert digest(a.fixtures/name)==sha,name |
Contributor
There was a problem hiding this comment.
P1: Running this verifier with Python optimization (-O or PYTHONOPTIMIZE) removes the hash, count, and probe-status checks, allowing invalid fixtures or a failing probe to produce a PASS verdict. Replace these assertions with explicit exceptions or return-code checks.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At server/tools/ds4v_image_prepare/verify.py, line 27:
<comment>Running this verifier with Python optimization (`-O` or `PYTHONOPTIMIZE`) removes the hash, count, and probe-status checks, allowing invalid fixtures or a failing probe to produce a `PASS` verdict. Replace these assertions with explicit exceptions or return-code checks.</comment>
<file context>
@@ -0,0 +1,36 @@
+verified={}
+for name,sha in expected.items():
+ if name.split('/')[0] in ('corn','carrots'):
+ assert digest(a.fixtures/name)==sha,name
+ verified[name]=sha
+assert len(verified)==28
</file context>
| if name.split('/')[0] not in ('corn','carrots'): | ||
| continue | ||
| actual=hashlib.sha256((a.fixtures/name).read_bytes()).hexdigest() | ||
| assert actual==digest,name |
Contributor
There was a problem hiding this comment.
P1: When this verifier runs with python -O or -OO, Python removes the hash, count, and probe-status assertions, so invalid fixtures or a failing probe can still produce PASS. Replace these assertions with explicit exceptions or return-code checks.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At server/tools/ds4v_image_prompt/verify_fixtures.py, line 20:
<comment>When this verifier runs with `python -O` or `-OO`, Python removes the hash, count, and probe-status assertions, so invalid fixtures or a failing probe can still produce `PASS`. Replace these assertions with explicit exceptions or return-code checks.</comment>
<file context>
@@ -0,0 +1,29 @@
+ if name.split('/')[0] not in ('corn','carrots'):
+ continue
+ actual=hashlib.sha256((a.fixtures/name).read_bytes()).hexdigest()
+ assert actual==digest,name
+ verified[name]=digest
+assert len(verified)==28
</file context>
| manifest = json.loads((a.reference / 'manifest.json').read_text()) | ||
| results = {} | ||
| passed = True | ||
| for label, entry in manifest['images'].items(): |
Contributor
There was a problem hiding this comment.
P1: When the reference manifest is empty or omits one required image, this loop skips that case and the comparator still exits 0. Validate the expected corn and carrots image set before iterating.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At server/tools/ds4v_vision/compare.py, line 22:
<comment>When the reference manifest is empty or omits one required image, this loop skips that case and the comparator still exits 0. Validate the expected `corn` and `carrots` image set before iterating.</comment>
<file context>
@@ -0,0 +1,45 @@
+manifest = json.loads((a.reference / 'manifest.json').read_text())
+results = {}
+passed = True
+for label, entry in manifest['images'].items():
+ results[label] = {}
+ patches = entry['patches']
</file context>
|
|
||
| option(DFLASH27B_DS4_MIX_CONVERTER | ||
| "Build the CPU-only DeepSeek-V4 safetensors to MIX GGUF converter" ON) | ||
| if(DFLASH27B_DS4_MIX_CONVERTER) |
Contributor
There was a problem hiding this comment.
P2: On Windows, the default build enters this branch and fails compiling ds4_mix_converter because its source is POSIX-only. Gate this target on non-Windows or port the converter to the Win32 file APIs.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At server/CMakeLists.txt, line 383:
<comment>On Windows, the default build enters this branch and fails compiling `ds4_mix_converter` because its source is POSIX-only. Gate this target on non-Windows or port the converter to the Win32 file APIs.</comment>
<file context>
@@ -377,6 +377,22 @@ if(DFLASH27B_MIXED_GGML_SHARED)
+
+option(DFLASH27B_DS4_MIX_CONVERTER
+ "Build the CPU-only DeepSeek-V4 safetensors to MIX GGUF converter" ON)
+if(DFLASH27B_DS4_MIX_CONVERTER)
+ add_executable(ds4_mix_converter
+ tools/ds4_mix_converter/ds4_mix_converter.cpp)
</file context>
Suggested change
| if(DFLASH27B_DS4_MIX_CONVERTER) | |
| if(DFLASH27B_DS4_MIX_CONVERTER AND NOT WIN32) |
| print(run.stdout,end='') | ||
| print(run.stderr,end='') | ||
| assert run.returncode==0,run.returncode | ||
| a.output.write_text(json.dumps({'verdict':'PASS','verified_source_fixture_hashes':verified, |
Contributor
There was a problem hiding this comment.
P2: On a fresh checkout, the documented verification reaches this line and fails because the output parent directory does not exist. Create a.output.parent before writing the verdict.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At server/tools/ds4v_image_prompt/verify_fixtures.py, line 27:
<comment>On a fresh checkout, the documented verification reaches this line and fails because the output parent directory does not exist. Create `a.output.parent` before writing the verdict.</comment>
<file context>
@@ -0,0 +1,29 @@
+print(run.stdout,end='')
+print(run.stderr,end='')
+assert run.returncode==0,run.returncode
+a.output.write_text(json.dumps({'verdict':'PASS','verified_source_fixture_hashes':verified,
+ 'single_image_cases':10,'ordered_pair_cases':2,'decoder_calls':0,'tower_calls':0,
+ 'scope':'Unintegrated prompt preparation using accepted source RGB core layouts and BF16 patch fixtures'},indent=2)+'\n')
</file context>
| ? render_chat_template(chat,ChatFormat::DEEPSEEK4,true,false,tools) | ||
| : render_chat_template_jinja(jinja,chat,"","",true,false,tools); | ||
| auto tokens=tokenizer.encode(rendered); | ||
| if (std::count(tokens.begin(),tokens.end(),129264)!=static_cast<ptrdiff_t>(encoded.size())) |
Contributor
There was a problem hiding this comment.
P2: When a custom Jinja template preserves the marker count but reorders or relocates markers, this check passes. prepare_image_prompt then pairs extraction-order images with rendered-order markers, attaching pixels to the wrong prompt position. Preserve per-image identity through rendering or reject templates whose marker sequence changes.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At server/tools/ds4v_image_prepare/compose.cpp, line 46:
<comment>When a custom Jinja template preserves the marker count but reorders or relocates markers, this check passes. `prepare_image_prompt` then pairs extraction-order images with rendered-order markers, attaching pixels to the wrong prompt position. Preserve per-image identity through rendering or reject templates whose marker sequence changes.</comment>
<file context>
@@ -0,0 +1,67 @@
+ ? render_chat_template(chat,ChatFormat::DEEPSEEK4,true,false,tools)
+ : render_chat_template_jinja(jinja,chat,"","",true,false,tools);
+ auto tokens=tokenizer.encode(rendered);
+ if (std::count(tokens.begin(),tokens.end(),129264)!=static_cast<ptrdiff_t>(encoded.size()))
+ return fail("cardinality: final image marker count differs from image count");
+ Composition result;
</file context>
| GGML_OP_RMS_NORM_VISION_F32==106 && GGML_OP_SOFT_MAX_VISION_F32==107 && GGML_OP_MUL_MAT_VISION_AV_F32==108 && GGML_OP_COUNT==109,"operation ABI changed"); | ||
| static void rejected(int mode) { | ||
| const pid_t pid=fork(); check(pid>=0,"fork failed"); | ||
| if(pid==0) { |
Contributor
There was a problem hiding this comment.
P2: When core dumps are enabled, each intentionally aborted child can write a dump, so this test may create ten core files and exhaust CI disk or leave large artifacts. Disable core dumps in the child before exercising rejected inputs, as attention_contract.cpp does.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At server/tools/ds4v_vision/norm_contract.cpp, line 17:
<comment>When core dumps are enabled, each intentionally aborted child can write a dump, so this test may create ten core files and exhaust CI disk or leave large artifacts. Disable core dumps in the child before exercising rejected inputs, as `attention_contract.cpp` does.</comment>
<file context>
@@ -0,0 +1,63 @@
+ GGML_OP_RMS_NORM_VISION_F32==106 && GGML_OP_SOFT_MAX_VISION_F32==107 && GGML_OP_MUL_MAT_VISION_AV_F32==108 && GGML_OP_COUNT==109,"operation ABI changed");
+static void rejected(int mode) {
+ const pid_t pid=fork(); check(pid>=0,"fork failed");
+ if(pid==0) {
+ auto c=ggml_init({1024*1024,nullptr,true});
+ auto x=ggml_new_tensor_2d(c,mode==1?GGML_TYPE_BF16:GGML_TYPE_F32,mode==2?512:1024,mode==3?15:16);
</file context>
| static_assert(GGML_OP_PAGED_ATTN==104 && GGML_OP_MUL_MAT_BIAS_BF16==105 && GGML_OP_RMS_NORM_VISION_F32==106 && GGML_OP_SOFT_MAX_VISION_F32==107 && GGML_OP_MUL_MAT_VISION_AV_F32==108 && GGML_OP_COUNT==109,"operation ABI changed"); | ||
| static void rejected(int mode,bool with_bias=true) { | ||
| pid_t pid=fork(); check(pid>=0,"fork failed"); | ||
| if(pid==0) { |
Contributor
There was a problem hiding this comment.
P2: Each invalid-input child can emit a core dump, and this test launches 21 such aborting children. Disable core dumps in the child before constructing invalid tensors, as attention_contract.cpp does.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At server/tools/ds4v_vision/linear_contract.cpp, line 13:
<comment>Each invalid-input child can emit a core dump, and this test launches 21 such aborting children. Disable core dumps in the child before constructing invalid tensors, as `attention_contract.cpp` does.</comment>
<file context>
@@ -0,0 +1,68 @@
+static_assert(GGML_OP_PAGED_ATTN==104 && GGML_OP_MUL_MAT_BIAS_BF16==105 && GGML_OP_RMS_NORM_VISION_F32==106 && GGML_OP_SOFT_MAX_VISION_F32==107 && GGML_OP_MUL_MAT_VISION_AV_F32==108 && GGML_OP_COUNT==109,"operation ABI changed");
+static void rejected(int mode,bool with_bias=true) {
+ pid_t pid=fork(); check(pid>=0,"fork failed");
+ if(pid==0) {
+ auto c=ggml_init({1024*1024,nullptr,true});
+ auto w=ggml_new_tensor_2d(c,mode==0?GGML_TYPE_F32:GGML_TYPE_BF16,64,32);
</file context>
| } | ||
| weight = imatrix[i]; | ||
| } | ||
| error += weight * delta * delta; |
Contributor
There was a problem hiding this comment.
P2: When a finite input is large enough for delta * delta to overflow, every candidate scores INFINITY and the MIX encoder rejects the row. Accumulate or clamp each error term before comparing candidates so finite inputs follow the format's saturation behavior.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At server/deps/llama.cpp/ggml/rocmfpx/rocmfpx.c, line 1357:
<comment>When a finite input is large enough for `delta * delta` to overflow, every candidate scores `INFINITY` and the MIX encoder rejects the row. Accumulate or clamp each error term before comparing candidates so finite inputs follow the format's saturation behavior.</comment>
<file context>
@@ -1292,3 +1292,248 @@ bool rocmfpx_validate_row_data_fp8(const void * data, size_t nbytes) {
+ }
+ weight = imatrix[i];
+ }
+ error += weight * delta * delta;
+ if (error > best) {
+ break;
</file context>
Suggested change
| error += weight * delta * delta; | |
| const float term = weight == 0.0f ? 0.0f : weight * delta * delta; | |
| error = isfinite(term) && error <= FLT_MAX - term | |
| ? error + term : FLT_MAX; |
| @@ -0,0 +1,242 @@ | |||
| # DS4V uncensored vision GGUF for Strix Halo plus 7900XT plan | |||
|
|
|||
| Build an uncensored DeepSeek V4 Flash Vision GGUF that runs on the operator pair. | |||
Contributor
There was a problem hiding this comment.
P2: The plan's deliverable is a model explicitly designed to comply with harmful requests, and Appendix C's only mitigation is 'keep them local and never serve them publicly,' yet DS4V-3's verification exercises the model through a persistent OpenAI-compatible /v1/chat/completions endpoint and the broader PR wires serving-runtime integration for it. Nothing in the plan enforces the 'never serve publicly' constraint, so a misconfigured deployment would expose the uncensored model. Make the public-exposure restriction an explicit, checked gate (e.g., bind to loopback / auth required, and a lane that asserts it) rather than a prose note.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At docs/ds4v-uncensored-vision-plan.md, line 3:
<comment>The plan's deliverable is a model explicitly designed to comply with harmful requests, and Appendix C's only mitigation is 'keep them local and never serve them publicly,' yet DS4V-3's verification exercises the model through a persistent OpenAI-compatible /v1/chat/completions endpoint and the broader PR wires serving-runtime integration for it. Nothing in the plan enforces the 'never serve publicly' constraint, so a misconfigured deployment would expose the uncensored model. Make the public-exposure restriction an explicit, checked gate (e.g., bind to loopback / auth required, and a lane that asserts it) rather than a prose note.</comment>
<file context>
@@ -0,0 +1,242 @@
+# DS4V uncensored vision GGUF for Strix Halo plus 7900XT plan
+
+Build an uncensored DeepSeek V4 Flash Vision GGUF that runs on the operator pair.
+Start from the OrcaRouter abliterated parent with vision intact.
+Quant with the prometheusAIR imatrix recipe for a 128 GiB plus 24 GiB budget.
</file context>
Contributor
Author
Handoff: where this branch actually standsPosting this so nobody has to reconstruct the state of the work from the diff. Short version: the serving/text path is proven, the vision path is not qualified, and the smaller-model experiment failed its quality and speed gates and was not installed. Nothing here should be read as a claim that vision works end to end. IntentBring DS4V (DeepSeek V4 Flash Vision, abliterated parent) image input into the dflash HIP serving path on the Strix Halo + 7900 XT pair, with the memory/prefill behaviour needed to hold a long context alongside vision, plus a bounded converter for a smaller (80-85 GB) variant of the same model. What is in the diff
Verified
Not verified / not passing
Blockers
Unresolved questions for you
|
Contributor
Author
|
@davide221 — tagging you since you reviewed #604. This is where I stopped on bringing DS4V vision input into the dflash HIP serving path; details are in the handoff comment above. Headline: the text path passes ( Also in the PR: The CI runs for this head are sitting in |
Contributor
Author
Refinement to the vision numbers aboveThe handoff comment quotes the first native HIP run; the scoped fix moved those numbers, so here is the current state of the tower gate:
Status of the image path itself, to avoid reading the above as "images do not run":
So: an image request can be submitted and answered in this runtime, but the vision tower does not meet its numerical contract and image-chat acceptance is not claimed. |
Adds harness/qualification/deepseek4/ds4v-vision/ so the DS4V vision numerical gate can be reproduced without rebuilding the measurement setup: the component-only and runtime qualification harnesses, the per-component qualification scripts with their review notes, the adopted same-GPU reference policy, the original-source reference tooling and environment receipts (constraints, libtorch/MIOpen linkage, script hashes), the frozen three-way comparison, and the native-hip-first and native-hip-scoped run summaries. Documents the current gate values, the open question of whether the target is the frozen same-GPU source or the absolute 0.9995 threshold, the next experiments, and the known gaps (reference images and guard journals are not included). Text only, about 0.5 MB.
Contributor
Author
Reproduction kit added (
|
Contributor
There was a problem hiding this comment.
1 existing issue remains and 40 new issues found across 198 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="harness/qualification/deepseek4/ds4v-vision/hip-qualification.sh">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/hip-qualification.sh:19">
P1: The harness gates HIP output against the CPU fixtures instead of the frozen same-GPU source reference. Point `reference` at `~/ds4v-work/source-rocm210-reference/source-hip-reference` and pin its `677b5ef0...` manifest; retain the CPU manifest only for a separate portability comparison.</violation>
<violation number="2" location="harness/qualification/deepseek4/ds4v-vision/hip-qualification.sh:109">
P2: On interruption or timeout, `stop_owned()` writes incomplete process timing evidence because it reaps the child with `Popen.wait()` instead of recording `wait4` usage and elapsed time. Capture resource usage while reaping the killed child so failed runs retain the documented diagnostics.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/hip-lt-concurrent-qualification.py">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/hip-lt-concurrent-qualification.py:57">
P1: If SIGTERM or SIGINT arrives between `Popen` and handler installation, the parent exits without reaping the guard, leaving its GPU child active. Install or block the handlers before spawning the guard, then restore them after cleanup.</violation>
<violation number="2" location="harness/qualification/deepseek4/ds4v-vision/hip-lt-concurrent-qualification.py:82">
P1: The adapter hash is only recorded, so a modified qualification script can change the acceptance logic while every external artifact pin still passes. Require the configured reviewed adapter hash to equal `digest(__file__)` before using the configuration.</violation>
<violation number="3" location="harness/qualification/deepseek4/ds4v-vision/hip-lt-concurrent-qualification.py:154">
P2: When two lane policies share a `run_name`, the later lane overwrites the earlier lane’s guard evidence and the qualification is no longer independently auditable. Require all resolved lane output directories to be distinct before launching any lane.</violation>
<violation number="4" location="harness/qualification/deepseek4/ds4v-vision/hip-lt-concurrent-qualification.py:165">
P2: A malformed guard report can bypass the proof check because both fields are tested by truthiness. Require `result['pass'] is True` and `result['device_proof_verified'] is True`.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/hip-lt-retry-qualification.py">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/hip-lt-retry-qualification.py:81">
P1: The config and guard hashes are not anchored to a reviewed value, so a modified guard can fabricate lane reports and bypass this numerical qualification. Verify the config and guard/policy artifacts against a trusted release receipt or hard-coded approved digests, rather than accepting a digest supplied alongside the config.</violation>
<violation number="2" location="harness/qualification/deepseek4/ds4v-vision/hip-lt-retry-qualification.py:162">
P1: When two lane policies use the same `run_name`, their executions share `out` and the later copies cannot represent independent lanes. Reject duplicate output directories before launching any lane.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/source-rocm-reference/compare-corn-three-way.py">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/source-rocm-reference/compare-corn-three-way.py:17">
P2: When this comparison runs with Python optimization enabled, its input-identity and hash checks are skipped and mismatched tensors can be reported as if they were the pinned run. Use explicit exceptions instead of `assert` for these runtime checks.</violation>
<violation number="2" location="harness/qualification/deepseek4/ds4v-vision/source-rocm-reference/compare-corn-three-way.py:20">
P1: When `source/report.json` or its output files are replaced, this check still accepts the replacement because the report supplies both the output path and its expected hash. The three-way result can therefore compare native output against a substituted source tensor instead of the frozen reference; pin the two frozen source output hashes (or verify a pinned source manifest/report digest) before reading the source files.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/mmproj-byte-proof.py">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/mmproj-byte-proof.py:15">
P1: When this proof runs under an optimized Python interpreter, every validation is removed but the script still writes a PASS manifest. Replace validation `assert` statements with explicit exceptions, or reject optimized execution before producing evidence.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/hip-unbiased-qualification.py">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/hip-unbiased-qualification.py:58">
P2: If SIGTERM or SIGINT arrives immediately after spawning a guard, the default handler bypasses cleanup and leaves the guard potentially running a GPU lane. Install or block the parent handlers before spawning the guard, then restore them after cleanup.</violation>
<violation number="2" location="harness/qualification/deepseek4/ds4v-vision/hip-unbiased-qualification.py:172">
P1: When two lane policies share `run_name`, sequential runs overwrite the same output files, so native/repeat comparisons can use the wrong lane and falsely pass. Reject duplicate lane output directories before launching.</violation>
</file>
<file name="server/deps/llama.cpp/ggml/src/ggml.c">
<violation number="1" location="server/deps/llama.cpp/ggml/src/ggml.c:1203">
P1: These entries shift the pre-existing `GGML_OP_DS4_MOE_COMBINE` ordinal, breaking the operation ABI for old graph or protocol encodings. Append the new operations after `DS4_MOE_COMBINE` and update the enum and parallel name tables together.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/source-rocm-reference/freeze-source-reference.py">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/source-rocm-reference/freeze-source-reference.py:15">
P1: When this script is invoked with `python -O`, Python removes every validation `assert`, so arbitrary or mismatched artifacts can be copied into the canonical reference without any checks. Reject optimized execution and use explicit exceptions for the integrity checks.</violation>
<violation number="2" location="harness/qualification/deepseek4/ds4v-vision/source-rocm-reference/freeze-source-reference.py:57">
P2: If a source run used stale or mismatched patches, this freeze would still accept its outputs because it checks only the manifest hash. Compare `report['patch_sha256']` with the original image patch hash before copying.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/hip-lt-qualification.sh">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/hip-lt-qualification.sh:81">
P2: `--component-only` always fails this check, so the documented standalone qualification can never reach `idle_window()` or produce evidence. Remove this guard or remove the mode from the usage and documentation.</violation>
<violation number="2" location="harness/qualification/deepseek4/ds4v-vision/hip-lt-qualification.sh:92">
P2: The prerequisite validates only a text-proof directory and three `text_*` verdicts; it never reads or hashes the required source-linear acceptance. Require an immutable linear receipt tied to `source_sha` before launching these lanes.</violation>
<violation number="3" location="harness/qualification/deepseek4/ds4v-vision/hip-lt-qualification.sh:196">
P2: This only checks that the `libggml-hip` soname appears in `ldd`; it never compares the resolved path with the pinned files, so a different GGML or ROCm library can qualify. Validate every effective soname path against its pinned path before running the probe.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/runtime-qualification.sh">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/runtime-qualification.sh:8">
P2: This qualification can validate stale or modified external binaries and fixtures instead of the PR runtime. Pin and verify the probe, source commit, comparator, projector, and reference manifest before recording qualification evidence.</violation>
<violation number="2" location="harness/qualification/deepseek4/ds4v-vision/runtime-qualification.sh:47">
P2: When this qualification runs with Python optimization enabled, its finite, shape, and byte-identity gates disappear while the verdict still records PASS. Use explicit exceptions for these checks and run the comparator with a non-optimized isolated interpreter.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/reference-fixtures.py">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/reference-fixtures.py:17">
P2: This limits only intra-op parallelism and leaves inter-op scheduling at its default, so fixture generation can oversubscribe CPUs and differ from the controlled two-thread reference. Set both Torch thread pools to two before importing the model.</violation>
<violation number="2" location="harness/qualification/deepseek4/ds4v-vision/reference-fixtures.py:26">
P2: When `output` contains a prior run, an interrupted regeneration can leave manifest hashes paired with changed or partial files. Require a fresh output directory or stage the complete fixture set before publishing it.</violation>
<violation number="3" location="harness/qualification/deepseek4/ds4v-vision/reference-fixtures.py:38">
P2: When `--encode` uses a different or corrupted model shard, this manifest still looks valid because it does not bind the loaded weights. Hash the index and every loaded shard before publishing encoded fixtures.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/hip-linear-qualification.sh">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/hip-linear-qualification.sh:183">
P2: The harness can qualify with an alternate GGML library: this check does not prove that the probe loaded the pinned files, and the later hash is diagnostic only. Require each resolved GGML SONAME to match its pinned `ggml_build` path before launching the GPU lanes.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/capture-comparator-runtime.py">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/capture-comparator-runtime.py:7">
P2: Importing NumPy before disabling bytecode writes can modify the reference environment and contradict the read-only inventory. Set `sys.dont_write_bytecode = True` before importing NumPy.</violation>
<violation number="2" location="harness/qualification/deepseek4/ds4v-vision/capture-comparator-runtime.py:9">
P2: With `python -O`, this assertion is removed, so the script can inventory a non-isolated NumPy installation and emit pins for it. Raise an explicit exception when `sys.flags.isolated` is false.</violation>
<violation number="3" location="harness/qualification/deepseek4/ds4v-vision/capture-comparator-runtime.py:11">
P2: The manifest omits the Python standard-library files used by `compare.py`, so changing `/usr/lib/python3.12` without changing the executable can pass the runtime pin. Add the standard-library tree and its extension modules to `files`.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/hip-attention-qualification.py">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/hip-attention-qualification.py:65">
P2: When SIGINT or SIGTERM interrupts a lane, the broad exception handler converts it into exit 3. Catch `InterruptedError` separately and return `128 + signal` after cleanup.</violation>
<violation number="2" location="harness/qualification/deepseek4/ds4v-vision/hip-attention-qualification.py:69">
P2: If the external guard hangs, `guarded_run` waits forever because `child.wait()` has no deadline. Add a bounded supervisor timeout and guaranteed cleanup of the guard and its owned worker.</violation>
</file>
<file name="server/src/common/backend_factory.cpp">
<violation number="1" location="server/src/common/backend_factory.cpp:221">
P2: When `--mmproj` is used without sparse prefill and in-process distinct local HIP expert owners, this gate accepts the launch but `DeepSeek4Backend::load_model()` immediately rejects it. Validate the complete DS4V placement contract during factory admission, or centralize that validation so invalid image launches fail before backend construction with the correct reason.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/hip-norm-qualification.py">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/hip-norm-qualification.py:62">
P2: The guard inherits the caller's dynamic-loader and GPU/runtime environment, so pinned binaries do not guarantee the qualified process is the one being tested. Launch it with the same fixed clean environment used by the qualification harness, removing inherited loader, device-mask, and GGML/DS4/DFLASH variables.</violation>
<violation number="2" location="harness/qualification/deepseek4/ds4v-vision/hip-norm-qualification.py:69">
P2: A hung supervisor can wedge the qualification indefinitely and prevent the final report. Bound each lane wait and use the existing termination/reaping path when the timeout expires.</violation>
<violation number="3" location="harness/qualification/deepseek4/ds4v-vision/hip-norm-qualification.py:210">
P2: Preflight failures occur before the exception handler, so a stale or missing prerequisite produces a traceback and no `summary.json`. Wrap preflight and evidence creation in the same reporting failure path so every qualification failure emits structured evidence.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/patch-bias-diagnostic.py">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/patch-bias-diagnostic.py:40">
P2: This diagnostic does not perform the exact dyadic calculation its description promises, so it cannot establish tie-rounding behavior independently of Torch's CPU kernels. Implement an exact reference calculation or remove the claim before using this report to explain native differences.</violation>
<violation number="2" location="harness/qualification/deepseek4/ds4v-vision/patch-bias-diagnostic.py:47">
P2: If a reference patch file is stale or corrupted, this script consumes it without checking the manifest hash and can misattribute differences to rounding. Verify `meta['sha256']` before loading the patch.</violation>
<violation number="3" location="harness/qualification/deepseek4/ds4v-vision/patch-bias-diagnostic.py:56">
P2: When native output already matches the CPU reference, `np.flatnonzero(mismatch)[0]` raises `IndexError`, so the diagnostic cannot emit its success report. Record first-difference fields as null when `mismatch` is empty.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/source-rocm-reference/cpu-runtime-info.py">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/source-rocm-reference/cpu-runtime-info.py:11">
P2: Running this receipt with `python -O` or `PYTHONOPTIMIZE` strips the environment and provenance guards, allowing mismatched device masks, MIOpen, or Torch versions to be reported as valid. Replace these validation assertions with explicit checks that raise errors.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/source-rocm-reference/hip-control-initial-failure.md">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/source-rocm-reference/hip-control-initial-failure.md:17">
P2: This sentence is stale after the corrected follow-up landed: the repository now contains completed source-HIP/CPU and native-HIP/source-HIP comparisons. Describe the unavailability as applying to this initial attempt and link `hip-control-report.md` so readers do not mistake the initial failure for the final status.</violation>
</file>
<file name="harness/qualification/deepseek4/ds4v-vision/target-hip-qualification-policy-review.md">
<violation number="1" location="harness/qualification/deepseek4/ds4v-vision/target-hip-qualification-policy-review.md:36">
P2: The review calls fixture breadth the main scientific limitation, but the frozen source-HIP reference already fails the unchanged CPU feature gate. State that matching it cannot rule out a shared backend defect.</violation>
</file>
Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.
Re-trigger cubic
| root = home / 'lucebox-ds4v-runtime' | ||
| build = Path('/tmp/ds4v-runtime-hip-build') | ||
| binary = build / 'ds4v_vision_probe' | ||
| reference = home / 'lucebox-ds4v-mix-fix/artifacts/vision-reference' |
Contributor
There was a problem hiding this comment.
P1: The harness gates HIP output against the CPU fixtures instead of the frozen same-GPU source reference. Point reference at ~/ds4v-work/source-rocm210-reference/source-hip-reference and pin its 677b5ef0... manifest; retain the CPU manifest only for a separate portability comparison.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/qualification/deepseek4/ds4v-vision/hip-qualification.sh, line 19:
<comment>The harness gates HIP output against the CPU fixtures instead of the frozen same-GPU source reference. Point `reference` at `~/ds4v-work/source-rocm210-reference/source-hip-reference` and pin its `677b5ef0...` manifest; retain the CPU manifest only for a separate portability comparison.</comment>
<file context>
@@ -0,0 +1,226 @@
+root = home / 'lucebox-ds4v-runtime'
+build = Path('/tmp/ds4v-runtime-hip-build')
+binary = build / 'ds4v_vision_probe'
+reference = home / 'lucebox-ds4v-mix-fix/artifacts/vision-reference'
+mmproj = home / 'ds4v-work/ds4v-mmproj.gguf'
+compare = root / 'server/tools/ds4v_vision/compare.py'
</file context>
| args = parser.parse_args() | ||
| require(sys.flags.isolated and args.parent_radeon_window_released, 'explicit numerical release required') | ||
| require(digest(args.config) == args.config_sha, 'config changed') | ||
| cfg = json.loads(args.config.read_text()) |
Contributor
There was a problem hiding this comment.
P1: The adapter hash is only recorded, so a modified qualification script can change the acceptance logic while every external artifact pin still passes. Require the configured reviewed adapter hash to equal digest(__file__) before using the configuration.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/qualification/deepseek4/ds4v-vision/hip-lt-concurrent-qualification.py, line 82:
<comment>The adapter hash is only recorded, so a modified qualification script can change the acceptance logic while every external artifact pin still passes. Require the configured reviewed adapter hash to equal `digest(__file__)` before using the configuration.</comment>
<file context>
@@ -0,0 +1,230 @@
+ args = parser.parse_args()
+ require(sys.flags.isolated and args.parent_radeon_window_released, 'explicit numerical release required')
+ require(digest(args.config) == args.config_sha, 'config changed')
+ cfg = json.loads(args.config.read_text())
+ linear_path = Path(cfg['linear_receipt']['path'])
+ require(digest(linear_path) == cfg['linear_receipt']['sha256'], 'linear acceptance receipt changed')
</file context>
| parser.add_argument('--parent-radeon-window-released', action='store_true') | ||
| args = parser.parse_args() | ||
| require(sys.flags.isolated and args.parent_radeon_window_released, 'explicit numerical release required') | ||
| require(digest(args.config) == args.config_sha, 'config changed') |
Contributor
There was a problem hiding this comment.
P1: The config and guard hashes are not anchored to a reviewed value, so a modified guard can fabricate lane reports and bypass this numerical qualification. Verify the config and guard/policy artifacts against a trusted release receipt or hard-coded approved digests, rather than accepting a digest supplied alongside the config.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/qualification/deepseek4/ds4v-vision/hip-lt-retry-qualification.py, line 81:
<comment>The config and guard hashes are not anchored to a reviewed value, so a modified guard can fabricate lane reports and bypass this numerical qualification. Verify the config and guard/policy artifacts against a trusted release receipt or hard-coded approved digests, rather than accepting a digest supplied alongside the config.</comment>
<file context>
@@ -0,0 +1,243 @@
+ parser.add_argument('--parent-radeon-window-released', action='store_true')
+ args = parser.parse_args()
+ require(sys.flags.isolated and args.parent_radeon_window_released, 'explicit numerical release required')
+ require(digest(args.config) == args.config_sha, 'config changed')
+ cfg = json.loads(args.config.read_text())
+ require(cfg.get('adapter_sha256') == digest(__file__), 'qualification adapter changed')
</file context>
| assert sha(manifest_bytes) == '38d12f30b9a10ed2f0e99bf4d40a07357ab5e8c4880151c58cb90414e4a53f4f' | ||
| manifest = json.loads(manifest_bytes) | ||
| source_report = json.loads((source/'report.json').read_text()) | ||
| assert source_report['script_sha256'] == 'cd34fd3df07963bea9014ccc6f405ec95400a7b8b70e4233fe5d3ead7dc9468c' |
Contributor
There was a problem hiding this comment.
P1: When source/report.json or its output files are replaced, this check still accepts the replacement because the report supplies both the output path and its expected hash. The three-way result can therefore compare native output against a substituted source tensor instead of the frozen reference; pin the two frozen source output hashes (or verify a pinned source manifest/report digest) before reading the source files.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/qualification/deepseek4/ds4v-vision/source-rocm-reference/compare-corn-three-way.py, line 20:
<comment>When `source/report.json` or its output files are replaced, this check still accepts the replacement because the report supplies both the output path and its expected hash. The three-way result can therefore compare native output against a substituted source tensor instead of the frozen reference; pin the two frozen source output hashes (or verify a pinned source manifest/report digest) before reading the source files.</comment>
<file context>
@@ -0,0 +1,55 @@
+assert sha(manifest_bytes) == '38d12f30b9a10ed2f0e99bf4d40a07357ab5e8c4880151c58cb90414e4a53f4f'
+manifest = json.loads(manifest_bytes)
+source_report = json.loads((source/'report.json').read_text())
+assert source_report['script_sha256'] == 'cd34fd3df07963bea9014ccc6f405ec95400a7b8b70e4233fe5d3ead7dc9468c'
+assert source_report['device']['name'] == 'Radeon RX 7900 XT'
+assert source_report['device']['gcn_arch'].split(':')[0] == 'gfx1100'
</file context>
| str(out / f'{label}-embeddings.f32'): ((h+2)//3)*((w+2)//3)*4096*4} | ||
| require(set(p['required_outputs']) == set(sizes), 'wrong output contract') | ||
| require(all(p['required_outputs'][k]['bytes'] == v for k, v in sizes.items()), 'wrong output dimensions') | ||
| policies.append((p, out, label)) |
Contributor
There was a problem hiding this comment.
P1: When two lane policies use the same run_name, their executions share out and the later copies cannot represent independent lanes. Reject duplicate output directories before launching any lane.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/qualification/deepseek4/ds4v-vision/hip-lt-retry-qualification.py, line 162:
<comment>When two lane policies use the same `run_name`, their executions share `out` and the later copies cannot represent independent lanes. Reject duplicate output directories before launching any lane.</comment>
<file context>
@@ -0,0 +1,243 @@
+ str(out / f'{label}-embeddings.f32'): ((h+2)//3)*((w+2)//3)*4096*4}
+ require(set(p['required_outputs']) == set(sizes), 'wrong output contract')
+ require(all(p['required_outputs'][k]['bytes'] == v for k, v in sizes.items()), 'wrong output dimensions')
+ policies.append((p, out, label))
+ evidence = Path(cfg['evidence'])
+ require(evidence.is_absolute() and not evidence.exists(), 'fresh evidence required')
</file context>
| a.output.mkdir(parents=True,exist_ok=True) | ||
| for label in ('corn','carrots'): | ||
| meta=manifest['images'][label]['patches'] | ||
| patches=torch.from_numpy(np.fromfile(a.reference/meta['file'],np.float32).reshape(meta['shape'])).to(torch.bfloat16) |
Contributor
There was a problem hiding this comment.
P2: If a reference patch file is stale or corrupted, this script consumes it without checking the manifest hash and can misattribute differences to rounding. Verify meta['sha256'] before loading the patch.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/qualification/deepseek4/ds4v-vision/patch-bias-diagnostic.py, line 47:
<comment>If a reference patch file is stale or corrupted, this script consumes it without checking the manifest hash and can misattribute differences to rounding. Verify `meta['sha256']` before loading the patch.</comment>
<file context>
@@ -0,0 +1,58 @@
+a.output.mkdir(parents=True,exist_ok=True)
+for label in ('corn','carrots'):
+ meta=manifest['images'][label]['patches']
+ patches=torch.from_numpy(np.fromfile(a.reference/meta['file'],np.float32).reshape(meta['shape'])).to(torch.bfloat16)
+ with torch.inference_mode():
+ fused=module(patches)
</file context>
| child = subprocess.Popen(command, stdout=log, stderr=subprocess.STDOUT) | ||
| previous = {} | ||
| def interrupted(signum, frame): | ||
| raise InterruptedError(f'qualification interrupted: {signum}') |
Contributor
There was a problem hiding this comment.
P2: When SIGINT or SIGTERM interrupts a lane, the broad exception handler converts it into exit 3. Catch InterruptedError separately and return 128 + signal after cleanup.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/qualification/deepseek4/ds4v-vision/hip-attention-qualification.py, line 65:
<comment>When SIGINT or SIGTERM interrupts a lane, the broad exception handler converts it into exit 3. Catch `InterruptedError` separately and return `128 + signal` after cleanup.</comment>
<file context>
@@ -0,0 +1,314 @@
+ child = subprocess.Popen(command, stdout=log, stderr=subprocess.STDOUT)
+ previous = {}
+ def interrupted(signum, frame):
+ raise InterruptedError(f'qualification interrupted: {signum}')
+ try:
+ for signum in (signal.SIGTERM, signal.SIGINT):
</file context>
| import torch | ||
| from safetensors import safe_open | ||
|
|
||
| torch.set_num_threads(2) |
Contributor
There was a problem hiding this comment.
P2: This limits only intra-op parallelism and leaves inter-op scheduling at its default, so fixture generation can oversubscribe CPUs and differ from the controlled two-thread reference. Set both Torch thread pools to two before importing the model.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/qualification/deepseek4/ds4v-vision/reference-fixtures.py, line 17:
<comment>This limits only intra-op parallelism and leaves inter-op scheduling at its default, so fixture generation can oversubscribe CPUs and differ from the controlled two-thread reference. Set both Torch thread pools to two before importing the model.</comment>
<file context>
@@ -0,0 +1,65 @@
+import torch
+from safetensors import safe_open
+
+torch.set_num_threads(2)
+torch.set_default_dtype(torch.bfloat16)
+sys.path.insert(0, str(args.source / 'inference'))
</file context>
Suggested change
| torch.set_num_threads(2) | |
| torch.set_num_threads(2) | |
| torch.set_num_interop_threads(2) |
Comment on lines
+36
to
+39
| Those separate gates remain required. The main residual scientific limitation is | ||
| fixture breadth: two images establish the selected deployment gate, not general | ||
| cross-image or cross-backend equivalence. No tolerance, source implementation or | ||
| reference backend may be changed after seeing corrected candidate results. |
Contributor
There was a problem hiding this comment.
P2: The review calls fixture breadth the main scientific limitation, but the frozen source-HIP reference already fails the unchanged CPU feature gate. State that matching it cannot rule out a shared backend defect.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/qualification/deepseek4/ds4v-vision/target-hip-qualification-policy-review.md, line 36:
<comment>The review calls fixture breadth the main scientific limitation, but the frozen source-HIP reference already fails the unchanged CPU feature gate. State that matching it cannot rule out a shared backend defect.</comment>
<file context>
@@ -0,0 +1,42 @@
+Any PASS is limited to the native tower on the recorded 7900 XT/software tuple.
+It does not qualify CPU numerics, gfx1151, other accelerators, decoder behavior,
+HTTP image input, image-dependent answers, isolation, resource limits or cleanup.
+Those separate gates remain required. The main residual scientific limitation is
+fixture breadth: two images establish the selected deployment gate, not general
+cross-image or cross-backend equivalence. No tolerance, source implementation or
</file context>
Suggested change
| Those separate gates remain required. The main residual scientific limitation is | |
| fixture breadth: two images establish the selected deployment gate, not general | |
| cross-image or cross-backend equivalence. No tolerance, source implementation or | |
| reference backend may be changed after seeing corrected candidate results. | |
| Those separate gates remain required. Scientific limitations include the source-HIP | |
| reference's unchanged CPU feature failure, so matching it cannot rule out a shared | |
| backend defect, and fixture breadth: two images establish the selected deployment | |
| gate, not general cross-image or cross-backend equivalence. No tolerance, source implementation or | |
| reference backend may be changed after seeing corrected candidate results. |
| import pathlib | ||
| import subprocess | ||
| import sys | ||
| import numpy |
Contributor
There was a problem hiding this comment.
P2: Importing NumPy before disabling bytecode writes can modify the reference environment and contradict the read-only inventory. Set sys.dont_write_bytecode = True before importing NumPy.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/qualification/deepseek4/ds4v-vision/capture-comparator-runtime.py, line 7:
<comment>Importing NumPy before disabling bytecode writes can modify the reference environment and contradict the read-only inventory. Set `sys.dont_write_bytecode = True` before importing NumPy.</comment>
<file context>
@@ -0,0 +1,27 @@
+import pathlib
+import subprocess
+import sys
+import numpy
+
+assert sys.flags.isolated
</file context>
Suggested change
| import numpy | |
| sys.dont_write_bytecode = True | |
| import numpy |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
DeepSeek V4 Flash Vision serving on the Strix Halo + 7900 XT pair, plus a bounded IQ85 converter for the abliterated parent. 83 commits, 135 files, +14651/−59.
Vision serving
03edb38), with image policy, transport, preprocess and prompt-ownership units and a CPU composition proof.export_ds4v_mmproj.py) verified lossless: 267 tensors, 932786176 payload bytes compared against the parent by an independent reader.Serving memory, prefill and decode
Text correctness
4bf7270.Tooling
server/tools/ds4v_vision/probes and contracts,ds4v_preprocess_probe, and--recipe iq85inds4_mix_converter: IQ2_XXS gate/up, IQ2_XS down, Q8_0 for dense text matrices, vision/aligner/router/indexer/norms/biases preserved, reading the original safetensors rather than a previously quantized GGUF. Newtest_ds4_iq_converter.Verification status — gates unchanged, nothing here is a quality claim
7071946, binaryc32e5ae3); short decode 15.9/16.4/17.4 t/s.0.99822935, first native HIP run0.9859(exit 3). Tower A is the experimental base; the numerical verdict is still ISSUES and runtime integration stays blocked.954433dc…), but the frozen quality gate failed (12/16 content, 5/16 strict) and median decode was 20.1 t/s against a 35 t/s target. No candidate was installed; the original service was restored.Full receipts:
artifacts/ds4v-quant85/README.md,docs/ds4v-continuation-status.md,decisions-ds4v-continuation.tsv.Notes
Binary trial evidence (requests, responses, journals) is retained outside the repository. The converter path is experimental and, per the evidence above, does not yet meet the target quality or speed.
Upstream sync
upstream/main(2f0eff05) is merged into the branch (6d762d8e) so the PR applies cleanly again. Conflict resolutions:GGML_OP_DS4_MOE_COMBINE);GGML_OP_COUNT105 → 110 in the enum and in bothggml.ctables.compute_ds4_hybrid_budget_infonow takes bothwith_visionandpaged; it charges paged KV on the primary target and still reserves the vision scratch allowance viavision::remaining_expert_budget.build_mla_attention_lane_coretakes upstream's prepared lane andout_attn_contextalongside the image span view; the contiguous wrapper forwards image spans so existing callers keep working.deepseek4_snapshot.cpp; this branch's inline copies were byte-identical to the merge base, so they were dropped anddeepseek4_release_image_scratchwas kept.Local checks on macOS with clang (C++17, HIP headers stubbed — Apple has no HIP):
ggml.ccompiles clean and the op enum plus both tables contain 110 entries;deepseek4_graph.cpp,deepseek4_backend.cpp,deepseek4_snapshot.cpp,deepseek4_paged_cache.cpp, the image units andtest_deepseek4_unit.cppall pass-fsyntax-only. No link or GPU-run verification was possible here. Upstream CI runs for this head are inaction_required(maintainer approval pending), so the CUDA build has not executed yet.