Skip to content

Commit 0d2ad5e

Browse files
Chi-Shan0707Chi-Shan0707
committed
Add NLC hypothesis and SSL evolution story
Key additions: - NLC (Natural Language Linearization Confusion) as core framework - Three knot types: branches (16%), loops (32%), calls (12%) - Perfect example: "If left < right" executing both paths simultaneously - SSL v1 failure (loss stalled at 0.432) → v2 success with per-domain encoders - Added threshold values (cond_rev_per1k > 3.0 for heavy branch) - Removed less relevant call-heavy example to focus on strongest evidence Sources: - NAD_Next research on NLC hypothesis - SSL v1 → v2 evolution experiments - Quantitative breakdown of knot types
1 parent 01e4951 commit 0d2ad5e

1 file changed

Lines changed: 24 additions & 23 deletions

File tree

‎demo/index.html‎

Lines changed: 24 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -47,15 +47,15 @@ <h4>📐 Math & Science: Linear Reasoning</h4>
4747
</div>
4848

4949
<div class="motivation-card branching">
50-
<h4>💻 Coding: Branching & State</h4>
51-
<p>Programming involves control flow that natural language cannot represent:</p>
50+
<h4>💻 Coding: The NLC Problem</h4>
51+
<p>Coding involves <strong>Natural Language Linearization Confusion (NLC)</strong>:</p>
5252
<ul>
53-
<li><strong>Loops:</strong> Iterative state updates that linear text flattens</li>
54-
<li><strong>Branches:</strong> Conditional logic with multiple paths</li>
55-
<li><strong>State:</strong> Variable bindings, memory, runtime errors</li>
56-
<li>Confident text ≠ correct code (off-by-one errors, edge cases)</li>
53+
<li><strong>Branches (16% of runs):</strong> If-else paths get flattened—model executes both simultaneously</li>
54+
<li><strong>Loops (32% of runs):</strong> Iteration state lost in linear text</li>
55+
<li><strong>Calls (12% of runs):</strong> Caller/callee frames get mixed</li>
56+
<li>Confident reasoning ≠ correct execution</li>
5757
</ul>
58-
<p style="margin-top: 16px; font-size: 15px;"><strong>Result:</strong> AUROC 0.407 (below random!)</p>
58+
<p style="margin-top: 16px; font-size: 15px;"><strong>Result:</strong> AUROC 0.434 (52% gap from math)</p>
5959
</div>
6060
</div>
6161

@@ -82,28 +82,27 @@ <h2>2. What "Heavy Branching" Looks Like</h2>
8282
</div>
8383

8484
<div class="heavy-branch-grid">
85+
<div class="heavy-branch-card branch-heavy">
86+
<span class="heavy-branch-badge">NLC: SIMULTANEOUS BRANCHES</span>
87+
<h5>The Perfect Example: Executing Both Paths</h5>
88+
<div class="heavy-branch-snippet">"If left < right, we move the pointer. So we check if nums[left] == target. Actually, if left >= right, we return. But we already moved left, so now left < right again..."</div>
89+
<p style="font-size: 13px; margin-top: 12px;"><strong>Pattern:</strong> Model says "If left < right" but then realizes it already moved left—so now left < right again. It's <em>simultaneously executing both branches</em> because linear text can't represent exclusive paths.</p>
90+
</div>
91+
8592
<div class="heavy-branch-card branch-heavy">
8693
<span class="heavy-branch-badge">CONDITIONAL BRANCH</span>
8794
<h5>Heavy Branch: "but note" Syndrome</h5>
88-
<div class="heavy-branch-metric">cond_rev_per1k: 5.40 (highest percentile)</div>
89-
<div class="heavy-branch-snippet">t=6 which is divisible by 3. but note: the condition is "divisible by t". since the product can be 0 (if any digit is 0) then that aut... from n upwards and check the condition for each number. however, note the example: for n=1... igits must be divisible by t. note that if the number contains a 0, then</div>
90-
<p style="font-size: 13px; margin-top: 12px;"><strong>Pattern:</strong> Model repeatedly checks conditions ("but note", "however")—trapped in conditional logic maze.</p>
95+
<div class="heavy-branch-metric">cond_rev_per1k: 5.40 (threshold: >3.0)</div>
96+
<div class="heavy-branch-snippet">t=6 which is divisible by 3. but note: the condition is "divisible by t". since the product can be 0 (if any digit is 0) then that aut... from n upwards and check the condition for each number. however, note the example: for n=1</div>
97+
<p style="font-size: 13px; margin-top: 12px;"><strong>Pattern:</strong> Model trapped in conditional logic maze—constantly checking "but note", "however" because it lost track of which branch it's in.</p>
9198
</div>
9299

93100
<div class="heavy-branch-card loop-heavy">
94101
<span class="heavy-branch-badge">LOOP KNOT</span>
95102
<h5>Heavy Loop: Nested Index Hell</h5>
96103
<div class="heavy-branch-metric">loop_rev_per1k: 1.81</div>
97-
<div class="heavy-branch-snippet">ctions in their reduced form. but that might lead to floating-point issues. another approach: for each pair (i, j) with i < j, store... s) that also have product x, but with indices such that p < q < r < s and the gaps are satisfied. but then, for each pair (p, r), i need to consid... ch q between p+1 and r-1, and for each s between r+1 and the end, check the condition.</div>
98-
<p style="font-size: 13px; margin-top: 12px;"><strong>Pattern:</strong> Model tries to track loop indices (i, j, p, q, r, s) in natural language—state gets lost.</p>
99-
</div>
100-
101-
<div class="heavy-branch-card call-heavy">
102-
<span class="heavy-branch-badge">CALL CONFUSION</span>
103-
<h5>Heavy Call: API Ambiguity</h5>
104-
<div class="heavy-branch-metric">frame_rev_per1k: 0.87</div>
105-
<div class="heavy-branch-snippet">- update the current array 3. return the operation count. but wait, what if after removal, the a... t segment is t[0:1] -> '-' -> but wait, no: the split function in python splits the string a... we can also use the replace function but with caution. actually, we can do: we want to re</div>
106-
<p style="font-size: 13px; margin-top: 12px;"><strong>Pattern:</strong> Model uncertain about function behavior—"but wait" indicates confusion about API semantics.</p>
104+
<div class="heavy-branch-snippet">for each pair (i, j) with i < j, store... s) that also have product x, but with indices such that p < q < r < s and the gaps are satisfied. but then, for each pair (p, r), i need to consid... ch q between p+1 and r-1, and for each s between r+1 and the end</div>
105+
<p style="font-size: 13px; margin-top: 12px;"><strong>Pattern:</strong> Model tries to track 6 loop indices (i, j, p, q, r, s) in natural language—iteration state gets lost.</p>
107106
</div>
108107
</div>
109108

@@ -247,9 +246,11 @@ <h4>Method 1: Feature Sweep (83 Coding-Specific Features)</h4>
247246

248247
<div class="method-card">
249248
<h4>Method 2: SSL Pre-training (42K Unlabeled Traces)</h4>
250-
<p><strong>Approach:</strong> Used self-supervised learning (like BERT) on 42K unlabeled coding traces to learn domain-specific text structure.</p>
251-
<p><strong>Result:</strong> <span class="method-result poor">Full-label ceiling: AUROC 0.454</span></p>
252-
<p><strong>Interpretation:</strong> SSL learns <em>text structure</em> (syntax patterns, common phrases), but code correctness isn't a text-structure problem—it's an execution problem. No amount of representation learning can extract what isn't there.</p>
249+
<p><strong>Approach v1:</strong> Used self-supervised learning on 42K unlabeled coding traces with a shared encoder across all domains.</p>
250+
<p><strong>Result v1:</strong> <span class="method-result poor">Complete failure—loss stalled at 0.432, rank=8 collapsed</span></p>
251+
<p><strong>Approach v2:</strong> Per-domain encoders with enhanced 50D features (including META features) and multi-objective VICReg training.</p>
252+
<p><strong>Result v2:</strong> <span class="method-result neutral">Science +6.7pp, Coding +4.9pp</span> (but only at low labels)</p>
253+
<p><strong>Interpretation:</strong> SSL v1's failure revealed that domains are <em>fundamentally different</em>. A universal text encoder can't work. SSL v2's partial success shows that domain-specific treatment helps, but coding still lags far behind. SSL learns <em>text structure</em>, but code correctness isn't in the structure—it's in the execution.</p>
253254
</div>
254255

255256
<div class="method-card">

0 commit comments

Comments
 (0)