Skip to content

Commit 48cca73

Browse files
committed
Professionalize the repository and cut v1.0.0
Splits the detection engine out of the UI, puts a test suite and CI behind it, adds the community-health files the project was missing, and regenerates the README figures from a script instead of by hand. Engine/UI split - detector.js: the heuristics, evidence and scoring, as pure functions with no DOM, storage or network. A guarded CommonJS footer means Node can require() it while the browser keeps loading a plain script, so there is still no build step. - script.js is now only the UI layer. aggregate() no longer reaches into the DOM through a default argument; readThresholds() reads the inputs and the engine's normaliser stays pure. Tests (217 assertions, zero dependencies, Node's built-in runner) - Every heuristic twice: a fixture it must fire on and one it must stay quiet on. A registered heuristic with no fixture pair fails the build. - Scoring arithmetic, verdict band edges, threshold normalisation, the shared string helpers, and degenerate input. - The gallery checked against the live engine, including the expectedSignals self-test that examples.js has always advertised and nothing ever ran. - The repository itself: version consistency across package.json, the masthead and the changelog; docs links resolving; no orphaned or missing screenshot; no third-party origin in the markup. Bugs the tests found - Threshold normalisation clamped to 0..100 before ordering mixed below ai, so ordering could push a value back out of range: Mixed>=100 with AI>=0 gave { mixed: -1, ai: 101 }, and with nothing able to reach 101 every snippet came back "Mixed signals". Both ends now stay in range. - A line beginning with * counted as a comment, so an operator continuation ("const area = width" / " * height;") read as a comment and skewed both comment-density signals. Block comments are now tracked by scanning for delimiters anywhere on the line, including mid-line opens, which is what let the * shortcut go. JSDoc bodies are still counted. - A raw NUL byte in the evidence de-duplication key made script.js register as binary to grep, file(1) and most editors. - Three gallery annotations cited signals the detector does not fire on those samples, one of them claiming "30+ lines of code" for a sample with 21 non-empty lines, below that signal's 25-line gate. Annotations can now carry scored:false to describe a tell the engine doesn't score; the card renders a "not scored" marker so a reader isn't told otherwise. Tooling - scripts/screenshots.mjs regenerates every README figure at a fixed viewport with fonts loaded, transitions frozen and localStorage cleared. - scripts/browser.mjs resolves a Chromium instead of insisting on the exact revision Playwright pins, which two scripts that only read colours and take pictures do not need. - CI: the suite on Node 20/22/24 with no install, the WCAG AA contrast audit in both themes, and a docs/parse hygiene job. Docs - LICENSE, which the README has claimed for a while with no file behind it. - CONTRIBUTING, CODE_OF_CONDUCT, SECURITY, CHANGELOG, .editorconfig, issue and PR templates, Dependabot. - README rebuilt with the regenerated figures and a "Known limitations" section recording two measured calibration gaps: the count-gated signals make any large file look generated (this repo's own script.js scores 40.3%, above six of eight known-generated gallery samples), and nothing in the gallery reaches the default 50% AI threshold. Both are documented rather than tuned, since changing either moves every verdict. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MTEQELhDNNSBMGL9eyZqFk
1 parent c2bf1cd commit 48cca73

43 files changed

Lines changed: 4175 additions & 907 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.editorconfig

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Indentation is a signal this project literally scores, so keeping it
2+
# consistent is not just tidiness — a stray tab changes what the detector says
3+
# about its own source.
4+
root = true
5+
6+
[*]
7+
charset = utf-8
8+
end_of_line = lf
9+
insert_final_newline = true
10+
trim_trailing_whitespace = true
11+
indent_style = space
12+
indent_size = 4
13+
14+
# The gallery data is dense and deeply nested; 2 keeps entries readable.
15+
[examples.js]
16+
indent_size = 2
17+
18+
[*.{json,yml,yaml}]
19+
indent_size = 2
20+
21+
[*.md]
22+
# Two trailing spaces are a hard line break in Markdown.
23+
trim_trailing_whitespace = false
24+
25+
[LICENSE]
26+
indent_style = unset
27+
indent_size = unset
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
name: Bug report
2+
description: Something in the app is broken — the UI, the editor, the gallery, the keyboard handling
3+
labels: ['bug']
4+
body:
5+
- type: markdown
6+
attributes:
7+
value: |
8+
For a wrong *verdict*, use the "False positive or false negative"
9+
template instead — those are calibration, not breakage.
10+
11+
- type: textarea
12+
id: what
13+
attributes:
14+
label: What happens
15+
validations:
16+
required: true
17+
18+
- type: textarea
19+
id: steps
20+
attributes:
21+
label: Steps to reproduce
22+
placeholder: |
23+
1. Open index.html
24+
2. Pick "Over-commented FizzBuzz" from the example picker
25+
3. ...
26+
validations:
27+
required: true
28+
29+
- type: textarea
30+
id: expected
31+
attributes:
32+
label: What you expected
33+
validations:
34+
required: true
35+
36+
- type: input
37+
id: browser
38+
attributes:
39+
label: Browser and version
40+
placeholder: 'Firefox 141 / Safari 18.2 / Chrome 139'
41+
validations:
42+
required: true
43+
44+
- type: dropdown
45+
id: theme
46+
attributes:
47+
label: Theme
48+
options:
49+
- Light
50+
- Dark
51+
- Both
52+
- Not relevant
53+
54+
- type: dropdown
55+
id: served
56+
attributes:
57+
label: How were you running it?
58+
options:
59+
- Opened index.html from the filesystem
60+
- Served over http (npm run serve, or another static server)
61+
- A deployed copy
62+
63+
- type: textarea
64+
id: console
65+
attributes:
66+
label: Anything in the browser console
67+
render: text

.github/ISSUE_TEMPLATE/config.yml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
blank_issues_enabled: true
2+
contact_links:
3+
- name: Security vulnerability
4+
url: https://github.com/Gabriel-Dalton/AI-Code-Detector/security/advisories/new
5+
about: Report privately via a security advisory, not a public issue. See SECURITY.md.
6+
- name: Contributing guide
7+
url: https://github.com/Gabriel-Dalton/AI-Code-Detector/blob/main/CONTRIBUTING.md
8+
about: How the code is laid out, and what adding a heuristic or a gallery sample involves.
Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
name: False positive or false negative
2+
description: The detector got a verdict wrong on a snippet
3+
title: 'Verdict: '
4+
labels: ['calibration']
5+
body:
6+
- type: markdown
7+
attributes:
8+
value: |
9+
This is the most useful report the project can get. The heuristics are
10+
published in `detector.js`, so a concrete snippet that fools them is
11+
directly actionable.
12+
13+
Before filing: the detector is calibrated for **snippets, not whole
14+
files**. Several signals are gated on absolute counts, so pasting a
15+
600-line file will light up signals for reasons unrelated to who wrote
16+
it. That's a known limitation already recorded in the README — no need
17+
to file it again unless you have a fix in mind.
18+
19+
- type: dropdown
20+
id: direction
21+
attributes:
22+
label: Which way was it wrong?
23+
options:
24+
- Human-written code was scored as AI
25+
- AI-generated code was scored as human
26+
- A specific signal fired when it shouldn't
27+
- A specific signal didn't fire when it should
28+
validations:
29+
required: true
30+
31+
- type: textarea
32+
id: snippet
33+
attributes:
34+
label: The snippet
35+
description: The smallest version that still reproduces it.
36+
render: text
37+
validations:
38+
required: true
39+
40+
- type: input
41+
id: signal
42+
attributes:
43+
label: Which signal
44+
description: The id from the Heuristics tab, e.g. `over_commenting_trivial_ops`. Leave blank if it's about the overall score.
45+
46+
- type: input
47+
id: score
48+
attributes:
49+
label: Score and verdict shown
50+
placeholder: '40.9% — Mixed signals'
51+
52+
- type: textarea
53+
id: expected
54+
attributes:
55+
label: What you expected instead, and why
56+
validations:
57+
required: true
58+
59+
- type: input
60+
id: provenance
61+
attributes:
62+
label: Where the code came from
63+
description: Helps a lot for the AI-scored-as-human direction — which model, and was it edited afterwards?
64+
placeholder: 'Hand-written in 2019 / GPT-4 output, unedited / Copilot completion I then rewrote'
Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
name: Propose a heuristic
2+
description: A tell the detector doesn't look for yet
3+
title: 'Heuristic: '
4+
labels: ['heuristic']
5+
body:
6+
- type: markdown
7+
attributes:
8+
value: |
9+
A good heuristic is one that fires on generated code and *stays quiet*
10+
on the human equivalent. The second half is the hard part and the part
11+
worth thinking about before filing — a signal that fires on everything
12+
adds noise to every verdict, since the score is normalised over the sum
13+
of all weights.
14+
15+
- type: input
16+
id: name
17+
attributes:
18+
label: What would you call it
19+
placeholder: 'Exhaustive enum switch with an unreachable default'
20+
validations:
21+
required: true
22+
23+
- type: textarea
24+
id: fires
25+
attributes:
26+
label: Code it should fire on
27+
render: text
28+
validations:
29+
required: true
30+
31+
- type: textarea
32+
id: quiet
33+
attributes:
34+
label: Similar code it should NOT fire on
35+
description: The same shape, written by a human, without the tell.
36+
render: text
37+
validations:
38+
required: true
39+
40+
- type: textarea
41+
id: why
42+
attributes:
43+
label: Why this is a tell
44+
description: What is it about how models generate code that produces this?
45+
validations:
46+
required: true
47+
48+
- type: input
49+
id: weight
50+
attributes:
51+
label: Suggested weight
52+
description: Existing weights run 0.4 to 2.0 for AI tells; negative means it's a human signal. Roughly, how strongly does it point?
53+
placeholder: '1.2'
54+
55+
- type: input
56+
id: languages
57+
attributes:
58+
label: Which languages does it apply to
59+
placeholder: 'JS/TS only / any C-family / Python'
60+
61+
- type: checkboxes
62+
id: checked
63+
attributes:
64+
label: Before filing
65+
options:
66+
- label: I checked the Heuristics tab and this isn't already covered
67+
required: true
68+
- label: My "should not fire" example is genuinely code a person would write, not a strawman
69+
required: true

.github/dependabot.yml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
version: 2
2+
3+
# Only dev tooling is versioned here — the app itself has no runtime
4+
# dependencies. Monthly is right for two dev packages; weekly would be noise.
5+
updates:
6+
- package-ecosystem: npm
7+
directory: '/'
8+
schedule:
9+
interval: monthly
10+
open-pull-requests-limit: 3
11+
commit-message:
12+
prefix: 'deps'
13+
groups:
14+
dev-tooling:
15+
patterns: ['*']
16+
17+
- package-ecosystem: github-actions
18+
directory: '/'
19+
schedule:
20+
interval: monthly
21+
commit-message:
22+
prefix: 'ci'

.github/pull_request_template.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
<!--
2+
Thanks for contributing. Keep this short — the diff is the record. Delete any
3+
section that doesn't apply rather than filling it with "n/a".
4+
-->
5+
6+
## What this changes
7+
8+
<!-- One or two sentences. What is different afterwards? -->
9+
10+
## Why
11+
12+
<!-- The problem, not the patch. If it fixes an issue, link it: Fixes #123 -->
13+
14+
## How I know it works
15+
16+
<!--
17+
For a heuristic change, the useful thing is the snippet: what fires now that
18+
didn't, or what stopped firing that shouldn't have.
19+
-->
20+
21+
- [ ] `npm test` passes
22+
- [ ] `npm run audit:contrast` passes (only needed if CSS or markup changed)
23+
- [ ] Added or updated a fixture pair for any heuristic I touched
24+
- [ ] Regenerated the README figures with `npm run screenshots` (only if the UI
25+
looks different)
26+
27+
## What I'd expect to break if I got this wrong
28+
29+
<!--
30+
Genuinely the most useful line in this template. It tells a reviewer where to
31+
look, and writing it often finds the bug before review does.
32+
-->

.github/workflows/ci.yml

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: ['**']
6+
pull_request:
7+
workflow_dispatch:
8+
9+
# A new push to the same branch makes the in-flight run obsolete.
10+
concurrency:
11+
group: ${{ github.workflow }}-${{ github.ref }}
12+
cancel-in-progress: true
13+
14+
permissions:
15+
contents: read
16+
17+
jobs:
18+
test:
19+
name: test (node ${{ matrix.node }})
20+
runs-on: ubuntu-latest
21+
strategy:
22+
fail-fast: false
23+
matrix:
24+
# 20 is the engines floor, 24 is current. If the suite ever needs a
25+
# dependency to run, that is a regression worth noticing here.
26+
node: ['20', '22', '24']
27+
steps:
28+
- uses: actions/checkout@v4
29+
30+
- uses: actions/setup-node@v4
31+
with:
32+
node-version: ${{ matrix.node }}
33+
34+
# Deliberately no `npm ci`. The engine and its tests have zero
35+
# dependencies, and running them on a bare checkout is the check that
36+
# keeps it that way.
37+
- name: Run the test suite
38+
run: node --test
39+
40+
a11y:
41+
name: WCAG AA contrast
42+
runs-on: ubuntu-latest
43+
steps:
44+
- uses: actions/checkout@v4
45+
46+
- uses: actions/setup-node@v4
47+
with:
48+
node-version: '22'
49+
cache: npm
50+
51+
- run: npm ci
52+
53+
- name: Install Chromium
54+
run: npx playwright install --with-deps chromium
55+
56+
# Polled with curl rather than a wait-for-it package: adding a dependency
57+
# to wait for a server is how a zero-dependency project stops being one.
58+
- name: Serve the page
59+
run: |
60+
npm run serve &
61+
for _ in $(seq 1 30); do
62+
curl -sfo /dev/null http://127.0.0.1:8899/index.html && exit 0
63+
sleep 1
64+
done
65+
echo "static server did not come up on :8899" >&2
66+
exit 1
67+
68+
# Renders the real page in both themes across six interaction states and
69+
# exits non-zero on any AA failure.
70+
- name: Audit contrast
71+
run: npm run audit:contrast
72+
73+
hygiene:
74+
name: repository hygiene
75+
runs-on: ubuntu-latest
76+
steps:
77+
- uses: actions/checkout@v4
78+
79+
- uses: actions/setup-node@v4
80+
with:
81+
node-version: '22'
82+
83+
# Every source file has to parse as the kind of script index.html loads
84+
# it as. Cheap, and it catches a bad merge before the tests do.
85+
- name: Check that every script parses
86+
run: |
87+
for f in detector.js script.js examples.js scripts/*.mjs test/*.mjs; do
88+
node --check "$f" || exit 1
89+
done
90+
91+
# Version consistency across package.json, the masthead and the changelog,
92+
# plus every relative link in the docs actually resolving.
93+
- name: Check the docs
94+
run: node --test test/docs.test.mjs

0 commit comments

Comments
 (0)