Skip to content

README.md

Latest commit

 

History

History
140 lines (96 loc) · 6.09 KB

README.md

File metadata and controls

140 lines (96 loc) · 6.09 KB
Raw

humanizer-metrics (optional CLI)

A small, zero-dependency Node tool that computes the writing signals the skill talks about, so you can score docs deterministically and gate them in CI.

The pure-Markdown skill in ../skills/humanizer/SKILL.md needs none of this. This directory is a separate, optional layer for power users and teams. Nothing here runs unless you run it.

What it computes

For any text it reports:

  • Burstiness (coefficient of variation of sentence length) and Goh-Barabasi burstiness
  • Type-token ratio and MATTR (length-stable lexical diversity)
  • Trigram repetition (the treadmill signal, P43)
  • AI-vocabulary tell density (tiered blacklist, mirrors P7 and the tiered vocab in the skill)
  • Flesch-Kincaid grade and reading ease
  • A transparent weighted 0-100 AI-tell score (higher = more AI smell)

The score is a deterministic proxy: 0.28 low-burstiness + 0.18 low-diversity + 0.14 high-repetition + 0.40 lexical-tells. It is not a trained detector and does not call any network. The /humanizer skill's LLM-estimated score is more holistic; this one is fast and reproducible. Use them together.

Requirements

Node 18 or newer. No npm install needed.

Usage

# Score one file (or - for stdin)
node cli/index.js score README.md
cat draft.md | node cli/index.js score -

# Ignore code fences and quoted examples so they don't inflate the score
node cli/index.js score docs/api.md --ignore-code --ignore-quotes

# Scan a whole tree, or a list of files
node cli/index.js scan docs/
node cli/index.js scan README.md docs/faq.md

# Show the before/after delta
node cli/index.js compare --before draft.md --after final.md

# Fail if the rewrite dropped a fact
node cli/index.js compare --before draft.md --after final.md --check-facts

# JSON for tooling
node cli/index.js score README.md --json

Checking that a rewrite kept the facts

--check-facts compares the hard tokens in the two files and exits 1 if anything in --before is missing from --after:

Kind Examples Normalized so that
numbers 2200, 5,400, 12.5 5,400 and 5400 are the same fact
percentages 59%, 12.5 % spacing does not matter
dates 2026-04-16, April 16, 2026, 16 April 2026 all three are one token
versions v0.7.0, 0.7.0 the v prefix does not matter, and dropping it does not read as a loss
URLs https://example.com/a?b=1 trailing sentence punctuation is trimmed, balanced brackets are kept
names AWS, RDS, P18 any run of two or more capitals and digits, minus GitHub alert markers and shouted words

Facts are read from the raw text, so a number inside a code fence still counts even under --ignore-code. A fact the rewrite moved into a link still counts. Adding a fact is never reported, because adding detail is a writing choice.

What it does not catch

This checks that every hard token survived. It is not a proof of factual equivalence.

  • Swapped values. The check is set-based. Turning "7 errors in 2 regions" into "2 errors in 7 regions" keeps both numbers, so it passes. It catches deletions and edits, not reorderings.
  • Facts with no hard token. "most customers" becoming "all customers" changes the claim and contains nothing to extract.
  • Meaning. A number kept in a sentence that now says the opposite still counts as present.

CI quality gate

Fail the build when docs get too AI-flavored, or only when they regress against a saved baseline:

# Hard threshold
node cli/index.js scan docs/ --fail-above 40

# Save a baseline once, then fail only on regressions (existing debt is grandfathered)
node cli/index.js scan docs/ --baseline .humanizer-baseline.json --write-baseline
node cli/index.js scan docs/ --baseline .humanizer-baseline.json --fail-on-regression

Exit codes: 0 pass, 1 gate failed, 2 usage or IO error.

GitHub Action

A reusable composite action lives at .github/actions/humanizer-gate:

- uses: Aboudjem/humanizer-skill/.github/actions/humanizer-gate@main
  with:
    path: docs/
    fail-above: '40'

Pre-commit hook

For a fast local gate (the GitHub Action above is the CI-side backstop), the repo ships a .pre-commit-hooks.yaml for the Python pre-commit framework. It wraps the same zero-dependency CLI, language: script (no install step, just Node), and re-scans the repo on every commit that touches a Markdown or text file.

Add this to your own repo's .pre-commit-config.yaml:

repos:
  - repo: https://github.com/Aboudjem/humanizer-skill
    rev: main  # pin to a tagged release or commit SHA in production; `pre-commit autoupdate` will not update a mutable ref
    hooks:
      - id: humanizer-scan

Then pre-commit install once. The hook fails the commit if any scanned file scores above 40; adjust the threshold by overriding entry in your own config, e.g. entry: cli/index.js scan . --fail-above 60.

Not using the Python pre-commit framework? Husky (the most common JS-native alternative) works too. The CLI is not published to npm today (see below), so vendor it, for example as a git submodule at tools/humanizer-skill, then add one line to .husky/pre-commit:

node tools/humanizer-skill/cli/index.js scan . --fail-above 40

Tests

cd cli && node --test

64 tests, no dependencies. Covers tokenization, every metric, the composite score and its per-signal breakdown, code/quote masking, the vocabulary layer, fact extraction and the fact-loss diff, multi-path scanning, the packaged bin (shebang, executable bit, and that no shipped module requires a file outside the tarball), and CLI exit codes (threshold gate, baseline regression, fact check, bad input).

Running the CLI needs Node 18 or newer, and CI runs the suite on both 18 and 20. Contributing or publishing needs Node 20.19 or newer, because eslint 10 does not run on 18.

Linting

cd cli && npm install && npm run lint

ESLint is a devDependency only, used for local and CI linting; it never ships with the published package and does not affect the CLI's zero-runtime-dependency claim.