Skip to content

validation.md

Latest commit

 

History

History
183 lines (136 loc) · 7.18 KB

validation.md

File metadata and controls

183 lines (136 loc) · 7.18 KB
Raw

Validation

Eliware Test runs deterministic checks for the selected profiles. It does not publish, deploy, release, or change external systems. It does not run semantic reviews.

Aggregate validation

Run this command from the consumer repository root:

eliware-test

The CLI selects checks from package.json.eliware.apply. The audit stage has a 30-second runtime limit. If it reaches the limit, Eliware Test stops the process and reports a warning. Validation continues, and the timeout does not change the aggregate exit code.

The CLI runs these stages at the same time:

  1. Lint.
  2. Format check.
  3. npm audit.
  4. npm outdated.

The CLI waits until all four stages complete. If any stage fails, the CLI skips the remaining checks and Jest.

If all four stages pass, the CLI runs the enabled package stages and convention checks at the same time:

  1. Package check when npm-published applies.
  2. Typecheck when library applies.
  3. Build when web applies.
  4. All selected convention checks.

The CLI waits until all stages and checks complete. If any stage or check fails, the CLI skips Jest. If all stages and checks pass, the CLI runs Jest and coverage. The CLI reports all failures and returns the highest failure code. The CLI runs Jest without the consumer's npm test script.

Focused validation

Pass one or more test paths to select a focused run:

eliware-test tests/example.test.mjs
eliware-test tests/one.test.mjs tests/two.test.mjs

The CLI runs only the selected Jest tests. It checks lint, format, and applicable layout rules for each selected test and matching source module. It checks coverage for each matching source module. It runs other checks only when their rules can use the selected files. It skips repository-wide checks that cannot use the focused file list.

Pass a Jest option after --:

eliware-test tests/example.test.mjs -- --testNamePattern "case name"

Put each test path before --. Do not pass tool modes with focused paths. The CLI rejects absolute paths, parent traversal, and paths that do not start with tests/. The CLI also rejects options that replace its Jest configuration, reporters, coverage, or test selection.

Application and library profiles require one .test.mjs file for each maintained .mjs source. Each application or library test must contain an executable Jest test. Store each library declaration file in the directory of its matching .mjs source.

Jest and coverage

The CLI uses consumer Jest when available. If not, it uses bundled Jest. It reads supported Jest settings from package.json. It rejects Jest configuration files not in package.json. Configured reporter names must be strings. Reporter option objects are not supported.

Jest uses its default worker pool on each run. The CLI enables the Jest cache in coverage/cache. Pass --detectOpenHandles to find open handles. Jest then runs in-band.

It adds a reporter and coverage reports. It keeps the consumer's required coverage settings. The selected profile rules define required coverage thresholds.

Before each Jest run, the CLI removes earlier reports from the consumer's coverage/ directory. It keeps coverage/cache for later runs. Jest writes coverage reports to that directory. The CLI saves redacted Jest output as plain text in coverage/jest.log. The file does not contain harness progress records. On failure, the CLI shows the failed suite and its assertion or error details.

The package lock check accepts a transitive version outside its parent range when a root override selects that version. It checks that the locked version satisfies the override. It keeps the parent range check when no root override applies.

The CLI leaves coverage/eliware-test.log after each validation run. When Jest runs, the CLI also leaves coverage/jest.log and coverage reports. The CLI writes stage, check, and suite timing to coverage/eliware-test.log.

When validation starts, the CLI prints Logging to ./coverage/eliware-test.log.... When Jest is about to run, the CLI prints Logging to ./coverage/jest.log.... The CLI keeps the timing log while it clears other coverage files. The CLI does not use system temp or user folders for its temporary files.

Each test suite has a ten-second limit. A 15-second watchdog also covers startup and pauses between suites. Output does not change the limits. The CLI requests a graceful stop after a timeout. It forces a stop after one second. The CLI reports an error if the child command does not close.

Read the summary

After a clean aggregate or focused run, the CLI reports:

  • The number of checks, Jest suites, and tests.
  • The number of lint warnings, format issues, audit issues, and pack issues.
  • The time for checks, Jest, and the full run.
  • The exit code on the final line.

Use --debug-timing to show stage, check, and suite timings on the console. The CLI writes timing lines to coverage/eliware-test.log when validation starts. The --help and --version commands exit before validation starts.

README links

The general profile checks local README links and Markdown fragments. It checks supported links in Markdown and quoted HTML attributes. It does not fetch external links or fragments. It ignores links in code spans and fenced code blocks. Read the general convention files for the supported link forms.

Repository specifications

The CLI parses all YAML files in specs/. The CLI checks YAML syntax in all files. It checks the directives schema only when a YAML document has a top-level directives key. It checks unique rule IDs across all directive files. It checks each specs/ index against YAML files in its folder and child indexes. Read the profile rules for the full specs/ layout.

Repository map and links

The general check reads ../docs/repo-map.yaml when that file exists. When it finds a matching entry, it compares the repository ID, description, keywords, and profiles. Without a map, the package metadata supplies the repository description and rule ID namespace. The CLI checks local README links and fragments without network access. It does not check if an external website responds.

Data and validation scope

The CLI checks tracked links through the Git index. It rejects tracked file and directory symlinks. It does not resolve symlink targets.

Keep secrets out of child command output. The CLI redacts known secrets when it writes diagnostics. That redaction cannot find all secrets.

README and AGENTS sections

The general checks require content after every required level-two heading in README.md and AGENTS.md. The checks report a section when it contains only blank lines or other headings. The semantic conventions define the information that belongs in each section.

Documentation and release notes

The General profile requires documentation and release notes in every repository. Keep a docs/README.md index and link it from the root README.md. Index each maintained Markdown file in docs/, except the index itself. Keep RELEASE_NOTES.md and link it from the root README Usage section. Read the General convention files for release-note format and content rules.

Eliware Test runs validation only. It does not change a release or deployment system.

Return to documentation.