|
1 | 1 | # pluginval Change List |
2 | 2 |
|
3 | 3 | ### 2.0.0 |
4 | | -- Updated JUCE to 9.0.4 |
5 | | -- Fixed the crash handler so a backtrace is shown again when a plugin crashes, and made it async-signal-safe [#144] |
6 | | -- Added `--junit-xml <file>` and `--json-results <file>` to both `pluginval validate` and `pluginval test` to write machine-readable reports for CI (also settable as `JUNIT_XML` / `JSON_RESULTS`). A report that can't be written fails the run. See `docs/Adding pluginval to CI.md` |
| 4 | + |
| 5 | +**Upgrading from 1.x:** the command line now uses subcommands (`pluginval validate [options] <plugin>`; `pluginval <plugin>` still works) and is stricter: unrecognised options and invalid values are errors instead of being ignored, and `--vst3validator` has been removed because the VST3 validator is now built in. CI scripts that download `releases/latest` will get 2.0.0 automatically, so check the **Breaking** items below. |
| 6 | + |
| 7 | +#### New |
7 | 8 | - Added acceptance testing: `pluginval test <config.json>` renders a plugin with a known state, input (audio/MIDI) and optional playhead, then compares the output against a checked-in reference WAV, exiting `0` on a match and `1` on a mismatch (with a diff WAV). Use `--record-missing` to record new references. See `docs/Acceptance testing.md` |
8 | 9 | - Added `--plugin <plugin>` to `pluginval test` to run a config against a given plugin instead of the one it names (the config's `plugin` can then be omitted) |
| 10 | +- Added `--junit-xml <file>` and `--json-results <file>` to both `pluginval validate` and `pluginval test` to write machine-readable reports for CI (also settable as `JUNIT_XML` / `JSON_RESULTS`). A report that can't be written fails the run. See `docs/Adding pluginval to CI.md` |
9 | 11 | - Added two GitHub Actions for use from other repositories: `Tracktion/pluginval/validate` (validate a list of plugins) and `Tracktion/pluginval/acceptance` (run every acceptance config in a directory against a built plugin). See `.github/actions/README.md` |
| 12 | +- Added real-time safety checking on macOS, using [rtcheck](https://github.com/Tracktion/rtcheck): `--rtcheck enabled` checks the plugin's process calls during the tests for real-time-unsafe behaviour, and `--rtcheck relaxed` skips the first process call to allow for initialisation. Off by default [#156] |
| 13 | +- The Steinberg VST3 validator is now built into pluginval and runs automatically for VST3 plugins at strictness level 5 and above, so it no longer needs downloading and passing in separately [#162] |
| 14 | +- Added `--config <file.json>` to load settings from JSON (repeatable; later files win per key) |
| 15 | +- Added a "Discrete parameter state restoration" test (strictness 8) which checks a discrete parameter has the same value after restoring a state, whatever value it held beforehand. This fails for parameters that keep the raw normalised value from the host instead of mapping it to a step (e.g. `juce::AudioParameterBool`) |
| 16 | +- Added drag-and-drop of plug-in files onto the main window, with two drop zones to either validate the plug-in or add it to the plugin list [#170] |
| 17 | +- Enabled the editor stress and extreme (real-time allocation / oversized block) tests that were present in the source tree but had not been compiled into the build |
| 18 | + |
| 19 | +#### Changes |
| 20 | +- Updated JUCE to 9.0.4 |
10 | 21 | - Restructured the command line into subcommands: `pluginval validate [options] <plugin>` (the default), `pluginval run-tests` and `pluginval strictness-help [level]`. The plugin path is now a positional argument of `validate` |
11 | 22 | - **Deprecated:** the flat flags `--validate <plugin>`, `--run-tests` and `--strictness-help [level]` still work as aliases (with a one-line notice) but will be removed in a future version. `pluginval <plugin>` remains a silent shorthand for `validate` |
12 | 23 | - Replaced the hand-rolled command-line parser with CLI11 and a single JSON-based settings pipeline |
13 | | -- Added `--config <file.json>` to load settings from JSON (repeatable; later files win per key) |
14 | 24 | - Settings precedence is now (lowest to highest): defaults, environment variables, `--config`, command-line options |
15 | 25 | - Environment variable support is now automatic for every option (including `DISABLED_TESTS`) |
16 | | -- **Breaking:** unrecognised options now produce an error instead of being ignored |
17 | | -- **Breaking:** invalid option values now error instead of silently falling back (e.g. `--rtcheck banana`, `--strictness-level abc`) |
18 | | -- **Breaking:** falsey flag environment variables now mean off (e.g. `SKIP_GUI_TESTS=0` no longer enables skipping) |
19 | 26 | - `--help` output is now generated by CLI11 |
20 | | -- Added a "Discrete parameter state restoration" test (strictness 8) which checks a discrete parameter has the same value after restoring a state, whatever value it held beforehand. This fails for parameters that keep the raw normalised value from the host instead of mapping it to a step (e.g. `juce::AudioParameterBool`) |
21 | | -- Fixed false failures in the "Plugin state restoration" test where the first parameter was compared against the last value sent by a previous test rather than the plugin's own value |
22 | 27 | - Below strictness level 8, two-step (on/off) parameters are now only sent 0 or 1 when tests randomise parameter values. Arbitrary values are still sent to them at level 8 and above, and to all other parameters at every level |
| 28 | +- Added static linking to the Windows runtime so it should run on more Windows systems (particularly non-dev machines) |
| 29 | +- Made `PluginInfoTest` run on the message thread as the functions it calls aren't thread-safe |
| 30 | + |
| 31 | +#### Breaking |
| 32 | +- Removed `--vst3validator <path>`: the VST3 validator is now built in (see above), and passing the option is an error [#162] |
| 33 | +- Unrecognised options now produce an error instead of being ignored |
| 34 | +- Invalid option values now error instead of silently falling back (e.g. `--rtcheck banana`, `--strictness-level abc`) |
| 35 | +- Falsey flag environment variables now mean off (e.g. `SKIP_GUI_TESTS=0` no longer enables skipping) |
| 36 | + |
| 37 | +#### Fixes |
| 38 | +- Fixed the crash handler so a backtrace is shown again when a plugin crashes, and made it async-signal-safe [#144] |
| 39 | +- Fixed false failures in the "Plugin state restoration" test where the first parameter was compared against the last value sent by a previous test rather than the plugin's own value |
23 | 40 | - Fixed the "Plugin state restoration" test's exact binary state comparison never running. It was documented as running at strictness level 8 and above but checked the test's own level instead of the run's |
| 41 | +- Fixed races between a timeout and the writing of the log and report files, which could leave a report that disagreed with the exit code [#198] |
| 42 | +- Fixed garbled non-ASCII characters in output from the validation child process when a character was split across reads [#198] |
| 43 | +- Fixed `--output-filename` without `--output-dir` not being resolved against the working directory [#198] |
| 44 | +- pluginval's `operator delete` replacement no longer throws (which ended the process), and its `operator new` throws `std::bad_alloc` instead of returning null [#198] |
24 | 45 | - Fixed the macOS app failing to start on any machine other than the one that built it ("Library not loaded: @rpath/librtcheck.dylib"); this affected the `v2.0.0-beta1` and `v2.0.0-beta2` releases |
25 | | -- Enabled the editor stress and extreme (real-time allocation / oversized block) tests that were present in the source tree but had not been compiled into the build |
26 | | - |
27 | | -### 1.0.5 |
28 | | -- Added drag-and-drop of plug-in files onto the main window, with two drop zones to either validate the plug-in or add it to the plugin list [#170] |
29 | | -- Added static linking to the Windows runtime so it should run on more Windows systems (particularly non-dev machines) |
30 | | -- Made `PluginInfoTest` run on the message thread as the functions it calls aren't thread-safe |
31 | 46 |
|
32 | 47 | ### 1.0.4 |
33 | 48 | - Limit auval's stress test to 20 seconds (vs 600) [#135] |
|
0 commit comments