LearnLib CLI (#166)

* initial work on CLI application

* update build profiles

* github: add worflow for attaching cli artifacts

* cli: include OS classifiers in artifact name

* do not include cli module in releases for now

* update docs

* tidy:pom

* add support for SAF serialization

* cli: add ADT and LSHARP algorithm

* cli oracles still need some rework regarding error handling

* cleanup CLI oracles

* experiment: allow for logging intermediate hypotheses

* jacoco: also depend on cli module before aggregating reports

* cli: improve documentation

* fix code-analysis

* cli: add support for parallel oracles

* cli: add unit- and integration-tests

* add support for snapshotting

* use LocalDateTime

* simplify --eqo-* param names

* Revert "simplify --eqo-* param names"

This reverts commit d209e96.

* add MalerPnueli + RivestSchapire learner and adjust learner names

* include license information in jlink artifact

* cleanup checkerframework config

* add rudimentary class validation during deserialization + cleanups

* add changelog

* add missing --add-opens + test

* cleanup

* wording

* handle python availability more gracefully

* conditionally set --add-opens option

* cli: invoke script via python interpreter

as on Windows the scripts alone are not detected as executable applications

* sett full (and platform-specific) path in jlink IT

* wording

* coveralls: s/Example/Experiment/

Author: mtf90

.github/workflows/ci.yml

@@ -47,7 +47,7 @@ jobs:
           mvn -B install -DskipTests
           cd $GITHUB_WORKSPACE
       - name: Run Maven
-        run: mvn -B install site -Pintegration-tests,code-analysis,bundles,jlink
+        run: mvn -B install site -Pintegration-tests,code-analysis,bundles
   platform-integration:
     name: "Platform Integration (JDK: ${{ matrix.jdk }}, OS: ${{ matrix.os }})"
     needs: [ tests-and-analysis ]
@@ -81,7 +81,7 @@ jobs:
           mvn -B install -DskipTests
           cd $GITHUB_WORKSPACE
       - name: Run Maven
-        run: mvn -B install -Pjlink
+        run: mvn -B install -Pcli
   coverage:
     name: "Coverage"
     needs: [ platform-integration ]

.github/workflows/release.yml

@@ -0,0 +1,35 @@
+name: Attach artifacts to GitHub release
+on:
+  release:
+    types: [published]
+jobs:
+  publish:
+    name: "Build and attach artifacts"
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ ubuntu-22.04, windows-2022, macos-15-intel, macos-15 ]
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/checkout@v6
+      - uses: actions/setup-java@v5
+        with:
+          java-version: '25'
+          distribution: 'temurin'
+          cache: 'maven'
+      - name: Build AutomataLib # can be removed on actual stable releases
+        shell: bash
+        run: |
+          git clone -b develop --single-branch https://github.com/LearnLib/automatalib.git ${HOME}/automatalib-git
+          cd ${HOME}/automatalib-git
+          mvn -B install -DskipTests
+          cd $GITHUB_WORKSPACE
+      - name: Build
+        run: mvn -B package -DskipTests -Pcli
+      - name: Release
+        uses: softprops/action-gh-release@v3
+        with:
+          working_directory: cli/target
+          files: learnlib-cli-*.zip
+          fail_on_unmatched_files: true

CHANGELOG.md

@@ -8,6 +8,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ### Added
 
+* GitHub releases now provide `learnlib-cli` artifacts for using LearnLib via the command-line interface without the need for Java or Maven.
 * Added a new (L*-based) learning algorithm for *Mealy machines with local timers* (MMLTs), including support for parallel queries, caching, and conformance testing (thanks to [Paul Kogel](https://github.com/pdev55)).
 * Added the L<sup>s</sup> active learning algorithm for Mealy machines (thanks to [Wolffhardt Schwabe](https://github.com/stateMachinist)).
 * Added an `EarlyExitEQOracle` which for a given `AdaptiveMembershipOracle` and `TestWordGenerator` stops the evaluation of (potentially long) Mealy-based equivalence tests as soon as a mismatch with the hypothesis is detected, potentially improving the symbol performance of the given equivalence oracle.
@@ -21,12 +22,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 * Statistics collection has received a major rework. Previously, classes would implement the `StatisticCollector` interface and return a `StatisticData` object which 1) only allows for describing a very limited amount of data, and 2) requires you to keep track of all the objects that collect data. This approach has been *replaced* by a new `StatisticsService`. Instances of this service can be obtained similar to a logger via `Statistics.getService()` and require you to provide an implementation of this service on the classpath (a default one is provided by the `learnlib-statistics` module). The new service allows arbitrary components to collect various data which can be conveniently extracted based on the new `StatisticsKey`s used by the components. For more details on advanced scenarios (such as multi-threaded benchmarking), see the documentation of the respective classes. While this may require you to adjust the way you are collecting statistics, all functionality from beforehand should still be available.
   * `SimpleProfiler` has been replaced by the new clock-based statistics.
 * Most learners now more rigorously implement the `LearningAlgorithm` contract that, e.g., duplicate invocations of `startLearning` or calling `refineHypothesis` / `getHypothesisModel` before `startLearning` throw `IllegalStateException`s.
+* `Experiment` now has type variables for the input symbol type and output domain type.
+* `{DFA,Mealy,Moore}Experiment` have been moved to the `de.learnlib.util` package.
 * The `generateTestWords` method of `AbstractTestWordEQOracle` now needs to be public.
 * The classes of `de.learnlib.testsupport.it.learner` have been split into the packages `de.learnlib.testsupport.it{,testcase,util,variant}` in the same module (`de.learnlib.testsupport:learnlib-learner-it-support`).
 
 ### Removed
 
-* All *adapters* from the `learnlib-procedural` learner have been removed to due main learners implementing `AccessSequenceTransformer` now. Use the constructors of the main learners instead.
+* All *adapters* from the `learnlib-procedural` learner have been removed due to main learners implementing `AccessSequenceTransformer` now. Use the constructors of the main learners instead.
 
 ### Fixed
 

algorithms/active/aaar/src/test/java/de/learnlib/algorithm/aaar/AbstractAAARTest.java

@@ -52,12 +52,12 @@ public void testAbstractHypothesisEquivalence() {
         final WpMethodTestsIterator<I> iter = new WpMethodTestsIterator<>(automaton, alphabet);
         final List<Word<I>> testCases = IteratorUtil.list(iter);
 
-        final SampleSetEQOracle<I, D> eqo = new SampleSetEQOracle<>();
-        eqo.addAll(new SimulatorOracle<>(automaton), testCases);
+        final SampleSetEQOracle<I, D> eqo =
+                new SampleSetEQOracle<I, D>().addAll(new SimulatorOracle<>(automaton), testCases);
 
         final LearningAlgorithm<A, I, D> learner =
                 new TranslatingLearnerWrapper<>((AbstractAAARLearner<?, A, A, I, I, D>) aaarLearner);
-        final Experiment<A> exp = new Experiment<>(learner, eqo, alphabet);
+        final Experiment<A, I, D> exp = new Experiment<>(learner, eqo, alphabet);
 
         exp.run();
 

algorithms/active/adt/src/main/java/de/learnlib/algorithm/adt/learner/ADTLearner.java

@@ -60,6 +60,7 @@
 import de.learnlib.query.DefaultQuery;
 import de.learnlib.tooling.annotation.builder.GenerateBuilder;
 import de.learnlib.util.MQUtil;
+import de.learnlib.util.mealy.Adaptive2MembershipWrapper;
 import net.automatalib.alphabet.Alphabet;
 import net.automatalib.alphabet.SupportsGrowingAlphabet;
 import net.automatalib.automaton.transducer.MealyMachine;
@@ -104,12 +105,26 @@ public class ADTLearner<I, O> implements LearningAlgorithm.MealyLearner<I, O>,
     private ADTHypothesis<I, O> hypothesis;
     private ADT<ADTState<I, O>, I, O> adt;
 
+    public ADTLearner(Alphabet<I> alphabet, AdaptiveMembershipOracle<I, O> oracle) {
+        this(alphabet,
+             oracle,
+             BuilderDefaults.leafSplitter(),
+             BuilderDefaults.adtExtender(),
+             BuilderDefaults.subtreeReplacer());
+    }
+
     public ADTLearner(Alphabet<I> alphabet,
                       AdaptiveMembershipOracle<I, O> oracle,
                       LeafSplitter leafSplitter,
                       ADTExtender adtExtender,
                       SubtreeReplacer subtreeReplacer) {
-        this(alphabet, oracle, leafSplitter, adtExtender, subtreeReplacer, true, LocalSuffixFinders.RIVEST_SCHAPIRE);
+        this(alphabet,
+             oracle,
+             leafSplitter,
+             adtExtender,
+             subtreeReplacer,
+             BuilderDefaults.useObservationTree(),
+             BuilderDefaults.suffixFinder());
     }
 
     @GenerateBuilder(defaults = BuilderDefaults.class)

algorithms/active/adt/src/test/java/de/learnlib/algorithm/adt/it/ADTIT.java

@@ -47,7 +47,7 @@
 import de.learnlib.testsupport.MQ2AQWrapper;
 import de.learnlib.testsupport.it.AbstractMealyLearnerIT;
 import de.learnlib.testsupport.it.variant.LearnerVariantList;
-import de.learnlib.util.Experiment.MealyExperiment;
+import de.learnlib.util.MealyExperiment;
 import net.automatalib.alphabet.Alphabet;
 import net.automatalib.automaton.transducer.impl.CompactMealy;
 import net.automatalib.exception.FormatException;

algorithms/active/lambda/src/test/java/de/learnlib/algorithm/lambda/AbstractCounterexampleQueueTest.java

@@ -19,7 +19,7 @@
 import de.learnlib.oracle.MembershipOracle.DFAMembershipOracle;
 import de.learnlib.oracle.equivalence.SampleSetEQOracle;
 import de.learnlib.oracle.membership.DFASimulatorOracle;
-import de.learnlib.util.Experiment.DFAExperiment;
+import de.learnlib.util.Experiment;
 import net.automatalib.alphabet.Alphabet;
 import net.automatalib.alphabet.impl.Alphabets;
 import net.automatalib.automaton.fsa.DFA;
@@ -80,7 +80,8 @@ public void testPop() {
         final Word<Character> b = new WordBuilder<>('b', 9).toWord();
         eqOracle.addAll(mqOracle, Word.fromWords(b, a, b, a, b, a, b, a));
 
-        final DFAExperiment<Character> experiment = new DFAExperiment<>(learner, eqOracle, alphabet);
+        final Experiment<DFA<?, Character>, Character, Boolean> experiment =
+                new Experiment<>(learner, eqOracle, alphabet);
         experiment.run();
 
         final DFA<?, Character> result = experiment.getFinalHypothesis();

algorithms/active/lambda/src/test/java/de/learnlib/algorithm/lambda/lstar/mealy/it/LLambdaMealyIT.java

@@ -30,7 +30,7 @@
 import de.learnlib.query.DefaultQuery;
 import de.learnlib.testsupport.it.AbstractMealyLearnerIT;
 import de.learnlib.testsupport.it.variant.LearnerVariantList.MealyLearnerVariantList;
-import de.learnlib.util.Experiment.MealyExperiment;
+import de.learnlib.util.MealyExperiment;
 import de.learnlib.util.mealy.MealyUtil;
 import net.automatalib.alphabet.Alphabet;
 import net.automatalib.automaton.transducer.MealyMachine;

algorithms/active/nlstar/src/test/java/de/learnlib/algorithm/nlstar/NLStarTest.java

@@ -62,7 +62,8 @@ public void testIssue70() {
 
         final NLStarLearner<Character> learner = new NLStarLearner<>(alphabet, mqOracle);
 
-        final Experiment<NFA<?, Character>> experiment = new Experiment<>(learner, eqOracle, alphabet);
+        final Experiment<NFA<?, Character>, Character, Boolean> experiment =
+                new Experiment<>(learner, eqOracle, alphabet);
         experiment.run();
         final NFA<?, Character> hyp = experiment.getFinalHypothesis();
 

algorithms/active/observation-pack-vpa/src/test/java/de/learnlib/algorithm/observationpack/vpa/DTVisualizationTest.java

@@ -50,7 +50,7 @@ public DTVisualizationTest() {
         final SimulatorEQOracle<Character> eqo = new SimulatorEQOracle<>(vpa);
         this.learner = new OPLearnerVPA<>(alphabet, mqo, AcexAnalyzers.BINARY_SEARCH_FWD);
 
-        final Experiment<OneSEVPA<?, Character>> exp = new Experiment<>(learner, eqo, alphabet);
+        final Experiment<OneSEVPA<?, Character>, Character, Boolean> exp = new Experiment<>(learner, eqo, alphabet);
         exp.run();
     }