General-purpose optimization algorithms for parameter fitting, training, and control.
| Algorithm | Description |
|---|---|
| Bayesian Optimization | Gradient-free global optimization using Gaussian Process surrogate and Expected Improvement |
| Gradient Descent | Iterative batch optimization via gradient-based weight updates |
| SGD / Adam Step Optimisers | Stateful per-step SGD (with optional momentum/Nesterov) and Adam with bias correction |
| Levenberg–Marquardt | Damped Gauss–Newton for small nonlinear least squares with a retraction (manifold / bounded states) and pluggable damping |