Skip to content

Commit e0b3ce4

Browse files
feat: add lyapunov sylvester (#203)
* add lyapunov sylvester * Update doc/solvers/LyapunovSylvester.md Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * merge * fix doc * fix readme * fix sonar findings --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
1 parent e3f73ac commit e0b3ce4

12 files changed

Lines changed: 437 additions & 167 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ Refer to the documentation to quickly integrate and utilize the library's signal
2626
| [Neural Network](doc/neural_network/README.md) | Layers, activations, losses, model |
2727
| [Optimization](doc/optimization/README.md) | Gradient Descent |
2828
| [Regularization](doc/regularization/README.md) | L1 (Lasso), L2 (Ridge) |
29-
| [Math](doc/math/README.md) | CORDIC, Quaternion, MatrixNorms, Step Response Metrics, MatrixExponential |
29+
| [Math](doc/math/README.md) | CORDIC, Quaternion, MatrixNorms, Step Response Metrics, MatrixExponential |
3030
| [Solvers](doc/solvers/README.md) | Gaussian Elimination, Levinson-Durbin, Durand-Kerner, Cholesky, DARE, Runge-Kutta ODE Integrators (RK4 + Dormand-Prince), Spectral Radius & Discrete Stability Margin, QR Decomposition (Householder / Givens), LU Decomposition with Partial Pivoting |
3131
| [Performance Optimization](doc/performance-optimization/README.md) | Compiler optimizations, SIMD |
3232

‎ROADMAP.md‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,6 @@ Difficulty legend:
2828
| # | Component | Target module | Difficulty |
2929
|----|------------------------------------------------------|---------------------------|------------|
3030
| 30 | Continuous → discrete conversion (`c2d`) | `math` | ★★★★☆ |
31-
| 31 | Lyapunov / Sylvester equation solvers | `solvers` | ★★★★☆ |
3231
| 32 | Transfer-function ↔ state-space conversion | `control_analysis` | ★★★★☆ |
3332
| 33 | Madgwick / Mahony AHRS | `filters/active` | ★★★★☆ |
3433
| 34 | Sliding Mode Control (SMC) | `robust_control` (new) | ★★★★☆ |

‎doc/solvers/LyapunovSylvester.md‎

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
# Lyapunov and Sylvester Equation Solvers
2+
3+
## Overview & Motivation
4+
5+
The Sylvester equation $AX + XB = C$ and its symmetric specialisations — the continuous Lyapunov
6+
$AX + XA^\top = -Q$ and the discrete Lyapunov $AXA^\top - X = -Q$ — are the central matrix
7+
equations of stability analysis, gramian computation, and robust-control synthesis. A
8+
positive-definite Lyapunov solution is a formal certificate that the system $\dot{x} = Ax$ is
9+
asymptotically stable; the same equations produce the **controllability and observability gramians**
10+
that measure how effectively a linear system can be driven and observed.
11+
12+
## Mathematical Theory
13+
14+
### Sylvester Equation
15+
16+
Given matrices $A \in \mathbb{R}^{N \times N}$, $B \in \mathbb{R}^{M \times M}$, and
17+
$C \in \mathbb{R}^{N \times M}$, the Sylvester equation is
18+
19+
$$AX + XB = C,$$
20+
21+
with unknown $X \in \mathbb{R}^{N \times M}$. The equation has a unique solution if and only if
22+
$A$ and $-B$ share no eigenvalue, equivalently $\lambda_i(A) + \lambda_j(B) \neq 0$ for all
23+
$i, j$.
24+
25+
### Kronecker Vectorisation
26+
27+
Stacking $X$ column-by-column into $\operatorname{vec}(X) \in \mathbb{R}^{NM}$ and applying
28+
the identities $\operatorname{vec}(AXI) = (I_M \otimes A)\operatorname{vec}(X)$ and
29+
$\operatorname{vec}(IXB) = (B^\top \otimes I_N)\operatorname{vec}(X)$ converts the matrix
30+
equation into the standard linear system
31+
32+
$$\underbrace{(I_M \otimes A + B^\top \otimes I_N)}_{K \in \mathbb{R}^{NM \times NM}}
33+
\operatorname{vec}(X) = \operatorname{vec}(C).$$
34+
35+
The $(iN+k, jN+l)$ entry of $K$ is $\delta_{ij} A_{kl} + B_{ji} \delta_{kl}$. The system
36+
is solved by Gaussian elimination with partial pivoting; a near-zero pivot signals the
37+
unsolvable (degenerate eigenvalue) case.
38+
39+
### Continuous Lyapunov Equation
40+
41+
Setting $B = A^\top$ and $C = -Q$ reduces the Sylvester equation to the continuous Lyapunov form
42+
43+
$$AX + XA^\top = -Q.$$
44+
45+
When $A$ is Hurwitz (all eigenvalues in the open left half-plane) and $Q$ is symmetric
46+
positive semi-definite, the unique solution $X$ is symmetric positive definite.
47+
48+
### Discrete Lyapunov Equation
49+
50+
The discrete counterpart
51+
52+
$$AXA^\top - X = -Q$$
53+
54+
is solved by the Kronecker system $(A \otimes A - I_{N^2})\operatorname{vec}(X) = -\operatorname{vec}(Q)$,
55+
with the $(iN+k, jN+l)$ entry of $A \otimes A - I$ equal to $A_{ij}A_{kl} - \delta_{ij}\delta_{kl}$.
56+
When $A$ is Schur-stable ($\rho(A) < 1$) and $Q \succeq 0$, the unique solution equals the
57+
controllability gramian $X = \sum_{k=0}^{\infty} A^k Q (A^\top)^k$.
58+
59+
## Complexity Analysis
60+
61+
| Case | Time | Space | Notes |
62+
|-------------------------------|----------------|-----------------------------|---------------------------------------------------------------|
63+
| $N{=}M{=}2$ | $O(64)$ flops | $O(NM)^2 = 16$ floats for K | 4×4 linear system |
64+
| $N{=}M{=}3$ | $O(729)$ flops | 81 floats for K | 9×9 linear system |
65+
| General $N{=}M$ | $O(N^6)$ flops | $O(N^4)$ words | Gaussian elimination on the $N^2 \times N^2$ Kronecker system |
66+
| Alternative (Bartels–Stewart) | $O(N^3)$ | $O(N^2)$ | Requires Schur decomposition; preferred for $N \geq 6$ |
67+
68+
For the embedded sizes ($N \leq 5$) targeted by this library the Kronecker approach is negligibly
69+
more expensive than Bartels–Stewart and avoids the full Schur machinery.
70+
71+
## Step-by-Step Walkthrough
72+
73+
Solve $AX + XB = C$ for the $2 \times 2$ case
74+
$A = \bigl[\begin{smallmatrix}-3&1\\0&-2\end{smallmatrix}\bigr]$,
75+
$B = \bigl[\begin{smallmatrix}1&2\\0&1\end{smallmatrix}\bigr]$,
76+
$C = I_2$.
77+
78+
**Build $K$** (the $4 \times 4$ Kronecker matrix):
79+
80+
$$K = \begin{pmatrix}-2&1&0&0\\0&-1&0&0\\2&0&-2&1\\0&2&0&-1\end{pmatrix}.$$
81+
82+
**Build $\operatorname{vec}(C) = (1, 0, 0, 1)^\top$.**
83+
84+
**Gaussian elimination** with partial pivoting produces $X_{00} = -0.5$, $X_{10} = 0$,
85+
$X_{01} = -1$, $X_{11} = -1$.
86+
87+
**Verify:** $AX + XB = I_2$. ✓
88+
89+
## Pitfalls & Edge Cases
90+
91+
- **Unsolvable case:** if $\lambda_i(A) + \lambda_j(B) = 0$ (continuous) or
92+
$\lambda_i(A)\lambda_j(B) = 1$ (discrete) for any pair, $K$ is singular and no unique solution
93+
exists. The solver returns `false` and leaves the solution unchanged.
94+
- **Ill-conditioning:** near-degenerate eigenvalue pairs yield large solution norms and reduced
95+
accuracy. Monitor the residual $\|AX + XB - C\|$ after solving.
96+
- **Symmetry:** the continuous Lyapunov solution is guaranteed symmetric only when $Q$ is
97+
symmetric; numerical symmetry should be enforced by the caller if required downstream.
98+
99+
## Variants & Generalizations
100+
101+
- **Bartels–Stewart algorithm:** reduces to quasi-upper-triangular (Schur) form via the double-shift
102+
QR algorithm, then back-substitutes column by column in $O(N^3)$. Preferred for $N \geq 6$.
103+
- **Smith iteration:** fixed-point iteration $X_{k+1} = AX_k A^\top + Q$ for the discrete
104+
Lyapunov equation; simple but slow for $\rho(A)$ close to 1.
105+
- **Cholesky factor updates:** propagate a Cholesky factor of $X$ for guaranteed positive-definiteness
106+
in square-root filtering (see `SquareRootKalmanFilter`).
107+
108+
## Applications
109+
110+
- **LQR / LQG design:** the continuous Lyapunov equation $AX + XA^\top + Q = 0$ yields the
111+
closed-loop covariance, and the DARE solution can be cross-validated via its Lyapunov residual.
112+
- **Kalman filter (observability Gramian):** $AW + WA^\top + BB^\top = 0$ gives the
113+
controllability Gramian used to evaluate reachability in balanced truncation.
114+
- **Pole-placement observers (Luenberger):** the Sylvester equation $AX - XF = LC$ arises when
115+
placing observer poles while decoupling the error dynamics.
116+
- **H₂ norm computation:** $\|G\|_2^2 = \operatorname{tr}(B^\top X B)$ where $X$ satisfies the
117+
observability Gramian Lyapunov equation.
118+
119+
## Connections to Other Algorithms
120+
121+
- **LU Decomposition:** the Kronecker-vectorisation approach converts any Sylvester equation into
122+
a dense linear system solved by the `LuDecomposition` solver.
123+
- **QR Decomposition:** the Bartels–Stewart and Hammarling algorithms use QR / Schur
124+
decomposition to reduce the problem to quasi-triangular form before back-substitution.
125+
- **DARE:** the discrete algebraic Riccati equation can be warm-started or verified using
126+
one step of the discrete Lyapunov equation.
127+
- **Spectral Radius:** solvability of the continuous equation requires $\lambda_i(A) + \lambda_j(A) \neq 0$;
128+
checking the spectral radius is a fast necessary condition.
129+
130+
## References & Further Reading
131+
132+
- Bartels, R. H. & Stewart, G. W. (1972). "Solution of the matrix equation AX + XB = C."
133+
*Communications of the ACM*, 15(9), 820–826.
134+
- Golub, G. H., Nash, S. & Van Loan, C. (1979). "A Hessenberg-Schur method for the problem
135+
AX + XB = C." *IEEE Transactions on Automatic Control*, 24(6), 909–913.
136+
- Horn, R. A. & Johnson, C. R. (1994). *Topics in Matrix Analysis*. Cambridge University Press.
137+
(Chapter 4: Kronecker products and the vec operator.)
138+
- Hammarling, S. J. (1982). "Numerical solution of the stable, non-negative definite Lyapunov
139+
equation." *IMA Journal of Numerical Analysis*, 2(3), 303–323.

‎doc/solvers/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,3 +16,4 @@ Numerical solvers for linear systems, polynomial roots, and matrix equations.
1616
| [Condition Number](ConditionNumber.md) | 1-norm condition number estimate via column-wise inverse solve |
1717
| [QR Decomposition](QrDecomposition.md) | Householder factorization and Givens streaming row update for least-squares solves |
1818
| [LU Decomposition](LuDecomposition.md) | PA = LU factorization with partial pivoting for general dense linear systems |
19+
| [Lyapunov / Sylvester Solvers](LyapunovSylvester.md) | Sylvester AX+XB=C and continuous/discrete Lyapunov solvers via Kronecker vectorisation |

‎numerical/solvers/CMakeLists.txt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ target_sources(numerical.solver PRIVATE
1919
GaussianElimination.hpp
2020
LevinsonDurbin.hpp
2121
LuDecomposition.hpp
22+
LyapunovSylvester.hpp
2223
OdeSystem.hpp
2324
QrDecomposition.hpp
2425
RungeKuttaIntegrators.hpp
@@ -31,6 +32,7 @@ numerical_add_coverage_sources(numerical.solver
3132
DurandKerner.cpp
3233
GaussianElimination.cpp
3334
LuDecomposition.cpp
35+
LyapunovSylvester.cpp
3436
QrDecomposition.cpp
3537
RungeKuttaIntegrators.cpp
3638
SpectralRadius.cpp
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
#include "numerical/solvers/LyapunovSylvester.hpp"
2+
3+
namespace solvers
4+
{
5+
template class SylvesterSolver<float, 2, 2>;
6+
template class SylvesterSolver<float, 3, 3>;
7+
}
Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
#pragma once
2+
3+
#if defined(__GNUC__) || defined(__clang__)
4+
#pragma GCC optimize("O3", "fast-math")
5+
#endif
6+
7+
#include "numerical/math/CompilerOptimizations.hpp"
8+
#include "numerical/math/Matrix.hpp"
9+
#include "numerical/solvers/LuDecomposition.hpp"
10+
#include <cstddef>
11+
#include <type_traits>
12+
13+
namespace solvers
14+
{
15+
template<typename T, std::size_t N, std::size_t M>
16+
class SylvesterSolver
17+
{
18+
static_assert(std::is_floating_point_v<T>, "SylvesterSolver supports floating-point types only");
19+
static_assert(N > 0 && M > 0, "SylvesterSolver requires positive dimensions");
20+
21+
static constexpr std::size_t NM = N * M;
22+
23+
public:
24+
SylvesterSolver() = default;
25+
26+
OPTIMIZE_FOR_SPEED bool SolveSylvester(
27+
const math::SquareMatrix<T, N>& a,
28+
const math::SquareMatrix<T, M>& b,
29+
const math::Matrix<T, N, M>& c);
30+
31+
OPTIMIZE_FOR_SPEED bool SolveContinuousLyapunov(
32+
const math::SquareMatrix<T, N>& a,
33+
const math::SquareMatrix<T, N>& q);
34+
35+
OPTIMIZE_FOR_SPEED bool SolveDiscreteLyapunov(
36+
const math::SquareMatrix<T, N>& a,
37+
const math::SquareMatrix<T, N>& q);
38+
39+
[[nodiscard]] const math::Matrix<T, N, M>& Solution() const
40+
{
41+
return solution;
42+
}
43+
44+
private:
45+
static math::Matrix<T, N * M, 1> Vec(const math::Matrix<T, N, M>& x);
46+
static math::Matrix<T, N, M> UnVec(const math::Matrix<T, N * M, 1>& v);
47+
48+
math::Matrix<T, N, M> solution{};
49+
};
50+
51+
template<typename T, std::size_t N, std::size_t M>
52+
OPTIMIZE_FOR_SPEED bool SylvesterSolver<T, N, M>::SolveSylvester(
53+
const math::SquareMatrix<T, N>& a,
54+
const math::SquareMatrix<T, M>& b,
55+
const math::Matrix<T, N, M>& c)
56+
{
57+
math::Matrix<T, NM, NM> K{};
58+
for (std::size_t row = 0; row < NM; ++row)
59+
for (std::size_t col = 0; col < NM; ++col)
60+
{
61+
const std::size_t i = row / N;
62+
const std::size_t k = row % N;
63+
const std::size_t j = col / N;
64+
const std::size_t l = col % N;
65+
T val{};
66+
if (i == j)
67+
val += a.at(k, l);
68+
if (k == l)
69+
val += b.at(j, i);
70+
K.at(row, col) = val;
71+
}
72+
73+
math::Matrix<T, NM, 1> vecC = Vec(c);
74+
75+
LuDecomposition<T, NM> lu{};
76+
if (!lu.Decompose(K))
77+
return false;
78+
79+
solution = UnVec(lu.Solve(vecC));
80+
return true;
81+
}
82+
83+
template<typename T, std::size_t N, std::size_t M>
84+
OPTIMIZE_FOR_SPEED bool SylvesterSolver<T, N, M>::SolveContinuousLyapunov(
85+
const math::SquareMatrix<T, N>& a,
86+
const math::SquareMatrix<T, N>& q)
87+
{
88+
static_assert(N == M, "SolveContinuousLyapunov requires square solver (N == M)");
89+
90+
return SolveSylvester(a, a.Transpose(), q * T(-1));
91+
}
92+
93+
template<typename T, std::size_t N, std::size_t M>
94+
OPTIMIZE_FOR_SPEED bool SylvesterSolver<T, N, M>::SolveDiscreteLyapunov(
95+
const math::SquareMatrix<T, N>& a,
96+
const math::SquareMatrix<T, N>& q)
97+
{
98+
static_assert(N == M, "SolveDiscreteLyapunov requires square solver (N == M)");
99+
100+
math::Matrix<T, NM, NM> K{};
101+
for (std::size_t row = 0; row < NM; ++row)
102+
for (std::size_t col = 0; col < NM; ++col)
103+
{
104+
const std::size_t i = row / N;
105+
const std::size_t k = row % N;
106+
const std::size_t j = col / N;
107+
const std::size_t l = col % N;
108+
T val = a.at(i, j) * a.at(k, l);
109+
if (i == j && k == l)
110+
val -= T(1);
111+
K.at(row, col) = val;
112+
}
113+
114+
math::Matrix<T, NM, 1> rhs = Vec(q * T(-1));
115+
116+
LuDecomposition<T, NM> lu{};
117+
if (!lu.Decompose(K))
118+
return false;
119+
120+
solution = UnVec(lu.Solve(rhs));
121+
return true;
122+
}
123+
124+
template<typename T, std::size_t N, std::size_t M>
125+
math::Matrix<T, N * M, 1> SylvesterSolver<T, N, M>::Vec(const math::Matrix<T, N, M>& x)
126+
{
127+
math::Matrix<T, NM, 1> v{};
128+
for (std::size_t i = 0; i < M; ++i)
129+
for (std::size_t k = 0; k < N; ++k)
130+
v.at(i * N + k, 0) = x.at(k, i);
131+
return v;
132+
}
133+
134+
template<typename T, std::size_t N, std::size_t M>
135+
math::Matrix<T, N, M> SylvesterSolver<T, N, M>::UnVec(const math::Matrix<T, N * M, 1>& v)
136+
{
137+
math::Matrix<T, N, M> out{};
138+
for (std::size_t i = 0; i < M; ++i)
139+
for (std::size_t k = 0; k < N; ++k)
140+
out.at(k, i) = v.at(i * N + k, 0);
141+
return out;
142+
}
143+
144+
#ifdef NUMERICAL_TOOLBOX_COVERAGE_BUILD
145+
extern template class SylvesterSolver<float, 2, 2>;
146+
extern template class SylvesterSolver<float, 3, 3>;
147+
#endif
148+
}

‎numerical/solvers/test/CMakeLists.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ target_sources(numerical.solvers_test PRIVATE
1414
TestGaussianElimination.cpp
1515
TestLevinsonDurbin.cpp
1616
TestLuDecomposition.cpp
17+
TestLyapunovSylvester.cpp
1718
TestQrDecomposition.cpp
1819
TestRungeKuttaIntegrators.cpp
1920
TestSpectralRadius.cpp

0 commit comments

Comments
 (0)