|
| 1 | +# Lyapunov and Sylvester Equation Solvers |
| 2 | + |
| 3 | +## Overview & Motivation |
| 4 | + |
| 5 | +The Sylvester equation $AX + XB = C$ and its symmetric specialisations — the continuous Lyapunov |
| 6 | +$AX + XA^\top = -Q$ and the discrete Lyapunov $AXA^\top - X = -Q$ — are the central matrix |
| 7 | +equations of stability analysis, gramian computation, and robust-control synthesis. A |
| 8 | +positive-definite Lyapunov solution is a formal certificate that the system $\dot{x} = Ax$ is |
| 9 | +asymptotically stable; the same equations produce the **controllability and observability gramians** |
| 10 | +that measure how effectively a linear system can be driven and observed. |
| 11 | + |
| 12 | +## Mathematical Theory |
| 13 | + |
| 14 | +### Sylvester Equation |
| 15 | + |
| 16 | +Given matrices $A \in \mathbb{R}^{N \times N}$, $B \in \mathbb{R}^{M \times M}$, and |
| 17 | +$C \in \mathbb{R}^{N \times M}$, the Sylvester equation is |
| 18 | + |
| 19 | +$$AX + XB = C,$$ |
| 20 | + |
| 21 | +with unknown $X \in \mathbb{R}^{N \times M}$. The equation has a unique solution if and only if |
| 22 | +$A$ and $-B$ share no eigenvalue, equivalently $\lambda_i(A) + \lambda_j(B) \neq 0$ for all |
| 23 | +$i, j$. |
| 24 | + |
| 25 | +### Kronecker Vectorisation |
| 26 | + |
| 27 | +Stacking $X$ column-by-column into $\operatorname{vec}(X) \in \mathbb{R}^{NM}$ and applying |
| 28 | +the identities $\operatorname{vec}(AXI) = (I_M \otimes A)\operatorname{vec}(X)$ and |
| 29 | +$\operatorname{vec}(IXB) = (B^\top \otimes I_N)\operatorname{vec}(X)$ converts the matrix |
| 30 | +equation into the standard linear system |
| 31 | + |
| 32 | +$$\underbrace{(I_M \otimes A + B^\top \otimes I_N)}_{K \in \mathbb{R}^{NM \times NM}} |
| 33 | +\operatorname{vec}(X) = \operatorname{vec}(C).$$ |
| 34 | + |
| 35 | +The $(iN+k, jN+l)$ entry of $K$ is $\delta_{ij} A_{kl} + B_{ji} \delta_{kl}$. The system |
| 36 | +is solved by Gaussian elimination with partial pivoting; a near-zero pivot signals the |
| 37 | +unsolvable (degenerate eigenvalue) case. |
| 38 | + |
| 39 | +### Continuous Lyapunov Equation |
| 40 | + |
| 41 | +Setting $B = A^\top$ and $C = -Q$ reduces the Sylvester equation to the continuous Lyapunov form |
| 42 | + |
| 43 | +$$AX + XA^\top = -Q.$$ |
| 44 | + |
| 45 | +When $A$ is Hurwitz (all eigenvalues in the open left half-plane) and $Q$ is symmetric |
| 46 | +positive semi-definite, the unique solution $X$ is symmetric positive definite. |
| 47 | + |
| 48 | +### Discrete Lyapunov Equation |
| 49 | + |
| 50 | +The discrete counterpart |
| 51 | + |
| 52 | +$$AXA^\top - X = -Q$$ |
| 53 | + |
| 54 | +is solved by the Kronecker system $(A \otimes A - I_{N^2})\operatorname{vec}(X) = -\operatorname{vec}(Q)$, |
| 55 | +with the $(iN+k, jN+l)$ entry of $A \otimes A - I$ equal to $A_{ij}A_{kl} - \delta_{ij}\delta_{kl}$. |
| 56 | +When $A$ is Schur-stable ($\rho(A) < 1$) and $Q \succeq 0$, the unique solution equals the |
| 57 | +controllability gramian $X = \sum_{k=0}^{\infty} A^k Q (A^\top)^k$. |
| 58 | + |
| 59 | +## Complexity Analysis |
| 60 | + |
| 61 | +| Case | Time | Space | Notes | |
| 62 | +|-------------------------------|----------------|-----------------------------|---------------------------------------------------------------| |
| 63 | +| $N{=}M{=}2$ | $O(64)$ flops | $O(NM)^2 = 16$ floats for K | 4×4 linear system | |
| 64 | +| $N{=}M{=}3$ | $O(729)$ flops | 81 floats for K | 9×9 linear system | |
| 65 | +| General $N{=}M$ | $O(N^6)$ flops | $O(N^4)$ words | Gaussian elimination on the $N^2 \times N^2$ Kronecker system | |
| 66 | +| Alternative (Bartels–Stewart) | $O(N^3)$ | $O(N^2)$ | Requires Schur decomposition; preferred for $N \geq 6$ | |
| 67 | + |
| 68 | +For the embedded sizes ($N \leq 5$) targeted by this library the Kronecker approach is negligibly |
| 69 | +more expensive than Bartels–Stewart and avoids the full Schur machinery. |
| 70 | + |
| 71 | +## Step-by-Step Walkthrough |
| 72 | + |
| 73 | +Solve $AX + XB = C$ for the $2 \times 2$ case |
| 74 | +$A = \bigl[\begin{smallmatrix}-3&1\\0&-2\end{smallmatrix}\bigr]$, |
| 75 | +$B = \bigl[\begin{smallmatrix}1&2\\0&1\end{smallmatrix}\bigr]$, |
| 76 | +$C = I_2$. |
| 77 | + |
| 78 | +**Build $K$** (the $4 \times 4$ Kronecker matrix): |
| 79 | + |
| 80 | +$$K = \begin{pmatrix}-2&1&0&0\\0&-1&0&0\\2&0&-2&1\\0&2&0&-1\end{pmatrix}.$$ |
| 81 | + |
| 82 | +**Build $\operatorname{vec}(C) = (1, 0, 0, 1)^\top$.** |
| 83 | + |
| 84 | +**Gaussian elimination** with partial pivoting produces $X_{00} = -0.5$, $X_{10} = 0$, |
| 85 | +$X_{01} = -1$, $X_{11} = -1$. |
| 86 | + |
| 87 | +**Verify:** $AX + XB = I_2$. ✓ |
| 88 | + |
| 89 | +## Pitfalls & Edge Cases |
| 90 | + |
| 91 | +- **Unsolvable case:** if $\lambda_i(A) + \lambda_j(B) = 0$ (continuous) or |
| 92 | + $\lambda_i(A)\lambda_j(B) = 1$ (discrete) for any pair, $K$ is singular and no unique solution |
| 93 | + exists. The solver returns `false` and leaves the solution unchanged. |
| 94 | +- **Ill-conditioning:** near-degenerate eigenvalue pairs yield large solution norms and reduced |
| 95 | + accuracy. Monitor the residual $\|AX + XB - C\|$ after solving. |
| 96 | +- **Symmetry:** the continuous Lyapunov solution is guaranteed symmetric only when $Q$ is |
| 97 | + symmetric; numerical symmetry should be enforced by the caller if required downstream. |
| 98 | + |
| 99 | +## Variants & Generalizations |
| 100 | + |
| 101 | +- **Bartels–Stewart algorithm:** reduces to quasi-upper-triangular (Schur) form via the double-shift |
| 102 | + QR algorithm, then back-substitutes column by column in $O(N^3)$. Preferred for $N \geq 6$. |
| 103 | +- **Smith iteration:** fixed-point iteration $X_{k+1} = AX_k A^\top + Q$ for the discrete |
| 104 | + Lyapunov equation; simple but slow for $\rho(A)$ close to 1. |
| 105 | +- **Cholesky factor updates:** propagate a Cholesky factor of $X$ for guaranteed positive-definiteness |
| 106 | + in square-root filtering (see `SquareRootKalmanFilter`). |
| 107 | + |
| 108 | +## Applications |
| 109 | + |
| 110 | +- **LQR / LQG design:** the continuous Lyapunov equation $AX + XA^\top + Q = 0$ yields the |
| 111 | + closed-loop covariance, and the DARE solution can be cross-validated via its Lyapunov residual. |
| 112 | +- **Kalman filter (observability Gramian):** $AW + WA^\top + BB^\top = 0$ gives the |
| 113 | + controllability Gramian used to evaluate reachability in balanced truncation. |
| 114 | +- **Pole-placement observers (Luenberger):** the Sylvester equation $AX - XF = LC$ arises when |
| 115 | + placing observer poles while decoupling the error dynamics. |
| 116 | +- **H₂ norm computation:** $\|G\|_2^2 = \operatorname{tr}(B^\top X B)$ where $X$ satisfies the |
| 117 | + observability Gramian Lyapunov equation. |
| 118 | + |
| 119 | +## Connections to Other Algorithms |
| 120 | + |
| 121 | +- **LU Decomposition:** the Kronecker-vectorisation approach converts any Sylvester equation into |
| 122 | + a dense linear system solved by the `LuDecomposition` solver. |
| 123 | +- **QR Decomposition:** the Bartels–Stewart and Hammarling algorithms use QR / Schur |
| 124 | + decomposition to reduce the problem to quasi-triangular form before back-substitution. |
| 125 | +- **DARE:** the discrete algebraic Riccati equation can be warm-started or verified using |
| 126 | + one step of the discrete Lyapunov equation. |
| 127 | +- **Spectral Radius:** solvability of the continuous equation requires $\lambda_i(A) + \lambda_j(A) \neq 0$; |
| 128 | + checking the spectral radius is a fast necessary condition. |
| 129 | + |
| 130 | +## References & Further Reading |
| 131 | + |
| 132 | +- Bartels, R. H. & Stewart, G. W. (1972). "Solution of the matrix equation AX + XB = C." |
| 133 | + *Communications of the ACM*, 15(9), 820–826. |
| 134 | +- Golub, G. H., Nash, S. & Van Loan, C. (1979). "A Hessenberg-Schur method for the problem |
| 135 | + AX + XB = C." *IEEE Transactions on Automatic Control*, 24(6), 909–913. |
| 136 | +- Horn, R. A. & Johnson, C. R. (1994). *Topics in Matrix Analysis*. Cambridge University Press. |
| 137 | + (Chapter 4: Kronecker products and the vec operator.) |
| 138 | +- Hammarling, S. J. (1982). "Numerical solution of the stable, non-negative definite Lyapunov |
| 139 | + equation." *IMA Journal of Numerical Analysis*, 2(3), 303–323. |
0 commit comments