Source: The Palindrome by Tivadar Danka
Published: Aug 06, 2025
Subtitle: A complete guide to linear algebra, calculus, and probability theory
Machine learning is built upon three pillars: linear algebra, calculus, and probability theory.
Predictive models such as neural networks are described using linear algebraic concepts, such as matrix multiplication. Linear algebra is the most important topic for machine learning engineers working on real-life problems.
A set of vectors V is a vector space over the real numbers if you can add and scale vectors in a straightforward way. You can think of each point in the plane as a tuple x = (x1, x2), represented by an arrow pointing from the origin to (x1, x2).
Algebraic operations:
Visual addition:
Euclidean space notation:
A normed space provides tools for measuring distance in vector spaces.
3D extension:
General norm properties:
p-norm family:
Supremum norm:
Distance definition:
Manhattan distance visual:
Inner product properties:
Euclidean inner product:
Inner product to norm:
Standard basis vectors:
Basis span property:
Linear transformation properties:
Matrix definitions:
Multiplication formula:
Visual multiplication guide:
Diagonal matrix computation:
Derivative limit definition:
Tangent line visual:
Second derivative test:
Chain rule:
Fundamental theorem of calculus:
Integration area visual:
Surface plot visual:
Partial derivative definitions:
Tangent plane visual:
Gradient vector definition:
Linear layer function:
Vector function components:
Total derivative matrix:
Linear component decomposition:
Inner product decomposition:
Multivariate chain rule:
Optimization problem:
Hessian matrix:
Event space Omega:
Probability axioms:
Set operations visual:
Conditional probability:
Bayes' theorem:
Discrete expected value:
Summation formula:
Continuous expected value:
LLN formula:
Discrete entropy:
Continuous entropy:
Cross-entropy loss:
KL Divergence:
- The Mathematics of Machine Learning by Tivadar Danka — A single resource with a full breakdown of the entire roadmap (linear algebra, calculus, probability theory)
- Graph Theory Track
- Foundations of Mathematics Track
- Neural Networks from Scratch Track
- Subscribe to The Palindrome Premium
All resources compiled from: The Roadmap of Mathematics for Machine Learning — The Palindrome