ROADMAP.md

ML from Scratch — Roadmap

Following roadmap.sh/machine-learning

---

1. Prerequisites

Math Foundations

• Linear Algebra (vectors, matrices, dot product, eigenvalues)
• Calculus (derivatives, partial derivatives, chain rule, gradients)
• Statistics & Probability (distributions, Bayes theorem, descriptive & inferential stats)

Python & Libraries

• NumPy
• Pandas
• Matplotlib / Seaborn

---

2. Data Preprocessing

• Handling missing values
• Feature scaling & normalization (StandardScaler, MinMaxScaler)
• Feature engineering & selection
• Encoding categorical variables (Label Encoding, One-Hot Encoding)
• Dimensionality reduction (PCA)

---

3. Supervised Learning — Regression

• Simple Linear Regression
• Multiple Linear Regression
• Ridge Regression
• Lasso Regression
• ElasticNet Regression

---

4. Supervised Learning — Classification

• KNN (K-Nearest Neighbors)
• Logistic Regression
• SVM (Support Vector Machine)
• Decision Trees
• Random Forest
• Gradient Boosting (XGBoost, LightGBM)
• Naive Bayes

---

5. Model Evaluation

• Confusion Matrix
• Accuracy, Precision, Recall, F1-Score
• ROC-AUC Curve
• Cross Validation (K-Fold, LOOCV)
• Bias-Variance Tradeoff

---

6. Unsupervised Learning

• K-Means Clustering
• Hierarchical Clustering
• DBSCAN
• PCA (Principal Component Analysis)

---

7. Deep Learning

• Perceptron & MLPs
• Activation Functions (Sigmoid, Softmax)
• Forward & Back Propagation
• Loss Functions (BCE, Categorical Cross-Entropy)
• Optimizers from scratch (SGD, Momentum, Adam)
• End-to-end project: Fashion-MNIST classifier (87.62% test accuracy)
• CNNs (Convolutional Neural Networks)
• RNNs (Recurrent Neural Networks)
• LSTMs & GRUs
• Attention & Transformers

---

8. NLP (Natural Language Processing)

• Tokenization
• Stemming & Lemmatization
• TF-IDF
• Word Embeddings (Word2Vec, GloVe)
• Transformers for NLP

---

9. Reinforcement Learning

• Q-Learning
• Deep Q-Networks
• Policy Gradient
• Actor-Critic