Complete reference for both parts of the project: the Jupyter notebook implementation and the web visualizer codebase.
- Project Overview
- Notebook — Implementation Details
- Web Visualizer — Architecture
- State Model
- Data Generation (Web)
- Gradient Descent Engine (Web)
- Training Loop & Animation
- Scatter Plot Rendering
- Loss Chart Rendering
- Canvas & DPI Handling
- UI & Controls
- Deployment
- Key Differences: Notebook vs Web App
- Known Limitations & Future Work
This project has two deliverables that share the same underlying algorithm but serve different purposes:
| Notebook | Web Visualizer | |
|---|---|---|
| Language | Python (NumPy) | Vanilla JavaScript |
| Purpose | Education — understand the math deeply | Intuition — see it work visually |
| Dataset | Salary vs. experience (1–15 years) | Synthetic, x ∈ [−2, 2] |
| Scaling | Standardization (z-score) applied | Normalised to [0,1] coordinates |
| Evaluation | RMSE, R², sklearn comparison | MSE, R² displayed live |
| Extra | Learning rate experiment, residual plot | Animated fit line, true line overlay |
TRUE_SLOPE = 10_000
TRUE_INTERCEPT = 30_000
N_SAMPLES = 100
X = np.linspace(1, 15, N_SAMPLES) # Years of experience
noise = np.random.normal(loc=0, scale=8000, size=N_SAMPLES)
y = TRUE_SLOPE * X + TRUE_INTERCEPT + noise # SalaryThe true parameters are known in advance — this lets us verify whether the model recovers them correctly. The noise scale (8000) is intentionally significant to make the problem realistic.
Train/test split (manual):
indices = np.random.permutation(N_SAMPLES)
train_idx, test_idx = indices[:split_idx], indices[split_idx:]Indices are shuffled before splitting to avoid any ordering bias in the generated X (which was created with linspace — monotonically increasing).
Gradient descent on raw salary data (scale ~10,000–170,000) is unstable. Without scaling, gradients are enormous and a usable learning rate would be extremely small. Standardization fixes this:
X_mean = X_train.mean()
X_std = X_train.std()
X_train_scaled = (X_train - X_mean) / X_std # mean=0, std=1
X_test_scaled = (X_test - X_mean) / X_std # Same transform, not recomputedCritical: the mean and std are computed only on training data and then applied to the test set. Computing them on the full dataset would constitute data leakage — the test set would influence the transformation used during training.
After scaling, a learning rate of 0.01 converges cleanly in ~200–500 epochs.
class LinearRegressionScratch:
def __init__(self, learning_rate=0.01, epochs=1000):
self.lr = learning_rate
self.epochs = epochs
self.m = None # slope (w)
self.b = None # intercept
self.loss_history = []fit(X, y) — runs gradient descent:
def fit(self, X, y):
self.m = 0.0
self.b = 0.0
n = len(X)
for epoch in range(self.epochs):
y_pred = self.m * X + self.b
error = y_pred - y
dm = (2/n) * np.dot(error, X) # ∂L/∂m
db = (2/n) * np.sum(error) # ∂L/∂b
self.m -= self.lr * dm
self.b -= self.lr * db
self.loss_history.append(self.mse(y, y_pred))predict(X):
def predict(self, X):
return self.m * X + self.bmse(y, y_pred):
def mse(self, y, y_pred):
return np.mean((y - y_pred) ** 2)score(X, y) — R² score:
def score(self, X, y):
y_pred = self.predict(X)
ss_res = np.sum((y - y_pred) ** 2)
ss_tot = np.sum((y - y.mean()) ** 2)
return 1 - ss_res / ss_totStarting from MSE:
Partial derivative with respect to m:
Partial derivative with respect to b:
The factor 2/n comes from the chain rule on the squared term and averaging over n samples. In practice, the 2 is absorbed by the learning rate (scaling α by 0.5 gives equivalent results), but it's kept here for mathematical correctness.
RMSE (Root Mean Squared Error):
train_rmse = np.sqrt(model.mse(y_train, model.predict(X_train_scaled)))RMSE is in the same units as y (dollars), making it interpretable: "on average, predictions are off by $X."
R² Score:
R² = 1 - SS_res / SS_tot- R² = 1.0 → perfect fit
- R² = 0.0 → model performs as well as predicting the mean
- R² < 0 → model is worse than predicting the mean (only possible with non-training-set evaluation)
After convergence, the notebook's model achieves R² ≈ 0.97–0.99 on the training set (varies with random seed/noise).
Four learning rates are compared: [0.001, 0.01, 0.1, 0.5]
| α | Behaviour |
|---|---|
| 0.001 | Too slow — still converging at epoch 500 |
| 0.01 | Just right — smooth convergence in ~200 epochs |
| 0.1 | Fast — converges in ~50 epochs, slightly noisier |
| 0.5 | Too high — loss may oscillate or diverge |
The experiment plots loss curves and fitted lines side by side, making the instability of high learning rates immediately visible.
The notebook compares the scratch implementation against sklearn.LinearRegression, which uses the Normal Equation (closed-form analytical solution):
This is not gradient descent — it finds the exact optimum in one step. The comparison validates that the gradient descent implementation converges to the correct solution:
print(f"Slope (m): My Model = {model.m:,.4f} | Sklearn = {sklearn_model.coef_[0]:,.4f}")
print(f"Intercept (b): My Model = {model.b:,.4f} | Sklearn = {sklearn_model.intercept_:,.4f}")Results should match to 2–3 decimal places (minor differences due to numerical precision in gradient descent vs. exact closed form).
The web visualizer is a single-page application with no dependencies:
index.html → 3-column layout: left sidebar, center canvas, right loss chart
style.css → CSS custom properties, responsive layout, slider styling
script.js → All logic: data, gradient descent, training loop, canvas rendering
vercel.json → Deployment headers and routing
External resources: Google Fonts only (DM Serif Display, DM Sans, JetBrains Mono).
All application state lives in a single state object:
const state = {
data: { train: [], test: [], all: [] }, // Normalised point arrays
model: { w: 0, b: 0 }, // Current learned parameters
lossHistory: [], // MSE per epoch
isTraining: false, // Lock flag
animFrame: null, // rAF handle
trueW: 2.1, // Ground truth slope (randomized on init)
trueB: 0.8, // Ground truth intercept (randomized on init)
};Point object shape:
{ x: 0.42, y: 1.73 } // Raw values in data space, not normalised to canvasx values are sampled from [-2, 2], y values follow trueW * x + trueB + noise.
On init, trueW and trueB are slightly randomized:
state.trueW = 1.5 + (Math.random() - 0.5) * 1.5;
state.trueB = (Math.random() - 0.5) * 1.5;This ensures the visualizer shows a different line each time the page loads, keeping it fresh.
function generateData() {
const n = parseInt(els.numPoints.value); // 20–300
const noise = parseFloat(els.noiseLevel.value); // 0–1
const split = parseFloat(els.trainSplit.value); // 0.5–0.9
for (let i = 0; i < n; i++) {
const x = (Math.random() * 4) - 2; // x ∈ [−2, 2]
const y = trueW * x + trueB + randn() * noise * 2.5;
points.push({ x, y });
}
}Box-Muller transform for Gaussian noise:
const randn = () => {
let u = 0, v = 0;
while (u === 0) u = Math.random();
while (v === 0) v = Math.random();
return Math.sqrt(-2 * Math.log(u)) * Math.cos(2 * Math.PI * v);
};This converts two uniform random samples into a standard normal (mean=0, std=1). The while (u===0) guards against log(0).
The noise strength is randn() * noise * 2.5 — the noise slider (0–1) scales the noise σ, and the 2.5 multiplier gives a useful visual range of scatter.
Points are shuffled and split by index — no sorting or stratification.
Three pure functions implement the math:
Loss (MSE):
function computeLoss(pts, w, b) {
let sum = 0;
for (const p of pts) {
const diff = (w * p.x + b) - p.y;
sum += diff * diff;
}
return sum / pts.length;
}R² Score:
function computeR2(pts, w, b) {
const yMean = pts.reduce((s, p) => s + p.y, 0) / pts.length;
let ssTot = 0, ssRes = 0;
for (const p of pts) {
ssTot += (p.y - yMean) ** 2;
ssRes += (p.y - (w * p.x + b)) ** 2;
}
return 1 - ssRes / ssTot;
}One gradient step:
function gradientStep(pts, w, b, lr) {
const n = pts.length;
let dw = 0, db = 0;
for (const p of pts) {
const err = (w * p.x + b) - p.y;
dw += err * p.x;
db += err;
}
dw = (2 / n) * dw;
db = (2 / n) * db;
return { w: w - lr * dw, b: b - lr * db };
}These are pure functions — they don't touch state directly, making them easy to reason about and test.
Initialization options:
| Method | w | b |
|---|---|---|
| Zeros | 0 | 0 |
| Random | Uniform in [−1, 1] | Uniform in [−1, 1] |
Random initialization lets users see that gradient descent reaches the same optimum regardless of starting point (for convex loss surfaces like MSE).
Training uses requestAnimationFrame to stay non-blocking — the browser can repaint between gradient steps, making the animation smooth:
const BATCH_SIZE = 10; // Gradient steps per animation frame
function step() {
for (let i = 0; i < BATCH_SIZE && epoch < epochs; i++, epoch++) {
const updated = gradientStep(train, state.model.w, state.model.b, lr);
state.model.w = updated.w;
state.model.b = updated.b;
state.lossHistory.push(computeLoss(train, state.model.w, state.model.b));
}
// Update UI, draw canvases
drawScatter(true);
drawLoss();
if (epoch < epochs) {
trainingQueue = requestAnimationFrame(step);
} else {
// Finalise
}
}
trainingQueue = requestAnimationFrame(step);Why BATCH_SIZE = 10? Each rAF callback runs ~16ms (60fps target). Running 10 gradient steps per frame gives ~600 steps/second — fast enough to train 200 epochs in under a second visually, while still showing smooth animation.
Stop mechanism:
function stopTraining() {
if (trainingQueue) cancelAnimationFrame(trainingQueue);
state.isTraining = false;
}cancelAnimationFrame prevents the next step() call from being queued, immediately halting the loop.
drawScatter(showLine) renders the center canvas. Key design decisions:
Auto-scaling axes: bounds are computed from the actual data range + 12% padding on each side:
const xPad = (xMax - xMin) * 0.12 + 0.3;
const yPad = (yMax - yMin) * 0.12 + 0.3;The + 0.3 minimum ensures padding even for very tight datasets.
Coordinate mapping:
const toX = v => pad.left + ((v - xMin) / (xMax - xMin)) * pw;
const toY = v => pad.top + (1 - (v - yMin) / (yMax - yMin)) * ph;
// Note: Y is flipped — canvas origin is top-left, math origin is bottom-leftPoint radius scaling: smaller points for dense datasets:
const r = Math.max(3, Math.min(5, 200 / allPts.length));Two regression lines drawn when model is active:
- Fitted line (solid green) — current
state.model.wandstate.model.b - True line (dashed grey) —
state.trueWandstate.trueB
The true line lets users visually see how close the learned model is to the ground truth parameters.
Colour scheme:
| Element | Colour |
|---|---|
| Train points | #2B4C7E (navy blue) |
| Test points | #D97B4A (orange) |
| Fitted line | #1A9E6B (green) |
| True line | #E8E4DE (dashed grey) |
drawLoss() renders the right-panel loss curve manually on #lossCanvas.
Auto-scaled Y axis:
yMin = Math.max(0, yMin - yRange * 0.08); // Never below 0
yMax = yMax + yRange * 0.12; // 12% headroomGradient fill under curve:
const grd = ctx.createLinearGradient(0, pad.top, 0, y0);
grd.addColorStop(0, 'rgba(139,61,190,.34)'); // Purple, opaque at top
grd.addColorStop(1, 'rgba(139,61,190,.02)'); // Nearly transparent at baselineLoss summary stats (shown after training completes):
const reduction = ((1 - fin / init) * 100).toFixed(1) + '%';Shows initial MSE, final MSE, and percentage reduction — a quick intuitive measure of how much training helped.
Both canvases use devicePixelRatio scaling for crisp rendering on retina screens:
function resizeCanvas(canvas, wrap) {
const dpr = window.devicePixelRatio || 1;
const rect = wrap.getBoundingClientRect();
canvas.width = rect.width * dpr;
canvas.height = rect.height * dpr;
canvas.style.width = rect.width + 'px';
canvas.style.height = rect.height + 'px';
const ctx = canvas.getContext('2d');
ctx.scale(dpr, dpr); // All subsequent drawing is in CSS pixels
return { ctx, w: rect.width, h: rect.height };
}resizeCanvas is called at the start of every draw call — not just on resize. This means canvases always reflect the current layout dimensions, at the correct DPI, before any drawing occurs.
On window.resize, drawing is debounced with an 80ms timer to avoid excessive redraws while the user is actively resizing.
All sliders call generateData() immediately on input for dataset sliders (points, noise, split) so the scatter plot updates live. Training sliders (epochs, learning rate) only update their display value — they don't retrain automatically.
els.numPoints.addEventListener('input', () => {
els.numPointsVal.textContent = els.numPoints.value;
generateData(); // Immediate live update
});Three states with distinct CSS classes:
| State | Class | Shown when |
|---|---|---|
| Ready | .ready |
App loaded, no training in progress |
| Training | .training |
requestAnimationFrame loop is running |
| Complete | .done |
All epochs finished |
A simple CSS width transition on #progressBar:
els.progressBar.style.width = pct + '%';Updated every BATCH_SIZE epochs (every animation frame).
vercel.json:
{
"cleanUrls": true,
"trailingSlash": false,
"headers": [
{
"source": "/(.*)",
"headers": [
{ "key": "X-Content-Type-Options", "value": "nosniff" },
{ "key": "X-Frame-Options", "value": "DENY" },
{ "key": "Cache-Control", "value": "public, max-age=86400" }
]
}
]
}The live deployment is at lin-reg-scratch.vercel.app.
| Aspect | Notebook | Web App |
|---|---|---|
| X range | 1–15 (years of experience) | −2 to 2 (abstract) |
| Y range | ~30,000–180,000 (salary) | Varies with trueW/trueB |
| Feature scaling | Z-score standardization applied | Not applied (x already small) |
| Loss computed on | Training set only during fit | Training set only during fit |
| Initialization | Always zeros | Zeros or random (user choice) |
| Evaluation shown | RMSE, R² on both sets | MSE, R² on both sets (live) |
| True line shown | Dashed on matplotlib scatter | Dashed grey overlay on canvas |
| Sklearn comparison | Yes (section 9) | No |
| Area | Current State | Possible Improvement |
|---|---|---|
| Regression type | Simple (1 feature) | Multiple linear regression |
| Loss function | MSE only | Add MAE, Huber loss options |
| Data | Generated only | CSV upload for real datasets |
| Initialisation | Zeros or uniform random | Xavier / He initialization |
| Regularisation | None | L1 (Lasso), L2 (Ridge) toggles |
| Polynomial features | No | Degree selector (1–4) |
| Residual plot | Notebook only | Add residual view to web app |
| Feature scaling | Not in web app | Toggle standardization |
| Notebook | Static | Add interactive widgets (ipywidgets) |
Documentation written for Linear Regression from Scratch — © Umair Zia