Profit Prediction using Multiple Linear Regression

Predicting startup profit from R&D, Administration, and Marketing spend using OLS regression with Backward Elimination for feature selection.

---

Problem Statement

Given a dataset of 50 startups with expenditure across R&D, Administration, and Marketing departments (across California, Florida, and New York), the goal is to build a regression model that accurately predicts the net profit of a company.

This project also explores which spending categories are statistically significant using Backward Elimination on p-values, and compares model performance before and after feature selection.

---

Dataset

Feature	Description
RND	Research & Development expenditure
ADMIN	Administration expenditure
MKT	Marketing expenditure
STATE	State of operation — California, Florida, New York (categorical)
PROFIT	Target variable

• Rows: 50 | Features: 4 (+ 1 target)
• Missing values: 1 missing value in MKT — imputed using column mean
• Profit range: $14,681 – $192,261 | Mean profit: ~$112,013

---

Approach

1. Data Preprocessing

• Custom replacer() function: numerical columns filled with mean, categorical with mode
• One-Hot Encoding on STATE with drop_first=True to avoid the Dummy Variable Trap
• Features standardized using StandardScaler

2. Full Model — OLS Regression (statsmodels)

• All 5 features used: RND, ADMIN, MKT, STATE_Florida, STATE_New York
• OLS summary analyzed for p-values, R², Adjusted R², F-statistic

3. Backward Elimination

• Started with all features
• Removed ADMIN (p-value = 0.596, highest and > 0.05)
• STATE_Florida (p = 0.989) and STATE_New York (p = 0.885) also statistically insignificant but retained for comparison
• Final significant predictors: RND (p < 0.001) and MKT (p = 0.048)

4. Model Evaluation

• Train/test split: 70% train / 30% test (random_state=21)
• Evaluated using MAE and R² on test data

---

Results

OLS Summary — Full Model (on entire dataset)

Metric	Value
R-squared	0.952
Adj. R-squared	0.946
F-statistic	172.7 (p = 9.46e-28)

P-values — Full Model

Feature	Coefficient	p-value	Significant?
const	112,200	0.000	✅
RND	36,090	0.000	✅
ADMIN	-758.46	0.596	❌ Eliminated
MKT	3,854	0.080	⚠️ Borderline
STATE_Florida	7.26	0.998	❌
STATE_New York	-456.06	0.889	❌

Performance Comparison (Test Set)

Model	MAE	R²
Before Backward Elimination	5,319.46	0.96
After Backward Elimination (ADMIN removed)	5,489.13	0.96

Key Finding: Removing ADMIN did not reduce R² at all (both 0.96), confirming it had no meaningful contribution to profit prediction. RND is by far the strongest predictor with a coefficient of ~36,000 and p < 0.001 — every 1-unit increase in scaled R&D spend associates with ~₹36,000 increase in profit. State of operation had virtually no effect on profit.

---

Tech Stack

---

Project Structure

profit-prediction-multiple-linear-regression/
│
├── data/
│   └── 50_Startups.csv                    # Dataset
│
├── notebooks/
│   ├── profit_prediction.ipynb            # Base MLR model
│   └── profit_prediction_backward_elim.ipynb  # OLS + Backward Elimination
│
├── requirements.txt
└── README.md

---

How to Run

# 1. Clone the repository
git clone https://github.com/divyajagtap28/profit-prediction-multiple-linear-regression.git
cd profit-prediction-multiple-linear-regression

# 2. Install dependencies
pip install -r requirements.txt

# 3. Open notebooks
jupyter notebook notebooks/

---

Key Concepts Demonstrated

• Multiple Linear Regression — modeling profit as a linear function of multiple inputs
• OLS (Ordinary Least Squares) — minimizing residual sum of squares; provides p-values and confidence intervals unlike sklearn
• Backward Elimination — iterative removal of features with p-value > 0.05
• Dummy Variable Trap — avoided using drop_first=True in One-Hot Encoding
• StandardScaler — normalized features for fair coefficient comparison
• R² vs Adjusted R² — Adj. R² penalizes unnecessary features; more reliable for model comparison

---

What I Learned

• A model with fewer features can match a full model's performance — simpler is better
• OLS summary provides far richer statistical insight than sklearn's .score() alone
• State of operation had near-zero impact on profit; R&D investment is the dominant driver
• Comparing MAE before vs after elimination is a practical way to validate feature selection decisions

---

Author

Divya Jagtap
Second Year Computer Engineering | PCCOER, Pune