🤖 AI Engineer With Python Portfolio Project

A production-ready AI engineering project covering: Python · FastAPI · PyTorch · Scikit-learn · NLP · MLflow · Docker · AWS ECS

---

📁 Project Structure

ai-engineer-project/
├── app/
│   ├── main.py                  # FastAPI app entry point
│   ├── config.py                # Env-based configuration
│   └── routers/
│       ├── sentiment.py         # NLP: sentiment analysis (VADER / HuggingFace)
│       ├── classification.py    # Scikit-learn: Iris classifier + MLflow
│       └── regression.py        # PyTorch: London house price prediction
├── pipelines/
│   ├── data_pipeline.py         # ETL: ingest → clean → feature engineer → split
│   └── airflow_dag.py           # Airflow DAG (or standalone runner)
├── mlops/
│   ├── experiments.py           # MLflow experiment tracking & hyperparameter search
│   └── monitoring.py            # Data drift (KS test, PSI) + prediction logging
├── tests/
│   └── test_suite.py            # pytest: API + pipeline + monitoring tests
├── infra/
│   └── aws_deploy.py            # AWS ECS Fargate + ECR + CloudWatch deployment
├── Dockerfile
├── docker-compose.yml           # API + MLflow + Redis
└── requirements.txt

---

🚀 Quick Start (Local — no Docker)

# 1. Install dependencies
pip install -r requirements.txt

# 2. Start the API
uvicorn app.main:app --reload --port 8000

# 3. Open interactive docs
open http://localhost:8000/docs

---

🐳 Docker Compose (Recommended — includes MLflow + Redis)

# Start everything
docker-compose up --build

# API:    http://localhost:8000/docs
# MLflow: http://localhost:5000

# Run the data pipeline
docker-compose --profile pipeline up pipeline

# Run experiments
docker-compose --profile experiments up experiments

---

🔌 API Endpoints

Method	Endpoint	Description
GET	/health	Health check
GET	/docs	Interactive Swagger UI
POST	/api/v1/nlp/sentiment	Sentiment analysis (VADER / HuggingFace)
POST	/api/v1/nlp/batch-sentiment	Batch NLP inference
GET	/api/v1/classify/train	Retrain Iris classifier
POST	/api/v1/classify/iris	Classify iris flower
POST	/api/v1/classify/iris/batch	Batch classification
GET	/api/v1/predict/train	Train PyTorch house price model
POST	/api/v1/predict/house-price	Predict London house price

Example — Sentiment Analysis

curl -X POST http://localhost:8000/api/v1/nlp/sentiment \
  -H "Content-Type: application/json" \
  -d '{"text": "This product is absolutely fantastic!", "model": "vader"}'

Example — Iris Classification

curl -X POST http://localhost:8000/api/v1/classify/iris \
  -H "Content-Type: application/json" \
  -d '{"sepal_length": 5.1, "sepal_width": 3.5, "petal_length": 1.4, "petal_width": 0.2}'

Example — House Price Prediction (PyTorch)

curl -X POST http://localhost:8000/api/v1/predict/house-price \
  -H "Content-Type: application/json" \
  -d '{
    "sqft": 1500, "bedrooms": 3, "bathrooms": 2.0,
    "age_years": 15, "garage": true, "garden": true, "location_score": 7.5
  }'

---

🧪 Tests

pytest tests/test_suite.py -v

---

📊 MLflow Experiment Tracking

# Start MLflow server locally
mlflow server --host 0.0.0.0 --port 5000

# Run experiments
python mlops/experiments.py

# View UI at http://localhost:5000

---

🔄 Data Pipeline

# Run the full ETL pipeline
python pipelines/data_pipeline.py

# Or the Airflow DAG standalone (no Airflow needed)
python pipelines/airflow_dag.py

---

📉 Model Monitoring

python mlops/monitoring.py

Outputs a drift report using KS test and PSI (Population Stability Index).

---

☁️ AWS Deployment (ECS Fargate)

# Set your AWS credentials
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_REGION=eu-west-2

# Build + push to ECR, register task, create ECS service
python infra/aws_deploy.py

Architecture: Route 53 → ALB → ECS Fargate → ECR image Models stored in S3, tracking via CloudWatch alarms.

---

🛠️ Skills Demonstrated

Category	Technologies
Python / ML	NumPy, Pandas, Scikit-learn, SciPy
Deep Learning	PyTorch (MLP with BatchNorm + Dropout)
NLP	VADER sentiment, HuggingFace Transformers
APIs	FastAPI, Pydantic v2, async endpoints
MLOps	MLflow tracking, model registry, experiment comparison
Data pipelines	ETL, feature engineering, Parquet, SQLite
Monitoring	KS drift test, PSI, prediction logging
Orchestration	Airflow DAG (also runnable standalone)
Cloud	AWS ECR, ECS Fargate, S3, CloudWatch, CloudFormation
Containerisation	Docker, Docker Compose
Testing	pytest, FastAPI TestClient
Databases	SQLite (dev), S3 (model storage), Redis (cache)