🎬 IMDb Top 250 Movies Web Scraper

An end-to-end Web Scraping project built using Python, Selenium, BeautifulSoup, and Pandas to extract IMDb's Top 250 Movies data. The scraper automates browser interactions, parses dynamically loaded content, cleans the extracted data, and exports it into a structured dataset for further analysis.

---

📌 Project Overview

This project demonstrates how to scrape data from a dynamic website using Selenium and BeautifulSoup. It automates navigation to IMDb's Top 250 Movies page, extracts movie information, performs basic data cleaning, and stores the results in a Pandas DataFrame.

This project is ideal for learning:

• Web Scraping
• Browser Automation
• HTML Parsing
• Data Cleaning
• Data Extraction from Dynamic Websites

---

🚀 Features

• Automated browser interaction using Selenium
• Scrapes IMDb Top 250 Movies
• Parses HTML using BeautifulSoup
• Extracts structured movie information
• Cleans scraped data using Pandas
• Exports the dataset for analysis
• Well-structured and beginner-friendly notebook

---

🛠️ Technologies Used

• Python
• Selenium
• BeautifulSoup (bs4)
• Pandas
• Chrome WebDriver
• Jupyter Notebook

---

📊 Extracted Data

The scraper extracts the following information for each movie:

Column	Description
Rank	IMDb Ranking
Title	Movie Title
Release Year	Year of Release
Duration	Runtime
Rating	IMDb Rating
Review	Total Number of User Ratings

---

📁 Project Structure

web_scrapping_imdb_250/
│
├── notebook/
│   └── imdb_top250.ipynb
│
└── README.md

---

⚙️ Installation

Clone the repository

git clone https://github.com/your-username/web_scrapping_imdb_250.git

Navigate to the project folder

cd web_scrapping_imdb_250

Install the required libraries

pip install selenium beautifulsoup4 pandas lxml

---

▶️ How to Run

1. Open the project in Jupyter Notebook.
2. Open:

notebook/imdb_top250.ipynb

3. Run all notebook cells sequentially.
4. The scraper will:
   • Launch Chrome
   • Navigate to IMDb
   • Open the Top 250 Movies page
   • Extract movie details
   • Clean the data
   • Display the final dataset

---

🔄 Workflow

Launch Chrome
      │
      ▼
Open IMDb Website
      │
      ▼
Navigate to Top 250 Movies
      │
      ▼
Load Movie List
      │
      ▼
Extract Page Source
      │
      ▼
Parse HTML using BeautifulSoup
      │
      ▼
Extract Movie Details
      │
      ▼
Clean Data
      │
      ▼
Create Pandas DataFrame

---

🧹 Data Cleaning

The notebook performs basic cleaning operations, including:

• Removing "#" from movie rankings
• Removing unnecessary parentheses from review counts
• Organizing extracted data into a structured DataFrame

---

📚 Skills Demonstrated

• Web Scraping
• Selenium Automation
• BeautifulSoup
• HTML Parsing
• XPath
• Dynamic Web Page Handling
• Data Cleaning
• Pandas
• Data Extraction
• Browser Automation

---

📷 Sample Output

Rank	Title	Release Year	Duration	Rating	Reviews
1	The Shawshank Redemption	1994	2h 22m	9.3	3M
2	The Godfather	1972	2h 55m	9.2	2.2M

---

💡 Future Improvements

• Export data to CSV and Excel
• Store data in SQL databases
• Add movie genres
• Extract directors and cast
• Include movie poster URLs
• Schedule automated scraping
• Implement retry logic for robust scraping

---

⚠️ Disclaimer

This project is intended for educational and learning purposes only. Please ensure that your use of IMDb data complies with IMDb's Terms of Service and robots.txt guidelines.

---

👨‍💻 Author

Md Ainain Ahmed

Aspiring Data Analyst | Python | SQL | Power BI | Web Scraping

GitHub: https://github.com/ainainDev