An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Azure Data Lake. Other processing takes place on Azure Data Factory, Azure Synapse and Tableau.
This project builds an End-to-End Azure Data Engineering Pipeline, performing ETL and Analytics Reporting on the AdventureWorks2017LT Database.
A comprehensive ETL pipeline and sales analysis project leveraging Microsoft Azure and PySpark, designed to optimize e-commerce sales by providing actionable insights through detailed data analysis.
End-to-end Azure Databricks retail data engineering project using Medallion Architecture (Bronze, Silver, Gold). Implements Auto Loader, Unity Catalog, Delta Lake, SCD Type 1 & 2 dimensions, and Fact Orders for analytics-ready star schema modeling.
Formula 1 race data engineering project which utilises azure services and databricks to ingest and analyse the data.
This project builds an End-to-End Azure Data Engineering Pipeline, performing ETL and Analytics Reporting on the AdventureWorks2022LT Database.
Tokyo-olympic-azure-data-engineering-end-to-end-project
This project demonstrates an end-to-end data pipeline built on cloud technologies, following the Medallion Architecture (Bronze–Silver–Gold). . Ingests raw data from cloud sources, processes via ETL/ELT in Databricks & DBT, orchestrated with Data Factory, and delivers clean, analytics-ready datasets stored in a scalable data lake.
Azure pipeline for data analytics on Tokyo Olympics data
This project builds an End-to-End Azure Data Engineering Pipeline, performing ETL and Analytics Reporting on the AdventureWorks2022LT Database.
Real-Time Stock-Market Data Streaming Using Kafka
Conducted comprehensive data analysis on Retail Sales data leveraging Azure Core Services, Azure Databricks and Power BI
Azure End-to-End Data Engineering Project
Data Engineering Project - Python, PySpark & SQL - Azure Data Factory (ADF), DataBricks, Synapse Analytics, Azure Data Lake Storage (ADLS) Gen2, Power BI, Tableau and Looker Studio
Formula1 ADF pipeline
End-to-end Azure data engineering and analytics pipeline for NHTSA vehicle complaints, with a Ford case study using ADF, Synapse Serverless SQL, and Power BI.
Data Engineering Project using Tokyo Olympic Data
A lightweight toolkit for Azure Data Lake Storage Gen2 operations, featuring AzCopy commands and Databricks integration examples. Includes sample data and notebooks for quick experimentation with data lake architectures.
A cloud-based end-to-end data engineering project built on Microsoft Azure, implementing a Medallion Architecture for Healthcare Revenue Cycle Management (RCM) data.
An end-to-end data engineering pipeline that fetches data from the BingAPI, cleans and transforms it with Azure Databricks.Sentiment Analysis is performed in AzureML and the data is visualized using Tableau.
To associate your repository with the azure-data-lake-gen2 topic, visit your repo's landing page and select "manage topics."