An Apache Airflow data pipeline that demos an end-to-end processing job starting from a file download, to creating a Hive table then processing the data with a Spark job.
See the Airflow DAG config here
| Name | Name | Last commit date | ||
|---|---|---|---|---|
An Apache Airflow data pipeline that demos an end-to-end processing job starting from a file download, to creating a Hive table then processing the data with a Spark job.
See the Airflow DAG config here