Andre Rodrigues – DBA | DBRE | Data Engineer
Passionate about Automation, Performance, and Reliability
CI/CD workflow for ETL pipelines using Apache Hop, Jenkins, Docker, and GitHub
This project demonstrates a complete CI/CD workflow using Apache Hop, Jenkins, and Docker. The main goal is to provide an automated, reproducible, and scalable environment for ETL pipeline deployment and execution.
| Component | Purpose |
|---|---|
| Apache Hop | Data pipeline orchestration |
| Jenkins | CI/CD automation |
| Docker | Containerised environment |
| GitHub | Version control |
GitHub → Jenkins → Docker Build → Apache Hop Pipeline
Every push to the repository triggers the Jenkins pipeline, validates the workflow, and deploys the latest container version.
- Automated deployments
- Version control and traceability
- Reproducible environments
- Fast rollback and recovery
- Easy CI/CD integration
This project can be extended with monitoring dashboards using Grafana, Prometheus, and Percona dashboards for performance visibility and alerting.
- Kubernetes / Argo CD integration
- Cloud deployment (AWS / Azure)
- Automated testing stage
- Performance metrics dashboards
Andre Rodrigues
DBA | DBRE | Data Engineer