|
1 | | -# Stream-Seat: Microservices Migration to AWS Fargate |
| 1 | +# Stream-Seat: Cloud-Native Microservices Migration |
2 | 2 |
|
3 | | -## Project Overview |
4 | | -**Stream-Seat** is a distributed microservices platform designed for real-time ticket booking. This project documents the transition from a localized development environment to a production-grade, serverless **AWS Cloud** infrastructure. |
| 3 | +## Project Overview |
| 4 | +**Stream-Seat** is a distributed microservices platform designed for real-time ticket booking. This project documents the transition from a localized development environment to a production-grade, serverless **AWS Cloud** infrastructure, focusing on high availability and secure service orchestration. |
5 | 5 |
|
6 | | -## Cloud Architecture (The "80%" Milestone) |
7 | | -The project has successfully moved into advanced orchestration, shifting from manual IP-based networking to automated DNS-based Service Discovery and isolated data tier management. |
| 6 | +--- |
| 7 | + |
| 8 | +## Cloud Architecture (The "100% Infrastructure Milestone") |
| 9 | +The project has successfully reached full infrastructure deployment, transitioning from manual task management to a fully routed, discovery-aware microservices ecosystem. |
8 | 10 |
|
9 | 11 | ### Current Infrastructure Components: |
10 | | -* **Compute:** AWS Fargate (Serverless) utilizing the **AMD64** runtime environment. |
11 | | -* **Service Discovery:** Netflix Eureka Server facilitating internal service-to-service communication via Private VPC routing. |
12 | | -* **Database:** Amazon RDS (MySQL) deployed within private subnets for enhanced data security. |
13 | | -* **Registry:** Amazon ECR for multi-region container image management and distribution. |
14 | | -* **Networking:** Custom VPC with a multi-AZ, split-subnet architecture (Public/Private). |
| 12 | +* **Compute:** AWS ECS Fargate (Serverless) utilizing the **AMD64** runtime environment. |
| 13 | +* **Service Discovery:** **AWS Cloud Map** with Private DNS Namespaces (`stream-seat.local`) for stable, hostname-based internal routing. |
| 14 | +* **API Gateway:** **Spring Cloud Gateway** deployed in public subnets as the unified entry point for all external traffic. |
| 15 | +* **Database:** Amazon RDS (MySQL 8.0) isolated in private subnets with administrative access via **EC2 Bastion Hosts**. |
| 16 | +* **Networking:** Custom VPC with multi-AZ redundancy and **Security Group Nesting** for zero-trust inter-service communication. |
15 | 17 |
|
16 | 18 |
|
17 | 19 |
|
18 | 20 | --- |
19 | 21 |
|
20 | | -## Technical Challenges & Solutions |
| 22 | +## 🛠️ Technical Challenges & Solutions |
21 | 23 |
|
22 | 24 | ### 1. Cross-Platform Container Compatibility |
23 | | -* **Issue:** `CannotPullContainerError: image Manifest does not contain descriptor matching platform 'linux/amd64'`. |
24 | | -* **Cause:** Development on Apple Silicon (ARM64) created an architecture mismatch when deploying to standard AMD64 Fargate nodes. |
25 | | -* **Fix:** Implemented **Docker multi-platform builds** using the `--platform linux/amd64` flag. This ensured that images built on ARM64 hardware remained fully compatible with cloud-native x86_64 runtimes. |
26 | | - |
27 | | -### 2. Private Database Orchestration |
28 | | -* **Issue:** `SQLGrammarException: Unknown database` and connection timeouts during service startup. |
29 | | -* **Cause:** Strict VPC isolation prevented external schema initialization, and the application required a pre-existing schema for Hibernate DDL execution. |
30 | | -* **Fix:** Implemented **Security Group Nesting** to allow traffic specifically from the Application Security Group (`app-sg`) to the Database Security Group (`rds-sg`). Leveraged Spring Boot’s **Externalized Configuration** to map the JDBC connection to the default `mysql` schema for initial VPC handshake validation. |
31 | | - |
32 | | -### 3. Internal Service Discovery Handshake |
33 | | -* **Issue:** Microservices successfully reached a `RUNNING` status but failed to appear in the Eureka registry. |
34 | | -* **Cause:** Communication was blocked by default Security Group ingress rules and hindered by the dynamic nature of Fargate Private IPs. |
35 | | -* **Fix:** Established a **Self-Referencing Security Group rule** for port 8761 and updated environment variables to use current **VPC Private IPs**. This ensured all registry traffic stayed within the AWS internal network, eliminating data egress costs and minimizing latency. |
36 | | - |
37 | | -### 4. Ephemeral IP Management & Service Discovery |
38 | | -* **Issue:** Fargate tasks receive new Private IPs on every restart, forcing manual updates to the EUREKA_CLIENT_SERVICEURL_DEFAULTZONE variable. |
39 | | -* **Fix:** Implemented AWS Cloud Map. By creating a Private DNS Namespace and linking the Eureka service to an A record (eureka.stream-seat.local), microservices now find the registry via a consistent hostname regardless of underlying IP changes. |
40 | | - |
41 | | -### 5. MySQL 8.0 "Access Denied" for System Schemas |
42 | | -* **Issue:** Access to data dictionary table 'mysql.events' is rejected. |
43 | | -* **Cause:** Application tables were being created in the mysql system schema, which is restricted in MySQL 8.0+ for security reasons. |
44 | | -* **Fix:** Utilized a Bastion Host (EC2) to perform remote administrative tasks. Created a dedicated stream_seat_db application schema, isolating application data from system metadata and adhering to security best practices. |
| 25 | +* **Issue:** `CannotPullContainerError` due to ARM64 (Apple Silicon) vs. AMD64 (AWS Fargate) architecture mismatch. |
| 26 | +* **Fix:** Implemented **Docker multi-platform builds** using the `--platform linux/amd64` flag to ensure binary compatibility across development and cloud environments. |
| 27 | + |
| 28 | +### 2. Private Database Orchestration & Schema Initialization |
| 29 | +* **Issue:** Security isolation prevented RDS schema creation, causing `SQLGrammarException` during service startup. |
| 30 | +* **Fix:** Utilized a **Bastion Host (EC2)** to bridge into the private subnet. Manually initialized the `stream_seat_db` schema to support Hibernate's DDL execution while maintaining a restricted database attack surface. |
| 31 | + |
| 32 | +### 3. Non-Routable Link-Local Address Conflict |
| 33 | +* **Issue:** Fargate tasks registered with Eureka using `169.254.x.x` addresses, which are non-routable between different tasks, causing "Connection Refused" errors at the Gateway. |
| 34 | +* **Fix:** Configured **Spring Cloud InetUtils** via environment variables (`SPRING_CLOUD_INETUTILS_PREFERRED_NETWORKS=10.0`) to force services to register using their routable VPC Private IPs. |
| 35 | + |
| 36 | +### 4. Ephemeral IP Management |
| 37 | +* **Issue:** Dynamic Fargate IPs broke hardcoded service links on every task restart, leading to frequent manual configuration updates. |
| 38 | +* **Fix:** Integrated **AWS Cloud Map** to provide a persistent Private DNS record (`eureka.stream-seat.local`), decoupling the registry location from its ephemeral IP address. |
| 39 | + |
45 | 40 | --- |
46 | 41 |
|
47 | | -## Deployment Status |
| 42 | +## 🚦 Deployment Status |
48 | 43 |
|
49 | 44 | | Service | Status | Port | Networking | Registry | |
50 | 45 | | :--- | :--- | :--- | :--- | :--- | |
51 | 46 | | **Discovery Server (Eureka)** | **RUNNING** | 8761 | Public Subnet | ECR (us-east-1) | |
| 47 | +| **API Gateway** | **RUNNING** | 8082 | Public Subnet | ECR (us-east-1) | |
52 | 48 | | **Booking Service** | **RUNNING** | 8080 | Private Subnet | ECR (us-east-1) | |
53 | 49 | | **Event Service** | **RUNNING** | 8081 | Private Subnet | ECR (us-east-1) | |
54 | | -| **API Gateway** | **IMAGE READY** | 8082 | Public Subnet | ECR (us-east-1) | |
55 | 50 | | **RDS MySQL** | **AVAILABLE** | 3306 | Private Subnet | AWS RDS | |
56 | 51 |
|
57 | 52 | --- |
58 | 53 |
|
59 | | -## Key Skills Demonstrated |
60 | | -* **Advanced Containerization:** Expertise in managing multi-architecture Docker manifests and ECR lifecycle policies. |
61 | | -* **Infrastructure Troubleshooting:** Proficiency in analyzing CloudWatch logs and Spring Boot stack traces to resolve network-level and application-level failures. |
62 | | -* **Cloud Security:** Implementation of the **Principle of Least Privilege** through granular Security Group rules and private network isolation. |
63 | | -* **Externalized Configuration:** Utilizing Spring Boot profiles and environment variable injection to maintain environment parity across local and cloud deployments. |
64 | | -* **Cloud Orchestration:** Managing service lifecycles via AWS ECS and Cloud Map. |
| 54 | +## 📈 Future Roadmap (CI/CD) |
| 55 | +* [ ] **GitHub Actions Integration:** Automate the build-test-push cycle for all microservices. |
| 56 | +* [ ] **Automated Deployment:** Configure ECS task definitions to update automatically upon new image pushes. |
| 57 | +* [ ] **Secret Management:** Move AWS credentials and database passwords to **GitHub Secrets** and **AWS Secrets Manager**. |
| 58 | + |
| 59 | +--- |
| 60 | + |
| 61 | +## 🛠️ Key Skills Demonstrated |
| 62 | +* **Cloud Orchestration:** Managing complex service lifecycles via AWS ECS and Cloud Map. |
| 63 | +* **Network Security:** Designing multi-tier VPCs and implementing **Bastion Host** architectures. |
| 64 | +* **Infrastructure Troubleshooting:** Debugging Link-Local networking and cross-architecture container issues. |
| 65 | +* **Externalized Configuration:** Utilizing Spring Boot environment variables to override production settings dynamically without image rebuilds. |
| 66 | + |
| 67 | +--- |
0 commit comments