Sample code that shows the important aspects of developing custom connectors for Kafka Connect. It provides the resources for building, deploying, and running the code on-premises using Docker, as well as running the code in the cloud.
Managed ELKK stack implemented with the AWS CDK
Technical solution to implement event processing prioritization with Apache Kafke using the concept of buckets.
🦾 Ultra-compact agent framework for Kafka in Go. Operate a distributed swarm.
After you migrate from an existing graph database to Amazon Neptune, you might want to capture and process changed data in real time. Continuous replication of databases using the change data capture technique allows you to unlock your data and make it available to other systems for use cases such as distributed data processing, building an ente…
Data Pipeline for CDC data from MySQL DB to Amazon S3 through Amazon MSK using Amazon MSK Connect (Debezium).
Kafka KStreams example using Amazon MSK with IAM Auth
Real-time streaming pipeline using AWS CDK with Amazon MSK Serverless, Amazon Managed Flink, Amazon Kinesis Data Firehose, Amazon S3, and Amazon SageMaker Feature Store for stock market data processing.
AWS Secrets Manager Config Provider for Apache Kafka
A tool that can be deployed to process posting and receiving text and audio files from and into a data lake, apply transformation in a distributed manner, and load it into a warehouse in a suitable format to train a speech-to-text model
Kafka DMZ pattern: Kong Event Gateway on ECS Fargate protecting Amazon MSK with OAuth authentication
Real-time AWS ecommerce data pipeline using MSK Serverless, MSK Connect, S3, AWS Glue, Delta Lake, SCD Type 2, and Redshift Serverless.
Real-time data pipeline using Apache Kafka & AWS MSK. Streams CDC data from RDS to S3 via Kafka Connect. Ideal for mastering cloud-native big data integration
Advanced networking architecture utilizing HAProxy and TCP SNI routing to securely connect external cloud providers (Azure, GCP) to private Amazon MSK brokers without a VPN.
Demo event analytics platform based on Apache Kafka (Confluent).
Real-time AWS ecommerce data pipeline using MSK Serverless, MSK Connect, S3, AWS Glue, Delta Lake, SCD Type 2, and Redshift Serverless.
Multi-source CDC lakehouse on AWS: stream changes from Oracle, Aurora PostgreSQL, CockroachDB, and Kafka into Apache Iceberg via Amazon Data Firehose or Debezium with Managed Flink, and query from Athena and Snowflake
Enterprise-scale migration of 65+ connectors and 200+ topics from self-managed Kafka to Amazon MSK, featuring automated staggered ingestion and AWS Secrets Manager integration.
To associate your repository with the amazon-msk topic, visit your repo's landing page and select "manage topics."