Amazon Managed Service for Apache Flink Examples

This repository contains sample applications for Amazon Managed Service for Apache Flink, organized by language and operational use cases.

Prerequisites

• A valid LocalStack for AWS license, which provides a LOCALSTACK_AUTH_TOKEN required to run these samples with Managed Flink in LocalStack.
• Docker for running LocalStack.
• AWS CLI (required by lstk aws) and lstk, the LocalStack CLI.
• Java and Maven for Java-based examples.
• Python and pip for Python-based examples.

Run the following command before starting LocalStack with your preferred method:

export LOCALSTACK_AUTH_TOKEN=<your-auth-token>

Samples

• Java S3 Sink sample
• Python Getting Started sample
• All Java examples
• All Python examples
• Scala getting started example
• Infrastructure utilities

Testing on LocalStack

test.sh deploys a Java example as a Managed Flink application on LocalStack and checks its output. CI runs it for two examples:

• S3Sink, the example from the LocalStack Managed Flink guide: generated stock prices must land as JSON lines in the sink bucket.
• GettingStarted: records put on the input Kinesis stream must come out of the output stream unchanged.

lstk start
mvn -B package -DskipTests -f java/S3Sink/pom.xml
./test.sh S3Sink

Applications that use the Kinesis connectors need two extra runtime properties on LocalStack, which test.sh sets for GettingStarted:

• aws.endpoint set to https://localhost.localstack.cloud:4566 on each stream's property group. By default the connectors call kinesis.<region>.amazonaws.com, and LocalStack's TLS certificate does not cover that name.
• aws.http.protocol.version set to HTTP1_1 on the sink's property group. The sink's AWS SDK v2 client uses HTTP/2 by default, which fails against LocalStack.

Security

See CONTRIBUTING for more information.

License

This sample code is made available under the MIT-0 license. See LICENSE.