ECS Scaling at 20% CPU usage. - #16693
Conversation
🚀 Image pushed to AWS ECRImage digest: 🐛 Run the image locallyThe following can be used to run the image locally: # Refer to image using the immutable digest. Find alternatives below.
IMAGE_IDENTIFIER="@sha256:32c4d87d751a767c56cc9d7fe6dfb7843df38dee560fa70300842e6eb258273e"
# Refer to image using branch tag
# IMAGE_IDENTIFIER=":branch-dcr-1percent"
# Refer to image using build tag
# IMAGE_IDENTIFIER=":build-31167"
# Set environment variables for the AWS CLI
AWS_PROFILE="<A_PROFILE_FROM_JANUS>"
AWS_DEFAULT_REGION="eu-west-1"
IMAGE_ACCOUNT_ID=$(aws ssm get-parameter --name /organisation/accounts/artifacts --query "Parameter.Value" --output text)
REGISTRY="${IMAGE_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com"
IMAGE="${REGISTRY}/guardian/dotcom-rendering${IMAGE_IDENTIFIER}"
# Login to AWS ECR https://docs.aws.amazon.com/AmazonECR/latest/userguide/registry_auth.html
aws ecr get-login-password | docker login --username AWS --password-stdin $REGISTRY
# Pull the image
docker pull $IMAGE
# Run the image. You'll likely need to set additional flags. See https://docs.docker.com/reference/cli/docker/container/run.
docker run $IMAGE |
7dc4abe to
96eedbc
Compare
akash1810
left a comment
There was a problem hiding this comment.
IIUC we don't currently scale EC2 CODE - do we need to scale ECS CODE? Not a blocking comment, as we can update in a later PR.
| scaleInCooldown: Duration.seconds(60), | ||
| scaleOutCooldown: Duration.seconds(60), |
There was a problem hiding this comment.
The default is Duration.seconds(300), which I actually think I like more as it gives more time to prevent fluctuations in capacity.
We had the 60 seconds before this change when doing our load testing, however I think that may have been to speed up scaling events to reduce wait time to watch them happen. Not for any meaningful reason.
There was a problem hiding this comment.
@akash1810 and I discussed this and we are going to stick with 60 seconds for each cooldown for now. We can tune this more later!
60 seconds seams reasonable for now, and it is hard to know how this will impact scaling and the services ability to maintain its SLI under increasing load without watching it in production. We did a load test in code and it coped fine, so we have no reason to believe 60 seconds is a bad choice.
4ab965b to
72e6092
Compare
72e6092 to
f0284bd
Compare
|
Seen on PROD (merged by @connoromalleyatwork 9 minutes and 4 seconds ago) Please check your changes! |
What does this change?
Adds target scaling to ECS for tag page rendering in both CODE and PROD.
Target scaling was added to the cdk in
GuLoadBalancedAppExperimentalin pr ECS CPU Target Scaling - #2962