ECS currently provides lifecycle events through EventBridge (ECS Deployment State Change, ECS Task State Change, etc.), which are useful for reacting to runtime transitions.
What is currently missing is a lightweight, consistent way for external control loops to observe changes to an ECS service's desired-state configuration.
I'm building an open-source GitOps reconciliation controller for ECS and would like ECS to expose a primitive that allows external controllers to efficiently determine whether a service's desired configuration has changed without repeatedly fetching and diffing the complete service configuration.
I operate environments with 30+ ECS services in a cluster, where I want GitOps workflows that behave like a proper continuous reconciliation system rather than relying on CI/CD push-based workarounds.
The ideal solution could be one or more of the following:
- A monotonic
desiredStateRevision identifier on the Service, returned by DescribeServices, which increments whenever the service's desired configuration changes. (Note: If strict monotonic integers are too complex for the distributed backend, a deterministic state hash or ETag would also work).
- A resumable change stream using a sequence token / resourceVersion-style mechanism, allowing an external controller to resume observation after being offline without missing changes.
- EventBridge events for service desired-configuration changes, in addition to the existing deployment and task lifecycle events.
A revision or ETag would allow controllers to determine whether a full configuration read and reconciliation are necessary, while avoiding continuous full-state polling.
A push-based workflow can deploy a change when Git changes, but it does not provide a reliable control loop for continuously determining whether the actual ECS state still matches the desired state.
For example, infrastructure changes can indirectly modify services, operators can make manual changes, Terraform can register task-definition revisions, and other AWS APIs or automation can modify service configuration outside the GitOps workflow.
However, building a reliable ECS control loop currently has a state-observation problem. Today, an external controller generally has to periodically call DescribeServices and compare the full configuration against its desired state.
At scale, this creates several problems:
- Polling load increases with the number of services even when nothing changes.
- Frequent polling creates unnecessary API calls and potential throttling pressure.
- Infrequent polling increases drift-detection latency.
- There is no simple, cheap "has this service changed since revision X?" primitive.
Why is this different from existing ECS EventBridge events?
ECS EventBridge events are excellent for answering: "What happened to the workload?"
For example: A deployment started / A deployment completed / A task started / A task stopped.
A control loop also needs to answer a different question: "Has the desired configuration of this service changed since I last observed it?"
These are different requirements:
- ECS lifecycle events -> "What happened?"
- Desired-state revision -> "Did the desired configuration change?"
While we can route UpdateService CloudTrail events to EventBridge, it fails as a reliable control loop primitive. CloudTrail events can experience delivery latency, and more importantly, they are transient. If the controller goes offline, it has no resumable sequence to know what UpdateService calls it missed, forcing it to fall back to the heavy periodic full-state polling sweep we are trying to avoid.
The simplest implementation would be a monotonically increasing revision associated with the service's desired configuration (or a deterministic ETag/Hash).
Conceptually:
{
"serviceArn": "arn:aws:ecs:...",
"desiredStateRevision": 1837
}
ECS currently provides lifecycle events through EventBridge (ECS Deployment State Change, ECS Task State Change, etc.), which are useful for reacting to runtime transitions.
What is currently missing is a lightweight, consistent way for external control loops to observe changes to an ECS service's desired-state configuration.
I'm building an open-source GitOps reconciliation controller for ECS and would like ECS to expose a primitive that allows external controllers to efficiently determine whether a service's desired configuration has changed without repeatedly fetching and diffing the complete service configuration.
I operate environments with 30+ ECS services in a cluster, where I want GitOps workflows that behave like a proper continuous reconciliation system rather than relying on CI/CD push-based workarounds.
The ideal solution could be one or more of the following:
desiredStateRevisionidentifier on the Service, returned byDescribeServices, which increments whenever the service's desired configuration changes. (Note: If strict monotonic integers are too complex for the distributed backend, a deterministic state hash or ETag would also work).A revision or ETag would allow controllers to determine whether a full configuration read and reconciliation are necessary, while avoiding continuous full-state polling.
A push-based workflow can deploy a change when Git changes, but it does not provide a reliable control loop for continuously determining whether the actual ECS state still matches the desired state.
For example, infrastructure changes can indirectly modify services, operators can make manual changes, Terraform can register task-definition revisions, and other AWS APIs or automation can modify service configuration outside the GitOps workflow.
However, building a reliable ECS control loop currently has a state-observation problem. Today, an external controller generally has to periodically call
DescribeServicesand compare the full configuration against its desired state.At scale, this creates several problems:
Why is this different from existing ECS EventBridge events?
ECS EventBridge events are excellent for answering: "What happened to the workload?"
For example: A deployment started / A deployment completed / A task started / A task stopped.
A control loop also needs to answer a different question: "Has the desired configuration of this service changed since I last observed it?"
These are different requirements:
While we can route
UpdateServiceCloudTrail events to EventBridge, it fails as a reliable control loop primitive. CloudTrail events can experience delivery latency, and more importantly, they are transient. If the controller goes offline, it has no resumable sequence to know whatUpdateServicecalls it missed, forcing it to fall back to the heavy periodic full-state polling sweep we are trying to avoid.The simplest implementation would be a monotonically increasing revision associated with the service's desired configuration (or a deterministic ETag/Hash).
Conceptually:
{ "serviceArn": "arn:aws:ecs:...", "desiredStateRevision": 1837 }