Skip to content

Commit 1b50cb4

Browse files
committed
feat: add Timeout pattern (#2845)
1 parent 41625d8 commit 1b50cb4

17 files changed

Lines changed: 1446 additions & 0 deletions

‎pom.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -260,6 +260,7 @@
260260
<module>rate-limiting-pattern</module>
261261
<module>fallback</module>
262262
<module>onion-architecture</module>
263+
<module>timeout</module>
263264
</modules>
264265
<repositories>
265266
<repository>

‎timeout/README.md‎

Lines changed: 287 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,287 @@
1+
---
2+
title: "Timeout Pattern in Java: Bounding the Wait for Slow Dependencies"
3+
shortTitle: Timeout
4+
description: "Learn the Timeout pattern in Java: give every downstream call a per-service time limit, cancel calls that overrun, log and count the events, and continue with a fallback so slow dependencies cannot stall the whole system."
5+
category: Resilience
6+
language: en
7+
tag:
8+
- Asynchronous
9+
- Cloud distributed
10+
- Fault tolerance
11+
- Microservices
12+
- Resilience
13+
---
14+
15+
## Also known as
16+
17+
* Time Limiter
18+
* Deadline
19+
20+
## Intent of Timeout Design Pattern
21+
22+
Bound how long a caller waits for a downstream service. When the limit is exceeded the call is abandoned, the event is recorded, and the caller continues with a fallback, so the latency of one slow dependency never becomes the latency of the whole system.
23+
24+
## Detailed Explanation of Timeout Pattern with Real-World Examples
25+
26+
Real-world example
27+
28+
> A pizza chain's online shop asks a separate recommendation engine which side dishes to suggest during checkout. One evening the recommendation engine starts taking twenty seconds per request. Without a limit, every checkout waits those twenty seconds, threads pile up, and soon nobody can order a pizza at all. With a 100 ms limit, the shop stops waiting, shows the always available "most popular sides" list instead, and the order goes through. The slow engine is logged and counted so the on-call engineer can look at it in the morning.
29+
30+
In plain words
31+
32+
> Decide up front how long you are willing to wait for a dependency, and when the time is up, stop waiting and move on with a plan B.
33+
34+
microservices.io says
35+
36+
> Prevent a client from waiting indefinitely for a response from a service by aborting the request after a specified time period.
37+
38+
Sequence diagram
39+
40+
```mermaid
41+
sequenceDiagram
42+
participant Caller
43+
participant TimeoutExecutor
44+
participant Worker as Worker thread
45+
participant Service as Downstream service
46+
47+
Caller->>TimeoutExecutor: execute(policy, call, fallback)
48+
TimeoutExecutor->>Worker: submit(call)
49+
Worker->>Service: invoke
50+
TimeoutExecutor->>TimeoutExecutor: wait at most policy.timeout()
51+
alt response arrives in time
52+
Service-->>Worker: result
53+
Worker-->>TimeoutExecutor: result
54+
TimeoutExecutor-->>Caller: result
55+
else limit exceeded
56+
TimeoutExecutor->>Worker: cancel(interrupt)
57+
TimeoutExecutor->>TimeoutExecutor: log warning, count timeout
58+
TimeoutExecutor-->>Caller: fallback.get()
59+
end
60+
```
61+
62+
## Programmatic Example of Timeout Pattern in Java
63+
64+
The example models an online shop that calls two downstream services. The product catalog is fast; the recommendation engine is slow. Each gets its own time limit.
65+
66+
1. **Declare a limit per service**
67+
68+
A `TimeoutPolicy` couples a service name with the maximum time the caller is willing to wait. The record validates that the limit is positive.
69+
70+
```java
71+
public record TimeoutPolicy(String serviceName, Duration timeout) {
72+
73+
public TimeoutPolicy {
74+
Objects.requireNonNull(serviceName, "serviceName");
75+
Objects.requireNonNull(timeout, "timeout");
76+
if (serviceName.isBlank()) {
77+
throw new IllegalArgumentException("serviceName must not be blank");
78+
}
79+
if (timeout.isZero() || timeout.isNegative()) {
80+
throw new IllegalArgumentException("timeout must be positive");
81+
}
82+
}
83+
84+
public static TimeoutPolicy of(String serviceName, long millis) {
85+
return new TimeoutPolicy(serviceName, Duration.ofMillis(millis));
86+
}
87+
}
88+
```
89+
90+
2. **Keep the limits configurable in one place**
91+
92+
A `TimeoutRegistry` stores the policies and hands out a default for services nobody configured explicitly.
93+
94+
```java
95+
public class TimeoutRegistry {
96+
97+
private final Map<String, TimeoutPolicy> policies = new ConcurrentHashMap<>();
98+
private final Duration defaultTimeout;
99+
100+
public TimeoutRegistry(Duration defaultTimeout) {
101+
this.defaultTimeout = Objects.requireNonNull(defaultTimeout, "defaultTimeout");
102+
}
103+
104+
public TimeoutRegistry register(TimeoutPolicy policy) {
105+
policies.put(policy.serviceName(), policy);
106+
return this;
107+
}
108+
109+
public TimeoutPolicy policyFor(String serviceName) {
110+
return policies.getOrDefault(serviceName, new TimeoutPolicy(serviceName, defaultTimeout));
111+
}
112+
}
113+
```
114+
115+
3. **Enforce the limit**
116+
117+
`TimeoutExecutor` runs the call on a worker thread and waits for at most the configured duration. On a timeout it cancels the worker with an interrupt, logs the event, counts it in `TimeoutMetrics`, and returns the fallback. A failure raised by the service is not a timeout and is rethrown as `ServiceCallException`.
118+
119+
```java
120+
@Slf4j
121+
public class TimeoutExecutor implements AutoCloseable {
122+
123+
private final ExecutorService executor;
124+
private final TimeoutMetrics metrics = new TimeoutMetrics();
125+
126+
public TimeoutExecutor() {
127+
this(Executors.newVirtualThreadPerTaskExecutor());
128+
}
129+
130+
public <T> T execute(TimeoutPolicy policy, Callable<T> call, Supplier<T> fallback) {
131+
var serviceName = policy.serviceName();
132+
var limitMillis = policy.timeout().toMillis();
133+
var future = executor.submit(call);
134+
try {
135+
var result = future.get(limitMillis, TimeUnit.MILLISECONDS);
136+
LOGGER.info("{} responded within its {} ms limit", serviceName, limitMillis);
137+
return result;
138+
} catch (TimeoutException e) {
139+
future.cancel(true);
140+
metrics.recordTimeout(serviceName);
141+
LOGGER.warn(
142+
"{} exceeded its {} ms limit; call cancelled, using fallback", serviceName, limitMillis);
143+
return fallback.get();
144+
} catch (ExecutionException e) {
145+
throw new ServiceCallException(serviceName, e.getCause());
146+
} catch (InterruptedException e) {
147+
future.cancel(true);
148+
Thread.currentThread().interrupt();
149+
throw new ServiceCallException(serviceName, e);
150+
}
151+
}
152+
153+
@Override
154+
public void close() {
155+
executor.shutdownNow();
156+
}
157+
}
158+
```
159+
160+
4. **Make the slow service cooperate with cancellation**
161+
162+
The simulated `RecommendationService` sleeps interruptibly, so the interrupt sent by the executor actually stops the work instead of leaving it running in the background.
163+
164+
```java
165+
public List<String> recommendationsFor(String customer) throws InterruptedException {
166+
LOGGER.info(
167+
"{}: computing recommendations for {}, expected latency {} ms",
168+
NAME,
169+
customer,
170+
latency.toMillis());
171+
try {
172+
Thread.sleep(latency);
173+
} catch (InterruptedException e) {
174+
LOGGER.info("{}: interrupted, abandoning the computation for {}", NAME, customer);
175+
throw e;
176+
}
177+
return List.of("Mechanical keyboard", "USB-C dock");
178+
}
179+
```
180+
181+
5. **Wire it together**
182+
183+
`App` registers a 500 ms limit for the catalog and a 100 ms limit for recommendations. Each call lives in a small helper that pairs the policy with the call and its fallback; `main` runs both. The catalog answers in time; the recommendation engine needs 400 ms, so the customer sees popular items instead and the timeout counter shows one event.
184+
185+
```java
186+
static List<String> loadProducts(
187+
TimeoutExecutor executor, TimeoutRegistry registry, ProductCatalogService catalog) {
188+
return executor.execute(
189+
registry.policyFor(ProductCatalogService.NAME), catalog::fetchProducts, List::of);
190+
}
191+
192+
static List<String> loadRecommendations(
193+
TimeoutExecutor executor,
194+
TimeoutRegistry registry,
195+
RecommendationService recommendations,
196+
String customer) {
197+
return executor.execute(
198+
registry.policyFor(RecommendationService.NAME),
199+
() -> recommendations.recommendationsFor(customer),
200+
() -> POPULAR_ITEMS);
201+
}
202+
```
203+
204+
```java
205+
var registry =
206+
new TimeoutRegistry(Duration.ofMillis(300))
207+
.register(TimeoutPolicy.of(ProductCatalogService.NAME, 500))
208+
.register(TimeoutPolicy.of(RecommendationService.NAME, 100));
209+
210+
var catalog = new ProductCatalogService(Duration.ofMillis(50));
211+
var recommendations = new RecommendationService(Duration.ofMillis(400));
212+
213+
try (var executor = new TimeoutExecutor()) {
214+
var products = loadProducts(executor, registry, catalog);
215+
LOGGER.info("Products: {}", products);
216+
217+
var suggested = loadRecommendations(executor, registry, recommendations, "alice");
218+
LOGGER.info("Recommendations shown to alice: {}", suggested);
219+
220+
LOGGER.info("Timeouts per service: {}", executor.metrics().snapshot());
221+
}
222+
```
223+
224+
Running the application produces output along these lines:
225+
226+
```
227+
Configured per-service limits: catalog 500 ms, recommendations 100 ms
228+
Calling product-catalog
229+
product-catalog: fetching products, expected latency 50 ms
230+
product-catalog responded within its 500 ms limit
231+
Products: [Laptop, Headphones, Monitor]
232+
Calling recommendations
233+
recommendations: computing recommendations for alice, expected latency 400 ms
234+
recommendations exceeded its 100 ms limit; call cancelled, using fallback
235+
recommendations: interrupted, abandoning the computation for alice
236+
Recommendations shown to alice: [Wireless mouse, Webcam]
237+
Timeouts per service: {recommendations=1}
238+
```
239+
240+
## Class diagram
241+
242+
See [timeout.urm.puml](./etc/timeout.urm.puml) for the PlantUML class diagram.
243+
244+
## When to Use the Timeout Pattern in Java
245+
246+
* Whenever a call leaves the process: HTTP and gRPC calls, database queries, message broker round trips, third-party APIs.
247+
* When a degraded answer delivered on time is worth more than a perfect answer delivered late.
248+
* When threads, connections or other pooled resources are held for the duration of a call and must not be tied up by a stalled dependency.
249+
* When different dependencies have different latency profiles and need individually tuned limits.
250+
251+
## Real-World Applications of Timeout Pattern in Java
252+
253+
* [Resilience4j TimeLimiter](https://resilience4j.readme.io/docs/timelimiter) wraps a `CompletableFuture` or `Future` with a configurable limit and optional cancellation.
254+
* [Netflix Hystrix](https://github.com/Netflix/Hystrix/wiki/Configuration#execution.isolation.thread.timeoutInMilliseconds) applied a per-command execution timeout before falling back.
255+
* [gRPC deadlines](https://grpc.io/docs/guides/deadlines/) propagate a limit across service hops.
256+
* `java.net.http.HttpClient` connect and request timeouts, JDBC `queryTimeout`, and `Future.get(long, TimeUnit)` in the JDK.
257+
258+
## Benefits and Trade-offs of Timeout Pattern
259+
260+
Benefits:
261+
262+
* **Predictable latency**: The caller's worst case is the configured limit plus the fallback cost, not the dependency's worst case.
263+
* **Failure containment**: Stalled dependencies stop consuming threads and connections, which prevents cascading failures.
264+
* **Observability**: Every timeout is logged and counted, exposing dependencies that regularly miss their budget.
265+
* **Independent tuning**: Each service gets a limit that matches its normal latency.
266+
267+
Trade-offs:
268+
269+
* **Choosing the value is hard**: Too short causes false alarms under normal jitter; too long defeats the purpose.
270+
* **Wasted work**: A cancelled call may already have done its side effects, so operations that are not idempotent need care.
271+
* **Cooperative cancellation**: An interrupt only stops code that checks for it; blocking calls that ignore interrupts keep running until they finish on their own.
272+
* **Fallback quality**: The fallback must be genuinely cheap and safe, otherwise the pattern only moves the problem.
273+
274+
## Related Java Design Patterns
275+
276+
* [Fallback](../fallback): Supplies the alternative answer once a timeout fires. The fallback module treats the time limit as one of several triggers; this module makes the limit itself the subject, with per-service configuration, cancellation and metrics.
277+
* [Circuit Breaker](../circuit-breaker): Counts timeouts as failures and stops calling a dependency that keeps overrunning its limit.
278+
* [Retry](../retry): Retries a call that timed out, ideally with a total deadline so retries cannot multiply the wait.
279+
* Bulkhead: Limits how many concurrent calls a dependency may hold, complementing the limit on how long each call may take.
280+
281+
## References and Credits
282+
283+
* [Timeout pattern (microservices.io)](https://microservices.io/patterns/reliability/timeout.html)
284+
* [Release It! Design and Deploy Production-Ready Software](https://amzn.to/4aqTNEP)
285+
* [Microservices Patterns: With examples in Java](https://amzn.to/3xaZwk0)
286+
* [Resilience4j TimeLimiter documentation](https://resilience4j.readme.io/docs/timelimiter)
287+
* [gRPC deadlines](https://grpc.io/docs/guides/deadlines/)

‎timeout/etc/timeout.urm.puml‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
@startuml
2+
package com.iluwatar.timeout {
3+
class TimeoutPolicy {
4+
- serviceName : String
5+
- timeout : Duration
6+
+ TimeoutPolicy(serviceName : String, timeout : Duration)
7+
+ of(serviceName : String, millis : long) : TimeoutPolicy {static}
8+
+ serviceName() : String
9+
+ timeout() : Duration
10+
}
11+
class TimeoutRegistry {
12+
- policies : Map<String, TimeoutPolicy>
13+
- defaultTimeout : Duration
14+
+ TimeoutRegistry(defaultTimeout : Duration)
15+
+ register(policy : TimeoutPolicy) : TimeoutRegistry
16+
+ policyFor(serviceName : String) : TimeoutPolicy
17+
}
18+
class TimeoutMetrics {
19+
- timeouts : ConcurrentMap<String, AtomicInteger>
20+
+ TimeoutMetrics()
21+
+ recordTimeout(serviceName : String) : void
22+
+ timeoutCount(serviceName : String) : int
23+
+ snapshot() : Map<String, Integer>
24+
}
25+
class TimeoutExecutor {
26+
- executor : ExecutorService
27+
- metrics : TimeoutMetrics
28+
+ TimeoutExecutor()
29+
+ TimeoutExecutor(executor : ExecutorService)
30+
+ execute(policy : TimeoutPolicy, call : Callable<T>, fallback : Supplier<T>) : T
31+
+ metrics() : TimeoutMetrics
32+
+ close() : void
33+
}
34+
class ServiceCallException {
35+
+ ServiceCallException(serviceName : String, cause : Throwable)
36+
}
37+
class ProductCatalogService {
38+
+ NAME : String {static}
39+
- latency : Duration
40+
+ ProductCatalogService(latency : Duration)
41+
+ fetchProducts() : List<String>
42+
}
43+
class RecommendationService {
44+
+ NAME : String {static}
45+
- latency : Duration
46+
+ RecommendationService(latency : Duration)
47+
+ recommendationsFor(customer : String) : List<String>
48+
}
49+
class App {
50+
- POPULAR_ITEMS : List<String> {static}
51+
+ App()
52+
+ main(args : String[]) : void
53+
~ loadProducts(executor : TimeoutExecutor, registry : TimeoutRegistry, catalog : ProductCatalogService) : List<String> {static}
54+
~ loadRecommendations(executor : TimeoutExecutor, registry : TimeoutRegistry, recommendations : RecommendationService, customer : String) : List<String> {static}
55+
}
56+
}
57+
TimeoutExecutor --> TimeoutMetrics
58+
TimeoutExecutor ..> TimeoutPolicy
59+
TimeoutExecutor ..> ServiceCallException
60+
TimeoutRegistry --> "*" TimeoutPolicy
61+
App ..> TimeoutRegistry
62+
App ..> TimeoutExecutor
63+
App ..> ProductCatalogService
64+
App ..> RecommendationService
65+
@enduml

0 commit comments

Comments
 (0)