Skip to content

Commit 797b601

Browse files
namedgraphclaude
andcommitted
The SPARQL endpoint refuses a query result longer than it can answer with
A CONSTRUCT on the Northwind demo's root document matched the whole dataset: its { BIND(?resource AS ?s) ?s ?p ?o } branch sees ?resource unbound, so it returned 83,251 triples (13.8 MB as N-Triples) where 182 were meant. Nothing bounded the result on its way: the endpoint buffered and hashed all of it, and Saxon-JS held the page's thread for ~30 s tabling it. The SPARQL client is exempt from MAX_CONTENT_LENGTH since b7261d9, rightly - that limit bounds untrusted input, and the platform's own lookups must read its store whatever its size. What bounds a query result is what its consumer can take, so it is a limit of its own, MAX_QUERY_RESPONSE_LENGTH (5 MiB unless set), applied only to HTTP callers of /sparql: SPARQLEndpointImpl takes a capped accessor from ServiceContext.getQueryEndpointAccessor(), and Access and the Graph Store keep the uncapped one. ContentLengthLimitFilter honours a per-request MAX_CONTENT_LENGTH property, which ResponseLengthLimitFilter sets and enforces, so the query limit replaces the client-wide one rather than stacking under it. The stream is cut where it passes the limit, so the result is never buffered whole. A refusal, not truncation: a caller that needs more pages with LIMIT/OFFSET. The refusal stays 502, as the proxy's does, and its description is typed ldh:ResponseTooLarge, so the client can tell it from an upstream that could not be reached: ldh:http-error-key($response) reads the type, and every call site holding the response uses it ("The result is larger than this server returns. Narrow the query or add a LIMIT."). The SSR error card in document.xsl reads the same type. Verified: unit test for the per-request limit (ContentLengthLimitFilterTest 9/9); a scratch probe through Jersey + Jena against a chunked server - an oversized CONSTRUCT and SELECT raise ResponseContentTooLargeException unwrapped, and a 2.5 MB result passes a 3 MB query limit under a 2 MB client-wide one; make sef; the SSR stylesheet compiles. NOT yet verified end to end: the image has not been rebuilt, so the new HTTP test (POST-sparql-response-too-large-502.sh) and the query block UI spec case have not run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent eb3b3a2 commit 797b601

38 files changed

Lines changed: 359 additions & 40 deletions

File tree

‎.env_sample‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,9 @@ OWNER_STATE_OR_PROVINCE=Denmark
1717
OWNER_COUNTRY_NAME=DK
1818

1919
MAX_CONTENT_LENGTH=2097152
20+
# the longest result, in bytes, the /sparql endpoint answers a query with; a longer one is refused with 502. Counted on the
21+
# store's response to the platform, not on what the caller receives. 5242880 (5 MiB) when unset
22+
# MAX_QUERY_RESPONSE_LENGTH=5242880
2023

2124
# the assistant is opt-in: its plan service (the web-algebra container) is a proprietary image, distributed without its
2225
# source. Uncomment to run it and offer it in the UI

‎AGENTS.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,6 +138,7 @@ A single instance hosts multiple **dataspaces**, each a subdomain (origin). Each
138138
- **`403` where you expect `404`.** The ACL query fails closed on a typeless resource, so a non-owner request for a URL that does not exist is answered `403`.
139139
- **`428`** on any write to an existing document that carries no `If-Match`; **`412`** when the tag quoted was read under a different negotiated variant. See *Conditional writes*.
140140
- **`502`** (rather than `413`) when an upstream response exceeds `MAX_CONTENT_LENGTH`: what was too large is the upstream's response. Requests to the deployment's own SPARQL, Graph Store and Quad Store services are exempt from the bound.
141+
- **`502`** from `/sparql` when the query's result exceeds `MAX_QUERY_RESPONSE_LENGTH` (5 MiB unless set), its body typed `ldh:ResponseTooLarge`. The result is refused, not truncated: page through it with `LIMIT`/`OFFSET`.
141142
- **`406`** for `application/link-format` on anything but `?timemap`.
142143

143144
## Tooling

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -246,6 +246,8 @@ _:warning: Do not use blank nodes to identify applications or services. We recom
246246
<dd>The period for which the data import can keep an open HTTP connection before it times out, in ms. The larger files are being imported, the longer it has to be in order for the import to complete.</dd>
247247
<dt><code>MAX_CONTENT_LENGTH</code></dt>
248248
<dd>Maximum allowed size of the request body, in bytes</dd>
249+
<dt><code>MAX_QUERY_RESPONSE_LENGTH</code></dt>
250+
<dd>Maximum length of a query result the <code>/sparql</code> endpoint answers with, in bytes. A longer one is refused with <code>502</code>. 5242880 (5 MiB) by default</dd>
249251
<dt><code>COMPOSE_PROFILES</code></dt>
250252
<dd><code>assistant</code> to run and offer the <a href="#assistant">assistant</a>. Disabled by default</dd>
251253
<dt><code>MAIL_SMTP_HOST</code></dt>

‎docker-compose.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ services:
6666
- SELF_SIGNED_CERT=true # only on localhost
6767
- SIGN_UP_CERT_VALIDITY=180
6868
- MAX_CONTENT_LENGTH=${MAX_CONTENT_LENGTH:-2097152}
69+
- MAX_QUERY_RESPONSE_LENGTH=${MAX_QUERY_RESPONSE_LENGTH:-} # 5 MiB unless set (Application.DEFAULT_MAX_QUERY_RESPONSE_LENGTH)
6970
- ALLOW_INTERNAL_URLS=${ALLOW_INTERNAL_URLS:-}
7071
- ENABLE_SITEMAP=${ENABLE_SITEMAP:-} # opt-in: /sitemap.xml queries every public document of the dataspace
7172
- COMPOSE_PROFILES=${COMPOSE_PROFILES:-} # the "assistant" profile also makes the UI offer the assistant (entrypoint.sh)

‎platform/context.xsl‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ xmlns:orcid="&orcid;"
3737
<xsl:param name="ldhc:ownerAuthQuery"/>
3838
<xsl:param name="ldhc:enableLinkedDataProxy"/>
3939
<xsl:param name="ldhc:maxContentLength"/>
40+
<xsl:param name="ldhc:maxQueryResponseLength"/>
4041
<xsl:param name="ldhc:maxImportThreads"/>
4142
<xsl:param name="ldhc:notificationAddress"/>
4243
<xsl:param name="ldhc:enableWebIDSignUp"/>
@@ -125,6 +126,9 @@ xmlns:orcid="&orcid;"
125126
<xsl:if test="$ldhc:maxContentLength">
126127
<Parameter name="&ldhc;maxContentLength" value="{$ldhc:maxContentLength}" override="false"/>
127128
</xsl:if>
129+
<xsl:if test="$ldhc:maxQueryResponseLength">
130+
<Parameter name="&ldhc;maxQueryResponseLength" value="{$ldhc:maxQueryResponseLength}" override="false"/>
131+
</xsl:if>
128132
<xsl:if test="$ldhc:maxImportThreads">
129133
<Parameter name="&ldhc;maxImportThreads" value="{$ldhc:maxImportThreads}" override="false"/>
130134
</xsl:if>

‎platform/entrypoint.sh‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1149,6 +1149,10 @@ if [ -n "$MAX_CONTENT_LENGTH" ]; then
11491149
MAX_CONTENT_LENGTH_PARAM="--stringparam ldhc:maxContentLength '$MAX_CONTENT_LENGTH' "
11501150
fi
11511151

1152+
if [ -n "$MAX_QUERY_RESPONSE_LENGTH" ]; then
1153+
MAX_QUERY_RESPONSE_LENGTH_PARAM="--stringparam ldhc:maxQueryResponseLength '$MAX_QUERY_RESPONSE_LENGTH' "
1154+
fi
1155+
11521156
if [ -n "$MAX_IMPORT_THREADS" ]; then
11531157
MAX_IMPORT_THREADS_PARAM="--stringparam ldhc:maxImportThreads '$MAX_IMPORT_THREADS' "
11541158
fi
@@ -1228,6 +1232,7 @@ transform="xsltproc \
12281232
$OWNER_AUTH_QUERY_PARAM \
12291233
$ENABLE_LINKED_DATA_PROXY_PARAM \
12301234
$MAX_CONTENT_LENGTH_PARAM \
1235+
$MAX_QUERY_RESPONSE_LENGTH_PARAM \
12311236
$MAX_IMPORT_THREADS_PARAM \
12321237
$NOTIFICATION_ADDRESS_PARAM \
12331238
$ENABLE_WEBID_SIGNUP_PARAM \

‎src/main/java/com/atomgraph/linkeddatahub/Application.java‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
import com.atomgraph.linkeddatahub.server.util.OntologyRepository;
2525
import org.apache.jena.riot.RDFParser;
2626
import com.atomgraph.linkeddatahub.server.mapper.HttpHostConnectExceptionMapper;
27+
import com.atomgraph.linkeddatahub.server.mapper.ResponseContentTooLargeExceptionMapper;
2728
import com.atomgraph.linkeddatahub.server.mapper.InternalURLExceptionMapper;
2829
import com.atomgraph.linkeddatahub.server.mapper.MessagingExceptionMapper;
2930
import com.atomgraph.linkeddatahub.server.mapper.auth.webid.WebIDLoadingExceptionMapper;
@@ -270,6 +271,8 @@ public class Application extends ResourceConfig
270271
public static final String CLIENT_SEF_PATH = "/static/com/atomgraph/linkeddatahub/xsl/client.xsl.sef.json";
271272
/** Milliseconds a request waits for a connection from the client pool when nothing configures it; the image sets the same through CONNECTION_REQUEST_TIMEOUT */
272273
public static final int DEFAULT_CONNECTION_REQUEST_TIMEOUT = 30000;
274+
/** Bytes of a SPARQL endpoint query result when nothing configures it; the image sets the same through MAX_QUERY_RESPONSE_LENGTH */
275+
public static final int DEFAULT_MAX_QUERY_RESPONSE_LENGTH = 5242880;
273276
/** ORCID's production environment, used when no issuer is configured; a client registered on the sandbox sets <code>https://sandbox.orcid.org</code> */
274277
public static final String ORCID_ISSUER = "https://orcid.org";
275278
/** Path of the client stylesheet source in the webapp, composed with package stylesheets per import set */
@@ -302,6 +305,7 @@ public class Application extends ResourceConfig
302305
private final boolean allowInternalUrls;
303306
private final URLValidator urlValidator;
304307
private final Integer maxContentLength;
308+
private final int maxQueryResponseLength;
305309
private final Address notificationAddress;
306310
private final Authenticator authenticator;
307311
private final Properties emailProperties = new Properties();
@@ -378,6 +382,8 @@ public Application(@Context ServletConfig servletConfig) throws URISyntaxExcepti
378382
System.getProperty("com.atomgraph.linkeddatahub.allowInternalUrls") != null ? Boolean.parseBoolean(System.getProperty("com.atomgraph.linkeddatahub.allowInternalUrls")) :
379383
servletConfig.getServletContext().getInitParameter(LDHC.allowInternalUrls.getURI()) != null ? Boolean.parseBoolean(servletConfig.getServletContext().getInitParameter(LDHC.allowInternalUrls.getURI())) : false,
380384
servletConfig.getServletContext().getInitParameter(LDHC.maxContentLength.getURI()) != null ? Integer.valueOf(servletConfig.getServletContext().getInitParameter(LDHC.maxContentLength.getURI())) : null,
385+
servletConfig.getServletContext().getInitParameter(LDHC.maxQueryResponseLength.getURI()) != null ? Integer.parseInt(servletConfig.getServletContext().getInitParameter(LDHC.maxQueryResponseLength.getURI())) :
386+
DEFAULT_MAX_QUERY_RESPONSE_LENGTH, // unset means the endpoint buffers, and a page renders, a result of any length
381387
System.getProperty("com.atomgraph.linkeddatahub.maxConnPerRoute") != null ? Integer.valueOf(System.getProperty("com.atomgraph.linkeddatahub.maxConnPerRoute")) :
382388
servletConfig.getServletContext().getInitParameter(LDHC.maxConnPerRoute.getURI()) != null ? Integer.valueOf(servletConfig.getServletContext().getInitParameter(LDHC.maxConnPerRoute.getURI())) : null,
383389
System.getProperty("com.atomgraph.linkeddatahub.maxTotalConn") != null ? Integer.valueOf(System.getProperty("com.atomgraph.linkeddatahub.maxTotalConn")) :
@@ -449,6 +455,7 @@ public Application(@Context ServletConfig servletConfig) throws URISyntaxExcepti
449455
* @param enableAssistant true if the assistant's plan service is deployed beside this instance
450456
* @param allowInternalUrls true if internal/private network URLs are allowed (disables SSRF protection)
451457
* @param maxContentLength maximum size of request entity
458+
* @param maxQueryResponseLength maximum length of a SPARQL endpoint query result in bytes
452459
* @param maxConnPerRoute maximum client connections per rout
453460
* @param maxTotalConn maximum total client connections
454461
* @param maxRequestRetries maximum number of times that the HTTP client will retry a request
@@ -480,7 +487,7 @@ public Application(final ServletConfig servletConfig, final MediaTypes mediaType
480487
final String webIDQueryString, final String agentQueryString, final String userAccountQueryString, final String ontologyQueryString,
481488
final String baseURIString, final String proxyScheme, final String proxyHostname, final Integer proxyPort,
482489
final String uploadRootString, final String sefRootString, final String sefCompilerString, final String clientStylesheetString, final boolean invalidateCache,
483-
final Integer cookieMaxAge, final boolean enableLinkedDataProxy, final boolean enableAssistant, final boolean allowInternalUrls, final Integer maxContentLength,
490+
final Integer cookieMaxAge, final boolean enableLinkedDataProxy, final boolean enableAssistant, final boolean allowInternalUrls, final Integer maxContentLength, final int maxQueryResponseLength,
484491
final Integer maxConnPerRoute, final Integer maxTotalConn, final Integer maxRequestRetries, final Integer connectionRequestTimeout,
485492
final Integer socketTimeout, final Integer connectTimeout, final Integer selfRequestTimeout, final Long connectionTimeToLive, final Integer validateAfterInactivity, final Integer maxImportThreads,
486493
final String notificationAddressString, final boolean enableWebIDSignUp, final String oidcRefreshTokensPropertiesPath,
@@ -622,6 +629,7 @@ public Application(final ServletConfig servletConfig, final MediaTypes mediaType
622629
this.urlValidator = new URLValidator(allowInternalUrls);
623630
configureServiceExecution(allowInternalUrls);
624631
this.maxContentLength = maxContentLength;
632+
this.maxQueryResponseLength = maxQueryResponseLength;
625633
this.invalidateCache = invalidateCache;
626634
this.enableWebIDSignUp = enableWebIDSignUp;
627635
this.oidcRefreshTokensPropertiesPath = oidcRefreshTokensPropertiesPath;
@@ -1288,6 +1296,7 @@ protected void registerExceptionMappers()
12881296
register(NotAcceptableExceptionMapper.class);
12891297
register(ClientErrorExceptionMapper.class);
12901298
register(HttpHostConnectExceptionMapper.class);
1299+
register(ResponseContentTooLargeExceptionMapper.class);
12911300
register(InternalURLExceptionMapper.class);
12921301
register(BadGatewayExceptionMapper.class);
12931302
register(OntClassNotFoundExceptionMapper.class);
@@ -2514,6 +2523,16 @@ public Integer getMaxContentLength()
25142523
return maxContentLength;
25152524
}
25162525

2526+
/**
2527+
* Maximum length of the result the SPARQL endpoint answers a query with.
2528+
*
2529+
* @return length in bytes
2530+
*/
2531+
public int getMaxQueryResponseLength()
2532+
{
2533+
return maxQueryResponseLength;
2534+
}
2535+
25172536
/**
25182537
* Keystore of the HTTP client.
25192538
*

‎src/main/java/com/atomgraph/linkeddatahub/client/exception/ResponseContentTooLargeException.java‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,9 +33,10 @@ public class ResponseContentTooLargeException extends WebApplicationException
3333
* @param maxContentLength maximum content length in bytes
3434
* @param contentLength actual content length in bytes
3535
*/
36-
public ResponseContentTooLargeException(long maxContentLength, long contentLength) // TO-DO: use sizes to generate a message?
36+
public ResponseContentTooLargeException(long maxContentLength, long contentLength)
3737
{
38-
super(Response.Status.BAD_GATEWAY);
38+
// a streamed response is refused once it passes the limit, so its count is where it stopped, not its length
39+
super("Response is larger than the limit of " + maxContentLength + " bytes", Response.Status.BAD_GATEWAY);
3940
}
4041

4142
}
Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
/**
2+
* Copyright 2026 Martynas Jusevičius <martynas@atomgraph.com>
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*
16+
*/
17+
package com.atomgraph.linkeddatahub.client.filter;
18+
19+
import com.atomgraph.linkeddatahub.server.filter.request.ContentLengthLimitFilter;
20+
import java.io.IOException;
21+
import jakarta.ws.rs.client.ClientRequestContext;
22+
import jakarta.ws.rs.client.ClientRequestFilter;
23+
import jakarta.ws.rs.client.ClientResponseContext;
24+
import jakarta.ws.rs.client.ClientResponseFilter;
25+
26+
/**
27+
* Client filter that bounds a request's response with a limit of its own, in place of the configured one.
28+
* <p>
29+
* The configured limit ({@link ContentLengthLimitFilter}) bounds content from origins nobody controls, and
30+
* this deployment's own store is exempt from it ({@link UnlimitedContentLengthFilter}). A query result
31+
* answered to an HTTP caller of the SPARQL endpoint is bounded for a different reason: the endpoint
32+
* buffers it whole, and so does whatever receives it - a page renders it in the browser's only thread.
33+
* That bound is not the configured one, which is a limit on input. So the request carries its own,
34+
* which the configured filter on the same client honours, and this filter applies it itself for a
35+
* client without one.
36+
*
37+
* @author {@literal Martynas Jusevičius <martynas@atomgraph.com>}
38+
* @see ContentLengthLimitFilter#MAX_CONTENT_LENGTH
39+
*/
40+
public class ResponseLengthLimitFilter implements ClientRequestFilter, ClientResponseFilter
41+
{
42+
43+
private final ContentLengthLimitFilter limitFilter;
44+
45+
/**
46+
* Constructs filter.
47+
*
48+
* @param maxContentLength maximum response length in bytes
49+
*/
50+
public ResponseLengthLimitFilter(int maxContentLength)
51+
{
52+
this.limitFilter = new ContentLengthLimitFilter(maxContentLength);
53+
}
54+
55+
@Override
56+
public void filter(ClientRequestContext requestContext) throws IOException
57+
{
58+
requestContext.setProperty(ContentLengthLimitFilter.MAX_CONTENT_LENGTH, getLimitFilter().getMaxContentLength());
59+
}
60+
61+
@Override
62+
public void filter(ClientRequestContext requestContext, ClientResponseContext responseContext) throws IOException
63+
{
64+
getLimitFilter().filter(requestContext, responseContext);
65+
}
66+
67+
/**
68+
* Returns the filter that applies the limit.
69+
*
70+
* @return content length limit filter
71+
*/
72+
public ContentLengthLimitFilter getLimitFilter()
73+
{
74+
return limitFilter;
75+
}
76+
77+
}

‎src/main/java/com/atomgraph/linkeddatahub/model/ServiceContext.java‎

Lines changed: 31 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,10 @@
2222
import com.atomgraph.core.client.SPARQLClient;
2323
import com.atomgraph.core.model.EndpointAccessor;
2424
import com.atomgraph.core.model.impl.remote.EndpointAccessorImpl;
25+
import com.atomgraph.linkeddatahub.client.filter.ResponseLengthLimitFilter;
2526
import com.atomgraph.linkeddatahub.client.filter.UnlimitedContentLengthFilter;
2627
import jakarta.ws.rs.client.Client;
28+
import jakarta.ws.rs.client.ClientRequestFilter;
2729
import jakarta.ws.rs.client.WebTarget;
2830
import jakarta.ws.rs.core.UriBuilder;
2931
import java.net.URI;
@@ -105,6 +107,20 @@ public SPARQLClient getSPARQLClient()
105107
* @return SPARQL client
106108
*/
107109
public SPARQLClient getSPARQLClient(WebTarget webTarget)
110+
{
111+
// this service is this deployment's own store, not an origin the content limit is meant to bound.
112+
// Registered per client rather than taken off the outbound client, which also dereferences WebIDs
113+
return getSPARQLClient(webTarget, new UnlimitedContentLengthFilter());
114+
}
115+
116+
/**
117+
* Creates a SPARQL Protocol client for the specified URI web target, its responses bounded by the given filter.
118+
*
119+
* @param webTarget URI web target
120+
* @param lengthFilter filter that sets the response length limit
121+
* @return SPARQL client
122+
*/
123+
protected SPARQLClient getSPARQLClient(WebTarget webTarget, ClientRequestFilter lengthFilter)
108124
{
109125
SPARQLClient sparqlClient;
110126

@@ -113,9 +129,7 @@ public SPARQLClient getSPARQLClient(WebTarget webTarget)
113129
else
114130
sparqlClient = SPARQLClient.create(getMediaTypes(), webTarget);
115131

116-
// this service is this deployment's own store, not an origin the content limit is meant to bound.
117-
// Registered per client rather than taken off the outbound client, which also dereferences WebIDs
118-
sparqlClient.getEndpoint().register(new UnlimitedContentLengthFilter());
132+
sparqlClient.getEndpoint().register(lengthFilter);
119133

120134
if (getService().getAuthUser() != null && getService().getAuthPwd() != null)
121135
{
@@ -139,6 +153,20 @@ public EndpointAccessor getEndpointAccessor()
139153
return new EndpointAccessorImpl(getSPARQLClient());
140154
}
141155

156+
/**
157+
* Returns the endpoint accessor that answers HTTP callers of this service's SPARQL endpoint.
158+
* Unlike {@link #getEndpointAccessor()}, which the platform's own lookups use, a result longer than the
159+
* given limit is refused: the endpoint buffers it whole, and so does whatever receives it.
160+
*
161+
* @param maxResponseLength maximum length of the store's response in bytes
162+
* @return endpoint accessor
163+
*/
164+
public EndpointAccessor getQueryEndpointAccessor(int maxResponseLength)
165+
{
166+
return new EndpointAccessorImpl(getSPARQLClient(getClient().target(getProxiedURI(URI.create(getService().getSPARQLEndpoint().getURI()))),
167+
new ResponseLengthLimitFilter(maxResponseLength)));
168+
}
169+
142170
/**
143171
* Returns the Graph Store Protocol client for this service, with proxy routing applied.
144172
*

0 commit comments

Comments
 (0)