Skip to content

Commit c62e53b

Browse files
namedgraphclaude
andcommitted
The front Varnish caches anonymous SPARQL queries on their body
The client POSTs every query (the SPARQL 1.1 Protocol's query via POST), and varnish-frontend passed every POST, so each visitor's views reached the platform even when the store's own Varnish already held the result: there a hit is ~2 ms, through the platform ~0.3 s, and in a page's burst 1-4 s. A URL-keyed cache cannot tell two POSTs to /sparql apart, so the body is cached (std.cache_req_body, up to 64 KB) and hashed into the key (bodyaccess, which the varnish image ships). Only reads, and only anonymous ones: application/sparql-query, or a form with query= and no update=, to /sparql or /ns, without Client-Cert, On-Behalf-Of or an id_token cookie - the same rule as the GET branch, since an authenticated response carries the agent's identity. A miss is fetched as the POST it was. Results are kept for an hour; any write that passes through bans them all, as the end-user Varnish does for GETs, and a cached query no longer bans /sparql as if it were a write. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 6aa8346 commit c62e53b

1 file changed

Lines changed: 52 additions & 0 deletions

File tree

‎docker-compose.yml‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -488,6 +488,7 @@ configs:
488488
vcl 4.0;
489489
490490
import std;
491+
import bodyaccess;
491492
492493
backend default {
493494
.host = "${VARNISH_FRONTEND_BACKEND_HOST:-linkeddatahub}";
@@ -535,6 +536,25 @@ configs:
535536
return (pipe);
536537
}
537538
539+
# Anonymous SPARQL queries are cached on their body. The platform's client POSTs every query (the
540+
# SPARQL 1.1 Protocol's "query via POST"), and a URL-keyed cache cannot tell two of those apart, so
541+
# without this every visitor's views reach the platform. Only reads are cached - a query body, never
542+
# an update - and only anonymous ones, for the same reason the GET branch below passes authenticated
543+
# requests: the response would carry the agent's identity. Bodies over 64 KB pass.
544+
if (req.method == "POST" && req.url ~ "^/(sparql|ns)(\?|$)" &&
545+
!req.http.Client-Cert && !req.http.On-Behalf-Of && req.http.Cookie !~ "LinkedDataHub\.id_token=" &&
546+
(req.http.Content-Type ~ "^application/sparql-query" || req.http.Content-Type ~ "^application/x-www-form-urlencoded")) {
547+
if (std.cache_req_body(64KB)) {
548+
if (req.http.Content-Type ~ "^application/sparql-query" ||
549+
(bodyaccess.rematch_req_body("(^|&)query=") == 1 && bodyaccess.rematch_req_body("(^|&)update=") == 0)) {
550+
set req.http.X-SPARQL-Query = "cache";
551+
unset req.http.Cookie;
552+
return (hash);
553+
}
554+
}
555+
}
556+
unset req.http.X-SPARQL-Query;
557+
538558
if (req.method != "GET" && req.method != "HEAD") {
539559
/* We only deal with GET and HEAD by default */
540560
return (pass);
@@ -576,7 +596,33 @@ configs:
576596
return (hash);
577597
}
578598
599+
sub vcl_hash {
600+
# a cached query is keyed on its body as well as the URL (and, through Vary, the Accept header)
601+
if (req.http.X-SPARQL-Query == "cache") {
602+
bodyaccess.hash_req_body();
603+
}
604+
}
605+
606+
sub vcl_backend_fetch {
607+
# a cache miss is fetched as a GET unless told otherwise; the query is the POST's body
608+
if (bereq.http.X-SPARQL-Query == "cache") {
609+
set bereq.method = "POST";
610+
}
611+
}
612+
579613
sub vcl_backend_response {
614+
if (bereq.http.X-SPARQL-Query == "cache") {
615+
# an error is not kept; a result is kept for an hour at most, and dropped by any write (below)
616+
if (beresp.status != 200) {
617+
set beresp.uncacheable = true;
618+
set beresp.ttl = 0s;
619+
} else {
620+
set beresp.ttl = 1h;
621+
set beresp.http.X-SPARQL-Query = "cached";
622+
}
623+
return (deliver);
624+
}
625+
580626
/* Add Vary: Origin for static files to enable proper CORS caching */
581627
if (bereq.url ~ "^/static/") {
582628
if (beresp.http.Vary) {
@@ -590,6 +636,8 @@ configs:
590636
if ((beresp.status == 200 || beresp.status == 201 || beresp.status == 204) && bereq.method ~ "POST|PUT|DELETE|PATCH") {
591637
set beresp.http.X-LinkedDataHub = "Banned";
592638
ban("req.url == " + bereq.url + " && req.http.host == " + bereq.http.host);
639+
# a write can change any query's result
640+
ban("obj.http.X-SPARQL-Query == cached");
593641
}
594642
595643
/* NS (ontology) SPARQL queries from authenticated clients are cache-busted per-request with _nc;
@@ -602,6 +650,10 @@ configs:
602650
603651
return (deliver);
604652
}
653+
654+
sub vcl_deliver {
655+
unset resp.http.X-SPARQL-Query;
656+
}
605657
varnish-admin_vcl:
606658
content: |
607659
vcl 4.0;

0 commit comments

Comments
 (0)