Skip to content

Commit c932a18

Browse files
committed
controlplane, console: stop listing expired nodes on every refresh
GET /admin/status and GET /user/status returned every node row kept for the retention period, so each console refresh grew with history. At 25k rows, mostly expired probe identities, one refresh moved 54.6 MB and took about 650 ms of control-plane time. Both lists now hold admitted and banned nodes only: 94 KB in 33 ms. A user's list is filtered by owner in SQL instead of in Go after loading the whole mesh. storage.Store lists nodes through one ListNodes(ctx, NodeFilter), which selects by state (admitted, expired, banned), owner and role, with keyset paging. GET /user/nodes pages NodeListResponse, newest enrollment first, narrowed by state, role and, for an administrator, owner_id. An expired OIDC enrollment can renew in place, so the console pages state=expired in its Expired Nodes section, only when the section is opened. /metrics counts nodes with one GROUP BY instead of loading every row. Migration 19 indexes owner_id and (enrolled_at, peer_id). controlplane.Options.Default() sets NodeRetention to 30 days, so agentmesh-one deletes lapsed rows too. A negative retention keeps them, and --node-retention 0 still means keep forever.
1 parent c98fc2e commit c932a18

26 files changed

Lines changed: 1329 additions & 305 deletions

File tree

‎api/agentmesh.pb.go‎

Lines changed: 235 additions & 173 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎api/agentmesh.proto‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -767,7 +767,7 @@ message NodeServices {
767767
}
768768

769769
// AdminStatusResponse answers GET /admin/status: everything the console
770-
// shows an administrator.
770+
// shows an administrator, except expired nodes (see GET /user/nodes).
771771
message AdminStatusResponse {
772772
repeated User users = 1;
773773
repeated RouterLease active_routers = 2;
@@ -779,9 +779,9 @@ message AdminStatusResponse {
779779
map<string, NodeServices> node_catalog = 7;
780780
}
781781

782-
// UserStatusResponse answers GET /user/status: the caller and what it owns.
783-
// The router fleet and the mesh policy describe the whole mesh and are set
784-
// for an administrator only.
782+
// UserStatusResponse answers GET /user/status: the caller and what it owns,
783+
// except expired nodes (see GET /user/nodes). The router fleet and the mesh
784+
// policy describe the whole mesh and are set for an administrator only.
785785
message UserStatusResponse {
786786
User user = 1;
787787
repeated EnrolledNode enrolled_nodes = 2;
@@ -790,6 +790,16 @@ message UserStatusResponse {
790790
PolicyConfig policy = 5;
791791
}
792792

793+
// NodeListResponse answers GET /user/nodes: enrolled nodes, newest enrollment
794+
// first, page_size (default 50, at most 200) at a time. state (admitted,
795+
// expired or banned, repeatable) and role narrow the list. An administrator
796+
// sees every owner's nodes and may pick one with owner_id; a user sees its own.
797+
message NodeListResponse {
798+
repeated EnrolledNode nodes = 1;
799+
// Pass as page_token for the next page; empty on the last page.
800+
string next_page_token = 2;
801+
}
802+
793803
// ============================================================================
794804
// Identity Evidence API
795805
// ============================================================================

‎cmd/agentmesh-control-plane/main.go‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,12 @@ func main() {
121121
}
122122
}()
123123

124+
// Options reads 0 as the default, so the flag's 0 (keep forever)
125+
// travels as a negative retention.
126+
if nodeRetention == 0 {
127+
nodeRetention = -1
128+
}
129+
124130
opts := controlplane.Options{
125131
ListenAddr: bindAddress,
126132
DriverName: dbDriver,

‎internal/console/public/app.js‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -146,6 +146,7 @@ function showApp() {
146146

147147
function showLandingPage() {
148148
stopAutoRefresh();
149+
resetExpiredNodes();
149150
document.getElementById('landing-page').classList.add('active');
150151
document.getElementById('app-container').style.display = 'none';
151152
}
Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
// Copyright 2026 Google LLC
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
const EXPIRED_NODES_PAGE_SIZE = 50;
16+
17+
// Where the expired-nodes list resumes, and which opening of the section a
18+
// fetch belongs to, so a slow page never lands in a list reset since.
19+
let expiredNodesPageToken = '';
20+
let expiredNodesGeneration = 0;
21+
22+
document.addEventListener('DOMContentLoaded', () => {
23+
const expiredNodes = document.getElementById('expired-nodes');
24+
expiredNodes.addEventListener('toggle', () => {
25+
if (expiredNodes.open) {
26+
loadExpiredNodes(true);
27+
}
28+
});
29+
document.getElementById('expired-nodes-more').addEventListener('click', () => loadExpiredNodes(false));
30+
document.getElementById('table-expired-nodes').addEventListener('click', (e) => {
31+
const button = e.target.closest('button[data-peer-id]');
32+
if (button) {
33+
revokeExpiredNode(button);
34+
}
35+
});
36+
});
37+
38+
// Fetched on demand, never by the background refresh: expired rows stay for
39+
// the retention period, and listing them all would slow every refresh.
40+
async function loadExpiredNodes(reset) {
41+
const more = document.getElementById('expired-nodes-more');
42+
if (reset) {
43+
expiredNodesGeneration++;
44+
expiredNodesPageToken = '';
45+
more.hidden = true;
46+
setTableMessage('table-expired-nodes', 5, 'Loading...');
47+
}
48+
const generation = expiredNodesGeneration;
49+
let url = `api/user/nodes?state=expired&page_size=${EXPIRED_NODES_PAGE_SIZE}`;
50+
if (expiredNodesPageToken) {
51+
url += `&page_token=${encodeURIComponent(expiredNodesPageToken)}`;
52+
}
53+
54+
more.disabled = true;
55+
try {
56+
const response = await fetch(url);
57+
if (response.status === 401 || response.status === 403) {
58+
showLandingPage();
59+
throw new Error('Unauthorized. Please login again.');
60+
}
61+
if (!response.ok) {
62+
throw new Error(`HTTP error! status: ${response.status}`);
63+
}
64+
const data = await response.json();
65+
if (generation !== expiredNodesGeneration) {
66+
return;
67+
}
68+
69+
const tbody = document.getElementById('table-expired-nodes');
70+
const rows = (data.nodes || []).map(expiredNodeRow).join('');
71+
if (reset) {
72+
tbody.innerHTML = rows;
73+
} else {
74+
tbody.insertAdjacentHTML('beforeend', rows);
75+
}
76+
if (!tbody.querySelector('tr')) {
77+
setTableMessage('table-expired-nodes', 5, 'No expired nodes');
78+
}
79+
expiredNodesPageToken = data.next_page_token || '';
80+
more.hidden = !expiredNodesPageToken;
81+
applySearchFilter();
82+
} catch (error) {
83+
if (generation !== expiredNodesGeneration) {
84+
return;
85+
}
86+
console.error('Failed to load expired nodes:', error);
87+
// A failed "Load more" keeps the rows already shown and the button to retry.
88+
if (reset) {
89+
setTableMessage('table-expired-nodes', 5, `Error loading expired nodes: ${error.message}`, true);
90+
} else {
91+
showToast(`Failed to load more expired nodes: ${error.message}`, 'error');
92+
}
93+
} finally {
94+
more.disabled = false;
95+
}
96+
}
97+
98+
// The Revoke button carries the peer ID as data, not inside an inline handler,
99+
// where an HTML-escaped quote would be decoded back into the script.
100+
function expiredNodeRow(node) {
101+
const expired = node.expire_time ? new Date(node.expire_time).toLocaleString() : '';
102+
return `
103+
<tr>
104+
<td>${peerCell(node.peer_id, node.labels)}</td>
105+
<td>${escapeHTML(node.role)}</td>
106+
<td>${escapeHTML(node.owner_id || '')}</td>
107+
<td>${escapeHTML(expired)}</td>
108+
<td><button class="btn btn-sm btn-danger" type="button" data-peer-id="${escapeHTML(node.peer_id)}">Revoke</button></td>
109+
</tr>
110+
`;
111+
}
112+
113+
// Paging resumes after the last row served, so dropping a revoked row never
114+
// shifts the next page.
115+
async function revokeExpiredNode(button) {
116+
if (!confirm('Revoke this node? It can no longer renew its expired session or rejoin the mesh.')) {
117+
return;
118+
}
119+
button.disabled = true;
120+
try {
121+
await actionRequest(`api/user/revoke?id=${encodeURIComponent(button.dataset.peerId)}`);
122+
} catch (error) {
123+
// actionRequest already reported the failure.
124+
button.disabled = false;
125+
return;
126+
}
127+
const tbody = document.getElementById('table-expired-nodes');
128+
button.closest('tr').remove();
129+
if (!tbody.querySelector('tr:not([data-filter-empty])')) {
130+
loadExpiredNodes(true);
131+
}
132+
}
133+
134+
// The landing page hides the app without a reload, so the next identity to log
135+
// in must not inherit rows, or an in-flight page, fetched for this one.
136+
function resetExpiredNodes() {
137+
expiredNodesGeneration++;
138+
expiredNodesPageToken = '';
139+
document.getElementById('expired-nodes').open = false;
140+
document.getElementById('table-expired-nodes').innerHTML = '';
141+
document.getElementById('expired-nodes-more').hidden = true;
142+
}

‎internal/console/public/index.html‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -226,6 +226,28 @@ <h2>Revoked Nodes</h2>
226226
</div>
227227
</div>
228228
</div>
229+
<!-- Loaded on open, never by the refresh: expired rows pile up for the
230+
retention period, yet an expired OIDC node can renew until revoked. -->
231+
<details id="expired-nodes" class="section-header-spaced">
232+
<summary class="section-header"><h2>Expired Nodes</h2></summary>
233+
<div class="card">
234+
<div class="table-responsive">
235+
<table class="data-table">
236+
<thead>
237+
<tr>
238+
<th>Node ID</th>
239+
<th>Type</th>
240+
<th>Owner</th>
241+
<th>Expired</th>
242+
<th>Actions</th>
243+
</tr>
244+
</thead>
245+
<tbody id="table-expired-nodes"></tbody>
246+
</table>
247+
</div>
248+
<button id="expired-nodes-more" class="btn btn-sm load-more" type="button" hidden>Load more</button>
249+
</div>
250+
</details>
229251
</section>
230252

231253
<!-- Services View: self-reported by each node (see /nodes/catalog);
@@ -405,5 +427,6 @@ <h2>Mesh Security Policies</h2>
405427
<div id="toast-container" class="toast-container" role="status" aria-live="polite"></div>
406428
<script src="vendor/js-yaml.min.js"></script>
407429
<script src="app.js"></script>
430+
<script src="expired-nodes.js"></script>
408431
</body>
409432
</html>

‎internal/console/public/style.css‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -523,6 +523,24 @@ body {
523523
font-weight: 600;
524524
}
525525

526+
/* A collapsible section: the disclosure marker sits beside its title. */
527+
summary.section-header {
528+
cursor: pointer;
529+
}
530+
531+
summary.section-header h2 {
532+
display: inline;
533+
}
534+
535+
.load-more {
536+
margin-top: 16px;
537+
}
538+
539+
/* .btn sets display, which would otherwise beat the hidden attribute. */
540+
.load-more[hidden] {
541+
display: none;
542+
}
543+
526544
/* Buttons */
527545
.btn {
528546
display: inline-flex;

‎internal/console/server_test.go‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -95,8 +95,11 @@ func TestNewServerEmbeddedAssets(t *testing.T) {
9595
if code, body := get("/"); code != http.StatusOK || !strings.Contains(body, "<html") {
9696
t.Errorf("GET / = %d, want 200 with the embedded index.html", code)
9797
}
98-
if code, _ := get("/app.js"); code != http.StatusOK {
99-
t.Errorf("GET /app.js = %d, want 200", code)
98+
// A missing file would get the SPA fallback, so a script must not be index.html.
99+
for _, asset := range []string{"/app.js", "/expired-nodes.js"} {
100+
if code, body := get(asset); code != http.StatusOK || strings.Contains(body, "<html") {
101+
t.Errorf("GET %s = %d, want 200 with the embedded script", asset, code)
102+
}
100103
}
101104
// SPA fallback: unknown paths serve index.html.
102105
if code, body := get("/some/client/route"); code != http.StatusOK || !strings.Contains(body, "<html") {

‎internal/controlplane/catalog_test.go‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -82,8 +82,8 @@ func catalogBody(t *testing.T, services ...*api.ServiceInfo) []byte {
8282
return body
8383
}
8484

85-
// adminNodeCatalog fetches /admin/status and returns its node_catalog value.
86-
func adminNodeCatalog(t *testing.T, cpURL, adminToken string) map[string]*api.NodeServices {
85+
// adminStatus fetches /admin/status with the admin token.
86+
func adminStatus(t *testing.T, cpURL, adminToken string) *api.AdminStatusResponse {
8787
t.Helper()
8888

8989
req, err := http.NewRequest(http.MethodGet, cpURL+"/admin/status", nil)
@@ -107,7 +107,7 @@ func adminNodeCatalog(t *testing.T, cpURL, adminToken string) map[string]*api.No
107107
if err := protojson.Unmarshal(body, status); err != nil {
108108
t.Fatalf("decode /admin/status: %v", err)
109109
}
110-
return status.GetNodeCatalog()
110+
return status
111111
}
112112

113113
func TestHandleNodeCatalog(t *testing.T) {
@@ -155,7 +155,7 @@ func TestHandleNodeCatalog(t *testing.T) {
155155
}
156156

157157
// The console sees the reported services with the report time.
158-
view := adminNodeCatalog(t, cpURL, srv.config.AdminToken)
158+
view := adminStatus(t, cpURL, srv.config.AdminToken).GetNodeCatalog()
159159
got, ok := view[nodePeer.String()]
160160
if !ok || len(view) != 1 {
161161
t.Fatalf("expected the reporting peer in node_catalog, got %v", view)
@@ -169,7 +169,7 @@ func TestHandleNodeCatalog(t *testing.T) {
169169
if got := postCatalog(t, cpURL, bearer(biscuitBytes), priv, catalogBody(t)); got != http.StatusNoContent {
170170
t.Fatalf("empty report: got status %d, want %d", got, http.StatusNoContent)
171171
}
172-
if view := adminNodeCatalog(t, cpURL, srv.config.AdminToken); len(view[nodePeer.String()].GetServices()) != 0 {
172+
if view := adminStatus(t, cpURL, srv.config.AdminToken).GetNodeCatalog(); len(view[nodePeer.String()].GetServices()) != 0 {
173173
t.Fatalf("empty report must clear services, got %+v", view)
174174
}
175175
}
@@ -280,7 +280,7 @@ func TestHandleNodeCatalog_Admission(t *testing.T) {
280280
if got := postCatalog(t, cpURL, bearer(biscuitBytes), priv, body); got != http.StatusNoContent {
281281
t.Fatalf("admitted node: got status %d, want %d", got, http.StatusNoContent)
282282
}
283-
if _, ok := adminNodeCatalog(t, cpURL, srv.config.AdminToken)[nodePeer.String()]; !ok {
283+
if _, ok := adminStatus(t, cpURL, srv.config.AdminToken).GetNodeCatalog()[nodePeer.String()]; !ok {
284284
t.Fatal("admitted node's report missing from node_catalog")
285285
}
286286

@@ -296,9 +296,13 @@ func TestHandleNodeCatalog_Admission(t *testing.T) {
296296
if got := postCatalog(t, cpURL, bearer(biscuitBytes), priv, body); got != http.StatusUnauthorized {
297297
t.Fatalf("expired session: got status %d, want %d", got, http.StatusUnauthorized)
298298
}
299-
if view := adminNodeCatalog(t, cpURL, srv.config.AdminToken); len(view) != 0 {
299+
expired := adminStatus(t, cpURL, srv.config.AdminToken)
300+
if view := expired.GetNodeCatalog(); len(view) != 0 {
300301
t.Fatalf("expired node must not appear in node_catalog, got %v", view)
301302
}
303+
if nodes := expired.GetEnrolledNodes(); len(nodes) != 0 {
304+
t.Fatalf("expired node must not appear in enrolled_nodes, got %v", nodes)
305+
}
302306

303307
// Banned through the server path: the entry is evicted outright.
304308
record.ExpiresAt = time.Now().Add(time.Hour)
@@ -359,7 +363,7 @@ func TestCatalogPeerIDCanonicalization(t *testing.T) {
359363

360364
// The view must join the CIDv1 record with the canonically-keyed entry,
361365
// displayed under the record's own spelling.
362-
view := adminNodeCatalog(t, cpURL, srv.config.AdminToken)
366+
view := adminStatus(t, cpURL, srv.config.AdminToken).GetNodeCatalog()
363367
if _, ok := view[cidForm]; !ok || len(view[cidForm].GetServices()) != 1 {
364368
t.Fatalf("CIDv1-enrolled node's report missing from node_catalog, got %v", view)
365369
}

‎internal/controlplane/config.go‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -49,9 +49,10 @@ type Options struct {
4949
OIDCSessionTTL time.Duration // How long an OIDC enrollment stays refreshable before the identity must re-authenticate interactively; defaults to api.OIDCSessionTTL
5050
WorkloadSessionTTL time.Duration // Session TTL for enrollments from a WorkloadIssuer; defaults to DefaultWorkloadSessionTTL (48h)
5151
// NodeRetention is how long an enrolled node's row is kept after its
52-
// session expired before being deleted; 0 keeps rows forever. Every
53-
// pod restart without a persistent data dir enrolls a fresh identity,
54-
// so without this the nodes table only ever grows.
52+
// session expired before being deleted; 0 is DefaultNodeRetention and a
53+
// negative value keeps rows forever. Every pod restart without a
54+
// persistent data dir enrolls a fresh identity, so without this the
55+
// nodes table only ever grows.
5556
NodeRetention time.Duration
5657
AdminToken string // Optional: administrative bearer token for protecting policy and enrollment queue REST APIs
5758
AutoApproveEnrollment bool // If true, valid bootstrap token enrollment requests are immediately approved without administrative manual gate
@@ -124,6 +125,9 @@ func (o *Options) Default() {
124125
if o.WorkloadSessionTTL <= 0 {
125126
o.WorkloadSessionTTL = DefaultWorkloadSessionTTL
126127
}
128+
if o.NodeRetention == 0 {
129+
o.NodeRetention = DefaultNodeRetention
130+
}
127131
if o.DelegatedBiscuitTTL <= 0 {
128132
o.DelegatedBiscuitTTL = 1 * time.Hour
129133
}

0 commit comments

Comments
 (0)