Skip to content

Commit 236609a

Browse files
joocerclaude
andcommitted
Stage 1,000,000 discoveries an hour for the next deployment
http 300000, https and ssh 250000, ftp and smtp 100000. Every rate recomputed to hold its existing window rather than spend it, the same method as the last two steps: 98/82/94/132/132 keeps all five runs within 20s of where they are, so the cron layout and flock margins are untouched. Measured at 800000 first, one full cycle: 67% CPU average, ~70% peak, drift +0.70% to +0.98% on the long protocols, no skipped ticks, 7442 hosts an hour. http came out of that step with 527s of margin having *gained* 25000 candidates. Two things get materially bigger and neither has been seen at this size. Peak ZMap send 322pps -> 406pps. Discovery grabs 7444 -> 9150 an hour, +23%, and grabs are what move CPU - the last three candidate steps cost about 2 points each, so ~70% is the expectation. That is an extrapolation from three points, and variables.tf already records two CPU projections that were confidently wrong, so it is written down as something to measure rather than to rely on. Not applied. Rides along with the next change that rolls the instance, same as the 74pps did. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent b8e52dc commit 236609a

1 file changed

Lines changed: 29 additions & 5 deletions

File tree

terraform/variables.tf

Lines changed: 29 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -302,18 +302,42 @@ variable "scan_protocol_budgets" {
302302
at 82+66+75, plus one banner at 99). Grabs, which are what actually move CPU here,
303303
rise about 20% on the discovery side - roughly 6290 to 7550 an hour - and 8% in
304304
total once refresh's fixed 9900 is counted.
305+
306+
800000 -> 1000000, same method again: every rate recomputed to hold its window.
307+
308+
http 250000@82 3052s -> 300000@ 98 3064s margin 536s
309+
https 200000@66 3033s -> 250000@ 82 3052s margin 548s
310+
ssh 200000@75 2670s -> 250000@ 94 2663s margin 937s
311+
ftp 75000@99 761s -> 100000@132 761s margin 2839s
312+
smtp 75000@99 761s -> 100000@132 761s margin 2839s
313+
314+
Measured at 800000 before this step, one full cycle: CPU 67% average and ~70% peak,
315+
drift +0.70% to +0.98% on the three long protocols, no skipped ticks, 7442 hosts an
316+
hour discovered. http came out of the previous step with 527s of margin having
317+
gained 25000 candidates, which is what 82pps bought.
318+
319+
Two things get materially bigger here and neither has been observed yet at this
320+
size. Peak ZMap send goes 322pps -> 406pps. Discovery grabs go 7444 -> 9150 an hour,
321+
+23%, and grabs are the thing that actually moves CPU - the three previous candidate
322+
steps each cost about 2 points, so ~70% is the expectation, but that is an
323+
extrapolation from three points and this file records two earlier CPU projections
324+
that were both wrong. Measure it; do not trust the number.
325+
326+
If CPU does land near 80% or drift starts eating the ~540s margins, this is the
327+
point where the answer stops being tuning: refresh already holds a fixed 9900 grabs
328+
an hour and c6g.xlarge doubles the cores.
305329
EOT
306330
type = map(object({
307331
candidates = number
308332
rate_pps = number
309333
minute = number
310334
}))
311335
default = {
312-
http = { candidates = 250000, rate_pps = 82, minute = 6 }
313-
https = { candidates = 200000, rate_pps = 66, minute = 6 }
314-
ssh = { candidates = 200000, rate_pps = 75, minute = 6 }
315-
ftp = { candidates = 75000, rate_pps = 99, minute = 10 }
316-
smtp = { candidates = 75000, rate_pps = 99, minute = 50 }
336+
http = { candidates = 300000, rate_pps = 98, minute = 6 }
337+
https = { candidates = 250000, rate_pps = 82, minute = 6 }
338+
ssh = { candidates = 250000, rate_pps = 94, minute = 6 }
339+
ftp = { candidates = 100000, rate_pps = 132, minute = 10 }
340+
smtp = { candidates = 100000, rate_pps = 132, minute = 50 }
317341
}
318342
}
319343

0 commit comments

Comments
 (0)