Repository navigation
Commit d82c422
perf: cut per-event CPU on the runtime hot paths (#7370)
* perf: cut per-event CPU on the runtime hot paths
Every backend event pays a fixed cost in the event processor and a per-value
cost in the state machinery: reading fields and computed vars, wrapping
values in mutable proxies, building the delta, encoding it to JSON and, with
a redis or disk manager, pickling the touched states. Each of those paths did
work per call that only has to happen once per class, per handler or per
type, and a few did work that was never needed at all.
Computed vars: a cache hit re-validated the whole cached value against the
return type on every read (a 1000-element cached list cost ~1 ms per read;
now ~0.2 us), the three cache attribute names were formatted per access, the
last-updated timestamp was stored for every var although only interval vars
read it, and marking a var dirty walked dependency dicts for names that have
no dependents. The type mismatch is now logged once per computed value.
Fields: scalar values skip the mutability check, and the dirty marking does
not re-assign an already set touched flag through the dev-mode setattr hook.
Mutable proxies: a proxy built for a list element no longer goes through the
class lookup and the type-call dispatch, immutable values return before any
tracking check in __getattr__, __getitem__ and __iter__, the dataclass frame
check only runs for proxied dataclasses (via sys._getframe instead of a
five-frame walk), and a method that is rebound to the proxy is no longer
wrapped in a FunctionWrapper first.
Event processor: the event context is built once per enqueue (fork takes the
router_data), transaction ids come straight from os.urandom, the handler's
coroutine-ness, background and supersedes flags and type hints are resolved
once per handler, the per-event imports are hoisted, a handler that yields
nothing skips the yield validation, the memory manager's modify_state and
modify_state_with_links are plain async context managers instead of async
generators, substate lookup resolves the class path once, and build_delta
reads field values without constructing a proxy to unwrap.
Serialization: json_dumps shares one encoder per set of separators instead
of building a JSONEncoder per call, dataclasses serialize through a per-type
serializer instead of dataclasses.fields() per instance, proxied values
resolve their serializer once instead of dispatching twice, and StateUpdate
serializes from a precomputed field tuple.
State managers: redis persists every touched state of a tree in one pipeline
instead of one write task (and one lock check) per state, deserializing a
state updates the instance dict and drops the stale keys instead of filtering
per key, the state-size check only runs on states over the limit, and the
disk manager only derives a token for the states it writes.
Measured on an idle 4-core VM (min of interleaved rounds):
- in-process event pipeline, counter handler with 4 computed vars:
266 -> 205 us/event; 1000-row filtered/sorted table: 8.5 -> 4.4 ms/event
- proxy iteration of 1000 dataclass rows: -55%; sorted(): -62%;
attribute chains: -25%; write paths unchanged
- cached computed var hit: 1.32 -> 0.19 us (scalar), 992 -> 0.19 us (list of
1000); scalar field read -35%, write -36%
- JSON: 4-var delta 4.5 -> 2.7 us; 1000 proxied dataclass rows 2.5 -> 1.6 ms
- redis manager, 35-state tree: 8.1 -> 2.1 ms/event; pickle 5.5 -> 3.3 us
Claude-Session: https://claude.ai/code/session_01DKCRZTdcqM4Smu9MKrjfhx
* perf: cut the per-operation cost of building Vars
Every derived Var a page builds (arithmetic, comparisons, string and array
methods, rx.cond, rx.foreach) goes through LiteralVar.create for its
operands, the var_operation wrapper, and Var.guess_type on the result. Each
of those did work per call that only depends on the type involved, or
imported a module on every call:
- LiteralVar.create ran two relative imports before looking at the value;
the exact-type lookup now comes first and the imports (which register the
literal classes) only run on a miss.
- Var.guess_type resolved the type alias, unwrapped Optional, tested the
origin and union, ran two registry caches and the whole Var.to dispatch
on every call; a plain cast now comes from a per-type cache that the
registry resets when a Var subclass is added.
- Var.to imported ObjectVar and ran the serializer probe before consulting
the registry for a Var-class target.
- ToOperation.__getattr__ imported ObjectVar to answer _js_expr, which
every operand interpolation reads.
- The var_operation wrapper built two dicts, two item lists and a
concatenation per call to pass a tuple of pairs along.
- Each cached Var property added one __del__ link that raised and caught
AttributeError for every property never read; the owner now has one
__del__ that probes the instance dict for all of its cached fields.
- A color name made of plain literals built six Vars and a concat
operation for a string that is known at compile time.
The generated code is unchanged (124 snapshots of a Var corpus and the
compiled benchmark pages are identical). Measured, min of interleaved
rounds on an idle core: arithmetic/comparison/string/array operations
-47 to -55%, match -35%, iteration -28%; a var-heavy page evaluates -29%,
the stateful page -17%, a full page compile -7 to -11%.
Claude-Session: https://claude.ai/code/session_01DKCRZTdcqM4Smu9MKrjfhx
* Assert which states the batched redis write stores
The batched set_state tests only counted pipeline constructions, so a batch
that also wrote an untouched state, or a direct write for an untouched tree,
still passed. Check the stored keys of the tree: only the touched states are
written, and an untouched tree writes nothing at all.
Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
* Check the redis lock atomically with the batched state write
The batched set_state checked the lock once, before serializing the whole
tree, where the per-state writes it replaced each re-checked it before
their own write. A tree whose serialization outlasted the lock could then
overwrite the writes of the next lock holder. A held lock now writes the
touched states through a script that compares the lock id and sets the
keys atomically, still in one round trip (and cheaper than the MULTI/EXEC
pipeline), raising LockExpiredError when the lock changed hands. The mock
redis runs the script for the unit tests.
Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
* Cache a computed var whose value is dataclasses.MISSING
The cached computed var read used dataclasses.MISSING to mean "no cached
value", so a getter returning MISSING ran again, and marked the state
touched again, on every read. Use a private sentinel in the sync and async
paths.
Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
* Release the state lock when linking the state fails
modify_state_with_links released the lock only for errors raised while
entering the linked states: an error from the link lookup or the
shared-state imports, after the lock was taken, left the token locked.
Guard the whole link setup.
Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
* Cache proxy classes for dataclass types only
_proxy_class cached its answer for every mutable type it saw, so it kept
each one alive for the life of the process, including model classes an app
generates at runtime, although any non-dataclass type is proxied through
the base class. Check for a dataclass first and cache only the classes
generated for dataclass types, as before the cache was introduced.
Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
* Read get_value through a StateProxy on the state it wraps
get_value took its field fast path for a StateProxy too, since the proxy
reports the wrapped state's class, and Field._get_raw then looked up the
proxy's parent_state, which a StateProxy refuses outside `async with self`.
Reading a value through a background task's proxy raised
ImmutableStateError. Take the fast path for a state itself only; a proxy
reads through getattr, as before.
Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
* Reuse the empty type hints of an unannotated event handler
process_event fell back to resolving a handler's type hints whenever its
cached hints were falsy, which includes the empty mapping cached for a
handler without annotations, so those handlers resolved their hints again
on every event. Fall back only when the hints were not resolvable at
registration (None).
Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
* Shorten the runtime performance news fragments
Say what got faster for users in a sentence or two, keeping the two
behavior notes (the computed var type log and the once-per-handler
is_background/supersedes reads), and leave the implementation narrative
to the PR.
Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
* Fix type errors in the lock-checked write and its tests
Cast the script's untyped reply to the lock id it carries, as _lock does
for GETDEL, and patch reflex.utils.types directly in the type hints test
instead of reaching it through the processor module, which does not
export it.
Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
* Type the dataclass proxy-class cache for the minimum dependency versions
At the declared wrapt floor (1.17.0, which ships no type information)
pyright resolves wrapt.ObjectProxy to Unknown, so it cannot prove
type[MutableProxy] hashable, and functools.cache's Hashable parameters
rejected the base proxy class: check-min-deps failed for reflex. Keep the
classes generated for dataclass types in a dict per base proxy class,
keyed by the dataclass type, instead. Other types are still never cached,
and a cached lookup no longer builds and hashes a key tuple.
Claude-Session: https://claude.ai/code/session_01PXcFKAgnzhgdnSvhuCmxWE
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 8ee5d41 commit d82c422
26 files changed
Lines changed: 1665 additions & 310 deletions
File tree
- news
- packages/reflex-base
- news
- src/reflex_base
- event
- processor
- utils
- vars
- reflex
- istate
- manager
- tests/units
- istate
- manager
- reflex_base
- event/processor
- vars
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
| 12 | + | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| |||
611 | 611 | | |
612 | 612 | | |
613 | 613 | | |
614 | | - | |
| 614 | + | |
615 | 615 | | |
616 | 616 | | |
617 | 617 | | |
| 618 | + | |
| 619 | + | |
618 | 620 | | |
619 | 621 | | |
620 | 622 | | |
621 | 623 | | |
622 | 624 | | |
623 | | - | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
624 | 635 | | |
625 | 636 | | |
626 | 637 | | |
| |||
636 | 647 | | |
637 | 648 | | |
638 | 649 | | |
| 650 | + | |
639 | 651 | | |
640 | 652 | | |
641 | 653 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
| 7 | + | |
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
34 | 43 | | |
35 | 44 | | |
36 | 45 | | |
| |||
89 | 98 | | |
90 | 99 | | |
91 | 100 | | |
92 | | - | |
| 101 | + | |
93 | 102 | | |
94 | 103 | | |
95 | 104 | | |
| |||
110 | 119 | | |
111 | 120 | | |
112 | 121 | | |
113 | | - | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
114 | 125 | | |
115 | 126 | | |
116 | 127 | | |
117 | 128 | | |
| 129 | + | |
118 | 130 | | |
119 | 131 | | |
120 | 132 | | |
| |||
127 | 139 | | |
128 | 140 | | |
129 | 141 | | |
130 | | - | |
| 142 | + | |
131 | 143 | | |
132 | 144 | | |
133 | 145 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
21 | 22 | | |
22 | 23 | | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| 27 | + | |
26 | 28 | | |
27 | 29 | | |
28 | 30 | | |
29 | 31 | | |
30 | | - | |
31 | 32 | | |
32 | 33 | | |
33 | 34 | | |
| |||
76 | 77 | | |
77 | 78 | | |
78 | 79 | | |
79 | | - | |
80 | 80 | | |
81 | 81 | | |
82 | 82 | | |
| |||
121 | 121 | | |
122 | 122 | | |
123 | 123 | | |
124 | | - | |
125 | | - | |
126 | | - | |
127 | 124 | | |
128 | 125 | | |
129 | 126 | | |
| |||
135 | 132 | | |
136 | 133 | | |
137 | 134 | | |
| 135 | + | |
| 136 | + | |
138 | 137 | | |
139 | 138 | | |
140 | 139 | | |
| |||
181 | 180 | | |
182 | 181 | | |
183 | 182 | | |
184 | | - | |
| 183 | + | |
185 | 184 | | |
186 | 185 | | |
187 | 186 | | |
| |||
231 | 230 | | |
232 | 231 | | |
233 | 232 | | |
234 | | - | |
235 | | - | |
236 | 233 | | |
237 | 234 | | |
238 | 235 | | |
| |||
250 | 247 | | |
251 | 248 | | |
252 | 249 | | |
253 | | - | |
254 | | - | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
255 | 254 | | |
256 | 255 | | |
257 | 256 | | |
| |||
305 | 304 | | |
306 | 305 | | |
307 | 306 | | |
308 | | - | |
309 | | - | |
310 | | - | |
311 | | - | |
| 307 | + | |
| 308 | + | |
312 | 309 | | |
313 | 310 | | |
314 | | - | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
315 | 317 | | |
316 | 318 | | |
317 | 319 | | |
| |||
320 | 322 | | |
321 | 323 | | |
322 | 324 | | |
323 | | - | |
324 | | - | |
| 325 | + | |
| 326 | + | |
325 | 327 | | |
326 | 328 | | |
327 | 329 | | |
328 | | - | |
| 330 | + | |
329 | 331 | | |
330 | 332 | | |
331 | 333 | | |
| |||
468 | 470 | | |
469 | 471 | | |
470 | 472 | | |
| 473 | + | |
471 | 474 | | |
472 | 475 | | |
473 | | - | |
| 476 | + | |
474 | 477 | | |
475 | 478 | | |
476 | 479 | | |
477 | | - | |
478 | | - | |
479 | | - | |
480 | | - | |
| 480 | + | |
481 | 481 | | |
482 | 482 | | |
483 | 483 | | |
| |||
503 | 503 | | |
504 | 504 | | |
505 | 505 | | |
506 | | - | |
| 506 | + | |
507 | 507 | | |
508 | 508 | | |
509 | 509 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
402 | 402 | | |
403 | 403 | | |
404 | 404 | | |
| 405 | + | |
| 406 | + | |
405 | 407 | | |
406 | 408 | | |
407 | | - | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
408 | 412 | | |
409 | 413 | | |
410 | | - | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
411 | 417 | | |
412 | 418 | | |
413 | 419 | | |
414 | | - | |
415 | | - | |
416 | | - | |
| 420 | + | |
417 | 421 | | |
418 | 422 | | |
419 | 423 | | |
| |||
579 | 583 | | |
580 | 584 | | |
581 | 585 | | |
582 | | - | |
| 586 | + | |
583 | 587 | | |
584 | 588 | | |
585 | 589 | | |
| |||
761 | 765 | | |
762 | 766 | | |
763 | 767 | | |
764 | | - | |
| 768 | + | |
765 | 769 | | |
766 | 770 | | |
767 | 771 | | |
| |||
934 | 938 | | |
935 | 939 | | |
936 | 940 | | |
937 | | - | |
938 | | - | |
939 | 941 | | |
940 | 942 | | |
941 | 943 | | |
942 | 944 | | |
943 | 945 | | |
944 | 946 | | |
| 947 | + | |
| 948 | + | |
945 | 949 | | |
946 | 950 | | |
947 | 951 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
718 | 718 | | |
719 | 719 | | |
720 | 720 | | |
721 | | - | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
| 736 | + | |
| 737 | + | |
| 738 | + | |
| 739 | + | |
| 740 | + | |
722 | 741 | | |
723 | 742 | | |
724 | 743 | | |
725 | 744 | | |
| 745 | + | |
726 | 746 | | |
727 | 747 | | |
728 | 748 | | |
729 | 749 | | |
730 | 750 | | |
| 751 | + | |
| 752 | + | |
| 753 | + | |
731 | 754 | | |
732 | 755 | | |
733 | 756 | | |
734 | | - | |
| 757 | + | |
735 | 758 | | |
736 | 759 | | |
737 | 760 | | |
| |||
0 commit comments