Repository navigation
perf: reuse serializer wrapper models instead of rebuilding them on every call - #1335
Merged
jason-schrader-openrouter merged 2 commits intoNov 13, 2026
Merged
Conversation
…very call marshal_json and unmarshal call pydantic.create_model on every call, so each chat request built about 19 model classes. An SDK init hook now caches one wrapper model per type. Fixes #1056.
Contributor
|
I'll fix CI failures and address comments from users with write access that start with 'Devin'.
|
christineschen
approved these changes
Nov 13, 2026
jason-schrader-openrouter
deleted the
devin/1794545319-cache-serializer-models
branch
November 13, 2026 19:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1056.
utils/serializers.pycallspydantic.create_model("Unmarshaller" | "Marshaller", body=(typ, ...), __config__=...)every timemarshal_jsonorunmarshalruns. As a result, eachchat.send/chat.send_asynccall builds and compiles about 19 throwaway model classes. That file is Speakeasy template code and gets overwritten on regeneration, so this PR fixes it from_hooks/, which regeneration doesn't overwrite:_hooks/serializer_cache.py.install()replacesserializers.create_modelwith a version that builds one model per(name, type, config)and keeps it in a boundedlru_cache(_CACHE_SIZE = 1024)._type_key(typ), nottypalone. Annotations that compare equal can still validate differently, so the key keeps union member order and the type of each leaf:unmarshal("1", int | float)followed byunmarshal("1", float | int)returned1(int) from the cache instead of1.0(float).pydantic.create_model. If the template changes, the SDK keeps working; it just stops caching.init_hooks()inregistration.pycallsinstall(), so the cache is active as soon as anOpenRouterclient is created.Measured on 1.3.173 with a local stub server (Python 3.12, pydantic 2.14.3):
tests/test_serializer_cache.pychecks that:chat.sendbuilds no new models;Optional, generics and validation errors,unmarshalandmarshal_jsongive the same result type, value or error with the cache as with plainpydantic.create_model, after warming the cache in forward and reverse order;_CACHE_SIZE.The equivalence and bound tests fail against the previous cache key. Delete this hook once the upstream template caches these models itself.