Skip to content

Commit 78a3211

Browse files
badGarnetclaude
andcommitted
fix(etl-uvicorn): do not require a body when every input field is optional
A pydantic body parameter with no default is mandatory even when every field inside the model is optional. `wrap_in_fastapi` chose its `/invoke` signature on parameter *presence*, so a plugin whose parameters are all optional got a required body that no caller has a reason to populate -- and before it grew those parameters the same plugin accepted no body at all. Adding an optional parameter therefore looked backward-compatible while flipping the HTTP contract to 422 for every bodyless caller. Observed in production: the playground indexer gained `invocation_settings`/`invocation_context` (both defaulting to None) and every ephemeral job began failing with [{"type":"missing","loc":["body"],"msg":"Field required","input":null}] The indexer is the first node in the DAG and the source of all documents, so nothing was indexed, every downstream node idled, and the job still reported COMPLETED -- with total_docs 0 and an empty failed-files list. An absent body now resolves each field to its own default, which is what the signature already promised. Plugins with at least one required field keep a mandatory body, so a downloader invoked without `file_data` still fails validation rather than receiving None. The two model-bearing branches share one handler so they cannot drift. Verified against the real playground indexer: a bodyless POST returns 200 and indexes from the settings-file fallback, while a populated body still takes precedence, so the wire-settings migration keeps working. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 4aea423 commit 78a3211

4 files changed

Lines changed: 123 additions & 14 deletions

File tree

CHANGELOG.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,14 @@
1+
## 0.0.45
2+
3+
* **`/invoke` no longer demands a body from a plugin whose parameters are all optional.** A pydantic
4+
body parameter with no default is mandatory even when every field inside the model is optional, so
5+
such a plugin required a body that no caller has a reason to populate — and before it grew those
6+
parameters the same plugin accepted no body at all, which made adding one look
7+
backward-compatible while silently flipping the HTTP contract to 422 for every bodyless caller.
8+
An absent body now resolves each field to its own default, which is what the signature already
9+
promised. Plugins with at least one required field are unchanged: a missing `file_data` still
10+
fails validation rather than arriving as `None`.
11+
112
## 0.0.44
213

314
* **Ignore SIGTERM in plugin uvicorn Servers**: plugin webservers now keep

test/api/test_api.py

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -470,3 +470,79 @@ def test_streaming_unstructured_ingest_error_with_none_status_code():
470470
"Async gen test UnstructuredIngestError with None status_code"
471471
in invoke_response.status_code_text
472472
)
473+
474+
475+
# --- optional-body contract -------------------------------------------------------------------
476+
#
477+
# A pydantic body parameter with no default is mandatory even when every field inside the model is
478+
# optional. A plugin whose parameters are ALL optional therefore used to demand a body that no
479+
# caller has a reason to populate: before it grew those parameters the same plugin accepted no body
480+
# at all, so adding one silently flipped its HTTP contract and every bodyless caller got a 422.
481+
482+
483+
class _Echo(BaseModel):
484+
settings: Optional[dict] = None
485+
context: Optional[dict] = None
486+
received: Optional[str] = None
487+
488+
489+
def _all_optional(
490+
invocation_settings: Optional[dict] = None, invocation_context: Optional[dict] = None
491+
) -> _Echo:
492+
return _Echo(settings=invocation_settings, context=invocation_context)
493+
494+
495+
def _no_params() -> _Echo:
496+
return _Echo(received="ok")
497+
498+
499+
def _has_required(element_dicts: str, invocation_context: Optional[dict] = None) -> _Echo:
500+
return _Echo(received=element_dicts)
501+
502+
503+
@pytest.mark.parametrize("body", [None, {}])
504+
def test_all_optional_params_accept_absent_or_empty_body(body):
505+
client = TestClient(wrap_in_fastapi(func=_all_optional, plugin_id="mock_plugin"))
506+
507+
kwargs = {} if body is None else {"json": body}
508+
resp = client.post("/invoke", **kwargs)
509+
510+
assert resp.status_code == 200
511+
invoke_response = InvokeResponse.model_validate(resp.json())
512+
invoke_response.generic_validation()
513+
# Each field resolves to its own default, which is what the signature already promised.
514+
assert invoke_response.output == {"settings": None, "context": None, "received": None}
515+
516+
517+
def test_all_optional_params_still_receive_a_populated_body():
518+
# The tolerance must not swallow a body that IS supplied, or the wire-settings plane silently
519+
# stops working while every request keeps returning 200.
520+
client = TestClient(wrap_in_fastapi(func=_all_optional, plugin_id="mock_plugin"))
521+
522+
resp = client.post("/invoke", json={"invocation_settings": {"k": "v"}})
523+
524+
assert resp.status_code == 200
525+
assert InvokeResponse.model_validate(resp.json()).output == {
526+
"settings": {"k": "v"},
527+
"context": None,
528+
"received": None,
529+
}
530+
531+
532+
def test_required_param_still_rejects_an_absent_body():
533+
# The tolerance must not leak into plugins that genuinely need input: a downloader invoked
534+
# without `file_data` has to fail loudly rather than receive None.
535+
client = TestClient(wrap_in_fastapi(func=_has_required, plugin_id="mock_plugin"))
536+
537+
assert client.post("/invoke").status_code == 422
538+
assert client.post("/invoke", json={}).status_code == 422
539+
assert client.post("/invoke", json={"element_dicts": "x"}).status_code == 200
540+
541+
542+
def test_no_param_plugin_still_accepts_a_bodyless_post():
543+
client = TestClient(wrap_in_fastapi(func=_no_params, plugin_id="mock_plugin"))
544+
545+
resp = client.post("/invoke")
546+
547+
assert resp.status_code == 200
548+
assert InvokeResponse.model_validate(resp.json()).output["received"] == "ok"
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__version__ = "0.0.44" # pragma: no cover
1+
__version__ = "0.0.45" # pragma: no cover

unstructured_platform_plugins/etl_uvicorn/api_generator.py

Lines changed: 35 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -254,22 +254,44 @@ async def _stream_response():
254254
file_data=request_dict.get("file_data", None),
255255
)
256256

257-
if input_schema_model.model_fields:
257+
async def run_job_with_body(request: BaseModel) -> ResponseType:
258+
log_func_and_body(func=func, body=request.json())
259+
# Create dictionary from pydantic model while preserving underlying types
260+
request_dict = {f: getattr(request, f) for f in request.model_fields}
261+
# Make sure nested classes get instantiated correctly
262+
if "file_data" in request_dict:
263+
request_dict["file_data"] = file_data_from_dict(request_dict["file_data"].model_dump())
264+
map_inputs(func=func, raw_inputs=request_dict)
265+
if logger.level == LOG_LEVELS.get("trace", logging.NOTSET):
266+
logger.log(level=logger.level, msg=f"passing inputs to function: {request_dict}")
267+
return await wrap_fn(func=func, kwargs=request_dict)
268+
269+
# A pydantic body parameter with no default is mandatory even when every field inside the model
270+
# is optional. So a plugin whose parameters are ALL optional would demand a body that no caller
271+
# has a reason to populate -- and before it grew those parameters that same plugin accepted no
272+
# body at all, so adding one flips the HTTP contract while looking backward-compatible. Default
273+
# the body in that case: an absent body resolves each field to its own default, which is exactly
274+
# what the function signature already promises.
275+
#
276+
# A plugin with at least one required field keeps a mandatory body, so an indexer that needs
277+
# `file_data` still fails validation rather than silently receiving None.
278+
body_is_optional = input_schema_model.model_fields and not any(
279+
field.is_required() for field in input_schema_model.model_fields.values()
280+
)
281+
282+
if body_is_optional:
283+
284+
@fastapi_app.post("/invoke", response_model=InvokeResponse)
285+
async def run_job(request: Optional[input_schema_model] = None) -> ResponseType:
286+
return await run_job_with_body(
287+
request if request is not None else input_schema_model()
288+
)
289+
290+
elif input_schema_model.model_fields:
258291

259292
@fastapi_app.post("/invoke", response_model=InvokeResponse)
260293
async def run_job(request: input_schema_model) -> ResponseType:
261-
log_func_and_body(func=func, body=request.json())
262-
# Create dictionary from pydantic model while preserving underlying types
263-
request_dict = {f: getattr(request, f) for f in request.model_fields}
264-
# Make sure nested classes get instantiated correctly
265-
if "file_data" in request_dict:
266-
request_dict["file_data"] = file_data_from_dict(
267-
request_dict["file_data"].model_dump()
268-
)
269-
map_inputs(func=func, raw_inputs=request_dict)
270-
if logger.level == LOG_LEVELS.get("trace", logging.NOTSET):
271-
logger.log(level=logger.level, msg=f"passing inputs to function: {request_dict}")
272-
return await wrap_fn(func=func, kwargs=request_dict)
294+
return await run_job_with_body(request)
273295

274296
else:
275297

0 commit comments

Comments
 (0)