Repository navigation
Commit 27553e2
fix: stop failing on rejected jobs and add skip_preflight (0.47.0) (#356)
## What
Jobs the new preflight check rejects no longer break the SDK. Workflows
can also opt out of preflight from the SDK.
**Before**, after a job is rejected:
```
client.jobs.list_jobs(...) → ValidationError: 'REJECTED' is not a valid JobStatus (whole page lost)
client.jobs.get_job(...) → ValidationError
```
**After:**
```
job.status → JobStatus.REJECTED
job.status → "SOME_FUTURE_STATUS" # a status this SDK version doesn't know comes back as a plain str instead of raising
details.node_stats[0].failure_category → PreflightFailureCategory.AUTH_INVALID_CREDENTIAL
details.node_stats[0].failure_reason → "The AWS Access Key Id you provided does not exist."
```
- `JobStatus` and `JobProcessingStatus` gain `REJECTED`.
`JobInformation.status` and `JobDetails.processing_status` also accept
unknown values. Known values are still enum members, so `job.status ==
JobStatus.COMPLETED` and `.value` behave as before.
- `JobNodeDetails` gains `failure_category`
(`Union[PreflightFailureCategory, str]`) and `failure_reason`, so a
caller can see why a job was rejected. The server sets them only on the
node that failed preflight; before this, pydantic dropped them.
- `skip_preflight` is now on `CreateWorkflow`, `UpdateWorkflow` (unset =
unchanged) and `WorkflowInformation`. For a one-off `create_job`, the
API defines the flag inside the `request_data` JSON. The docs now say to
set it there; the SDK does not rewrite that string.
- Releases as **0.47.0**.
## Why
The SDK models used to be regenerated from the API spec every night.
That generator has been dead since January (removed in #349). Since
then, new server values reach the SDK only when someone adds them by
hand. `REJECTED` didn't get added, and a closed enum turns a new value
into a hard failure.
This PR also adds a way to catch that kind of drift before a customer
does (DTPL-696 acceptance criterion 4):
```
weekly CI → fetch live OpenAPI specs → compare with SDK models → new missing field/enum value → open/update a GitHub issue
(a failed download fails the run but opens no issue)
```
- `make check-spec-drift` runs the check locally. Gaps we already know
about are recorded in `scripts/spec_drift_baseline.json` (20 schemas
today, e.g. `JobInformation.processing_status`, the `teradata` connector
type). The check fails only on new drift, or on a baseline entry that
has since been covered.
- `CONTRIBUTING.md` documents how to resolve a drift issue.
Replaces #353, which got here through a custom `_missing_` enum hook and
code that spliced bytes into `request_data`. This version uses a plain
pydantic union and leaves `request_data` alone.
## Notes for consumers
- Type checkers now see these two status fields as `Union[JobStatus,
str]` / `Union[JobProcessingStatus, str]`. Code that reads `.value` or
`.name` on them needs to handle the `str` case. Runtime behavior for
known statuses is unchanged.
- Follow-up, not in this PR: the SDK's other "open" enums (connector
types, partition `Strategy`, etc.) use `OpenEnumMeta` /
`validate_open_enum`. On Python 3.12+ those return a raw `str` even for
known values, and they emit pydantic serializer warnings. The long-term
fix is to move them to the same union pattern used here. That changes
their runtime types, so it belongs in its own PR.
## Validation
- `make test-unit`: 419 passed. `make test-contract`: 72 passed. The new
job/workflow contract tests pass on Python 3.11, 3.12 and 3.13, and they
fail without the model changes.
- `make lint` is clean. `make check-spec-drift` passes against the live
specs.
- Two rounds of Codex review. Round 1 found that enums declared inline
on a property escaped the drift check; that is fixed and has a test.
Round 2 was clean.
- cubic: three findings fixed (no drift issue on a fetch failure, no
duplicate issues from overlapping runs, changelog wording). Re-review
clean, and full CI is green, including integration on 3.11–3.13.
## Maintenance validation (2026-10-19, keepalive base refresh)
- Synchronized main through its 0.46.5 keepalive release while retaining
the authored 0.47.0 version and user-agent. Manual conflict resolution
touched only the changelog and version file; main runtime and test
changes merged automatically.
- Local validation: 419 unit tests and 71 contract tests passed (one
existing expected unit failure). Four localhost socket tests needed bind
permission and passed without source changes.
- Exact maintenance-head CI:
https://github.com/Unstructured-IO/unstructured-python-client/actions/runs/39329146842.
## Review follow-up (2026-10-20)
- Added `failure_category` / `failure_reason` to `JobNodeDetails` with a
new `PreflightFailureCategory` enum, and dropped both from the drift
baseline. A new contract test covers a known category, an unknown
category (plain `str`) and a passing node (both null); it passes on
3.11, 3.12 and 3.13.
- Removed the `JobStatus` / `JobProcessingStatus` entries from
`overlay_client.yaml`. Nothing reads the overlay since the generator was
removed, and `x-speakeasy-unknown-values` would describe the
`OpenEnumMeta` style, not the union shipped here.
- `make test-unit` 419 passed, `make test-contract` 72 passed, `make
lint` clean, `make check-spec-drift` clean against the live specs.
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>1 parent 91410b0 commit 27553e2
30 files changed
Lines changed: 676 additions & 32 deletions
File tree
- .github/workflows
- _test_contract/platform_api
- _test_unstructured_client/unit
- scripts
- src/unstructured_client
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
1 | 10 | | |
2 | 11 | | |
3 | 12 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
20 | 33 | | |
21 | 34 | | |
22 | 35 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
71 | 75 | | |
72 | 76 | | |
73 | 77 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
222 | 222 | | |
223 | 223 | | |
224 | 224 | | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
1 | 2 | | |
2 | 3 | | |
3 | 4 | | |
| |||
256 | 257 | | |
257 | 258 | | |
258 | 259 | | |
259 | | - | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
| 9 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
0 commit comments