overrides: add makeRequired DSL + 6 input-shape quirks + 5 service-method JSDocs - #13
Merged
Merged
Conversation
The SCHEMA_OVERRIDES DSL handled "spec marks required, API doesn't" (makeOptional) and "field missing from spec entirely" (addRequiredFields) but had no way to express "spec marks optional, API requires it" — a common case where the spec is structurally correct but doesn't capture empirical platform behavior. Add `makeRequired: string[]` symmetric to `makeOptional`. Also tighten validation. Existing override loader fails generation when a SCHEMA name is removed from the spec; extend the same loud-failure rule to FIELD names referenced by makeOptional / makeRequired / fieldTypes. A field renamed upstream would have silently no-op'd before, leaving the override stale. The validation runs alongside the existing application loop so per-field checks share the collectFlatProperties view. Reject contradictions: a single field appearing in both makeOptional and makeRequired is a config error, not last-wins. Negative-tested: introducing a stale field name to an existing override now fails generation with a clear message; reverted to the working set, all 5 existing overrides still emit. Existing 215 tests pass.
Four schemas affected by API behaviors not captured in the OpenAPI spec:
ConnectionSchemaFieldInputRepresentation
Server NPEs ("Cannot invoke java.lang.CharSequence.length() because
this.text is null") when `label` is omitted from a schema field. The
upsert handler dereferences label unconditionally despite the spec
marking it optional. New `makeRequired: ["label"]`.
DataStreamFieldMappingInputRepresentation
Two related issues:
1. POST input uses `sourceFieldLabel` but GET responses echo
`sourceFieldName` for the same field. Round-tripping GET→POST
without renaming fails with JSON_PARSER_ERROR.
2. `targetFieldReturntype` is required by the create handler —
when omitted, the mapping is silently dropped from the saved
data stream with no error.
Documented in the note; `makeRequired: ["targetFieldReturntype"]`.
DataStreamSourceFieldInputRepresentation
Same pattern as #1 above: POST uses `dataType` (camelCase), GET
echoes `datatype` (lowercase). Documented; no shape change.
VectorEmbeddingInputRepresentation
Server NPEs when vectorEmbeddingRelatedFields is omitted, empty, or
null. Spec marks it optional. New `makeRequired:
["vectorEmbeddingRelatedFields"]`.
Also extend two existing override notes:
DataStreamInputRepresentation
Add routing note: `dataAccessMode = "Direct_Access"` is required for
federated/BYOL connectors (Snowflake, Databricks, BigQuery, Iceberg)
— without it the create returns the misleading
`400 INTERNAL_ERROR: Unable to post Data Stream: DATA_CONNECTORS is
not supported`. Direct_Access streams must also OMIT `datasource`.
SemanticSearchInputRepresentation
Add note: input rejects output-only display-name fields
(sourceDmoName, sourceDmoFieldName, relatedDmoName,
relatedDmoFieldName) that GET responses include — round-tripping
GET→POST returns 500 UNKNOWN_EXCEPTION with no diagnostic body.
Spec drift (unrelated): regenerated output also picks up new fields
on Activation* schemas (activationType, sourceDmoName) added to the
upstream spec.
215 tests pass; typecheck clean.
Schema-shape quirks live in SCHEMA_OVERRIDES; behavioral quirks
(timing, race conditions, asymmetric path-param semantics) belong on
the service-class methods themselves. Convert three thin re-exports
into subclasses solely to host the JSDoc.
ConnectionsService
- create: name-lock race after DELETE → DUPLICATES_DETECTED for
several seconds; recommend ~30s retry budget on that error code.
IngestApi name-rewrite caveat (`<label>_<uuid>`).
- delete (new override): transient 500 UNKNOWN_EXCEPTION from the
platform's teardown race even when deletable=true; recommend ~30s
5xx retry budget.
CalculatedInsightsService
- create: server-side validation+schedule registration commonly
takes 30–60s; SDK's default 30s timeout aborts. Recommend
`{ timeout: 120_000 }`.
- delete: asynchronous teardown — 204 immediately but record sits
at status=DELETING for minutes. GET still returns the record
during that window; same-name re-create returns
DUPLICATES_DETECTED until teardown completes.
SearchIndexService
- delete: path parameter accepts the platform-assigned ID but NOT
the developer name. Asymmetric vs. GET (which accepts either).
Each method retains its original signature and delegates via super.*;
no behavior change, only JSDoc. Existing 215 tests still pass; type
check clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three independent changes, one PR for review economy. Each is its own commit
so they can be split if needed.
generate-types: extend the override DSL withmakeRequiredandadd field-level validation. Symmetric to the existing
makeOptional;covers the case "spec marks optional, API requires it" — distinct from
addRequiredFields(used when the spec omits the field entirely).SCHEMA_OVERRIDES: 4 new entries + 2 extended notes for input-shapequirks observed during integration testing.
src/resources/: behavioral JSDoc on five service methods —timing, async-teardown, asymmetric path-params, name-rewrite quirks
that don't fit the type-shape DSL.
215 tests pass.
tsc --noEmitclean. No public-API breakage; the onlytype-narrowing changes are on input shapes that the API was already
rejecting at runtime when the new requirements were violated.
Why this matters
The SDK already has precedent for runtime overrides on schemas where the
OpenAPI spec is structurally accurate but doesn't capture empirical platform
behavior. This PR continues that work: each change is grounded in a concrete
server response that the prior typings would have let consumers compile and
ship.
A representative example — without this PR, the following compiles cleanly
but fails at deploy:
```ts
await client.searchIndex.create({
developerName: "MyIdx",
label: "MyIdx",
searchType: "HYBRID",
processingType: "NEAR_REALTIME",
sourceDmoDeveloperName: "X__dlm",
chunkDmoDeveloperName: "MyIdx_chunk",
chunkDmoName: "MyIdx_chunk",
vectorDmoDeveloperName: "MyIdx_index",
vectorDmoName: "MyIdx_index",
chunkingConfiguration: { fieldLevelConfigurations: [/.../] },
vectorEmbedding: {}, // ← server NPEs here
vectorEmbeddingConfiguration: { /.../ },
});
// → 500 UNKNOWN_EXCEPTION (empty diagnostic body)
```
After this PR, `vectorEmbeddingRelatedFields` is required on
`VectorEmbeddingInputRepresentation` and the omission is caught at
compile time with a helpful `@override` JSDoc explaining why.
Commit 1: `generate-types` — `makeRequired` + field validation
The override loader had two gaps:
triggers when the field is absent from the schema — wrong tool when the
field exists and just needs its required-ness flipped.
but a renamed/removed field referenced by `makeOptional` or `fieldTypes`
silently no-op'd, leaving stale overrides invisible until they were
actively misleading.
Adds:
— a stale field name now fails generation with the same loud message
the schema-name validation already produces.
and `makeRequired` errors at config-load time rather than last-wins.
Negative-tested: introducing a stale field name to an existing override
produces
Commit 2: 4 new overrides + 2 extended notes
`ConnectionSchemaFieldInputRepresentation` — `makeRequired: ["label"]`
`PUT /ssot/connections/{id}/schema` returns
`500 INTERNAL_SERVER_ERROR` with the Java NPE
`Cannot invoke java.lang.CharSequence.length() because "this.text" is null`
when any field in the schema is missing `label`.
Reproducer:
```http
PUT /services/data/v66.0/ssot/connections/{id}/schema
{
"schemas": [{
"schemaType": "IngestApi",
"name": "KB",
"label": "KB",
"fields": [
{ "name": "Id", "dataType": "Text" } // ← no label → NPE
]
}]
}
```
Adding `label: "Id"` succeeds. Spec marks `label` optional on the base
`ConnectionSchemaFieldInputRepresentation`; the upsert handler dereferences
it unconditionally.
`DataStreamFieldMappingInputRepresentation` — `makeRequired: ["targetFieldReturntype"]`
Two related issues, one structural and one documentary.
Structural. `targetFieldReturntype` is required by the data-stream
create handler, but the spec marks it optional. When omitted, the create
returns 201 but the mapping is silently dropped from the saved data
stream — no error, no warning, just missing data flow at runtime.
Documentary (note only). POST input uses `sourceFieldLabel`, but GET
responses echo `sourceFieldName` for the same field. Round-tripping a GET
shape into a POST without renaming fails with
`JSON_PARSER_ERROR: Unrecognized field "sourceFieldName"`.
`DataStreamSourceFieldInputRepresentation` — note only
Same input/output asymmetry pattern as above: POST uses `dataType`
(camelCase), GET echoes `datatype` (lowercase). Round-tripping fails with
`JSON_PARSER_ERROR: Unrecognized field "datatype"`.
`VectorEmbeddingInputRepresentation` — `makeRequired: ["vectorEmbeddingRelatedFields"]`
Server NPEs when the array is omitted, empty, or null. Spec marks it
optional. Typical minimum is a single entry pointing at the source DMO's
primary key.
`DataStreamInputRepresentation` — extended note
The spec types `dataAccessMode?: "Direct_Access" | "Ingest"` correctly,
but doesn't document that `Direct_Access` is required for federated /
BYOL connectors (Snowflake, Databricks, BigQuery, Iceberg). Without it,
the create handler routes to the ingest path and returns the misleading
error `400 INTERNAL_ERROR: Unable to post Data Stream: DATA_CONNECTORS
is not supported` even when the connector is GA. The connector's
`features` array exposes `BYOL` as a hint, but that's the only public
signal.
Direct_Access streams must also OMIT the top-level `datasource` field —
the server returns
`400 BAD_REQUEST: DataSource name should be empty for External data streams`
otherwise. The connection binding is established through
`connectorInfo.connectorDetails.name` instead.
`SemanticSearchInputRepresentation` — extended note
The input handler rejects output-only display-name fields that GET
responses include alongside the developer-name fields. Specifically:
`sourceDmoName`, `sourceDmoFieldName`, `relatedDmoName`,
`relatedDmoFieldName`. Round-tripping a GET shape into a POST returns
`500 UNKNOWN_EXCEPTION` with no diagnostic body. Pass developer-name
fields only.
Commit 3: Service-class JSDoc
These don't fit the type-shape DSL because they describe behavior rather
than shape: timing windows, async teardowns, retry guidance, path-param
asymmetries. Three resource files converted from thin re-exports into
subclasses solely to host the JSDoc; signatures unchanged, all methods
delegate via `super.*`.
`DUPLICATES_DETECTED` for several seconds; recommend ~30s retry budget
on that error code. IngestApi name-rewrite (`_`) caveat.
from the platform's teardown race, even when `deletable: true` and no
live dependents. Recommend ~30s 5xx retry budget.
schedule registration commonly takes 30–60s; SDK's default 30s timeout
aborts. Recommend `{ timeout: 120_000 }`.
`status = "DELETING"` for minutes; same-name re-create fails until
teardown completes.
platform-assigned ID but NOT the developer name (asymmetric vs. GET
which accepts either).
A note on the diff
`src/generated/openapi.{yaml,d.ts}` and parts of `src/schemas.ts` include
unrelated spec drift picked up by `npm run generate` (new `activationType`
and `sourceDmoName` fields on Activation* schemas). Reviewing on a
branch where the maintainer regenerates fresh would suppress these.
Test plan
upstream spec
an override fails generation with a clear message)
on each platform quirk