Skip to content

Commit 648bdc8

Browse files
committed
feat(feature-store): add BatchWriteRecord and ListRecords to FeatureGroup
Add two new FeatureStore Runtime operations to the FeatureGroup resource: - batch_write_record: writes a batch of records to one or more FeatureGroups - list_records: lists RecordIdentifier values stored in a FeatureGroup's OnlineStore Changes: - Update service-2.json from botocore 1.43.38 - Add shape classes (BatchWriteRecordEntry, BatchWriteRecordResponse, etc.) - Add shape_dag entries and additional_operations mapping - Add exclude_resource_attributes support to codegen to prevent FeatureGroup.next_token from being incorrectly mapped to ListRecords.NextToken (different pagination contexts) - Regenerate resources.py and shapes.py via codegen --- X-AI-Prompt: add batch_write_record and list_records to sagemaker-core FeatureGroup, fix codegen next_token bug X-AI-Tool: kiro-cli
1 parent 3024a57 commit 648bdc8

7 files changed

Lines changed: 800 additions & 12 deletions

File tree

‎sagemaker-core/sample/sagemaker-featurestore-runtime/2020-07-01/service-2.json‎

Lines changed: 188 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,13 @@
55
"endpointPrefix":"featurestore-runtime.sagemaker",
66
"jsonVersion":"1.1",
77
"protocol":"rest-json",
8+
"protocols":["rest-json"],
89
"serviceFullName":"Amazon SageMaker Feature Store Runtime",
910
"serviceId":"SageMaker FeatureStore Runtime",
1011
"signatureVersion":"v4",
1112
"signingName":"sagemaker",
12-
"uid":"sagemaker-featurestore-runtime-2020-07-01"
13+
"uid":"sagemaker-featurestore-runtime-2020-07-01",
14+
"auth":["aws.auth#sigv4"]
1315
},
1416
"operations":{
1517
"BatchGetRecord":{
@@ -28,6 +30,23 @@
2830
],
2931
"documentation":"<p>Retrieves a batch of <code>Records</code> from a <code>FeatureGroup</code>.</p>"
3032
},
33+
"BatchWriteRecord":{
34+
"name":"BatchWriteRecord",
35+
"http":{
36+
"method":"POST",
37+
"requestUri":"/BatchWriteRecord"
38+
},
39+
"input":{"shape":"BatchWriteRecordRequest"},
40+
"output":{"shape":"BatchWriteRecordResponse"},
41+
"errors":[
42+
{"shape":"ValidationError"},
43+
{"shape":"ResourceNotFound"},
44+
{"shape":"InternalFailure"},
45+
{"shape":"ServiceUnavailable"},
46+
{"shape":"AccessForbidden"}
47+
],
48+
"documentation":"<p>Writes a batch of <code>Records</code> to one or more <code>FeatureGroup</code>s. Use this API for bulk ingestion of records into the <code>OnlineStore</code> and <code>OfflineStore</code>.</p> <p>You can set the ingested records to expire at a given time to live (TTL) duration after the record's event time by specifying the <code>TtlDuration</code> parameter. A request level <code>TtlDuration</code> applies to all entries that do not specify their own <code>TtlDuration</code>.</p>"
49+
},
3150
"DeleteRecord":{
3251
"name":"DeleteRecord",
3352
"http":{
@@ -41,7 +60,7 @@
4160
{"shape":"ServiceUnavailable"},
4261
{"shape":"AccessForbidden"}
4362
],
44-
"documentation":"<p>Deletes a <code>Record</code> from a <code>FeatureGroup</code> in the <code>OnlineStore</code>. Feature Store supports both <code>SoftDelete</code> and <code>HardDelete</code>. For <code>SoftDelete</code> (default), feature columns are set to <code>null</code> and the record is no longer retrievable by <code>GetRecord</code> or <code>BatchGetRecord</code>. For <code>HardDelete</code>, the complete <code>Record</code> is removed from the <code>OnlineStore</code>. In both cases, Feature Store appends the deleted record marker to the <code>OfflineStore</code>. The deleted record marker is a record with the same <code>RecordIdentifer</code> as the original, but with <code>is_deleted</code> value set to <code>True</code>, <code>EventTime</code> set to the delete input <code>EventTime</code>, and other feature values set to <code>null</code>.</p> <p>Note that the <code>EventTime</code> specified in <code>DeleteRecord</code> should be set later than the <code>EventTime</code> of the existing record in the <code>OnlineStore</code> for that <code>RecordIdentifer</code>. If it is not, the deletion does not occur:</p> <ul> <li> <p>For <code>SoftDelete</code>, the existing (not deleted) record remains in the <code>OnlineStore</code>, though the delete record marker is still written to the <code>OfflineStore</code>.</p> </li> <li> <p> <code>HardDelete</code> returns <code>EventTime</code>: <code>400 ValidationException</code> to indicate that the delete operation failed. No delete record marker is written to the <code>OfflineStore</code>.</p> </li> </ul> <p>When a record is deleted from the <code>OnlineStore</code>, the deleted record marker is appended to the <code>OfflineStore</code>. If you have the Iceberg table format enabled for your <code>OfflineStore</code>, you can remove all history of a record from the <code>OfflineStore</code> using Amazon Athena or Apache Spark. For information on how to hard delete a record from the <code>OfflineStore</code> with the Iceberg table format enabled, see <a href=\"https://docs.aws.amazon.com/sagemaker/latest/dg/feature-store-delete-records-offline-store.html#feature-store-delete-records-offline-store\">Delete records from the offline store</a>.</p>"
63+
"documentation":"<p>Deletes a <code>Record</code> from a <code>FeatureGroup</code> in the <code>OnlineStore</code>. Feature Store supports both <code>SoftDelete</code> and <code>HardDelete</code>. For <code>SoftDelete</code> (default), feature columns are set to <code>null</code> and the record is no longer retrievable by <code>GetRecord</code> or <code>BatchGetRecord</code>. For <code>HardDelete</code>, the complete <code>Record</code> is removed from the <code>OnlineStore</code>. In both cases, Feature Store appends the deleted record marker to the <code>OfflineStore</code>. The deleted record marker is a record with the same <code>RecordIdentifer</code> as the original, but with <code>is_deleted</code> value set to <code>True</code>, <code>EventTime</code> set to the delete input <code>EventTime</code>, and other feature values set to <code>null</code>.</p> <p>Note that the <code>EventTime</code> specified in <code>DeleteRecord</code> should be set later than the <code>EventTime</code> of the existing record in the <code>OnlineStore</code> for that <code>RecordIdentifer</code>. If it is not, the deletion does not occur:</p> <ul> <li> <p>For <code>SoftDelete</code>, the existing (not deleted) record remains in the <code>OnlineStore</code>, though the delete record marker is still written to the <code>OfflineStore</code>.</p> </li> <li> <p> <code>HardDelete</code> returns <code>EventTime</code>: <code>400 ValidationException</code> to indicate that the delete operation failed. No delete record marker is written to the <code>OfflineStore</code>.</p> </li> </ul> <p>When a record is deleted from the <code>OnlineStore</code>, the deleted record marker is appended to the <code>OfflineStore</code>. If you have the Iceberg table format enabled for your <code>OfflineStore</code>, you can remove all history of a record from the <code>OfflineStore</code> using Amazon Athena or Apache Spark. For information on how to hard delete a record from the <code>OfflineStore</code> with the Iceberg table format enabled, see <a href=\"https://docs.aws.amazon.com/sagemaker/latest/dg/feature-store-delete-records.html#feature-store-delete-records-offline-store\">Delete records from the offline store</a>.</p>"
4564
},
4665
"GetRecord":{
4766
"name":"GetRecord",
@@ -60,6 +79,24 @@
6079
],
6180
"documentation":"<p>Use for <code>OnlineStore</code> serving from a <code>FeatureStore</code>. Only the latest records stored in the <code>OnlineStore</code> can be retrieved. If no Record with <code>RecordIdentifierValue</code> is found, then an empty result is returned. </p>"
6281
},
82+
"ListRecords":{
83+
"name":"ListRecords",
84+
"http":{
85+
"method":"POST",
86+
"requestUri":"/FeatureGroup/{FeatureGroupName}/ListRecords"
87+
},
88+
"input":{"shape":"ListRecordsRequest"},
89+
"output":{"shape":"ListRecordsResponse"},
90+
"errors":[
91+
{"shape":"ValidationError"},
92+
{"shape":"ResourceNotFound"},
93+
{"shape":"InternalFailure"},
94+
{"shape":"ServiceUnavailable"},
95+
{"shape":"AccessForbidden"}
96+
],
97+
"documentation":"<p>Lists the <code>RecordIdentifier</code> values of all records stored in a <code>FeatureGroup</code>'s <code>OnlineStore</code>. This enables you to discover which records exist without retrieving the full record data.</p>",
98+
"readonly":true
99+
},
63100
"PutRecord":{
64101
"name":"PutRecord",
65102
"http":{
@@ -216,6 +253,98 @@
216253
"member":{"shape":"BatchGetRecordResultDetail"},
217254
"min":0
218255
},
256+
"BatchWriteRecordEntries":{
257+
"type":"list",
258+
"member":{"shape":"BatchWriteRecordEntry"},
259+
"max":25,
260+
"min":1
261+
},
262+
"BatchWriteRecordEntry":{
263+
"type":"structure",
264+
"required":[
265+
"FeatureGroupName",
266+
"Record"
267+
],
268+
"members":{
269+
"FeatureGroupName":{
270+
"shape":"FeatureGroupNameOrArn",
271+
"documentation":"<p>The name or Amazon Resource Name (ARN) of the <code>FeatureGroup</code> to write the record to.</p>"
272+
},
273+
"Record":{
274+
"shape":"Record",
275+
"documentation":"<p>List of FeatureValues to be inserted. This will be a full over-write.</p>"
276+
},
277+
"TargetStores":{
278+
"shape":"TargetStores",
279+
"documentation":"<p>A list of stores to which you're adding the record. By default, Feature Store adds the record to all of the stores that you're using for the <code>FeatureGroup</code>.</p>"
280+
},
281+
"TtlDuration":{
282+
"shape":"TtlDuration",
283+
"documentation":"<p>Time to live duration for this entry, where the record is hard deleted after the expiration time is reached; <code>ExpiresAt</code> = <code>EventTime</code> + <code>TtlDuration</code>. This overrides the request level <code>TtlDuration</code>.</p>"
284+
}
285+
},
286+
"documentation":"<p>An entry to write as part of a <code>BatchWriteRecord</code> request.</p>"
287+
},
288+
"BatchWriteRecordError":{
289+
"type":"structure",
290+
"required":[
291+
"Entry",
292+
"ErrorCode",
293+
"ErrorMessage"
294+
],
295+
"members":{
296+
"Entry":{
297+
"shape":"BatchWriteRecordEntry",
298+
"documentation":"<p>The entry that failed to be written.</p>"
299+
},
300+
"ErrorCode":{
301+
"shape":"ValueAsString",
302+
"documentation":"<p>The error code for the failed record write.</p>"
303+
},
304+
"ErrorMessage":{
305+
"shape":"Message",
306+
"documentation":"<p>The error message for the failed record write.</p>"
307+
}
308+
},
309+
"documentation":"<p>The error that has occurred when attempting to write a record in a batch.</p>"
310+
},
311+
"BatchWriteRecordErrors":{
312+
"type":"list",
313+
"member":{"shape":"BatchWriteRecordError"},
314+
"min":0
315+
},
316+
"BatchWriteRecordRequest":{
317+
"type":"structure",
318+
"required":["Entries"],
319+
"members":{
320+
"Entries":{
321+
"shape":"BatchWriteRecordEntries",
322+
"documentation":"<p>A list of records to write. Each entry specifies the <code>FeatureGroup</code>, the record data, and optionally target stores and a TTL duration.</p>"
323+
},
324+
"TtlDuration":{
325+
"shape":"TtlDuration",
326+
"documentation":"<p>Time to live duration applied to all entries in the batch that do not specify their own <code>TtlDuration</code>; <code>ExpiresAt</code> = <code>EventTime</code> + <code>TtlDuration</code>. For information on HardDelete, see the <a href=\"https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_feature_store_DeleteRecord.html\">DeleteRecord</a> API in the Amazon SageMaker API Reference guide.</p>"
327+
}
328+
}
329+
},
330+
"BatchWriteRecordResponse":{
331+
"type":"structure",
332+
"required":[
333+
"Errors",
334+
"UnprocessedEntries"
335+
],
336+
"members":{
337+
"Errors":{
338+
"shape":"BatchWriteRecordErrors",
339+
"documentation":"<p>A list of errors that occurred when writing records in the batch.</p>"
340+
},
341+
"UnprocessedEntries":{
342+
"shape":"UnprocessedBatchWriteRecordEntries",
343+
"documentation":"<p>A list of entries that were not processed. These entries can be retried.</p>"
344+
}
345+
}
346+
},
347+
"Boolean":{"type":"boolean"},
219348
"DeleteRecordRequest":{
220349
"type":"structure",
221350
"required":[
@@ -364,6 +493,54 @@
364493
"fault":true,
365494
"synthetic":true
366495
},
496+
"ListRecordsMaxResults":{
497+
"type":"integer",
498+
"max":100,
499+
"min":1
500+
},
501+
"ListRecordsNextToken":{
502+
"type":"string",
503+
"max":8192,
504+
"min":1
505+
},
506+
"ListRecordsRequest":{
507+
"type":"structure",
508+
"required":["FeatureGroupName"],
509+
"members":{
510+
"FeatureGroupName":{
511+
"shape":"FeatureGroupNameOrArn",
512+
"documentation":"<p>The name or Amazon Resource Name (ARN) of the feature group to list records from.</p>",
513+
"location":"uri",
514+
"locationName":"FeatureGroupName"
515+
},
516+
"MaxResults":{
517+
"shape":"ListRecordsMaxResults",
518+
"documentation":"<p>The maximum number of record identifiers to return in a single page of results. For the <code>InMemory</code> tier, this value is a hint and not a strict requirement. The response may contain more or fewer results than the specified <code>MaxResults</code>.</p>"
519+
},
520+
"NextToken":{
521+
"shape":"ListRecordsNextToken",
522+
"documentation":"<p>A token to resume pagination of <code>ListRecords</code> results.</p>"
523+
},
524+
"IncludeSoftDeletedRecords":{
525+
"shape":"Boolean",
526+
"documentation":"<p>If set to <code>true</code>, the result includes records that have been soft deleted.</p>"
527+
}
528+
}
529+
},
530+
"ListRecordsResponse":{
531+
"type":"structure",
532+
"required":["RecordIdentifiers"],
533+
"members":{
534+
"RecordIdentifiers":{
535+
"shape":"RecordIdentifierList",
536+
"documentation":"<p>A list of record identifier values for the records stored in the <code>OnlineStore</code>.</p>"
537+
},
538+
"NextToken":{
539+
"shape":"ListRecordsNextToken",
540+
"documentation":"<p>A token to resume pagination if the response includes more record identifiers than <code>MaxResults</code>.</p>"
541+
}
542+
}
543+
},
367544
"Message":{
368545
"type":"string",
369546
"max":2048
@@ -400,6 +577,10 @@
400577
"member":{"shape":"FeatureValue"},
401578
"min":1
402579
},
580+
"RecordIdentifierList":{
581+
"type":"list",
582+
"member":{"shape":"ValueAsString"}
583+
},
403584
"RecordIdentifiers":{
404585
"type":"list",
405586
"member":{"shape":"ValueAsString"},
@@ -471,6 +652,11 @@
471652
"type":"integer",
472653
"min":1
473654
},
655+
"UnprocessedBatchWriteRecordEntries":{
656+
"type":"list",
657+
"member":{"shape":"BatchWriteRecordEntry"},
658+
"min":0
659+
},
474660
"UnprocessedIdentifiers":{
475661
"type":"list",
476662
"member":{"shape":"BatchGetRecordIdentifier"},

‎sagemaker-core/src/sagemaker/core/resources.py‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12394,6 +12394,122 @@ def batch_get_record(
1239412394
transformed_response = transform(response, "BatchGetRecordResponse")
1239512395
return BatchGetRecordResponse(**transformed_response)
1239612396

12397+
@Base.add_validate_call
12398+
def batch_write_record(
12399+
self,
12400+
entries: List[BatchWriteRecordEntry],
12401+
ttl_duration: Optional[TtlDuration] = Unassigned(),
12402+
session: Optional[Session] = None,
12403+
region: Optional[str] = None,
12404+
) -> Optional[BatchWriteRecordResponse]:
12405+
"""
12406+
Writes a batch of Records to one or more FeatureGroups.
12407+
12408+
Parameters:
12409+
entries: A list of records to write. Each entry specifies the FeatureGroup, the record data, and optionally target stores and a TTL duration.
12410+
ttl_duration: Time to live duration applied to all entries in the batch that do not specify their own TtlDuration; ExpiresAt = EventTime + TtlDuration. For information on HardDelete, see the DeleteRecord API in the Amazon SageMaker API Reference guide.
12411+
session: Boto3 session.
12412+
region: Region name.
12413+
12414+
Returns:
12415+
BatchWriteRecordResponse
12416+
12417+
Raises:
12418+
botocore.exceptions.ClientError: This exception is raised for AWS service related errors.
12419+
The error message and error code can be parsed from the exception as follows:
12420+
```
12421+
try:
12422+
# AWS service call here
12423+
except botocore.exceptions.ClientError as e:
12424+
error_message = e.response['Error']['Message']
12425+
error_code = e.response['Error']['Code']
12426+
```
12427+
AccessForbidden: You do not have permission to perform an action.
12428+
InternalFailure: An internal failure occurred. Try your request again. If the problem persists, contact Amazon Web Services customer support.
12429+
ResourceNotFound: Resource being access is not found.
12430+
ServiceUnavailable: The service is currently unavailable.
12431+
ValidationError: There was an error validating your request.
12432+
"""
12433+
12434+
operation_input_args = {
12435+
"Entries": entries,
12436+
"TtlDuration": ttl_duration,
12437+
}
12438+
# serialize the input request
12439+
operation_input_args = serialize(operation_input_args)
12440+
logger.debug(f"Serialized input request: {operation_input_args}")
12441+
12442+
client = Base.get_sagemaker_client(
12443+
session=session, region_name=region, service_name="sagemaker-featurestore-runtime"
12444+
)
12445+
12446+
logger.debug(f"Calling batch_write_record API")
12447+
response = client.batch_write_record(**operation_input_args)
12448+
logger.debug(f"Response: {response}")
12449+
12450+
transformed_response = transform(response, "BatchWriteRecordResponse")
12451+
return BatchWriteRecordResponse(**transformed_response)
12452+
12453+
@Base.add_validate_call
12454+
def list_records(
12455+
self,
12456+
max_results: Optional[int] = Unassigned(),
12457+
next_token: Optional[StrPipeVar] = Unassigned(),
12458+
include_soft_deleted_records: Optional[bool] = Unassigned(),
12459+
session: Optional[Session] = None,
12460+
region: Optional[str] = None,
12461+
) -> Optional[ListRecordsResponse]:
12462+
"""
12463+
Lists the RecordIdentifier values of all records stored in a FeatureGroup's OnlineStore.
12464+
12465+
Parameters:
12466+
max_results: The maximum number of record identifiers to return in a single page of results. For the InMemory tier, this value is a hint and not a strict requirement. The response may contain more or fewer results than the specified MaxResults.
12467+
next_token: A token to resume pagination of ListRecords results.
12468+
include_soft_deleted_records: If set to true, the result includes records that have been soft deleted.
12469+
session: Boto3 session.
12470+
region: Region name.
12471+
12472+
Returns:
12473+
ListRecordsResponse
12474+
12475+
Raises:
12476+
botocore.exceptions.ClientError: This exception is raised for AWS service related errors.
12477+
The error message and error code can be parsed from the exception as follows:
12478+
```
12479+
try:
12480+
# AWS service call here
12481+
except botocore.exceptions.ClientError as e:
12482+
error_message = e.response['Error']['Message']
12483+
error_code = e.response['Error']['Code']
12484+
```
12485+
AccessForbidden: You do not have permission to perform an action.
12486+
InternalFailure: An internal failure occurred. Try your request again. If the problem persists, contact Amazon Web Services customer support.
12487+
ResourceNotFound: Resource being access is not found.
12488+
ServiceUnavailable: The service is currently unavailable.
12489+
ValidationError: There was an error validating your request.
12490+
"""
12491+
12492+
operation_input_args = {
12493+
"FeatureGroupName": self.feature_group_name,
12494+
"MaxResults": max_results,
12495+
"NextToken": next_token,
12496+
"IncludeSoftDeletedRecords": include_soft_deleted_records,
12497+
}
12498+
# serialize the input request
12499+
operation_input_args = serialize(operation_input_args)
12500+
logger.debug(f"Serialized input request: {operation_input_args}")
12501+
12502+
client = Base.get_sagemaker_client(
12503+
session=session, region_name=region, service_name="sagemaker-featurestore-runtime"
12504+
)
12505+
12506+
logger.debug(f"Calling list_records API")
12507+
response = client.list_records(**operation_input_args)
12508+
logger.debug(f"Response: {response}")
12509+
12510+
transformed_response = transform(response, "ListRecordsResponse")
12511+
return ListRecordsResponse(**transformed_response)
12512+
1239712513

1239812514
class FeatureMetadata(Base):
1239912515
"""

0 commit comments

Comments
 (0)