Skip to content

Commit 8d924cb

Browse files
romiikRomik Amipara
andauthored
feat(feature-store): add UpdateRecord API and Standard_V2 storage type (#6247)
Adds feature-level writes to SageMaker Feature Store: sagemaker-core (generated layer, regenerated from updated models): - Botocore models: UpdateRecord op (PATCH /FeatureGroup/{name}/Record), UpdateRecordRequest/Features shapes, ConflictException/ThrottlingException, and Standard_V2 value on the sagemaker StorageType enum. - FeatureGroup.update_record resource method + shape_dag serialization entries; registered in additional_operations.json. sagemaker-mlops (convenience layer): - OnlineStoreStorageTypeEnum.STANDARD_V2. - feature_utils.update_record(): partial-write helper delegating to core, with client-side validation (<=100 features, no duplicates, offline-only rejected, ttl requires EventTime) and dict/FeatureValue coercion. - Exported update_record from feature_store package. Tests & docs: - Core plumbing tests, mlops unit tests, integ tests, and a usage doc. Task: P495083396 Co-authored-by: Romik Amipara <romikdev@amazon.com>
1 parent 19fa86c commit 8d924cb

14 files changed

Lines changed: 661 additions & 5 deletions

File tree

‎docs/ml_ops/index.rst‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -667,6 +667,73 @@ To include soft-deleted records in the listing:
667667
region="us-west-2",
668668
)
669669
670+
**Feature-level writes with UpdateRecord (Standard_V2):**
671+
672+
``UpdateRecord`` performs a partial write to a record in a feature group whose online store uses
673+
the ``Standard_V2`` or ``InMemory`` storage type. Only the features you supply are written; features
674+
you do not list are preserved. This avoids the ``GetRecord`` -> merge -> ``PutRecord`` round trip and
675+
prevents lost writes when independent pipelines own different features on the same record. The record
676+
must already exist in the online store (use ``PutRecord`` to create it).
677+
678+
Create the feature group with ``Standard_V2`` storage (feature-level writes require ``Standard_V2``
679+
or ``InMemory``; they are not supported on the default ``Standard`` tier):
680+
681+
.. code-block:: python
682+
683+
from sagemaker.mlops.feature_store import FeatureGroupManager, OnlineStoreStorageTypeEnum
684+
from sagemaker.core.shapes import OnlineStoreConfig
685+
686+
feature_group = FeatureGroupManager.create(
687+
feature_group_name="customer-features",
688+
record_identifier_feature_name="customer_id",
689+
event_time_feature_name="event_time",
690+
feature_definitions=feature_definitions,
691+
online_store_config=OnlineStoreConfig(
692+
enable_online_store=True,
693+
storage_type=OnlineStoreStorageTypeEnum.STANDARD_V2.value,
694+
),
695+
role_arn=role,
696+
)
697+
698+
You can migrate an existing ``Standard`` feature group to ``Standard_V2`` with ``UpdateFeatureGroup``.
699+
This migration is one-way and cannot be reversed:
700+
701+
.. code-block:: python
702+
703+
from sagemaker.core.resources import FeatureGroup
704+
from sagemaker.core.shapes import OnlineStoreConfigUpdate
705+
706+
feature_group = FeatureGroup.get(feature_group_name="customer-features")
707+
feature_group.update(
708+
online_store_config=OnlineStoreConfigUpdate(storage_type="Standard_V2"),
709+
)
710+
711+
Use ``update_record`` to write only the features that changed. Pass ``EventTime`` as a feature
712+
(not a top-level parameter); features you do not include are preserved:
713+
714+
.. code-block:: python
715+
716+
from sagemaker.mlops.feature_store import update_record
717+
718+
update_record(
719+
feature_group_name="customer-features",
720+
record_identifier_value_as_string="cust-1",
721+
features=[
722+
{"feature_name": "purchase_count", "value_as_string": "11"},
723+
{"feature_name": "event_time", "value_as_string": "2026-01-02T00:00:00Z"},
724+
],
725+
region="us-west-2",
726+
)
727+
728+
Notes:
729+
730+
* Supply at most 100 features per call. If the supplied ``EventTime`` is not greater than the
731+
record's current ``EventTime``, the update is rejected with a ``ConflictException``.
732+
* ``ttl_duration`` requires the record's event-time feature to be present in ``features``.
733+
``target_stores`` defaults to all stores on the feature group; a value resolving to the
734+
``OfflineStore`` only is rejected.
735+
* ``UpdateRecord`` is not supported on ``Standard`` (V1) feature groups.
736+
670737

671738

672739
Migration from V2

‎sagemaker-core/sample/sagemaker-featurestore-runtime/2020-07-01/service-2.json‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,23 @@
111111
{"shape":"AccessForbidden"}
112112
],
113113
"documentation":"<p>The <code>PutRecord</code> API is used to ingest a list of <code>Records</code> into your feature group. </p> <p>If a new record’s <code>EventTime</code> is greater, the new record is written to both the <code>OnlineStore</code> and <code>OfflineStore</code>. Otherwise, the record is a historic record and it is written only to the <code>OfflineStore</code>. </p> <p>You can specify the ingestion to be applied to the <code>OnlineStore</code>, <code>OfflineStore</code>, or both by using the <code>TargetStores</code> request parameter. </p> <p>You can set the ingested record to expire at a given time to live (TTL) duration after the record’s event time, <code>ExpiresAt</code> = <code>EventTime</code> + <code>TtlDuration</code>, by specifying the <code>TtlDuration</code> parameter. A record level <code>TtlDuration</code> is set when specifying the <code>TtlDuration</code> parameter using the <code>PutRecord</code> API call. If the input <code>TtlDuration</code> is <code>null</code> or unspecified, <code>TtlDuration</code> is set to the default feature group level <code>TtlDuration</code>. A record level <code>TtlDuration</code> supersedes the group level <code>TtlDuration</code>.</p>"
114+
},
115+
"UpdateRecord":{
116+
"name":"UpdateRecord",
117+
"http":{
118+
"method":"POST",
119+
"requestUri":"/FeatureGroup/{FeatureGroupName}/Record"
120+
},
121+
"input":{"shape":"UpdateRecordRequest"},
122+
"errors":[
123+
{"shape":"ValidationError"},
124+
{"shape":"InternalFailure"},
125+
{"shape":"ServiceUnavailable"},
126+
{"shape":"AccessForbidden"},
127+
{"shape":"ResourceNotFound"},
128+
{"shape":"ConflictException"}
129+
],
130+
"documentation":"<p>Updates one or more feature values for an existing record in the specified feature group. Features that you do not include in the request remain unchanged. You can update up to 100 features per call. This operation requires the online store and is available only for feature groups that use the <code>Standard_V2</code> or <code>InMemory</code> online store type.</p> <p>The record must already exist in the online store; if it does not exist or has been soft-deleted, the operation returns a <code>ResourceNotFound</code> error. To create a record, use <code>PutRecord</code>.</p> <p>Pass <code>EventTime</code> as a feature in the <code>Features</code> list rather than as a top-level parameter. If you provide an <code>EventTime</code> that is older than the record's current <code>EventTime</code>, the update is rejected with a <code>ConflictException</code>; if it is equal to or newer, the update is applied; if you omit it, the record's existing <code>EventTime</code> is kept. If you specify <code>TtlDuration</code> you must also provide an <code>EventTime</code>, otherwise a <code>ValidationError</code> is returned. <code>TargetStores</code> must include the <code>OnlineStore</code>; a value that resolves to the <code>OfflineStore</code> only is rejected.</p>"
114131
}
115132
},
116133
"shapes":{
@@ -124,6 +141,15 @@
124141
"exception":true,
125142
"synthetic":true
126143
},
144+
"ConflictException":{
145+
"type":"structure",
146+
"members":{
147+
"Message":{"shape":"Message"}
148+
},
149+
"documentation":"<p>The request conflicts with the current state of the record. This is returned by <code>UpdateRecord</code> when the supplied <code>EventTime</code> is not greater than the record's current <code>EventTime</code>.</p>",
150+
"error":{"httpStatusCode":409},
151+
"exception":true
152+
},
127153
"BatchGetRecordError":{
128154
"type":"structure",
129155
"required":[
@@ -662,6 +688,38 @@
662688
"member":{"shape":"BatchGetRecordIdentifier"},
663689
"min":0
664690
},
691+
"UpdateRecordRequest":{
692+
"type":"structure",
693+
"required":[
694+
"FeatureGroupName",
695+
"RecordIdentifierValueAsString",
696+
"Features"
697+
],
698+
"members":{
699+
"FeatureGroupName":{
700+
"shape":"FeatureGroupNameOrArn",
701+
"documentation":"<p>The name or Amazon Resource Name (ARN) of the feature group that contains the record you want to update. The feature group must use an <code>OnlineStoreConfig</code> <code>StorageType</code> of <code>Standard_V2</code> or <code>InMemory</code>.</p>",
702+
"location":"uri",
703+
"locationName":"FeatureGroupName"
704+
},
705+
"RecordIdentifierValueAsString":{
706+
"shape":"ValueAsString",
707+
"documentation":"<p>The value for the <code>RecordIdentifier</code> that uniquely identifies the record to update, in string format.</p>"
708+
},
709+
"Features":{
710+
"shape":"Record",
711+
"documentation":"<p>The feature values to write to the record. Only the features included here are written; features that are not listed are preserved. Pass <code>EventTime</code> as a feature in this list. You can update up to 100 features per call.</p>"
712+
},
713+
"TargetStores":{
714+
"shape":"TargetStores",
715+
"documentation":"<p>A list of stores to which the update is applied. By default, Feature Store applies the update to all of the stores that you're using for the <code>FeatureGroup</code>. A value that resolves to the <code>OfflineStore</code> only is rejected.</p>"
716+
},
717+
"TtlDuration":{
718+
"shape":"TtlDuration",
719+
"documentation":"<p>Time to live duration, where the record is hard deleted after the expiration time is reached; <code>ExpiresAt</code> = <code>EventTime</code> + <code>TtlDuration</code>. Specifying <code>TtlDuration</code> requires <code>EventTime</code> to be present in <code>Features</code>.</p>"
720+
}
721+
}
722+
},
665723
"ValidationError":{
666724
"type":"structure",
667725
"members":{

‎sagemaker-core/sample/sagemaker/2017-07-24/service-2.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38675,6 +38675,10 @@
3867538675
"TtlDuration":{
3867638676
"shape":"TtlDuration",
3867738677
"documentation":"<p>Time to live duration, where the record is hard deleted after the expiration time is reached; <code>ExpiresAt</code> = <code>EventTime</code> + <code>TtlDuration</code>. For information on HardDelete, see the <a href=\"https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_feature_store_DeleteRecord.html\">DeleteRecord</a> API in the Amazon SageMaker API Reference guide.</p>"
38678+
},
38679+
"StorageType":{
38680+
"shape":"StorageType",
38681+
"documentation":"<p>The online store storage type to migrate the feature group to. Use this parameter to migrate an existing feature group from <code>Standard</code> to <code>Standard_V2</code> storage format, enabling support for the <a href=\"https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_feature_store_UpdateRecord.html\">UpdateRecord</a> operation. Migration is a one-way operation and cannot be reversed.</p>"
3867838682
}
3867938683
},
3868038684
"documentation":"<p>Updates the feature group online store configuration.</p>"
@@ -45440,6 +45444,7 @@
4544045444
"type":"string",
4544145445
"enum":[
4544245446
"Standard",
45447+
"Standard_V2",
4544345448
"InMemory"
4544445449
]
4544545450
},

‎sagemaker-core/src/sagemaker/core/resources.py‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12282,6 +12282,64 @@ def put_record(
1228212282
response = client.put_record(**operation_input_args)
1228312283
logger.debug(f"Response: {response}")
1228412284

12285+
@Base.add_validate_call
12286+
def update_record(
12287+
self,
12288+
record_identifier_value_as_string: StrPipeVar,
12289+
features: List[FeatureValue],
12290+
target_stores: Optional[List[StrPipeVar]] = Unassigned(),
12291+
ttl_duration: Optional[TtlDuration] = Unassigned(),
12292+
session: Optional[Session] = None,
12293+
region: Optional[str] = None,
12294+
) -> None:
12295+
"""
12296+
The UpdateRecord API performs a feature-level write to a Record in a feature group whose OnlineStoreConfig StorageType is Standard_V2 or InMemory. Only the supplied Features are written; features not included are preserved. The record must already exist in the online store.
12297+
12298+
Parameters:
12299+
record_identifier_value_as_string: The value for the RecordIdentifier that uniquely identifies the record to update, in string format.
12300+
features: The list of FeatureValues to update. Only the features included here are written; features that are not listed are preserved. Pass EventTime as a feature in this list. A maximum of 100 features can be updated in a single request.
12301+
target_stores: A list of stores to which the update is applied. By default, Feature Store applies the update to all of the stores that you're using for the FeatureGroup. A value that resolves to the OfflineStore only is rejected.
12302+
ttl_duration: Time to live duration, where the record is hard deleted after the expiration time is reached; ExpiresAt = EventTime + TtlDuration. Specifying TtlDuration requires EventTime to be present in Features.
12303+
session: Boto3 session.
12304+
region: Region name.
12305+
12306+
Raises:
12307+
botocore.exceptions.ClientError: This exception is raised for AWS service related errors.
12308+
The error message and error code can be parsed from the exception as follows:
12309+
```
12310+
try:
12311+
# AWS service call here
12312+
except botocore.exceptions.ClientError as e:
12313+
error_message = e.response['Error']['Message']
12314+
error_code = e.response['Error']['Code']
12315+
```
12316+
AccessForbidden: You do not have permission to perform an action.
12317+
ConflictException: There was a conflict when you attempted to modify a record; the supplied EventTime was not greater than the record's current EventTime.
12318+
InternalFailure: An internal failure occurred. Try your request again. If the problem persists, contact Amazon Web Services customer support.
12319+
ResourceNotFound: A resource that is required to perform an action was not found.
12320+
ServiceUnavailable: The service is currently unavailable.
12321+
ValidationError: There was an error validating your request.
12322+
"""
12323+
12324+
operation_input_args = {
12325+
"FeatureGroupName": self.feature_group_name,
12326+
"RecordIdentifierValueAsString": record_identifier_value_as_string,
12327+
"Features": features,
12328+
"TargetStores": target_stores,
12329+
"TtlDuration": ttl_duration,
12330+
}
12331+
# serialize the input request
12332+
operation_input_args = serialize(operation_input_args)
12333+
logger.debug(f"Serialized input request: {operation_input_args}")
12334+
12335+
client = Base.get_sagemaker_client(
12336+
session=session, region_name=region, service_name="sagemaker-featurestore-runtime"
12337+
)
12338+
12339+
logger.debug(f"Calling update_record API")
12340+
response = client.update_record(**operation_input_args)
12341+
logger.debug(f"Response: {response}")
12342+
1228512343
@Base.add_validate_call
1228612344
def delete_record(
1228712345
self,

‎sagemaker-core/src/sagemaker/core/shapes/shapes.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14670,9 +14670,11 @@ class OnlineStoreConfigUpdate(Base):
1467014670
Attributes
1467114671
----------------------
1467214672
ttl_duration: Time to live duration, where the record is hard deleted after the expiration time is reached; ExpiresAt = EventTime + TtlDuration. For information on HardDelete, see the DeleteRecord API in the Amazon SageMaker API Reference guide.
14673+
storage_type: The online store storage type to migrate the feature group to. Use this parameter to migrate an existing feature group from Standard to Standard_V2 storage format, enabling support for the UpdateRecord operation. Migration is a one-way operation and cannot be reversed.
1467314674
"""
1467414675

1467514676
ttl_duration: Optional[TtlDuration] = Unassigned()
14677+
storage_type: Optional[StrPipeVar] = Unassigned()
1467614678

1467714679

1467814680
class Parent(Base):

‎sagemaker-core/src/sagemaker/core/tools/additional_operations.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -459,6 +459,14 @@
459459
"method_type": "object",
460460
"service_name": "sagemaker-featurestore-runtime"
461461
},
462+
"UpdateRecord": {
463+
"operation_name": "UpdateRecord",
464+
"resource_name": "FeatureGroup",
465+
"method_name": "update_record",
466+
"return_type": "None",
467+
"method_type": "object",
468+
"service_name": "sagemaker-featurestore-runtime"
469+
},
462470
"DeleteRecord": {
463471
"operation_name": "DeleteRecord",
464472
"resource_name": "FeatureGroup",

‎sagemaker-core/src/sagemaker/core/utils/code_injection/shape_dag.py‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14113,7 +14113,10 @@
1411314113
"type": "structure",
1411414114
},
1411514115
"OnlineStoreConfigUpdate": {
14116-
"members": [{"name": "TtlDuration", "shape": "TtlDuration", "type": "structure"}],
14116+
"members": [
14117+
{"name": "TtlDuration", "shape": "TtlDuration", "type": "structure"},
14118+
{"name": "StorageType", "shape": "StorageType", "type": "string"},
14119+
],
1411714120
"type": "structure",
1411814121
},
1411914122
"OnlineStoreSecurityConfig": {
@@ -18731,6 +18734,16 @@
1873118734
],
1873218735
"type": "structure",
1873318736
},
18737+
"UpdateRecordRequest": {
18738+
"members": [
18739+
{"name": "FeatureGroupName", "shape": "FeatureGroupNameOrArn", "type": "string"},
18740+
{"name": "RecordIdentifierValueAsString", "shape": "ValueAsString", "type": "string"},
18741+
{"name": "Features", "shape": "Record", "type": "list"},
18742+
{"name": "TargetStores", "shape": "TargetStores", "type": "list"},
18743+
{"name": "TtlDuration", "shape": "TtlDuration", "type": "structure"},
18744+
],
18745+
"type": "structure",
18746+
},
1873418747
"UpdateTrialComponentResponse": {
1873518748
"members": [{"name": "TrialComponentArn", "shape": "TrialComponentArn", "type": "string"}],
1873618749
"type": "structure",

0 commit comments

Comments
 (0)