Skip to content

Error in Invoking Universal Sentence Encoder Sagemaker Endpoint  #1712

Closed

Description

@singularity014

Describe the bug
Hello,
I deployed a multilingual universal sentence encoder model on using SageMaker notebook instance.
The model got deployed successfully. But when I am trying to predict it from the notebook instance.
I am getting the following error -

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (404) from model with message "{ "error": "[Derived]{{function_node __inference_signature_wrapper_227768}} {{function_node __inference_signature_wrapper_227768}} Op type not registered 'SentencepieceOp' in binary running on model.aws.local. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) tf.contrib.resampler should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.\n\t [[{{node StatefulPartitionedCall}}]]\n\t [[StatefulPartitionedCall]]" }". See https://eu-west-1.console.aws.amazon.com/cloudwatch/home?region=eu-west-1#logEventViewer:group=/aws/sagemaker/Endpoints/museA in account 647725471721 for more information.

image

To reproduce
from sagemaker import get_execution_role
import sagemaker
role = get_execution_role()
from sagemaker.tensorflow.serving import Model, Predictor

sagemaker_model = Model(
model_data = 's3://naister-platform-models/universal-sentence-encoder-multilingual-large_3.tar.gz',
role = role,
framework_version='2.0.0'
)

predictor = sagemaker_model.deploy(
initial_instance_count=1,
instance_type='ml.t2.medium',
endpoint_name="museA"
)

input = ['Winter is coming', 'Hold the door, Hodor']
output = predictor.predict(input)

Expected behavior
Sentence Embeddings

System information
A description of your system. Please provide:

  • Using Sagemaker Notebook type conda_amazonei_tensorflow2_p36:
  • Framework name ( Tensorflow), model (Multilingual Universal sentence encoder large):
  • Framework version: 2.2.0
  • Python version: 3.6

Assistance requested regarding this problem.

Activity

  1. chuyang-deng commented on Jul 15, 2020

    @chuyang-deng
    Contributor

    Hi @singularity014,

    The ModelError you had seems to be a TensorFlow specific error, which I've found a potential solution to it: tensorflow/hub#463

    Are you using an inference.py script for predictions? If so you could try adding tensorflow_text and import it in the inference script.

    Here's instruction on how to bring in custom modules (towards the end of the Pre/Post-Processing section):
    https://github.com/aws/sagemaker-tensorflow-serving-container#prepost-processing

  2. singularity014 commented on Aug 4, 2020

    @singularity014
    Author

    Hi..I think it is because of the sentence piece and tensorflow-text missing dependency from the Tensorflow Hub container used by SageMaker.

  3. locked and limited conversation to collaborators on May 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions