Repository navigation
Universal Sentence Encoder Failing to Deploy on SageMaker Notebook instance] #1669
Description
Hello I am trying to use TensorFlow Serving for using Universal Sentence Encoder using
Sagemaker's NoteBook instance.
It throws the error -
" The primary container for production variant AllTraffic did not pass the ping health check. Please check CloudWatch logs for this endpoint..! "
""
To reproduce
A clear, step-by-step set of instructions to reproduce the bug.
from sagemaker import get_execution_role
import sagemaker
role = get_execution_role()
from sagemaker.tensorflow.serving import Model, Predictor
sagemaker_model = Model(
model_data = 's3://naister-platform-models/MUSE_TAR/universal-sentence-encoder.tar.gz',
role = role
)
predictor = sagemaker_model.deploy(
initial_instance_count=1,
instance_type='ml.t2.medium',
endpoint_name="MuseEndpointLg"
)
input = ['I am a good guy.', 'I am a bad guy, very bad.']
output = predictor.predict(input)
Expected behavior
Similarity score between the two input sentences.
Screenshots or logs
" The primary container for production variant AllTraffic did not pass the ping health check. Please check CloudWatch logs for this endpoint..! "
""
Activity
metrizable commentedon Jul 6, 2020
@singularity014 Thanks for filing the issue. I have a few questions:
ml.t2.medium. Is your tar.gz of reasonable size?singularity014 commentedon Jul 6, 2020
@metrizable thanks for addressing the question.
The tar.gz model is of 318 MB in size.
Yes, the enpoint creation process was taking very long :
predictor = sagemaker_model.deploy(
initial_instance_count=1,
instance_type='ml.t2.medium',
endpoint_name="MuseEndpointLg"
)
It took almost 15 minutes to run this , before it threw the error.
laurenyu commentedon Jul 23, 2020
sorry for the delayed response here. what are the contents of
s3://naister-platform-models/MUSE_TAR/universal-sentence-encoder.tar.gz?