Multi Model Endpoint Deployment Issue #1250

Activity

1. nmakarun

   added

   type: bug

   on Jan 20, 2020
2. preshen-goobiah commented on Feb 19, 2020

   

   preshen-goobiah

   on Feb 19, 2020

   Last edited by laurenyu

   Contributor

   More actions

   Receiving the same error when trying to deploy multiple models locally.

   I used this example

   from sagemaker import multidatamodel
   
   mdm = multidatamodel.MultiDataModel(name="MultipleModelsLocally",
                                       model_data_prefix="s3://sagemaker-eu-west-1-test/DEMO_MME_SCIKIT/multi_model_artifacts/", 
                                       image="<account_id>.dkr.ecr.eu-west-1.amazonaws.com/multi-model-sklearn:latest",
                                       role="arn:aws:iam::<account_id>:role/service-role/AmazonSageMaker-ExecutionRole-20190401T141434"
                                      )
   
   predictor = mdm.deploy(initial_instance_count=1,instance_type='local', endpoint_name="MDMModel")

   

   UPDATE: I was able to deploy locally by including the require env variables for the model

   from sagemaker import multidatamodel
   
   mdm = multidatamodel.MultiDataModel(name="MultipleModelsLocally2",
                                       model_data_prefix="s3://sagemaker-eu-west-1-test/DEMO_MME_SCIKIT/multi_model_artifacts/", 
                                       image="<account_id>.dkr.ecr.eu-west-1.amazonaws.com/multi-model-sklearn:latest",
                                       role="arn:aws:iam::<account_id>:role/service-role/AmazonSageMaker-ExecutionRole-20190401T141434",
    env={
               'SAGEMAKER_PROGRAM' : "script.py",
               'SAGEMAKER_SUBMIT_DIRECTORY' : "s3://sagemaker-eu-west-1-test/DEMO_MME_SCIKIT/inference-code/user_code.tar.gz"
           }
                                      )
   
   predictor = mdm.deploy(initial_instance_count=1,instance_type='local', endpoint_name="MDMModel2")
3. laurenyu

   added

   Pending information

   type: question

   and removed

   type: bug

   on Feb 24, 2020
4. laurenyu

   added

   NON-PY-SDK

   on May 8, 2020
5. dlragha commented on Dec 12, 2020

   

   dlragha

   on Dec 12, 2020

   Last edited by dlragha

   Contributor

   More actions

   Thanks for reaching out! You are correct, if building an BYO image with SageMaker Scikit-learn image (SKLearn) as the base, the inference container expects SAGEMAKER_PROGRAM and SAGEMAKER_SUBMIT_DIRECTORY environment variables to be set explicitly while creating the model object. Creating an SKLearn estimator and deploying the model using the SageMaker Python SDK as seen in the example below sets these env variables automatically without having to pass them in explicitly.

   AWS Managed Scikit-learn now provides native support for launching multi-model endpoints. Please see the updated example for usage.
6. bveeramani

   closed this as completedon May 20, 2021
7. aws

   locked and limited conversation to collaborators on May 20, 2021