Skip to content

Accessing inference endpoint environment / config from within the inference code is undocumented #1338

Closed

Description

@ram-nadella

What did you find confusing? Please describe.
I was looking for a way to get information about the inference endpoint from the inference endpoint code at runtime (eg. within a Flask request handler). Similar to how training and processing have access to JSON config in a file under the /opt/ml/ structure or an env var.

Describe how documentation can be improved
Need docs for the right /opt/ml/ config file that would provide info about the current inference endpoint (the name would be a good starting point as the rest of the info about model etc. can be pulled from this using the SDK)

Additional context
Example of how this info is accessible for training:

SM_TRAINING_ENV has a lot of info about the job itself.
https://docs.aws.amazon.com/sagemaker/latest/dg/docker-container-environmental-variables.html

Is there a similar file/env var for inference?

FYI, we are using a custom container for inference.

Activity

  1. laurenyu commented on Mar 18, 2020

    @laurenyu
    Contributor

    @ram-nadella sorry for the delayed response here

    is your custom image based on one of the prebuilt SageMaker images or completely built from scratch?

  2. ram-nadella commented on Mar 18, 2020

    @ram-nadella
    Author

    Hi @laurenyu it's completely from scratch.

    (we're operating in a python environment, using a base of continuumio/miniconda3:4.6.14)

  3. laurenyu commented on Mar 19, 2020

    @laurenyu
    Contributor

    Thanks for clarifying! I've passed this along to the SageMaker Hosting team to see if they can help.

  4. locked and limited conversation to collaborators on May 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions