feat(serve): size workloads from an endpoint's traffic

Add read_endpoint_traffic(), which reads the traffic one model on an
endpoint served over the last 1 to 7 days: the busiest hour's concurrency
for the whole deployment (ConcurrentRequestsPerCopy or
ConcurrentRequestsPerModel times the current copies or instances) and,
with detailed observability on, the request count and the mean and
standard deviation of the prompt and generated tokens of the requests a
vLLM container completed. The token histograms are read through
CloudWatch's Prometheus-compatible query API with the session's
credentials.

The histograms are cumulative, so each series' traffic in the window is
its newest snapshot in the window less its last one before it, read with
one instant query a day.

An adapter's requests run in its base component's container, which
publishes the concurrency and the token histograms for them, so an
adapter reads its base component, and finding the component skips
adapters.

Add Workload.from_endpoint(), which builds a synthetic workload from
them, so a benchmark or recommendation job can be sized from what an
endpoint serves today.

Author: ZealSV

sagemaker-serve/src/sagemaker/serve/ai_inference_recommender/__init__.py

@@ -39,6 +39,11 @@
     compare_benchmarks,
 )
 from sagemaker.serve.ai_inference_recommender.secrets import Secret
+from sagemaker.serve.ai_inference_recommender.traffic import (
+    EndpointTraffic,
+    TokenStats,
+    read_endpoint_traffic,
+)
 from sagemaker.serve.ai_inference_recommender.workload import Workload
 from sagemaker.serve.ai_inference_recommender._model_builder_methods import (
     start_benchmark,
@@ -51,15 +56,18 @@
     "BenchmarkMetrics",
     "BenchmarkResult",
     "BenchmarkSearchResult",
+    "EndpointTraffic",
     "FeatureGatedError",
     "InferenceFramework",
     "PerformanceTarget",
     "RecommendationJob",
     "Secret",
+    "TokenStats",
     "Workload",
     "WorkloadValidationError",
     "compare_benchmarks",
     "list_benchmarks",
     "list_recommendations",
+    "read_endpoint_traffic",
     "start_benchmark",
 ]