feat(serve): describe an endpoint's traffic and size workloads from it

Add describe_endpoint_traffic(), which describes the traffic one model on
an endpoint served over a window, relative (lookback back from end_time,
seven days by default) or absolute (start_time to end_time), up to seven
days. Times must be timezone-aware and are reported in UTC.

- Concurrency for the whole deployment: each hour's maximum
  ConcurrentRequestsPerCopy or ConcurrentRequestsPerModel times the
  current copies or instances.
- With detailed observability on, the request count and the prompt and
  generated tokens of the requests the model completed, from the serving
  container's token histograms, read through CloudWatch's
  Prometheus-compatible query API with the session's credentials. The
  histograms are listed per serving framework (vLLM today), and the
  result names the framework read. They aren't read when the endpoint
  configuration turns detailed observability off.
- Tokens and concurrency are TrafficStats: min, p25, p50, p70, p80, p90,
  p95, p99, max (p100), mean and standard deviation. Token percentiles
  interpolate within the histograms' buckets, and the token means are
  exact. Concurrency's are over the window's hourly maximums, so its max
  is the busiest hour's.
- The variant's Data Capture prefix, when the endpoint captures requests.

The histograms are cumulative, so each series' traffic in the window is
its newest snapshot in the window less its snapshot at the start, read
with one instant query a day.

An adapter's requests run in its base component's container, which
publishes the concurrency and the token histograms for them, so an
adapter is described by its base component, and finding the component
skips adapters.

Add Workload.from_endpoint(), which replays the captured requests when
the endpoint has Data Capture on (use_data_capture=False opts out) and
otherwise builds a synthetic workload from the token metrics, at the
busiest hour's concurrency, so a benchmark or recommendation job can be
sized from what an endpoint serves today. Other workload parameters, such
as tokenizer and hf_token, pass through to the workload.

Author: ZealSV

sagemaker-serve/src/sagemaker/serve/ai_inference_recommender/__init__.py

@@ -39,6 +39,11 @@
     compare_benchmarks,
 )
 from sagemaker.serve.ai_inference_recommender.secrets import Secret
+from sagemaker.serve.ai_inference_recommender.traffic import (
+    EndpointTraffic,
+    TrafficStats,
+    describe_endpoint_traffic,
+)
 from sagemaker.serve.ai_inference_recommender.workload import Workload
 from sagemaker.serve.ai_inference_recommender._model_builder_methods import (
     start_benchmark,
@@ -51,14 +56,17 @@
     "BenchmarkMetrics",
     "BenchmarkResult",
     "BenchmarkSearchResult",
+    "EndpointTraffic",
     "FeatureGatedError",
     "InferenceFramework",
     "PerformanceTarget",
     "RecommendationJob",
     "Secret",
+    "TrafficStats",
     "Workload",
     "WorkloadValidationError",
     "compare_benchmarks",
+    "describe_endpoint_traffic",
     "list_benchmarks",
     "list_recommendations",
     "start_benchmark",