Skip to content

Add flyte proxy app: authenticated localhost proxy into a Flyte App - #1435

Open
mhotan wants to merge 2 commits into
mainfrom
mike/flyte-proxy-app
Open

Add flyte proxy app: authenticated localhost proxy into a Flyte App#1435
mhotan wants to merge 2 commits into
mainfrom
mike/flyte-proxy-app

Conversation

@mhotan

@mhotan mhotan commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Overview

Adds a flyte proxy app NAME command — an authenticated localhost reverse proxy into a no-auth-inside Flyte App. It reuses the CLI's own authenticator to inject a fresh bearer on every request, so a local HTTP client (an MCP server, curl, a browser) can reach an app-edge-gated app with no token handling. Think kubectl port-forward for Flyte Apps.

Details

  • Resolves the target by app name (App.get) or an explicit --url.
  • Binds loopback on both 127.0.0.1 and ::1 so localhost — which resolves to ::1 first on macOS — can't be silently shadowed by a wildcard listener from another process (e.g. a container runtime binding *:PORT with SO_REUSEPORT). Previously an IPv4-only bind still "succeeded" while every request landed on the other listener and reset; now a genuine conflict fails loudly with EADDRINUSE. A non-loopback --address binds a single family and warns (the proxy is ambient authority for the caller's identity).
  • Streams responses unbuffered and preserves Content-Encoding — SSE-safe for MCP streamable-HTTP.
  • Drops inbound Authorization/hop-by-hop headers, injects the fresh bearer on the standard authorization header (the SDK auth flow may emit it under a gRPC-style metadata key the HTTP app edge never reads), and refreshes + retries once when the edge bounces an expired token to /login.
  • --emit-mcp-config prints a generic HTTP-MCP client block pointed at the local port.
  • WebSocket upgrade proxying is deferred to a follow-up.

Also included

  • App-serde: materialize delayed parameters on the pod_template path. When an app parameter is a delayed value (e.g. RunOutput), it is materialized to a resolved File/Dir URI at deploy time. The container path already did this; the pod_template serialization path did not — so an app that set a pod_template serialized the raw, unresolved getter and the serve entrypoint couldn't download it. Threads parameter_overrides through _get_k8s_pod / _serialized_pod_spec so the pod command carries the resolved value. Independent of the proxy — can be split into its own PR on request.

Testing

  • Unit tests (tests/cli/test_proxy.py): inbound-auth/hop-by-hop stripping, Content-Encoding preservation, JWT identity display, and the MCP-config block.
  • Validated end-to-end against a running app with an upstream mcp-grafana server driven by the official MCP Python client SDK (ClientSession): initialize -> 65 tools -> list_datasources and query_prometheus returned live data through the proxy with no real credential in the MCP config.
  • Dual-stack bind verified: with a wildcard listener holding the port, curl localhost:PORT reaches the proxy (not the other listener); a conflicting specific bind fails with EADDRINUSE.

🤖 Generated with Claude Code

@mhotan
mhotan force-pushed the mike/flyte-proxy-app branch from af3a5ee to 7aa0069 Compare August 14, 2026 04:28
@mhotan
mhotan force-pushed the mike/flyte-proxy-app branch 4 times, most recently from 4be3f27 to c425c88 Compare September 1, 2026 10:13
# resolved value — matching the container path (get_proto_container). Without
# this the pod path serializes the raw, unresolved parameter and the serve
# entrypoint can't download it.
container.command = app_env.container_cmd(serialization_context, parameter_overrides)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually a bug fix I think I found.

@mhotan
mhotan force-pushed the mike/flyte-proxy-app branch from faa78b4 to 4e53058 Compare September 1, 2026 11:18
Comment thread src/flyte/cli/_proxy.py Outdated

import rich_click as click

import flyte.remote as remote

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we import lazily?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 39fd218 — moved import flyte.remote as remote into the app command's name-resolution branch, so the heavy remote client stack only loads when resolving an app by name (the --url path skips it entirely). Module top now imports only stdlib + rich_click (needed for the decorators).

Adds a `flyte proxy app NAME` command — an authenticated localhost reverse
proxy into a no-auth-inside Flyte App. Reuses the CLI authenticator to inject a
fresh bearer on every request so a local HTTP client (an MCP server, curl, a
browser) reaches an app-edge-gated app with no token handling.

- Resolve the target by app name (App.get) or an explicit --url.
- Bind loopback on both 127.0.0.1 and ::1 so `localhost` (which resolves to ::1
  first on macOS) is not silently shadowed by a wildcard listener from another
  process (e.g. a container runtime on *:PORT with SO_REUSEPORT); a genuine
  conflict now fails loudly with EADDRINUSE instead of half-binding.
- Stream responses unbuffered and preserve Content-Encoding (SSE-safe for MCP
  streamable-HTTP).
- Drop inbound Authorization/hop-by-hop headers; inject the fresh bearer on the
  standard `authorization` header (the SDK auth flow may emit it under a
  gRPC-style metadata key the HTTP app edge never reads); refresh + retry once
  when the edge bounces an expired token to /login.
- --emit-mcp-config prints a generic HTTP-MCP client block.

Also materialize delayed app parameters (e.g. RunOutput) on the pod_template
serialization path: the container path already resolved them to a File/Dir URI,
but the pod_template path serialized the raw getter, so an app that set a
pod_template could not download its input. Thread parameter_overrides through
_get_k8s_pod / _serialized_pod_spec.

Unit tests cover header filtering, Content-Encoding preservation, JWT identity
display, and the MCP-config block.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Michael Hotan <mike@union.ai>
@mhotan
mhotan force-pushed the mike/flyte-proxy-app branch from efedc30 to fe7405c Compare September 3, 2026 05:18
Addresses review feedback: the flyte.remote client stack is heavy and was imported
at module top, slowing every 'flyte' CLI invocation. Move it into the app command's
name-resolution branch so it only loads when resolving an app by name (the --url
path skips it entirely).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@mhotan
mhotan force-pushed the mike/flyte-proxy-app branch from 39fd218 to f1b26ca Compare September 4, 2026 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants