Skip to content

docsy(v2): document tracked local runs (--tracked / --tracked-strict) - #1462

Merged
ppiegaze merged 3 commits into
mainfrom
docsy/v2-tracked-runs
Sep 3, 2026
Merged

docsy(v2): document tracked local runs (--tracked / --tracked-strict)#1462
ppiegaze merged 3 commits into
mainfrom
docsy/v2-tracked-runs

Conversation

@ppiegaze

@ppiegaze ppiegaze commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

The gap

Regen #1424 published --tracked, --tracked-strict and flyte create config --local-tracked in the generated CLI reference, but no narrative page mentioned any of them. Two pages were actively misleading as a result:

  • run-command-options.md opens with "The flyte run command provides the following options:" and an exhaustive table that lists --local but not --tracked.
  • running-locally.md and the run-modes comparison table describe local runs as leaving no trace on the control plane, which is exactly what --tracked changes.

Surfaced by the review of #1459, which claimed the four API regens left two narrative gaps. This was a third.

Publish gate: cleared

This PR was drafted on 2026-08-19 under a "do not merge until released" hold. That hold was already stale when it was written, and is now released.

Gate Evidence
Released The reporter landed as flyte-sdk@f1f80a69 on 2026-08-11. The first tag containing it is v2.5.20, published to PyPI 2026-08-12 — a week before this PR was drafted. The docs site is pinned at v2.6.13.0.
Reachable trackedRuns.enabled: true for staging, canary and production in both services: cloud:executions/deploy/values-{staging,canary,production}.yaml (registers TrackedRunService) and cloud:dataproxy/deploy/dataproxy.yaml trackedRunsEnabled (serves tracked-run metadata from the per-env control-plane bucket). No per-org gate.

Verified against fetched origin/main at unionai/cloud@d3f7660474.

What I verified, and where

All against fetched origin/main, not a local checkout and not memory. Re-verified 2026-09-03 against flyte-sdk@1a0d78f7, eleven releases after the original pass; every row still holds.

Claim Source
--tracked implies --local; needs endpoint, project, domain src/flyte/cli/_run.py tracked field + _run.py:524-526
--tracked-strict only valid with --tracked; raises otherwise _run.py:528-529
Reports actions, attempts, phases, inputs, outputs, reports, cache status src/flyte/_persistence/_remote_reporter.py record_*, _upload_terminal_artifacts
Logs are not reported no log-link path anywhere in _remote_reporter.py
Best-effort: never fails or blocks the local run _remote_reporter.py module docstring; bounded retries + bounded terminal flush
Auto-name local-<8 hex>; user names <=30 chars, no leading u/r generate_tracked_run_name, validate_tracked_run_name (_remote_reporter.py:96-122)
local.tracked / local.tracked_strict config keys src/flyte/config/_internal.py; cli/_create.py:477,575
Console link printed at start src/flyte/_run.py; remote/_client/controlplane.py
Tracked runs appear under their own nav section, not in Runs cloud:clientsv2/src/components/NavPanel/NavItemConfigs.tsxTrackedRunsLink (display text "Tracked Runs", href /domain/<d>/project/<p>/tracked-runs) is defined separately from RunsLink; the Runs list has no tracked-run handling
flyte run hello runs a built-in example with no files flyte-sdk#1496, merged 2026-09-01; src/flyte/cli/_hello.py; present in the pinned CLI reference at flyte run hello

Two claims were settled by running the CLI rather than reading it, on flyte 2.6.13 in an empty directory:

$ flyte run --local hello
  Using the built-in example from .../flyte-hello-<user>/task/hello.py
  Completed Local Run   Outputs: ActionOutputs(o0=14.0)

$ flyte run --tracked hello
  Completed Local Run
  Path: https://<tenant>/v2/domain/development/project/<project>/tracked-runs/local-ffa90918
  Outputs: ActionOutputs(o0=14.0)

That confirms both that hello composes with --tracked, and the /tracked-runs/ URL shape.

Design calls made, and why

Tracking is documented as a modifier on local execution, not a fourth run mode. --tracked implies --local and the task still runs in the local Python process, so a peer page next to Local/Devbox/Remote would misrepresent it. It lands as a section in running-locally.md plus an option section in run-command-options.md.

This is the taxonomy problem DOC-1343 predicted: a run that executes on your laptop but registers with the control plane is locally executed and remotely visible, a combination the current Local/Devbox/Remote vocabulary cannot express. DOC-1326 should ratify this framing before it is encoded in style-spec.md.

Gated to the union variant. flyteorg/flyte carries only the flyteidl2 protos and generated stubs for TrackedRunService and has no server implementation; it is implemented in unionai/cloud. Verified no leakage on the rebased build: the flyte variant contains zero occurrences of "tracked" across all three pages, the union variant has them.

No rows added to the shared options table. #1439 established the precedent of documenting Union-only run options in a gated section rather than duplicating the table, and adding union-only rows to a shared table would be a variant-safety bug.

Open question for eng

Self-hosted enablement is undocumented and may not work. (Corrected 2026-09-03: this was previously filed against Self-managed, which was the wrong audience. Self-managed runs the customer's data plane against Union's control plane, where tracked runs is already enabled — see the publish-gate table above. The gap is self-hosted, where the customer operates the control plane too.)

The dataproxy gate trackedRuns.enabled is "Dark by default" (cloud:dataproxy/config/config.go:48-53). The self-hosted control-plane values (cloud:infra/terraform/envs/staging/adam-selfhosted/control-plane/values.yaml) set no trackedRuns key, the chart default is enabled: false (dataproxy/deploy/values.yaml), and the tenant chart exposes only trackedRunsRateLimit, no enablement value. So a self-hosted operator appears to have no supported way to turn this on.

If that is right, the self-hosted deployment docs (#932) need the Helm value and a prerequisite note. Not blocking this PR — the pages here are +union-gated and self-hosted has no published page to carry the caveat yet.

Checks

Full make dist green on the rebased branch (both variants, 1m17s), asset references clean (43,758 checked, 0 missing), variant gating re-verified in the built output. Rendered HTML inspected: the new "Tracked Runs" paragraph and the hello block render as top-level siblings inside the gated section.

Also found

DOC-1439 "Missing documentation for tracked runs feature" (Triage, unlabelled) duplicates DOC-1343. Recommend closing it as Duplicate. Your call, not mine.


— docsy · automated docs agent · DOC-1343

@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown

GHA build & deploy preview

Built by .github/workflows/build-pr.yml and deployed to the docs CF Pages project by .github/workflows/deploy-pr-preview.yml.

Branch alias https://pr-1462-docsy-v2-tracked-run.docs-dog.pages.dev
This commit https://f45574bc.docs-dog.pages.dev
Commit SHA 427b914e64f5ad0d1a2931e557bf7496781f02c5

Updated automatically on every push.

ppiegaze and others added 3 commits September 3, 2026 09:51
Regen #1424 published --tracked, --tracked-strict and
flyte create config --local-tracked in the generated CLI reference, but no
narrative page mentions them. A reader searching the user guide for "tracked"
finds nothing, and running-locally describes local runs as leaving no trace on
the control plane, which --tracked changes.

Document tracking as a modifier on local execution rather than a fourth run
mode, since --tracked implies --local and the task still executes in the local
Python process:

- run-command-options: a --tracked / --tracked-strict section covering what is
  reported (actions, attempts, phases, inputs, outputs, reports, cache status),
  what is not (logs stay in the terminal), the best-effort reporting policy and
  why strict mode exists, the run-name contract (<=30 chars, no leading u or r,
  auto-named local-<id>), and the local.tracked config key.
- running-locally: a "Track local runs in the console" section, plus a note on
  the "What works locally" table, which otherwise implies local runs are
  invisible to the control plane.
- run-modes/_index: the union comparison table listed "No (TUI only)" under
  Flyte UI for local execution, which is no longer accurate.

Gated to the union variant: flyteorg/flyte carries only the flyteidl2 protos
and generated stubs for TrackedRunService, with no server implementation. It is
implemented in the Union control plane.

NOT FOR MERGE until tracked runs is released. Drafted under the publish gate.

Co-Authored-By: docsy <docsy@union.ai>
Signed-off-by: Peeter Piegaze <1153481+ppiegaze@users.noreply.github.com>
The two rules validate_tracked_run_name enforces (<=30 chars, no leading u or
r) are specific to tracked runs. Phrasing them as the naming contract read as
exhaustive, which would mislead a reader who then hits the separate --name
charset behavior reported in DOC-1425 (unresolved: no eng verdict, and the
constraint is not in the SDK or in Union CreateRun, which rejects only a
leading 'u').

Co-Authored-By: docsy <docsy@union.ai>
Signed-off-by: Peeter Piegaze <1153481+ppiegaze@users.noreply.github.com>
Two corrections to the tracked-runs pages, both verified against
unionai/cloud@origin/main and by running the CLI.

Where tracked runs appear. Both pages said a tracked run shows up
"alongside" or "next to" your remote runs, which reads as the Runs list.
It is not: clientsv2/src/components/NavPanel/NavItemConfigs.tsx defines
TrackedRunsLink as its own sidebar item pointing at
/domain/<domain>/project/<project>/tracked-runs, separate from RunsLink,
and the Runs list has no tracked-run handling at all. Running
`flyte run --tracked hello` printed a link ending
/project/<project>/tracked-runs/local-ffa90918. A reader following the
old wording looks in the wrong list, so both pages now name the section.

The zero-file path. `flyte run hello` landed in flyte-sdk #1496 on
2026-09-01 and is already in the pinned CLI reference. It writes a
built-in example to a scratch directory and runs it, so
`flyte run --tracked hello` takes someone from a fresh config to a run
in the console without writing a file first. Verified by execution on
flyte 2.6.13, not by reading: local and tracked both complete with
ActionOutputs(o0=14.0).

Co-Authored-By: docsy <docsy@union.ai>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017CJ89R9ri86y11FFhPdVJV
Signed-off-by: Peeter Piegaze <1153481+ppiegaze@users.noreply.github.com>
@ppiegaze
ppiegaze force-pushed the docsy/v2-tracked-runs branch from 55681a1 to 427b914 Compare September 3, 2026 07:56
@ppiegaze
ppiegaze marked this pull request as ready for review September 3, 2026 07:57
@ppiegaze
ppiegaze requested a review from EngHabu as a code owner September 3, 2026 07:57
Copilot AI lite review requested due to automatic review settings September 3, 2026 07:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A couple of small documentation clarity issues (missing prerequisite context for --tracked and ambiguous “console” wording in the run-modes table) should be addressed to avoid reader confusion.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR closes a narrative documentation gap for Union-only “tracked local runs” by adding flyte run --tracked/--tracked-strict guidance and updating the local run-mode docs so they no longer imply that local runs are always invisible to the control plane.

Changes:

  • Add Union-variant documentation for --tracked / --tracked-strict, including naming rules and flyte create config --local-tracked configuration keys.
  • Update the “Run locally in Python” page to describe tracked runs, where they appear in the console UI, and how they behave when reporting fails.
  • Update the Union run-modes comparison table to reflect that local runs can be visible in the console when --tracked is used.
File summaries
File Description
content/user-guide/tasks/task-deployment/run-command-options.md Adds Union-only sections and option-table entries for tracked local runs and related config keys.
content/user-guide/get-started/run-modes/running-locally.md Updates the local execution guide to explain tracked runs, visibility in the console, and related commands/links.
content/user-guide/get-started/run-modes/_index.md Updates the Union run-modes comparison table to reflect console visibility for local runs via --tracked.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines 49 to 54
| Aspect | Local (`--local`) | Devbox | Remote |
|--------|-------------------|--------|--------|
| **⚡️ Execution** | In-process Python | Containerized, local Docker | Containerized, cloud |
| **🐳 Docker required** | No | Yes | No (remote build) |
| **💻 Flyte UI** | No (TUI only) | Yes (`localhost:30080`) | Yes |
| **💻 Flyte UI** | TUI, or the console with `--tracked` | Yes (`localhost:30080`) | Yes |
| **📦 Container images** | Ignored | Built locally | Built locally or remotely |
Comment on lines +130 to +134
A local run normally leaves no trace on the control plane. Add `--tracked` and the run still executes on your machine, but its progress is reported to {{< key product_name >}} so you can watch it in the console:

```bash
flyte run --tracked hello.py main
```
@ppiegaze
ppiegaze merged commit a436f1d into main Sep 3, 2026
17 checks passed
@ppiegaze
ppiegaze deleted the docsy/v2-tracked-runs branch September 3, 2026 08:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants