Add Jira plugin: tasks, webhook app, MCP server - #1507
Closed
cosmicBboy wants to merge 7 commits into
Closed
Conversation
Adds plugins/jira (flyteplugins-jira), a Jira Cloud integration for Flyte: - JiraClient: async REST v3 client covering projects, issues, JQL search, comments, and workflow transitions, with automatic ADF conversion for descriptions/comments and 429 retries. Basic auth over account email + API token. - JiraAppEnvironment: setup/management dashboard plus a webhook receiver protected by a shared X-Webhook-Token header (Jira webhooks are not signed), normalizing events and dispatching to handlers with a project allowlist. - launch_task: idempotent event-driven run launching with dedupe labels. - MCP server builder exposing the read/write surface to agents on Flyte (read-only by default, delete_issue opt-in destructive). Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
- The webhook token compares as bytes: a non-ASCII X-Webhook-Token made compare_digest raise TypeError, turning a 401 into a 500. ASGI servers hand Starlette raw header bytes, which it decodes as latin-1, so a non-ASCII token really does reach the comparison. - The project_keys allowlist now fails closed: events it cannot attribute to a project are skipped rather than dispatched, matching github/slack. - Retry-After is parsed defensively and clamped; an HTTP-date value raised ValueError, and a large value would have slept unbounded. - The dashboard's recent-events table read the 25 oldest events, not the newest, so it froze after 25 events; allowlist values are now escaped. - _dispatch docstring said "Slack retries" (copy-paste). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VKZrTNjjWVzTZUxDFbn4Nk Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
- launch_task and blocking_run are now async-first, wrapped with flyte's @Syncify. Handlers await launch_task.aio(...), so a launch no longer blocks the app's event loop while every other in-flight request waits behind it. The synchronous form still works for scripts. - Idempotency is now entirely label-based. The run-name allocation (probe up to 32 candidate names via Run.get, then launch under the winner) is gone, along with run_name_for/RUN_NAME_MAX and the prefix and run_name_base arguments. Names race, cap how many runs one key can ever have, and were never identity — the control plane assigns them now. - The dedupe key is documented as caller-supplied: dedupe_key() is a default, not a requirement. - Examples and READMEs use the await form. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VKZrTNjjWVzTZUxDFbn4Nk Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
Wraps the 12 client methods with flyte's @Syncify, matching how the SDK itself exposes Run.listall, flyte.run and flyte.serve. Each method now has two forms: `client.foo(...)` blocks, `await client.foo.aio(...)` does not. - Adds __enter__/__exit__ so the blocking form is actually usable. They run __aenter__/__aexit__ on syncify's background loop -- the same loop the syncified methods run on -- so the httpx.AsyncClient is created and used on a single loop. - Internal self-calls use .aio(). The blocking form would deadlock when called from syncify's own loop thread. - The MCP tool bridge uses .aio(). `await getattr(client, name)(...)` would otherwise raise TypeError on the returned value, and would stall the MCP server's event loop for the duration of every tool call. - Tests, examples and READMEs use .aio() on async paths, and document both forms plus when not to reach for the blocking one. The syncified client type-checks clean: mypy resolves methods to SyncFunction[...] and the error count on the package is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VKZrTNjjWVzTZUxDFbn4Nk Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
The class docstring still demonstrated the pre-syncify async call, which no longer works as written. It now shows the async form with .aio() and the blocking form under a plain `with`, and says which belongs where. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VKZrTNjjWVzTZUxDFbn4Nk Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
cosmicBboy
force-pushed
the
nielsb/integrations-jira
branch
from
September 1, 2026 13:31
5c04b64 to
e36de1e
Compare
Adds flyteplugins.jira.events, so handlers register against constants
instead of hand-copied strings:
@app_env.on_event(events.Issue.CREATED)
Follows the ActionPhase pattern in flyte.models: `str` enums grouped by
event type, so a member is drop-in wherever a pattern string is accepted and
a typo fails at import rather than by silently never matching. Raw strings
still work, for events the constants do not cover yet.
- The enum base pins __str__/__format__ to str's. Python 3.11+ would
otherwise render members as "Class.MEMBER" rather than the wire value,
which would corrupt the dashboard and /api/status output.
- Tests assert the constants equal what the parsers actually produce, that
ANY is the bare event type every action shares as a prefix, and that no
value appears in two classes.
- Examples, READMEs, module docstrings and the on_event docstring use them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VKZrTNjjWVzTZUxDFbn4Nk
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
A step-by-step pass a human can follow against a real account: create the credentials, verify the client standalone, deploy the task the receiver launches, run it directly, deploy the app, wire the provider up, trigger a real event, and confirm idempotency. Ends with a troubleshooting table mapping each failure mode to its cause. Ordered so each step fails in isolation: the client is exercised before the platform, and the launched task is deployed before the app that looks it up. Also adds the `triage_issue` task the webhook example launches. It was looked up by name but defined nowhere, so the example could not work as written. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VKZrTNjjWVzTZUxDFbn4Nk Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
cosmicBboy
added a commit
that referenced
this pull request
Sep 3, 2026
First of a stack that replaces the six per-product integration plugins (#1502–#1507) with the parts that are genuinely Flyte's. Structured like `plugins/agents`: **core holds the contract, each product is its own thin package.** **Stack:** this → github → slack → linear → clickup → jira → examples. Each PR's diff is just its own package. ## What core owns | | | | --- | --- | | `WebhookAppEnvironment` | One app: a dashboard and a verified receiver at `/webhook/{provider}`, for whichever providers it is handed | | `Provider` | The contract a product plugin implements | | `WebhookEvent` | The normalized event every provider parses into | | `run_once` | Launch a run once per event key, returning the run that covers it either way | | `EventType` | Base for the typed event constants each plugin ships | | verification primitives | `constant_time_equals`, `hex_hmac_sha256`, `json_body`, `lower_headers` | | `testing.assert_provider_conforms` | The conformance check every plugin runs | A product plugin owns only what is specific to it — usually under 150 lines. Core is tested against a **stub provider**, so it depends on no product package. ## Two decisions worth reviewing **Identity lives on a run label, never a name.** `run_once` launches only when no run carrying the same `dedupe` label is live or succeeded; when one is, it returns that run with `created=False` rather than launching a second or raising, so a handler can answer "already handled" and link to it. Names race against concurrent launches and cap how many runs one key can ever have, so the control plane assigns them. It is async-first behind `@syncify` — the blocking form stalls an app's event loop, and webhook senders time out in seconds. A test asserts four concurrent launches overlap; against a blocking implementation it measures 0.81s instead of 0.42s and fails. **`constant_time_equals` compares bytes.** `hmac.compare_digest` raises `TypeError` on `str` operands containing non-ASCII, and these headers come off the wire — ASGI hands Starlette raw bytes which it decodes latin-1, so a crafted header would turn a clean 401 into a 500. Writing it once here is the main argument for a shared core. ## The conformance harness Each product plugin ships `SAMPLE_DELIVERY`, a real payload. The harness signs and replays it, so `verify` and `parse` are checked against an actual payload rather than against each other — without one, every other check is vacuous. It earned its keep during development twice: - It caught that **Jira's shared token is not bound to the request body**, unlike the four HMAC providers. That is inherent to an unsigned webhook, not a bug, so `signed=False` now opts out of the body-tampering check *and* makes the dashboard say the product does not sign. - It caught **a hole in itself**: the hostile-header check sent a bare non-ASCII value, which GitHub and Slack reject on their scheme prefix before reaching the comparison — leaving the check vacuous for exactly the two providers it mattered most for. Hostile values now preserve the prefix, and a `str`-based `compare_digest` fails conformance as intended. I verified it discriminates by injecting three regressions (plain `str, Enum` constants, a `str` comparison, a constant that no longer matches the parser); each one fails with a specific message. ## Notes No new dependencies land in `flyte`: the shared core lives in `flyte.extras.webhooks` behind the same soft-import pattern the other `flyte.extras` apps use, and each product plugin ships its own provider. 46 core tests plus each plugin's suite. `make fmt`, `make mypy` (864 files), `make ty`, ruff, and codespell pass. The caveat from the review that prompted this: the parsers still encode payload-shape assumptions I cannot verify without live accounts. That surface is much smaller now — the six API clients and their ~213 field assumptions are gone — and `SAMPLE_DELIVERY` pins what remains, but one real pass per product is still the thing that would confirm it. --------- Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
plugins/jira(flyteplugins-jira), a Jira Cloud integration for Flyte 2 (sixth of the integration plugins alongside GitHub/Slack/Linear/ClickUp/Notion).What's included
Read/write from tasks —
JiraClient, an async REST v3 client covering projects, issues, JQL search, comments, and workflow transitions. Plain-text descriptions/comments are converted to Atlassian Document Format automatically (and back to plain text on read). Basic auth over account email + API token, all mounted fromflyte.Secrets (JIRA_BASE_URL,JIRA_EMAIL,JIRA_API_TOKEN). 429 rate limits retried; errors raised asJiraAPIErrorwith Jira's error messages.React to Jira events —
JiraAppEnvironmentserves a setup/management dashboard (/) with end-to-end setup instructions (API token creation,flyte create secretcommands, webhook config) and status/verify endpoints, plus a webhook receiver (/webhook) protected by a sharedX-Webhook-Tokenheader — Jira webhooks are not signed, and the dashboard honestly documents the token-via-gateway pattern and the network-level fallback. Payloads are normalized intoJiraEventobjects, matched against an optional project allowlist, and dispatched toon_eventhandlers (jira:issue_created,jira:issue_updated,comment_created, ...).launch_tasklaunches runs idempotently via dedupe labels, so webhook redeliveries never launch duplicate runs.MCP server for agents on Flyte — the read/write surface doubles as MCP tools via
build_mcp_server/jira_mcp_app_env. Read-only by default;read_only=Falseadds issue creation/updates/commenting/transitions;include_destructive=Trueadditionally exposesdelete_issue. Tool annotations come from the registry. Event ingestion is intentionally not an MCP tool — that stays with the app environment.Tests
57 tests covering the client (respx-mocked Jira API, including basic-auth header, ADF conversion, and transition-by-name resolution), webhook token verification and event normalization, the MCP tool registry/server, dispatch idempotency, and the app dashboard/receiver endpoints.
Examples
examples/manage_ticket.py— open/transition/search ticketsexamples/react_to_jira_events.py— webhook app launching idempotent runsexamples/jira_mcp_server.py— MCP server for agentsReview pass
A follow-up commit fixes issues found while reviewing this branch.
Robustness and security
hmac.compare_digestraisesTypeErroronstroperands containing non-ASCII, andX-Webhook-Tokenis attacker-controlled: ASGI servers hand Starlette raw header bytes which it decodes as latin-1, so a crafted header turned a clean 401 into a 500. This matters more here than elsewhere, since the token is the whole authentication story for unsigned Jira webhooks.project_keysallowlist now fails closed. Events it cannot attribute to a project are skipped rather than dispatched, matching the GitHub and Slack plugins. Previously an event with no project key bypassed the allowlist entirely.Retry-Afteris parsed defensively and clamped at 60s. The header may carry an HTTP-date, which raisedValueError, and a large value would previously have slept unbounded.Dashboard and docs
[:25]on a deque that appends on the right — so it froze after the first 25 events._dispatchmodule docstring said "Slack retries on non-2xx" (copy-paste from the Slack plugin).3 new tests cover these.
Follow-up: async launching and label-only idempotency
launch_taskno longer blocks the app's event loop. It andblocking_runare now async-first, wrapped with flyte's@syncify, so handlersawait launch_task.aio(...). Previously the synchronous call stalled the whole event loop for the duration of two control-plane round-trips plus a launch — with every other in-flight webhook queued behind it, against sender timeouts measured in seconds. The synchronous form still works for scripts. A regression test asserts four concurrent launches overlap; against the old blocking call it measures 0.81s instead of 0.42s and fails.Idempotency is now purely label-based. The run-name allocation is gone — it probed up to 32 candidate names via
Run.getbefore launching under the winner. That raced with concurrent launches, silently capped how many runs one dedupe key could ever have, and treated a name as an identity it never was. Runs now carry only thededupe=<key>label and the control plane assigns the name.run_name_for,RUN_NAME_MAX, and theprefix/run_name_basearguments are removed.The dedupe key is explicitly caller-supplied.
dedupe_key()is a sensible default, not a requirement — any string chooses a different idempotency scope.The module docstring is also honest about the residual race: the label check is a read followed by a launch, so two simultaneous deliveries of one event can both launch. Redeliveries are seconds to minutes apart and dedupe reliably; closing the concurrent case needs a compare-and-set the control plane does not expose.
Follow-up: sync and async call forms on
JiraClientThe 12 client methods are wrapped with flyte's
@syncify, so each has two call forms — matching how the SDK itself exposesRun.listall,flyte.run, andflyte.serve:__enter__/__exit__were added so the blocking form is usable at all. They run__aenter__/__aexit__on syncify's background loop — the same loop the syncified methods run on — so thehttpx.AsyncClientis created and used on a single loop rather than straddling two.Two call sites had to move to
.aio()for correctness, not just style. Internalself.calls would otherwise deadlock, since syncify raises on a blocking call made from its own loop thread. And the MCP tool bridge (await getattr(client, name)(...)) would raiseTypeError: object dict can't be used in 'await' expressionon the returned value — and, had it not raised, would have stalled the MCP server's event loop for the duration of every tool call. Measured: 0 event-loop ticks during one blocking call.The dynamic
sync_*-method alternative was rejected: it works at runtime but both mypy and ty report"JiraClient" has no attribute "sync_..."and fall back toAny, so every typed caller breaks.@syncifyin place resolves toSyncFunction[[...], ...]in both checkers. The package's mypy error count is unchanged at 12, all pre-existing, none in_client.py.Tests, examples, and READMEs use
.aio()on async paths and document both forms, including when not to reach for the blocking one.Follow-up: typed event constants
on_eventno longer needs hand-copied strings:flyteplugins.jira.eventsadds 6 classes / 22 constants, following theActionPhasepattern inflyte.models—strenums grouped by event type, so a member drops in wherever a pattern string is accepted and a typo fails at import rather than by silently never matching. The constants carry Jira's exact wire values, so you need not remember which names take thejira:prefix. Raw strings still work, for events the constants do not cover yet.The enum base pins
__str__/__format__tostr's. Without that, Python 3.11+ renders members asClass.MEMBERrather than the wire value — which would have leaked enum names into the dashboard and/api/status, since handler patterns flow into both.Tests assert the constants equal what the parser actually produces (through the app's own
_matches), that everyANYis the bare type its actions prefix, and that no value appears in two classes.Follow-up: an end-to-end Testing guide
The README now ends with a numbered walkthrough against a real account, ordered so each step fails in isolation — exercise the client standalone before involving the platform, deploy the launched task before the app that looks it up, run it directly before any webhook is in play. Then wire up the provider, trigger a real event, and check the four places evidence lands: the provider's delivery log,
/api/events,flyte get runs, and the resource itself. It ends with a troubleshooting table mapping each symptom to its cause, since 401, 503, and "200 but no run" are three different failures.It also walks through the idempotency behaviour by hand, which is the part unit tests can only simulate.
Writing the guide surfaced a gap and it is fixed here: the
triage_issuetaskreact_to_jira_events.pylaunches was looked up by name but defined nowhere, so anyone following the README end to end would have hit a task-not-found error at the first real event.Verified mechanically rather than by eye: every referenced example path resolves, every
flyte create secretname matches the actual_config.pydefault, everyflyte deploy <file> <env>names a realTaskEnvironment, everyflyte runtask is defined, and every example file still executes.