Skip to content

Commit 08c96a7

Browse files
cosmicBboyclaude
andcommitted
feat(webhooks): add flyteplugins-webhooks-github
Receive GitHub webhooks in Flyte. Exports `GitHubProvider`, a Provider subclass with its defaults pre-wired, so wiring it up reads: WebhookAppEnvironment(providers=[GitHubProvider()]) Implements the contract from flyteplugins-webhooks-core: which environment variable holds the secret, how to verify a delivery, how to parse one into a WebhookEvent, plus typed constants for every event GitHub can send. examples/github_webhooks.py runs with no GitHub account at all — `--local` replays the plugin's own SAMPLE_DELIVERY through the app, so you can watch a delivery be verified, normalized, and dispatched before wiring anything up. Runs the shared conformance check, which replays that same sample through verify and parse rather than trusting them to agree with each other. Calling the GitHub API is deliberately not this plugin's job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VKZrTNjjWVzTZUxDFbn4Nk Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
1 parent 46ac7e6 commit 08c96a7

9 files changed

Lines changed: 2205 additions & 0 deletions

File tree

plugins/webhooks/github/README.md

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# flyteplugins-webhooks-github
2+
3+
Receive GitHub webhooks in Flyte.
4+
5+
```bash
6+
pip install "flyteplugins-webhooks-core[app]" flyteplugins-webhooks-github
7+
```
8+
9+
## Using it
10+
11+
Hand a `GitHubProvider()` to a `WebhookAppEnvironment` and register handlers with the
12+
typed constants in `events`:
13+
14+
```python
15+
import flyte
16+
from flyteplugins.webhooks.core import DuplicateRun, WebhookAppEnvironment, idempotent_run
17+
from flyteplugins.webhooks.github import GitHubProvider, events
18+
19+
app_env = WebhookAppEnvironment(
20+
name="github-webhooks",
21+
providers=[GitHubProvider()],
22+
secrets=[flyte.Secret("GITHUB_WEBHOOK_SECRET", as_env_var="GITHUB_WEBHOOK_SECRET")],
23+
)
24+
25+
26+
@app_env.on_event(events.PullRequest.OPENED)
27+
async def handle(event):
28+
import flyte.remote as remote
29+
30+
task = remote.Task.get(name="my-env.my_task", auto_version="latest")
31+
try:
32+
run = await idempotent_run.aio(task, key=event.dedupe_key(), resource=event.resource_id)
33+
except DuplicateRun as exc:
34+
return {"skipped": str(exc)}
35+
return {"run": run.name}
36+
37+
38+
flyte.serve(app_env)
39+
```
40+
41+
Handlers must `await idempotent_run.aio(...)`. The blocking form stalls the
42+
app's event loop, and GitHub times deliveries out in seconds.
43+
44+
One app can serve several products at once — hand it one provider per product.
45+
46+
## Try it
47+
48+
`examples/github_webhooks.py` runs two ways. The first needs no GitHub account:
49+
50+
```bash
51+
python examples/github_webhooks.py --local # replay a real sample delivery in-process
52+
python examples/github_webhooks.py # deploy the receiver to Flyte
53+
```
54+
55+
`--local` posts this plugin's `SAMPLE_DELIVERY` through the app with FastAPI's
56+
test client, so you see a delivery verified, normalized, and dispatched — plus
57+
an unsigned one refused with a 401, and the same delivery replayed to show the
58+
dedupe key is stable.
59+
60+
## Setup
61+
62+
1. Store the secret and mount it on the app:
63+
```bash
64+
flyte create secret GITHUB_WEBHOOK_SECRET --value <secret>
65+
```
66+
2. Point GitHub at `<app-url>/webhook/github`, from
67+
repository Settings → Webhooks → Add webhook, content type `application/json`.
68+
69+
GitHub sends a `ping` when the webhook is created; it is answered automatically, so a green check in *Recent Deliveries* means the app is reachable.
70+
71+
**Verification:** HMAC-SHA256 over the raw body (`X-Hub-Signature-256`).
72+
73+
Comment and review events fold the comment id into `resource_id`, so two comments on one issue are two events rather than a redelivery of the first.
74+
75+
## Event constants
76+
77+
`events` spells every event this plugin can dispatch, as `str` enums grouped by
78+
event type, so a typo fails at import rather than by silently never matching.
79+
Raw strings still work, for events the constants do not cover yet.
80+
81+
## What this plugin does not do
82+
83+
Call the GitHub API. Use `PyGithub` directly from your tasks — see
84+
`examples/external_saas_integrations`. This plugin owns only the part that is
85+
Flyte's: authenticating an inbound delivery and turning it into a run.
Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
"""Receive GitHub webhooks in Flyte, and see one arrive without leaving your laptop.
2+
3+
Two ways to run this. The second needs no GitHub account at all:
4+
5+
python github_webhooks.py --local # replay a real sample delivery in-process
6+
python github_webhooks.py # deploy the receiver to Flyte
7+
8+
`--local` runs the app through FastAPI's test client and posts this plugin's
9+
`SAMPLE_DELIVERY` — a `pull_request.opened` delivery — signed with a throwaway secret. You see the
10+
delivery verified, normalized, and dispatched to a handler, which is the whole
11+
path a real webhook takes.
12+
13+
To receive real events, deploy it and point GitHub at `<app-url>/webhook/github`
14+
from repository Settings -> Webhooks -> Add webhook, content type `application/json`.
15+
16+
Setup for the real thing:
17+
flyte create secret GITHUB_WEBHOOK_SECRET --value <secret>
18+
19+
A fine-grained token is not needed to *receive* webhooks — only the shared secret you set on the webhook itself.
20+
"""
21+
22+
import os
23+
import sys
24+
25+
import flyte
26+
from flyteplugins.webhooks.core import WebhookAppEnvironment
27+
28+
from flyteplugins.webhooks.github import DEFAULT_SECRET_ENV, SAMPLE_DELIVERY, GitHubProvider, events
29+
30+
image = flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
31+
"flyteplugins-webhooks-core[app]", "flyteplugins-webhooks-github"
32+
)
33+
34+
app_env = WebhookAppEnvironment(
35+
name="github-webhooks",
36+
providers=[GitHubProvider()],
37+
image=image,
38+
secrets=[flyte.Secret(DEFAULT_SECRET_ENV, as_env_var=DEFAULT_SECRET_ENV)],
39+
)
40+
41+
42+
@app_env.on_event(events.PullRequest.OPENED)
43+
async def on_primary(event):
44+
"""React to the event this plugin's sample delivery carries.
45+
46+
Returning a dict is enough to see the path working. To do real work, launch
47+
a deployed task instead — see `launch_a_task` below.
48+
"""
49+
return {
50+
"saw": event.qualified_type,
51+
"resource": event.resource_id,
52+
"title": event.title,
53+
# The key `idempotent_run` would dedupe on. Replaying the same delivery
54+
# produces the same key, which is what makes a redelivery a no-op.
55+
"dedupe_key": event.dedupe_key(),
56+
}
57+
58+
59+
@app_env.on_event(events.Issues.OPENED)
60+
async def on_secondary(event):
61+
"""A second handler, to show dispatch picking the right one per event."""
62+
return {"saw": event.qualified_type, "resource": event.resource_id}
63+
64+
65+
async def launch_a_task(event):
66+
"""What a handler looks like once it does real work.
67+
68+
Not registered above, because it needs `github-triage.triage_pr` deployed first
69+
and a Flyte backend to launch into. Wire it up with:
70+
71+
@app_env.on_event(events.PullRequest.OPENED)
72+
73+
`idempotent_run` refuses to launch when a run carrying the same dedupe key
74+
is already live or has succeeded, so GitHub redelivering an event — which
75+
it does on any non-2xx — never starts a second run.
76+
"""
77+
import flyte.remote as remote
78+
from flyteplugins.webhooks.core import DuplicateRun, idempotent_run
79+
80+
task = remote.Task.get(name="github-triage.triage_pr", auto_version="latest")
81+
try:
82+
# Always `.aio`: the blocking form stalls the app's event loop, and
83+
# webhook senders time deliveries out in seconds.
84+
run = await idempotent_run.aio(
85+
task, key=event.dedupe_key(), repo=event.scope, number=int((event.resource_id or "#0").split("#")[1])
86+
)
87+
except DuplicateRun as exc:
88+
return {"skipped": str(exc)}
89+
return {"run": run.name}
90+
91+
92+
def _try_locally() -> None:
93+
"""Post this plugin's sample delivery to the app, in-process."""
94+
from fastapi.testclient import TestClient
95+
96+
secret = os.environ.setdefault(DEFAULT_SECRET_ENV, "local-trial-secret")
97+
build_headers, body = SAMPLE_DELIVERY
98+
client = TestClient(app_env.app)
99+
100+
print("POST /webhook/github (signed with a throwaway secret)")
101+
response = client.post("/webhook/github", content=body, headers=build_headers(body, secret))
102+
print(f" {response.status_code} {response.json()}\n")
103+
104+
print("the same delivery again — note the identical dedupe_key:")
105+
again = client.post("/webhook/github", content=body, headers=build_headers(body, secret))
106+
print(f" {again.status_code} {again.json()}\n")
107+
108+
print("an unsigned delivery is refused:")
109+
bad = client.post("/webhook/github", content=body, headers={})
110+
print(f" {bad.status_code} {bad.json()}\n")
111+
112+
print("normalized events the app has seen:")
113+
for seen in client.get("/api/events").json():
114+
print(f" {seen['provider']} {seen['qualified_type']} resource={seen['resource_id']}")
115+
116+
117+
if __name__ == "__main__":
118+
if "--local" in sys.argv:
119+
_try_locally()
120+
else:
121+
flyte.init_from_config()
122+
handle = flyte.serve(app_env)
123+
handle.activate(wait=True)
124+
print(f"Dashboard ready at {handle.endpoint}")
125+
print(f"Point GitHub at {handle.endpoint}/webhook/github")
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
[project]
2+
name = "flyteplugins-webhooks-github"
3+
dynamic = ["version"]
4+
description = "Receive GitHub webhooks in Flyte."
5+
readme = "README.md"
6+
authors = [{ name = "Flyte Contributors" }]
7+
requires-python = ">=3.10"
8+
dependencies = ["flyteplugins-webhooks-core"]
9+
10+
[build-system]
11+
requires = ["setuptools", "setuptools_scm"]
12+
build-backend = "setuptools.build_meta"
13+
14+
[dependency-groups]
15+
dev = [
16+
"pytest>=8.3.5",
17+
"pytest-asyncio>=0.26.0",
18+
"fastapi>=0.115",
19+
"uvicorn>=0.30",
20+
"httpx>=0.27",
21+
]
22+
23+
[tool.setuptools]
24+
include-package-data = true
25+
26+
[tool.setuptools.packages.find]
27+
where = ["src"]
28+
include = ["flyteplugins*"]
29+
30+
[tool.setuptools_scm]
31+
root = "../../.."
32+
33+
[tool.pytest.ini_options]
34+
norecursedirs = []
35+
log_cli = true
36+
log_cli_level = 20
37+
markers = []
38+
asyncio_mode = "auto"
39+
asyncio_default_fixture_loop_scope = "function"
40+
41+
[tool.coverage.run]
42+
branch = true
43+
44+
[tool.ruff]
45+
line-length = 120
46+
47+
[tool.ruff.lint]
48+
select = ["E", "W", "F", "I", "PLW", "YTT", "ASYNC", "C4", "T10", "EXE", "ISC", "LOG", "PIE", "Q", "RSE", "FLY", "PGH", "PLC", "PLE", "FURB", "RUF"]
49+
ignore = ["PGH003", "PLC0415"]
50+
51+
[tool.ruff.lint.per-file-ignores]
52+
"examples/*" = ["E402"]
53+
54+
[tool.uv.sources]
55+
flyte = { path = "../../..", editable = true }
56+
flyteplugins-webhooks-core = { path = "../core", editable = true }
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
"""GitHub webhooks for Flyte.
2+
3+
Hand a `GitHubProvider()` to a `WebhookAppEnvironment` and register handlers with the
4+
typed constants in `events`:
5+
6+
```python
7+
import flyte
8+
from flyteplugins.webhooks.core import DuplicateRun, WebhookAppEnvironment, idempotent_run
9+
from flyteplugins.webhooks.github import GitHubProvider, events
10+
11+
app_env = WebhookAppEnvironment(
12+
name="github-webhooks",
13+
providers=[GitHubProvider()],
14+
secrets=[flyte.Secret("GITHUB_WEBHOOK_SECRET", as_env_var="GITHUB_WEBHOOK_SECRET")],
15+
)
16+
17+
18+
@app_env.on_event(events.PullRequest.OPENED)
19+
async def triage(event):
20+
import flyte.remote as remote
21+
22+
task = remote.Task.get(name="github-triage.triage_pr", auto_version="latest")
23+
try:
24+
run = await idempotent_run.aio(task, key=event.dedupe_key(), repo=event.scope)
25+
except DuplicateRun as exc:
26+
return {"skipped": str(exc)}
27+
return {"run": run.name}
28+
```
29+
30+
Calling the GitHub API is not this plugin's job — use `PyGithub` from your
31+
tasks. See `examples/external_saas_integrations`.
32+
"""
33+
34+
import hashlib
35+
import hmac
36+
37+
from . import events
38+
from ._provider import DEFAULT_SECRET_ENV, GitHubProvider, handshake, parse, verify
39+
40+
__all__ = ["DEFAULT_SECRET_ENV", "SAMPLE_DELIVERY", "GitHubProvider", "events", "handshake", "parse", "verify"]
41+
42+
43+
def _sample_headers(body: bytes, secret: str) -> dict[str, str]:
44+
signature = hmac.new(secret.encode(), body, hashlib.sha256).hexdigest()
45+
return {
46+
"X-GitHub-Event": "pull_request",
47+
"X-GitHub-Delivery": "00000000-0000-0000-0000-000000000000",
48+
"X-Hub-Signature-256": f"sha256={signature}",
49+
}
50+
51+
52+
#: A real `pull_request.opened` delivery, trimmed to the fields the parser reads.
53+
#: The conformance harness signs and replays it, so `verify` and `parse` are
54+
#: checked against an actual payload rather than against each other.
55+
SAMPLE_DELIVERY = (
56+
_sample_headers,
57+
(
58+
b'{"action": "opened", "number": 7,'
59+
b' "pull_request": {"number": 7, "title": "Add a feature",'
60+
b' "html_url": "https://github.com/octo/repo/pull/7", "updated_at": "2024-01-01T00:00:00Z"},'
61+
b' "repository": {"full_name": "octo/repo"}, "sender": {"login": "octocat"}}'
62+
),
63+
)

0 commit comments

Comments
 (0)