Skip to content

Commit 152b371

Browse files
committed
docs(github): add an end-to-end Testing guide to the README
A step-by-step pass a human can follow against a real account: create the credentials, verify the client standalone, deploy the task the receiver launches, run it directly, deploy the app, wire the provider up, trigger a real event, and confirm idempotency. Ends with a troubleshooting table mapping each failure mode to its cause. Ordered so each step fails in isolation: the client is exercised before the platform, and the launched task is deployed before the app that looks it up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VKZrTNjjWVzTZUxDFbn4Nk Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
1 parent 93f4bf8 commit 152b371

1 file changed

Lines changed: 103 additions & 0 deletions

File tree

plugins/github/README.md

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,3 +193,106 @@ agent = Agent(
193193
API base URL (GitHub Enterprise Server), timeouts, and retries. The module
194194
exports `default_config`; pass a custom `Config` to `GitHubClient`,
195195
`build_mcp_server`, or `collect_review_context` when you need it.
196+
197+
## Testing
198+
199+
An end-to-end pass against a real repository. Use a scratch repo you own —
200+
step 4 comments on and labels a real pull request.
201+
202+
**1. Create the credentials.**
203+
204+
A fine-grained personal access token (GitHub → Settings → Developer settings →
205+
Fine-grained tokens) scoped to your test repo, with *Pull requests: read &
206+
write*, *Issues: read & write*, and *Checks: read & write*. Then pick any
207+
random string as the webhook secret:
208+
209+
```bash
210+
flyte create secret GITHUB_TOKEN --value <token>
211+
flyte create secret GITHUB_WEBHOOK_SECRET --value <random-string>
212+
```
213+
214+
**2. Check the client works before involving the platform.** Everything below
215+
is easier to debug once you know the token is good:
216+
217+
```bash
218+
export GITHUB_TOKEN=<token>
219+
python -c "
220+
from flyteplugins.github import GitHubClient
221+
with GitHubClient() as c:
222+
print(c.get_repository('<owner>/<repo>')['full_name'])
223+
"
224+
```
225+
226+
**3. Deploy the task the webhook will launch.**
227+
228+
```bash
229+
flyte deploy plugins/github/examples/read_write_pr.py env
230+
```
231+
232+
`react_to_pr_events.py` looks this task up by name (`triage_pr`), so it has to
233+
exist before the app can launch it.
234+
235+
**4. Run the read/write task directly**, to confirm writes land before any
236+
webhook is in play. Open a PR in your test repo, then:
237+
238+
```bash
239+
flyte run plugins/github/examples/read_write_pr.py triage_pr \
240+
--repo <owner>/<repo> --number <pr-number>
241+
```
242+
243+
The PR should pick up a `flyte-triage` label, a comment with its diff stats,
244+
and a check run.
245+
246+
**5. Deploy the webhook app.**
247+
248+
```bash
249+
python plugins/github/examples/react_to_pr_events.py
250+
```
251+
252+
It prints the app URL. Open it: the dashboard should show both secrets as
253+
mounted, and *Verify GitHub credentials* should return your login.
254+
255+
**6. Point GitHub at the app.** In the repo's Settings → Webhooks → Add
256+
webhook:
257+
258+
- Payload URL: `<app-url>/webhook`
259+
- Content type: `application/json`
260+
- Secret: the same value you used for `GITHUB_WEBHOOK_SECRET`
261+
- Events: *Let me select individual events* → Pull requests, Issues
262+
263+
GitHub immediately sends a `ping`, which the receiver answers with
264+
`{"ok": true, "ping": true}` — a green checkmark in the webhook's *Recent
265+
Deliveries* tab means the URL is reachable.
266+
267+
**7. Trigger a real event.** Open a new pull request. Then check, in order:
268+
269+
- GitHub's *Recent Deliveries* tab — the `pull_request` delivery should be 200,
270+
and the response body names the handler that ran and the run it launched.
271+
- `<app-url>/api/events` — the normalized event.
272+
- `flyte get runs` — a run whose `dedupe` label matches the event.
273+
- The PR itself — label, comment, and check run.
274+
275+
**8. Confirm idempotency.** Hit *Redeliver* on that same delivery in GitHub.
276+
The response should report `skipped` with a `DuplicateRun` message, and no
277+
second run should appear. This is the behaviour worth checking by hand, since
278+
it is the one a webhook sender will exercise on its own during an outage.
279+
280+
**9. Optional — the MCP server.**
281+
282+
```bash
283+
python plugins/github/examples/github_mcp_server.py
284+
claude mcp add --transport http github-mcp <app-url>/mcp/mcp
285+
```
286+
287+
Ask an agent to summarize a PR in your test repo. The default surface is
288+
read-only, so it can look but not touch.
289+
290+
### Troubleshooting
291+
292+
| Symptom | Cause |
293+
| --- | --- |
294+
| Webhook delivery returns 401 | The secret in GitHub does not match `GITHUB_WEBHOOK_SECRET`. |
295+
| Delivery returns 503 | `GITHUB_WEBHOOK_SECRET` is not mounted on the app; check `/api/status`. |
296+
| Delivery is 200 but no run appears | No handler matched. `/api/status` lists the registered patterns. |
297+
| Handler reports a task-not-found error | Step 3 was skipped, or the task deployed under a different name. |
298+
| Second delivery launches a second run | Expected when the first run failed — failed runs do not block a retry. |

0 commit comments

Comments
 (0)