Skip to content

Commit 67204ec

Browse files
committed
feat: add download subcommand for user-attachments URLs (#42)
Adds `gh image download` to fetch attachments back out of GitHub, closing the half of the attachment problem the tool did not cover. An attachment URL answers with a 302 to a presigned storage URL, so the flow has two legs with opposite credential requirements: the first needs a credential, the second must carry none at all — an Authorization header on the S3 bucket is rejected with a 400, and that failure is invisible on the other storage host. Neither client follows redirects; the redirect is classified explicitly so a login interstitial or error page can never be written to disk as a plausible attachment. The resolve leg takes the same two routes as upload: the gh CLI's bearer token first, the browser session as fallback, so a run that stays on the fast path never touches the cookie store. The routing is simpler than upload's — the bearer upload endpoint accepts only a narrow set of content types, while one credential reaches every attachment on the way back out, so a single rejection turns the fast route off for the rest of the run rather than being remembered per content type. A 404 is what triggers the fallback, since GitHub answers the same way for an absent asset and for one the credential cannot read; any other status is surfaced as-is, because it says nothing about the credential. Output follows curl's conventions: gh image download <url>... derived names in the cwd gh image download --output-dir <dir> ... derived names in a directory gh image download --output <file> <url> an exact path gh image download --output - <url> stream to stdout Existing files are overwritten, as curl -O does; --no-clobber suffixes .1, .2 instead. Filenames come from the URL rather than any response header: /files/ URLs carry their name and GitHub validates it, while /assets/ URLs carry only a uuid, so the extension comes from the presigned path. The destination is opened only after the fetch returns 200, so a failed request leaves no 0-byte file, and a partial write is removed rather than left behind. Protocol notes are in documentation/github-attachment-download-flow.md.
1 parent d59ac2b commit 67204ec

9 files changed

Lines changed: 1988 additions & 38 deletions

File tree

README.md

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,18 @@ https://github.com/user-attachments/assets/…
8888

8989
If any upload fails, the error is printed to stderr and the process exits non-zero — other files in the batch still upload.
9090

91+
### Download
92+
93+
```bash
94+
# Fetch attachments, named from the URL, into the current directory
95+
gh image download <url>... [--output-dir <dir>] [--no-clobber]
96+
97+
# Or send a single attachment somewhere specific — `-` for stdout
98+
gh image download <url> --output <file>
99+
```
100+
101+
Existing files are overwritten unless `--no-clobber` is passed, which writes `name.1`, `name.2` instead. As with upload, a failed URL is reported to stderr and the process exits non-zero — the rest of the batch still downloads.
102+
91103
### Pipe directly into an issue, PR, or comment
92104

93105
From inside the repo's working directory, both `gh image` and `gh issue create` infer the target repository automatically:
@@ -151,7 +163,7 @@ A <code>demo-videos</code> skill publishes the <b>README demo reels</b> &mdash;
151163

152164
## Authentication
153165

154-
`gh-image` authenticates with credentials you already have — **nothing to provision, no OAuth scopes to configure**. Images and video going to a repository you can push to are uploaded with your `gh` CLI token; everything else — other file types, and repositories you cannot push to — falls back to your existing GitHub session, read as the `user_session` cookie from your browser's encrypted cookie store.
166+
`gh-image` authenticates with credentials you already have — **nothing to provision, no OAuth scopes to configure**. Images and video going to a repository you can push to are uploaded with your `gh` CLI token; everything else — other file types, and repositories you cannot push to — falls back to your existing GitHub session, read as the `user_session` cookie from your browser's encrypted cookie store. Downloads take the same two routes: the `gh` token first, your browser session as fallback.
155167

156168
**Supported browsers:** Chrome · Brave · Chromium · Edge · Firefox · Opera · Safari
157169

SECURITY.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
## What this tool handles
44

5-
`gh-image` reads your GitHub `user_session` cookie from your browser's encrypted cookie store (or from an explicit token source) and uses it to authenticate against GitHub's internal image upload API.
5+
`gh-image` reads your GitHub `user_session` cookie from your browser's encrypted cookie store (or from an explicit token source) and uses it to authenticate against GitHub's internal attachment APIs.
66

77
The cookie grants **full account access** — equivalent to your GitHub password, and not scoped like a personal access token. See the [Authentication](README.md#authentication) section of the README for full details on how the cookie is sourced and used.
88

documentation/architecture.md

Lines changed: 41 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
## Overview
44

5-
`gh-image` is a Go CLI tool distributed as a `gh` extension. It uploads files — images and other GitHub-supported attachments like PDFs and zips — to GitHub using the same internal API that the web UI uses when you drag-and-drop or paste an attachment. The tool resolves a GitHub session (from a flag, env var, or browser cookie store), negotiates upload tokens, performs an S3 presigned upload, then prints the resulting markdown reference to stdout (an image embed for images, a download link for other files).
5+
`gh-image` is a Go CLI tool distributed as a `gh` extension. It uploads and downloads files — images and other GitHub-supported attachments like PDFs and zips — to GitHub using the same internal API that the web UI uses when you drag-and-drop or paste an attachment. The tool resolves a GitHub session (from a flag, env var, or browser cookie store), negotiates upload tokens, performs an S3 presigned upload, then prints the resulting markdown reference to stdout (an image embed for images, a download link for other files).
66

77
## Project Structure
88

@@ -21,6 +21,9 @@ gh-image/
2121
│ ├── session/
2222
│ │ ├── session.go # Session token validation (check-token)
2323
│ │ └── session_test.go
24+
│ ├── download/
25+
│ │ ├── download.go # 2-leg attachment download + filename derivation
26+
│ │ └── download_test.go
2427
│ ├── httputil/
2528
│ │ └── httputil.go # Shared User-Agent constant
2629
│ ├── upload/
@@ -33,8 +36,9 @@ gh-image/
3336
│ ├── repo.go # Infers owner/repo from git remote, resolves repo ID
3437
│ └── repo_test.go
3538
├── documentation/
36-
│ ├── architecture.md # This file
37-
│ └── github-image-upload-flow.md # Reverse-engineered upload protocol
39+
│ ├── architecture.md # This file
40+
│ ├── github-image-upload-flow.md # Reverse-engineered upload protocol
41+
│ └── github-attachment-download-flow.md # Reverse-engineered download protocol
3842
└── .github/
3943
└── workflows/
4044
└── release.yml # GoReleaser cross-compilation + release
@@ -44,11 +48,13 @@ gh-image/
4448

4549
```
4650
gh image [--repo owner/repo] [--token <value>] <file-path>...
51+
gh image download [--output <file>|-] [--output-dir <dir>] [--no-clobber] [--token <value>] <url>...
4752
gh image extract-token
4853
gh image check-token [--token <value>]
4954
```
5055

5156
- **Default mode** uploads one or more files and prints markdown references to stdout. Flags may appear before or after positional args; use `--` to pass filenames that begin with `-`.
57+
- **`download`** fetches one or more `user-attachments` URLs. With no output flag each lands in the current directory under a name derived from the URL; `--output-dir` picks a different directory, `--output <file>` an exact path, and `--output -` streams to stdout. Existing files are overwritten unless `--no-clobber` is given, which suffixes `.1`, `.2` instead.
5258
- **`extract-token`** reads the session cookie from the browser and prints the raw token value to stdout (status info to stderr). Useful for piping into CI secrets.
5359
- **`check-token`** resolves a token using the standard precedence (flag → env → browser) and verifies it against GitHub, printing the authenticated username on success.
5460

@@ -200,7 +206,31 @@ finalizeUpload() ──→ PUT {asset_upload_url}
200206

201207
Handles the multipart form construction for the S3 presigned upload. Separated from the main orchestration because the S3 request has different requirements (no cookies, no GitHub headers, just the presigned form fields and file data).
202208

203-
### 7. CLI Entrypoint (`main.go`)
209+
### 7. Download Flow (`internal/download/`)
210+
211+
Implements the download protocol documented in [github-attachment-download-flow.md](github-attachment-download-flow.md). A `GET` on the attachment URL answers with a `302` to a presigned storage URL, so there are two legs with **opposite** credential requirements: the first needs the session cookie pair, the second must carry none at all — an `Authorization` header on the S3 bucket is rejected with a 400, and the presigned URL is its own capability.
212+
213+
```go
214+
// NewClient builds both HTTP clients from the session cookie.
215+
func NewClient(sessionCookie *http.Cookie) *Client
216+
217+
// Save resolves ref and writes it, returning the path written.
218+
func (c *Client) Save(ref Ref, dest Dest) (string, error)
219+
220+
// Stream resolves ref and writes its bytes to w (the --output - path).
221+
func (c *Client) Stream(ref Ref, w io.Writer) (int64, error)
222+
```
223+
224+
**Key implementation details:**
225+
226+
- **Two credential routes, bearer first.** The `gh` token is tried before the browser session, so a run that stays on the fast path never touches the cookie store and never prompts for it. A `404` is what triggers the fallback, and it is deliberately ambiguous — GitHub answers the same way for an absent asset and for one the credential cannot read — so the session is tried once before the run reports failure. Any other status is surfaced as-is rather than retried, since it says nothing about the credential. An explicit `--token` or `GH_SESSION_TOKEN` pins the run to the session route.
227+
- **Neither client follows redirects.** The resolve leg classifies its own redirect; the fetch leg must not hop onward past that classification.
228+
- **A `302` is classified in a fixed order:** a `/login` target *on github.com* means the session is stale; a target carrying `X-Amz-Signature` is the asset; anything else is a hard error. The order matters — checking the signature first would report an expired session as an unusable-target error. The host check matters too: matching `/login` by path alone would let an unrelated host claim the credential is stale.
229+
- **The destination is opened only after the fetch returns 200**, so a failed request leaves no 0-byte file. A partial write is removed rather than left in place.
230+
- **Filenames come from the URL, never from a response header.** `/files/<id>/<name>` carries its name and GitHub validates it; `/assets/<uuid>` carries none, so the extension is taken from the presigned path. `filepath.Base` at the join keeps a traversal-shaped name inside the destination.
231+
- **Timeouts mirror the upload side:** 30s for the header-only resolve leg, 120s for the transfer, matching `s3.go`.
232+
233+
### 8. CLI Entrypoint (`main.go`)
204234

205235
`main()` is a one-line entrypoint that delegates to a testable
206236
`run(args []string, stdout, stderr io.Writer, deps) int`: it returns an exit code
@@ -215,6 +245,7 @@ Responsibilities:
215245
- **Subcommand dispatch** for `extract-token` and `check-token`, with validation that disallowed flag combinations are rejected before any work is done.
216246
- **Session resolution** via `resolveSessionCookie`, which applies the flag → env → browser precedence and wraps raw token values into a properly scoped `*http.Cookie`.
217247
- **Multi-file upload loop**: each positional path is uploaded independently. A failure on one file is reported to stderr and the loop continues; the process exits non-zero if any upload failed.
248+
- **Download loop**: same contract for URLs. All flag validation happens before any request, so a bad combination fails immediately. `--output -` streams to `run`'s injected stdout writer rather than `os.Stdout`, keeping the path testable.
218249

219250
## Data Flow
220251

@@ -257,8 +288,13 @@ flowchart TD
257288
| Step 2 (S3 upload) | None | Presigned policy from step 1 |
258289
| Repo ID lookup | OAuth token | `gh` CLI (via `gh auth`) |
259290
| `check-token` validation | Same `user_session` pair | Same precedence as upload |
291+
| Download resolve leg (fast) | `Authorization: Bearer` | `gh auth token`, skipped when a session token is named |
292+
| Download resolve leg (fallback) | `user_session` + `__Host-user_session_same_site` cookies | `--token` flag, `GH_SESSION_TOKEN`, or browser cookie DB |
293+
| Download fetch leg | **None** | Presigned URL from the resolve leg |
294+
295+
Download uses the same two routes as upload, for the same reason: the `gh` token avoids the browser entirely when it works. The routing is simpler here, though. Upload has to remember rejections *per content type*, because the bearer upload endpoint accepts a narrow set; download has no such split — one credential reaches every attachment — so a single rejection turns the fast route off for the rest of the run.
260296
261-
The session-cookie path and the `gh` CLI auth path are independent. The cookie provides a browser-equivalent session for the undocumented upload API, while `gh` handles the standard REST API used only for looking up the numeric repository ID.
297+
The session-cookie path and the `gh` CLI auth path are independent. The cookie provides a browser-equivalent session for the undocumented attachment APIs, while `gh` handles the standard REST API used only for looking up the numeric repository ID.
262298
263299
## Distribution
264300
Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
# GitHub Attachment Download Flow
2+
3+
## Overview
4+
5+
Fetching a `user-attachments` asset is a two-leg operation. A `GET` on the
6+
attachment URL answers with a `302` to a presigned storage URL; the bytes come
7+
from that second URL. The legs have **opposite** credential requirements, which
8+
is the single most important thing to know about this protocol.
9+
10+
Companion to [github-image-upload-flow.md](github-image-upload-flow.md).
11+
12+
## The two URL shapes
13+
14+
GitHub routes uploads to one of two shapes depending on the file type, and they
15+
behave differently on the way back out.
16+
17+
| | `/user-attachments/assets/<uuid>` | `/user-attachments/files/<id>/<name>` |
18+
|---|---|---|
19+
| Used for | images and videos | PDF, zip, log, txt — everything else |
20+
| Filename in the URL | **no**, only a uuid | **yes** |
21+
| Presigned host | `github-production-user-asset-*.s3.amazonaws.com` | `objects.githubusercontent.com` |
22+
| `response-content-disposition` on the redirect | no | yes |
23+
24+
Both redirect targets carry `X-Amz-Expires=300` and `response-content-type`.
25+
26+
## The Flow
27+
28+
### Step 1: Resolve
29+
30+
Either credential works:
31+
32+
```
33+
GET https://github.com/user-attachments/assets/<uuid>
34+
Authorization: Bearer <gh auth token>
35+
User-Agent: …
36+
```
37+
38+
```
39+
GET https://github.com/user-attachments/assets/<uuid>
40+
Cookie: user_session=…; __Host-user_session_same_site=…
41+
User-Agent: …
42+
```
43+
44+
The bearer token is the fast path — it needs no browser and no keychain access.
45+
Verified against a private repository, for an asset uploaded by a different
46+
user, so the grant follows repository read permission rather than uploader
47+
identity. Both URL shapes work, unlike the bearer *upload* endpoint, which
48+
accepts only a narrow set of content types.
49+
50+
On the cookie route both cookies are required. Sending only `user_session` looks
51+
correct and fails to authenticate — the same requirement the browser-session
52+
upload endpoints have.
53+
54+
The response is a `302`. **Do not follow it automatically**; classify it:
55+
56+
| Redirect target | Meaning |
57+
|---|---|
58+
| `/login`, resolved host exactly `github.com` | the session token is invalid or expired |
59+
| absolute URL carrying `X-Amz-Signature` | the asset — proceed to step 2 |
60+
| anything else | refuse; do not fetch |
61+
62+
Order matters. Checking for the signature first turns an expired session into a
63+
confusing "unusable target" error. The host check on `/login` matters too:
64+
matching by path alone would let `https://attacker.example/login` be read as
65+
"your credential is stale", which on a tool that reads a browser cookie store is
66+
a prompt an attacker should not be able to trigger.
67+
68+
The third row is what prevents an SSO interstitial or error page from flowing
69+
into step 2, returning `200` with a consistent `Content-Length`, and landing on
70+
disk as a perfectly plausible attachment.
71+
72+
### Step 2: Fetch
73+
74+
```
75+
GET <presigned URL>
76+
User-Agent: …
77+
```
78+
79+
**No credentials of any kind.** The presigned URL is its own capability.
80+
81+
An `Authorization` header here returns **400** on the S3 bucket — AWS rejects
82+
two auth mechanisms on one request. `objects.githubusercontent.com` tolerates
83+
it, so the mistake is invisible on half the assets and easy to ship.
84+
85+
Verify the byte count against `Content-Length` when present, so a truncated
86+
transfer never passes for a complete file.
87+
88+
## Filenames
89+
90+
Never taken from a response header:
91+
92+
| URL shape | Name from |
93+
|---|---|
94+
| `/files/<id>/<name>` | the `<name>` segment of the URL |
95+
| `/assets/<uuid>` | the uuid plus the extension of the presigned path |
96+
97+
The `/files/` name is server-validated — a valid file id with a tampered
98+
filename returns `404` — so the URL is a trustworthy source. An `/assets/` URL
99+
carries no name at all, so the extension has to come out of the redirect, which
100+
is why `curl -LO` on one produces an extensionless uuid.
101+
102+
GitHub sanitizes names at upload time: `paren (1) test.txt` is stored as
103+
`paren.1.test.txt`.
104+
105+
## Authentication Summary
106+
107+
| Leg | Auth |
108+
|---|---|
109+
| Resolve (github.com) | `Authorization: Bearer` **or** `user_session` + `__Host-user_session_same_site` |
110+
| Fetch (presigned) | none |
111+
112+
A `404` cannot distinguish "no such asset" from "this credential cannot read
113+
it", so a client holding both credentials should try the second before
114+
reporting failure.
115+
116+
## Observed Responses
117+
118+
| Condition | Result |
119+
|---|---|
120+
| anonymous, private asset | `404` |
121+
| valid session, private asset | `302` to presigned |
122+
| valid bearer token, private asset | `302` to presigned |
123+
| invalid bearer token | `404` |
124+
| stale session, `/assets/` | `302` to `/login` |
125+
| stale session, `/files/` | `404` |
126+
| well-formed but nonexistent uuid | `404` |
127+
| nonexistent file id | `404` |
128+
| valid file id, tampered filename | `404` |
129+
| stray `Authorization` on the presigned leg | `400` (S3), tolerated (objects host) |
130+
| `HEAD` on the presigned leg | `403` (S3) — signed for `GET` only |
131+
| 30 sequential authenticated resolves | `302` every time; no `429`, no rate-limit headers |
132+
133+
A `404` conflates "no such asset" with "exists but you cannot read it", so error
134+
messages have to name both.
135+
136+
## Caveats
137+
138+
- The endpoints are **undocumented** and may change without notice.
139+
- `HEAD` is not usable for metadata on the S3 host, so asset size cannot be
140+
probed without fetching.
141+
- Presigned URLs expire; treat one as valid only for the request that
142+
immediately follows its resolve.
143+
- `curl -O` on an attachment URL writes **0 bytes and exits 0** — without `-L`
144+
it saves the empty redirect body and reports success.

0 commit comments

Comments
 (0)