Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,18 @@ https://github.com/user-attachments/assets/…

If any upload fails, the error is printed to stderr and the process exits non-zero — other files in the batch still upload.

### Download

```bash
# Fetch attachments, named from the URL, into the current directory
gh image download <url>... [--output-dir <dir>] [--no-clobber]

# Or send a single attachment somewhere specific — `-` for stdout
gh image download <url> --output <file>
```

Existing files are overwritten unless `--no-clobber` is passed, which writes `name.1`, `name.2` instead. As with upload, a failed URL is reported to stderr and the process exits non-zero — the rest of the batch still downloads.

### Pipe directly into an issue, PR, or comment

From inside the repo's working directory, both `gh image` and `gh issue create` infer the target repository automatically:
Expand Down Expand Up @@ -153,7 +165,7 @@ A <code>demo-videos</code> skill publishes the <b>README demo reels</b> &mdash;

## Authentication

`gh-image` authenticates with credentials you already have — **nothing to provision, no OAuth scopes to configure**. Images and video going to a repository you can push to are uploaded with your `gh` CLI token; everything else — other file types, and repositories you cannot push to — falls back to your existing GitHub session, read as the `user_session` cookie from your browser's encrypted cookie store.
`gh-image` authenticates with credentials you already have — **nothing to provision, no OAuth scopes to configure**. Images and video going to a repository you can push to are uploaded with your `gh` CLI token; everything else — other file types, and repositories you cannot push to — falls back to your existing GitHub session, read as the `user_session` cookie from your browser's encrypted cookie store. Downloads take the same two routes: the `gh` token first, your browser session as fallback.

**Supported browsers:** Chrome · Brave · Chromium · Edge · Firefox · Opera · Safari

Expand Down
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## What this tool handles

`gh-image` reads your GitHub `user_session` cookie from your browser's encrypted cookie store (or from an explicit token source) and uses it to authenticate against GitHub's internal image upload API.
`gh-image` reads your GitHub `user_session` cookie from your browser's encrypted cookie store (or from an explicit token source) and uses it to authenticate against GitHub's internal attachment APIs.

The cookie grants **full account access** — equivalent to your GitHub password, and not scoped like a personal access token. See the [Authentication](README.md#authentication) section of the README for full details on how the cookie is sourced and used.

Expand Down
47 changes: 42 additions & 5 deletions documentation/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Overview

`gh-image` is a Go CLI tool distributed as a `gh` extension. It uploads files — images and other GitHub-supported attachments like PDFs and zips — to GitHub using the same internal API that the web UI uses when you drag-and-drop or paste an attachment. The tool resolves a GitHub session (from a flag, env var, or browser cookie store), negotiates upload tokens, performs an S3 presigned upload, then prints the resulting markdown reference to stdout (an image embed for images, a download link for other files).
`gh-image` is a Go CLI tool distributed as a `gh` extension. It uploads and downloads files — images and other GitHub-supported attachments like PDFs and zips — to GitHub using the same internal API that the web UI uses when you drag-and-drop or paste an attachment. The tool resolves a GitHub session (from a flag, env var, or browser cookie store), negotiates upload tokens, performs an S3 presigned upload, then prints the resulting markdown reference to stdout (an image embed for images, a download link for other files).

## Project Structure

Expand All @@ -24,6 +24,9 @@ gh-image/
│ ├── session/
│ │ ├── session.go # Session token validation (check-token)
│ │ └── session_test.go
│ ├── download/
│ │ ├── download.go # 2-leg attachment download + filename derivation
│ │ └── download_test.go
│ ├── httputil/
│ │ └── httputil.go # Shared User-Agent constant
│ ├── upload/
Expand All @@ -36,8 +39,9 @@ gh-image/
│ ├── repo.go # Infers owner/repo from git remote, resolves repo ID
│ └── repo_test.go
├── documentation/
│ ├── architecture.md # This file
│ └── github-image-upload-flow.md # Reverse-engineered upload protocol
│ ├── architecture.md # This file
│ ├── github-image-upload-flow.md # Reverse-engineered upload protocol
│ └── github-attachment-download-flow.md # Reverse-engineered download protocol
└── .github/
└── workflows/
└── release.yml # GoReleaser cross-compilation + release
Expand All @@ -47,11 +51,13 @@ gh-image/

```
gh image [--repo owner/repo] [--token <value>] <file-path>...
gh image download [--output <file>|-] [--output-dir <dir>] [--no-clobber] [--token <value>] <url>...
gh image extract-token
gh image check-token [--token <value>]
```

- **Default mode** uploads one or more files and prints markdown references to stdout. Flags may appear before or after positional args; use `--` to pass filenames that begin with `-`.
- **`download`** fetches one or more `user-attachments` URLs. With no output flag each lands in the current directory under a name derived from the URL; `--output-dir` picks a different directory, `--output <file>` an exact path, and `--output -` streams to stdout. Existing files are overwritten unless `--no-clobber` is given, which suffixes `.1`, `.2` instead.
- **`extract-token`** reads the session cookie from the browser and prints the raw token value to stdout (status info to stderr). Useful for piping into CI secrets.
- **`check-token`** resolves a token using the standard precedence (flag → env → browser) and verifies it against GitHub, printing the authenticated username on success.

Expand Down Expand Up @@ -203,7 +209,32 @@ finalizeUpload() ──→ PUT {asset_upload_url}

Handles the multipart form construction for the S3 presigned upload. Separated from the main orchestration because the S3 request has different requirements (no cookies, no GitHub headers, just the presigned form fields and file data).

### 7. CLI Entrypoint (`main.go`)
### 7. Download Flow (`internal/download/`)

Implements the download protocol documented in [github-attachment-download-flow.md](github-attachment-download-flow.md). A `GET` on the attachment URL answers with a `302` to a presigned storage URL, so there are two legs with **opposite** credential requirements: the first needs the session cookie pair, the second must carry none at all — an `Authorization` header on the S3 bucket is rejected with a 400, and the presigned URL is its own capability.

```go
// NewClient builds both HTTP clients and both credential routes; a nil
// newBearer pins the run to the session cookie.
func NewClient(newBearer func() (string, error), newCookie func() (*http.Cookie, error), notify func(string)) *Client

// Save resolves ref and writes it, returning the path written.
func (c *Client) Save(ref Ref, dest Dest) (string, error)

// Stream resolves ref and writes its bytes to w (the --output - path).
func (c *Client) Stream(ref Ref, w io.Writer) (int64, error)
```

**Key implementation details:**

- **Two credential routes, bearer first.** The `gh` token is tried before the browser session, so a run that stays on the fast path never touches the cookie store and never prompts for it. A `404` or a `/login` redirect triggers the fallback, since both describe the credential rather than the asset: a `404` is deliberately ambiguous — GitHub answers the same way for an absent asset and for one the credential cannot read — while a redirect to `/login` says outright that the credential is not valid. Either way the session is tried once before the run reports failure. Any other status is surfaced as-is rather than retried, since it says nothing about the credential. An explicit `--token` or `GH_SESSION_TOKEN` pins the run to the session route.
- **Neither client follows redirects.** The resolve leg classifies its own redirect; the fetch leg must not hop onward past that classification.
- **A `302` is classified in a fixed order:** a `/login` target *on github.com* means the session is stale; a target carrying `X-Amz-Signature` is the asset; anything else is a hard error. The order matters — checking the signature first would report an expired session as an unusable-target error. The host check matters too: matching `/login` by path alone would let an unrelated host claim the credential is stale.
- **The destination is opened only after the fetch returns 200**, so a failed request leaves no 0-byte file. A partial write is removed rather than left in place.
- **Filenames come from the URL, never from a response header.** `/files/<id>/<name>` carries its name and GitHub validates it; `/assets/<uuid>` carries none, so the extension is taken from the presigned path. `filepath.Base` at the join keeps a traversal-shaped name inside the destination.
- **Timeouts mirror the upload side:** 30s for the header-only resolve leg, 120s for the transfer, matching `s3.go`.

### 8. CLI Entrypoint (`main.go`)

`main()` is a one-line entrypoint that delegates to a testable
`run(args []string, stdout, stderr io.Writer, deps) int`: it returns an exit code
Expand All @@ -218,6 +249,7 @@ Responsibilities:
- **Subcommand dispatch** for `extract-token` and `check-token`, with validation that disallowed flag combinations are rejected before any work is done.
- **Session resolution** via `resolveSessionCookie`, which applies the flag → env → browser precedence and wraps raw token values into a properly scoped `*http.Cookie`.
- **Multi-file upload loop**: each positional path is uploaded independently. A failure on one file is reported to stderr and the loop continues; the process exits non-zero if any upload failed.
- **Download loop**: same contract for URLs. All flag validation happens before any request, so a bad combination fails immediately. `--output -` streams to `run`'s injected stdout writer rather than `os.Stdout`, keeping the path testable.

## Data Flow

Expand Down Expand Up @@ -260,8 +292,13 @@ flowchart TD
| Step 2 (S3 upload) | None | Presigned policy from step 1 |
| Repo ID lookup | OAuth token | `gh` CLI (via `gh auth`) |
| `check-token` validation | Same `user_session` pair | Same precedence as upload |
| Download resolve leg (fast) | `Authorization: Bearer` | `gh auth token`, skipped when a session token is named |
| Download resolve leg (fallback) | `user_session` + `__Host-user_session_same_site` cookies | `--token` flag, `GH_SESSION_TOKEN`, or browser cookie DB |
| Download fetch leg | **None** | Presigned URL from the resolve leg |

Download uses the same two routes as upload, for the same reason: the `gh` token avoids the browser entirely when it works. The routing is simpler here, though. Upload has to remember rejections *per content type*, because the bearer upload endpoint accepts a narrow set; download has no such split — one credential reaches every attachment — so a single rejection turns the fast route off for the rest of the run.

The session-cookie path and the `gh` CLI auth path are independent. The cookie provides a browser-equivalent session for the undocumented upload API, while `gh` handles the standard REST API used only for looking up the numeric repository ID.
The session-cookie path and the `gh` CLI auth path are independent. The cookie provides a browser-equivalent session for the undocumented attachment APIs, while `gh` handles the standard REST API used only for looking up the numeric repository ID.

## Distribution

Expand Down
144 changes: 144 additions & 0 deletions documentation/github-attachment-download-flow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
# GitHub Attachment Download Flow

## Overview

Fetching a `user-attachments` asset is a two-leg operation. A `GET` on the
attachment URL answers with a `302` to a presigned storage URL; the bytes come
from that second URL. The legs have **opposite** credential requirements, which
is the single most important thing to know about this protocol.

Companion to [github-image-upload-flow.md](github-image-upload-flow.md).

## The two URL shapes

GitHub routes uploads to one of two shapes depending on the file type, and they
behave differently on the way back out.

| | `/user-attachments/assets/<uuid>` | `/user-attachments/files/<id>/<name>` |
|---|---|---|
| Used for | images and videos | PDF, zip, log, txt — everything else |
| Filename in the URL | **no**, only a uuid | **yes** |
| Presigned host | `github-production-user-asset-*.s3.amazonaws.com` | `objects.githubusercontent.com` |
| `response-content-disposition` on the redirect | no | yes |

Both redirect targets carry `X-Amz-Expires=300` and `response-content-type`.

## The Flow

### Step 1: Resolve

Either credential works:

```
GET https://github.com/user-attachments/assets/<uuid>
Authorization: Bearer <gh auth token>
User-Agent: …
```

```
GET https://github.com/user-attachments/assets/<uuid>
Cookie: user_session=…; __Host-user_session_same_site=…
User-Agent: …
```

The bearer token is the fast path — it needs no browser and no keychain access.
Verified against a private repository, for an asset uploaded by a different
user, so the grant follows repository read permission rather than uploader
identity. Both URL shapes work, unlike the bearer *upload* endpoint, which
accepts only a narrow set of content types.

On the cookie route both cookies are required. Sending only `user_session` looks
correct and fails to authenticate — the same requirement the browser-session
upload endpoints have.

The response is a `302`. **Do not follow it automatically**; classify it:

| Redirect target | Meaning |
|---|---|
| `/login`, resolved host exactly `github.com` | the session token is invalid or expired |
| absolute URL carrying `X-Amz-Signature` | the asset — proceed to step 2 |
| anything else | refuse; do not fetch |

Order matters. Checking for the signature first turns an expired session into a
confusing "unusable target" error. The host check on `/login` matters too:
matching by path alone would let `https://attacker.example/login` be read as
"your credential is stale", which on a tool that reads a browser cookie store is
a prompt an attacker should not be able to trigger.

The third row is what prevents an SSO interstitial or error page from flowing
into step 2, returning `200` with a consistent `Content-Length`, and landing on
disk as a perfectly plausible attachment.

### Step 2: Fetch

```
GET <presigned URL>
User-Agent: …
```

**No credentials of any kind.** The presigned URL is its own capability.

An `Authorization` header here returns **400** on the S3 bucket — AWS rejects
two auth mechanisms on one request. `objects.githubusercontent.com` tolerates
it, so the mistake is invisible on half the assets and easy to ship.

Verify the byte count against `Content-Length` when present, so a truncated
transfer never passes for a complete file.

## Filenames

Never taken from a response header:

| URL shape | Name from |
|---|---|
| `/files/<id>/<name>` | the `<name>` segment of the URL |
| `/assets/<uuid>` | the uuid plus the extension of the presigned path |

The `/files/` name is server-validated — a valid file id with a tampered
filename returns `404` — so the URL is a trustworthy source. An `/assets/` URL
carries no name at all, so the extension has to come out of the redirect, which
is why `curl -LO` on one produces an extensionless uuid.

GitHub sanitizes names at upload time: `paren (1) test.txt` is stored as
`paren.1.test.txt`.

## Authentication Summary

| Leg | Auth |
|---|---|
| Resolve (github.com) | `Authorization: Bearer` **or** `user_session` + `__Host-user_session_same_site` |
| Fetch (presigned) | none |

A `404` cannot distinguish "no such asset" from "this credential cannot read
it", so a client holding both credentials should try the second before
reporting failure.

## Observed Responses

| Condition | Result |
|---|---|
| anonymous, private asset | `404` |
| valid session, private asset | `302` to presigned |
| valid bearer token, private asset | `302` to presigned |
| invalid bearer token | `404` |
| stale session, `/assets/` | `302` to `/login` |
| stale session, `/files/` | `404` |
| well-formed but nonexistent uuid | `404` |
| nonexistent file id | `404` |
| valid file id, tampered filename | `404` |
| stray `Authorization` on the presigned leg | `400` (S3), tolerated (objects host) |
| `HEAD` on the presigned leg | `403` (S3) — signed for `GET` only |
| 30 sequential authenticated resolves | `302` every time; no `429`, no rate-limit headers |

A `404` conflates "no such asset" with "exists but you cannot read it", so error
messages have to name both.

## Caveats

- The endpoints are **undocumented** and may change without notice.
- `HEAD` is not usable for metadata on the S3 host, so asset size cannot be
probed without fetching.
- Presigned URLs expire; treat one as valid only for the request that
immediately follows its resolve.
- `curl -O` on an attachment URL writes **0 bytes and exits 0** — without `-L`
it saves the empty redirect body and reports success.
Loading