Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Agent Guide: quay-builder

## Purpose

Single-job Quay build worker. It connects to Quay build manager over gRPC, fetches build context, builds with Podman or Docker, pushes the image, reports logs/status, and exits.

## Start Here

- Entrypoint: `cmd/quay-builder/main.go`
- Context fetch: `buildctx/`
- Build execution: `buildpack/`
- Runtime abstraction: `containerclient/`
- Quay gRPC client: `rpc/`
- Protocol bindings: `buildman_pb/`

## Common Tasks

- Add context source: update `buildctx/`, request parsing, and tests.
- Debug build failure: check context fetch, Dockerfile parse, runtime logs, gRPC phase/log updates.
- Change runtime behavior: use `containerclient/`; do not call Podman/Docker directly from unrelated packages.

## Commands

```bash
make test
make build
make build-ubi8
```

## Guardrails

- Keep build directories isolated and cleaned up.
- Do not leak registry credentials or build args into logs.
- Resource limits are enforced outside the worker; avoid bypassing them in code.
- `quay/quay` owns the build manager server under `buildman/`.
139 changes: 139 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# quay-builder Architecture

## Purpose

Automated container build worker for Quay using Podman/Docker.

```mermaid
flowchart LR
quay[Quay build manager]
grpc[gRPC job and log stream]
builder[quay-builder]
context[buildctx source fetch]
runtime[Podman or Docker]
registry[Quay registry]

quay <--> grpc <--> builder
builder --> context
builder --> runtime
runtime --> registry
builder --> quay
```

## High-Level Design

```
Quay (build trigger)
↓ gRPC
quay-builder
↓ Build execution
Podman/Docker
↓ Image push
Quay registry
```

## Components

### `/cmd/quay-builder`
Worker entrypoint:
- Connects to Quay via gRPC
- Polls for build jobs
- Reports build status

### `/buildctx`
Build context fetchers:
- **Git**: Clone from GitHub/GitLab/Bitbucket
- **Tarball**: Download and extract
- **Inline**: Dockerfile from request body

### `/buildpack`
Build execution:
- Dockerfile parsing
- Multi-stage build handling
- Build arg injection
- Layer caching

### `/containerclient`
Container runtime abstraction:
- Podman client
- Docker client
- Runtime auto-detection

### `/rpc`
gRPC client for Quay:
- Job polling
- Build log streaming
- Status updates
- Image push coordination

## Build Flow

```
1. Quay → quay-builder: BuildRequest{repo, dockerfile_url, context}
2. quay-builder → buildctx: Fetch context (git clone / tar download)
3. quay-builder → buildpack: Parse Dockerfile
4. buildpack → Podman: Execute build
5. Podman → buildpack: Stream logs
6. quay-builder → Quay: Stream build logs (gRPC)
7. Podman: Build complete, tag image
8. quay-builder → Quay registry: Push image
9. quay-builder → Quay: BuildComplete{image_id, digest}
```

## Build Context Handling

### Git Context
```
1. Clone repo to /tmp/build-<uuid>/
2. Checkout commit/branch/tag
3. Apply submodules (if .gitmodules exists)
4. Build from repo root (or dockerfile_path)
```

### Tarball Context
```
1. Download tarball
2. Extract to /tmp/build-<uuid>/
3. Build from extracted root
```

### Inline Context
```
1. Write Dockerfile to /tmp/build-<uuid>/Dockerfile
2. Build with no context (FROM only)
```

## Configuration

```yaml
worker:
name: builder-01
concurrency: 4

quay:
endpoint: grpc://quay.example.com:50051
token: ${QUAY_TOKEN}

runtime:
type: podman # or docker
socket: unix:///var/run/podman/podman.sock

build:
cache_dir: /var/cache/quay-builder
max_context_size_mb: 2048
timeout: 3600
```

## Performance

- Parallel builds (configurable concurrency)
- Layer caching (shared across builds)
- Build context caching (git repos)
- Cleanup after build (temp dirs, dangling images)

## Security

- Isolated build contexts (separate temp dirs)
- No arbitrary command execution
- Dockerfile validation before build
- Resource limits (CPU, memory, timeout)
53 changes: 53 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Contributing to quay-builder

## Setup

```bash
# Requires podman or docker
make build

# Run worker locally
./bin/quay-builder --config worker.yaml
```

## Development

Build worker that:
- Pulls Dockerfiles from Quay build triggers
- Executes builds via Podman/Docker
- Pushes results back to Quay

## Testing

```bash
# Unit tests
make test

# Integration test (requires Quay instance)
./test/integration.sh

# Local build test
./bin/quay-builder build --dockerfile testdata/Dockerfile
```

## Build Context

Supports multiple context sources:
- Git (GitHub, GitLab, Bitbucket)
- Tarball upload
- Inline Dockerfile

## Pull Requests

- Test with both Podman and Docker
- Update buildpack logic tests
- Security: validate Dockerfile sources, no arbitrary command injection
- Performance: benchmark large builds (>1GB context)

## Code Structure

- `cmd/quay-builder/` - worker entrypoint
- `buildctx/` - context fetchers (git, tar, http)
- `buildpack/` - Docker/Podman build execution
- `containerclient/` - container runtime abstraction
- `rpc/` - gRPC API to Quay
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,23 @@ This generates image with tag `quay.io/projectquay/quay-builder:latest-alpine`.
make build-centos
```
This generates image with tag `quay.io/projectquay/quay-builder:latest-centos`.

## Contextification Addendum

```mermaid
flowchart LR
quay[Quay build manager]
builder[quay-builder]
context[buildctx]
runtime[Podman or Docker]
registry[Quay registry]

quay <--> builder
builder --> context
builder --> runtime
runtime --> registry
```

Key paths: `cmd/quay-builder/main.go`, `buildctx/`, `buildpack/`, `containerclient/`, `rpc/`, and `buildman_pb/`. Use `make test`, `make build`, and `make build-ubi8`.

Keep build directories isolated and cleaned up. Do not log registry credentials or build args.
Loading