Skip to content

registry.md

Latest commit

 

History

History
393 lines (277 loc) · 22.9 KB

registry.md

File metadata and controls

393 lines (277 loc) · 22.9 KB
Raw

Image Registry — Template, Push, Browse, Delete

Introduced: v7.5.0 (template + push + browse), v7.6.0 (delete) Works in standalone and HA mode.

Docker Dash provides a complete workflow for managing OCI image registries: deploy your own private registry from a one-click template, push local images to any configured registry with live progress, browse registry contents, and delete tags safely.

This document is the unified guide. Per-feature implementation notes live in the CHANGELOG.md entries for the respective releases.

1. The four pieces

Piece Where in the UI Introduced Who can use it
Template "Private Registry (Distribution)" Templates → DevOps category v7.5.0 admin (deploy any template)
Push to Registry action on every image row Images → action button (cloud icon) or right-click menu v7.5.0 admin + operator
Registry Browser page Sidebar → Registries (admin-only) v7.5.0 admin
Delete tag button Registry Browser → manifest panel v7.6.0 admin (with two-step confirmation)

The credential store (Settings → Registries) is older and predates v7.5.0 — credentials are encrypted at rest with AES-GCM (legacy XOR/base64 entries are auto-migrated on first read).

2. Quick start — your own private registry

Five minutes from "no registry" to "I just pushed an image to it":

# 1. Deploy the template (Templates → DevOps → Private Registry (Distribution))
#    The template ships a single container + named volume + htpasswd auth.
#    After deploy, the registry runs at http://<host>:5000 — but auth is
#    not yet configured (no users in the htpasswd file).

# 2. Generate a user (on the host, NOT inside Docker Dash):
mkdir -p ./auth
docker run --rm --entrypoint htpasswd httpd:2 -Bbn alice s3cret > ./auth/htpasswd

# 3. The template volume bind ./auth into /auth:ro, so re-create the container
#    to pick up the htpasswd file. From the Docker Dash UI:
#    Stacks → docker-registry → Recreate
#    (or: docker compose restart docker-registry on the host)

# 4. In the Docker Dash UI:
#    Settings → Registries → New
#    Name:  My Private Registry
#    URL:   http://<host>:5000
#    Username: alice
#    Password: s3cret
#    → Save. Click Test Connection to verify.

# 5. Push your first image:
#    Images → pick an image → click the cloud-upload icon (or right-click → Push to Registry)
#    Pick the registry, repo (auto-filled from source), tag → Push.
#    Live progress per layer streams into the modal.

# 6. Browse what you pushed:
#    Sidebar → Registries → pick the registry → click a repo → click a tag.
#    Manifest panel shows digest, layer breakdown, size.

That's the whole loop. Everything else in this doc is detail.

3. Why Distribution (and not Harbor)?

We deliberately ship a Distribution template, not Harbor. Quick comparison:

Distribution (registry:3) Harbor
Containers 1 9–11 (Postgres + Redis + Trivy + Notary + nginx + jobservice + ...)
RAM idle ~30 MB 4–8 GB
Compose file Static (one we ship) Dynamic (generated by Harbor's install.sh)
OCI v2 API ✅ ✅ (uses Distribution under the hood)
UI None Yes (good)
RBAC, scanning, replication, signing None Yes

If you want enterprise features (RBAC, vulnerability scanning, replication, signing), deploy Harbor via its official installer and add it as a Registry credential in Docker Dash. The push, browse, and delete features all work against Harbor identically — same UX from our end. We don't bundle Harbor because tracking its installer + 11-container topology is a maintenance commitment we don't want to take on.

4. Push action — what really happens

Trigger:

  • Inline button on each image row (cloud-upload icon, accent color)
  • Right-click → Push to Registry in the context menu

Modal flow:

  1. Lists all configured registries (read-only — manage them in Settings).
  2. Pre-fills target repo from source image name (last segment of repo/image:tag → image).
  3. Pre-fills target tag from source tag (or latest if source has no tag).
  4. Shows a live preview: <registry-host>/<repo>:<tag>.
  5. Submit → Server-Sent Events stream from POST /api/registries/:id/push:
    • Each layer renders one row with status + percentage, updated in place.
    • "Layer already exists" rows surface in green (registry deduplication).
    • On error anywhere in the stream, modal switches to a red status with the message.
  6. On success: toast notification + audit log entry.

Backend implementation (src/services/registry.js):

// 1. Tag the local image under the registry host
await docker.getImage(sourceImage).tag({
  repo: `${registryHost}/${targetRepo}`,
  tag: targetTag,
});

// 2. Push the newly-tagged image
const stream = await docker.getImage(fullImage).push({
  authconfig: {
    username, password, serveraddress: reg.url,  // dockerode encodes this as base64-JSON for X-Registry-Auth
  },
});

// 3. Forward NDJSON events to SSE
docker.modem.followProgress(stream, onDone, onProgress);

RBAC

Role Can push
viewer No
operator Yes
admin Yes

Operators legitimately deploy app images (it's their job). Viewers cannot mutate registry state.

Audit log

  • registry_push on success — { registry, sourceImage, targetRepo, targetTag, durationMs }
  • registry_push_failed on either init failure (tag/auth/network) or stream failure (registry-side error) — same fields plus error (truncated to 300 chars)

Limitations

  • Multi-arch manifest lists are NOT supported. Dockerode pushes whatever the local engine has tagged — typically a single platform. Multi-arch images need docker buildx imagetools push or skopeo. The push modal includes a yellow info note so users aren't surprised.
  • Insecure registries (HTTP) require the host's Docker daemon to allow them — set "insecure-registries": ["host:5000"] in /etc/docker/daemon.json and restart Docker. We surface a clear error from the push stream when the daemon refuses.
  • Long-lived tokens only. GHCR/ECR/GCR use OAuth-issued tokens that expire after 12h. Re-enter the token in Settings → Registries when expired. (A token-refresh flow is on the roadmap if there's demand.)

5. Browse action — Registry Browser page

Sidebar → Registries (admin-only, separate from Settings → Registries which manages credentials).

Two-pane layout:

  • Left: repository list with client-side filter (good for ≤ a few hundred repos)
  • Right: tag list + manifest inspector

When you click a tag, the manifest panel shows:

  • Digest (sha256:...)
  • Content type (application/vnd.docker.distribution.manifest.v2+json or application/vnd.oci.image.index.v1+json for multi-arch)
  • Schema version, layer count + total size
  • Collapsible per-layer breakdown
  • For multi-arch images: per-platform manifests with <os>/<arch>/<variant> + per-arch digest + size
  • Copy pull command button → docker pull <host>/<repo>:<tag> to clipboard

Last-selected registry persists in sessionStorage so re-visiting the page lands on the same context.

Backend endpoints (admin-required for all)

Endpoint What it does
GET /api/registries/:id/catalog List all repositories (Distribution V2 /v2/_catalog)
GET /api/registries/:id/tags/*repo List tags for one repo (/v2/<repo>/tags/list)
GET /api/registries/:id/manifest/*ref Inspect a manifest (/v2/<repo>/manifests/<ref>) — returns raw manifest + digest from headers

These have existed since v7.5.0; the manifest endpoint is new in v7.5.0.

6. Delete tag — what's safe and what's not

Introduced in v7.6.0 — closes the explicit v7.6 commitment from v7.5.0.

The Distribution V2 API only supports delete by digest, not by tag. We hide that detail from the UI: the Browse page Delete button takes a tag, and the backend resolves tag → digest first.

UI flow:

  1. Click Delete this tag in the manifest panel (admin-only — the button is hidden for operators + viewers).
  2. Confirmation modal:
    • Shows the full repo:tag + the resolved digest.
    • Warns that anyone (or any CI job) currently pulling this tag will fail.
    • Two-step gate: type the full repo:tag string into a text field. The Delete button stays disabled until the input matches exactly.
  3. Submit → DELETE /api/registries/:id/tag/<repo>:<tag> → backend HEADs the manifest to resolve digest, then DELETEs by digest.
  4. On success: toast + tag list refreshes (deleted tag is gone, manifest panel hides).

Audit log

  • registry_tag_delete on success — { repo, tag, digest }
  • registry_tag_delete_failed — { error }

Idempotent

A 404 from the delete (manifest already gone) is treated as success. Safe to retry.

Failure mode you'll likely hit first

"Registry has deletion disabled. Set REGISTRY_STORAGE_DELETE_ENABLED=true and restart it."

Distribution returns HTTP 405 or 501 for delete when this env var is unset. Our shipped template sets it to true (so delete works out of the box), but third-party Distribution deployments often have it off. Fix: set the env var, restart the registry container.

Garbage collection (operator responsibility)

Deleting a tag's manifest does NOT reclaim the disk space used by its layer blobs. The blobs remain because they may be shared with other tags. To actually free disk:

# Stop the registry (or put it in read-only mode) — unsafe to GC while serving writes
docker exec docker-dash-registry registry garbage-collect /etc/distribution/config.yml

We deliberately don't expose a "Run GC" button in the UI:

  • Running GC while the registry is accepting writes can corrupt data.
  • Putting the registry in read-only mode requires either restarting with a different config or a kill -SIGUSR2-style toggle that Distribution doesn't have.
  • This is an operator decision tied to a maintenance window — UI button would be a footgun.

The Delete modal includes an info note pointing operators at this command.

7. Programmatic API (non-UI usage)

Everything the UI does is available as REST. Useful for CI scripts that want to push from outside Docker Dash but reuse our credential store.

# Push (requires admin or operator role, returns SSE stream)
curl -s -X POST http://docker-dash:8101/api/registries/1/push \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"sourceImage":"myapp:latest","targetRepo":"team/myapp","targetTag":"v1.2.3"}' \
  -N

# Catalog
curl -s -H "Authorization: Bearer $TOKEN" \
  http://docker-dash:8101/api/registries/1/catalog

# Tags
curl -s -H "Authorization: Bearer $TOKEN" \
  http://docker-dash:8101/api/registries/1/tags/team/myapp

# Manifest inspect
curl -s -H "Authorization: Bearer $TOKEN" \
  http://docker-dash:8101/api/registries/1/manifest/team/myapp:v1.2.3

# Delete tag (admin only — two-step gate is UI-only; the API delete is direct)
curl -s -X DELETE -H "Authorization: Bearer $TOKEN" \
  http://docker-dash:8101/api/registries/1/tag/team/myapp:v1.2.3

$TOKEN is the Bearer token returned by POST /api/auth/login. Same auth as any other Docker Dash API.

8. Troubleshooting

Symptom Cause Fix
Push modal shows "Network error" immediately Registry URL unreachable from inside the Docker Dash container Check the registry's hostname resolves from the Docker Dash container; for HTTP registries on the same host, use the container name (not localhost) on a shared network
Push streams "received unexpected HTTP status: 401 Unauthorized" Wrong credentials Settings → Registries → edit → re-enter password → Test Connection
Push streams "x509: certificate signed by unknown authority" Self-signed TLS cert on the registry Either switch to HTTP + add to daemon insecure-registries, or import the registry's CA into the host's trust store
Push streams "no basic auth credentials" Image was tagged for the wrong host (or no host at all) The push modal handles this for you; if you tag manually with docker tag, use the registry's exact host:port
Browse: "404 Not Found" on catalog Registry URL incorrect (e.g. trailing slash, wrong port) Settings → Registries → edit URL
Browse: catalog returns empty even though images exist Some registries (Harbor, ECR) require per-project tokens; the global credential doesn't have list-repos scope Use a token with catalog read scope (Harbor: Project Admin or above)
Delete: "Registry has deletion disabled" REGISTRY_STORAGE_DELETE_ENABLED is false Set it to true on the registry, restart the container
Disk usage doesn't drop after deleting tags Distribution doesn't auto-GC Run registry garbage-collect manually (see §6)

9. Build Provenance Panel (v8.1.0)

A read-only panel that surfaces what an image's manifest already tells you — without requiring you to learn oras inspect syntax.

Where it lives: Registry Browser → pick a tag → manifest inspect modal → Build Provenance collapsible card (collapsed by default; click to expand).

What it surfaces (parses OCI annotations from the manifest's manifests[].annotations and the config's annotations):

Field Annotation key UI behavior
Source repository org.opencontainers.image.source Linkified for github.com / gitlab.com / bitbucket.org / codeberg.org / gitea.com (other hosts shown as plain text)
Commit SHA org.opencontainers.image.revision Truncated 8 chars, full SHA in tooltip; combined with source URL → linkified to <source>/commit/<sha>
Authors org.opencontainers.image.authors Plain text
License org.opencontainers.image.licenses Plain text (SPDX identifier expected)
Vendor org.opencontainers.image.vendor Plain text
Version label org.opencontainers.image.version Plain text
Base image org.opencontainers.image.base.name + .base.digest Plain text + truncated digest
Cosign signature presence Layer with application/vnd.dev.cosign.simplesigning.v1+json mediaType, OR sibling :sha256-XXX.sig tag in same repo "🔒 Signed" badge — presence only, NOT cryptographic verification

What the panel does NOT do:

  • Run cosign verify against the signature bytes — that needs the cosign binary, key management UX, and trust policy. Deferred to a future v8.x.
  • Fetch the linked GitHub/GitLab commit metadata. The link is one click away; we don't proxy.
  • Validate that base.digest is a digest you trust. We surface what the build wrote.

Show-all toggle: an expander labeled "Show all annotations" lists every key/value pair from manifests[].annotations and config.annotations — useful for power users who care about a non-canonical key (com.acme.build-id, etc.).

Backend: src/services/registry-provenance.js — pure function parse(manifestData) → { hasProvenance, known, other, otherCount, totalAnnotations }. 15-case test corpus covers all 5 supported source hosts, signature presence/absence, missing annotations, and edge cases (empty manifest, malformed annotations).

10. Retention Policies with Dry-Run (v8.1.0)

Per-repo cleanup rules. Five safety layers stacked because "I just deleted production by accident" is the dominant operator fear with retention features.

Where it lives: Registry Browser → pick a repo → Repository Settings expander (collapsed by default) → Retention Policy section.

The five safety layers:

  1. Default disabled (dry-run only). A new policy is created in dry-run mode. Clicking Preview shows what would be deleted; clicking Apply is a separate, deliberate action that requires the operator to flip the enabled toggle.
  2. Hard floor: minimum 3 tags kept. Even if the rule says "keep 1", the evaluator floors at 3. Rationale: defense against off-by-one errors and YAML typos that wipe out a repo.
  3. Default protected patterns. New policies pre-fill the protect-glob list with latest, v*, main, master, prod-*, stable. Operators can edit, but the defaults catch the common cases.
  4. Server-side cap of 200 deletions per run. If the evaluator wants to delete 500 tags, only 200 are processed; remainder is logged for the operator to investigate. Prevents a runaway evaluation from wiping a year of CI artifacts in one cron tick.
  5. Audit per delete. Every individual registry_tag_delete event is hash-chain-logged with the policy ID and the rule version that triggered it. If something does go wrong, you can prove what was configured at the time.

4 rule templates:

Template Rule shape Use case
Keep last N tags { keepLastN: 10 } Rolling release tags
Delete untagged older than X days { deleteUntaggedOlderThanDays: 30 } Build artifacts that lost their tag via a re-tag
Keep last N + delete untagged combination Most common — apply both rules
Custom JSON raw rule_json editor Power users with multi-clause rules

Cron: Daily at 17 3 * * * (off-:00 to avoid clashing with the daily DB backup). Leader-only in HA mode (registered via cluster.onBecomeLeader).

Backend: src/services/retention.js — pure function evaluate({tags, rule}) → {toDelete, toKeep, summary}. 27-case test corpus covers all 5 safety layers, glob pattern edge cases (v* matching v1/v1.2.3/v-rc1, NOT matching version), date arithmetic, and the empty-input case.

Migration: 063_registry_repos_and_retention.js creates retention_policies (1:1 with registry_repos, ON DELETE CASCADE both ways).

11. Remote/Virtual Repositories (v8.1.0)

JFrog-style local/remote/virtual repo taxonomy adapted to the OCI Distribution constraint of "one upstream per registry instance."

Three repo types (stored on registry_repos.type):

Type What it is Use case
local The default — repos hosted directly in your Distribution registry Your own pushed images
remote A proxy to an upstream public registry (Docker Hub, GHCR, Quay) — Distribution caches on first pull Survive Docker Hub rate limits + offline operation after first cache
virtual A logical group of local + remote repos served under one path Single pull URL that resolves transparently across upstreams

Where it lives: Registry Browser → pick a repo → Repository Settings expander → Repository Type section. Editor is a radio (Local / Remote / Virtual) with conditional fields:

  • Remote type reveals: upstream URL, upstream username (optional), upstream password (encrypted at rest).
  • Virtual type reveals: drag-and-drop list of member local and remote repos.

The "Private Registry + Cache" template (v8.1.0):

Templates → DevOps → Private Registry + Cache (4 containers). Ships:

  • 1× registry:3 for local repos (your pushes)
  • 1× registry:3 configured as a Docker Hub proxy via proxy: { remoteurl: 'https://registry-1.docker.io' }
  • 1× registry:3 configured as a GHCR proxy
  • 1× Caddy router with strip_prefix rules so /local/*, /dockerhub/*, /ghcr/* resolve to the right backend

This is the JFrog "virtual" pattern adapted to OCI Distribution's one-upstream-per-instance constraint. Each remote needs its own container (Distribution doesn't multiplex upstreams within one process), but Caddy hides the multi-container shape from operators behind a single hostname.

What this solves:

  • Docker Hub rate limit relief. First pull populates the cache; subsequent pulls hit your local copy. Anonymous Hub limit (100/6h) becomes "your team's first 100 unique pulls in 6 hours, ever."
  • Offline operation. After the cache is warm for the images you actually use, an air-gapped or transient-network host can still pull.
  • Single pull URL across upstreams. docker pull registry.internal/redis:7-alpine resolves to Hub via the proxy; docker pull registry.internal/myorg/myapp:v1.2.3 resolves to your local repo. Operators don't need to remember which upstream a tag came from.

Backend: src/services/registry.js gains listRepos(), upsertRepo(), deleteRepo(), resolveVirtual(). The _authConfigForRegistry() helper picks per-repo upstream credentials when present, falling back to the registry's default credential.

Migration: 063_registry_repos_and_retention.js creates registry_repos (registry_id + repo_path UNIQUE, type CHECK constraint, upstream_url + upstream_password_encrypted nullable for local).

Audit: 8 new audit actions — registry_repo_create, registry_repo_update, registry_repo_delete, retention_policy_create, retention_policy_update, retention_policy_delete, retention_dry_run, retention_executed.

12. What's NOT here (and why)

Feature Status Why
Multi-arch manifest list push Not supported Dockerode doesn't expose manifest-list APIs — needs docker buildx imagetools or skopeo
Auto garbage collection Not supported Risk of data loss without explicit operator confirmation + read-only mode
Pagination of the catalog UI Client-side filter only V2 catalog supports ?n=&last= — we use ?n=100. Add real pagination if you have > 100 repos and the filter isn't enough
Per-repo permissions Not in scope Our credential store is one (user, password) per registry. For per-project tokens, configure multiple Registry credentials in Settings
Pull-with-progress UI Not built (CLI handles this) Docker Dash already has Pull from Settings → Registries; that's an operator workflow, the rest of pull-management lives in Stacks/Templates
Image signing (Cosign / Notary) Not in scope Major feature on its own — if there's demand, file a discussion
Webhook receiver for registry events Not in scope Distribution can POST events on push/pull/delete; a webhook receiver is a different feature than a registry browser

These are deliberate decisions, not oversights. If any becomes important enough to file as an issue, we'll re-evaluate.

13. See also