This guide covers the product-level operations around the runnable Diamond
Forge example. Read examples/git-browser/README.md first for initial
PostgreSQL, storage, OAuth, and server setup.
Diamond owns accounts, authorization, repository metadata, job scheduling,
artifact metadata, Pages deployments, and custom-domain mappings. It delegates
Git semantics to the installed git executable, isolated job execution to the
configured systemd or Bubblewrap backend, PostgreSQL durability to PostgreSQL,
and public TLS and DNS enforcement to the deployment edge.
Run these as distinct supervised services:
- two or more
diamond serveweb nodes behind one HTTPS reverse proxy; - one
project-pages-serveprocess on a cookie-free Pages origin; - a pool of repeatedly invoked
project-delivery-workworkers; - a pool of repeatedly invoked
project-ops-workworkers; - scheduled
project-git-reconcileruns; and - PostgreSQL with tested continuous archiving and point-in-time recovery.
All nodes use the same PostgreSQL database and declared shared-object, Git,
delivery, and operations stores. Store directories must be canonical, owned by
the service account, mode 0700, and unavailable to unrelated services. Never
place two writable filesystem stores behind one binding: each repository has
exactly one current shard placement and promotion fences the old generation.
Before admitting traffic, run:
diamond check examples/git-browser/diamond.toml
diamond project-deploy-check examples/git-browser/diamond.toml \
--postgres-env DIAMOND_POSTGRES \
--runtime-env DIAMOND_RUNTIME \
--origin https://forge.example.com \
--identity-mailbox /srv/diamond/identity-mailCollect structured service logs and alert on readiness, stale heartbeats, queue age, failed jobs, Git reconciliation errors, storage integrity errors, and recovery-catalog failures. Drain a web node from Scale & recovery before replacing it.
Google and GitHub callbacks are:
https://forge.example.com/_diamond/oauth/google/callback
https://forge.example.com/_diamond/oauth/github/callback
Keep provider secrets in the environment variables named by DIAMOND_RUNTIME;
do not put secrets in diamond.toml. Keep the OAuth transaction key independent
from either provider secret and use at least 32 random characters.
The example supports four installation policies:
public: anyone may create a local account or enter through an enabled OAuth provider;restricted: admit configured email domains, Google tenants, GitHub organizations, or exact provider identities;invite: require a currently valid single-purpose invitation; andclosed: only administrators provision users.
Choose the narrowest policy that fits the installation. Provider admission and local registration are separate decisions. A provider-authenticated address is not sufficient for a company installation unless the tenant, organization, domain, or identity rule also matches.
Diamond does not invent an archive format. Export all hosted Git refs with a mirror clone:
git clone --mirror \
https://diamond:SOURCE_CREDENTIAL@forge.example.com/_diamond/git/repos/SOURCE_REPOSITORY.git \
repository-export.git
git -C repository-export.git fsck --strict
git -C repository-export.git show-refThe adapter accepts branches, tags, and Git notes. A mirror clone preserves
annotated tag objects and refs/notes/*, as well as Unicode path names and file
bytes. LFS content is stored outside the bare repository, so export it with a
Git LFS client when the repository uses LFS:
git -C repository-export.git lfs fetch --allTo import, create and initialize the destination repository in Diamond, mint a write credential scoped to it, and push the mirror:
git -C repository-export.git push --mirror \
https://diamond:DESTINATION_CREDENTIAL@forge.example.com/_diamond/git/repos/DESTINATION_REPOSITORY.gitProtected-ref policy remains authoritative during import. After an import,
clone the destination into a new directory, run git fsck --strict, and compare
git show-ref output before retiring the source.
Each delivery binding declares between one and eight named executor profiles
and must include standard:
[delivery.bindings.builds.executors.standard]
memory_bytes = 536870912
max_processes = 128
max_file_bytes = 67108864
[delivery.bindings.builds.executors.small]
memory_bytes = 134217728
max_processes = 32
max_file_bytes = 16777216A repository selects only one declared name in .diamond-ci.json:
{
"version": 1,
"jobs": [
{
"name": "test",
"executor": "small",
"script": "./ci/test.sh",
"timeout_seconds": 300
}
]
}Unknown profiles reject the pipeline before a runner starts. The runtime applies the profile's memory, process, file-size, and wall-time limits to the configured isolation backend. Isolation failure still fails closed. Public, untrusted jobs require a disposable VM or equivalently reviewed sandbox outside Diamond's process.
Generic artifact downloads support HEAD, one HTTP byte range, ETag, and
If-None-Match. Multiple ranges are intentionally unsupported. A repository
writer may delete artifact metadata from CI & delivery; authorization is
rechecked before bytes stream, so the download route is fenced immediately.
Physical content-addressed bytes are removed later by held, grace-bounded
garbage collection.
Pages deployments are immutable. Activating an older retained deployment is an atomic rollback. Stopping an environment changes its durable state without deleting its deployment history or artifacts.
Only a Diamond account administrator may add or remove a custom-domain mapping.
Diamond accepts canonical lowercase DNS hosts, routes the exact Host to the
selected site, and stops routing it immediately after removal. A mapping is not
proof of DNS ownership and does not issue a certificate.
Use this deployment sequence:
- Verify control of the requested domain outside Diamond.
- Configure the reverse proxy or CDN to route that host to the separate Pages listener.
- Obtain and renew its certificate through the edge's maintained ACME client.
- Add the mapping from CI & delivery as a Diamond administrator.
- Verify HTTPS, the expected site, isolation headers, and the absence of Diamond session cookies before publishing DNS broadly.
The edge must reject unknown hosts and must not route control-plane session cookies to Pages. Removing a mapping in Diamond does not remove public DNS or a certificate; retire those separately at the edge.
Each user controls notifications per repository. They may disable a repository or request unread-only results. Preferences affect the user's web feed; they do not change authorization or reveal an event the user can no longer read.
Global search normalizes surrounding whitespace, treats punctuation literally, and ranks exact titles and title terms above body-only matches. Results are reauthorized against current repository policy when read, so revocation takes effect without waiting for reindexing. Search indexes are rebuildable projections; PostgreSQL and Git remain authoritative.
Create an encrypted snapshot into a new private directory:
diamond project-backup examples/git-browser/diamond.toml \
--postgres-env DIAMOND_POSTGRES \
--runtime-env DIAMOND_RUNTIME \
--out /srv/diamond/backups/2026-10-23 \
--backup-key-env DIAMOND_BACKUP_KEYThe snapshot covers PostgreSQL and every declared shared-object, Git, delivery, and operations root. CI workspaces and rebuildable search projections are not recovery authorities. Copy encrypted backup artifacts off-host under a separate credential and retention policy.
Test restoration only into an empty database and empty canonical roots:
diamond project-restore examples/git-browser/diamond.toml \
--postgres-env DIAMOND_RESTORE_POSTGRES \
--runtime-env DIAMOND_RESTORE_RUNTIME \
--from /srv/diamond/backups/2026-10-23 \
--backup-key-env DIAMOND_BACKUP_KEYRestore verifies ciphertext and per-file hashes before import, repairs root-relative Git hook paths, revokes restored sessions and API credentials, and validates routed Git access. Keep restored services isolated until independent counts, refs, objects, permissions, artifacts, Pages output, and audit continuity match the recovery plan.
When a repository update has an uncertain outcome, do not blindly replay it. Inspect the durable push event and run:
diamond project-git-reconcile examples/git-browser/diamond.toml \
--postgres-env DIAMOND_POSTGRES --runtime-env DIAMOND_RUNTIMEFor an uncertain job outcome, inspect immutable pipeline and fencing state before retrying. Revoking an account, repository grant, API credential, deploy key, or SSH certificate principal takes precedence over cached UI state.
Before upgrades or storage work:
- record a fresh encrypted backup and pass an isolated restore;
- drain affected web or worker nodes;
- confirm the current repository placement and generation;
- preserve the event spool and all declared roots;
- apply the change to one node or shard;
- replay health, clone/fetch, push policy, artifact, Pages, and job checks; and
- return capacity only after readiness and reconciliation are clean.
The retained qualification commands and bounded coverage are documented in
verification/git_*_results.md. Passing them is evidence for the tested
profile, not a substitute for validating an installation's reverse proxy, DNS,
TLS, PostgreSQL, runner images, operating system, and backup custody.