Skip to content

FORUM-POST-FINAL.md

Latest commit

 

History

History
286 lines (185 loc) · 15.2 KB

FORUM-POST-FINAL.md

File metadata and controls

286 lines (185 loc) · 15.2 KB
Raw

Docassemble on Cloudron: Working Custom Package + Packaging Guide

We have successfully packaged docassemble as a custom Cloudron app. It is working: interviews run, documents generate, the Playground is functional, and data persists across restarts. The source is available at [your-repo-url] and we welcome testing, feedback, and contributions.

This post is structured for three audiences:

  1. Deployers who want to run docassemble on their Cloudron today
  2. Packagers who might want to make this an officially supported Cloudron app
  3. Cloudron maintainers who could improve the platform to make apps like this easier

What is docassemble and why would you want it?

Docassemble is a free, open-source platform for building guided interviews that generate documents. You define questions in YAML, write logic in Python, and it produces a mobile-friendly web application that walks users through a process step by step, then assembles customised PDFs, DOCX files, or other outputs.

It is the leading open-source tool in its niche. Courts, legal aid organisations, government agencies, and NGOs use it for everything from benefits applications to tenancy agreements to immigration forms. The commercial alternatives (HotDocs, Documate, Lawyaw) cost thousands per year and lock you into their cloud.

What makes it worth the packaging effort:

  • Genuinely smart conditional logic. Not just yes/no branching. The Python engine evaluates complex conditions, queries external APIs mid-interview, and adapts the flow dynamically.
  • Document assembly that handles real-world messiness. PDF form filling, DOCX generation from Jinja2 templates, touchscreen signatures, OCR, multilingual support.
  • Browser-based development. The Playground lets you write, test, and package interviews without any local tooling.
  • Self-hosted and MIT licensed. Your data stays on your server. No per-user fees. No SaaS dependency. Critical for legal, healthcare, and government contexts.

Part 1: Deploying docassemble on Cloudron (for users)

Prerequisites

  • A Cloudron instance with at least 12 GB total RAM (docassemble needs ~8 GB allocated to it)
  • Cloudron CLI installed and logged in
  • A domain or subdomain configured in Cloudron
  • About 45 minutes for the first deployment (most of this is waiting for builds and initialization)

Step 1: Get the package files (~2 minutes)

Clone the repository:

git clone [your-repo-url] docassemble-cloudron
cd docassemble-cloudron

Step 2: Install the app (~10-15 minutes)

cloudron install --location docassemble.yourdomain.com

The build takes 10-15 minutes. The Cloudron build service downloads the ~4 GB upstream Docker image, removes RabbitMQ, installs LavinMQ, and prepares seed directories.

Gotcha: Build timeout. If the build times out on the Cloudron build service, you can build locally instead:

docker build -t your-registry/docassemble-cloudron:0.1 .
docker push your-registry/docassemble-cloudron:0.1
cloudron install --image your-registry/docassemble-cloudron:0.1 --location docassemble.yourdomain.com

Step 3: Set the memory limit (~1 minute)

This step is critical. Immediately after installation, go to your Cloudron dashboard, find the docassemble app, open its settings, and set the memory limit to at least 8 GB (8192 MiB).

Gotcha: Manifest memory limit is ignored. Cloudron does not honour the memoryLimit field in CloudronManifest.json for custom (unpublished) apps. The default is 256 MB, which is not enough to load Python's import chain. You must set it manually in the dashboard. The app will restart automatically after you change it.

Step 4: Wait for first boot (~15-20 minutes)

First boot is slow. The app needs to:

  • Copy the ~2 GB Python virtual environment to persistent storage (~3-5 minutes)
  • Run database migrations and populate package tables (~2-3 minutes)
  • Generate nginx and application configuration (~1 minute)
  • Start all services (nginx, uWSGI, Celery, WebSockets) (~2-3 minutes)

You can monitor progress:

cloudron logs -f --app docassemble.yourdomain.com

Look for Finished initializing in the logs. After that, services will start one by one. When you see celery, nginx, and uwsgi all running, the app is ready.

Gotcha: "Not responding" during init. The Cloudron dashboard may show "not responding" for up to 20 minutes during first boot. This is normal. Do not restart the app during this time.

Step 5: Log in (~1 minute)

Visit https://docassemble.yourdomain.com. The default admin account is:

  • Email: admin@example.com
  • Password: password

Change this password immediately after logging in. Go to User List from the menu, click Edit next to the admin account, and change both the email and password.

Step 6: Verify it works (~5 minutes)

  1. Go to Playground from the menu
  2. Paste this test interview:
question: |
  Hello from Cloudron!
subquestion: |
  Docassemble is running at ${ url_of('root') }
mandatory: True
  1. Click Save and Run. If the interview loads, the core engine works.
  2. Go to Configuration from the menu and verify it shows your PostgreSQL and Redis connection details.
  3. Go to Logs and confirm entries are visible.

Subsequent boots (~2-3 minutes)

After the first boot, restarts are much faster because the database tables already exist and the virtual environment is already copied. Expect 2-3 minutes for the app to become responsive after a restart.

Part 2: How the package works (for packagers)

Docassemble is one of the most complex applications you can package for Cloudron. The upstream Docker image is a monolith running ~10 services under supervisord. Here is how we solved each major challenge.

Architecture

Cloudron reverse proxy (TLS)
  |
  v
nginx:8080 --> uWSGI (Flask/Python app)
                 |
                 +--> Cloudron PostgreSQL addon (external)
                 +--> Cloudron Redis addon (external)
                 +--> LavinMQ localhost:5672 (internal, replaces RabbitMQ)
                 +--> Celery workers (2 processes, 2 queues)
                 +--> WebSocket server (live help)
                 +--> cron (scheduled tasks)

Challenge 1: Read-only filesystem

Cloudron locks everything except /app/data, /tmp, and /run. Docassemble writes to approximately 25 different filesystem paths during initialization and normal operation.

Solution: Two categories of symlinks.

Persistent paths (survive restarts, backed up by Cloudron) are symlinked to /app/data/:

  • /usr/share/docassemble/config → /app/data/config
  • /usr/share/docassemble/files → /app/data/files
  • /usr/share/docassemble/log → /app/data/log
  • /usr/share/docassemble/backup → /app/data/backup
  • /usr/share/docassemble/certs → /app/data/certs
  • /usr/share/docassemble/cron → /app/data/cron
  • /usr/share/docassemble/local3.12 (Python venv) → /app/data/venv

Volatile paths (regenerated every boot) are symlinked to /run/:

  • /etc/nginx/sites-available and /etc/nginx/sites-enabled
  • /var/log/nginx, /var/log/supervisor, /var/log/apache2
  • /etc/ssl, /var/www, /var/cache/debconf
  • /etc/localtime, /etc/timezone, /etc/locale.gen, /etc/locale.conf
  • /var/lib/nginx, /var/lib/postgresql
  • /etc/cron.daily, /etc/hasbeeninitialized

The Dockerfile creates all symlinks at build time. The start.sh entrypoint recreates volatile directories and seeds them from saved copies on every boot.

Gotcha: Python's shutil.rmtree() refuses symlinks. Docassemble's install_certs.py calls shutil.rmtree() on /var/www/.certs and /etc/ssl/docassemble. If these are symlinks, Python raises OSError: Cannot call rmtree on a symbolic link. The fix is to symlink the entire parent directory (/var/www → /run/www, /etc/ssl → /run/ssl) and seed them as real directories from saved copies at boot time.

Gotcha: nginx temp directory. Nginx needs writable temp directories at /var/lib/nginx/body, /var/lib/nginx/proxy, etc. Symlink /var/lib/nginx → /run/nginx-lib and create the subdirectories in start.sh.

Challenge 2: Replacing RabbitMQ

Docassemble uses RabbitMQ as the Celery message broker. Cloudron has no RabbitMQ addon, and RabbitMQ uses ~200 MB of RAM at rest (Erlang overhead).

Solution: LavinMQ. LavinMQ is an AMQP 0.9.1 wire-compatible message broker written in Crystal. It is a drop-in replacement: same protocol, same broker URL format (amqp://guest:guest@localhost:5672), same Celery compatibility. Memory usage is ~40 MB at rest.

The Dockerfile removes RabbitMQ and Erlang, then installs LavinMQ from the official apt repository. The start.sh script starts LavinMQ before supervisord and waits for it to accept connections.

Installation in the Dockerfile:

RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
    apt-get remove -y rabbitmq-server erlang-base erlang-nox || true && \
    apt-get autoremove -y && \
    curl -fsSL https://packagecloud.io/cloudamqp/lavinmq/gpgkey | \
      gpg --dearmor -o /usr/share/keyrings/lavinmq.gpg && \
    echo "deb [signed-by=/usr/share/keyrings/lavinmq.gpg] https://packagecloud.io/cloudamqp/lavinmq/ubuntu noble main" \
      > /etc/apt/sources.list.d/lavinmq.list && \
    apt-get update && \
    apt-get install -y lavinmq && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

Challenge 3: External PostgreSQL and Redis

Cloudron provides managed PostgreSQL and Redis as addons. Docassemble supports external services via environment variables and config.yml.

Solution: Map Cloudron env vars to docassemble env vars in start.sh, then generate config.yml on every boot.

Key mappings:

  • CLOUDRON_POSTGRESQL_HOST → DBHOST
  • CLOUDRON_POSTGRESQL_DATABASE → DBNAME
  • CLOUDRON_POSTGRESQL_USERNAME → DBUSER
  • CLOUDRON_POSTGRESQL_PASSWORD → DBPASSWORD
  • CLOUDRON_REDIS_* → redis: directive in config.yml
  • CLOUDRON_APP_DOMAIN → DAHOSTNAME
  • BEHINDHTTPSLOADBALANCER=true (always)
  • PORT=8080 (matches manifest httpPort)

Gotcha: CONTAINERROLE must not be "all". Setting CONTAINERROLE=all tells initialize.sh to start internal PostgreSQL and Redis via supervisorctl, regardless of supervisor's autostart setting. Set CONTAINERROLE=web:celery:cron:log:mail to skip the sql, redis, and rabbitmq roles.

Gotcha: Cloudron PostgreSQL does not use SSL. Do not set ssl mode: require in the database configuration. Cloudron's internal PostgreSQL rejects SSL connections.

Gotcha: pg_hba.conf errors are non-fatal. Initialize.sh tries to connect to the postgres system database to check if the app database exists. Cloudron only allows connections to the app-specific database, so this check fails. It is harmless: the script falls through and connects to the correct database.

Challenge 4: Supervisor overrides

We need to disable services that Cloudron provides or that we have replaced.

Solution: A supervisor conf file that sets autostart=false for postgres, redis, rabbitmq, apache2, and syslogng.

Gotcha: Must COPY in Dockerfile, not at runtime. /etc/supervisor/conf.d/ is read-only at runtime. The override must be placed there during the Docker build.

Challenge 5: Python virtual environment persistence

Docassemble writes to its Python virtual environment at runtime (Playground package installs, pip config changes). The venv is ~2 GB.

Solution: Save the venv as a seed in the Docker image, then copy it to /app/data/venv on first boot. The Dockerfile copies the venv to /app/code/seed-venv, then symlinks /usr/share/docassemble/local3.12 → /app/data/venv. On first boot, start.sh copies the seed to persistent storage.

Gotcha: Build service timeout. The 2 GB venv copy can time out on Cloudron's remote build service. If this happens, build locally with docker build and push to a registry.

Challenge 6: config.yml regeneration

Cloudron may change environment variables between restarts (domain changes, credential rotation). Config.yml is regenerated from environment variables on every boot.

The secretkey (used for encrypting session data) is generated once and persisted at /app/data/config/.secret so sessions survive restarts.

Part 3: Suggestions for Cloudron maintainers

Several platform improvements would make packaging complex apps like docassemble significantly easier.

1. Honour memoryLimit in manifests for custom apps

Currently, the memoryLimit field in CloudronManifest.json is ignored for custom (unpublished) apps. The default is 256 MB, which is insufficient for any substantial Python application. Users must manually set memory in the dashboard after installation, which is an undiscoverable step that causes confusing failures.

Suggestion: Apply the manifest's memoryLimit during installation, or at minimum, show a warning if the default is being used despite a higher value in the manifest.

2. Provide a LavinMQ addon

LavinMQ is a lightweight, AMQP 0.9.1 compatible message broker that could serve any app currently needing RabbitMQ. At ~40 MB RAM and a single binary, it is a natural fit for the addon model. A managed LavinMQ addon would simplify packaging for apps that use Celery, Sidekiq, or any AMQP-based task queue.

There is a separate forum request for LavinMQ on Cloudron.

3. Configurable stop timeout per app

Docassemble's upstream Docker setup uses --stop-timeout 600 for graceful shutdown. Cloudron's default stop timeout is shorter. A per-app configurable stop timeout (in the manifest or dashboard) would help apps that need time to flush state.

4. Writable /etc subdirectories as an opt-in manifest option

Many complex apps write to /etc/ subdirectories during initialization (timezone, locale, SSL, cron). Currently, packagers must individually symlink each path. A manifest option like "writableEtcDirs": ["/etc/nginx", "/etc/ssl", "/etc/cron.daily"] would reduce boilerplate significantly.

5. Build service timeout for large images

The Cloudron build service can time out when Dockerfile steps involve copying large directories (2 GB+ in our case). A configurable build timeout or a progress-aware timeout (that resets when output is being produced) would help.

Known limitations (v0.1)

  • Memory must be set manually to ~8 GB in the Cloudron dashboard after installation
  • First boot takes 15-20 minutes (database migration, venv copy)
  • Exim4 (email receiving) does not work (read-only /etc/exim4). Outbound email via Cloudron's sendmail addon works fine.
  • The initialize supervisor process hangs after completing. Non-blocking but wastes some memory.
  • Playground package installs persist across restarts but not across Cloudron updates (the venv is re-seeded from the image on update)
  • No OIDC/LDAP integration with Cloudron user management yet

Repository

[your-repo-url]

Contains: Dockerfile, start.sh, CloudronManifest.json, supervisor/docassemble-cloudron.conf, DESCRIPTION.md, icon.png, documentation.

Tested on Cloudron 9.x with the jhpyle/docassemble:latest image (v1.8.17, Ubuntu 24.04, Python 3.12).

Feedback, bug reports, and pull requests welcome.