Skip to content
View spatialite-geopackage-automation's full-sized avatar

Block or report spatialite-geopackage-automation

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse

SpatiaLite & GeoPackage Automation with Python

SpatiaLite & GeoPackage Automation with Python

Production-grade patterns for lightweight, serverless, offline-first spatial databases.

🌐 Live site: www.spatialite-geopackage-automation.org

A free, deep, and continuously growing knowledge base for engineers who build spatial software that has to work without a database server and without a network — on field tablets, mobile apps, edge devices, and serverless pipelines. Every page is hands-on: real Python, SQL, and shell you can copy and run, benchmark tables you can act on, and validation checklists you can tick off as you harden a deployment.

If you have ever wrestled with a no such function: ST_Intersects error, a silently out-of-sync R-tree index, a database is locked crash during a field sync, or the exact byte layout of a GeoPackage geometry blob — this site is written for you.


Why this resource exists

Most GIS documentation assumes a heavyweight, server-based stack (PostGIS, ArcGIS Enterprise). But a huge and growing class of work happens on single-file spatial databases built on SQLiteSpatiaLite and the OGC GeoPackage format. These engines run inside your application process, which shifts every responsibility — extension loading, transaction scoping, geometry serialization, concurrency, and sync — directly into your code.

This site fills that gap with 50+ in-depth guides (100,000+ words) covering the format internals, the Python integration layer, and the command-line automation and offline-sync patterns that production field systems actually depend on.

Who it's for: field GIS technicians, Python data engineers, mobile app developers, and offline-first platform builders working far from a reliable network.


What you'll find

The library is organized into three deep sections, each with topic guides and focused, task-oriented how-to pages you can read in any order.

The engine internals and the OGC format contracts every automation script depends on.

  • The GeoPackage specification, mandatory metadata tables, and how to validate OGC compliance
  • SpatiaLite metadata tables, the SRS registry, and coordinate reference systems
  • File-structure and header analysis (SQLite header, GeoPackage application_id)
  • Extension compatibility, version detection, and cross-platform mod_spatialite loading
  • Security boundaries: SQLCipher encryption, read-only WAL distribution, safe temp-file handling

The in-process library layer: connection lifecycle, serialization, and transaction control.

  • Native sqlite3 + mod_spatialite, connection pooling, and lifecycle management
  • Fiona / OGR driver configuration and GeoPandas ↔ GeoPackage integration
  • Geometry serialization — WKB and GeoPackage Binary (GPB) round-trips with Shapely 2.0
  • Transaction scoping, WAL mode, BEGIN IMMEDIATE, and retrying locked writes
  • End-to-end automation recipes: ingest → create → R-tree index → validate → sync

The layer above your library code: shell automation and offline-first synchronization.

  • Driving sqlite3, spatialite, ogr2ogr, and ogrinfo from scripts and cron
  • Change tracking with triggers, delta transport, and deterministic conflict resolution
  • Idempotent sync-push scripts that survive crashes and retries
  • SpatiaLite vs GeoPackage trade-offs for mobile offline sync

What makes it useful

  • Runnable, verified code. Every code block targets Python 3.9+, sqlite3, GDAL/OGR 3.4+, and Shapely 2.0+, with copy buttons and syntax highlighting.
  • Diagnostics-first. Guides name real failure modes — exact error messages, causes, and fixes — not just the happy path.
  • Original diagrams. Hand-authored, theme-aware SVG architecture and data-flow diagrams explain the hardest concepts.
  • Densely cross-linked. Concepts link to their in-depth reference the first time they appear, so you can go as deep as you need.
  • Fast & installable. A static, no-backend site that installs as a Progressive Web App and works offline — appropriate for an audience that is often disconnected.
  • Accessible & responsive. Built to WCAG 2 A/AA, tuned from mobile through widescreen.

Built with

Static output, no runtime backend — generated with Eleventy (11ty), Markdown-it, Prism syntax highlighting, and KaTeX, and deployed to Cloudflare.

Local development

npm install
npm run dev      # local dev server with live reload
npm run build    # production build into _site/

Set the production URL via an env var (used for canonical URLs, sitemap, and OG tags):

cp .env.example .env   # then edit PUBLIC_SITE_URL

Project layout

content/                 Source docs (one index.md per page; optional YAML frontmatter)
src/
  _data/                 Global data (site, docs tree, build id)
  _includes/             base / header / footer / page layouts (Nunjucks)
  assets/{css,js,img}    Styles, progressive-enhancement JS, logo & icons
  static/                Files copied to site root (favicon.ico)
  index.njk              Home page
  docs.njk               Paginated template that renders every content page
  404 / offline / sitemap / robots / manifest / sw
lib/
  markdown.mjs           Shared markdown-it renderer (code, tables, tasks, KaTeX…)
  content.mjs            Reads content/, builds URLs, nav tree, breadcrumbs
scripts/gen-icons.mjs    One-time PNG/ICO icon generation from the logo
eleventy.config.mjs      Eleventy config (filters, collections, HTML minify)

Authoring content

Add a folder under content/ with an index.md. The first # H1 becomes the page title and the URL mirrors the folder path (e.g. content/python-integration-database-workflows/index.md/python-integration-database-workflows/). Optional YAML frontmatter (title, description, dates) is stripped before rendering. Navigation, breadcrumbs, and "related content" cards are generated automatically from the directory tree.

Supported markdown features (all styled to the site palette):

  • Code blocks — build-time syntax highlighting (Python/SQL/Bash/…) with a copy button and a light, palette-matched theme.
  • Task lists- [ ] render as interactive checkboxes; checking one strikes the text through and persists per-page in localStorage.
  • Tables — automatically wrapped for horizontal scrolling on small screens.
  • In-page links — heading anchors with sticky-header-aware scroll offset.
  • FAQ accordions — authored <details> disclosure blocks, styled as accordions.
  • Diagrams — hand-authored inline SVGs using the site's dg-node / dg-edge design tokens, responsive and theme-aware.
  • KaTeX$inline$ and $$block$$ math is rendered at build time.

Regenerating icons

Icons (favicon, Apple touch, PWA maskable) derive from src/assets/img/logo.svg:

node scripts/gen-icons.mjs

PWA

manifest.webmanifest + sw.js make the site installable and offline-capable (app-shell precache + stale-while-revalidate; navigations fall back to a cached copy then /offline/). Apple meta tags cover iOS home-screen installs.


Explore the full library at www.spatialite-geopackage-automation.org.

Popular repositories Loading

  1. spatialite-geopackage-automation spatialite-geopackage-automation Public

    Production-focused guides and runnable Python, SQL & CLI recipes for automating SpatiaLite & GeoPackage — offline-first spatial databases: format internals, serialization, R-tree indexing, transact…

    CSS