@@ -1,8 +1,10 @@
 # sqlitexplorer
 
-A small command-line explorer for SQLite databases. It lists tables, prints
-schemas, dumps rows and runs ad-hoc queries, rendering the results as terminal
-tables with [outfancy](https://github.com/carlosplanchon/outfancy).
+A command-line explorer for SQLite databases. It lists tables, prints schemas,
+dumps rows, computes statistics, searches values, runs ad-hoc queries, draws
+charts, exports and imports data, and offers an interactive shell. Tables are
+rendered with [outfancy](https://github.com/carlosplanchon/outfancy) and charts
+with [plotille](https://github.com/tammoippen/plotille).
 
 ## Installation
 
@@ -15,31 +17,132 @@ uv tool install .
 pip install .
 ```
 
-## Usage
+## Quick tour
 
 ```sh
-sqlitexplorer tables app.db                 # tables and views with their row counts
-sqlitexplorer schema app.db                 # every CREATE statement in the database
-sqlitexplorer schema app.db users           # one table with its indexes and triggers
-sqlitexplorer describe app.db users         # columns: type, NOT NULL, default, primary key
-sqlitexplorer show app.db users             # all the rows of a table or view
-sqlitexplorer show app.db users -n 20 --offset 40
-sqlitexplorer query app.db "SELECT name, age FROM users WHERE age > 30"
+sqlitexplorer tables app.db                     # tables and views with their row counts
+sqlitexplorer schema app.db [users]             # CREATE statements, in creation order
+sqlitexplorer describe app.db users             # columns: type, NOT NULL, default, primary key
+sqlitexplorer indexes app.db [users]            # indexes and the columns they cover
+sqlitexplorer foreign-keys app.db [posts]       # foreign keys
+sqlitexplorer info app.db [--check]             # size, pragmas, object counts, integrity check
+sqlitexplorer show app.db users                 # rows of a table or view
+sqlitexplorer stats app.db users                # nulls, distinct, min, max, top values per column
+sqlitexplorer search app.db "marie"             # find a text in every column of every table
+sqlitexplorer query app.db "SELECT ..."         # run SQL
+sqlitexplorer chart app.db "SELECT day, total FROM sales ORDER BY day"
+sqlitexplorer dump app.db                       # the whole database as SQL
+sqlitexplorer export app.db users -o users.csv  # rows to a file
+sqlitexplorer import app.db people people.csv   # a file into a table
+sqlitexplorer diff app.db backup.db             # compare two schemas
+sqlitexplorer shell app.db                      # interactive shell
+```
+
+Every command has `--help`. Databases are opened read-only and are never
+created; commands that change data need `--write`, except `import`, which
+always writes.
+
+## Exploring
+
+`show` accepts filters so most questions need no SQL:
+
+```sh
+sqlitexplorer show app.db users -c name,age --where "age > 18" --order-by age --desc -n 20 --offset 40
+```
+
+`--columns` and `--order-by` are validated against the table; `--where` is a
+raw SQL condition.
+
+`stats` reports, for each column, the declared type, how many NULLs, how many
+distinct values, the minimum, the maximum and the most frequent values
+(`--top N`). `search` looks for a case-insensitive substring in every non-BLOB
+column of every table (`--table` restricts it, `--limit` stops early) and
+prints the table, column, rowid and value of each match.
+
+## Output options
+
+Every command that prints rows takes the same options:
+
+| Option | Effect |
+|---|---|
+| `--format`, `-f` | `table` (default), `csv`, `tsv`, `json` or `markdown` |
+| `--null TEXT` | text shown for NULL values (default `NULL`) |
+| `--truncate N` | cut values longer than N characters |
+| `--page N`, `--page-size M` | print only one page of the rows; the footer goes to stderr |
+| `--pager` | send the output to `$PAGER` (default `less -R`) when on a terminal |
+| `--color` / `--no-color` | force or disable ANSI colors; by default only on a terminal, never when `NO_COLOR` is set |
+| `--width N` | width to fit tables into (default: the terminal width) |
+
+`NULL` and `BLOB` values are printed as SQL literals (`NULL`, `X'0102'`). In
+the JSON format NULL becomes `null`, BLOBs are base64 strings and values are
+never truncated.
+
+## Queries
+
+```sh
+sqlitexplorer query app.db "SELECT name, age FROM users WHERE age > :min" -p min=30
+sqlitexplorer query app.db --file report.sql            # several statements, one transaction
 echo "SELECT COUNT(*) FROM users" | sqlitexplorer query app.db -
+sqlitexplorer query app.db "SELECT ..." --attach old=backup.db   # then use old.users
+sqlitexplorer query app.db "SELECT ..." --explain        # EXPLAIN QUERY PLAN as a tree
+sqlitexplorer query app.db "SELECT ..." --time           # rows and elapsed time on stderr
+sqlitexplorer query app.db "SELECT COUNT(*) FROM jobs" --watch 2   # re-run every 2 s, Ctrl-C stops
 sqlitexplorer query app.db "DELETE FROM users WHERE age IS NULL" --write
 ```
 
-Databases are opened read-only and are never created. Pass `--write` to
-`query` to run statements that modify data; the change is committed when the
-statement succeeds and rolled back otherwise. `NULL` and `BLOB` values are
-printed as SQL literals (`NULL`, `X'0102'`).
+Several statements can be given at once (separated by `;`); they run in one
+transaction that is committed at the end and rolled back on the first error.
+`--param` values that look like numbers are bound as numbers. Attached
+databases follow the read-only rule of the main one.
+
+## Charts
+
+```sh
+sqlitexplorer chart app.db "SELECT day, sales, returns FROM daily ORDER BY day"
+sqlitexplorer chart app.db "SELECT age FROM users" --kind hist --bins 20
+```
+
+The first column is the X axis (numbers or ISO dates), every other column is a
+series named after the column; rows with NULLs are skipped and counted on
+stderr. `--kind` selects `line` (default), `scatter` or `hist` (first column
+only, `--bins`). `--height`, `--width`, `--x-label`, `--y-label` and `--color`
+adjust the drawing.
+
+## Export, import, dump and diff
+
+```sh
+sqlitexplorer dump app.db -o backup.sql                  # replayable SQL, like .dump
+sqlitexplorer export app.db users -f json -o users.json
+sqlitexplorer export app.db --all -o exported/ -f csv    # one file per table and view
+sqlitexplorer import app.db people people.csv            # csv, tsv or json (list of objects)
+sqlitexplorer diff app.db backup.db                      # exit status 1 when the schemas differ
+```
+
+`import` creates the table when it does not exist, inferring INTEGER, REAL or
+TEXT for each column; empty CSV cells become NULL. The format comes from the
+file extension unless `--format` is given.
+
+## Shell
+
+```
+$ sqlitexplorer shell app.db
+sqlitexplorer> SELECT name
+        ...> FROM users;
+sqlitexplorer> .tables
+sqlitexplorer> .format json
+sqlitexplorer> .quit
+```
 
-Tables adapt to the terminal width. Use `--width N` to render for another width
-(for example when redirecting the output to a file) and `--color` /
-`--no-color` to override the automatic color detection (`NO_COLOR` is
-honoured).
+Statements end with `;` and may span several lines. Dot-commands: `.tables`,
+`.schema [NAME]`, `.describe TABLE`, `.indexes [TABLE]`, `.stats TABLE`,
+`.format FORMAT`, `.null TEXT`, `.truncate N|off`, `.help` and `.quit`. Tab
+completes SQL keywords, table and column names; the history is kept in
+`$XDG_STATE_HOME/sqlitexplorer/history` (`~/.local/state` by default). With
+`--write` every statement is committed as soon as it succeeds. SQL can also be
+piped into the shell.
 
-Run `sqlitexplorer --help` or `sqlitexplorer <command> --help` for every option.
+Completion of the command line itself (commands, options and table names) is
+installed with `sqlitexplorer --install-completion`.
 
 ## Development
 
 
@@ -5,15 +5,15 @@ build-backend = "setuptools.build_meta"
 [project]
 name = "sqlitexplorer"
 version = "0.2.0"
-description = "Explore SQLite databases from the terminal: tables, schema, rows and ad-hoc queries rendered with outfancy."
+description = "Explore SQLite databases from the terminal: tables, schema, stats, search, queries, charts, export/import and an interactive shell."
 readme = "README.md"
 license = "MIT"
 license-files = ["LICENSE"]
 requires-python = ">=3.10"
 authors = [
     { name = "Carlos A. Planchón", email = "carlosandresplanchonprestes@gmail.com" },
 ]
-keywords = ["sqlite", "sqlite3", "database", "cli", "terminal", "explorer"]
+keywords = ["sqlite", "sqlite3", "database", "cli", "terminal", "explorer", "chart", "shell", "repl"]
 classifiers = [
     "Environment :: Console",
     "Intended Audience :: Developers",
@@ -28,6 +28,7 @@ classifiers = [
 ]
 dependencies = [
     "outfancy>=0.11",
+    "plotille>=6",
     "typer>=0.12",
 ]
 
 
@@ -0,0 +1,203 @@
+"""Charts drawn with plotille from a :class:`ResultSet`.
+
+The first column of the result is the X axis (numbers or ISO dates) and every
+other column is a numeric series. Histograms use the first column only.
+"""
+
+from __future__ import annotations
+
+import os
+from collections.abc import Iterator, Sequence
+from contextlib import contextmanager
+from dataclasses import dataclass
+from datetime import datetime
+from enum import Enum
+
+import plotille
+
+from sqlitexplorer.core import ExplorerError, ResultSet
+
+__all__ = [
+    "ChartKind",
+    "Series",
+    "histogram_values",
+    "render_chart",
+    "render_histogram",
+    "series_from_result",
+]
+
+PALETTE = ("red", "green", "yellow", "blue", "magenta", "cyan")
+# plotille reserves this many characters for the Y axis label.
+AXIS_LABEL_WIDTH = 8
+# Characters taken by the Y axis (ticks, label and separator) next to the canvas.
+AXIS_WIDTH = 12
+
+
+class ChartKind(str, Enum):
+    LINE = "line"
+    SCATTER = "scatter"
+    HIST = "hist"
+
+
+@dataclass(frozen=True)
+class Series:
+    label: str
+    x: list[float | datetime]
+    y: list[float]
+
+
+def _number(value: object) -> float | None:
+    if isinstance(value, bool | int | float):
+        return float(value)
+    if isinstance(value, str):
+        try:
+            return float(value)
+        except ValueError:
+            return None
+    return None
+
+
+def _x_value(value: object) -> float | datetime | None:
+    number = _number(value)
+    if number is not None:
+        return number
+    if isinstance(value, str):
+        try:
+            return datetime.fromisoformat(value)
+        except ValueError:
+            return None
+    return None
+
+
+def series_from_result(result: ResultSet) -> tuple[list[Series], int]:
+    """Split *result* into one series per numeric column after the first.
+
+    Rows with a NULL in the X column or in any series are skipped; the second
+    item of the returned tuple counts them.
+    """
+    if len(result.columns) < 2:
+        raise ExplorerError("need an X column and at least one numeric column")
+    x_name, y_names = result.columns[0], result.columns[1:]
+    xs: list[float | datetime] = []
+    ys: list[list[float]] = [[] for _ in y_names]
+    x_type: type | None = None
+    skipped = 0
+    for row in result.rows:
+        if any(value is None for value in row):
+            skipped += 1
+            continue
+        x = _x_value(row[0])
+        if x is None:
+            raise ExplorerError(f"column {x_name} is neither numeric nor an ISO date: {row[0]!r}")
+        if x_type is None:
+            x_type = type(x)
+        elif not isinstance(x, x_type):
+            raise ExplorerError(f"column {x_name} mixes numbers and dates")
+        numbers = []
+        for name, value in zip(y_names, row[1:], strict=True):
+            number = _number(value)
+            if number is None:
+                raise ExplorerError(f"column {name} is not numeric: {value!r}")
+            numbers.append(number)
+        xs.append(x)
+        for bucket, number in zip(ys, numbers, strict=True):
+            bucket.append(number)
+    if not xs:
+        raise ExplorerError("no rows to plot")
+    return [
+        Series(label=name, x=xs, y=bucket) for name, bucket in zip(y_names, ys, strict=True)
+    ], skipped
+
+
+def histogram_values(result: ResultSet) -> tuple[list[float], int]:
+    """Numeric values of the first column of *result*, and how many NULLs were skipped."""
+    if not result.columns:
+        raise ExplorerError("no rows to plot")
+    name = result.columns[0]
+    values: list[float] = []
+    skipped = 0
+    for row in result.rows:
+        if row[0] is None:
+            skipped += 1
+            continue
+        number = _number(row[0])
+        if number is None:
+            raise ExplorerError(f"column {name} is not numeric: {row[0]!r}")
+        values.append(number)
+    if not values:
+        raise ExplorerError("no rows to plot")
+    return values, skipped
+
+
+@contextmanager
+def _color_environment(enabled: bool) -> Iterator[None]:
+    """plotille checks FORCE_COLOR, NO_COLOR and isatty itself; make it follow *enabled*."""
+    saved = {name: os.environ.get(name) for name in ("FORCE_COLOR", "NO_COLOR")}
+    if enabled:
+        os.environ["FORCE_COLOR"] = "1"
+        os.environ.pop("NO_COLOR", None)
+    else:
+        os.environ["NO_COLOR"] = "1"
+    try:
+        yield
+    finally:
+        for name, value in saved.items():
+            if value is None:
+                os.environ.pop(name, None)
+            else:
+                os.environ[name] = value
+
+
+def _canvas_width(width: int) -> int:
+    return max(10, width - AXIS_WIDTH)
+
+
+def render_chart(
+    series: Sequence[Series],
+    *,
+    kind: ChartKind,
+    width: int,
+    height: int,
+    color: bool,
+    x_label: str,
+    y_label: str,
+) -> str:
+    """Draw *series* as a line chart or scatter plot, with a legend when there are several."""
+    figure = plotille.Figure()
+    figure.width = _canvas_width(width)
+    figure.height = max(3, height)
+    figure.with_colors = color
+    figure.color_mode = "names"
+    figure.x_label = x_label
+    figure.y_label = y_label[:AXIS_LABEL_WIDTH]
+    for index, item in enumerate(series):
+        line_color = PALETTE[index % len(PALETTE)] if color else None
+        if kind is ChartKind.SCATTER:
+            figure.scatter(item.x, item.y, lc=line_color, label=item.label)
+        else:
+            figure.plot(item.x, item.y, lc=line_color, label=item.label)
+    with _color_environment(color):
+        return figure.show(legend=len(series) > 1)
+
+
+def render_histogram(
+    values: Sequence[float],
+    *,
+    bins: int,
+    width: int,
+    height: int,
+    color: bool,
+    x_label: str,
+    y_label: str,
+) -> str:
+    """Draw the distribution of *values*."""
+    with _color_environment(color):
+        return plotille.histogram(
+            list(values),
+            bins=bins,
+            width=_canvas_width(width),
+            height=max(3, height),
+            X_label=x_label,
+            Y_label=y_label[:AXIS_LABEL_WIDTH],
+            lc=PALETTE[0] if color else None,
+        )