Skip to content

Commit c332891

Browse files
committed
Merge feature/nitrite-bridge, and release 5.2.0
Brings in `nitrite-bridge`, the Nitrite adapter for the dbinspect wire protocol, together with the skip-without-decoding paging work the branch also carried. The module is deliberately outside the reactor `<modules>` — `org.dizitart:dbinspect-bridge` is not on Maven Central until the dbinspect release, so a build of a clean checkout would fail on it. Build it on its own with `dbinspect-bridge` installed locally. Everything goes to 5.2.0. CHANGELOG.md carries the detail. Reactor green (1634 + 93 + 104 tests), bridge green (76 tests) over MVStore, RocksDB and in-memory.
2 parents 874f585 + dcbf7df commit c332891

49 files changed

Lines changed: 4392 additions & 22 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CHANGELOG.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,32 @@
1+
## Release 5.2.0 - Aug 30, 2026
2+
3+
### New Features
4+
5+
- **`nitrite-bridge` — inspect a running Nitrite database from a desktop client.** The Nitrite adapter for the `dbinspect` wire protocol: collections and handed-in repositories as browsable stores, schema inferred by sampling documents and always flagged as inferred, paging over `FindOptions` skip/limit/orderBy, the JSON filter DSL, and watch over Nitrite's collection subscription. The engine-neutral core it plugs into is `org.dizitart:dbinspect-bridge`, which has no database in its dependency tree at all.
6+
- **Row editing is behind `allowWrite`, and whole-store `snapshot` behind `allowSnapshot`** — both `false` unless the embedding application asks, and absent from the reported capabilities while they are. A row is addressed by `_id`; an update is partial; `changes: 0` means the row was not there. `_id` inside an update's `values` is refused, because Nitrite merges an update document and it would rewrite the identity of the row it just matched.
7+
- **`regex` is off unless `allowRegex` is set**, and length-capped with a nested-quantifier check when it is on. `java.util.regex` backtracks and a match cannot be interrupted, so the default is the mitigation that matters.
8+
- Every client-supplied store name is resolved against the set the adapter reported. `Nitrite.getCollection` creates a collection that does not exist, so an unchecked name would let a paired client make one; writes go through the same allow-list.
9+
- The module is deliberately **absent from the reactor `<modules>`**: `dbinspect-bridge` is not on Maven Central until the dbinspect release, so a build of a clean checkout would fail on it. Build it on its own, with `dbinspect-bridge` installed locally.
10+
11+
- **The bridge can open a transaction** (`docs/PROTOCOL.md` §3.1). A transaction is a second adapter over the same database rather than a mode on the first, so one connection's uncommitted documents can never reach another connection's reads. Nitrite's transaction lives above the storage engine — a `TransactionStore` buffers the writes and a journal replays them on commit — so this works identically on MVStore, RocksDB and in memory.
12+
- `capabilities.transactions` follows `allowWrite`: that flag is the permission, this reports what the engine can undo. The transactional twin reports it `false`, because Nitrite does not nest one.
13+
- `listStores` counts through the transaction, since a total that left out the rows the person just staged is not read-your-own-writes.
14+
15+
### Performance
16+
17+
- Paging a collection no longer costs every row before the page - `find(filter, skipBy(n).limit(m))` now advances the storage iterator without decoding what it passes over.
18+
- `BoundedStream` skipped by calling `next()` on the iterator beneath it, and that iterator is below the document layer: it deserialised **every skipped document** in order to throw it away. Page latency was therefore linear in the page number - about 9 µs per skipped row on both persistent stores - so a 50 000-row collection paged at 200 rows took 3 ms for the first page and 350 ms for the two-hundredth. It is an ordinary browse, not an edge case.
19+
- New `SkippableIterator`, implemented by the storage-level iterators that can do better and by nothing else. `BoundedStream` uses it when it is there and falls back to the loop when it is not, so a stream that has to look at a document to know whether to skip it - a collection-scan filter, a blocking sort - is unaffected and still correct.
20+
- **MVStore descends the tree by index.** An MVStore page records how many entries sit beneath it, so `MVMap.getKey(n)` is a O(log n) descent rather than a scan; the skip seeks with it and continues from a cursor. Measured over 50 000 rows, 200 to a page: p95 **583.9 ms → 5.8 ms**, and the first and last pages now cost the same 0.1 ms.
21+
- **RocksDB steps the native iterator.** There is no seek-by-index to use, but the decode is what a skipped row actually cost; advancing past one is now a memcmp inside the block the iterator is already on. p95 **3.8 ms** over the same 50 000 rows.
22+
- An index scan skips its own id set the same way, without the map lookup and decode `next()` would have paid for each row.
23+
24+
### Issue Fixes
25+
26+
- Fix the index dirty marker not being persisted ([#1281](https://github.com/nitrite/nitrite-java/pull/1281))
27+
- `markDirty()` flipped `isDirty` on the `IndexMeta` that `get()` returned and never put it back, so nothing told the store the entry had changed and the new value was not guaranteed to be written. Both `beginIndexing()` and `endIndexing()` go through it, so the marker was unreliable in both directions: an index left half-built by a crash could come back reading as clean, and a completed one could stay marked dirty and be rebuilt on the first write after every open.
28+
29+
130
## Release 5.1.0 - Aug 22, 2026
231

332
### Improvements

‎nitrite-bom/pom.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
<parent>
55
<groupId>org.dizitart</groupId>
66
<artifactId>nitrite-java</artifactId>
7-
<version>5.1.0</version>
7+
<version>5.2.0</version>
88
</parent>
99

1010
<artifactId>nitrite-bom</artifactId>

‎nitrite-bridge/README.md‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# nitrite-bridge
2+
3+
Inspect a running Nitrite database from a desktop client, over a paired, loopback-by-default
4+
connection.
5+
6+
This artifact is the **Nitrite adapter only**. The wire protocol, pairing, transport and the
7+
release guard are in `org.dizitart:dbinspect-bridge`, which knows about no database at all — so a
8+
JVM developer inspecting an H2, MapDB or JDBC database takes that artifact and writes their own
9+
adapter, and nothing about that path routes through a Nitrite repository.
10+
11+
```java
12+
BridgeServer bridge = DbInspect.start(
13+
DbInspect.options("my_app", List.of(
14+
NitriteAdapter.builder(db, "main", "app data")
15+
.repositories(orders, customers)
16+
.build())));
17+
```
18+
19+
## Two things to know before you ship it
20+
21+
**Declare `INTERNET`.** A bridge binds a socket; on Android, without the permission it fails with
22+
a `SecurityException` that reads like a bug in us.
23+
24+
**Keep it out of your release build.** Everything below is off unless you turn it on, but the
25+
strongest guard is the artifact not being there at all:
26+
27+
- **Desktop** — `provided` scope, *and* `-Ddbinspect.bridge.enabled=true` at launch. Two signals,
28+
neither of which survives into a production launch script by accident.
29+
- **Android** — `debugImplementation`. An application has no command line, so the system property
30+
can never be set and a guard requiring it could never be satisfied; the packaging is the guard,
31+
which is a stronger property rather than a weaker one. R8 shrinking is **not** the guard.
32+
33+
Both example applications carry a script that greps the built artifact for the protocol strings,
34+
and both build twice so the check has a negative control.
35+
36+
## Capabilities, and what is off by default
37+
38+
`edit`, `snapshot` and `regex` are off unless you asked for them on that adapter — `allowWrite`,
39+
`allowSnapshot`, `allowRegex` on the builder. What is off is *absent from the reported
40+
capabilities*, not merely refused at call time, so the client greys it out rather than offering
41+
something that will fail.
42+
43+
**A row is addressed by `_id`.** With `allowWrite(true)`, `updateRow` and `deleteRow` take the
44+
value the grid showed in that column — a `long` here, and the bracketed `[1755…]NO₂` rendering is
45+
accepted too, so an id pasted from another runtime's grid works. `_id` inside an update's `values`
46+
is refused: Nitrite merges an update document, so it would rewrite the identity of the row it just
47+
matched. An update is partial, and `changes: 0` means the row was not there, which is an answer
48+
rather than an error.
49+
50+
`capabilities.filterOps` reports what this implementation actually has. Two notes on it:
51+
52+
- **`exists` needs nitrite 5.0.0**, which is the floor this artifact sets. It tests presence only:
53+
a field explicitly set to null is present and matches, and "does not have the field" is `not`
54+
around it, never `exists` with `value: false`.
55+
- **`text` needs a full-text index on the field.** Nitrite's own query planner refuses a text
56+
filter it cannot serve from an index, and `filterOps` is a flat operator list that cannot say
57+
"text, but only on `bio`". The adapter refuses per field, with a message that names the fix.
58+
59+
## Repositories are handed in; collections are discovered
60+
61+
`listRepositories()` answers with entity names, and an entity name is not a `Class`. Turning one
62+
back into a type means loading a class named by a paired client, and the underlying document
63+
collection cannot be reached by name either — `getCollection` refuses a name a repository owns,
64+
and re-opening the map behind one would give you a second collection object with its own event
65+
bus, so `watch` would silently never fire.
66+
67+
So pass the repositories you want inspected. One that was not passed in is not listed, rather than
68+
listed and unopenable.
69+
70+
## Building it
71+
72+
`dbinspect-bridge` is not on Maven Central yet, so this module is deliberately **not** in
73+
`nitrite-java`'s `<modules>` — a clean checkout could not resolve it and the whole reactor would
74+
fail. Install the core, then build this on its own:
75+
76+
```bash
77+
mvn -f <dbinspect>/jvm/dbinspect-bridge/pom.xml install
78+
mvn -f nitrite-bridge/pom.xml test
79+
```
80+
81+
## Examples
82+
83+
| Path | What it is |
84+
|---|---|
85+
| `example/` | a plain `java -jar` desktop application, and `tool/verify_release_jar.sh` |
86+
| `example-android/` | a native Android (Kotlin) application, and `tool/verify_release_apk.sh` |
87+
88+
Both are browsable from the desktop client, and the dbinspect conformance suite passes unmodified
89+
against either. `tool/run_reference_bridge.sh` starts a bridge over a memory, MVStore or RocksDB
90+
database for the suite to be pointed at.
Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# minSdk 26, and what pinned it
2+
3+
`PLAN.md` §6 M3 asks for a floor set by measurement rather than assumption, and for the finding
4+
that set it. Here is both. Measured 2026-08-06 with AGP 8.9.1, Gradle 9.2, `desugar_jdk_libs`
5+
2.1.5 and core library desugaring on.
6+
7+
The measurement is `assembleRelease` at each candidate level: D8 refuses to dex what it cannot
8+
desugar, so the floor is a build failure rather than a runtime surprise.
9+
10+
| minSdk | this app (Nitrite + MVStore + bridge) | bridge without MVStore |
11+
|---|---|---|
12+
| 21 | **fails to dex** | dexes |
13+
| 24 | **fails to dex** | dexes |
14+
| 26 | dexes | dexes |
15+
16+
## What sets it, and it is not what the plan expected
17+
18+
The plan predicted the gap would be in `java.nio` / `java.time`, since that is where desugaring
19+
usually falls short of Java 11. It is not. Every one of the four errors is the same shape:
20+
21+
```
22+
D8: MethodHandle.invoke and MethodHandle.invokeExact are only supported starting with
23+
Android O (--min-api 26): Lorg/h2/util/Bits;uuidToBytes(JJ)[B
24+
```
25+
26+
…and all four are in **`h2-mvstore`**, the storage engine behind `nitrite-mvstore-adapter`:
27+
`Bits.uuidToBytes`, `AES.encryptBlock`, `AES.decryptBlock`, `SHA256.getPBKDF2`. H2 reaches for
28+
`VarHandle`/`MethodHandle` polymorphic signatures for byte access, and a polymorphic signature is
29+
not something desugaring can rewrite — the call is resolved by the VM, so Android O is the floor
30+
and no library version of ours moves it.
31+
32+
**So the floor is the store, not the bridge.** `dbinspect-bridge` (Java-WebSocket, Gson, slf4j),
33+
`nitrite-bridge` and Nitrite's own core all dex cleanly at API 21 with desugaring on. If a future
34+
Android target needs to go below 26, the question to ask is which store it uses, not what the
35+
bridge does — and the answer is likely the in-memory store or a different adapter, since
36+
`nitrite-rocksdb-adapter` carries a native library that has no Android build at all.
37+
38+
## What this does not prove
39+
40+
Dexing is not running. The app was run on an API 37 emulator (Pixel 10 Pro XL) and the M1
41+
conformance suite passes against it unmodified over `adb forward` — 111 checks, 108 passed, 0
42+
failed, 3 skipped. **API 26 itself has not been run**: no API 26 system image is installed on the
43+
build machine. Dexing at 26 is a hard floor for the artifact and running at 37 shows the bridge
44+
works on the platform, but a run on an API 26 image is what would close this properly, and it is
45+
one `sdkmanager` download away.
46+
47+
`animal-sniffer` in `nitrite-bridge`'s own build checks its sources against the API 26 signature
48+
on every `mvn test`, so a regression in the adapter itself fails there rather than here.
Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
plugins {
2+
id("com.android.application")
3+
id("org.jetbrains.kotlin.android")
4+
}
5+
6+
android {
7+
namespace = "org.dizitart.example.android"
8+
compileSdk = 35
9+
10+
defaultConfig {
11+
applicationId = "org.dizitart.example.android"
12+
// See MinSdk.md beside this file. Not a guess and not yet a measurement either — the
13+
// number is what the artifacts on the compile path require; PLAN.md M3 asks for a run.
14+
minSdk = 26
15+
targetSdk = 35
16+
versionCode = 1
17+
versionName = "1.0"
18+
}
19+
20+
compileOptions {
21+
sourceCompatibility = JavaVersion.VERSION_11
22+
targetCompatibility = JavaVersion.VERSION_11
23+
// dbinspect-bridge is Java 11 and uses java.time and java.nio; below API 26 those are
24+
// only there through desugaring, which is why this is on rather than assumed away.
25+
isCoreLibraryDesugaringEnabled = true
26+
}
27+
28+
kotlinOptions {
29+
jvmTarget = "11"
30+
}
31+
32+
buildTypes {
33+
release {
34+
// Deliberately off. THREAT-MODEL §7 criterion 2 must hold because the bridge is not
35+
// in the variant at all, not because R8 happened to shrink it away — R8 is
36+
// configurable away and is not the guard (see DbInspect#bridgeEnabled).
37+
isMinifyEnabled = false
38+
}
39+
}
40+
}
41+
42+
dependencies {
43+
implementation("org.dizitart:nitrite:5.0.0")
44+
implementation("org.dizitart:nitrite-mvstore-adapter:5.0.0")
45+
46+
// THE release guard on Android. An application has no command line, so the system property
47+
// -Ddbinspect.bridge.enabled can never be set and a guard requiring it could never be
48+
// satisfied; the packaging is the guard instead, and it is a stronger property than a runtime
49+
// flag rather than a weaker one. src/release/ carries a no-op DebugTools so that nothing in
50+
// the release variant so much as names a bridge class.
51+
debugImplementation("org.dizitart:nitrite-bridge:5.0.0")
52+
53+
coreLibraryDesugaring("com.android.tools:desugar_jdk_libs:2.1.5")
54+
}
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
<?xml version="1.0" encoding="utf-8"?>
2+
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
3+
4+
<!--
5+
mDNS only, and only in the debug variant. NsdManager needs a held multicast lock to see
6+
anything on most handsets, and the lock needs these two. adb forward is the primary path
7+
on Android and needs none of it, so a developer who never turns mDNS on loses nothing.
8+
-->
9+
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
10+
<uses-permission android:name="android.permission.CHANGE_WIFI_MULTICAST_STATE" />
11+
</manifest>
Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
package org.dizitart.example.android
2+
3+
import android.content.Context
4+
import android.net.nsd.NsdManager
5+
import android.net.nsd.NsdServiceInfo
6+
import android.net.wifi.WifiManager
7+
import android.util.Log
8+
import org.dizitart.dbinspect.BridgeAdapter
9+
import org.dizitart.dbinspect.BridgeServer
10+
import org.dizitart.dbinspect.DbInspect
11+
import org.dizitart.no2.Nitrite
12+
import org.dizitart.no2.bridge.NitriteAdapter
13+
import java.security.SecureRandom
14+
15+
/**
16+
* Everything this application knows about dbinspect, in the debug variant only.
17+
*
18+
* `src/release/` has a file with this name that returns null and names nothing, so [MainActivity]
19+
* calls one signature and the packaging decides which one exists. See app/build.gradle.kts.
20+
*
21+
* **The primary path on Android is `adb forward`, not mDNS**, and that needs nothing from this
22+
* file — the bridge binds loopback and `adb forward tcp:<local> tcp:<bridge>` reaches it. mDNS is
23+
* opt-in and secondary, and it is here because PLAN.md M3 asks for `NsdManager` rather than JmDNS
24+
* on this platform, with a multicast lock held.
25+
*/
26+
object DebugTools {
27+
28+
private const val TAG = "dbinspect"
29+
30+
/** Set to true to also advertise on the local network. Off is the default for a reason. */
31+
private const val ADVERTISE_OVER_MDNS = false
32+
33+
fun start(context: Context, db: Nitrite): String? {
34+
val adapters: List<BridgeAdapter> =
35+
listOf(NitriteAdapter.builder(db, "nitrite-main", "example.db").build())
36+
37+
// bridgeEnabled() is true on Dalvik/ART by construction: an application has no command
38+
// line, so the desktop system property could never be set and a guard requiring it could
39+
// never be satisfied. On this platform the packaging above is the guard.
40+
val bridge = DbInspect.start(DbInspect.options("example_app", adapters, android.os.Build.MODEL))
41+
?: return null
42+
43+
if (ADVERTISE_OVER_MDNS) {
44+
advertise(context, bridge)
45+
}
46+
47+
Log.i(TAG, bridge.banner())
48+
return buildString {
49+
append("dbinspect is listening\n\n")
50+
append("pairing code ").append(bridge.pairingCode().value()).append('\n')
51+
append("port ").append(bridge.port()).append("\n\n")
52+
append("adb forward tcp:").append(bridge.port())
53+
.append(" tcp:").append(bridge.port())
54+
}
55+
}
56+
57+
/**
58+
* NsdManager, with a multicast lock held for as long as the advertisement is up.
59+
*
60+
* Without the lock most handsets drop multicast frames while the screen is off and some drop
61+
* them always, which presents as "the device never appears" with nothing in any log.
62+
*
63+
* The TXT record carries a random per-session instance name and `proto`, and nothing else —
64+
* PROTOCOL.md §1: appName, deviceModel, bridgeVersion and the adapter list are answered after
65+
* pairing, because advertising them unauthenticated hands every peer on the subnet a target
66+
* list and a version number to look up.
67+
*/
68+
private fun advertise(context: Context, bridge: BridgeServer) {
69+
val wifi = context.applicationContext.getSystemService(Context.WIFI_SERVICE) as WifiManager
70+
val lock = wifi.createMulticastLock("dbinspect").apply {
71+
setReferenceCounted(false)
72+
acquire()
73+
}
74+
75+
val random = SecureRandom()
76+
val instance = (0 until 8)
77+
.map { "abcdefghijklmnopqrstuvwxyz0123456789"[random.nextInt(36)] }
78+
.joinToString("")
79+
80+
val service = NsdServiceInfo().apply {
81+
serviceName = instance
82+
serviceType = "_dbinspect._tcp"
83+
port = bridge.port()
84+
setAttribute("proto", "1")
85+
}
86+
87+
val manager = context.getSystemService(Context.NSD_SERVICE) as NsdManager
88+
manager.registerService(service, NsdManager.PROTOCOL_DNS_SD, object : NsdManager.RegistrationListener {
89+
override fun onServiceRegistered(info: NsdServiceInfo) {
90+
Log.i(TAG, "advertising as ${info.serviceName}")
91+
}
92+
93+
override fun onRegistrationFailed(info: NsdServiceInfo, errorCode: Int) {
94+
Log.w(TAG, "mDNS registration failed: $errorCode")
95+
lock.release()
96+
}
97+
98+
override fun onServiceUnregistered(info: NsdServiceInfo) {
99+
lock.release()
100+
}
101+
102+
override fun onUnregistrationFailed(info: NsdServiceInfo, errorCode: Int) {
103+
lock.release()
104+
}
105+
})
106+
}
107+
}

0 commit comments

Comments
 (0)