Skip to content

Commit 06c9b97

Browse files
Thoriumclaude
Thorium
and
claude
committed
Scaffold SQLProvider.Fable: one query runtime, two backends
Portable F# core (SqlValue, ISqlConnector, Row readers) with an ADO.NET implementation and a Fable/Rust implementation over rusqlite. The smoke suite is written once against ISqlConnector and runs unchanged on both: 14/14 pass under xunit on .NET and as a Rust binary. The native surface is five members, not System.Data: obj becomes a union (Rust would make it Lrc<dyn Any>), DBNull becomes option, parameters bind at execute time rather than on a mutable command collection, and there is no connection-state machine to model. Everything else in SQLProvider's ISqlProvider is schema introspection that stays on .NET. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
0 parents  commit 06c9b97

20 files changed

Lines changed: 1443 additions & 0 deletions

‎.gitignore‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# .NET
2+
bin/
3+
obj/
4+
*.user
5+
6+
# Rust / Cargo
7+
target/
8+
Cargo.lock
9+
10+
# Fable
11+
.fable/
12+
fable_modules/
13+
build/

‎README.md‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
# SQLProvider.Fable
2+
3+
A light SQL data-access runtime for F# that compiles to **Rust** through Fable, and
4+
to .NET through ADO.NET, from one source.
5+
6+
The shared smoke suite in `tests/SQLProvider.Fable.SmokeTests/Smoke.fs` is written
7+
once against `ISqlConnector` and runs on both backends unchanged — no `#if` in it.
8+
9+
## Status
10+
11+
| Backend | Driver | Verified |
12+
|---|---|---|
13+
| .NET | ADO.NET (`Microsoft.Data.Sqlite`) | 14/14 smoke tests pass |
14+
| Rust | `rusqlite` (bundled SQLite) | 14/14 smoke tests pass |
15+
16+
Not built yet: the schema code generator, the quotation-to-SQL translator, and
17+
backends other than SQLite. See [Not done yet](#not-done-yet).
18+
19+
## Why not just port SQLProvider's runtime?
20+
21+
Almost none of `System.Data` survives the trip to Rust, and almost none of it
22+
needs to. Tracing `executeQuery` in SQLProvider's `SqlRuntime.Linq.fs`, one query
23+
touches roughly twenty ADO members:
24+
25+
| Type | Members used |
26+
|---|---|
27+
| `IDbConnection` | `ConnectionString`, `Open`, `Close`, `State`, `Dispose` |
28+
| `IDbCommand` | `CommandText`, `CommandTimeout`, `Parameters.Add`, `ExecuteReader`, `ExecuteScalar`, `ExecuteNonQuery` |
29+
| `IDbDataParameter` | `ParameterName`, `Value`, `DbType`, `Direction` |
30+
| `IDataReader` | `Read`, `FieldCount`, `GetName`, `GetValue` |
31+
| `ConnectionState`, `DBNull`, `IDbTransaction`, `IsolationLevel` | — |
32+
33+
Everything else in SQLProvider's `ISqlProvider` — `GetTables`, `GetColumns`,
34+
`GetRelationships`, `GetSprocs`, `CreateTypeMappings`, the `DataTable` and SSDT
35+
machinery — is schema introspection. That runs on .NET at code-generation time
36+
and never reaches a Fable target.
37+
38+
But those twenty members should not be ported either, because each collides with
39+
Rust:
40+
41+
1. **`obj` values.** `reader.GetValue i : obj` becomes `Lrc<dyn Any>` in Rust, so
42+
every column read is a downcast. Replaced by the `SqlValue` union.
43+
2. **`DBNull`** has no Rust analogue and shouldn't get one. It's `option`.
44+
3. **A mutable `.Parameters` collection.** rusqlite and sqlx want parameters
45+
supplied at execute time, not accumulated on a command object.
46+
4. **`Open`/`Close`/`State`.** rusqlite opens on construct; sqlx has a pool.
47+
There is no ADO connection-state machine to model.
48+
5. **`System.Transactions.TransactionScope`** — SQLProvider's ambient-scope write
49+
model has no Rust counterpart. Transactions here are explicit.
50+
51+
So the native surface is one interface with five members
52+
(`src/SQLProvider.Fable.Core/Connector.fs`), and everything else is portable F#
53+
on top of it.
54+
55+
## Layout
56+
57+
```
58+
src/SQLProvider.Fable.Core portable F#: SqlValue, ISqlConnector, Row readers
59+
src/SQLProvider.Fable.Ado .NET backend over ADO.NET
60+
src/SQLProvider.Fable.Rust Fable/Rust backend: F# bindings + sqlite_native.rs
61+
tests/SQLProvider.Fable.SmokeTests the suite, written once, backend-agnostic
62+
tests/SQLProvider.Fable.Tests.Net runs it under xunit over ADO.NET
63+
tests/Rust runs it as a Rust binary over rusqlite
64+
```
65+
66+
`Core` and `SmokeTests` must stay free of `System.Data`, reflection,
67+
`System.Linq.Expressions` and async — anything there has to survive Fable's Rust
68+
backend.
69+
70+
## Running the tests
71+
72+
.NET:
73+
74+
```bash
75+
dotnet test tests/SQLProvider.Fable.Tests.Net/SQLProvider.Fable.Tests.Net.fsproj
76+
```
77+
78+
Rust (needs `rustup` and the `fable` CLI on PATH):
79+
80+
```bash
81+
pwsh tests/Rust/build.ps1
82+
```
83+
84+
That script compiles F# to Rust, stages `sqlite_native.rs` and `Cargo.toml` into
85+
`build/rust`, then runs the binary. It exits non-zero if any test fails.
86+
87+
## Design notes
88+
89+
### Result sets are materialised eagerly
90+
91+
Not laziness: a rusqlite `Rows` iterator borrows the `Statement` that produced
92+
it, so a reader kept alive across F# calls would need a self-referential struct.
93+
SQLProvider's own `Sql.dataReaderToArray` already reads the whole reader into an
94+
array before projecting, so nothing is lost.
95+
96+
### Mapping is generated, not reflected
97+
98+
Fable's Rust `FSharpPropertyInfo` carries only a name (see fable-library-rust
99+
`QuotationTypes.fs`), so `PropertyInfo.PropertyType` is unavailable and a generic
100+
"convert each column to whatever the record field wants" mapper has nothing to
101+
dispatch on. Mappers state conversions explicitly instead:
102+
103+
```fsharp
104+
let ofRow (r: SqlRow) : Customer =
105+
{ CustomerId = Row.int r "CustomerId"
106+
Name = Row.text r "Name"
107+
Country = Row.textOpt r "Country"
108+
Balance = Row.float r "Balance" }
109+
```
110+
111+
This is what the schema code generator will emit per table. It is also faster and
112+
clearer than reflection on .NET.
113+
114+
### The native boundary carries primitives only
115+
116+
`sqlite_native.rs` never sees a Fable-generated `SqlValue`. Values cross as an
117+
integer tag plus a typed accessor call, so a change in how Fable lays out unions
118+
cannot silently break the shim.
119+
120+
### Parameters
121+
122+
SQL text uses `@name` on every backend (rusqlite accepts `@`, `:` and `$`;
123+
Microsoft.Data.Sqlite accepts `@`). `SqlParam.Name` is the bare name; each
124+
connector adds the marker its driver wants.
125+
126+
## Fable Rust codegen gaps found while building this
127+
128+
Three, all worked around here, all worth filing upstream:
129+
130+
1. **Object expression calling back into its enclosing object.** Generates a
131+
struct field literally named `_`, a reserved identifier in Rust, and calls the
132+
enclosing object's method on the object-expression struct. Worked around with
133+
a concrete class (`RusqliteTransaction`). Matches the cases already commented
134+
out in Fable's own `MiscTests.fs`.
135+
2. **`this :> ISomeInterface` inside the type's own interface implementation.**
136+
`interface_cast!` expands to a plain `as`, which Rust rejects as a
137+
non-primitive cast. Worked around by moving the logic to free functions.
138+
3. **Doubly-nested closure over a captured value.** `Array.init n (fun r ->
139+
Array.init m (fun c -> ...raw...))` clones `raw` into the outer `Fn` closure
140+
but *moves* it into the inner one, so it does not borrow-check. Worked around
141+
with explicit loops and a top-level helper.
142+
143+
Also worth knowing: Fable emits the `#[path = "..."] mod ...` declaration for a
144+
`.rs` pulled in by `importAll`, but does not copy the file — the build script
145+
stages it.
146+
147+
## Not done yet
148+
149+
- **Schema code generator.** Reads SQLProvider's own offline schema JSON
150+
(`SchemaCache.Save`, the documented `ContextSchemaPath` format) and emits
151+
records plus `ofRow` mappers. File-format coupling only: no assembly
152+
reference to SQLProvider, no internals, no version lockstep.
153+
- **Quotation-to-SQL translator.** `<@ fun c -> c.Country = "UK" @>` to a WHERE
154+
clause, walking the `FSharpExpr` AST that Fable's Rust target now supports.
155+
- **Blob parameters on Rust** are bound but untested; blob *results* are covered
156+
by the shim, not by a test.
157+
- **Other backends.** Postgres/MySQL via sqlx or tokio-postgres will need the
158+
async story worked out — Fable's Rust async is its own machinery and bridging
159+
it to tokio is not free. rusqlite is synchronous, which is why it came first.
160+
- **Dates and decimals** currently travel as text/float. A real backend wants
161+
native cases on `SqlValue`.

‎SQLProvider.Fable.slnx‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
<Solution>
2+
<Folder Name="/src/">
3+
<Project Path="src/SQLProvider.Fable.Ado/SQLProvider.Fable.Ado.fsproj" />
4+
<Project Path="src/SQLProvider.Fable.Core/SQLProvider.Fable.Core.fsproj" />
5+
<Project Path="src/SQLProvider.Fable.Rust/SQLProvider.Fable.Rust.fsproj" />
6+
</Folder>
7+
<Folder Name="/tests/">
8+
<Project Path="tests/SQLProvider.Fable.SmokeTests/SQLProvider.Fable.SmokeTests.fsproj" />
9+
<Project Path="tests/SQLProvider.Fable.Tests.Net/SQLProvider.Fable.Tests.Net.fsproj" />
10+
</Folder>
11+
<Folder Name="/tests/Rust/">
12+
<Project Path="tests/Rust/SQLProvider.Fable.Tests.Rust.fsproj" />
13+
</Folder>
14+
</Solution>

‎src/SQLProvider.Fable.Ado/AdoConnector.fs‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
namespace SQLProvider.Fable.Ado
2+
3+
open System
4+
open System.Data
5+
open SQLProvider.Fable
6+
7+
/// ADO.NET implementation of ISqlConnector.
8+
///
9+
/// This is the whole of System.Data the runtime needs: IDbConnection
10+
/// (Open/Close/State/CreateCommand), IDbCommand (CommandText/Parameters/
11+
/// ExecuteReader/ExecuteScalar/ExecuteNonQuery), IDbDataParameter
12+
/// (ParameterName/Value) and IDataReader (Read/FieldCount/GetName/GetValue).
13+
/// Everything else in SQLProvider's ISqlProvider is schema introspection, which
14+
/// stays on .NET at code-generation time and never reaches a Fable target.
15+
module private Conv =
16+
17+
let toSqlValue (v: obj) : SqlValue =
18+
match v with
19+
| null -> SqlNull
20+
| :? DBNull -> SqlNull
21+
| :? bool as b -> SqlBool b
22+
| :? int8 as i -> SqlInt(int64 i)
23+
| :? uint8 as i -> SqlInt(int64 i)
24+
| :? int16 as i -> SqlInt(int64 i)
25+
| :? uint16 as i -> SqlInt(int64 i)
26+
| :? int32 as i -> SqlInt(int64 i)
27+
| :? uint32 as i -> SqlInt(int64 i)
28+
| :? int64 as i -> SqlInt i
29+
| :? float32 as f -> SqlFloat(float f)
30+
| :? float as f -> SqlFloat f
31+
| :? decimal as d -> SqlFloat(float d)
32+
| :? string as s -> SqlText s
33+
| :? (byte[]) as b -> SqlBlob b
34+
// Matching the placeholder the union documents: dates travel as
35+
// ISO-8601 text until the union grows a native date case.
36+
| :? DateTime as d -> SqlText(d.ToString "o")
37+
| :? Guid as g -> SqlText(g.ToString())
38+
| other -> SqlText(string other)
39+
40+
let toDbValue (v: SqlValue) : obj =
41+
match v with
42+
| SqlNull -> box DBNull.Value
43+
| SqlBool b -> box b
44+
| SqlInt i -> box i
45+
| SqlFloat f -> box f
46+
| SqlText s -> box s
47+
| SqlBlob b -> box b
48+
49+
/// Wraps an already-constructed ADO.NET connection. Ownership passes to the
50+
/// connector: Close/Dispose closes it.
51+
type AdoConnector(connection: IDbConnection, ?parameterPrefix: string) =
52+
53+
let prefix = defaultArg parameterPrefix "@"
54+
55+
// Microsoft.Data.Sqlite (and SqlClient) refuse to run a command while a
56+
// transaction is open on the connection unless the command is enlisted in
57+
// it explicitly, so the active one is tracked here.
58+
let mutable current: IDbTransaction = null
59+
60+
let ensureOpen () =
61+
if connection.State <> ConnectionState.Open then
62+
connection.Open()
63+
64+
let newCommand (sql: string) (ps: SqlParam[]) =
65+
let cmd = connection.CreateCommand()
66+
cmd.CommandText <- sql
67+
68+
if not (isNull current) then
69+
cmd.Transaction <- current
70+
71+
for p in ps do
72+
let dp = cmd.CreateParameter()
73+
dp.ParameterName <- prefix + p.Name
74+
dp.Value <- Conv.toDbValue p.Value
75+
cmd.Parameters.Add dp |> ignore
76+
77+
cmd
78+
79+
let readAll (reader: IDataReader) : ResultSet =
80+
let columns = Array.init reader.FieldCount reader.GetName
81+
let rows = ResizeArray<SqlValue[]>()
82+
83+
while reader.Read() do
84+
rows.Add(Array.init reader.FieldCount (fun i -> Conv.toSqlValue (reader.GetValue i)))
85+
86+
{ Columns = columns; Rows = rows.ToArray() }
87+
88+
member _.Connection = connection
89+
90+
interface ISqlConnector with
91+
92+
member _.Query(sql, ps) =
93+
ensureOpen ()
94+
use cmd = newCommand sql ps
95+
use reader = cmd.ExecuteReader()
96+
readAll reader
97+
98+
member _.Execute(sql, ps) =
99+
ensureOpen ()
100+
use cmd = newCommand sql ps
101+
cmd.ExecuteNonQuery()
102+
103+
member _.Scalar(sql, ps) =
104+
ensureOpen ()
105+
use cmd = newCommand sql ps
106+
Conv.toSqlValue (cmd.ExecuteScalar())
107+
108+
member _.BeginTransaction() =
109+
ensureOpen ()
110+
let tx = connection.BeginTransaction()
111+
current <- tx
112+
113+
{ new ISqlTransaction with
114+
member _.Commit() =
115+
tx.Commit()
116+
tx.Dispose()
117+
current <- null
118+
119+
member _.Rollback() =
120+
tx.Rollback()
121+
tx.Dispose()
122+
current <- null
123+
}
124+
125+
member _.Close() =
126+
if connection.State <> ConnectionState.Closed then
127+
connection.Close()
128+
129+
interface IDisposable with
130+
member _.Dispose() = connection.Dispose()

‎src/SQLProvider.Fable.Ado/SQLProvider.Fable.Ado.fsproj‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
<Project Sdk="Microsoft.NET.Sdk">
2+
3+
<PropertyGroup>
4+
<TargetFramework>net8.0</TargetFramework>
5+
</PropertyGroup>
6+
7+
<ItemGroup>
8+
<Compile Include="AdoConnector.fs" />
9+
</ItemGroup>
10+
11+
<ItemGroup>
12+
<ProjectReference Include="..\SQLProvider.Fable.Core\SQLProvider.Fable.Core.fsproj" />
13+
</ItemGroup>
14+
15+
</Project>

0 commit comments

Comments
 (0)