Skip to content

Commit 4fe0f3e

Browse files
ThoriumThorium
committed
🌅
1 parent 06c9b97 commit 4fe0f3e

50 files changed

Lines changed: 10435 additions & 1212 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.fantomasignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Golden file: byte-identical to what CodeGen emits, and a test asserts so.
2+
# Formatting it breaks the comparison -- regenerate it instead
3+
# (SQLPROVIDER_FABLE_REGENERATE=1).
4+
tests/SQLProvider.Fable.SmokeTests/GeneratedSchema.fs

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
bin/
33
obj/
44
*.user
5+
.vs/
56

67
# Rust / Cargo
78
target/

‎DESIGN.md‎

Lines changed: 244 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,244 @@
1+
# Design notes
2+
3+
Why SQLProvider.Fable is shaped the way it is. The user-facing documentation is
4+
in [README.md](README.md); the catalogue of Fable compiler gaps found while
5+
building this, with minimal repros, is in [GAPS.md](GAPS.md).
6+
7+
## Why not just port SQLProvider's runtime?
8+
9+
Almost none of `System.Data` survives the trip to Rust, and almost none of it
10+
needs to. Tracing `executeQuery` in SQLProvider's `SqlRuntime.Linq.fs`, one query
11+
touches roughly twenty ADO members:
12+
13+
| Type | Members used |
14+
|---|---|
15+
| `IDbConnection` | `ConnectionString`, `Open`, `Close`, `State`, `Dispose` |
16+
| `IDbCommand` | `CommandText`, `CommandTimeout`, `Parameters.Add`, `ExecuteReader`, `ExecuteScalar`, `ExecuteNonQuery` |
17+
| `IDbDataParameter` | `ParameterName`, `Value`, `DbType`, `Direction` |
18+
| `IDataReader` | `Read`, `FieldCount`, `GetName`, `GetValue` |
19+
| `ConnectionState`, `DBNull`, `IDbTransaction`, `IsolationLevel` | — |
20+
21+
Everything else in SQLProvider's `ISqlProvider` — `GetTables`, `GetColumns`,
22+
`GetRelationships`, `GetSprocs`, `CreateTypeMappings`, the `DataTable` and SSDT
23+
machinery — is schema introspection. That runs on .NET at code-generation time
24+
and never reaches a Fable target.
25+
26+
But those twenty members should not be ported either, because each collides with
27+
Rust:
28+
29+
1. **`obj` values.** `reader.GetValue i : obj` becomes `Lrc<dyn Any>` in Rust, so
30+
every column read is a downcast. Replaced by the `SqlValue` union.
31+
2. **`DBNull`** has no Rust analogue and shouldn't get one. It's `option`.
32+
3. **A mutable `.Parameters` collection.** sqlx wants parameters supplied at
33+
execute time, not accumulated on a command object.
34+
4. **`Open`/`Close`/`State`.** sqlx connects on construct and closes on drop.
35+
There is no ADO connection-state machine to model.
36+
5. **`System.Transactions.TransactionScope`** — SQLProvider's ambient-scope write
37+
model has no Rust counterpart. Transactions here are explicit.
38+
39+
So the native surface is one interface with eight members
40+
(`src/SQLProvider.Fable.Core/Connector.fs`), and everything else is portable F#
41+
on top of it.
42+
43+
## How it works
44+
45+
### The interface is async-only
46+
47+
PostgreSQL and MySQL are async-native, and a synchronous interface would park a
48+
thread per query. SQLite has no async to offer, so its connector completes
49+
immediately — the cost there is writing `let!` for work that never yields.
50+
51+
On Rust the query runs on a tokio runtime and the result is handed back over a
52+
oneshot channel, which Fable's `Async` awaits. The sqlx future never leaves
53+
tokio; only the receiver crosses into Fable's executor. That keeps the caller's
54+
thread free while the database is busy, which is the whole point, and it keeps
55+
`Async`'s `Send + Sync` bound off sqlx's futures.
56+
57+
### One SQL text, three placeholder syntaxes
58+
59+
sqlx does *not* normalise placeholders: PostgreSQL wants `$1`, MySQL and SQLite
60+
want a bare `?`, ADO.NET providers want `@name`. Query code here is always written
61+
with `@name`, and `Dialect.bind` rewrites it on the way down — reordering the
62+
values to match, duplicating an entry for a name used twice (neither positional
63+
form can refer back), and leaving alone any `@` that is not a parameter, such as
64+
an address inside a literal or a MySQL `@@version`.
65+
66+
This is pure logic with no database in it, which makes it the one part of the
67+
multi-vendor story that is fully testable anywhere: `DialectTests.fs` covers all
68+
three styles, and the .NET suite additionally runs the whole smoke suite a second
69+
time in `Numbered` mode against real SQLite, which accepts `$1` as a parameter
70+
name.
71+
72+
### Result sets are materialised eagerly
73+
74+
Not laziness: a sqlx row stream borrows the connection that produced it, so a
75+
reader kept alive across F# calls would need a self-referential struct.
76+
SQLProvider's own `Sql.dataReaderToArray` already reads the whole reader into an
77+
array before projecting, so nothing is lost.
78+
79+
### One connection, not a pool
80+
81+
A pool would hand successive statements to different connections, which breaks
82+
`BEGIN`/`COMMIT` — they are plain statements here — and, for `sqlite::memory:`,
83+
would give each connection its own empty database.
84+
85+
### Mapping is generated, not reflected
86+
87+
Fable's Rust `FSharpPropertyInfo` carries only a name (see fable-library-rust
88+
`QuotationTypes.fs`), so `PropertyInfo.PropertyType` is unavailable and a generic
89+
"convert each column to whatever the record field wants" mapper has nothing to
90+
dispatch on. Mappers state conversions explicitly instead:
91+
92+
```fsharp
93+
let ofRow (r: SqlRow) : Customer =
94+
{ CustomerId = Row.int r "CustomerId"
95+
Name = Row.text r "Name"
96+
Country = Row.textOpt r "Country"
97+
Balance = Row.float r "Balance" }
98+
```
99+
100+
This is what the schema code generator will emit per table. It is also faster and
101+
clearer than reflection on .NET.
102+
103+
### The native boundary carries primitives only
104+
105+
`sqlx_native.rs` never sees a Fable-generated `SqlValue`. Values cross as an
106+
integer tag plus a typed accessor call, so a change in how Fable lays out unions
107+
cannot silently break the shim.
108+
109+
### decimal, DateTime and Guid travel as text
110+
111+
sqlx's `Any` driver has no decimal, date or uuid kind, and SQLite has no such
112+
storage class either. Every backend encodes these three through the same
113+
`Convert` helpers, so identical bytes are stored whichever one wrote them, and
114+
the readers parse them back invariantly. The suite asserts the stored text, not
115+
only the round trip, because a locale-sensitive conversion sneaking in would
116+
still round-trip on the machine that wrote it.
117+
118+
Verified across engines rather than asserted: after a PostgreSQL run, `psql`
119+
shows `1234.5678` and `2026-08-27T10:30:00.0000000` in the table — the same bytes
120+
the .NET, Rust/SQLite and Node runs write.
121+
122+
`Convert` is hand-rolled, and every piece of it is there because a target
123+
disagreed:
124+
125+
- `Decimal`'s culture-taking overloads do not exist on Rust, and the culture-less
126+
ones throw under a comma-separator locale such as fi-FI.
127+
- `DateTime.ToString "o"` emits seven fractional digits on .NET and Rust but
128+
three on JavaScript, which goes through `Date.toISOString`. The format is
129+
therefore spelled out digit by digit rather than borrowed — caught by the
130+
"date stored as ISO-8601" assertion the first time the suite ran on Node.
131+
- `DateTime.AddTicks` is not mapped on BEAM (`DateTimeOffset.AddTicks` is), so
132+
the parser reconstructs from a tick count instead.
133+
134+
## Where this differs from SQLProvider
135+
136+
### What the query syntax cannot be
137+
138+
It is not `query { }`. SQLProvider writes
139+
`join order in ctx.Main.Orders on (customer.CustomerId = order.CustomerId)`;
140+
the closest here is
141+
`join Order.table (Customer.CustomerId == Order.CustomerId)` -- the table is
142+
named rather than bound to a variable, and the comparison needs its own
143+
operator. `select (Customer.Name, Customer.Country)` does read like
144+
SQLProvider's.
145+
146+
The gap is the price of having no expression trees: nothing can inspect a
147+
lambda, so every part of a query has to be a value the code builds rather than
148+
code the compiler hands over. Tuple projections into a typed *result* are the
149+
clearest thing still missing -- `select` takes typed columns, but the shaping
150+
into a record happens in the generated `ofRow` rather than in the query.
151+
152+
### On the objection this drew in SQLProvider itself
153+
154+
SQLProvider PR [#457](https://github.com/fsprojects/SQLProvider/pull/457) tried
155+
the same idea there -- a `sqlQuery` CE exposing only the operators that actually
156+
work -- and was closed on the maintainer's view that *"we should implement the
157+
whole LINQ support rather than limiting ourselves to some custom developer
158+
interface which is not familiar to anyone"*.
159+
160+
That objection is right where it was made and does not transfer here. In
161+
SQLProvider, `query { }` works and a narrower facade is a step back from
162+
finishing it. On a Fable target `System.Linq.Expressions` does not exist at all,
163+
so there is no LINQ support to complete and nothing to be narrower than.
164+
165+
The familiarity half of the objection does transfer, and it is why the
166+
operations are named after SQLProvider's rather than invented: a query ported
167+
from there mostly reads the same. Where this goes further than LINQ can --
168+
`fromAs`/`joinAs` for aliases, `whereAny`, `selectAs`, `sortByExpr` -- it is
169+
because the AST underneath is SQL, not because the CE is doing something clever.
170+
171+
That PR also ran into the async terminators (`countAsync` and friends) not
172+
fitting inside the CE. They are outside it here on purpose: the CE builds a
173+
`Query` and `Db.query`/`Db.count`/`Db.tryHead` run it, so nothing about
174+
execution has to be expressible as a custom operation.
175+
176+
### Why not `query { ... }`
177+
178+
SQLProvider's `query { }` builds a `System.Linq.Expressions` tree, which no
179+
Fable target has. F# *quotations* do survive to Rust, JavaScript and BEAM, so a
180+
`where <@ fun c -> c.Country = "UK" @>` front-end is the natural next step — but
181+
not yet, for two measured reasons:
182+
183+
- `DerivedPatterns.AndAlso`/`OrElse` and captured locals arriving as `Value`
184+
rather than `Var` both live on an **unmerged** Fable branch
185+
(`feat/quotation-derived-patterns`). On stock Fable, `&&` inside a quotation
186+
does not compile at all.
187+
- `PropertyGet` hands back a `PropertyInfo` on Rust but a bare string on the
188+
other targets, so reading a column name out of a quotation needs a per-target
189+
shim.
190+
191+
The AST underneath is what a quotation front-end would translate *into*, so
192+
nothing here is wasted when that lands.
193+
194+
## How the generated code is verified
195+
196+
`tests/SQLProvider.Fable.SmokeTests/GeneratedSchema.fs` is a checked-in
197+
generated file, compiled by the shared test project -- so it is compiled and
198+
exercised on .NET, Rust, JavaScript and BEAM, which is the only assertion that
199+
really matters about a generator. A test regenerates it from a live SQLite
200+
schema and compares, so the checked-in copy cannot drift
201+
(`SQLPROVIDER_FABLE_REGENERATE=1` rewrites it after a deliberate change).
202+
203+
## Fable Rust codegen gaps found while building this
204+
205+
Nine, all worked around here, all catalogued with repros in `GAPS.md`
206+
(G7, G8, G9, G20, G21, G23, G24, G25, G26). The last four came out of the query
207+
builder — the first code in this project with a non-trivial amount of pattern
208+
matching in it, and the first computation expression:
209+
210+
1. **Object expression calling back into its enclosing object.** Generates a
211+
struct field literally named `_`, a reserved identifier in Rust. Matches the
212+
cases already commented out in Fable's own `MiscTests.fs`.
213+
2. **`this :> ISomeInterface` inside the type's own interface implementation.**
214+
`interface_cast!` expands to a plain `as`, which Rust rejects as a
215+
non-primitive cast. Worked around by moving the logic to free functions.
216+
3. **Doubly-nested closure over a captured value.** `Array.init n (fun r ->
217+
Array.init m (fun c -> ...raw...))` clones `raw` into the outer `Fn` closure
218+
but *moves* it into the inner one, so it does not borrow-check. Worked around
219+
with explicit loops and a top-level helper.
220+
4. **The async builder has no `Combine`.** Any `async` block with a computation
221+
in statement position — `if cond then do! x`, a `for`, a `while` — emits
222+
`singleton.Combine(..)`, and neither `singleton` nor the method exists in
223+
`fable-library-rust`. Worked around by binding the offending `match` to a
224+
name. This one is a real constraint on generated code, not just on ours.
225+
5. **`Async<'T>` where `'T` is an interface** emits a trait object with no
226+
`Send + Sync` bound, which `Async` requires. Worked around by flattening
227+
`BeginTransaction`/`Commit`/`Rollback` onto `ISqlConnector`.
228+
6. **A fieldless DU matched with unit-valued branches** emits an integer tag
229+
pattern against the value itself. Worked around with equality.
230+
7. **A record field typed as an imported library class** (`StringBuilder`) is
231+
not boxed in the record's reflection info. Worked around with a
232+
`ResizeArray<string>`.
233+
8. **A guarded nested pattern** makes the decision-tree lowering `mem::zeroed`
234+
every bound array, list and option — which panics on an `Arc` at runtime,
235+
not at compile time. Worked around by moving the guard inside the branch.
236+
This is the one to fix upstream first: it compiles cleanly and then dies.
237+
9. **A member taking `unit`** is declared with the parameter but called without
238+
it, which breaks any computation-expression builder. Worked around by making
239+
the parameter generic.
240+
241+
Also worth knowing: Fable emits the `#[path = "..."] mod ...` declaration for a
242+
`.rs` pulled in by `importAll`, but does not copy the file — the build script
243+
stages it.
244+

0 commit comments

Comments
 (0)