Skip to content

NUMERIC reads: full digits arrive, the decode drops them #826

Description

@florinutz

The read-side follow-up promised in crate/sqlalchemy-cratedb#300, with the measurement #652 was missing. As @matriv said there, the full number reaches the client; the digits are dropped by this driver's decode.

On CrateDB 6.4.2, with 1.234567890123456789012345 stored in a NUMERIC(38, 24) column, the raw /_sql response carries every digit:

{"cols":["n"],"rows":[[1.234567890123456789012345]],"rowcount":1,"duration":105.078}

cursor.fetchone()[0] on the same query returns 1.2345678901234567, a float. The response is decoded with orjson.loads (http.py line 244), which parses every JSON number to a float64 and has no parse_float hook; stdlib json.loads(raw, parse_float=Decimal) on those bytes returns Decimal('1.234567890123456789012345').

Writes are already exact, since Decimal goes out as a string (#751). A stdlib decode with parse_float=Decimal would make reads match, but it changes the returned type for every float column and gives up orjson's speed, so it likely wants to be a connection option rather than the default.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions