Repository navigation
Commit 6f10781
authored
fix(bigquery-jdbc): preserve microsecond precision for
b/570669274
## Summary
Fixes sub-second truncation for BigQuery `TIME` values (including nested
`ARRAY<TIME>` and `STRUCT<TIME>`) when accessed as strings or
`java.time.LocalTime`, while keeping `java.sql.Time` strictly compliant
with the JDK and JDBC specifications (`HH:mm:ss` format, millisecond
backing).
## Problem
BigQuery `TIME` supports microsecond precision (`HH:mm:ss.SSSSSS`).
Previously:
1. `BigQueryTypeRegistry.createTimeDescriptor()` converted raw `TIME`
values (`String` from REST JSON or `Long` microseconds from Arrow
Storage Read API) to `String` and `LocalTime` by routing through
`java.sql.Time`. Because `java.sql.Time` only stores milliseconds and
`java.sql.Time.toString()` is specified by the JDK to return
`"hh:mm:ss"`, fractional seconds were truncated or dropped.
2. `BigQueryBaseResultSet.getObject(int, Class<T>)` called
`getObject(columnIndex)` (producing a truncated `java.sql.Time`) before
converting to `LocalTime.class` or `String.class`.
3. `BigQueryBaseArray.toString()` and `BigQueryBaseStruct.toString()`
formatted `TIME` elements/attributes via `java.sql.Time.toString()`,
truncating fractional seconds to `"HH:mm:ss"` when calling
`ResultSet.getString()` on `ARRAY<TIME>` or `STRUCT<TIME>` columns.
## Solution
- **`BigQueryTemporalUtility` & `BigQueryTypeRegistry`**:
- Added `TIME_FORMATTER` (`HH:mm:ss` with up to 9 optional fractional
digits) in `BigQueryTemporalUtility`.
- Updated `BigQueryTypeRegistry.createTimeDescriptor()` to convert raw
`String` and `Long` (epoch microseconds) directly to `LocalTime` and
`String` without intermediate boxing into `java.sql.Time`.
- **`BigQueryBaseResultSet` (`getObject(int, Class<T>)`)**:
- Exposed package-private `abstract Object getObjectInternal(int
columnIndex)` on `BigQueryBaseResultSet` (implemented by the existing
`getObjectInternal` methods in `BigQueryJsonResultSet` and
`BigQueryArrowResultSet`).
- For `StandardSQLTypeName.TIME` with target type `LocalTime.class` or
`String.class`, convert directly from `getObjectInternal(columnIndex)`
to preserve full microsecond precision.
- **`BigQueryBaseArray` & `BigQueryBaseStruct` (`toString()`)**:
- Added package-private `abstract Object getRawElement(int index)` on
`BigQueryBaseArray` and `abstract Object getRawAttribute(int index)` on
`BigQueryBaseStruct`, implemented in their JSON and Arrow subclasses.
- Updated `BigQueryBaseArray.toString()` and
`BigQueryBaseStruct.toString()` to format `StandardSQLTypeName.TIME`
elements/attributes from their raw values via
`BigQueryTypeRegistry.convert(..., StandardSQLTypeName.TIME,
String.class, null)`.
- `Array.getResultSet().getString(2)` also preserves full microsecond
precision for `ARRAY<TIME>` as `BigQueryJsonResultSet` and
`BigQueryArrowResultSet` read from the underlying raw values.
## Behavior Matrix
| Access Method | Top-Level `TIME` | `ARRAY<TIME>` | `STRUCT<TIME>` |
| :--- | :--- | :--- | :--- |
| `rs.getString(...)` | `"14:30:00.123456"` | `"[14:30:00.123456, ...]"`
| `"{\"t\":\"14:30:00.123456\"}"` |
| `rs.getObject(..., LocalTime.class)` | `LocalTime` (microsecond
precision) | N/A | N/A |
| `rs.getArray(...).getResultSet().getString(2)` | N/A |
`"14:30:00.123456"` | N/A |
| `rs.getTime(...)` / `getArray()` / `getAttributes()` | Standard
`java.sql.Time` (JDK spec compliant) | `java.sql.Time[]` | `Object[]`
with `java.sql.Time` |
## Testing
- Unit tests in `BigQueryTypeRegistryTest`,
`BigQueryJsonArrayOfPrimitivesTest`,
`BigQueryArrowArrayOfPrimitivesTest`, `BigQueryJsonStructTest`, and
`BigQueryArrowStructTest`.
- End-to-end integration test
`testTimeMicrosecondPrecisionInArrayAndStruct` in `ITBigQueryJDBCTest`
covering both REST JSON and Storage Read (Arrow) pipelines.TIME in getString() and LocalTime conversions (#14715)1 parent 11bc071 commit 6f10781
18 files changed
Lines changed: 220 additions & 38 deletions
File tree
- java-bigquery-jdbc/src
- main/java/com/google/cloud/bigquery/jdbc
- test/java/com/google/cloud/bigquery/jdbc
- it
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
111 | 111 | | |
112 | 112 | | |
113 | 113 | | |
114 | | - | |
115 | | - | |
116 | | - | |
117 | | - | |
| 114 | + | |
118 | 115 | | |
119 | 116 | | |
120 | 117 | | |
| |||
127 | 124 | | |
128 | 125 | | |
129 | 126 | | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
130 | 133 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
328 | 328 | | |
329 | 329 | | |
330 | 330 | | |
331 | | - | |
| 331 | + | |
| 332 | + | |
332 | 333 | | |
333 | 334 | | |
334 | 335 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
39 | | - | |
40 | | - | |
| 39 | + | |
41 | 40 | | |
42 | 41 | | |
43 | 42 | | |
| |||
50 | 49 | | |
51 | 50 | | |
52 | 51 | | |
53 | | - | |
| 52 | + | |
54 | 53 | | |
55 | 54 | | |
56 | 55 | | |
| |||
67 | 66 | | |
68 | 67 | | |
69 | 68 | | |
70 | | - | |
71 | 69 | | |
72 | 70 | | |
73 | 71 | | |
74 | | - | |
75 | | - | |
| 72 | + | |
76 | 73 | | |
77 | 74 | | |
78 | 75 | | |
79 | 76 | | |
80 | 77 | | |
81 | 78 | | |
82 | 79 | | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | 80 | | |
87 | 81 | | |
88 | 82 | | |
| |||
107 | 101 | | |
108 | 102 | | |
109 | 103 | | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
110 | 110 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
159 | 159 | | |
160 | 160 | | |
161 | 161 | | |
| 162 | + | |
| 163 | + | |
162 | 164 | | |
163 | 165 | | |
164 | 166 | | |
| |||
191 | 193 | | |
192 | 194 | | |
193 | 195 | | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
194 | 211 | | |
195 | 212 | | |
196 | 213 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
| 55 | + | |
55 | 56 | | |
56 | 57 | | |
57 | 58 | | |
| |||
310 | 311 | | |
311 | 312 | | |
312 | 313 | | |
| 314 | + | |
| 315 | + | |
313 | 316 | | |
314 | 317 | | |
315 | 318 | | |
316 | 319 | | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
317 | 326 | | |
318 | 327 | | |
319 | 328 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
| 53 | + | |
| 54 | + | |
53 | 55 | | |
54 | 56 | | |
55 | 57 | | |
| |||
95 | 97 | | |
96 | 98 | | |
97 | 99 | | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
98 | 107 | | |
99 | 108 | | |
100 | 109 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
112 | 112 | | |
113 | 113 | | |
114 | 114 | | |
115 | | - | |
116 | | - | |
| 115 | + | |
| 116 | + | |
117 | 117 | | |
118 | 118 | | |
119 | 119 | | |
| |||
131 | 131 | | |
132 | 132 | | |
133 | 133 | | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
134 | 140 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
339 | 339 | | |
340 | 340 | | |
341 | 341 | | |
342 | | - | |
| 342 | + | |
| 343 | + | |
343 | 344 | | |
344 | 345 | | |
345 | 346 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
62 | 65 | | |
63 | 66 | | |
64 | 67 | | |
65 | | - | |
66 | | - | |
| 68 | + | |
67 | 69 | | |
68 | 70 | | |
69 | 71 | | |
70 | 72 | | |
71 | 73 | | |
72 | 74 | | |
73 | 75 | | |
74 | | - | |
| 76 | + | |
75 | 77 | | |
76 | 78 | | |
77 | 79 | | |
| |||
94 | 96 | | |
95 | 97 | | |
96 | 98 | | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
97 | 105 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
60 | 65 | | |
61 | 66 | | |
62 | 67 | | |
| |||
0 commit comments