Skip to content

Commit 6f10781

Browse files
fix(bigquery-jdbc): preserve microsecond precision for TIME in getString() and LocalTime conversions (#14715)
b/570669274 ## Summary Fixes sub-second truncation for BigQuery `TIME` values (including nested `ARRAY<TIME>` and `STRUCT<TIME>`) when accessed as strings or `java.time.LocalTime`, while keeping `java.sql.Time` strictly compliant with the JDK and JDBC specifications (`HH:mm:ss` format, millisecond backing). ## Problem BigQuery `TIME` supports microsecond precision (`HH:mm:ss.SSSSSS`). Previously: 1. `BigQueryTypeRegistry.createTimeDescriptor()` converted raw `TIME` values (`String` from REST JSON or `Long` microseconds from Arrow Storage Read API) to `String` and `LocalTime` by routing through `java.sql.Time`. Because `java.sql.Time` only stores milliseconds and `java.sql.Time.toString()` is specified by the JDK to return `"hh:mm:ss"`, fractional seconds were truncated or dropped. 2. `BigQueryBaseResultSet.getObject(int, Class<T>)` called `getObject(columnIndex)` (producing a truncated `java.sql.Time`) before converting to `LocalTime.class` or `String.class`. 3. `BigQueryBaseArray.toString()` and `BigQueryBaseStruct.toString()` formatted `TIME` elements/attributes via `java.sql.Time.toString()`, truncating fractional seconds to `"HH:mm:ss"` when calling `ResultSet.getString()` on `ARRAY<TIME>` or `STRUCT<TIME>` columns. ## Solution - **`BigQueryTemporalUtility` & `BigQueryTypeRegistry`**: - Added `TIME_FORMATTER` (`HH:mm:ss` with up to 9 optional fractional digits) in `BigQueryTemporalUtility`. - Updated `BigQueryTypeRegistry.createTimeDescriptor()` to convert raw `String` and `Long` (epoch microseconds) directly to `LocalTime` and `String` without intermediate boxing into `java.sql.Time`. - **`BigQueryBaseResultSet` (`getObject(int, Class<T>)`)**: - Exposed package-private `abstract Object getObjectInternal(int columnIndex)` on `BigQueryBaseResultSet` (implemented by the existing `getObjectInternal` methods in `BigQueryJsonResultSet` and `BigQueryArrowResultSet`). - For `StandardSQLTypeName.TIME` with target type `LocalTime.class` or `String.class`, convert directly from `getObjectInternal(columnIndex)` to preserve full microsecond precision. - **`BigQueryBaseArray` & `BigQueryBaseStruct` (`toString()`)**: - Added package-private `abstract Object getRawElement(int index)` on `BigQueryBaseArray` and `abstract Object getRawAttribute(int index)` on `BigQueryBaseStruct`, implemented in their JSON and Arrow subclasses. - Updated `BigQueryBaseArray.toString()` and `BigQueryBaseStruct.toString()` to format `StandardSQLTypeName.TIME` elements/attributes from their raw values via `BigQueryTypeRegistry.convert(..., StandardSQLTypeName.TIME, String.class, null)`. - `Array.getResultSet().getString(2)` also preserves full microsecond precision for `ARRAY<TIME>` as `BigQueryJsonResultSet` and `BigQueryArrowResultSet` read from the underlying raw values. ## Behavior Matrix | Access Method | Top-Level `TIME` | `ARRAY<TIME>` | `STRUCT<TIME>` | | :--- | :--- | :--- | :--- | | `rs.getString(...)` | `"14:30:00.123456"` | `"[14:30:00.123456, ...]"` | `"{\"t\":\"14:30:00.123456\"}"` | | `rs.getObject(..., LocalTime.class)` | `LocalTime` (microsecond precision) | N/A | N/A | | `rs.getArray(...).getResultSet().getString(2)` | N/A | `"14:30:00.123456"` | N/A | | `rs.getTime(...)` / `getArray()` / `getAttributes()` | Standard `java.sql.Time` (JDK spec compliant) | `java.sql.Time[]` | `Object[]` with `java.sql.Time` | ## Testing - Unit tests in `BigQueryTypeRegistryTest`, `BigQueryJsonArrayOfPrimitivesTest`, `BigQueryArrowArrayOfPrimitivesTest`, `BigQueryJsonStructTest`, and `BigQueryArrowStructTest`. - End-to-end integration test `testTimeMicrosecondPrecisionInArrayAndStruct` in `ITBigQueryJDBCTest` covering both REST JSON and Storage Read (Arrow) pipelines.
1 parent 11bc071 commit 6f10781

18 files changed

Lines changed: 220 additions & 38 deletions

‎java-bigquery-jdbc/src/main/java/com/google/cloud/bigquery/jdbc/BigQueryArrowArray.java‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -111,10 +111,7 @@ public void free() {
111111
@Override
112112
Object getCoercedValue(int index) throws SQLException {
113113
LOG.finestTrace("getCoercedValue");
114-
Object value = this.values.get(index);
115-
if (value instanceof Text) {
116-
value = value.toString();
117-
}
114+
Object value = getRawElement(index);
118115
if (this.arrayOfStruct) {
119116
return new BigQueryArrowStruct(
120117
schema.getSubFields(),
@@ -127,4 +124,10 @@ Object getCoercedValue(int index) throws SQLException {
127124
}
128125
return BigQueryTypeRegistry.convert(value, this.schema.getType().getStandardType(), null);
129126
}
127+
128+
@Override
129+
Object getRawElement(int index) {
130+
Object value = this.values.get(index);
131+
return (value instanceof Text) ? value.toString() : value;
132+
}
130133
}

‎java-bigquery-jdbc/src/main/java/com/google/cloud/bigquery/jdbc/BigQueryArrowResultSet.java‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -328,7 +328,8 @@ else if (this.currentBatchRowIndex < this.vectorSchemaRoot.getRowCount()) {
328328
return false;
329329
}
330330

331-
private Object getObjectInternal(int columnIndex) throws SQLException {
331+
@Override
332+
Object getObjectInternal(int columnIndex) throws SQLException {
332333
LOG.finestTrace("getObjectInternal");
333334
checkClosed();
334335
Object value;

‎java-bigquery-jdbc/src/main/java/com/google/cloud/bigquery/jdbc/BigQueryArrowStruct.java‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -36,8 +36,7 @@
3636
class BigQueryArrowStruct extends BigQueryBaseStruct {
3737

3838
private final FieldList schema;
39-
40-
private final JsonStringHashMap<?, ?> values;
39+
private final List<?> values;
4140

4241
BigQueryArrowStruct(FieldList schema, JsonStringHashMap<?, ?> values) {
4342
this(schema, values, false, BigQueryJdbcResultSetLogger.getLogger(BigQueryArrowStruct.class));
@@ -50,7 +49,7 @@ class BigQueryArrowStruct extends BigQueryBaseStruct {
5049
BigQueryJdbcResultSetLogger log) {
5150
super(enableTimestampPicos, log);
5251
this.schema = schema;
53-
this.values = values;
52+
this.values = values == null ? null : new ArrayList<>(values.values());
5453
}
5554

5655
@Override
@@ -67,22 +66,17 @@ public Object[] getAttributes() throws SQLException {
6766
if (this.values == null) {
6867
return attributes;
6968
}
70-
List<?> structValues = new ArrayList<>(this.values.values());
7169

7270
for (int index = 0; index < size; index++) {
7371
Field currentSchema = this.schema.get(index);
74-
Object currentValue = structValues.get(index);
75-
Object coercedValue = getValue(currentSchema, currentValue);
72+
Object coercedValue = getValue(currentSchema, getRawAttribute(index));
7673
Array.set(attributes, index, coercedValue);
7774
}
7875
return attributes;
7976
}
8077

8178
private Object getValue(Field currentSchema, Object currentValue) throws SQLException {
8279
LOG.finestTrace("getValue");
83-
if (currentValue instanceof Text) {
84-
currentValue = currentValue.toString();
85-
}
8680
if (isArray(currentSchema)) {
8781
return new BigQueryArrowArray(
8882
currentSchema,
@@ -107,4 +101,10 @@ private Object getValue(Field currentSchema, Object currentValue) throws SQLExce
107101
return BigQueryTypeRegistry.convert(
108102
currentValue, currentSchema.getType().getStandardType(), null);
109103
}
104+
105+
@Override
106+
Object getRawAttribute(int index) {
107+
Object currentValue = this.values.get(index);
108+
return (currentValue instanceof Text) ? currentValue.toString() : currentValue;
109+
}
110110
}

‎java-bigquery-jdbc/src/main/java/com/google/cloud/bigquery/jdbc/BigQueryBaseArray.java‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -159,6 +159,8 @@ protected Class<?> getTargetClass() {
159159

160160
abstract Object getCoercedValue(int index) throws SQLException;
161161

162+
abstract Object getRawElement(int index);
163+
162164
static boolean isArray(Field currentSchema) {
163165
return currentSchema.getMode() == REPEATED;
164166
}
@@ -191,6 +193,21 @@ public String toString() {
191193
})
192194
.collect(Collectors.joining(", ", "[", "]"));
193195
}
196+
if (this.schema.getType().getStandardType() == StandardSQLTypeName.TIME) {
197+
StringBuilder sb = new StringBuilder("[");
198+
for (int i = 0; i < array.length; i++) {
199+
if (i > 0) {
200+
sb.append(", ");
201+
}
202+
sb.append(
203+
array[i] == null
204+
? "null"
205+
: BigQueryTypeRegistry.convert(
206+
getRawElement(i), StandardSQLTypeName.TIME, String.class, null));
207+
}
208+
sb.append("]");
209+
return sb.toString();
210+
}
194211
return Arrays.deepToString(array);
195212
} catch (SQLException e) {
196213
LOG.warning("Error converting array to string");

‎java-bigquery-jdbc/src/main/java/com/google/cloud/bigquery/jdbc/BigQueryBaseResultSet.java‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@
5252
import java.sql.Statement;
5353
import java.sql.Time;
5454
import java.sql.Timestamp;
55+
import java.time.LocalTime;
5556
import java.time.ZoneId;
5657
import java.util.Calendar;
5758
import java.util.List;
@@ -310,10 +311,18 @@ protected Object getObject(int columnIndex, ZoneId zone) throws SQLException {
310311
return getObject(columnIndex);
311312
}
312313

314+
abstract Object getObjectInternal(int columnIndex) throws SQLException;
315+
313316
@Override
314317
public <T> T getObject(int columnIndex, Class<T> type) throws SQLException {
315318
LOG.finestTrace("getObject");
316319
try {
320+
if (type == LocalTime.class || type == String.class) {
321+
StandardSQLTypeName bqType = getStandardSQLTypeName(columnIndex);
322+
if (bqType == StandardSQLTypeName.TIME) {
323+
return BigQueryTypeRegistry.convert(getObjectInternal(columnIndex), bqType, type, null);
324+
}
325+
}
317326
Object value = getObject(columnIndex);
318327
if (value == null) {
319328
return null;

‎java-bigquery-jdbc/src/main/java/com/google/cloud/bigquery/jdbc/BigQueryBaseStruct.java‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,8 @@ abstract class BigQueryBaseStruct implements java.sql.Struct {
5050

5151
abstract FieldList getSchema();
5252

53+
abstract Object getRawAttribute(int index);
54+
5355
@Override
5456
public final String getSQLTypeName() throws SQLException {
5557
return StandardSQLTypeName.STRUCT.name();
@@ -95,6 +97,13 @@ public String toString() {
9597
sb.append("\"")
9698
.append(BigQueryTemporalUtility.formatTimestampToUtcString((Timestamp) value))
9799
.append("\"");
100+
} else if (value instanceof Time
101+
&& field.getType().getStandardType() == StandardSQLTypeName.TIME) {
102+
sb.append("\"")
103+
.append(
104+
BigQueryTypeRegistry.convert(
105+
getRawAttribute(i), StandardSQLTypeName.TIME, String.class, null))
106+
.append("\"");
98107
} else if (value instanceof Timestamp || value instanceof Date || value instanceof Time) {
99108
sb.append("\"").append(value.toString()).append("\"");
100109
} else if (value instanceof byte[]) {

‎java-bigquery-jdbc/src/main/java/com/google/cloud/bigquery/jdbc/BigQueryJsonArray.java‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -112,8 +112,8 @@ public void free() {
112112
@Override
113113
Object getCoercedValue(int index) throws SQLException {
114114
LOG.finestTrace("getCoercedValue");
115-
FieldValue fieldValue = this.values.get(index);
116-
if (fieldValue == null || fieldValue.isNull()) {
115+
FieldValue fieldValue = getRawElement(index);
116+
if (fieldValue == null) {
117117
return null;
118118
}
119119
if (this.arrayOfStruct) {
@@ -131,4 +131,10 @@ Object getCoercedValue(int index) throws SQLException {
131131
}
132132
return BigQueryTypeRegistry.convert(fieldValue, this.schema.getType().getStandardType(), null);
133133
}
134+
135+
@Override
136+
FieldValue getRawElement(int index) {
137+
FieldValue fieldValue = this.values.get(index);
138+
return (fieldValue == null || fieldValue.isNull()) ? null : fieldValue;
139+
}
134140
}

‎java-bigquery-jdbc/src/main/java/com/google/cloud/bigquery/jdbc/BigQueryJsonResultSet.java‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -339,7 +339,8 @@ static boolean isRangeTimestamp(Field field) {
339339
* @param columnIndex the first column is 1, the second is 2, ...
340340
* @return an instance of {@link FieldValue} represents value at <code>columnIndex</code> column.
341341
*/
342-
private FieldValue getObjectInternal(int columnIndex) throws SQLException {
342+
@Override
343+
FieldValue getObjectInternal(int columnIndex) throws SQLException {
343344
checkClosed();
344345
LOG.finestTrace("getObjectInternal");
345346
FieldValue value;

‎java-bigquery-jdbc/src/main/java/com/google/cloud/bigquery/jdbc/BigQueryJsonStruct.java‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -59,19 +59,21 @@ public Object[] getAttributes() throws SQLException {
5959
LOG.finestTrace("getAttributes");
6060
int size = schema.size();
6161
Object[] attributes = (Object[]) Array.newInstance(Object.class, size);
62+
if (this.values == null) {
63+
return attributes;
64+
}
6265

6366
for (int index = 0; index < size; index++) {
6467
Field currentSchema = schema.get(index);
65-
FieldValue currentValue = values == null ? null : values.get(index);
66-
Object coercedValue = getValue(currentSchema, currentValue);
68+
Object coercedValue = getValue(currentSchema, getRawAttribute(index));
6769
Array.set(attributes, index, coercedValue);
6870
}
6971
return attributes;
7072
}
7173

7274
private Object getValue(Field currentSchema, FieldValue currentValue) throws SQLException {
7375
LOG.finestTrace("getValue");
74-
if (currentValue == null || currentValue.isNull()) {
76+
if (currentValue == null) {
7577
return null;
7678
}
7779
if (isArray(currentSchema)) {
@@ -94,4 +96,10 @@ private Object getValue(Field currentSchema, FieldValue currentValue) throws SQL
9496
return BigQueryTypeRegistry.convert(
9597
currentValue, currentSchema.getType().getStandardType(), null);
9698
}
99+
100+
@Override
101+
FieldValue getRawAttribute(int index) {
102+
FieldValue currentValue = this.values.get(index);
103+
return (currentValue == null || currentValue.isNull()) ? null : currentValue;
104+
}
97105
}

‎java-bigquery-jdbc/src/main/java/com/google/cloud/bigquery/jdbc/BigQueryTemporalUtility.java‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,11 @@ final class BigQueryTemporalUtility {
5757
.appendFraction(ChronoField.NANO_OF_SECOND, 1, 9, false)
5858
.toFormatter()
5959
.withZone(ZoneOffset.UTC);
60+
static final DateTimeFormatter TIME_FORMATTER =
61+
new DateTimeFormatterBuilder()
62+
.appendPattern("HH:mm:ss")
63+
.appendFraction(ChronoField.NANO_OF_SECOND, 0, 9, true)
64+
.toFormatter();
6065
static final String TIMESTAMP_PICOSECONDS_TYPE_NAME = "TIMESTAMP_PICOSECONDS";
6166

6267
private BigQueryTemporalUtility() {}

0 commit comments

Comments
 (0)