Skip to content

Commit b25100d

Browse files
test(bigquery-jdbc): add picosecond timestamp integration tests (#14392)
b/556664087 This PR adds ITs and and fixes one gap the ITs uncovered. ## Production fix `BigQueryConnection.getBigQueryConnection()` now sets `DataFormatOptions.TimestampFormatOptions.ISO8601_STRING` when `EnableTimestampPicos=1`. The REST JSON read path was silently truncating to microseconds. BigQuery serializes `TIMESTAMP` as FLOAT64 epoch seconds by default, which cannot represent sub-microsecond digits; `ISO8601_STRING` is the only output format that carries the full 12-digit fraction. The Arrow path was already correct, which is why unit tests and metadata assertions passed while REST values did not. Gated on the property so the wire format is unchanged for existing users. ## Tests New `ITJdbcTimestampPicosTest` (11 tests): Arrow and REST read paths, complex types (`ARRAY`/`STRUCT`), `ResultSetMetaData`, `DatabaseMetaData.getColumns`, `PreparedStatement` round trips, Legacy SQL rejection, and timezone invariance. Registered in the presubmit, nightly, and driver-agnostic suites. Six tests carry `@Tag("advanced")` where behavior is driver-specific; the remaining five were verified to pass against other drivers. ## Known limitation BigQuery truncates `TIMESTAMP`-typed query parameters to microseconds even against a `TIMESTAMP(12)` column (b/419328655), so `setTimestamp`/`setObject` cannot write picosecond values. `setString` works, since the value is coerced server side. Documented in `USER_GUIDE.md` and asserted in the tests so the expectations flip loudly when the backend fix lands.
1 parent c1aac66 commit b25100d

8 files changed

Lines changed: 610 additions & 0 deletions

File tree

‎java-bigquery-jdbc/docs/USER_GUIDE.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,6 +213,22 @@ String url = "jdbc:bigquery://https://bigquery.googleapis.com:443"
213213
| :--- | :---: | :--- |
214214
| `EnableTimestampPicos` | `false` | Enables 12-digit picosecond precision for `TIMESTAMP(12)` data types. When enabled, `TIMESTAMP(12)` columns are retrieved with 12 fractional digits formatted in UTC (`YYYY-MM-DD HH:MM:SS.ffffffffffff`) via `getString()` and `getObject()`, typed as `Types.VARCHAR` (`12`) with type name `"TIMESTAMP_PICOSECONDS"`. Picosecond precision is incompatible with Legacy SQL (`QueryDialect=BIG_QUERY`). |
215215

216+
> [!IMPORTANT]
217+
> **Writing picosecond values with a `PreparedStatement`.** BigQuery currently truncates
218+
> `TIMESTAMP`-typed query parameters to microsecond precision, even when the target column is
219+
> `TIMESTAMP(12)`. Binding a value via `setTimestamp()` or `setObject(..., Types.TIMESTAMP)`
220+
> therefore silently drops any sub-microsecond digits.
221+
>
222+
> To write full picosecond precision, bind the value as a string; it is coerced to `TIMESTAMP(12)`
223+
> server side with all 12 digits intact:
224+
>
225+
> ```java
226+
> ps.setString(1, "2026-10-05 08:15:30.987654321098");
227+
> ```
228+
>
229+
> Reading picosecond values is unaffected; `getString()` returns all 12 digits on both the REST and
230+
> Storage Read API paths.
231+
216232
### High-Throughput Storage & Write API Properties
217233
218234
| Property Name | Default Value | Description |

‎java-bigquery-jdbc/src/main/java/com/google/cloud/bigquery/jdbc/BigQueryConnection.java‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@
3030
import com.google.cloud.bigquery.BigQueryException;
3131
import com.google.cloud.bigquery.BigQueryOptions;
3232
import com.google.cloud.bigquery.ConnectionProperty;
33+
import com.google.cloud.bigquery.DataFormatOptions;
3334
import com.google.cloud.bigquery.DatasetId;
3435
import com.google.cloud.bigquery.Job;
3536
import com.google.cloud.bigquery.JobInfo;
@@ -1294,6 +1295,16 @@ private BigQuery getBigQueryConnection() {
12941295
bigQueryOptions.setRetrySettings(retry_settings_builder.build());
12951296
}
12961297

1298+
// ISO8601_STRING is the only output format carrying the full 12-digit fraction
1299+
// ("YYYY-MM-DDTHH:MM:SS.FFFFFFFFFFFFZ"). The default FLOAT64 and the INT64 alternative both
1300+
// cap out at microseconds.
1301+
if (this.enableTimestampPicos) {
1302+
bigQueryOptions.setDataFormatOptions(
1303+
DataFormatOptions.newBuilder()
1304+
.timestampFormatOptions(DataFormatOptions.TimestampFormatOptions.ISO8601_STRING)
1305+
.build());
1306+
}
1307+
12971308
if (this.catalog != null) {
12981309
bigQueryOptions.setProjectId(this.catalog);
12991310
}

‎java-bigquery-jdbc/src/test/java/com/google/cloud/bigquery/jdbc/BigQueryConnectionTest.java‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@
4444
import com.google.auth.oauth2.GoogleCredentials;
4545
import com.google.cloud.bigquery.BigQuery;
4646
import com.google.cloud.bigquery.BigQueryException;
47+
import com.google.cloud.bigquery.DataFormatOptions;
4748
import com.google.cloud.bigquery.Project;
4849
import com.google.cloud.bigquery.QueryJobConfiguration.JobCreationMode;
4950
import com.google.cloud.bigquery.exception.BigQueryJdbcException;
@@ -259,6 +260,24 @@ public void testWriteAPIConnectionProperties() throws SQLException {
259260
}
260261
}
261262

263+
@Test
264+
public void testTimestampPicosControlsDataFormatOptions() throws IOException, SQLException {
265+
try (BigQueryConnection connection =
266+
new BigQueryConnection(BASE_URL + "EnableTimestampPicos=1;")) {
267+
assertEquals(
268+
DataFormatOptions.TimestampFormatOptions.ISO8601_STRING,
269+
connection.getBigQuery().getOptions().getDataFormatOptions().timestampFormatOptions(),
270+
"EnableTimestampPicos=1 must request ISO8601 timestamp serialization");
271+
}
272+
273+
try (BigQueryConnection connection = new BigQueryConnection(BASE_URL)) {
274+
assertEquals(
275+
DataFormatOptions.TimestampFormatOptions.TIMESTAMP_OUTPUT_FORMAT_UNSPECIFIED,
276+
connection.getBigQuery().getOptions().getDataFormatOptions().timestampFormatOptions(),
277+
"Default connections must not alter the timestamp wire format");
278+
}
279+
}
280+
262281
@Test
263282
public void testGetWriteClient() throws SQLException {
264283
// Test without connection properties. Defaults to default values.

‎java-bigquery-jdbc/src/test/java/com/google/cloud/bigquery/jdbc/it/ITBase.java‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -283,6 +283,9 @@ public static String getBaseConnectionUrl() {
283283
getBaseConnectionUrl() + "ProjectId=" + DEFAULT_CATALOG + ";OAuthType=3;Timeout=3600;";
284284
public static final BigQuery bigQuery = BigQueryJdbcBaseTest.getBigQuery(connectionUrl);
285285

286+
public static final String FORCE_READ_API_PROPERTIES =
287+
"EnableHighThroughputAPI=1;HighThroughputActivationRatio=0;HighThroughputMinTableSize=0;MaxResults=1;";
288+
286289
public static final String createDatasetQuery =
287290
"CREATE SCHEMA IF NOT EXISTS `%s.%s` OPTIONS(default_table_expiration_days = 5)";
288291
public static final String dropSchema = "DROP SCHEMA IF EXISTS `%s.%s` CASCADE;";

0 commit comments

Comments
 (0)