You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 66c58d3
Browse filesBrowse the repository at this point in the historyBrowse files
fix(bigquery): apply DataFormatOptions to tabledata.list requests (#14889)
b/579291912
`BigQueryOptions.setDataFormatOptions(...)` is only sent on
`jobs.query`. Rows read through `tabledata.list` don't get it, so
BigQuery returns TIMESTAMPs as float64 there while the client decodes
them using the configured format. `tabledata.list` serves:
- `listTableData` / `Table.list`
- page 2+ of `query()`
- `Job.getQueryResults()`
Impact:
- `useInt64Timestamp(true)`: `getTimestampValue()` /
`getTimestampInstant()` throw `NumberFormatException`.
- `timestampFormatOptions(INT64)`: values are decoded as seconds, so
dates come out wrong. This also happens on page 1.
- `ISO8601_STRING`: rows from `tabledata.list` come back as float64.
- float64 loses microseconds, e.g. `9999-12-31 23:59:59.999999` becomes
`+10000-01-01`.
## Changes
- **`HttpBigQueryRpc`**: `listTableData` sends
`formatOptions.useInt64Timestamp` /
`formatOptions.timestampOutputFormat`, only when they differ from the
defaults. Default requests are unchanged.
- **`BigQueryImpl`**: rows are decoded as int64 when either
`useInt64Timestamp(true)` or `timestampFormatOptions(INT64)` is set.
No public API changes.
## Tests
- `HttpBigQueryRpcTest`: defaults send no `formatOptions`, and
configured options are sent.
- `BigQueryImplTest`: `INT64` values from `listTableData` decode as
int64.
- `ITBigQueryTest#testLosslessMaxTimestampIntegration`: now reads 2
pages (the second via `tabledata.list`) and adds an `INT64` case.
All new and changed tests fail without the fix and pass with it.
Copy file name to clipboardExpand all lines: java-bigquery/google-cloud-bigquery/src/test/java/com/google/cloud/bigquery/spi/v2/HttpBigQueryRpcTest.java
0 commit comments