Repository navigation
[#11925] feat(trino-connector): Support views in Trino connector - #11926
Conversation
Add view support to the Gravitino Trino connector: - HiveViewCatalogOperations: accept the trino SQL dialect in view creation and loading. - CatalogConnectorMetadata: delegate view CRUD to Gravitino's ViewCatalog API, only exposing views with a trino SQL representation. - GravitinoMetadata (+ per-Trino-version modules): implement listViews, getView, createView, dropView, renameView, isView.
There was a problem hiding this comment.
Pull request overview
This PR adds end-to-end view support in the Gravitino Trino connector, delegating CRUD to Gravitino’s ViewCatalog and ensuring only Trino-dialect view SQL is surfaced through Trino’s metadata APIs, with Hive backend updates to accept the trino dialect.
Changes:
- Implemented view metadata operations in the Trino connector (
listViews,getView,createView,dropView,renameView,isView) with per-Trino-version shims. - Added Trino↔Gravitino view metadata transformations (including
TypeId/TypeManagerhandling for column types). - Extended Hive view operations/tests to accept the
trinodialect and added integration testsets + documentation updates.
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| trino-connector/trino-connector/src/test/java/org/apache/gravitino/trino/connector/TestGravitinoMetadataView.java | Adds unit coverage for GravitinoMetadata view listing/loading and delegation. |
| trino-connector/trino-connector/src/test/java/org/apache/gravitino/trino/connector/catalog/TestCatalogConnectorMetadataView.java | Adds unit coverage for CatalogConnectorMetadata view support, visibility rules, and CRUD delegation/error paths. |
| trino-connector/trino-connector/src/main/java/org/apache/gravitino/trino/connector/util/json/JsonCodec.java | Introduces a cached TypeManager accessor to resolve TypeId back to Type. |
| trino-connector/trino-connector/src/main/java/org/apache/gravitino/trino/connector/metadata/GravitinoView.java | New view metadata wrapper for extracting/storing Trino-dialect SQL and related attributes. |
| trino-connector/trino-connector/src/main/java/org/apache/gravitino/trino/connector/GravitinoMetadata.java | Wires view listing/loading/delegation into connector metadata and merges views into listTables. |
| trino-connector/trino-connector/src/main/java/org/apache/gravitino/trino/connector/GravitinoErrorCode.java | Adds dedicated error codes for “view not exists” / “view already exists”. |
| trino-connector/trino-connector/src/main/java/org/apache/gravitino/trino/connector/catalog/CatalogConnectorMetadataAdapter.java | Implements Trino ConnectorViewDefinition ↔ Gravitino view conversions, including column type reconstruction. |
| trino-connector/trino-connector/src/main/java/org/apache/gravitino/trino/connector/catalog/CatalogConnectorMetadata.java | Adds ViewCatalog wiring + view CRUD/listing/visibility logic in the connector-side catalog facade. |
| trino-connector/trino-connector-473-478/src/main/java/org/apache/gravitino/trino/connector/GravitinoMetadata478.java | Adds Trino-version-specific createView/isView overrides delegating to shared helper. |
| trino-connector/trino-connector-469-472/src/main/java/org/apache/gravitino/trino/connector/GravitinoMetadata469.java | Adds Trino-version-specific createView/isView overrides delegating to shared helper. |
| trino-connector/trino-connector-452-468/src/main/java/org/apache/gravitino/trino/connector/GravitinoMetadata452.java | Adds Trino-version-specific createView/isView overrides delegating to shared helper. |
| trino-connector/trino-connector-446-451/src/main/java/org/apache/gravitino/trino/connector/GravitinoMetadata446.java | Adds Trino-version-specific createView override delegating to shared helper. |
| trino-connector/trino-connector-440-445/src/main/java/org/apache/gravitino/trino/connector/GravitinoMetadata440.java | Adds older Trino-version createView override (no properties map) delegating to helper. |
| trino-connector/trino-connector-435-439/src/main/java/org/apache/gravitino/trino/connector/GravitinoMetadata435.java | Adds older Trino-version createView override (no properties map) delegating to helper. |
| trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/lakehouse-iceberg/00014_view.txt | Adds expected output for Iceberg-backed view integration tests. |
| trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/lakehouse-iceberg/00014_view.sql | Adds Iceberg-backed SQL testset for view lifecycle operations. |
| trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00019_view.txt | Adds expected output for Hive-backed view integration tests (including error cases). |
| trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00019_view.sql | Adds Hive-backed SQL testset for view lifecycle operations. |
| docs/trino-connector/sql-support.md | Documents newly supported view-management SQL and visibility/ownership semantics. |
| catalogs/catalog-hive/src/test/java/org/apache/gravitino/catalog/hive/TestHiveCatalogOperations.java | Updates Hive catalog tests to assert Trino dialect views are accepted and HMS markers are set. |
| catalogs/catalog-hive/src/main/java/org/apache/gravitino/catalog/hive/HiveViewCatalogOperations.java | Allows trino dialect in Hive view create/load/replace and sets HMS presto_view marker. |
| catalogs/catalog-hive/src/main/java/org/apache/gravitino/catalog/hive/HiveView.java | Exposes presto_view marker key for shared package use (dialect detection/markers). |
Code Coverage Report
Files
|
mchades
left a comment
There was a problem hiding this comment.
Thanks for adding the view support. The main flow is well covered, but the current implementation still has backend-specific correctness gaps that make the documented Paimon creation path fail and can misread or persist inconsistent multi-representation views. Please address the inline comments before merge.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…hem as Hive dialect
mchades
left a comment
There was a problem hiding this comment.
Additional findings from the latest head review.
…path view metadata edge cases
| Optional.ofNullable(view.getDefaultSchema()), | ||
| columns, | ||
| Optional.ofNullable(view.getComment()), | ||
| Optional.empty(), |
There was a problem hiding this comment.
[P1] Could we round-trip the native Trino security and resolution context through reserved internal view properties? createView() drops the owner/security mode, native view loading also discards these fields and the SQL path, while this method always reconstructs an ownerless SECURITY INVOKER definition with an empty path. Trino creates views as SECURITY DEFINER by default, so this silently changes semantics after reload. Please namespace and protect these properties from direct mutation, use the decoded native payload as the source of truth on load, and cover DEFINER, INVOKER, and non-empty-path round trips.
There was a problem hiding this comment.
[P1] The native Hive security round trip is still incomplete on 5683227. The new owner property fixes the connector-local round trip, but HiveViewCatalogOperations.toHmsViewOriginalText() still writes owner=null, runAsInvoker=true, and toHiveView() does not restore the decoded owner/security mode. A Gravitino-created DEFINER view therefore becomes INVOKER through native Hive; a native DEFINER view becomes INVOKER through Gravitino. I reproduced both directions through the current conversion methods.
Please carry the owner/security mode through the native payload and add bidirectional DEFINER/INVOKER tests, including SHOW CREATE VIEW and access to an underlying table.
There was a problem hiding this comment.
[P2] Native views with a non-empty SQL path still lose that context on load. On f0fada9, toHiveView() does not preserve or reject decoded.path, and getViewDefinition() always returns List.of(). I reproduced a native payload with one path entry being loaded with an empty path. A view using that path to resolve an unqualified function can fail or bind differently.
The create/replace guards do not protect this read path. Please preserve the native path or explicitly reject such views during loading, and add a native-payload regression test.
There was a problem hiding this comment.
[P2] Native SQL path loss still reproduces on cc06630. A native view payload with one path entry is returned as a ConnectorViewDefinition with an empty path. toHiveView() still neither preserves nor rejects decoded.path, and getViewDefinition() returns List.of(). This can change unqualified function resolution; the create/replace guards do not protect loading native views.
Please preserve the path or reject these views on load, with a native-payload regression test.
There was a problem hiding this comment.
[P2] Native SQL path loss still reproduces on cda1ea5. A native payload containing one path entry still loads with an empty ConnectorViewDefinition.getPath(). toHiveView() neither preserves nor rejects decoded.path, so unqualified function resolution can change. The create/replace guards do not cover this read path. Please preserve the path or reject such views on load, with a native-payload regression test.
There was a problem hiding this comment.
Throw exception on this path
| view.getDefaultCatalog(), | ||
| view.getDefaultSchema(), | ||
| view.getComment())); | ||
| changes.addAll(computePropertyChanges(existingView.properties(), view.getProperties())); |
There was a problem hiding this comment.
[P2] For Trino 445+, how can a non-empty view-property map reach this code from SQL? No connector implementation exposes getViewProperties(), so Trino rejects CREATE VIEW ... WITH (...) before createViewInternal(), while the current unit test injects the property map directly. Could we delegate the internal connector’s view-property metadata in the applicable version shim(s), splitting the 440–445 range if necessary, and add an SQL-level IT?
There was a problem hiding this comment.
[P2] The new overrides still expose an empty property list on f0fada9. CatalogConnectorContext.getViewProperties() delegates to adapter.getViewProperties(), but no concrete adapter overrides the new default method, which returns emptyList(). I verified that HiveConnectorAdapter.getViewProperties() returns [], so a non-empty WITH (...) clause still fails Trino property validation.
Please delegate/register the actual view-property metadata and add an SQL-level test using Hive's extra_properties; the 445 shim also remains unwired.
There was a problem hiding this comment.
[P2] View-property registration is still empty on cc06630. I rechecked HiveConnectorAdapter.getViewProperties() and it returns []. The shims delegate through CatalogConnectorContext to the unchanged default method, with no concrete adapter supplying metadata. Consequently, WITH (extra_properties = MAP(ARRAY['k'], ARRAY['v'])) is rejected by Trino property validation before reaching createView().
Please expose the supported view-property metadata and add an SQL-level regression test.
There was a problem hiding this comment.
[P2] View-property registration remains empty on cda1ea5. HiveConnectorAdapter.getViewProperties() still returns [], so WITH (extra_properties = MAP(ARRAY['k'], ARRAY['v'])) fails Trino property validation before reaching createView(). Please register/delegate the supported view-property metadata and cover this with an SQL-level regression test.
There was a problem hiding this comment.
We don't support view properties in this PR,table/schema properties also not supported now
# Conflicts: # trino-connector/trino-connector-435-439/src/main/java/org/apache/gravitino/trino/connector/GravitinoMetadata435.java # trino-connector/trino-connector/src/main/java/org/apache/gravitino/trino/connector/GravitinoMetadata.java # trino-connector/trino-connector/src/main/java/org/apache/gravitino/trino/connector/catalog/CatalogConnectorMetadata.java # trino-connector/trino-connector/src/main/java/org/apache/gravitino/trino/connector/catalog/CatalogConnectorMetadataAdapter.java
Round-trip a Trino view's SECURITY DEFINER owner through a reserved Gravitino view property instead of always reconstructing it as SECURITY INVOKER. Return an empty list from listViews() when the schema does not exist, matching the ConnectorMetadata contract. Translate Iceberg's AlreadyExistsException to ViewAlreadyExistsException on view rename collisions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CquWGJb3F1vUjkR5EHYNYf
# Conflicts: # trino-connector/trino-connector/src/main/java/org/apache/gravitino/trino/connector/GravitinoMetadata.java
…default The owner round-trip fix makes CREATE VIEW (with no explicit security clause) correctly produce SECURITY DEFINER, matching Trino's documented default. Update the Hive and Iceberg view testset expected output accordingly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FNMq21XMMMsbL7dKMe7SRV
# Conflicts: # trino-connector/trino-connector/src/main/java/org/apache/gravitino/trino/connector/GravitinoMetadata.java
…fter merge GravitinoConfig.singleMetalakeMode() was replaced by catalogNameWithMetalake() (the config unification in apache#13301, merged from main). Update CatalogConnectorMetadata to use the inverse of the new method.
…eplace Trino's Connector.getViewProperties() only exists from 446 onward, so it is wired only in the version-specific connector subclasses (446, 452, 469, 478), not the shared base compiled against 440; CREATE VIEW ... WITH (...) is validated by the Trino engine before reaching createViewInternal(), matching what the unit test previously injected directly but could never reach in production. Also fix CREATE OR REPLACE VIEW leaving a stale SECURITY DEFINER owner property when the replacement view has no owner (SECURITY INVOKER), which made SHOW CREATE VIEW keep reporting SECURITY DEFINER after reload.
mchades
left a comment
There was a problem hiding this comment.
One additional issue with view output type conversion.
| columns.add( | ||
| new GravitinoColumn( | ||
| column.getName(), | ||
| dataTypeTransformer.getGravitinoType(trinoType), |
There was a problem hiding this comment.
[P2] View output types should not use the physical-table type restrictions. This transformer is catalog-specific: Iceberg rejects varchar(1) (e.g. the output of SELECT 'x' AS c), while Hive accepts timestamp(3) with time zone during creation but rejects it in getViewDefinition(). I reproduced both with the current adapter methods, so otherwise valid views either cannot be created or cannot be reloaded.
Please use view-specific type conversion that preserves the query's output types, and cover these cases with the actual Hive/Iceberg adapters rather than only GeneralDataTypeTransformer.
There was a problem hiding this comment.
Rechecked on f0fada9: this conversion is unchanged, and both failures still reproduce with the current adapter methods. Iceberg rejects varchar(1); Hive accepts timestamp(3) with time zone during creation but rejects it when loading the definition. Please keep this issue open until view-specific type conversion and regression coverage are added.
There was a problem hiding this comment.
[P2] The Iceberg type fix still fails at the server boundary on cc06630. The connector now accepts varchar(1), but IcebergViewCatalogOperations.createView() passes those columns to ConvertUtil.toIcebergSchema(), whose ToIcebergType rejects VarCharType. I reproduced this by passing the current adapter output to the server's createView() method; it throws before persistence. Thus CREATE VIEW v AS SELECT 'x' AS c still fails, while the new test stops at adapter conversion.
Please complete the Iceberg-side conversion and add a create/load regression through the server. The Hive timestamp-with-time-zone and unbounded varchar round trips now pass.
There was a problem hiding this comment.
[P2] Still reproducible on cda1ea5. CREATE VIEW v AS SELECT 'x' AS c now passes connector conversion, but the server's IcebergViewCatalogOperations.createView() still rejects VarCharType through ConvertUtil.toIcebergSchema(). I re-ran the current adapter-to-server call chain; the failure occurs before persistence. Please complete the server-side view type conversion and cover create/load through that boundary.
There was a problem hiding this comment.
[P2] Nested varchar still fails on 88266eb. Scalar varchar create/load/replace now passes against Iceberg's InMemoryCatalog. However, widenVarcharColumns() only checks the top-level column type: CREATE VIEW v AS SELECT ARRAY['x'] AS c still reaches ToIcebergType with a nested VarCharType and throws UnsupportedOperationException. Map keys/values and named row fields have the same problem; both create and replace fail before persistence.
Please recursively normalize varchar inside list/map/struct types and cover these nested cases through the server boundary.
…tive Hive payload Add a native_hive-read test to the Hive view testset (gt_hive creates, native_hive reads via SHOW CREATE VIEW and SELECT, including a timestamp(3) column). Generating its expected output exposed a real gap: HiveViewCatalogOperations always encoded native Trino view payloads with owner=null/runAsInvoker=true, so a SECURITY DEFINER view created through gt_hive read as SECURITY INVOKER through native_hive. Fix by threading the TRINO_VIEW_OWNER_PROPERTY reserved property into the native payload encode/decode paths in HiveViewCatalogOperations, and reorder CatalogConnectorMetadata's replace-view changes so the owner property update lands before ReplaceView, since Hive's ReplaceView handling encodes from whatever properties have been applied so far. Also fix TrinoQueryIT.executeSqlFileWithGenOutput(), which didn't strip SQL comments before matching statements (unlike its sibling methods), corrupting --gen_output for any testset file with -- comments.
mchades
left a comment
There was a problem hiding this comment.
One remaining issue in native-view replacement after the owner round-trip changes.
… type conversion Seed updatedProperties' owner from the decoded native payload before applying changes in HiveViewCatalogOperations.alterView(), so replacing a genuinely-native DEFINER view as the same owner no longer looks like no change to computePropertyChanges() and silently downgrades to INVOKER. Use an unrestricted GeneralDataTypeTransformer for view output columns instead of the catalog-specific one: view output types are never physically stored, so Iceberg's "no varchar(n)" and Hive's "no timestamp with time zone" table-column restrictions wrongly rejected otherwise- valid view output types, or accepted them on create only to reject the same type on reload.
…nsformer getGravitinoType() converted Trino's unbounded varchar into a bounded varchar(2147483646) instead of StringType, which broke view reloads once views started using this unrestricted transformer directly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FNMq21XMMMsbL7dKMe7SRV
# Conflicts: # trino-connector/trino-connector/src/main/java/org/apache/gravitino/trino/connector/GravitinoErrorCode.java # trino-connector/trino-connector/src/main/java/org/apache/gravitino/trino/connector/catalog/CatalogConnectorMetadata.java # trino-connector/trino-connector/src/test/java/org/apache/gravitino/trino/connector/catalog/TestCatalogConnectorMetadataView.java
Native Trino treats the HMS table owner as authoritative over the owner encoded in a view's payload (e.g. ALTER VIEW ... SET AUTHORIZATION updates only the HMS owner). Load now prefers the live HMS owner over the payload-decoded one, and create/replace keep the HMS owner in sync with the connector-managed owner property so the two don't immediately diverge again. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FNMq21XMMMsbL7dKMe7SRV
…g schema A view's output column may carry Gravitino's VarCharType (e.g. SELECT 'x' AS c produces varchar(1)), since the connector layer does not apply physical-table type restrictions to view columns. Iceberg has no length-bound string type, so ConvertUtil.toIcebergSchema() rejected it. Widen such columns to StringType before building the schema, scoped to the view create/replace call sites only. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FNMq21XMMMsbL7dKMe7SRV
…alter Gravitino's view model has no SQL path (SET PATH) concept, so silently dropping a non-empty path on load would let an unqualified function resolve differently than the native Trino view's creator intended. Reject such views on load instead. alterView() now rejects them upfront for every change type, not only ReplaceView: the load-path rejection also fires when building the return value, so a non-ReplaceView change (e.g. rename) previously mutated HMS successfully before throwing on the way out. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FNMq21XMMMsbL7dKMe7SRV
…gnment alterView() seeded the current owner from the (possibly stale) payload-encoded owner instead of the effective HMS owner, so a replace or rename could silently overwrite a correct HMS owner with a stale one. Separately, the owner-sync block ran unconditionally for any Trino view, so altering a genuine SECURITY INVOKER view (no owner property at all) fell back to the current principal and reassigned real HMS ownership on an unrelated change like a rename. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FNMq21XMMMsbL7dKMe7SRV
…ew columns widenVarcharColumns() only checked a column's top-level type, so a view output column like ARRAY['x'] (or a map/struct with a varchar element) still reached ConvertUtil.toIcebergSchema() with a nested VarCharType and was rejected the same way a top-level one was. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FNMq21XMMMsbL7dKMe7SRV
trinoRowTypeToGravitinoRowType() called field.getName().get() unconditionally, so an anonymous row field (e.g. ROW(1, 'x') with no AS) threw NoSuchElementException instead of converting. Synthesize a positional name, skipping any name already used by an explicitly named sibling field. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FNMq21XMMMsbL7dKMe7SRV
The docs claimed view owner is unsupported and views are always SECURITY INVOKER; that's no longer accurate now that owner is persisted and reloaded. Document current support and known limitations (no SET PATH, no view WITH (...) properties) instead. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FNMq21XMMMsbL7dKMe7SRV
| do { | ||
| name = "field" + anonymousSeq++; | ||
| } while (usedNames.contains(name)); |
There was a problem hiding this comment.
[P2] Avoid case-insensitive row field collisions. For CREATE VIEW v AS SELECT CAST(ROW(1, 'x') AS ROW(INTEGER, "FIELD0" VARCHAR(1))) AS r, this loop generates field0, so the persisted type contains both field0 and FIELD0. I reproduced that round trip with an in-memory Hive client and Iceberg InMemoryCatalog on f0ebaa38. Trino resolves row fields case-insensitively, so SELECT r.FIELD0 FROM v becomes ambiguous. Please reserve names case-insensitively while preserving explicit names, and add a regression test for this mixed anonymous/uppercase case.
Trino resolves row fields case-insensitively, so a synthesized anonymous-field name (e.g. field0) could still collide with an explicitly-named sibling that only differs in case (e.g. "FIELD0"), making the persisted type ambiguous on access. Compare and reserve names case-insensitively while preserving explicit casing in the output. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FNMq21XMMMsbL7dKMe7SRV
What changes were proposed in this pull request?
Add view support to the Gravitino Trino connector:
HiveViewCatalogOperations: accept thetrinoSQL dialect in view creation and loading.CatalogConnectorMetadata: delegate view CRUD to Gravitino'sViewCatalogAPI, only exposing views with atrinoSQL representation.GravitinoMetadata(+ per-Trino-version modules): implementlistViews,getView,createView,dropView,renameView,isView.Why are the changes needed?
Fix: #11925
Does this PR introduce any user-facing change?
Yes — users can now
CREATE VIEW/CREATE OR REPLACE VIEW/SHOW CREATE VIEW/DROP VIEW/ALTER VIEW ... RENAME TOvia Trino SQL on Hive/Iceberg-backed catalogs, and views appear inSHOW TABLES.How was this patch tested?
TestCatalogConnectorMetadataView,TestGravitinoMetadataView, Hive dialect tests inTestHiveCatalogOperations.