Skip to content

[#13800] feat(policy): Flat peer SerDe, DTO round-trips, minIntervalMs defaults - #13887

Open
lasdf1234 wants to merge 2 commits into
apache:branch-tms-schedulerfrom
lasdf1234:feature/tms-policy-peers-wire-13800
Open

lasdf1234 wants to merge 2 commits into
apache:branch-tms-schedulerfrom
lasdf1234:feature/tms-policy-peers-wire-13800

Conversation

@lasdf1234

@lasdf1234 lasdf1234 commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

What changes were proposed in this pull request?

Split from #13801 (2/2). Stack on #13885 (merge that first). Tracked by #13888.

Incremental scope (~1.5k vs #13885 ~1.1k):

  • Flat JSON mapping via ExecutablePolicyContentSerde + Iceberg PolicyContentDTO / DTOConverters
  • REST + POConverters round-trips for each executable built-in type
  • MaintenancePolicyDefinitions + per-type gravitino.maintenance.*.minIntervalMs + conf/docs

Until #13885 merges, the GitHub file list includes domain files from the stack base; review the second commit / common+core/maintenance+conf+docs paths.

Why are the changes needed?

Preserve flat wire/persistence JSON without flattening Spec into every type; server defaults for omitted minIntervalMs.

Fix: #13888

Does this PR introduce any user-facing change?

Yes — optional JSON peer fields on Iceberg maintenance policy contents, and:

  • gravitino.maintenance.compaction.minIntervalMs (3600000)
  • gravitino.maintenance.manifest-rewrite.minIntervalMs (3600000)
  • gravitino.maintenance.snapshot-expiry.minIntervalMs (3600000)
  • gravitino.maintenance.orphan-cleanup.minIntervalMs (86400000)

How was this patch tested?

./gradlew :common:test --tests 'org.apache.gravitino.dto.policy.*' \
  :core:test --tests 'org.apache.gravitino.storage.relational.utils.TestPOConverters' \
  --tests 'org.apache.gravitino.maintenance.policy.TestMaintenanceMinIntervalConfigs' \
  -PskipITs

@lasdf1234

Copy link
Copy Markdown
Collaborator Author

Tracked by #13888 (epic #13764). Stack on #13885 / #13800.

@lasdf1234
lasdf1234 force-pushed the feature/tms-policy-peers-wire-13800 branch from 20ec6e7 to 958f18c Compare October 15, 2026 12:15
@lasdf1234
lasdf1234 marked this pull request as draft October 15, 2026 12:22
@lasdf1234
lasdf1234 force-pushed the feature/tms-policy-peers-wire-13800 branch from 958f18c to dcd7ef5 Compare October 15, 2026 12:37
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Code Coverage Report

Overall Project 71.78% -0.11% 🟢
Files changed 57.65% 🔴

Module Coverage
aliyun 19.74% 🔴
api 54.37% 🟢
authorization-common 85.96% 🟢
authorization-ranger 14.44% 🔴
aws 53.54% 🟢
aws-bundle 0.0% 🔴
azure 32.1% 🔴
azure-bundle 0.0% 🔴
catalog-common 29.49% 🔴
catalog-fileset 82.37% 🟢
catalog-glue 73.24% 🟢
catalog-hive 83.63% 🟢
catalog-jdbc-common 57.98% 🟢
catalog-jdbc-doris 87.56% 🟢
catalog-jdbc-mysql 81.91% 🟢
catalog-jdbc-postgresql 84.02% 🟢
catalog-jdbc-starrocks 79.16% 🟢
catalog-kafka 75.21% 🟢
catalog-lakehouse-generic 84.58% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 86.0% 🟢
catalog-lakehouse-paimon 84.36% 🟢
catalog-model 77.99% 🟢
cli 44.62% 🟢
client-java 79.34% 🟢
client-java-runtime 0.0% 🔴
common 60.24% -0.6% 🟢
core 87.49% +0.02% 🟢
filesystem-hadoop3 76.89% 🟢
flink 55.44% 🟢
flink-common 32.2% 🔴
flink-runtime 0.0% 🔴
gcp 32.2% 🔴
hadoop-auth 68.0% 🟢
hadoop-common 17.84% 🔴
hive-metastore-common 62.3% 🟢
hive-metastore3-libs 58.8% 🟢
iceberg-aliyun-bundle 0.0% 🔴
iceberg-common 66.53% 🟢
iceberg-rest-server 77.08% 🟢
idp-basic 86.77% 🟢
integration-test-common 0.0% 🔴
jobs 71.35% 🟢
k8s-job-executor 99.81% 🟢
lance-common 39.9% 🔴
lance-rest-server 69.51% 🟢
lineage 59.39% 🟢
optimizer 84.2% 🟢
optimizer-api 46.58% 🟢
server 91.43% 🟢
server-common 83.85% 🟢
spark 61.79% 🟢
tencent 81.78% 🟢
tms 80.0% 🟢
trino-connector 63.31% 🟢
Files
Module File Coverage
common PolicyContentDTO.java 87.74% 🟢
DTOConverters.java 44.17% 🔴
core MaintenanceMinIntervalConfigs.java 100.0% 🟢
MaintenancePolicyDefinition.java 100.0% 🟢
MaintenancePolicyDefinitions.java 100.0% 🟢

roryqi pushed a commit that referenced this pull request Oct 16, 2026
#13885)

### What changes were proposed in this pull request?

Split from #13801 (1/2). **Domain** execution peers for Epic #13764 /
#13800.

- `PolicyExecutionSpec` / `ExecutablePolicyContent` /
`PolicyExecutionFields`
- Four Iceberg maintenance contents carry peers at construction
- TMS design notes on peers / `BuiltInType` identity

**Follow-up (stack):** #13888 / #13887 — wire/DTO/SerDe +
`minIntervalMs` registry.

### Why are the changes needed?

TMS needs shared trigger / cooldown / option-bag concepts without
freezing them onto every `PolicyContent`.

Fix: #13800

### Does this PR introduce _any_ user-facing change?

API/domain surface for optional peers on Iceberg maintenance contents
(REST wire lands in #13887).

### How was this patch tested?

```bash
./gradlew :api:test --tests 'org.apache.gravitino.policy.*' -PskipITs
```

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
@lasdf1234
lasdf1234 marked this pull request as ready for review October 16, 2026 03:41
lasdf1234 and others added 2 commits October 16, 2026 11:42
…efaults registry

REST PolicyContentDTO accessors and DTOConverters for flat peers; BuiltInType
minIntervalMs server defaults, conf template, and docs.

Co-authored-by: Cursor <cursoragent@cursor.com>
@lasdf1234
lasdf1234 force-pushed the feature/tms-policy-peers-wire-13800 branch from dcd7ef5 to ce91fa4 Compare October 16, 2026 03:50
@lasdf1234 lasdf1234 self-assigned this Oct 16, 2026
@lasdf1234
lasdf1234 requested a review from roryqi October 16, 2026 03:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant