Repository navigation
Conversation
This was referenced Oct 20, 2026
Wire the expand-pool handler to resolve the policy, batch-list tag-bound tables, enqueue idempotent cron-job rows, and reschedule or remove the expand lease per design §5.2.1. Co-authored-by: Cursor <cursoragent@cursor.com>
lasdf1234
force-pushed
the
feature/tms-expand-cron-enqueue-13984
branch
from
October 20, 2026 08:43
cf19220 to
79721f8
Compare
lasdf1234
marked this pull request as draft
October 20, 2026 08:43
Complete policy→cron-job fan-out: expand catalog/schema tag attachments, keep tables where this policy is the nearest BuiltInType winner, validate crontab before enqueue, and remove one-shot cron rows after the deferred submit callback so later expands can re-schedule. Co-authored-by: Cursor <cursoragent@cursor.com>
lasdf1234
changed the title
Skip cron-job row writes when OccupancyGate reports in-flight or within the resolved cooldown (policy minIntervalMs → server conf → type default). Co-authored-by: Cursor <cursoragent@cursor.com>
…icts Treat multiple same BuiltInType policies on the nearest object as a conflict: skip the table, log error, do not pick min(policy id). Co-authored-by: Cursor <cursoragent@cursor.com>
Apply OccupancyGate while scanning nearest-policy winners so cooldown or in-flight tables do not consume enqueueBatchSize and starve later tables. Co-authored-by: Cursor <cursoragent@cursor.com>
For ICEBERG_SNAPSHOT_EXPIRATION, merge enqueueable tables whose nearest
winner is schema/catalog into one batch:{hash}:{policyId} row with
CronJobData table ids; table-level winners stay per-table.
Co-authored-by: Cursor <cursoragent@cursor.com>
Collaborator
Author
|
Superseded by #14019 — expand/cron enqueue (including snapshot-expiry batching and incomplete-batch fill) continues there. |
Code Coverage Report
Files
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Implement expand-pool fan-out (§5.2.1) against
branch-tms-scheduler— full policy → cron-job row path:policy-expandno-op withPolicyExpandPipelinethat loads the policy, validatestimeTriggerbefore enqueue, lists tables, and enqueues one-shotcron-jobrows (table:{tableId}:{policyId}) viascheduleIfNotExistsNearestPolicyTableEnumeratorexpands tag-bound catalog/schema/table attachments and keeps tables where this policy is the nearest winner of itsBuiltInType(NearestMaintenancePolicyResolver)timeTrigger/ non-maintenanceSchedulerClientagainst the entity-store DataSource (defaultAutoCommit=false)Out of scope: policy-expand registration / PreCommitHook (#13897), cron-pool Spark submit, commit-job path.
Why are the changes needed?
Fix: #13984
Parent: #13760. Without a real expand pick handler, due
policy-expandleases never producecron-jobrows.Does this PR introduce any user-facing change?
No. Internal scheduler behavior only; uses existing
gravitino.maintenance.expand.enqueueBatchSize.How was this patch tested?
./gradlew :plugins:tms:test -PskipITs --tests 'org.apache.gravitino.maintenance.scheduler.Test*'