fix(chaining): pause chaining simulation - #8010
Open
EvaE-Filigran wants to merge 31 commits into
Open
EvaE-Filigran wants to merge 31 commits into
EvaE-Filigran wants to merge 31 commits into
Conversation
…ruption handling and tenant-scope background jobs (#7519)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…gJob and add & fix tests;
# Conflicts: # openaev-api/src/main/java/io/openaev/scheduler/jobs/QueueChainingJob.java # openaev-api/src/main/java/io/openaev/scheduler/jobs/WorkflowTimeoutJob.java # openaev-api/src/test/java/io/openaev/service/chaining/StepServiceTest.java
… and generation uuid by db
Contributor
|
📖 Documentation check — ✅ Passed 22 functional file(s), 0 doc file(s) changed. No documentation gaps detected. |
EvaE-Filigran
changed the title
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Blocking lifecycle, queue-concurrency, tenant-cleanup, migration, and resume re-evaluation issues remain unresolved.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds soft pause/resume support for chaining workflows, including timing persistence, delay-queue handling, lifecycle cleanup, UI controls, migrations, and tests.
Changes:
- Added workflow STOP/RUN pause lifecycle and timeout accounting.
- Updated queue processing, workflow termination, and tenant-scoped jobs.
- Added migrations, ADRs, frontend behavior, and regression tests.
File summaries
| File | Description |
|---|---|
WorkflowStateRepository.java |
Returns deleted-state count. |
WorkflowRepository.java |
Accounts for paused timeouts. |
StepRepository.java |
Restricts event lookups to active steps. |
StepDelayQueueRepository.java |
Adds pause-safe popping and atomic upsert. |
AssetAgentJobRepository.java |
Adds simulation-scoped cleanup. |
Workflow.java |
Maps pause timing fields. |
ExerciseHeader.tsx |
Enables pause for chained simulations. |
ExerciseHeader.test.tsx |
Updates lifecycle button tests. |
ExerciseServiceIntegrationTest.java |
Wires pause service. |
WorkflowServiceTest.java |
Updates workflow cleanup tests. |
WorkflowPauseServiceTest.java |
Tests pause/resume unit behavior. |
WorkflowPauseServiceIntegrationTest.java |
Tests pause scenarios. |
WorkflowEndServiceTest.java |
Updates end-of-life assertions. |
WorkflowEndOfLifeIntegrationTest.java |
Tests termination cleanup and logging. |
StepServiceTimeoutGuardsTest.java |
Updates step lookup stubs. |
StepServiceTest.java |
Tests active-step lookups and tenant scope. |
StepEventServiceTest.java |
Updates external event handling tests. |
StepDelayQueueServiceTest.java |
Tests queue upsert behavior. |
WorkflowTimeoutJobTest.java |
Tests tenant-scoped timeout handling. |
QueueChainingJobTest.java |
Tests transactional queue processing. |
ExerciseServiceTest.java |
Wires pause service mock. |
StepRepositoryTest.java |
Tests active-step repository queries. |
WorkflowUpdateEventAspectTest.java |
Updates active-step event tests. |
ExecutionTraceUtils.java |
Adds no-progress trace creation. |
WorkflowStateService.java |
Removes obsolete cleanup facade. |
WorkflowService.java |
Adds STOP handling and unified cleanup delegation. |
WorkflowPauseService.java |
Implements pause/resume orchestration. |
WorkflowEndService.java |
Centralizes workflow end cleanup. |
StepService.java |
Adds active-step helpers and end logging. |
StepEventService.java |
Guards READY execution during STOP. |
StepDelayQueueService.java |
Handles null and recalculated delay goals. |
QueueChainingJob.java |
Adds tenant-aware delay processing. |
ExerciseService.java |
Delegates simulation pause/resume. |
V6_20260916145000000__Add_workflow_pause_fields.java |
Adds pause persistence columns. |
V6_20260915143000000__Deduplicate_step_delay_queue_and_enforce_unique_key.java |
Deduplicates and constrains delay entries. |
V6_20260908142053000__Heal_legacy_ended_workflow_steps_and_states.java |
Heals legacy ended workflows. |
WorkflowUpdateEventAspect.java |
Sends updates only for active steps. |
ADR-010-workflow-pause-resume-lifecycle.md |
Documents pause/resume decisions. |
ADR-007-workflow-end-of-life-cancel-timeout-no-more-progress.md |
Documents unified end-of-life handling. |
Review details
- Files reviewed: 39/40 changed files
- Comments generated: 10
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+800
to
+802
| if (workflowService.isSimulationChaining(exercise.getId())) { | ||
| workflowPauseService.pauseSimulationWorkflowRuns(exercise.getId()); | ||
| } |
Comment on lines
+69
to
+70
| TenantContext.clearCurrentTenant(); | ||
| tenantTx.setScopeOnCurrentTransaction(TxCtx.missing()); |
Comment on lines
+90
to
+92
| stepDelayQueueService.deleteAllByWorkflowRun( | ||
| stepDelayQueue.getWorkflowRun(), | ||
| WorkflowEndService.WORKFLOW_END_CAUSE.NO_MORE_PROGRESS); |
| SELECT DISTINCT ON (sdq.steps_delay_queue_workflow_run_id) sdq.steps_delay_queue_id | ||
| FROM steps_delay_queue sdq | ||
| JOIN workflows w ON w.workflow_id = sdq.steps_delay_queue_workflow_run_id | ||
| WHERE w.workflow_status = 'RUN' |
Comment on lines
+30
to
+34
| "DELETE FROM workflow_states ws" | ||
| + " USING workflows w" | ||
| + " WHERE ws.workflow_execution_id = w.workflow_id" | ||
| + " AND w.workflow_status = 'END'" | ||
| + " AND w.workflow_keep_alive IS NOT TRUE;"); |
Comment on lines
+17
to
+21
| DELETE FROM steps_delay_queue sdq | ||
| USING ( | ||
| SELECT steps_delay_queue_id | ||
| FROM ( | ||
| SELECT steps_delay_queue_id, |
| statement.execute( | ||
| """ | ||
| ALTER TABLE workflows | ||
| ADD COLUMN IF NOT EXISTS workflow_pause_at timestamp; |
Comment on lines
+60
to
+65
| if (readySteps.isEmpty()) { | ||
| pausedRun = workflowService.evaluateWorkflowProgress(pausedRun); | ||
| if (pausedRun.getStatus() == WorkflowStatus.END) { | ||
| return true; | ||
| } | ||
| } |
| @@ -0,0 +1,99 @@ | |||
| # ADR-008: Workflow pause/resume lifecycle | |||
Comment on lines
+86
to
+87
| AND workflow_created_at | ||
| + ((workflow_timeout_seconds + COALESCE(workflow_pause_second, 0)) || ' seconds')::interval <= now() |
Stephanya Casanova (savacano28)
added
the
filigran team
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
WorkflowPauseServiceto handle STOP/RUN transitions,workflow_pause_at, cumulativeworkflow_pause_second, delay-goal nullification on pause, delay-goal recalculation on resume, and READY re-enqueue/re-evaluation on resume.ExerciseServicenow delegates chaining pause/resume lifecycle toWorkflowPauseServiceand removes the previous pause refusal path.V6_20260916145000000__Add_workflow_pause_fieldsaddsworkflow_pause_atandworkflow_pause_second.Workflowentity mapped with the new pause fields.WorkflowRepository.findAllExpiredRunWorkflowIdsnow includesworkflow_pause_secondin timeout expiration.StepDelayQueueRepository.popNextPerWorkflowRunnow pops only forRUNworkflows (pause safety barrier), plus repository support to nullify delay goals.StepEventServiceandWorkflowServicenow treatSTOPas non-runnable (likeEND) for READY execution/evaluation paths.WorkflowPauseServiceTestandWorkflowPauseServiceIntegrationTest(US.1/US.2/US.3).Testing Instructions
Create a chained scenario with at least a few steps that can produce READY and DELAY behavior, then launch a simulation.
Open the simulation execution timeline page and keep it visible during the whole test.
US.1 — Pause simulation chaining without timeout/rate limit
US.2 — Pause simulation chaining with timeout/rate limit
US.3 — Multi-pause scenario chaining without timeout/rate limit
Related issues
Checklist
Further comments
This implementation follows a “soft pause” model for chaining: while paused, READY/DELAY progression is blocked but in-flight RUN side effects can still complete. On resume, delay goals are restored and workflow progress is re-evaluated to create READY steps from outputs accumulated during pause.