Skip to content

Add agentic issue investigation workflow - #50606

Open
ROHIT SINGHAL (rohitsinghal4u) wants to merge 10 commits into
Azure:mainfrom
rohitsinghal4u:rohit/agentic-issue-investigation
Open

ROHIT SINGHAL (rohitsinghal4u) wants to merge 10 commits into
Azure:mainfrom
rohitsinghal4u:rohit/agentic-issue-investigation

Conversation

@rohitsinghal4u

@rohitsinghal4u ROHIT SINGHAL (rohitsinghal4u) commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds generalized post-triage issue investigation for Azure SDK for Java, based on Azure/azure-sdk-for-net#60514, without modifying the existing triage workflow.

Existing workflow compatibility

The initial integration incorrectly made regular triage outputs depend on owner notification. The implementation now restores issue-triage.md, its generated lock and actionlint.yml exactly to their base/main definitions.

A separate completed-workflow_run handoff checks investigation eligibility. It cannot delay or suppress the completed triage run's labels, assignments, comments, recovery or completion result. New tests run in a separate validation workflow.

Both live automatic legacy paths passed: prelabelled exit and manual-triage fallback. Their separate followers started after completion and suppressed investigation dispatch. The original fixture25 event was dropped while fork Actions were disabled; an already-approved prelabelled fixture26 supplied the fresh automatic probe. The dropped event remains preserved as an infrastructure observation.

Investigation and handoff

  • Validates successful same-repository/default-branch triage provenance, bounded activation/output artifacts, the pinned native threat verdict, a unique applied label/comment target and current issue state.
  • Uses the unique issue positively labeled and commented on, not a claimed originating-issue field: baseline activation JSON has no such field.
  • Rejects ambiguous, expired, changed, malformed or non-first-attempt automatic handoffs. Maintainers retain manual investigation dispatch.
  • Applies ordered version, specific-duplicate, missing-information, service-side and bounded SDK/documentation-fix decisions.
  • Uses read-only GitHub MCP tooling, no investigation shell, default-branch checkout and a bounded credential-free release-catalog lookup.
  • Preserves single-comment closure and source-citation provenance, with fourteen advisory Key Vault topics.
  • Deterministically rejects malformed issue numbers in a permission-free root job before activation, AI, detection or native conclusion reporting.

Reassessment and metadata clarification

The latest live follow-up identified a conflict between duplicate reassessment and comment deduplication. Preview31 still nooped by accepting its old unsupported automated duplicate assertion.

The new guidance makes reassessment happen first. An unsupported automated duplicate outcome is corrected once with a targeted information request; a supported existing outcome and equivalent next action still produce noop on unchanged reruns.

Management34 correctly requested missing information, rejected the unsupported duplicate and did not request a downgrade. Its remaining strict-test failure was unclear treatment of catalog2.55.5 versus newer Maven2.55.6. Catalog entries are now explicitly source-attributed candidates, not proof that no newer publication exists. Conflicting publisher evidence must be reconciled or reported as unresolved, rather than calling the older entry a verified latest version. The older-version reproduction rule is explicitly scoped to older versions.

The metadata implementation, schema, credentials, network destinations and version ordering policy are not changed by these wording corrections.

Validation and live results

  • 35 Node tests and29 Python test methods pass locally, including new reassessment and metadata-attribution contracts.
  • Exact legacy definitions, owner failure/recovery, handoff provenance, artifact bounds, native detection formats, Unicode hashing, label gates, malformed input and dispatch failures remain covered.
  • Strict gh-aw v0.87.1 compilation, direct actionlint1.7.12, spelling and strict link guidance pass; generated locks are regenerated and stable.
  • At b6e8062a3a, all13 targeted runs executed:11 passed and2 failed, with no infrastructure or dependency blocks. All five malformed inputs ran only preflight and made zero repository mutations. Valid-input customer/PR rejection, service closure, exact duplicate and unchanged reruns passed.
  • Service32 closed with exactly one complete comment as not_planned; its rerun made no changes. Precise duplicate49 linked issue29 with one comment; its rerun made no changes.
  • Preview31 reassessment and management34 metadata wording remain historical failed expectations. New live validation of the latest guidance is pending.
  • Current-head CI at b6e8062a3a, including macOS and Azure build6971110, passed. The older macOS download failure was superseded by that successful build; no manual retry succeeded.
  • Earlier full-batch23/24, closure3/4, citation3/3 and isolated matrix27pass/4fail/1infrastructure-blocked/1dependency-blocked results remain unchanged.

Scope and limitations

No Java SDK API, dependency, release-version or existing-workflow changes. No new secrets, external LLM endpoints or runtime upgrades.

The pinned runtime has no independent detection-tracker opt-out. Malformed inputs now cannot reach that reporter, but warnings on valid-number requests can still produce native tracking. Detection is not disabled or falsified; unexpected historical issues47/48 remain evidence.

Tool-name allowlists are pre-approval, and safe-output targets are defaults rather than strict issue-number confinement. Evidence decisions and comment deduplication remain model-driven; writes are not transactional. Actual coding-agent assignment, positive real external-author handoff and forced live outages remain unproven. Shared runner capacity cannot be guaranteed unchanged despite preserving legacy workflow code and dependencies.

All SDK Contribution checklist

  • No breaking SDK changes.
  • CHANGELOG update is not applicable: automation, tests and advisory documentation only.
  • Contribution guidelines read.
  • Existing triage and lint definitions preserved.
  • Regression coverage and live failures/limits documented honestly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 28, 2026 16:20
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
34 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved workflow security and correctness issues affect command permissions, checkout isolation, failure reporting, and triage output execution.

Review effort: Lite
Findings: 1 High severity · 1 Medium severity

Open (2)
What changed in this PR

Adds a post-triage agentic issue investigation workflow for Azure SDK for Java.

Changes:

  • Adds investigation rules, decisions, and safe outputs.
  • Integrates triage handoff and owner-recovery ordering.
  • Adds Key Vault troubleshooting context and generated workflow locks.
File Summary
sdk/​keyvault/​known-behaviors.md Adds advisory Key Vault service context.
.github/​workflows/​issue-triage.md Adds investigation handoff logic.
.github/​workflows/​issue-triage.lock.yml Regenerates the triage workflow.
.github/​workflows/​issue-investigation.md Defines investigation behavior and outputs.
.github/​workflows/​issue-investigation.lock.yml Compiles the investigation workflow.

singhalrohit4u and others added 7 commits September 28, 2026 17:43
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use the native close_issue body for the complete service-side explanation, instead of posting a separate comment before closure. Add source and generated-output regression checks to the existing Node test step.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep file citations tied to the repository, ref, and path successfully read. Verify each citation before publishing rather than replacing fork evidence with an unverified upstream-main URL. Retain trusted external metadata sources and add a guidance regression contract.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Restore the existing triage source, lock and actionlint workflow exactly. Dispatch investigation from a separate completed-run handoff with bounded artifact provenance, native threat parsing and fresh issue gates. Keep regression validation separate from existing jobs and cover owner failure recovery and handoff corner cases.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use the pinned producer whitespace semantics rather than Python strip defaults and reject non-finite JSON evidence. Compare body hashes directly with the pinned runtime implementation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
singhalrohit4u and others added 2 commits October 17, 2026 08:12
…te evidence

Add an independent read-only preflight that emits a local noop for invalid issue numbers before native AI/reporting jobs can run. Do not treat shared missing diagnostics or earlier unverified automated assertions as proof of duplication. Preserve native detection for valid requests and all legacy workflow definitions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Apply current evidence rules before comment deduplication and distinguish lagging catalog entries from verified latest publication evidence. Preserve valid unchanged outcomes and scope reproduction requests to older reported versions. Add regression contracts without changing runtime or legacy workflows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Untriaged

Development

Successfully merging this pull request may close these issues.

3 participants