Skip to content

Sync .github/skills directory with azure-sdk-tools for PR 16932 - #50381

Merged
Helen Gao (helen229) merged 1 commit into
mainfrom
sync-.github-skills-fix/skill-eval-environment-skills-6787054-16932
Sep 8, 2026
Merged

Helen Gao (helen229) merged 1 commit into
mainfrom
sync-.github-skills-fix/skill-eval-environment-skills-6787054-16932

Conversation

@azure-sdk-automation

Copy link
Copy Markdown
Contributor

Sync .github/skills directory with azure-sdk-tools for PR Azure/azure-sdk-tools#16932 See eng/common workflow

@azure-sdk-automation
azure-sdk-automation Bot requested review from Helen Gao (helen229) and a lite review from Copilot September 8, 2026 18:36
@azure-sdk-automation
azure-sdk-automation Bot requested a review from a team as a code owner September 8, 2026 18:36
@azure-sdk-automation azure-sdk-automation Bot added EngSys This issue is impacting the engineering system. Central-EngSys This issue is owned by the Engineering System team. labels Sep 8, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
36 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

One updated anti-trigger stimulus in the pipeline-analysis trigger eval does not require the intended boundary skill, weakening the routing test and allowing false passes.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR syncs the .github/skills trigger evaluation YAMLs with upstream changes from azure-sdk-tools PR 16932, primarily to make skill routing tests more deterministic by explicitly constraining which skills are available per stimulus.

Changes:

  • Adds per-stimulus environment.skills (via YAML anchors) to scope routing to the intended skill(s).
  • Updates several trigger prompts to more explicitly encode intent (e.g., including a release plan or build reference).
  • Strengthens multiple anti-trigger cases by requiring the “correct” alternative skill while disallowing the target skill.
File summaries
File Description
.github/skills/azsdk-common-sdk-release/evals/trigger.eval.yaml Adds scoped skill environments and updates trigger/anti-trigger prompts for release routing.
.github/skills/azsdk-common-prepare-release-plan/evals/trigger.eval.yaml Adds scoped skill environments and strengthens boundary anti-triggers for release-plan authoring.
.github/skills/azsdk-common-pipeline-fixer/evals/trigger.eval.yaml Adds scoped skill environments and an explicit analysis boundary (requires analysis skill, disallows fixer).
.github/skills/azsdk-common-pipeline-analysis/evals/trigger.eval.yaml Adds scoped skill environments and boundary anti-triggers; one TypeSpec boundary case remains weaker than others.
.github/skills/azsdk-common-apiview-feedback-resolution/evals/trigger.eval.yaml Adds scoped skill environments and boundary anti-triggers to avoid incorrect routing.
Review details
  • Files reviewed: 5/5 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

@helen229
Helen Gao (helen229) merged commit 6c8eed4 into main Sep 8, 2026
19 of 20 checks passed
@helen229
Helen Gao (helen229) deleted the sync-.github-skills-fix/skill-eval-environment-skills-6787054-16932 branch September 8, 2026 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Central-EngSys This issue is owned by the Engineering System team. EngSys This issue is impacting the engineering system.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants