Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
afd3248
server: preemption notices, asynchronous parks and exact concurrency,…
danielhanchen Sep 9, 2026
2834f07
Compose the server-side preemption pin with ggml-org#25731 (inkling) …
danielhanchen Sep 9, 2026
62cf502
Drop the fork's own workflow and script files from the pin tree; the …
danielhanchen Sep 9, 2026
1cfb42b
Pin that an exact-mode cross-sequence copy leaves both sequences as t…
danielhanchen Sep 9, 2026
17a0f3b
Condense the allocation granularity comment to two lines
danielhanchen Sep 9, 2026
1d55489
Park only when asked: --preempt-ram defaults to 0, so a server withou…
danielhanchen Sep 10, 2026
0b9caab
Read src[5] as a page table only for the op that has one
danielhanchen Sep 10, 2026
15bcc6c
Normalize sequence planes in every batch-invariant mode, not only und…
danielhanchen Sep 10, 2026
3e7d953
Fix the CI legs that broke on the last rebase: a portable setenv in t…
danielhanchen Sep 10, 2026
2176de1
Build test-exact-buft only where llama-impl.h links, queue the starte…
danielhanchen Sep 10, 2026
d0202ea
Condense the exact-mode weight check comment so it is not hard-wrapped
danielhanchen Sep 10, 2026
a05a318
tests : give the rotation test time to finish and stop the deliberate…
danielhanchen Sep 10, 2026
f8b7514
server : keep the tokens a recompute restore replays out of the reque…
danielhanchen Sep 10, 2026
0beed51
server : put the preemption record inside the completed response of a…
danielhanchen Sep 10, 2026
d040ae7
tests : condense the exact-pages preamble so it is not hard-wrapped
danielhanchen Sep 10, 2026
c5a89f7
llama : refuse an asynchronous state transfer for a model whose state…
danielhanchen Sep 10, 2026
7eb9cb7
tests : drive the started-slot trim from the prompt lengths instead o…
danielhanchen Sep 10, 2026
d8daa8f
tests : make the failed-save park a matter of lengths rather than of …
danielhanchen Sep 10, 2026
5b66acb
server : reset the prompt counters when a recompute park restarts a p…
danielhanchen Sep 10, 2026
b4433f5
server : number the preempt notices of every task of a batched reques…
danielhanchen Sep 10, 2026
41abdfb
tests : let the server log settle before asserting on its text
danielhanchen Sep 10, 2026
c5dc0f9
llama : keep the exact-mode host buffer comment to whole sentences
danielhanchen Sep 10, 2026
ae19539
tests : make the two generations that do not fit overlap by their len…
danielhanchen Sep 10, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 119 additions & 0 deletions .github/actions/prebuilt-alert/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.

name: Prebuilt failure alert
description: Open, update or close a deduplicated issue tracking prebuilt pipeline health.

inputs:
status:
description: 'failure or success'
required: true
key:
description: 'Dedup key; all alerts sharing it collapse onto one issue'
required: true
title:
description: 'Issue title'
required: true
details:
description: 'Markdown describing what broke'
required: false
default: ''
label:
description: 'Label applied to the issue'
required: false
default: 'prebuilt-failure'
token:
description: 'Token with issues:write'
required: true

runs:
using: composite
steps:
- name: Open, update or close the tracking issue
shell: bash
env:
GH_TOKEN: ${{ inputs.token }}
STATUS: ${{ inputs.status }}
KEY: ${{ inputs.key }}
TITLE: ${{ inputs.title }}
DETAILS: ${{ inputs.details }}
LABEL: ${{ inputs.label }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
REPO: ${{ github.repository }}
run: |
# GitHub runs `shell: bash` with -e, which `set -uo pipefail` does not
# undo. Alerting must never fail the run it reports on, so turn it off
# explicitly; every gh call below is already individually tolerated.
set +e
set -uo pipefail

case "$STATUS" in
failure|success) ;;
*) echo "::warning::prebuilt-alert: unknown status '$STATUS'"; exit 0 ;;
esac

MARKER="<!-- prebuilt-alert:${KEY} -->"

# Scan open labelled issues rather than the search index, which lags by
# minutes and would double-open on back-to-back runs.
EXISTING=""
while read -r num; do
[ -n "$num" ] || continue
if gh issue view "$num" --repo "$REPO" --json body --jq .body 2>/dev/null | grep -qF "$MARKER"; then
EXISTING="$num"; break
fi
done < <(gh issue list --repo "$REPO" --label "$LABEL" --state open \
--limit 100 --json number --jq '.[].number' 2>/dev/null || true)

if [ "$STATUS" = "success" ]; then
if [ -n "$EXISTING" ]; then
gh issue comment "$EXISTING" --repo "$REPO" \
--body "Recovered: [\`${GITHUB_WORKFLOW}\` run](${RUN_URL}) succeeded. Closing." >/dev/null 2>&1 || true
gh issue close "$EXISTING" --repo "$REPO" >/dev/null 2>&1 \
|| echo "::warning::prebuilt-alert: could not close #${EXISTING}"
echo "closed #${EXISTING} ($KEY)"
fi
exit 0
fi

BODY="$(printf '%s\n\n%s\n\n**Run:** %s\n\n%s\n' \
"$MARKER" \
"\`${GITHUB_WORKFLOW}\` failed on \`${GITHUB_EVENT_NAME}\`." \
"$RUN_URL" \
"$DETAILS")"

# The run summary always works: no token, no permission, no repo
# setting. Issues can be disabled (they are on a fresh fork), and an
# alert nobody can read is the failure this whole pipeline keeps
# having, so write the details somewhere visible before trying.
{
echo "## ${TITLE}"
echo
echo "$DETAILS"
} >> "${GITHUB_STEP_SUMMARY:-/dev/null}"

if [ -n "$EXISTING" ]; then
# Comment rather than open a second issue, so an outage is one thread.
gh issue comment "$EXISTING" --repo "$REPO" --body "$BODY" >/dev/null 2>&1 \
|| echo "::warning::prebuilt-alert: could not comment on #${EXISTING}"
echo "updated #${EXISTING} ($KEY)"
else
# gh refuses to create with a label that does not exist yet.
gh label create "$LABEL" --repo "$REPO" --color B60205 \
--description "Prebuilt release pipeline is failing" >/dev/null 2>&1 || true
NEW="$(gh issue create --repo "$REPO" --title "$TITLE" --label "$LABEL" \
--body "$BODY" 2>&1 | tail -1)"
case "$NEW" in
https://*) echo "opened $NEW ($KEY)" ;;
*)
# The one case worth shouting about: a real failure went
# unreported. Name the usual cause so it is actionable.
if [ "$(gh api "repos/${REPO}" --jq .has_issues 2>/dev/null)" = "false" ]; then
echo "::error::prebuilt-alert: issues are disabled on ${REPO}, so ${KEY} cannot be tracked; see the run summary for details"
else
echo "::error::prebuilt-alert: could not open an issue for ${KEY} (${NEW}); the failure at ${RUN_URL} is unreported"
fi
;;
esac
fi
exit 0
6 changes: 3 additions & 3 deletions .github/workflows/bench.yml.disabled
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ jobs:
tools/server/bench/*.log

- name: Commit status
uses: Sibz/github-status-action@v1
uses: Sibz/github-status-action@faaa4d96fecf273bd762985e0e7f9f933c774918 # v1
with:
authToken: ${{secrets.GITHUB_TOKEN}}
sha: ${{ inputs.sha || github.event.pull_request.head.sha || github.sha }}
Expand All @@ -172,7 +172,7 @@ jobs:
state: 'success'

- name: Upload benchmark images
uses: devicons/public-upload-to-imgur@v2.2.2
uses: devicons/public-upload-to-imgur@352cf5f2805c692539a96cfe49a09669e6fca88e # v2.2.2
continue-on-error: true # Important as it looks unstable: 503
id: imgur_step
with:
Expand Down Expand Up @@ -221,7 +221,7 @@ jobs:
echo "IMAGE_3=${{ fromJSON(steps.imgur_step.outputs.imgur_urls)[3] }}" >> $GITHUB_ENV

- name: Comment PR
uses: mshick/add-pr-comment@v2
uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2
id: comment_pr
if: ${{ github.event.pull_request != '' && matrix.pr_comment_enabled == 'true' }}
with:
Expand Down
Loading
Loading