Skip to content

Commit 2dca3f2

Browse files
accesswatchGitHub Workshop Botclaude
authored
fix(provisioning): seed Challenge 1 at provision time and tolerate pending invitations (#236)
Provisioned learning rooms started with an empty Issues tab: the Student Progression Bot only fires on issue close or manual dispatch, so nothing ever created Challenge 1 and the progression chain never began (reported in #226 and #216; every room in the self-paced-2026 cohort was affected). - provision-core: after content verification, seed the first challenge issue via an optional client capability; a seeding failure marks the learner failed and surfaces to the watchdog. - seed-first-challenge: new provisioning module that runs the learning-room challenge-progression script against the student repo with the App installation token. Idempotent via the script's own duplicate check. - challenge-progression: validate assignability in the repository (assignees endpoint) instead of global user existence. At provision time the collaborator invitation is still pending, so assignment 422s; the issue is now created unassigned and later challenges are assigned to whoever closes the previous one. - SPEC.md 7.2a/7.2b updated to match; HTML regenerated. Co-authored-by: GitHub Workshop Bot <workshop@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent e1fd386 commit 2dca3f2

10 files changed

Lines changed: 342 additions & 54 deletions

File tree

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
// Regression tests for the assignee handling in the student-facing challenge
2+
// progression script. At provisioning time the learner's collaborator
3+
// invitation is still pending, so they exist as a GitHub user but cannot yet
4+
// be assigned issues in the repository; the script must degrade to an
5+
// unassigned issue instead of failing with HTTP 422 (see learning rooms for
6+
// waphi and accesswatch-student on 2026-07-02).
7+
const test = require('node:test');
8+
const assert = require('node:assert/strict');
9+
const path = require('node:path');
10+
11+
process.env.GITHUB_TOKEN = 'test-token';
12+
process.env.GITHUB_REPOSITORY = 'org/room';
13+
process.env.START_CHALLENGE = '';
14+
process.env.GITHUB_EVENT_NAME = '';
15+
process.env.CLOSED_ISSUE_TITLE = '';
16+
17+
// The script resolves its issue-template directory from the working
18+
// directory, so requiring it must happen from inside learning-room/.
19+
process.chdir(path.join(__dirname, '..', '..', '..', 'learning-room'));
20+
21+
const { resolveValidAssignee } = require('../../../learning-room/.github/scripts/challenge-progression');
22+
23+
test('validates assignability in the repository, not global user existence', async () => {
24+
const routes = [];
25+
const request = async (route) => {
26+
routes.push(route);
27+
return null;
28+
};
29+
30+
const result = await resolveValidAssignee('waphi', request);
31+
32+
assert.equal(result, 'waphi');
33+
assert.deepEqual(routes, ['/repos/org/room/assignees/waphi']);
34+
});
35+
36+
test('returns null when the user cannot be assigned yet (pending invitation)', async () => {
37+
const request = async () => {
38+
throw new Error('GET /repos/org/room/assignees/waphi failed: 404');
39+
};
40+
41+
assert.equal(await resolveValidAssignee('waphi', request), null);
42+
});
43+
44+
test('returns null for a missing actor without calling the API', async () => {
45+
let called = false;
46+
const request = async () => {
47+
called = true;
48+
};
49+
50+
assert.equal(await resolveValidAssignee('', request), null);
51+
assert.equal(called, false);
52+
});

‎.github/scripts/provisioning/__tests__/provision-core.test.js‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -216,6 +216,54 @@ test('a failure does not stop the batch; others still provision', async () => {
216216
assert.equal(summary.created, 1);
217217
});
218218

219+
test('seeds the first challenge issue after workflows are verified', async () => {
220+
const seedCalls = [];
221+
const { client, state } = makeClient({
222+
async seedFirstChallenge({ owner, repo, assignee }) {
223+
seedCalls.push({ owner, repo, assignee, listCallsSoFar: state.calls.list });
224+
}
225+
});
226+
const roster = rosterWith({ github_handle: 'alice', cohort_id: 'c1' });
227+
const { roster: out, summary } = await provisionRoster({ roster, client, config, sleep: noSleep });
228+
229+
assert.equal(out.learners[0].provision_state, 'provisioned');
230+
assert.equal(summary.error, 0);
231+
assert.equal(seedCalls.length, 1);
232+
assert.equal(seedCalls[0].owner, 'Community-Access');
233+
assert.equal(seedCalls[0].repo, 'learning-room-c1-alice');
234+
assert.equal(seedCalls[0].assignee, 'alice');
235+
assert.ok(seedCalls[0].listCallsSoFar >= 1, 'seeding must happen after workflow verification');
236+
});
237+
238+
test('seeds the first challenge on the already-exists path too', async () => {
239+
const seedCalls = [];
240+
const { client, state } = makeClient({
241+
async seedFirstChallenge(args) {
242+
seedCalls.push(args);
243+
}
244+
});
245+
state.repos.set('learning-room-c1-alice', { workflows: [...REQUIRED] });
246+
const roster = rosterWith({ github_handle: 'alice', cohort_id: 'c1' });
247+
const { summary } = await provisionRoster({ roster, client, config, sleep: noSleep });
248+
249+
assert.equal(summary.already_exists, 1);
250+
assert.equal(seedCalls.length, 1);
251+
});
252+
253+
test('marks the learner failed when first-challenge seeding fails', async () => {
254+
const { client } = makeClient({
255+
async seedFirstChallenge() {
256+
throw new Error('seed boom');
257+
}
258+
});
259+
const roster = rosterWith({ github_handle: 'alice', cohort_id: 'c1' });
260+
const { roster: out, log, summary } = await provisionRoster({ roster, client, config, sleep: noSleep });
261+
262+
assert.equal(out.learners[0].provision_state, 'failed');
263+
assert.equal(summary.error, 1);
264+
assert.match(log[0].error_detail, /seed boom/);
265+
});
266+
219267
test('isSecondaryRateLimit detects status and message forms', () => {
220268
assert.ok(isSecondaryRateLimit({ status: 429 }));
221269
assert.ok(isSecondaryRateLimit({ message: 'oops HTTP 403 here' }));
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
const test = require('node:test');
2+
const assert = require('node:assert/strict');
3+
4+
const { createChallengeSeeder } = require('../seed-first-challenge');
5+
6+
function fakeSpawn(result) {
7+
const calls = [];
8+
const spawn = (cmd, args, opts) => {
9+
calls.push({ cmd, args, opts });
10+
return { status: 0, stdout: '', stderr: '', ...result };
11+
};
12+
return { calls, spawn };
13+
}
14+
15+
test('requires a token and a learning room directory', () => {
16+
assert.throws(() => createChallengeSeeder({ learningRoomDir: '/lr' }), /token/);
17+
assert.throws(() => createChallengeSeeder({ token: 't' }), /learningRoomDir/);
18+
});
19+
20+
test('runs the challenge progression script against the student repo', async () => {
21+
const { calls, spawn } = fakeSpawn({ stdout: 'Created Challenge 1' });
22+
const seeder = createChallengeSeeder({ token: 'tok', learningRoomDir: '/lr', spawn });
23+
24+
const result = await seeder.seedFirstChallenge({
25+
owner: 'org',
26+
repo: 'room',
27+
assignee: 'alice'
28+
});
29+
30+
assert.equal(calls.length, 1);
31+
const call = calls[0];
32+
assert.equal(call.cmd, process.execPath);
33+
assert.match(call.args[0], /challenge-progression\.js$/);
34+
assert.equal(call.opts.cwd, '/lr');
35+
assert.equal(call.opts.env.GITHUB_TOKEN, 'tok');
36+
assert.equal(call.opts.env.GITHUB_REPOSITORY, 'org/room');
37+
assert.equal(call.opts.env.START_CHALLENGE, '1');
38+
assert.equal(call.opts.env.CHALLENGE_ASSIGNEE, 'alice');
39+
assert.equal(result.output, 'Created Challenge 1');
40+
});
41+
42+
test('passes an empty assignee through as an empty string', async () => {
43+
const { calls, spawn } = fakeSpawn({});
44+
const seeder = createChallengeSeeder({ token: 'tok', learningRoomDir: '/lr', spawn });
45+
await seeder.seedFirstChallenge({ owner: 'org', repo: 'room' });
46+
assert.equal(calls[0].opts.env.CHALLENGE_ASSIGNEE, '');
47+
});
48+
49+
test('throws with script output when the script exits non-zero', async () => {
50+
const { spawn } = fakeSpawn({ status: 1, stderr: 'FATAL: creation exploded' });
51+
const seeder = createChallengeSeeder({ token: 'tok', learningRoomDir: '/lr', spawn });
52+
53+
await assert.rejects(
54+
() => seeder.seedFirstChallenge({ owner: 'org', repo: 'room', assignee: 'alice' }),
55+
/creation exploded/
56+
);
57+
});
58+
59+
test('throws when the script cannot be spawned at all', async () => {
60+
const { spawn } = fakeSpawn({ error: new Error('ENOENT node missing'), status: null });
61+
const seeder = createChallengeSeeder({ token: 'tok', learningRoomDir: '/lr', spawn });
62+
63+
await assert.rejects(
64+
() => seeder.seedFirstChallenge({ owner: 'org', repo: 'room', assignee: 'alice' }),
65+
/ENOENT/
66+
);
67+
});

‎.github/scripts/provisioning/provision-cli.js‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ const path = require('node:path');
2626
const { parseRoster, serializeRoster } = require('./roster');
2727
const { provisionRoster } = require('./provision-core');
2828
const { createFetchClient, REQUIRED_WORKFLOWS } = require('./github-client');
29+
const { createChallengeSeeder } = require('./seed-first-challenge');
2930
const { mintInstallationToken, discoverInstallationId } = require('./github-app-auth');
3031

3132
function parseArgs(argv) {
@@ -161,7 +162,13 @@ async function main() {
161162
}
162163

163164
const token = await resolveToken(env, apiBaseUrl);
164-
const client = createFetchClient({ token, apiBaseUrl });
165+
// The seeder runs the learning-room progression script from this checkout,
166+
// so provisioned rooms start with their Challenge 1 issue already open.
167+
const seeder = createChallengeSeeder({
168+
token,
169+
learningRoomDir: path.join(__dirname, '..', '..', '..', 'learning-room')
170+
});
171+
const client = { ...createFetchClient({ token, apiBaseUrl }), ...seeder };
165172

166173
// Preflight: fail with one clear message if the token cannot even see the
167174
// template, instead of a confusing per-learner error cascade.

‎.github/scripts/provisioning/provision-core.js‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,18 @@ async function provisionOne({
207207
);
208208
}
209209

210+
// A room without a Challenge 1 issue never starts the progression chain:
211+
// the Student Progression Bot only fires on issue close or manual dispatch.
212+
// Seed after content verification so the issue never points at an empty
213+
// repo. The seeder is idempotent (skips when a challenge issue exists).
214+
if (typeof client.seedFirstChallenge === 'function') {
215+
await client.seedFirstChallenge({
216+
owner: studentOwner,
217+
repo: repoName,
218+
assignee: handle
219+
});
220+
}
221+
210222
let templateSha = null;
211223
if (typeof client.getDefaultBranchSha === 'function') {
212224
templateSha = await client.getDefaultBranchSha({ owner: studentOwner, repo: repoName });
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
/**
2+
* First-challenge seeding for provisioning (SPEC.md section 7.2b step 3).
3+
*
4+
* A freshly provisioned learning room has no issues, and the Student
5+
* Progression Bot inside the room only fires when a challenge issue is closed
6+
* or manually dispatched - so without this step every new room starts with an
7+
* empty Issues tab and the progression chain never begins.
8+
*
9+
* Rather than duplicating the issue-rendering logic, this module runs the same
10+
* challenge-progression.js script the student repo uses, from the workshop
11+
* checkout's learning-room/ directory, pointed at the student repo with the
12+
* App installation token. The script is idempotent (it skips creation when a
13+
* matching challenge issue already exists) and degrades to an unassigned issue
14+
* when the learner's collaborator invitation is still pending.
15+
*
16+
* Note: challenge-progression.js talks to https://api.github.com directly; it
17+
* does not honor GITHUB_API_URL. That matches every current deployment.
18+
*/
19+
20+
'use strict';
21+
22+
const fs = require('node:fs');
23+
const path = require('node:path');
24+
const { spawnSync } = require('node:child_process');
25+
26+
function createChallengeSeeder({
27+
token,
28+
learningRoomDir,
29+
spawn = spawnSync,
30+
execPath = process.execPath
31+
}) {
32+
if (!token) throw new Error('createChallengeSeeder requires a token');
33+
if (!learningRoomDir) throw new Error('createChallengeSeeder requires learningRoomDir');
34+
35+
const scriptPath = path.join(learningRoomDir, '.github', 'scripts', 'challenge-progression.js');
36+
// Fail at construction, not per learner, when the checkout layout is wrong.
37+
if (spawn === spawnSync && !fs.existsSync(scriptPath)) {
38+
throw new Error(`createChallengeSeeder cannot find ${scriptPath}`);
39+
}
40+
41+
return {
42+
async seedFirstChallenge({ owner, repo, assignee }) {
43+
const result = spawn(execPath, [scriptPath], {
44+
cwd: learningRoomDir,
45+
encoding: 'utf8',
46+
env: {
47+
...process.env,
48+
GITHUB_TOKEN: token,
49+
GITHUB_REPOSITORY: `${owner}/${repo}`,
50+
GITHUB_EVENT_NAME: 'provisioning-seed',
51+
START_CHALLENGE: '1',
52+
CHALLENGE_ASSIGNEE: assignee || ''
53+
}
54+
});
55+
56+
if (result.error) throw result.error;
57+
if (result.status !== 0) {
58+
const detail = String(result.stderr || result.stdout || '')
59+
.trim()
60+
.split(/\r?\n/)
61+
.slice(-5)
62+
.join(' | ');
63+
throw new Error(
64+
`seedFirstChallenge failed for ${owner}/${repo} (exit ${result.status}): ${detail}`
65+
);
66+
}
67+
return { output: String(result.stdout || '').trim() };
68+
}
69+
};
70+
}
71+
72+
module.exports = { createChallengeSeeder };

‎SPEC.md‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -219,7 +219,7 @@ Grant only these. Anything beyond this list is over-privileged and fails the sec
219219
| Repository administration | Write | Create the per-student private repository from the template |
220220
| Contents | Write | Seed and heal repository content (workflows, issue templates) |
221221
| Metadata | Read | Mandatory baseline for any App |
222-
| Issues | Write | Seed the first challenge issue and provisioning status (optional; can defer to the Progression Bot) |
222+
| Issues | Write | Seed the first challenge issue at provision time. Not deferrable: the Progression Bot only fires on issue close or manual dispatch, so a room provisioned without a Challenge 1 issue never starts the progression chain |
223223

224224
App configuration rules:
225225

@@ -256,7 +256,15 @@ for each roster entry where provision_state in (pending, failed):
256256
- On failure: provision_state = failed, write error_detail,
257257
surface to watchdog. Do NOT leave a half-seeded repo silently.
258258
259-
5. Wait a short delay (1 to a few seconds) before the next entry.
259+
5. Seed the Challenge 1 issue if no challenge issue exists yet
260+
(idempotent; runs the learning-room progression script against the
261+
student repo). The learner's collaborator invitation is usually still
262+
pending at this point, so the issue is created unassigned; the
263+
Progression Bot assigns subsequent challenges to whoever closes the
264+
previous one. A seeding failure marks the learner failed and surfaces
265+
to the watchdog.
266+
267+
6. Wait a short delay (1 to a few seconds) before the next entry.
260268
On HTTP 403/429 (secondary limit), back off exponentially and retry
261269
the same entry; never skip it.
262270
```

‎html/SPEC.html‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -628,7 +628,7 @@ <h3 id="72a-github-app-permission-set">7.2a GitHub App permission set</h3>
628628
<tr>
629629
<td>Issues</td>
630630
<td>Write</td>
631-
<td>Seed the first challenge issue and provisioning status (optional; can defer to the Progression Bot)</td>
631+
<td>Seed the first challenge issue at provision time. Not deferrable: the Progression Bot only fires on issue close or manual dispatch, so a room provisioned without a Challenge 1 issue never starts the progression chain</td>
632632
</tr>
633633
</tbody></table>
634634
<p>App configuration rules:</p>
@@ -663,7 +663,15 @@ <h3 id="72b-provisioning-algorithm-idempotent-serial">7.2b Provisioning algorith
663663
- On failure: provision_state = failed, write error_detail,
664664
surface to watchdog. Do NOT leave a half-seeded repo silently.
665665

666-
5. Wait a short delay (1 to a few seconds) before the next entry.
666+
5. Seed the Challenge 1 issue if no challenge issue exists yet
667+
(idempotent; runs the learning-room progression script against the
668+
student repo). The learner&#x27;s collaborator invitation is usually still
669+
pending at this point, so the issue is created unassigned; the
670+
Progression Bot assigns subsequent challenges to whoever closes the
671+
previous one. A seeding failure marks the learner failed and surfaces
672+
to the watchdog.
673+
674+
6. Wait a short delay (1 to a few seconds) before the next entry.
667675
On HTTP 403/429 (secondary limit), back off exponentially and retry
668676
the same entry; never skip it.</code></pre>
669677
<p>Invariants: at-most-one repository per key; pinned template SHA; learner has access; required workflows present; every outcome written to the provisioning log; any failure visible to a human before a learner notices.</p>

0 commit comments

Comments
 (0)