maintainer-tools

Workflows

Workflows for use by maintainers. These should be run from your fork of this repository, with an encrypted secret called ACCESS_TOKEN that is a personal access token with repo and workflow scopes.

PR Script

The PR Script Workflow allows you to make a commit against a PR as a maintainer without having to check out the PR locally and push the change. The manual workflow takes as its inputs a link to the PR and a comma-separated list of quoted commands to run. As a convenience, you can also type "True" for the option to run pre-commit against the PR to fix up any pre-commit errors.

Actions

Base Setup

Use this action to consolidate setup steps and caching in your workflows. You can control the versions of Python and Node used by setting matrix.python-version and matrix.node-version, respectively. An example workflow file would be:

name: Tests

on:
  push:
    branches: ["main"]
  pull_request:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Base Setup
        uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
      - name: Install
        shell: bash
        run: pip install -e ".[test]"
      - name: Test
        shell: bash
        run: pytest

If you want to use your minimum dependencies, you can use the following option, which will create a constraints file and set the PIP_CONSTRAINT environment variable, so that installations will use that file. By default the Python version will be "3.10", which can be overridden with python_version. Note that the environment variable also works if you use virtual environments like hatch. Note: this does not work on Windows, and will error.

  minimum_version:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Base Setup
        uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
        with:
          dependency_type: minimum
      - name: Install
        run: pip install -e ".[test]"
      - name: Test
        run: pytest

If you want to use your pending dependencies, you can use the following option, which will create a constraints file and set the PIP_CONSTRAINT environment variable, so that installations will use that file. By default the Python version will be "3.15", which can be overridden with python_version. Note that the environment variable also works if you use virtual environments like hatch. Note: this does not work on Windows, and will error.

  prereleases:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Base Setup
        uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
        with:
          dependency_type: pre
      - name: Install
        run: pip install -e ".[test]"
      - name: Test
        run: pytest

Check Links

Use this action to check the links in your repo using pytest-check-links. It will ignore links to GitHub and cache links to save time.

When adding this to a repo, you may need to skip some files or links. If the build fails, you can copy the "Checking files with command" used in the build, and add the appropriate --ignore-glob and --check-links-ignore until the tests pass locally, and add them as ignore_glob and ignore_links inputs to the action, respectively.

name: Check Links

on:
  push:
    branches: ["main"]
  pull_request:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
      - uses: jupyterlab/maintainer-tools/.github/actions/check-links@v1

Enforce Labels

Use this action to enforce one of the triage labels on PRs in your repo (one of documentation, bug, enhancement, feature, maintenance). An example workflow file would be:

name: Enforce PR label

on:
  pull_request:
    types: [labeled, unlabeled, opened, edited, synchronize]

jobs:
  enforce-label:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
    steps:
      - name: enforce-triage-label
        uses: jupyterlab/maintainer-tools/.github/actions/enforce-label@v1

Pre-Commit Check

Use this action to run a pre commit check with a manual stage. It will print a suitable error message on failure.

name: Pre-Commit Check
on:

on:
  push:
    branches: ["main"]
  pull_request:

jobs:
  pre_commit:
   runs-on: ubuntu-latest
   steps:
    - uses: actions/checkout@v2
    - uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
    - uses: jupyterlab/maintainer-tools/.github/actions/pre-commit@v1

Test Downstream Libraries

Use this action to test a package against downstream libraries. This can be used to catch breaking changes prior to merging them. An example workflow file would be:

name: Downstream Tests

on:
  push:
    branches: ["main"]
  pull_request:

jobs:
  build:
    runs-on: ubuntu-latest
  steps:
    - name: Checkout
      uses: actions/checkout@v2
    - name: Base Setup
      uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
    - name: Test Against Foo
      uses: jupyterlab/maintainer-tools/.github/actions/downstream-test@v1
      with:
        package_name: foo
    - name: Test Against Bar
      uses: jupyterlab/maintainer-tools/.github/actions/downstream-test@v1
      with:
        package_name: bar
        env_values: "FIZZ=buzz NAME=snuffy"

To test against a prerelease use package_download_extra_args: "--pre".

Test SDist

Use this pair of actions to build an sdist for your package, and then test it in an isolated environment.

name: Test Sdist
on:
  push:
    branches: ["main"]
  pull_request:

jobs:
  make_sdist:
    name: Make SDist
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v2
      - uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
      - uses: jupyterlab/maintainer-tools/.github/actions/make-sdist@v1

  test_sdist:
    runs-on: ubuntu-latest
    needs: [make_sdist]
    name: Install from SDist and Test
    timeout-minutes: 20
    steps:
      - uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
      - uses: jupyterlab/maintainer-tools/.github/actions/test-sdist@v1

PR Binder Link

Use this action to add binder links for testing PRs, which show up as a comment. You can use the optional url_path parameter to use a different url than the default lab. An example workflow would be:

name: Binder Badge
on:
  pull_request_target:
    types: [opened]

jobs:
  binder:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
    steps:
      - uses: jupyterlab/maintainer-tools/.github/actions/binder-link@v1
        with:
          github_token: ${{ secrets.github_token }}

PR Script

You can use the PR Script action in your repo along with pull-request-comment-trigger to enable maintainers to comment on PRs to run a script against a pull request. The script can only be run by a org member, collaborator, or repo owner if the association parameter is used (as in the examples below).

Note that the resulting commit will not trigger the workflows to run again. You will have to close/reopen the PR, or push another commit for the workflows to run again. If this behavior is not desirable, you can use a personal access token instead of the default GitHub token provided to the workflow. Make sure the token used is of as limited scope as possible (preferably a bot account token with access to the public_repo scope only).

This first example allows maintainers to run pre-commit by commenting "auto run pre-commit" on a Pull Request.

name: Trigger Pre-Commit on a PR
on:
  issue_comment:
    types: [created]

permissions:
  contents: write
  pull-requests: write

jobs:
  pr-script:
    runs-on: ubuntu-latest
    steps:
      - uses: khan/pull-request-comment-trigger@1.0.0
        id: check
        with:
          trigger: "auto run pre-commit"
      - if: steps.check.outputs.triggered == 'true'
        uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
      - if: steps.check.outputs.triggered == 'true'
        uses: jupyterlab/maintainer-tools/.github/actions/pr-script@v1
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          pre_commit: true
          commit_message: "auto run pre-commit"
          target: ${{ github.event.issue.html_url }}
          association: ${{ github.event.comment.author_association }}

In this example, the repo has a custom script that should be run, which is triggered by a PR comment "auto run cleanup". Again, this can only be run by a org member, collaborator, or repo owner.

name: Trigger a Cleanup Script on a PR
on:
  issue_comment:
    types: [created]

permissions:
  contents: write
  pull-requests: write

jobs:
  pr-script:
    runs-on: ubuntu-latest
    steps:
      - uses: khan/pull-request-comment-trigger@1.0.0
        id: check
        with:
          trigger: "auto run cleanup"
      - if: steps.check.outputs.triggered == 'true'
        uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
      - if: steps.check.outputs.triggered == 'true'
        uses: jupyterlab/maintainer-tools/.github/actions/pr-script@v1
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          script: '["jlpm run integrity", "jlpm run lint"]'
          commit_message: "auto run cleanup"
          target: ${{ github.event.issue.html_url }}
          association: ${{ github.event.comment.author_association }}

Upload Coverage and Report Coverage

These actions are meant to be used together, to combine and enforce coverage. A coverage snapshot will be included in the workflow summary. If coverage is below threshold, the report-coverage action will fail and upload the html report.

name: Tests

on:
  push:
    branches: ["main"]
  pull_request:

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
    steps:
      - uses: actions/checkout@v2
      - uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
      - run: |
          pip install -e ".[test]"
          python -m coverage run -m pytest
      - uses: jupyterlab/maintainer-tools/.github/actions/upload-coverage@v1
  coverage_report:
    name: Combine & check coverage
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: jupyterlab/maintainer-tools/.github/actions/report-coverage@v1
        with:
          fail_under: 90

Inline Playwright Report

Use this action to inline image and video assets referenced by a Playwright HTML report into a single HTML file. This is useful for uploading and sharing one self-contained report artifact.

Inputs:

path (required): path to the Playwright report directory containing index.html
output (optional): output HTML path (defaults to output.html in the report directory)
verbose (optional): whether to print matched media references (yes/no, default: yes)
max_output_mb (optional): maximum output HTML size in MB (default: 200); images are always inlined first, then videos are inlined until the limit is reached.

Example workflow usage:

name: Publish Playwright Report

on:
  push:
    branches: ["main"]
  pull_request:

jobs:
  inline-report:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1

      - name: Generate Playwright HTML report
        run: |
          npm ci
          npx playwright test || true

      - name: Inline report assets
        uses: jupyterlab/maintainer-tools/.github/actions/inline-playwright-report@v1
        with:
          path: playwright-report
          output: playwright-report/report.html
          max_output_mb: 200

      - name: Upload inlined report
        if: always()
        uses: actions/upload-artifact@v7
        with:
          archive: false
          name: playwright-report-inlined
          path: playwright-report/report.html

UI Test Report Comment

Use this action from a PR comment workflow to add or update a badge linking to an inlined UI test report that opens directly in the browser. If a Binder preview comment exists, the badge is added to that comment; otherwise the action creates a standalone report comment.

The line it posts reads like this when every test passed:

View the UI test report in the browser:

and like this when some failed, where the badge links into the report already filtered to the failing tests:

View the UI test report in the browser:

It expects a completed UI test workflow run to upload a ui-test-report-comment-data artifact containing pr-comment-data.json:

{
  "prNumber": 123,
  "reportUrl": "https://github.com/OWNER/REPO/actions/runs/RUN_ID/artifacts/ARTIFACT_ID",
  "failing": 0,
  "flaky": 0
}

If failing or flaky is missing or not a non-negative integer, the badge shows an unknown status.

The PR comment workflow runs later with workflow_run and passes the completed UI test run id to this action.

For a pull request from a fork, the artifact is written by code the fork controls, so the action treats its contents as untrusted. It only comments when the run the artifact came from is the current head commit of the pull request it names, and it only accepts a report URL under that same run's artifact path, which is where upload-artifact puts them.

The action only ever edits comments written by a bot.

The line the badge sits on starts with View the UI test report in the browser: , and that prefix is also how the action finds a line it posted before so it can update it in place.

Example caller workflow:

name: Update PR comment with UI test report

on: # zizmor: ignore[dangerous-triggers] required to post from workflow_run artifact to PR comment
  workflow_run:
    workflows: ["UI Tests"]
    types: [completed]

permissions:
  actions: read
  pull-requests: write

jobs:
  update-preview-comment:
    if: ${{ github.event.workflow_run.event == 'pull_request' }}
    runs-on: ubuntu-latest
    steps:
      - uses: jupyterlab/maintainer-tools/.github/actions/ui-test-report-comment@v1
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          run_id: ${{ github.event.workflow_run.id }}

The name in workflows must match the name of the UI test workflow exactly, otherwise the workflow_run trigger never fires. The comment workflow runs whatever the UI tests concluded, so that a run with failing tests reports its failures. A run which was cancelled, or which died before it uploaded the artifact, has nothing to report and the comment workflow fails on the missing artifact.

If the JSON file in the artifact uses a different name, pass comment_data_file.

Matching UI test workflow, producing the report and the comment data:

name: UI Tests

on:
  pull_request:

jobs:
  ui-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1

      - name: Run Playwright tests
        run: |
          npm ci
          npx playwright test --reporter=html,json
        env:
          PLAYWRIGHT_JSON_OUTPUT_NAME: playwright-report.json

      - name: Inline report assets
        if: ${{ !cancelled() }}
        uses: jupyterlab/maintainer-tools/.github/actions/inline-playwright-report@v1
        with:
          path: playwright-report
          output: playwright-report/report.html

      - name: Upload inlined report
        id: upload-report
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@v7
        with:
          archive: false
          name: playwright-report-inlined
          path: playwright-report/report.html

      - name: Save PR comment data
        if: ${{ !cancelled() && github.event_name == 'pull_request' }}
        uses: actions/github-script@v8
        env:
          REPORT_URL: ${{ steps.upload-report.outputs.artifact-url }}
        with:
          script: |
            const fs = require('fs');
            let stats = {};
            try {
              stats = JSON.parse(fs.readFileSync('playwright-report.json', 'utf8')).stats;
            } catch {
              console.warn('No Playwright JSON report; the badge will show an unknown status.');
            }
            fs.writeFileSync('pr-comment-data.json', JSON.stringify({
              prNumber: context.payload.pull_request.number,
              reportUrl: process.env.REPORT_URL,
              failing: stats.unexpected,
              flaky: stats.flaky,
            }));

      - name: Upload PR comment data
        if: ${{ !cancelled() && github.event_name == 'pull_request' }}
        uses: actions/upload-artifact@v7
        with:
          name: ui-test-report-comment-data
          path: pr-comment-data.json
          retention-days: 1

The id on the upload step is what makes steps.upload-report.outputs.artifact-url resolve. The counts come from the stats object of the Playwright JSON reporter, where unexpected is the number of failing tests. Failing tests fail the job, and every step after them is guarded with !cancelled() so that the report and the comment data are still produced and the badge reports the failures.

Update snapshots

You can use update snapshots action to commit on a branch Playwright updated snapshots.

The requirements and constrains are:

You must be on the branch to which the snapshots will be committed
You must installed your project before calling the action
The action is using yarn package manager by default but can be configured with npm_client
The Playwright tests must be in TypeScript or JavaScript

An example of workflow that get triggered when a PR comment contains update playwright snapshots would be:

name: Update Playwright Snapshots

on:
  issue_comment:
    types: [created, edited]

permissions:
  contents: write
  pull-requests: write

jobs:
  update-snapshots:
    if: ${{ github.event.issue.pull_request && contains(github.event.comment.body, 'update playwright snapshots') }}
    runs-on: ubuntu-latest

    steps:
      - uses: jupyterlab/maintainer-tools/.github/actions/update-snapshots-checkout@main
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}

      - name: Install your project
        run: |
          # Execute the required installation command

      - name: Update snapshots
        uses: jupyterlab/maintainer-tools/.github/actions/update-snapshots@v1
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          # Test folder within your repository
          test_folder: playwright-tests

          # Optional npm scripts (the default values are displayed)
          # Script to start the server or 'null' if Playwright is taking care of it
          #   If not `null`, you must provide a `server_url` to listen to.
          start_server_script: start
          # Server url to wait for before updating the snapshots
          #  See specification for https://github.com/iFaxity/wait-on-action `resource`
          server_url: http-get://localhost:8888
          update_script: test:update

Name		Name	Last commit message	Last commit date
.github		.github
.pre-commit-config.yaml		.pre-commit-config.yaml
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
README.md		README.md
RELEASE.md		RELEASE.md
foobar.py		foobar.py
package.json		package.json
pyproject.toml		pyproject.toml

Navigation Menu

Uh oh!

Uh oh!

Repository files navigation

maintainer-tools

Workflows

PR Script

Actions

Base Setup

Check Links

Enforce Labels

Pre-Commit Check

Test Downstream Libraries

Test SDist

PR Binder Link

PR Script

Upload Coverage and Report Coverage

Inline Playwright Report

UI Test Report Comment

Update snapshots

About

Releases

Sponsor this project

Used by

Contributors

Languages

Uh oh!

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

maintainer-tools

Workflows

PR Script

Actions

Base Setup

Check Links

Enforce Labels

Pre-Commit Check

Test Downstream Libraries

Test SDist

PR Binder Link

PR Script

Upload Coverage and Report Coverage

Inline Playwright Report

UI Test Report Comment

Update snapshots

About

Topics

Resources

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

Sponsor this project

Used by

Contributors

Languages