Continuous integration

The reusable workflow

mutator provides a reusable workflow at PRL-PRG/mutator/.github/workflows/mutation-testing.yaml. A repository adopts it with a short caller workflow rather than by copying any script. The workflow checks out your package, installs it together with mutator and its dependencies, runs mutate_package(), uploads a results artifact, and writes a Markdown summary to the run’s Summary tab.

Add .github/workflows/mutation-testing.yaml to your repository:

on:
  pull_request:
  push:
    branches: [main, master]
  workflow_dispatch:

name: mutation-testing

jobs:
  mutation:
    uses: PRL-PRG/mutator/.github/workflows/[email protected]
    with:
      target-margin: "0.10"

Pin to a released tag such as @v0.2.0. The reusable workflow lives in the mutator package repository and is versioned with the package, so the tag you pin matches a mutator release. Bump the tag in your caller when you want to move to a newer release. You can also pin to a branch (for example @main) to track the latest changes, at the cost of reproducibility.

Inputs

All inputs are optional.

Enforcing a minimum mutation score

Set fail-under to the percentage below which the job should fail. The workflow computes the mutation score, and if it is below the threshold the job exits non-zero, which blocks the pull request:

jobs:
  mutation:
    uses: PRL-PRG/mutator/.github/workflows/[email protected]
    with:
      fail-under: "75"
      target-margin: "0.05"

Two points make a threshold gate reliable:

1. Reduce sampling noise. The score is estimated from a sample of mutants, so a run-to-run variation exists. Use a tighter target-margin (for example "0.05") or a large max-mutants when you gate on fail-under, so the estimate does not cross the threshold by chance. The half-width you choose is roughly how far the reported score can sit from the true score.
2. Ratchet the threshold. Start fail-under at, or slightly below, your current score, then raise it as you add tests. This locks in progress without failing the build on the day you adopt it.

Leave fail-under empty to report the score on every run without failing.

The measured score, its confidence interval, and the killed, survived, and hanged counts appear in the run’s Summary tab and in the uploaded artifact regardless of whether a threshold is set.

Badge

Set deploy-badge: true and grant the caller job contents: write. The workflow writes a shields.io endpoint JSON file and deploys it to gh-pages (or badge-branch):

jobs:
  mutation:
    permissions:
      contents: write
    uses: PRL-PRG/mutator/.github/workflows/[email protected]
    with:
      deploy-badge: true

Reference the badge in your README, replacing OWNER/REPO:

![mutator](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/OWNER/REPO/gh-pages/mutation-score.json)

This reads the JSON straight from badge-branch via raw.githubusercontent.com, so no GitHub Pages setup is needed. If you instead fetch it through a pkgdown Pages site (https://OWNER.github.io/REPO/mutation-score.json), GitHub Pages must be activated and serving from badge-branch, or the badge will not appear.

The badge is labelled mutator; its message shows the score and, when the run sampled mutants, a compact asymmetric confidence interval such as 83.2% -5.2/+4.7% (95% CI). The badge colour tracks the score: green above 80%, yellow above 60%, orange above 40%, and red below.

Set show-confidence-interval: false to show only the mutation score in both the badge and the job summary. The confidence interval remains part of the uploaded mutation results.

Permissions and private dependencies

The workflow reads GITHUB_TOKEN to install packages from GitHub, which is provided automatically. You only need to set permissions: contents: write on the caller job when deploy-badge is true, because publishing the badge pushes to a branch. Pull request runs never deploy the badge.

If your package depends on packages that are not on CRAN, pass them through extra-packages as pak specs:

    with:
      extra-packages: "github::owner/dependency, any::somePackage"

Tuning the run

Mutation testing runs the test suite once per mutant, so a large package can take a while. To keep CI fast:

• Cap work with max-mutants, or let target-margin size the sample for you.
• coverage-guided is on by default and runs only the tests that cover each mutated line. It applies to packages tested with testthat; for installed-tests packages the workflow warns and runs the full suite.
• Use exclude-files to skip generated or vendored files under R/.
• Raise cores on a larger runner, or leave it at "0" to use every core.

See the Configuration article for how these options behave in detail.