GitHub Actions

The ROOT project uses a GitHub Actions for continuous integration and delivery.


For building Docker images we use GitHub’s runners. For everything else, runners are hosted at CERN.

Registering a machine as a GitHub Actions Runner:

For Linux runners, see: #Adding Linux runners

Open Click the “New self-hosted runner” button and follow the instructions. (Or click here)

Runner requirements


Install the runner as a system service by following the instructions at the GitHub docs (click here for archive link)


When asked to install the runner as a service, enter yes.

Adding/Configuring Linux runners

note If you want to add python packages or dependencies to runners, this should be added to the images in The following is for managing the machines themselves.


Linux runners are configured using Puppet and managed with Foreman. The most relevant documentation for configuration can be found on ConfigDocs.

The Puppet configuration of the runners are stored on GitLab at:

Although using management tools is annoying (even awful), they simplify batch operations and ensure uniformity. Here are some examples of common tasks to show the strengths of Foreman/Puppet.

Creating a new runner

# Node management has to be done in aiadm

# Switch to appropriate project
eval $(ai-rc 'PH LCGAA')

# Create a foreman managed cs8 runner with 16 vcpus, 32 GB ram
ai-bs --nova-flavor m2.2xlarge  \
      -g lcgapp/build/root      \
      --cs8                     \
      --environment rootci_test \

Note that no further setup is needed. It takes about 20-30 minutes for the new runner to come online.

Adding 10 extra runners

The following example shows how to add ten new runners starting from githubci-lcgapp-21 to githubci-lcgapp-30.

# note! only been tested with bash

# Node management has to be done in aiadm

# Switch to relevant project
eval $(ai-rc 'PH LCGAA')

# Configuration (here are sensible defaults)
flavor='m2.2xlarge' # 16 vcpus, 32 GB ram
node='lcgapp/build/root' # Foreman hostgroup
environment='rootci_test' # which branch the config is on

# Create runners from 21 to 30
for i in $(seq -f "%02g" 21 30); do
    ai-bs --nova-flavor $flavor \
          -g            $node   \
          --cs8                 \
          --foreman-environment $environment \

Adding a script to clean old containers

The following is intended as an example to show how to change behavior of the runners (outside of the containers). In a perfect world, this will never have to be touched.

Clone the configuration repository in aiadm, edit the CI runners config:

git clone

cd it-puppet-hostgroup-lcgapp

$EDITOR code/manifests/build/root.pp

Add cleanup script and cronjob to run it:


  # Add cleanup script for github runners

  file { '/':
    ensure  => file,
    mode    => '0500',
    owner   => $runner_uid,
    content => '#!/bin/sh
podman system prune --force --volumes',

  -> cron { 'clean-job':
    ensure  => present,
    command => '/',
    user    => rootci,
    hour    => 2,
    minute  => 0,

Update github actions environment:

$EDITOR data/hostgroup/lcgapp/build/root.yaml
 github_actions_runner::group: 'rootci'
     repo_name: root

Now it usually takes about 1 hour for the changes to show up. To immidiately have a node pull the changes to verify that the new feature is functional, we can ssh into one of the nodes and run puppet agent -t.

Here is a utility script for running puppet agent -t on all machines.

# First machine's number

# Last machine's number

pullChange() {
    echo "refreshing $1"
    ssh -oStrictHostKeyChecking=no -f "$1" 'puppet agent -t'

for i in $(seq -f "%02g" $start $end); do
    pullChange root@githubci-lcgapp-$i &
    sleep 1