This project includes all necessary components to spin up the infrastructure for VM based GitHub self-hosted runners. This project was created with some inspiration from the Philips Lab AWS Solution with some opinionated changes on what our team at Liatrio has seen work well across different enterprises.

• Ephemeral Only
  • Runners should only run one job to avoid interference from one workflow run to the next
• Warm Pool by Default
  • Keeping idle runners on is a must to ensure quick feedback loops
• Custom Images
  • Images should be able to build most apps in organization without additional tool installation (example) ← TODO
  • Including necessary tools in VM Image to reduce startup time for most builds
• Security
  • Runner VMs are granted a single use registration token with no additional access to GitHub
  • Application can only interact with resources created in the Resource Group managed by this module

This Terraform module generates the infrastructure required to host the applications that will manage the self-hosted runners.

This application will serve as a filter for events being published from GitHub that are not intended to use these self-hosted runners and will process valid events to a Service Bus queue.

This application will act as the controller for the warm pool and ensure that the pool size adheres to the parameters specified in the Terraform module. It will consume events from the queue as necessary to create VMs and ensure a healthy number of VMs are always ready to process new workflow jobs.

This Packer template automates the creation of a shared image that is referenced by the Terraform module for use in Runner VM creation. Another image can be used if desired.

• GitHub App for Organization (owner access)
• Azure
  • Subscription
    • Note: Subscription quota for "Total Regional Low-priority vCPUs" should be increased to allow multiple spot instances
  • Resource Group
  • Subnet with internet access
  • KeyVault for GitHub App Credential
  • optional - Managed Image Id accessible by App B

Referencing the Packer Template repo, create an image and publish it to Azure Compute Gallery.

The GitHub App serves as the foundation for sending webhook events to App A and retrieving registration tokens to store in Azure Key Vault.

1. Navigate: Settings → Developer Settings → GitHub Apps → New GitHub App
2. Configure permissions
3. Configure settings, webhook settings will need to be changed later
4. Save App and take note of App ID, Client ID

*Note: You will need one GitHub App per org and allowing installation to "Any account" makes it near impossible to change access if installed on orgs outside your control.

Should this be deployed from local first or action first? 🐓 🐣 🥚 problem