Maintenance: Require integration tests before PR merge

[dreamorosi]

Summary

At the moment integration (e2e) tests are run manually by maintainers. Our standard operating procedure requires to run these tests before merging a PR and before making a release, however human errors happen and maintainers can forget to.

This issue serves to track the efforts needed to make sure that these tests are part of the requirements needed to merge a PR.

Maintainers should be able to bypass this check based on context (i.e. changes were made only to documentation).

Why is this needed?

In order to reduce the chances of human error. Automating this would enforce the requirement of running these tests.

Which area does this relate to?

Automation, Tests

Solution

• Investigate branch status checks
• Investigate how they can be enabled on PRs so that a specific workflow has to be successful before merge
• If required, investigate how the run-e2e-tests workflow can report its status into a PR under these checks

Acknowledgment

• This request meets Lambda Powertools Tenets
• Should this be considered in other Lambda Powertools languages? i.e. Python, Java

[github-actions]

This issue has not received a response in 2 weeks. If you still think there is a problem, please leave a comment to avoid the issue from automatically closing.

[am29d]

I have looked into that and I see two options: 1/ run e2e test workflow when PR is approved, 2/ run e2e test workflow when specific label was added

For 1 this could look like that:

on:
  pull_request_review:
    types: [submitted]

jobs:
  approved:
    if: github.event.review.state == 'APPROVED'
    runs-on: ubuntu-latest
    steps:
      - run: echo "This PR was approved"

For 2:

name: CI

on:
  pull_request:
    types: [ labeled ]

jobs:
  build:
    if: ${{ github.event.label.name == 'e2e-test' }}
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: Run a one-line script
      run: echo Hello, world!

Sadly, both approaches only execute the workflow once, so in cases when we need to rerun e2e tests, we need to remove label and add again or dismiss approval. Both will flood the status history of the PR.