ci(tests): Move hardware tests to GitLab #11890

lucasssvaz · 2025-10-01T20:53:27Z

Description of Change

Reworked test workflows to be able to move hardware tests to internal GitLab.

Test Scenarios

Tested in fork + internal GitLab

Related links

Test results:
https://github.com/lucasssvaz/arduino-esp32/blob/gh-pages/runtime-test-results/RUNTIME_TEST_RESULTS.md

github-actions · 2025-10-01T20:54:06Z

	Messages
📖	This PR seems to be quite large (total lines of code: 1209), you might consider splitting it into smaller PRs

👋 Hello lucasssvaz, we appreciate your contribution to this project!

📘 Please review the project's Contributions Guide for key guidelines on code, documentation, testing, and more.

🖊️ Please also make sure you have read and signed the Contributor License Agreement for this project.

Click to see more instructions ...

This automated output is generated by the PR linter DangerJS, which checks if your Pull Request meets the project's requirements and helps you fix potential issues.

DangerJS is triggered with each push event to a Pull Request and modify the contents of this comment.

Please consider the following:
- Danger mainly focuses on the PR structure and formatting and can't understand the meaning behind your code or changes.
- Danger is not a substitute for human code reviews; it's still important to request a code review from your colleagues.
- Addressing info messages (📖) is strongly recommended; they're less critical but valuable.
- To manually retry these Danger checks, please navigate to the Actions tab and re-run last Danger workflow.

Review and merge process you can expect ...

We do welcome contributions in the form of bug reports, feature requests and pull requests.

1. An internal issue has been created for the PR, we assign it to the relevant engineer.
2. They review the PR and either approve it or ask you for changes or clarifications.
3. Once the GitHub PR is approved we do the final review, collect approvals from core owners and make sure all the automated tests are passing.
- At this point we may do some adjustments to the proposed change, or extend it by adding tests or documentation.
4. If the change is approved and passes the tests it is merged into the default branch.

Generated by 🚫 dangerJS against 318070e

.github/workflows/tests_hw_wokwi.yml

+          echo "Ensuring GitLab sync has completed before triggering pipeline..."
+
+          # Use push time determined in get-artifacts job
+          push_time="${{ needs.get-artifacts.outputs.push_time }}"


To resolve the potential code injection, we should set the push_time value as an environment variable in the workflow step, using the workflow expression syntax. Then, within the shell script, reference $push_time (or $PUSH_TIME) using shell variable expansion, not expression interpolation. This ensures the shell receives the value directly, with no further parsing or evaluation of expression content by the shell.

Specifically, in the step at line 262 (run:), instead of:

push_time="${{ needs.get-artifacts.outputs.push_time }}"

we should:

Add an env: block to this step, setting PUSH_TIME: ${{ needs.get-artifacts.outputs.push_time }}.

In the shell script, replace occurrences of ${{ needs.get-artifacts.outputs.push_time }} or push_time="..." with push_time="$PUSH_TIME" or just use $PUSH_TIME directly.

No other behavior will be changed, but this ensures there is no workflow expression interpolation inside the shell script.

.github/workflows/tests_hw_wokwi.yml

+
+      - name: Trigger GitLab Pipeline and Download Artifacts
+        if: ${{ steps.check-tests.outputs.enabled == 'true' }}
+        uses: digital-blueprint/[email protected]


.github/workflows/tests_results.yml

        run: |
-          echo "Build, Hardware and QEMU tests: https://github.com/${{ github.repository }}/actions/runs/${{ env.original_run_id }}"
-          echo "Wokwi tests: https://github.com/${{ github.repository }}/actions/runs/${{ github.event.workflow_run.id }}"
+          echo "Build and QEMU tests: https://github.com/${{ github.repository }}/actions/runs/${{ steps.get-info.outputs.original_run_id }}"


To address the vulnerability, we should ensure that the value assigned to original_run_id comes only from trusted GitHub metadata (e.g., from ${{ github.event.workflow_run.id }}), and not from artifacts which can be influenced by untrusted jobs. The fix consists of:

In the get-artifacts job, remove the extraction of run_id.txt from ./artifacts/parent-artifacts/run_id.txt and instead explicitly set original_run_id using the trusted GitHub-provided context: ${{ github.event.workflow_run.id }}.

Adjust all lines depending on original_run_id in get-info step accordingly.

Update the output assignment to only use this trusted value.

This way, any code injection risk from artifact contents is eliminated, and only official workflow run IDs are used for subsequent privileged steps.

.github/workflows/tests_results.yml

        run: |
-          echo "Build, Hardware and QEMU tests: https://github.com/${{ github.repository }}/actions/runs/${{ env.original_run_id }}"
-          echo "Wokwi tests: https://github.com/${{ github.repository }}/actions/runs/${{ github.event.workflow_run.id }}"
+          echo "Build and QEMU tests: https://github.com/${{ github.repository }}/actions/runs/${{ steps.get-info.outputs.original_run_id }}"


To fix the issue, we should avoid direct interpolation of potentially untrusted output variables via ${{ ... }} in the shell command under run:. Instead, we should pass the potentially untrusted value as an environment variable, and then reference it in the shell script using native shell syntax (e.g., $ORIGINAL_RUN_ID). Specifically, in the "Print links to other runs" step, we will:

Add an env: block, passing ORIGINAL_RUN_ID: ${{ steps.get-info.outputs.original_run_id }}.

Change the script in the step to use $ORIGINAL_RUN_ID (not expression interpolation) in the echo statement.

Ensure no functional changes, just make the value's usage safe.

No new imports, methods, or definitions are required beyond this.

.github/workflows/tests_results.yml

        with:
          script: |
-            const ref = process.env.original_ref;
+            const ref = '${{ needs.get-artifacts.outputs.original_ref }}';


General fix:
To prevent code injection, values originating from untrusted sources (including artifact files) must not be injected via GitHub Actions expression syntax ("${{...}}") inside scripts. Instead, pass such values as environment variables via the env block, and within the script, access them using the shell or (process.env for JavaScript).

Detailed fix:

In the actions/github-script step that currently defines const ref = '${{ needs.get-artifacts.outputs.original_ref }}';, move the interpolation of ${{ needs.get-artifacts.outputs.original_ref }} into the environment (env) block as a variable, e.g., REF: ${{ needs.get-artifacts.outputs.original_ref }}.

In the script, reference this as const ref = process.env.REF;. Remove the direct interpolation via ${{...}} in the JavaScript source.

No changes to business logic or functional side effects.

Files/lines to change:

File: .github/workflows/tests_results.yml

In the actions/github-script step under unit-test-results, add a new env: block (or add to it if already present), passing REF: ${{ needs.get-artifacts.outputs.original_ref }}.

Edit the script: to set const ref = process.env.REF;, replacing the existing assignment.

Imports/definitions needed:

No new imports are needed. Only use standard JavaScript in the script field.

.github/workflows/tests_results.yml

            const key_prefix = 'test-' + ref + '-';

-            if (process.env.original_event == 'pull_request' && process.env.original_action != 'closed') {
+            if ('${{ needs.get-artifacts.outputs.original_event }}' == 'pull_request' && '${{ needs.get-artifacts.outputs.original_action }}' != 'closed') {


To fix this problem, update the workflow so that original_event, original_action, and original_ref (and any other required output values) are passed to the github-script step via environment variables, not via direct expression interpolation in JavaScript code.
That is:

Replace all usages like ${{ needs.get-artifacts.outputs.original_event }} in the script body with process.env.ORIGINAL_EVENT.

Set respective env: properties for the github-script step.

Update all JavaScript logic inside the script to use the environment variables.

No change to behavior; merely changing how values reach the script.

In .github/workflows/tests_results.yml, update the step "Clean up caches" (lines 140-170) to:

Add env: for environment variables (ORIGINAL_EVENT, ORIGINAL_ACTION, ORIGINAL_REF).

In the script, replace all references like '${{ needs.get-artifacts.outputs.original_event }}' with process.env.ORIGINAL_EVENT, and similarly for the others.

No packages to install; all changes are within the workflow YAML.

.github/workflows/tests_results.yml

            const key_prefix = 'test-' + ref + '-';

-            if (process.env.original_event == 'pull_request' && process.env.original_action != 'closed') {
+            if ('${{ needs.get-artifacts.outputs.original_event }}' == 'pull_request' && '${{ needs.get-artifacts.outputs.original_action }}' != 'closed') {


To fix the issue, we should never interpolate untrusted variables directly into the script: section of a github-script step via ${{ ... }} syntax. Instead, we should set the untrusted value as an environment variable in the workflow step (env: block), and then read it using process.env in the script—this avoids interpolation and instead relies on out-of-band environment setup, which mitigates code injection risks.

Specifically:

In the unit-test-results job, in the Clean up caches step (lines 139–170), set all required "original" variables as environment variables in the env: section.

In the script: block, replace direct expressions like ${{ needs.get-artifacts.outputs.original_action }} etc. with their safe environment equivalents (e.g., process.env.ORIGINAL_ACTION).

This change is within the shown workflow step, so no other parts of the file need editing.

.github/workflows/tests_results.yml

            const owner = '${{ github.repository_owner }}';
            const repo = '${{ github.repository }}'.split('/')[1];
-            const sha = process.env.original_sha;
+            const sha = '${{ needs.get-artifacts.outputs.original_sha }}';


General fix:
Do not pass user-controlled or potentially untrusted data directly into workflow expressions used in run: or script: contexts. Instead, use environment variables to pass such data, and reference them using the underlying shell or language's environment variable access method.

Detailed steps:

In the Report conclusion step (lines 171-191), change the reference to original_sha from using the dangerous ${{ needs.get-artifacts.outputs.original_sha }} directly in the script to injecting it as an environment variable.

Add an env block to the step, setting SHA: ${{ needs.get-artifacts.outputs.original_sha }}.

In the script: block, replace const sha = '${{ needs.get-artifacts.outputs.original_sha }}'; with const sha = process.env.SHA;

(Optional but recommended) Remove any other interpolated outputs in the script in favour of the same approach if they're user-controlled.

Lines/regions to change:

In .github/workflows/tests_results.yml, edit the step beginning with - name: Report conclusion at lines 171-191.

Requirements:

No new imports needed, as process.env is natively available in JavaScript.

Scripting syntax must be updated.

.github/workflows/tests_results.yml

            core.debug(`sha: ${sha}`);
            const { context: name, state } = (await github.rest.repos.createCommitStatus({
-              context: `Runtime Tests / Report results (${process.env.original_event} -> workflow_run -> workflow_run)`,
+              context: `Runtime Tests / Report results (${{ needs.get-artifacts.outputs.original_event }} -> workflow_run -> workflow_run)`,


To fix this problem, replace direct ${{ ... }} interpolation of untrusted inputs (specifically, the value of ${{ needs.get-artifacts.outputs.original_event }} on line 182) with usage of environment variables. Pass the sanitized workflow output as an environment variable to the step, and then reference that environment variable inside the script via process.env (JavaScript) rather than via expression interpolation. Specifically:

In the "Report conclusion" step (lines 170-191), add an env: block that sets ORIGINAL_EVENT: ${{ needs.get-artifacts.outputs.original_event }}.

In the script: block, reference this value as process.env.ORIGINAL_EVENT rather than using ${{ ... }}.

Refactor line 182 to use this environment variable when creating the context string.

Make similar adjustments for other workflow outputs if necessary and if they are user-controlled, but focus on ORIGINAL_EVENT as flagged by the alert.

No new dependencies or changes outside this region are required.

.gitlab/scripts/gen_hw_jobs.py

+# Ensure we run from repo root so relative paths work consistently
+try:
+    os.chdir(REPO_ROOT)
+except Exception:


.gitlab/scripts/gen_hw_jobs.py

+            try:
+                fixed = txt.replace("'", '"')
+                return [str(x).strip() for x in json.loads(fixed)]
+            except Exception:


github-actions · 2025-10-01T21:04:07Z

Test Results

76 files 76 suites 14m 50s ⏱️
38 tests 38 ✅ 0 💤 0 ❌
241 runs 241 ✅ 0 💤 0 ❌

Results for commit 318070e.

ci(tests): Move hardware tests to GitLab

318070e

lucasssvaz requested a review from me-no-dev October 1, 2025 20:53

lucasssvaz self-assigned this Oct 1, 2025

lucasssvaz requested a review from P-R-O-C-H-Y as a code owner October 1, 2025 20:53

lucasssvaz added the Type: CI & Testing Related to continuous integration, automated testing, or test infrastructure. label Oct 1, 2025

github-advanced-security bot found potential problems Oct 1, 2025

View reviewed changes

@@ -39,7 +39,8 @@
                       original_sha=$(cat ./artifacts/parent-artifacts/sha.txt)
                       original_ref=$(cat ./artifacts/parent-artifacts/ref.txt)
                       original_conclusion=$(cat ./artifacts/parent-artifacts/conclusion.txt)
-                      original_run_id=$(cat ./artifacts/parent-artifacts/run_id.txt)
+                      # Use trusted run ID from GitHub context
+                      original_run_id="${{ github.event.workflow_run.id }}"
                       # Sanitize the values to avoid security issues
@@ -58,8 +59,7 @@
                       # Conclusion: Allow alphabetical characters and underscores
                       original_conclusion=$(echo "$original_conclusion" | tr -cd '[:alpha:]_')
-                      # Run ID: Allow numeric characters
-                      original_run_id=$(echo "$original_run_id" | tr -cd '[:digit:]')
+                      # Run ID: Already trusted by assignment, no need to sanitize
                       echo "original_event=$original_event" >> $GITHUB_OUTPUT
                       echo "original_action=$original_action" >> $GITHUB_OUTPUT

@@ -76,8 +76,10 @@
                       echo "original_run_id = $original_run_id"
                   - name: Print links to other runs
+                    env:
+                      ORIGINAL_RUN_ID: ${{ steps.get-info.outputs.original_run_id }}
                     run: |
-                      echo "Build and QEMU tests: https://github.com/${{ github.repository }}/actions/runs/${{ steps.get-info.outputs.original_run_id }}"
+                      echo "Build and QEMU tests: https://github.com/${{ github.repository }}/actions/runs/$ORIGINAL_RUN_ID"
                       echo "Hardware and Wokwi tests: https://github.com/${{ github.repository }}/actions/runs/${{ github.event.workflow_run.id }}"
               unit-test-results:

@@ -139,12 +139,16 @@
                   - name: Clean up caches
                     if: always()
                     uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
+                    env:
+                      ORIGINAL_EVENT: ${{ needs.get-artifacts.outputs.original_event }}
+                      ORIGINAL_ACTION: ${{ needs.get-artifacts.outputs.original_action }}
+                      ORIGINAL_REF: ${{ needs.get-artifacts.outputs.original_ref }}
                     with:
                       script: |
-                        const ref = '${{ needs.get-artifacts.outputs.original_ref }}';
+                        const ref = process.env.ORIGINAL_REF;
                         const key_prefix = 'test-' + ref + '-';
-                        if ('${{ needs.get-artifacts.outputs.original_event }}' == 'pull_request' && '${{ needs.get-artifacts.outputs.original_action }}' != 'closed') {
+                        if (process.env.ORIGINAL_EVENT === 'pull_request' && process.env.ORIGINAL_ACTION !== 'closed') {
                           console.log('Skipping cache cleanup for open PR');
                           return;
                         }

@@ -139,12 +139,16 @@
                   - name: Clean up caches
                     if: always()
                     uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
+                    env:
+                      ORIGINAL_REF: ${{ needs.get-artifacts.outputs.original_ref }}
+                      ORIGINAL_EVENT: ${{ needs.get-artifacts.outputs.original_event }}
+                      ORIGINAL_ACTION: ${{ needs.get-artifacts.outputs.original_action }}
                     with:
                       script: |
-                        const ref = '${{ needs.get-artifacts.outputs.original_ref }}';
+                        const ref = process.env.ORIGINAL_REF;
                         const key_prefix = 'test-' + ref + '-';
-                        if ('${{ needs.get-artifacts.outputs.original_event }}' == 'pull_request' && '${{ needs.get-artifacts.outputs.original_action }}' != 'closed') {
+                        if (process.env.ORIGINAL_EVENT == 'pull_request' && process.env.ORIGINAL_ACTION != 'closed') {
                           console.log('Skipping cache cleanup for open PR');
                           return;
                         }

@@ -170,6 +170,8 @@
                   - name: Report conclusion
                     uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
                     if: always()
+                    env:
+                      ORIGINAL_EVENT: ${{ needs.get-artifacts.outputs.original_event }}
                     with:
                       script: |
                         const owner = '${{ github.repository_owner }}';
@@ -178,8 +180,9 @@
                         core.debug(`owner: ${owner}`);
                         core.debug(`repo: ${repo}`);
                         core.debug(`sha: ${sha}`);
+                        const event = process.env.ORIGINAL_EVENT;
                         const { context: name, state } = (await github.rest.repos.createCommitStatus({
-                          context: `Runtime Tests / Report results (${{ needs.get-artifacts.outputs.original_event }} -> workflow_run -> workflow_run)`,
+                          context: `Runtime Tests / Report results (${event} -> workflow_run -> workflow_run)`,
                           owner: owner,
                           repo: repo,
                           sha: sha,

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ci(tests): Move hardware tests to GitLab #11890

ci(tests): Move hardware tests to GitLab #11890

lucasssvaz commented Oct 1, 2025

Uh oh!

github-actions bot commented Oct 1, 2025

Uh oh!

Check failure

Copilot Autofix

Check warning

Check failure

Copilot Autofix

Check warning

Copilot Autofix

Check failure

Copilot Autofix

Check failure

Copilot Autofix

Check failure

Copilot Autofix

Check failure

Copilot Autofix

Check failure

Copilot Autofix

Check notice

Check notice

github-actions bot commented Oct 1, 2025

Uh oh!

Uh oh!

@@ -259,6 +259,8 @@
                   - name: Wait for GitLab sync
                     if: ${{ steps.check-tests.outputs.enabled == 'true' }}
+                    env:
+                      PUSH_TIME: ${{ needs.get-artifacts.outputs.push_time }}
                     run: |
                       # A webhook to sync the repository is sent to GitLab when a commit is pushed to GitHub
                       # We wait for 10 minutes after the push to GitHub to be safe
@@ -266,7 +268,7 @@
                       echo "Ensuring GitLab sync has completed before triggering pipeline..."
                       # Use push time determined in get-artifacts job
-                      push_time="${{ needs.get-artifacts.outputs.push_time }}"
+                      push_time="$PUSH_TIME"
                       if [ -n "$push_time" ]; then
                         echo "Push time: $push_time"

ci(tests): Move hardware tests to GitLab #11890

Are you sure you want to change the base?

ci(tests): Move hardware tests to GitLab #11890

Conversation

lucasssvaz commented Oct 1, 2025

Description of Change

Test Scenarios

Related links

Uh oh!

github-actions bot commented Oct 1, 2025

Uh oh!

Check failure

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot Autofix

Check warning

Uh oh!

Check failure

Uh oh!

Uh oh!

Copilot Autofix

Check warning

Uh oh!

Copilot Autofix

Check failure

Uh oh!

Uh oh!

Copilot Autofix

Check failure

Uh oh!

Uh oh!

Copilot Autofix

Check failure

Uh oh!

Uh oh!

Copilot Autofix

Check failure

Uh oh!

Uh oh!

Copilot Autofix

Check failure

Uh oh!

Uh oh!

Copilot Autofix

Check notice

Check notice

github-actions bot commented Oct 1, 2025

Test Results

Uh oh!

Uh oh!