Skip to content

ci: use aws codebuild for the dist-x86_64-linux job #140349

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 28, 2025
Merged

ci: use aws codebuild for the dist-x86_64-linux job #140349

merged 1 commit into from
Apr 28, 2025

Conversation

marcoieni
Copy link
Member

@marcoieni marcoieni commented Apr 26, 2025 •
edited by Kobzol
Loading

r? @ghost

We want to use aws credits for linux runners.

try-job: dist-x86_64-linux

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Apr 26, 2025
@marcoieni
Copy link
Member Author

@bors try

bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 26, 2025
@bors
Auto merge of rust-lang#140349 - marcoieni:codebuild-linux-large-runn…
1c29b93 
…ers, r=<try>

ci: use aws codebuild for x86 linux large runners

try-job: dist-x86_64-linux
try-job: dist-x86_64-linux-alt
try-job: x86_64-gnu-distcheck
@bors
Copy link
Collaborator

bors commented Apr 26, 2025

⌛ Trying commit c2922b4 with merge 1c29b93...

@bors
Copy link
Collaborator

bors commented Apr 26, 2025

💔 Test failed - checks-actions

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 26, 2025
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@marcoieni marcoieni force-pushed the codebuild-linux-large-runners branch 3 times, most recently from 48990e2 to ed20b7f Compare April 27, 2025 09:11
@marcoieni
Copy link
Member Author

@bors try

@bors
Copy link
Collaborator

bors commented Apr 27, 2025

⌛ Trying commit ed20b7f with merge 51349b0...

bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 27, 2025
@bors
Auto merge of rust-lang#140349 - marcoieni:codebuild-linux-large-runn…
51349b0 
…ers, r=<try>

ci: use aws codebuild for x86 linux large runners

try-job: dist-x86_64-linux
try-job: dist-x86_64-linux-alt
@bors
Copy link
Collaborator

bors commented Apr 27, 2025

☀️ Try build successful - checks-actions
Build commit: 51349b0 (51349b0d6ae41043c5e7b4925d19f587cdeb7c72)

@Kobzol
Copy link
Contributor

Kobzol commented Apr 27, 2025

@bors try

bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 27, 2025
@bors
Auto merge of rust-lang#140349 - marcoieni:codebuild-linux-large-runn…
0148c0b 
…ers, r=<try>

ci: use aws codebuild for x86 linux large runners

try-job: dist-x86_64-linux
try-job: dist-x86_64-linux-alt
@bors
Copy link
Collaborator

bors commented Apr 27, 2025

⌛ Trying commit ed20b7f with merge 0148c0b...

@bors
Copy link
Collaborator

bors commented Apr 27, 2025

☀️ Try build successful - checks-actions
Build commit: 0148c0b (0148c0b5e1aa4da2470d93e88991a3cb8e2148a1)

@marcoieni
Copy link
Member Author

@bors try

@bors
Copy link
Collaborator

bors commented Apr 27, 2025

⌛ Trying commit ed20b7f with merge 59ef764...

bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 27, 2025
@bors
Auto merge of rust-lang#140349 - marcoieni:codebuild-linux-large-runn…
59ef764 
…ers, r=<try>

ci: use aws codebuild for x86 linux large runners

try-job: dist-x86_64-linux
try-job: dist-x86_64-linux-alt
@marcoieni
Copy link
Member Author

marcoieni commented Apr 27, 2025 •
edited
Loading

oh I didn't see you already retried Jakub. It's quite weird that with 36 cores the jobs take so long. Maybe GitHub runners have faster cores? Or maybe some caching is not working. We need to analyze the logs.
image

@bors
Copy link
Collaborator

bors commented Apr 27, 2025

☀️ Try build successful - checks-actions
Build commit: 59ef764 (59ef764abee62044f1dc9818d9ccc95f65d962ef)

@marcoieni
Copy link
Member Author

marcoieni commented Apr 28, 2025 •
edited
Loading

I compared the logs for the dist-x86_64-linux-alt job (try build vs auto build in rust-lang-ci). Building LLVM takes 2x in the try build. Not sure why.

EDIT: I was comparing things in the wrong way. Apparently compiling llvm in the try job is faster. I need to check where the try job is slower.

@marcoieni marcoieni force-pushed the codebuild-linux-large-runners branch from ed20b7f to 7bc2b01 Compare April 28, 2025 06:46
@marcoieni marcoieni changed the title ci: use aws codebuild for x86 linux large runners ci: use aws codebuild for the dist-x86_64-linux job Apr 28, 2025
@marcoieni marcoieni force-pushed the codebuild-linux-large-runners branch from 7bc2b01 to 480d007 Compare April 28, 2025 06:48
@marcoieni
Copy link
Member Author

marcoieni commented Apr 28, 2025 •
edited
Loading

I don't know why the alt job is that slow. I edited this PR to only convert the dist-x86_64-linux job for now.

Imo this PR can be a rollup because we tried this job multiple times.

@marcoieni marcoieni marked this pull request as ready for review April 28, 2025 06:48
@marcoieni
Copy link
Member Author

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 28, 2025
@marcoieni marcoieni requested a review from Kobzol April 28, 2025 06:50
@Kobzol
Copy link
Contributor

Kobzol commented Apr 28, 2025

Hmm, it's weird. Most of the bootstrap steps were faster, some of them significantly, but the run as a whole was ~15 minutes slower. sccache stats also look good in both. I tried to go through both logs in parallel, but haven't found anything useful, it would be great to have some visualization of the log timeline. It's like the slowdown happened in between bootstrap steps or something.

But otherwise it looks fine.

@bors r+

@bors
Copy link
Collaborator

bors commented Apr 28, 2025

📌 Commit 480d007 has been approved by Kobzol

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 28, 2025
@GuillaumeGomez GuillaumeGomez mentioned this pull request Apr 28, 2025
Merged
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 28, 2025
@bors
Auto merge of rust-lang#140388 - GuillaumeGomez:rollup-aj9o3ch, r=Gui…
6655f5c 
…llaumeGomez

Rollup of 7 pull requests

Successful merges:

 - rust-lang#140056 (Fix a wrong error message in 2024 edition)
 - rust-lang#140220 (Fix detection of main function if there are expressions around it)
 - rust-lang#140249 (Remove `weak` alias terminology)
 - rust-lang#140316 (Introduce `BoxMarker` to improve pretty-printing correctness)
 - rust-lang#140347 (ci: clean more disk space in codebuild)
 - rust-lang#140349 (ci: use aws codebuild for the `dist-x86_64-linux` job)
 - rust-lang#140379 (rustc-dev-guide subtree update)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 28, 2025
@bors
Auto merge of rust-lang#140388 - GuillaumeGomez:rollup-aj9o3ch, r=Gui…
25cdf1f 
…llaumeGomez

Rollup of 7 pull requests

Successful merges:

 - rust-lang#140056 (Fix a wrong error message in 2024 edition)
 - rust-lang#140220 (Fix detection of main function if there are expressions around it)
 - rust-lang#140249 (Remove `weak` alias terminology)
 - rust-lang#140316 (Introduce `BoxMarker` to improve pretty-printing correctness)
 - rust-lang#140347 (ci: clean more disk space in codebuild)
 - rust-lang#140349 (ci: use aws codebuild for the `dist-x86_64-linux` job)
 - rust-lang#140379 (rustc-dev-guide subtree update)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 117202e into rust-lang:master Apr 28, 2025
6 checks passed
@rustbot rustbot added this to the 1.88.0 milestone Apr 28, 2025
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Apr 28, 2025
@rust-timer
Unrolled build for rust-lang#140349
58cb389 
Rollup merge of rust-lang#140349 - marcoieni:codebuild-linux-large-runners, r=Kobzol

ci: use aws codebuild for the `dist-x86_64-linux` job

try-job: dist-x86_64-linux
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Kobzol Kobzol Awaiting requested review from Kobzol

Assignees
No one assigned
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
1.88.0
Development

Successfully merging this pull request may close these issues.
5 participants
@marcoieni @bors @rust-log-analyzer @Kobzol @rustbot