Parallel frontend compiler support (perf backend only) by heinwol · Pull Request #2491 · rust-lang/rustc-perf

heinwol · 2026-07-01T12:05:25Z

Supersedes #2421, but only backend (profiler/bencher/database). As said in parent pr, parallel frontend support is implemented as additional dimension with corresponding row added to the database

from andrew.zhogin@gmail.com: 3ffe80f

as proposed by berykubik@gmail.com in the discussion of the preceding pr

Kobzol · 2026-07-01T12:48:01Z

Thanks for the PR! Let's revert the last commit; the CI failure shows why we need to have a custom environment variable, so that it will also work with a stable Rust compiler.

heinwol · 2026-07-01T14:45:46Z

I'm not sure I quite like a separate environment variable for that; don't we have a better mechanism? Also, maybe a collector run shouldn't silently revert to single-threaded scenario in an unsupported version, which it currently does, for my understanding? However, it seems like it's a common behavior in collector/src/lib.rs::async fn bench_published_artifact:

// ...
    let profiles = if collector::version_supports_doc(&toolchain.id) {
        vec![Profile::Check, Profile::Debug, Profile::Doc, Profile::Opt]
    } else {
        vec![Profile::Check, Profile::Debug, Profile::Opt]
    };
    let scenarios = if collector::version_supports_incremental(&toolchain.id) {
        Scenario::all()
    } else {
        Scenario::all_non_incr()
    };
    let frontend_threads_counts = if collector::version_supports_parallel_frontend(&toolchain.id) {
        FrontendThreads::default_opts()
    } else {
        vec![FrontendThreads(1)]
    };
// ...

So it might be i'm looking too hard into it...

Kobzol · 2026-07-02T07:28:46Z

bench_published_artifact is only used for benchmarking stable versions of the compiler, which is ~never done locally, and we need to keep compatibility with Rust going all the way back to 1.26.0, so I wouldn't worry about that.

Kobzol

Thanks for separating the PR! Did a first review pass. Adding a new benchmark axis is always cumbersome, as it requires changing many places, and this one is also tricky, because it doesn't have a closed set of values, but is just a number. But in general it looks good, just

Please revert all the site changes, the integration with the frontend and also the job queuing system should be done in a follow-up PR. For now everything on production will just assume frontend count 1.

Kobzol · 2026-07-02T07:32:16Z

+                            benchmark.name,
+                            profile,
+                            scenario,
+                            frontend_threads.par_n(),


Let's remove the par_n function, it contains a hardcoded string prefix that is then used in several places. That shouldn't belong in the database module.

Kobzol · 2026-07-02T07:33:23Z

+    #[arg(
+        long = "frontend-threads",
+        value_delimiter = ',',
+        default_value = "1,4"


Suggested change

default_value = "1,4"

default_value = "1"

The default should just be one for now, we don't need to test multiple values by default, it would just make local runs unnecessarily slower.

Kobzol · 2026-07-02T07:34:17Z

            },
            bench_rustc,
            targets: vec![job.target().into()],
+            frontend_threads: FrontendThreads::default_opts(),


Suggested change

frontend_threads: FrontendThreads::default_opts(),

frontend_threads: FrontendThreads::default_value(),

And also change the default to just [1].

Kobzol · 2026-07-02T07:37:58Z

+    let frontend_threads_counts = if collector::version_supports_parallel_frontend(&toolchain.id) {
+        FrontendThreads::default_opts()
+    } else {
+        vec![FrontendThreads(1)]


Hmm, this is tricky, because for the old version that doesn't support the flag, we would either:

Not pass the RUSTC_THREAD_COUNT at all (which happens now). That works for the old compiler, but if we change the default thread count in the future, then setting --frontend-threads=1 will actually use the default number of threads, which is misleading.

Still try to pass -Zthreads=1. That will ensure that we are using the right number of threads, but it will break benchmarking on Rust <1.33.0.

I would suggest changing this to always pass the flag. It is better to break benchmarking on Rust <1.33.0, which likely won't be needed in the near future, than to use an unexpected thread count by mistake.

Kobzol · 2026-07-02T07:40:08Z

+            if self.frontend_threads.0 != 1 {
+                cmd.env("RUSTC_THREAD_COUNT", self.frontend_threads.0.to_string());
+            }


Suggested change

if self.frontend_threads.0 != 1 {

cmd.env("RUSTC_THREAD_COUNT", self.frontend_threads.0.to_string());

}

cmd.env("RUSTC_THREAD_COUNT", self.frontend_threads.0.to_string());

Kobzol · 2026-07-02T07:48:59Z

            .raw()
            .prepare(
-                "select id, crate, profile, scenario, backend, target, metric from pstat_series;",
+                "select id, crate, profile, scenario, backend, target, metric, frontend_threads from pstat_series;",


Suggested change

"select id, crate, profile, scenario, backend, target, metric, frontend_threads from pstat_series;",

"select id, crate, profile, scenario, backend, target, frontend_threads, metric from pstat_series;",

And everywhere else.

Kobzol · 2026-07-02T07:49:18Z

+        Metric,
+        FrontendThreads,


Suggested change

Metric,

FrontendThreads,

FrontendThreads,

Metric,

Kobzol · 2026-07-02T07:50:11Z

    metric: Selector<crate::Metric>,
+    frontend_threads: Selector<FrontendThreads>,
    target: Selector<Target>,


Suggested change

metric: Selector<crate::Metric>,

frontend_threads: Selector<FrontendThreads>,

target: Selector<Target>,

target: Selector<Target>,

frontend_threads: Selector<FrontendThreads>,

metric: Selector<crate::Metric>,

Kobzol · 2026-07-02T07:50:41Z

+  │ metric            │  └──────────┘
+  │ target            |
+  │ frontend_threads  |


Suggested change

│ metric │ └──────────┘

│ target |

│ frontend_threads |

│ frontend_threads │ └──────────┘

│ target |

│ metric |

Kobzol · 2026-07-02T07:51:27Z

 * **metric** (`text`): the type of metric being collected.
+* **frontend_threads** (`text`): Parallel frontend thread count ('-Zthreads=N').


Suggested change

* **metric** (`text`): the type of metric being collected.

* **frontend_threads** (`text`): Parallel frontend thread count ('-Zthreads=N').

* **frontend_threads** (`text`): Parallel frontend thread count ('-Zthreads=N').

* **metric** (`text`): the type of metric being collected.

Seems like we are missing the target here (pre-existing), could you please add it? Thanks!

heinwol · 2026-07-02T08:50:27Z

Thanks for the fast yet sizable review! I'm sorry for calling out for you so early; i'm not sure this pr is quite review-ready. I'll try to resolve what you commented for now and then dive more into how this works! As until now all i did is brainless-ish renaming, lol

Kobzol · 2026-07-02T08:52:33Z

To be fair, adding a new benchmark axis mostly involves a lot of brainless-ish renaming :) I think that the PR is mostly good to go, though it will require a couple of follow-up PRs to:

Add integration in the website backend and frontend
Modify the job queue to be able to distinguish that different jobs can have different frontend thread counts

azhogin and others added 2 commits June 29, 2026 16:52

cherry-pick non-frontend files

4a524e5

from andrew.zhogin@gmail.com: 3ffe80f

Rename parallel -> frontend_threads

4f11ea4

as proposed by berykubik@gmail.com in the discussion of the preceding pr

heinwol force-pushed the par-front-scenario branch from 544995b to 4f11ea4 Compare July 1, 2026 14:37

Kobzol reviewed Jul 2, 2026

View reviewed changes

	frontend_threads: FrontendThreads::default_opts(),
	frontend_threads: FrontendThreads::default_value(),

	"select id, crate, profile, scenario, backend, target, metric, frontend_threads from pstat_series;",
	"select id, crate, profile, scenario, backend, target, frontend_threads, metric from pstat_series;",

		* metric (`text`): the type of metric being collected.
		* frontend_threads (`text`): Parallel frontend thread count ('-Zthreads=N').

Uh oh!

Conversation

heinwol commented Jul 1, 2026

Uh oh!

Kobzol commented Jul 1, 2026

Uh oh!

heinwol commented Jul 1, 2026

Uh oh!

Kobzol commented Jul 2, 2026

Uh oh!

Kobzol left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

heinwol commented Jul 2, 2026

Uh oh!

Kobzol commented Jul 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants