Add -Ycollect-statistics for collecting statistics without printing them
#10795
Conversation
|
Maybe we can change |
|
I thought about that but it's not obvious what the semantics should be or how the use of this feature by a build tool would compose with an explicit |
|
That makes sense, thanks. Looks good to me, @som-snytt maybe you have some more thoughts? |
|
Maybe ideally
At this late date, whatever keeps the ecosystem on life support.
All statistics are cold. Oh, I suppose that when they are represented in a visual form that roils the emotions, they become hot. Also, "damn lies, and statistics". |
This change adds a new flag `-Ycollect-statistics` that enables the same statistics gathering as `-Ystatistics` but without dumping all the statistics to the console. This is useful for build tools that want to collect statistics via a compiler plugin without interfering with the console output or the operation of `-Ystatistics` (if specified explicitly by the user). Note that there is an internal `YstatisticsEnabled` setting that may appear to do this already, but in fact it controls both collecting and printing together. Even if you switched it on internally (without enabling any phase statistics via `-Ystatistics` / `-Yhot-statistics`) you would still get at least the phase timings summary.
4627ee7 to
61ca27a
Compare
These are the statistics that are relatively cheap to collect, as opposed to Depending on how the performance analysis goes, I may end up splitting cold statistics in another PR that turns |
Backport of scala#10795: This change adds a new flag `-Ycollect-statistics` that enables the same statistics gathering as `-Ystatistics` but without dumping all the statistics to the console. This is useful for build tools that want to collect statistics via a compiler plugin without interfering with the console output or the operation of `-Ystatistics` (if specified explicitly by the user). Note that there is an internal `YstatisticsEnabled` setting that may appear to do this already, but in fact it controls both collecting and printing together. Even if you switched it on internally (without enabling any phase statistics via `-Ystatistics` / `-Yhot-statistics`) you would still get at least the phase timings summary.
|
After running some more benchmarking builds (a medium-size build of part of our codebase running in parallel on a 192-core machine), I'm getting an average build time of 770s without statistics and 820s with statistics, a 6.5% increase. This is purely from having cold statistics collecting enabled, without any output. Splitting off a subset of even colder statistics looks worthwhile. I wouldn't want to take a 6.5% performance hit in all our builds. |
|
The theory behind https://github.com/scala/scala/blob/2.13.x/src/reflect/scala/reflect/internal/util/AlmostFinalValue.java doesn't seem to work out, see also #6234, more follow up work was in https://github.com/scala/scala/pulls?q=is%3Apr+AlmostFinalValue+is%3Aclosed. Maybe it depends on the JVM version? |
|
At first glance it looks like #6234 defeats the point of this trick. Or is it still better when statistics have been enabled but are turned off at a later time? I would expect that never enabling statistics will result in similarly efficient code. I'll separate the phase timings from the rest (using their own AlmostFinal machinery) and run another benchmark with that. I hope that even with an inefficient AlmostFinal the overhead will be close to zero because the timers are only accessed a dozen or so times per build. |
Backport of scala#10795: This change adds a new flag `-Ycollect-statistics` that enables the same statistics gathering as `-Ystatistics` but without dumping all the statistics to the console. This is useful for build tools that want to collect statistics via a compiler plugin without interfering with the console output or the operation of `-Ystatistics` (if specified explicitly by the user). Note that there is an internal `YstatisticsEnabled` setting that may appear to do this already, but in fact it controls both collecting and printing together. Even if you switched it on internally (without enabling any phase statistics via `-Ystatistics` / `-Yhot-statistics`) you would still get at least the phase timings summary.
-Ycollect-statistics for collecting statistics without printing them
This change adds a new flag
-Ycollect-statisticsthat enables the same statistics gathering as-Ystatisticsbut without dumping all the statistics to the console. This is useful for build tools that want to collect statistics via a compiler plugin without interfering with the console output or the operation of-Ystatistics(if specified explicitly by the user).Note that there is an internal
YstatisticsEnabledsetting that may appear to do this already, but in fact it controls both collecting and printing together. Even if you switched it on internally (without enabling any phase statistics via-Ystatistics/-Yhot-statistics) you would still get at least the phase timings summary.I have a 2.12 version ready as well. It's not quite merge compatible for direct cherry-picking.