Documentation and automation for using JITWatch to analyse JMH benchmarks

[retronym]

I found JITWatch really useful to drill into regressions in HashMap performance recently, but its a pain to configure manually.

This PR automates the generation of a config file containing the sources (including source JARs of dependencies) and classes (including dependency JARs).

Instructions

(These are also in the benchmark README.md)

You can generate the hotspot.log file for a benchmark run by adding the required JVM options
to JMH benchmark execution:

sbt:root> bench/jmh:run scala.collection.mutable.ArrayOpsBenchmark.insertInteger -psize=1000 -f1 -jvmArgs -XX:+UnlockDiagnosticVMOptions -jvmArgs -XX:+TraceClassLoading -jvmArgs -XX:+LogCompilation -jvmArgs -XX:LogFile=hotspot.log -jvmArgs -XX:+PrintAssembly
...
[info] Loaded disassembler from /Users/jz/.jabba/jdk/1.8.172/Contents/Home/jre/lib/hsdis-amd64.dylib
[info] Decoding compiled method 0x0000000113f60bd0:
[info] Code:
[info] [Disassembling for mach='i386:x86-64']
[info] [Entry Point]
[info] [Constants]
[info]   # {method} {0x000000010ffa0000} 'hashCode' '()I' in 'java/lang/String'
[info]   #           [sp+0x40]  (sp of caller)
[info]   0x0000000113f60d40: mov    r10d,DWORD PTR [rsi+0x8]
[info]   0x0000000113f60d44: shl    r10,0x3
...
[info] # Run complete. Total time: 00:00:30
[info] Benchmark                        (size)  Mode  Cnt       Score      Error  Units
[info] ArrayOpsBenchmark.insertInteger    1000  avgt   10  188199.582 ± 5930.520  ns/op

JITWatch requires configuration of the class and source path. We can generate that with a custom
task in our build:

sbt> bench/jmh:jitwatchConfigFile
[info] Resolving jline#jline;2.14.6 ...
jmh
[info] ./launchUI.sh -Djitwatch.config.file=/Users/jz/code/scala/test/benchmarks/target/jitwatch-jmh.properties

sbt> ^C

Build jitwatch.

$ git clone https://github.com/AdoptOpenJDK/jitwatch
$ cd jitwatch
$ mvn install

Launch with the generated config file.

$ ./launchUI.sh -Djitwatch.config.file=/Users/jz/code/scala/test/benchmarks/target/jitwatch-jmh.properties

[retronym]

The best way to review/improve the PR would be to try to follow the instructions and see how far you get :)

[lrytz]

This is really great! In other words, please share more of the tricks in your wizardry toolbox 😃

I'll give it a try. The test failure would go away after a rebase (#7289 (comment)).

[lrytz]

typo "ing"

[smarter]

Very neat, can we expect an sbt-jitwatch ? :)

[retronym]

Very neat, can we expect an sbt-jitwatch

I'll push this up to sbt-jmh once it stabilizes a bit.

[lrytz]

Worked for me! 👍

[lrytz]

For macOS there are currently two files there: hsdis-amd64-darwin-67f6d23cbebd8998450a88b5bef362171f66f11a.dylib and hsdis-amd64-darwin-fdb13ef0d7d23d93dacaae9c98837bea0d4fc5a2.dylib. I usesd the fdb13... version, which seems to work, didn't try the other. Also I had to rename it to hsdis-amd64.dylib to make it work (and put it int the jre/lib/server directory).

[lrytz]

sbt logs an error for me

sbt:root> bench/jmh:jitwatchConfigFile
[error] 	local: bad info.apiURL found in /Users/luc/.ivy2/local/org.scala-lang/scala-reflect/2.13.0-pre-SNAPSHOT/ivys/ivy.xml: expected='http://www.scala-lang.org/api/2.13.0-pre-SNAPSHOT/' found='null'

and then

[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	::          UNRESOLVED DEPENDENCIES         ::
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	:: org.scala-lang#scala-reflect;2.13.0-pre-SNAPSHOT: java.text.ParseException: inconsistent module descriptor file found in '/Users/luc/.ivy2/local/org.scala-lang/scala-reflect/2.13.0-pre-SNAPSHOT/ivys/ivy.xml': bad info.apiURL found in /Users/luc/.ivy2/local/org.scala-lang/scala-reflect/2.13.0-pre-SNAPSHOT/ivys/ivy.xml: expected='http://www.scala-lang.org/api/2.13.0-pre-SNAPSHOT/' found='null';
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::

but the generated file works nevertheless.

[retronym]

Not sure how to get rid of that.