GCC crash in fullOptJS with ju.HashSet.remove

[tbje]

Scala version: 2.13.2
ScalaJS version: 1.1.0 or 1.0.1 (does not occur with sjs 0.6.x)
Minimal project: https://github.com/tbje/full-stack/tree/problem-hashset
Macros: no

Calling java.util.HashSet.remove twice results in fullOptJS compiler error.

Step to reproduce: fullOptJS in sample project.
No issue with fastOptJS.

Expected: generate optimized js file.
Actual: Compiler error

sbt:full-stack> last client/fullOptJS
[info] Full optimizing /home/tbje/proj/full-stack/js/target/scala-2.13/client-opt.js
[debug] Linker: Compute reachability: 447619 us
[debug] Linker: Assemble LinkedClasses: 32454 us
[debug] Linker: Check IR: 422904 us
[debug] Linker: 933965 us
[debug] Optimizer: Batch mode: true
[debug] Optimizer: Incremental part: 106715 us
[debug] Optimizer: Optimizing 5886 methods.
[debug] Optimizer: Optimizer part: 2042461 us
[debug] Optimizer: 2161914 us
[debug] Refiner: Compute reachability: 122983 us
[debug] Refiner: Assemble LinkedClasses: 8261 us
[debug] Refiner: 147066 us
[debug] Emitter: Generate classes: 746186 us
[debug] Emitter: Write trees: 19139 us
[debug] Emitter: Class tree cache stats: reused: 945 -- invalidated: 962
[debug] Emitter: Method tree cache stats: reused: 0 -- invalidated: 3789
[debug] Emitter: 811775 us
[debug] Closure: Create trees): 497838 us
[info] Closure: 0 error(s), 0 warning(s)
[error] java.lang.RuntimeException: INTERNAL COMPILER ERROR.
[error] Please report this problem.
[error]
[error] An enclosing scope is required for change reports but node NAME node$jscomp$inline_2106 114 [originalname: node] [source_file: https://raw.githubusercontent.com/scala-js/scala-js/v1.1.0/javalib/src/main/scala/java/util/HashMap.scala] [input_id: InputId: https://raw.githubusercontent.com/scala-js/scala-js/v1.1.0/javalib/src/main/scala/java/util/HashMap.scala] doesn't have one.
[error]   Node(BLOCK): https://raw.githubusercontent.com/scala-js/scala-js/v1.1.0/javalib/src/main/scala/java/util/HashMap.scala:112:37
[error] [source unknown]
[error]   Parent(BLOCK): https://raw.githubusercontent.com/scala-js/scala-js/v1.1.0/javalib/src/main/scala/java/util/HashSet.scala:43:20
[error] [source unknown]
[error]
[error] 	at com.google.javascript.jscomp.Compiler.getChangeScopeForNode(Compiler.java:2560)
[error] 	at com.google.javascript.jscomp.Compiler.reportChangeToEnclosingScope(Compiler.java:2607)
[error] 	at com.google.javascript.jscomp.InlineVariables$InliningBehavior.inlineValue(InlineVariables.java:395)
[error] 	at com.google.javascript.jscomp.InlineVariables$InliningBehavior.inlineWellDefinedVariable(InlineVariables.java:344)
[error] 	at com.google.javascript.jscomp.InlineVariables$InliningBehavior.inlineNonConstants(InlineVariables.java:265)
[error] 	at com.google.javascript.jscomp.InlineVariables$InliningBehavior.doInlinesForScope(InlineVariables.java:180)
[error] 	at com.google.javascript.jscomp.InlineVariables$InliningBehavior.afterExitScope(InlineVariables.java:124)
[error] 	at com.google.javascript.jscomp.ReferenceCollectingCallback.exitScope(ReferenceCollectingCallback.java:262)
[error] 	at com.google.javascript.jscomp.NodeTraversal.popScope(NodeTraversal.java:1048)
[error] 	at com.google.javascript.jscomp.NodeTraversal.popScope(NodeTraversal.java:1039)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseBranch(NodeTraversal.java:855)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseFunction(NodeTraversal.java:900)
[error] 	at com.google.javascript.jscomp.NodeTraversal.handleFunction(NodeTraversal.java:806)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseBranch(NodeTraversal.java:832)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseChildren(NodeTraversal.java:989)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseBranch(NodeTraversal.java:857)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseChildren(NodeTraversal.java:989)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseBranch(NodeTraversal.java:857)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseChildren(NodeTraversal.java:989)
[error] 	at com.google.javascript.jscomp.NodeTraversal.handleScript(NodeTraversal.java:794)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseBranch(NodeTraversal.java:829)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseChildren(NodeTraversal.java:989)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseBranch(NodeTraversal.java:857)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseRoots(NodeTraversal.java:397)
[error] 	at com.google.javascript.jscomp.ReferenceCollectingCallback.process(ReferenceCollectingCallback.java:111)
[error] 	at com.google.javascript.jscomp.InlineVariables.process(InlineVariables.java:88)
[error] 	at com.google.javascript.jscomp.PhaseOptimizer$NamedPass.process(PhaseOptimizer.java:317)
[error] 	at com.google.javascript.jscomp.PhaseOptimizer$Loop.process(PhaseOptimizer.java:462)
[error] 	at com.google.javascript.jscomp.PhaseOptimizer.process(PhaseOptimizer.java:232)
[error] 	at com.google.javascript.jscomp.Compiler.performOptimizations(Compiler.java:2418)
[error] 	at com.google.javascript.jscomp.Compiler.lambda$stage2Passes$1(Compiler.java:799)
[error] 	at com.google.javascript.jscomp.CompilerExecutor$2.call(CompilerExecutor.java:102)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error] 	at java.base/java.lang.Thread.run(Thread.java:834)
[error] Caused by: java.lang.IllegalStateException: An enclosing scope is required for change reports but node NAME node$jscomp$inline_2106 114 [originalname: node] [source_file: https://raw.githubusercontent.com/scala-js/scala-js/v1.1.0/javalib/src/main/scala/java/util/HashMap.scala] [input_id: InputId: https://raw.githubusercontent.com/scala-js/scala-js/v1.1.0/javalib/src/main/scala/java/util/HashMap.scala] doesn't have one.
[error] 	at com.google.javascript.jscomp.Compiler.getChangeScopeForNode(Compiler.java:2560)
[error] 	at com.google.javascript.jscomp.Compiler.reportChangeToEnclosingScope(Compiler.java:2607)
[error] 	at com.google.javascript.jscomp.InlineVariables$InliningBehavior.inlineValue(InlineVariables.java:395)
[error] 	at com.google.javascript.jscomp.InlineVariables$InliningBehavior.inlineWellDefinedVariable(InlineVariables.java:344)
[error] 	at com.google.javascript.jscomp.InlineVariables$InliningBehavior.inlineNonConstants(InlineVariables.java:265)
[error] 	at com.google.javascript.jscomp.InlineVariables$InliningBehavior.doInlinesForScope(InlineVariables.java:180)
[error] 	at com.google.javascript.jscomp.InlineVariables$InliningBehavior.afterExitScope(InlineVariables.java:124)
[error] 	at com.google.javascript.jscomp.ReferenceCollectingCallback.exitScope(ReferenceCollectingCallback.java:262)
[error] 	at com.google.javascript.jscomp.NodeTraversal.popScope(NodeTraversal.java:1048)
[error] 	at com.google.javascript.jscomp.NodeTraversal.popScope(NodeTraversal.java:1039)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseBranch(NodeTraversal.java:855)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseFunction(NodeTraversal.java:900)
[error] 	at com.google.javascript.jscomp.NodeTraversal.handleFunction(NodeTraversal.java:806)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseBranch(NodeTraversal.java:832)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseChildren(NodeTraversal.java:989)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseBranch(NodeTraversal.java:857)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseChildren(NodeTraversal.java:989)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseBranch(NodeTraversal.java:857)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseChildren(NodeTraversal.java:989)
[error] 	at com.google.javascript.jscomp.NodeTraversal.handleScript(NodeTraversal.java:794)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseBranch(NodeTraversal.java:829)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseChildren(NodeTraversal.java:989)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseBranch(NodeTraversal.java:857)
[error] 	at com.google.javascript.jscomp.NodeTraversal.traverseRoots(NodeTraversal.java:397)
[error] 	at com.google.javascript.jscomp.ReferenceCollectingCallback.process(ReferenceCollectingCallback.java:111)
[error] 	at com.google.javascript.jscomp.InlineVariables.process(InlineVariables.java:88)
[error] 	at com.google.javascript.jscomp.PhaseOptimizer$NamedPass.process(PhaseOptimizer.java:317)
[error] 	at com.google.javascript.jscomp.PhaseOptimizer$Loop.process(PhaseOptimizer.java:462)
[error] 	at com.google.javascript.jscomp.PhaseOptimizer.process(PhaseOptimizer.java:232)
[error] 	at com.google.javascript.jscomp.Compiler.performOptimizations(Compiler.java:2418)
[error] 	at com.google.javascript.jscomp.Compiler.lambda$stage2Passes$1(Compiler.java:799)
[error] 	at com.google.javascript.jscomp.CompilerExecutor$2.call(CompilerExecutor.java:102)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error] 	at java.base/java.lang.Thread.run(Thread.java:834)
[error] (client / Compile / fullOptJS) INTERNAL COMPILER ERROR.
[error] Please report this problem.
[error]
[error] An enclosing scope is required for change reports but node NAME node$jscomp$inline_2106 114 [originalname: node] [source_file: https://raw.githubusercontent.com/scala-js/scala-js/v1.1.0/javalib/src/main/scala/java/util/HashMap.scala] [input_id: InputId: https://raw.githubusercontent.com/scala-js/scala-js/v1.1.0/javalib/src/main/scala/java/util/HashMap.scala] doesn't have one.
[error]   Node(BLOCK): https://raw.githubusercontent.com/scala-js/scala-js/v1.1.0/javalib/src/main/scala/java/util/HashMap.scala:112:37
[error] [source unknown]
[error]   Parent(BLOCK): https://raw.githubusercontent.com/scala-js/scala-js/v1.1.0/javalib/src/main/scala/java/util/HashSet.scala:43:20
[error] [source unknown]

[tbje]

If removing scala-java-time-tzdb as a dependency the error does not occur.

[sjrd]

Thank you for the report. I am a bit overloaded until next Wednesday and then I'll have a few days of vacation, so it might take a few weeks before I can look at this issue.

[tbje]

No worries @sjrd, as one could guess from the Error message I manged to narrow it down a bit further.

[sjrd]

Oh thanks for the dependency-less reproduction. That is very useful. I can indeed reproduce this on the latest master, including in 2.12. For further reference, here is the code to put in the hello world:

package helloworld

object HelloWorld {
  def main(args: Array[String]): Unit = {
    val zones = new java.util.HashSet[String](3)
    zones.remove("")
    zones.remove("1") // only if remove is called twice
  }
}

and reproduction with:

> helloworld2_12/fullOptJS

[gzm0]

Note that upgrading GCC (#4100) does not resolve this issue.

[gzm0]

I have tried giving a fastopt.js file to GCC over the command line to see if this is a problem with our AST transformer. Unfortunately it chokes on the U+FF3F characters we generate:

scala-js/javalib/src/main/scala/java/util/HashMap.scala:133: ERROR - [JSC_PARSE_ERROR] Parse error. Invalid escape sequence

Or, with escaping disabled in javasciprt/Printers.scala:

scala-js/javalib/src/main/scala/java/util/HashMap.scala:133: ERROR - [JSC_PARSE_ERROR] Parse error. Character '＿' (U+FF3F) is not a valid identifier start char

[tbje]

According to @cquiroz the problem seems not to happen in all environments cquiroz/sbt-tzdb#36 (comment).

What system components could influence this?

tbje:~  > node --version
v14.3.0

[sjrd]

Node is not involved at this point. Everything happens on the JVM, using specific versions of the jars being used. There are very few environmental variables here. The only ones I can think of are JVM version and OS, but neither should impact what's happening here.

[sjrd]

Worth noting: the following line is displayed before the exception occurs:

[info] Closure: 0 error(s), 0 warning(s)

which one would think means GCC runs successfully at least some phases.

However, calls to our reporter's generateReport() method (which is responsible for printing this log line) are done in a finally block inside GCC. So this doesn't actually tell us anything.

[sjrd]

Other data point: everything works fine if we emit ES 5.1.

My assumption at this point is that this is bug in GCC. The next step will be to try and reproduce it using a vanilla invocation of GCC, instead of through our linker.

[gzm0]

I have not been able to reproduce this with a normal GCC invocation yet.

Code: gzm0@626d398
Invocation:

 java -jar ~/Downloads/closure-compiler-v20200614.jar --compilation_level ADVANCED_OPTIMIZATIONS --js helloworld-fastopt.js --js_output_file test.js