JsEnv timeouts start too early

[japgolly]

My travis builds often fail because PhantomJSEnv timeouts halfway through the tests. The fix for this was to change my .travis.yml from

script:
- sbt ++$TRAVIS_SCALA_VERSION test

to

script:
- sbt ++$TRAVIS_SCALA_VERSION test:fastOptJS
- sbt ++$TRAVIS_SCALA_VERSION test

This, coupled with the fact that I never get timeouts locally in long-running SBT sessions, implies that whatever timeout is in-place (I know we've discussed this before but I never come away with a proper understanding - sorry) begin when the batch of commands are issued. This should ideally be changed to begin when tests truly commence.

That being said, the workaround is trivial so if a fix isn't worth the effort, that's cool - feel free to close.

[sjrd]

Oh that's ... interesting. The workaround is actually very informative. We should be able to fix this.

[gzm0]

What is the error message you get when it times out?

[japgolly]

It just says

org.scalajs.jsenv.ComJSEnv$ComClosedException: Phantom.js isn't connected

Here's a sample full log: https://travis-ci.org/japgolly/scalajs-react/builds/106686299

[nicolasstucki]

Inspecting the log I can see that this is in fact not a time out. There are 2 possibilities: (i) the Jetty websocket was closed or (ii) the async VM run has finished. I do not have enough information to know which one it was and why it closed/finished.

@japgolly, if this issue is reproducible could you retest it on travis with the debug output:

script:
- sbt debug ++$TRAVIS_SCALA_VERSION test

[japgolly]

@nicolasstucki Sure, here you go: https://travis-ci.org/japgolly/scalajs-react/builds/107872634

I've also created a branch https://github.com/japgolly/scalajs-react/tree/tmp/timeout if you want to toy with the Travis config.

[nicolasstucki]

I've tried almost every combination of configurations imaginable to try to reproduce this on my machine, but was not able.

From the debug logs I was able to find that it is the jetty websocket connection that closes unexpectedly for no apparent reason. In that scenario, Phantom does not seem to have crashed, it is just a lost in the connection.

I do believe that it must be an issue with the version of Phantomjs that is used by travis, a custom build of phantom 2.0.0. I would wait until travis supports 2.1.1 to see if this issue is maybe fixed by it. Meanwhile we should use the simple work around stated in the first comment.

[japgolly]

I've tried almost every combination of configurations imaginable to try to reproduce this on my machine, but was not able.

@nicolasstucki I feel your pain. :) When you tested on your machine, did you change the tests to make them take a long enough time? I mean, if you checked out that branch I linked and just run locally it will pass because (←assumption) our machines are much faster than Travis so it won't timeout.

...off to the console and back...

Ok I can reproduce it locally (with PhantomJS 2.1.1) by adding a long-running test:

package omg

import utest._

object OMG extends TestSuite {
  override def tests = TestSuite {'x{
    var i = 0
    for {
      _ <- 0 to 1000000
      _ <- 0 to 1000000
    } i += i
    i
  }}
}

then running

sbt debug clean test | tee debug.txt

the output of which is here: debug.txt

[nicolasstucki]

Well, I tried testing that same branch while overloading my CPU with juck processes. It executed 10 slover than in Travis but still did not get any problem. I will try the code that you just gave me, maybe it will finally fail and I'll be happy (because Ill have a way to find the issue).

[japgolly]

It could also be an environmental issue. If there's any more info I could provide that would be helpful, please let me know.

[nicolasstucki]

I've been able to reproduce the failure. There is a timeout of 5 minutes on a WebSocket that we open using Jetty. I will add an option to set a new larger timeout on a PhanthomJSEnv.

[gzm0]

@nicolasstucki who does set the 5 minute timeout? I think in this case it might actually be justifiable to drop the timeout entirely or make it very large (hours, days).

Timeouts are usually a good thing in (long running) systems. But in this case there should be higher-level tools that catch these or you have a developer actually observing the build. So it seems that here, timeouts are more counter-productive.