Incredibly slow (near unusable) on Docker container

[mbrio]

I've begun using Docker containers for many of my projects and have noticed how incredibly slow Jest runs in a Docker container when executing on a shared folder on OS X. Within Docker you can mount a local path and map it to a path within the container (e.g. ~/Source/projecta:/usr/src/app). When using this configuration my tests which consists of a single file with a single unit test takes over 30 seconds; but when run locally outside of Docker it takes 1 second. I'm using the command jest specs/unit; which I got from another Jest issue suggesting that supplying the test folder could speed up tests, and it did, locally, from 3 seconds to 1 second; but made no change to how it runs in Docker. I would have left it at that and said that it's a Docker for Mac issue (especially because there is a speed issue with mounted volumes that is known) but I ran the same exact single test using mocha and it executed in milliseconds. The unit test is a simple expect(true).toBe(true) (or for mocha+chai expect(true).to.equal(true) and for some reason Jest, under Docker, executed in ~30 seconds vs mocha's less than 1 second.

Are there any configuration settings or optimizations that I could use to help speed up Jest? I would much rather use Jest over Mocha, but will obviously use the test platform that executes in a reasonable amount of time. This is my Jest configuration:

"jest": {
    "automock": false,
    "verbose": true,
    "testFileExtensions": [
      "js"
    ],
    "moduleFileExtensions": [
      "js",
      "json"
    ],
    "testDirectoryName": "spec/unit",
    "testPathIgnorePatterns": [
      "/node_modules/",
      "<rootDir>/lib/"
    ],
    "modulePathIgnorePatterns": [
      "/node_modules/",
      "<rootDir>/lib/"
    ]
  }

[cpojer]

Hmm, that is very weird. Is the file system access slow, that's really the only thing that should be different here between Jest and mocha? Jest crawls the file system from the root (package.json) and looks at all the files for static analysis. How many cores does the system report that it has? Can you try invoking Jest with -i?

[cpojer]

Another thing is that if you are using babel-jest, babel actually loads a ton of modules so if the fs is slow, this might contribute to that.

[boujeepossum]

i remember having similar kind of issues inside docker container. The problem was the slowness of fs access. I think it was related to mounting a directory through a virtual container

[boujeepossum]

found the issue docker/compose#631
not sure if it's still relevant

[aaronjensen]

This is a known docker for mac issue: https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076/158

[cpojer]

Yeah I'm gonna close this out because unfortunately it isn't actionable from our side :(

[mbrio]

Hi @cpojer, sorry I didn't mean to drop off like that, I did get a chance to run jest -i, I also installed native watchman and ran jest -i --watchman, neither of which sped up the processing of tests. As I mentioned in my original post I understood that there was a speed issue with Docker and mounted drives. I would like to point out that I have implemented mocha in conjunction with chokidar for watching files and still do not see the kind of slowdown I'm seeing with jest. Chokidar+mocha detects file changes in no more than 2 seconds and executed tests in under a second; jest takes over 30 seconds, sometimes up to a minute to detect a file change and executes the single test in a little over a second. Also, I'd like to note that I am not using babel-jest, I am using only the features of es6 that are available to node 6. I understand if this is something you are not looking to persue, but there does seem to be some sort of bottleneck in how jest watches files when compared to chokidar. Finally, as to your question about cores, my VM has 2 cores and 4gb of RAM.

[amkoehler]

@mbrio I decided to switch over to mocha, and the tests run immediately within my docker container without an issue. No changes were made to my Dockerfile or docker-compose configuration. This seems like a Jest issue and not a Docker issue. I had no issues with running my Jests tests locally, only within the container.

[mbrio]

@amkoehler yeah, I think there's a significant difference in the libraries that Jest and Mocha use for watching filesystem changes. I assume that Mocha is less thorough, (and/or) less active, and/or less compatible with all filesystems. The speed issue is definitely an issue with Docker for Mac as I have friends who have slightly different configurations for running Docker on a Mac that do not see as big of an issue. With that said, since Docker for Mac is the official way to run Docker instances, it would be great for the developers of Jest to consider looking into how to be a bit more compatible with it.

[wmartins]

I came to this issue lots of times when trying to figure it out why jest was slow.
Somehow, in my scenario, I was able to improve the performance. I'll just leave it here in case anyone is trying to figure it out too.

Scenario:

Building my application inside a docker container.

In my case, the container was based on Linux Alpine.

My Solution:

I changed the container base image to use Ubuntu instead of Alpine.
My tests started to run faster (from 3 ~ 4 min to 40s ~ 60s).

Hope this helps someone! :)

[alexgorbatchev]

This page pretty much says that osxfs is slow and that's being worked on...

As of Dec'17, running jest can be considered unusable in my opinion. I'm seeing 28s time for 3 basic test files inside a container. Exact same thing on the host takes 0.4s. This can work for CI purposes, but not for developer env.

There are a few clunky workarounds for this, all of which in my opinion mostly defeat the point of running development env inside containers. docker-sync and d4m-nfs are considered to be currently best solutions for this. Personally I'm not interested in trying either because if I can't use docker pull as the only command to "setup" my dev environment, then I may as well not bother until osxfs is fixed.

[thymikee]

@rogeliog is this something you could potentially find someone in Docker team to fix?