TestBed.configureTestingModule Performance Issue

[ollwenjones]

I'm submitting a ... (check one with "x")

[ ] bug report 
[x] feature request
[ ] support request 

Current behavior

Importing modules into TestBed.configureTestingModule can slow tests down substantially. We have a 'shared module' with parts, that lots of our components use, including a third party library or two. It is very convenient to import this module as a rather than cherry pick in the tests, however testing a small component yesterday, I saw test bootstrap taking two whole seconds and had to cherry-pick dependencies to make tests run reliably.

Expected behavior

TestBed.configureTestingModule should be performant.

Minimal reproduction of the problem with instructions

In this plunker

1. go to the src/simple.spec.ts and comment and un-comment the SharedModule import into configureTestingModule.
2. Observe time it takes for test to run.

I am seeing a jump from 0.079s to 0.241s. (~= 3x slower). Multiply that by 5 cases and this simple test takes a whole second. Make the shared module larger and you have real problems.

What is the motivation / use case for changing the behavior?

1. Slow tests hurt TDD workflow
2. Slow enough tests disconnect karma from the browser and fail the suite.

Please tell us about your environment:

Win10, VsCode, Webpack,

• Angular version: 2.0.X

  Yes.

• Browser: Locally, PhantomJS. Plunk tested in Chrome.

• Language: TypeScript (though tests are transpiled to ES5 prior to running, naturally)

Note: Even as I write this, I wonder if it's a reasonable expectation for large modules to go through TestBed quickly? Maybe I'm just doing something wrong architecturally, (use multiple, smaller shared modules, not one big one, etc.) However, tests are time-consuming enough without having to track down every individual dependency of a more complex component just to get started.

[ollwenjones]

Seems like the goal is really to have a fresh component instance each time, and for the most part the testing module is going to be static for a particular .spec file. It also seems like configuring the module at each iteration is the part that is really adding on the time. A better pattern could be configuring the module once, then testModule.createComponent multiple times.

TestBed.configureTestingModule once and calling TestBed.createComponent on that module multiple times doesn't work, though... maybe I'm missing an obvious way to preserve module state between iterations?

Edit: FWIW I tried logging the timestamp and it looks like most of the time passes during TestBed.createComponent, not TestBed.configureTestingModule - however I still suspect it's related to module configuration, since adding things to imported modules adds to the time, whether or not those imports are used in the component being tested.

[vmandy]

I have a noticed unit test execution performance issues that seem directly related to this issue.

I have an ionic 2 / angular 2 app. When trying to unit test my components using TestBed and ComponentFixture classes, there is easily 1-2 seconds spent in the setup of each test that seems directly related to the imports: [IonicModule.forRoot(MyComponent)] that I've added to my testing module via the TestBed.configureTestingModule({...}) method.

When I remove the import, my tests run much faster, but they obviously fail because my component's template no longer compiles.

[ollwenjones]

@vmandy I've been working around this by doing one or more of:

1. spending more time curating just what I need in the test module
2. mocking the component dependencies that are time-consuming to bootstrap
3. overriding the template of the component I'm testing so it doesn't depend on much.

All of which are time consuming and sub-optimal 👎 (except from the point of view that I am really forced to isolate the unit I am testing?)

It seems like there are some theoretical optimizations to this at least, because I know these components are not taking 1-2 secs to bootstrap at run-time.

[FlemmingBehrend]

Iam having similar problems. Thought it was caused by wallaby, but it seems to be the TestBed.

wallabyjs/public#885

[mlakmal]

I am seeing the same issue. Any unit test with component compilation is taking close to 1sec...

[ollwenjones]

Unfortunately I was pulled off the Angular 2 project and haven't looked at this in quite some time. It seems like a lot of the complications of TestBed (test-module setup, generally, not just performance) could be avoided by an approach similar to the enzyme.shallow render approach in the React ecosystem. - where just one unit is rendered and none of it's dependencies - the same thing can be accomplished by over-riding an Angular component's template in the .spec, but that is often tedious/time-consuming as well.

[mlakmal]

we have close to 2000 test cases getting executed just under 1 min in Angular 1 where Angular 2 takes 1min to execute just 100 test cases. It would be nice if we can get a resolution for this issue as it hinders the testing capabilities in large applications.

[ollwenjones]

@mlakmal I think there are lots of folks relying heavily on e2e tests, or just testing functional logic in component classes

let component = new ComponentClass(mockDep1, mockDep2);
expect(component.doAddition(2,2)).toBe(4);

Which doesn't test any of the template rendering (template logic)... but it does allow easy testing of class logic at least

[mlakmal]

@ollwenjones thanks, i am going to try that out. i really doesn't want to test any template logic since most of that gets covered by our automated tests, so just testing the component class code should be enough.

[ollwenjones]

@mlakmal sweet! Makes me happy to contribute something that actually helps someone. 🍻 - FWIW I'm often torn about template-rendering unit tests, because it's so easy to slip into testing the framework.

[michaelbromley]

I'm in the process of upgrading a pretty big project from rc.4 to 2.3, so it's the first time we're using the NgModule approach.

We have ~1000 unit tests, with maybe 1/2 of those testing components. Prior to upgrade, tests run pretty quickly - around 30s.

After upgrade, they take at leat twice as long, but the worst thing is that they seem kind of "flaky" - Karma will periodically lost the connection to Chrome and often I will need to refresh Chrome multiple times. I suspect that something is timing out but haven't isolated it yet.

@ollwenjones regarding "shallow" component tests, did you know about the NO_ERRORS_SCHEMA that you can set in the testing module?

import { NO_ERRORS_SCHEMA} from '@angular/core';
// ...
TestBed.configureTestingModule({
  declarations: [ /*... whatever */ ],
  schemas: [NO_ERRORS_SCHEMA]
});

This means that the compiler just ignores any elements it does not recognize, meaning you do not need to declare all the components used in the template of the component under test.

[marclaval]

Found the same issue in Angular's tests: #13500
I was able to improve the situation by creating bespoke testing modules for each test.