Testing a Yesod+Angular JS Application

By John Lenz. December 17, 2013.

Recently I have been investigating the best way of integrating AngularJs into a Yesod application, and I discussed the design I settled on in the previous post. One of the key design philosophies of Angular is to make testing easy. I was pleasantly surprised how easy it is to use the already developed testing tools with only a little glue code that I had to write. In particular, by separating out the Angular code into its own directory as standalone Javascript files, the excellent test runner karma can be used almost directly. This is a great boon since there has been a lot of work put into karma and Angular itself to make testing easy. Secondly, by not having any Javascript code inside the Yesod handlers and routes, the yesod-test package can also be used directly.

There are three kinds of testing that should be done: unit tests, mid-level tests, and end2end tests.

Unit Testing

In unit testing, we test individual functions or individual components. In our Yesod+Angular app, there are two kinds of unit tests:

Mid Level Testing

In mid-level testing, we want to test combinations of code and functions together. In our Yesod+Angular application, there are two natural mid-level tests we want to carry out.

Most likely the unit and mid-level tests will be combined together and this is fine: a single test section of the cabal file builds an executable containing both the unit tests and mid-level tests using hspec and yesod-test. Also a single instance of karma running both the unit and mid-level tests.

You might consider a third kind of mid-level test, where you try and test the Angular Javascript running against the real Yesod handlers (instead of mocking out the server). This is certainly possible, you could write the tests in Karma+Jasmine but have them such that you assume that the Yesod server is running (so you start yesod devel or something before running Karma). I feel that this type of testing is not needed, since you have already tested the components separately and you will be testing the Javascript and Yesod code together in end2end testing. I don't see a great need to test directly a combination of say an Angular service with the backend.

End To End Testing

In end to end testing, you test the entire application (client+server) by loading the site into a browser and interacting with the site just as a user would. Your test code does not call any Haskell code or call any Javascript code, instead you simulate clicks and key presses and check that the page in the browser looks as it should. At first glance, this seems quite complicated. But there is an excellent tool webdriver that makes this really easy to do.

WebDriver

Despite there being only one webdriver, there are a large number of libraries and projects that wrap webdriver and make it easier to use for specific languages and/or frameworks. The Angular project doing this is protractor so we might consider it. This would necessitate writing the tests in Javascript. Thankfully, the hs-webdriver and the webdriver-angular packages allows us to write this test code in Haskell because webdriver-angular includes some code from the protractor project to make using hs-webdriver on a page using Angular easier.

Since webdriver is used in so many different ways, it is hard to get a picture of what it does. So here is a brief description of webdriver, specifically targeted at how we will be using it. First, webdriver defines a network API that allows browsers to be controlled. It is at the moment a W3 Working Draft. Scanning the W3 draft gives a good overview of what is possible using webdriver. The browser side of the webdriver API has been implemented in all the major browsers, sometimes directly and sometimes as an extension. For example, there is a Firefox extension that implements the network API and controls the browser. The client side of the webdriver API has at the moment only one implementation, an implementation written in Java as part of the selenium project and called selenium-webdriver.

Thus if you write your test code in Java, you can just call the selenium-webdriver methods and these then communicate with the browser to automate it (you see a lot of examples of this on the selenium site and elsewhere online). But the selenium folks realized that not everyone wants to write their test code in Java, so they broke out the webdriver code into a standalone application called selenium-server-standalone. They also defined another network API allowing programs to communicate with selenium-server-standalone. This is how e.g hs-webdriver works. You write test code in Haskell calling methods in hs-webdriver which in turn makes network requests to selenium-server-standalone. Then selenium-server-standalone uses the W3 Webdriver network API to send commands to the browsers. This is how all the various webdriver language implementations work, e.g. webdriver-js and webdriver-python. It seems somewhat weird to me. Why not have Haskell and the various other language modules talk to the browsers directly using the W3 webdriver API? Perhaps it is because the W3 webdriver API is very new and still in development. Also, it does allow the so-called selenium grid, where you run selenium-server-standalone on a separate computer and have many VMs with various browser and operating system version combinations all registered with selenium-server-standalone. Then your test code in Haskell can then just communicate with selenium-server-standalone and have the tests run on all the different browser and OS combinations.

WebDriver and Angular

The webdriver Haskell package implements almost all of the webdriver API and is straightforward to use. The webdriver API allows looking up elements on the page by css selectors, ids, and several other ways. But usually when using Angular you are not adding ids to all elements. You just specify some binding which Angular automatically keeps up to date (if you were using JQuery you would need all these elements to have ids so you could update them yourself). Also, elements generated by ng-repeat can also be hard to load using only ids or css selectors. Thankfully, webdriver can also load elements on the page via Javascript. That is, a Javascript fragment can be sent from the test code to the browser over the network and the result of executing the Javascript is returned to the test code. This can be exploited to load elements using selectors based on Angular concepts.

While executing Javascript from the test code is very powerful, it is also not the easiest to use since to query elements by e.g. Angular bindings requires some internal knowledge of the properties that Angular sets on elements. Therefore, the Angular developers started the protractor project. It has a bunch of Javascript fragments that can lookup various page elements by their Angular attributes. This allows you to query the final HTML using selectors that reference the directives that are used by the file that is served. The protractor project then integrates these Javascript fragments into functions which extend the webdriver-js Javascript bindings.

I took these Javascript fragments from the protractor project and wrote a few Haskell functions which executed these Javascript fragments on the webpage. The webdriver Haskell package already had bindings to allow Javascript fragments to be executed, so all I had to do was load the fragments from protractor and call the webdriver functions. It was a surprisingly small amount of code, and it is contained in the webdriver-angular package.

With that, writing end to end tests using webdriver and webdriver-angular is quite easy. The only snag was that the webdriver package had the test code executing in the WD monad, so I had to write a small amount of boilerplate to turn WD monads into hspec Examples and also to lift several hspec expectations like shouldBe into the WD monad. But once that was done, writing end to end tests were quite easy.