Tools for Testing your Javascript
In our team’s testing process (and we do have a significant process to make sure that everything gets covered), we’ve noted in the past that we have quite a bit of overlap between what’s covered by our Javascript unit tests (typically with Jasmine) and by our browser-driving acceptance tests (Cucumber and Capybara with Selenium-Webdriver). Because we don’t want to duplicate our testing work if we don’t have to, we’ve had several conversations on to the theme of “When does it make sense to use Cucumber and when does it make sense to use Jasmine”.
Jasmine, if you haven’t used it, is a way to test your functionality in the same environment and at the same level as your javascript code. Much like when we’re working on the server-side, the test and the code share syntax and vocabulary (and running the resultant tests is very fast). The downside? Much of the time the javascript in our applications is working very closely with text and elements in the DOM of the web page that it’s included in. It could be argued that maybe this shouldn’t be the case, but if your use case is something like, “when I click this link, I want a modal dialog box to show up”, how far should the code really be abstracted away from the DOM that it’s working with? Because of this close relationship, jasmine tests often need to have a mocked DOM created and included that replicates the html we’d find coming from the application server, and this means keeping them in sync in order to preserve the usefulness of these tests (there is much to be said on HOW you can do this, but that is probably outside the scope of this post).
Cucumber, on the other hand, is the Golden Hammer of web application testing. It runs the whole stack, database to front-end javascript, so your front-end code is being tested in the exact environment that it runs in, including full interaction with your server code in it’s current state. A scenario will, in one fell swoop, log in to the site, load page, start clicking on things, and confirm that the visible results of your actions occur as you intend them to. Of course, as a result of this kitchen sink approach (which is necessary for full integration testing), the time it takes to run large suites of tests is not exactly low. 13 difference scenarios for testing different nuances of javascript-y interaction with a simple form can easily consume 30 seconds or more on each test run. Multiply that over every part of your application and you can be waiting a long time to make sure everything is still working.
During our latest team retrospective, in order to strike a balance, we’ve settled on a policy that I think leverages the strengths of each tool in order to reach a relative best-case scenario. Our core value is this: speed is critical because fast tests get run. Tests that run slowly just don’t get executed as often during the development cycle, and this quickly hurts their value to us. Because of this, we use jasmine for all of our front-end javascript. Just like our ruby code, nothing should be written without a unit test to cover it. In order to mitigate the issue with having HTML that accurately represents what’s being delivered by the server, we have a clever way of generating these fixtures of static html from the current server-side code base just prior to running the tests. Cucumber is in the mix, but only for what it was designed to be anyway: for integration and acceptance testing. Our cucumber scenarios (which, when done correctly, should be somewhat fewer in number than our unit tests) provide assurance that our main requirements from the customers are met and that all the elements of the application are working together successfully; meticulously executing every line of browser-based javascript is simply beyond the scope of such a tool.
As with all development tools, these things only have value to us insofar as they help us provide value to our customers. With our new strategy in place, I like to think that’s exactly what we’re accomplishing.