Thursday, 22 September 2011

Javascript Build Systems

Whew it’s been a while since I last posted here! I have moved to Malaysia and got married in that time :)

When I originally started working on Javascript, Ad products in particular, I wrote my code in one big Javascript file, tested everything manually and used jsmin to compress it. This was not an ideal solution, and I wasted a lot of time chasing down bugs due to missing semicolons, mis-matched braces and various typos. Another problem with this approach was that as the code got longer, things became harder to find and I found myself adding large comments at the start and end of sections: //START TOUCH EVENT HANDLERS and //END TOUCH EVENT HANDLERS.

Later on I discovered JSLint which saved me from a large number of “stupid” bugs, although I was still manually copying and pasting my code into the online linter :-O

I am using my own custom build system, with my code separated into files that deal with a certain piece of functionality which are then concatenated together, checked for errors and minified with one command line.

jQuery Build System

I based my system on this system (which can be found in the jQuery GitHub repository) which works as follows:
  1. make jquery which will concatenate all of the source files together and output the complete Javascript as one file (Looking at the Makefile you can see the source files listed under BASE_FILES and then wrapper with an intro and outro script in MODULES)
  2. make lint which will check the built file against JSLint and output any errors to the console
  3. make min which minifies the code using uglify.js

My Build System

As I am building a number of different Ad products, which often share similar functionality, I have my own build scripts which work in a similar way but with the ability to build different combinations of files.

Firstly, I use the product name as the target (e.g. make product.js) with rules in my makefile for the specific set of files required to build that product. This allows me to share files between the products easily, which used to involve copy and pasting blocks of code!

Also, instead of using JSLint I prefer to use JSHint as I find it’s rules more suited to my coding style. I pull in the JSHint repository as a submodule so that I can update it easily and track the version I am using.

Finally, I use the Google Closure Compiler instead of uglify.js (or various other alternatives). I like this as it really compresses my code a lot (Advanced mode, with the code written to work with this) and it provides additional checks to my code such as type checking (via JSDOC comments) and missing property checks. I use the downloadable JAR file to run this locally, although it does take a few seconds to run on larger code bases, I feel it is worth it.

Other Things

I also have a little python script which will run a local webserver for me to test builds on. This includes the ability to configure the Javascript files from XML config files and a basic templating engine so I can test on desktop or mobile sites.

My scripts also will push versioned builds to the live servers, to ease that step too.


Although it takes a little while to setup these automated scripts, it really is worth it in the long run. As so many bugs can be picked up immediately (without opening a browser or test suite), and changes can easily be made to a shared file and then all products using that can be rebuilt and receive the upgrades in a much shorter time period.

I would advise anyone to automate as much of their manual process as possible, as it will make your life easier.

Also, if you are not just building Javascript, but instead a whole site, then I recommend the HTML5 Boilerplate as that will manage this kind of process for you along with a lot of other nifty things (CSS compression, server optimisation etc etc).