Converting AMD to CommonJS

Browser technologies are evolving faster than most projects can keep up. If you started a project about a year ago, you probably evaluated some popular frameworks and ended up building your app with a set of tools which now feel outdated.

At Uber, we move incredibly fast and I found myself in the position of writing an app with tools which were somewhat perpendicular to the company's renewed focus. When the project started we used RequireJS as a module loader, Grunt as a task runner and bower as a package manager.

These things work great, but the company is moving in a different direction. There's no denying the simplicity of browserify and CommonJS. We've built a lot of build chain infrastructure around Gulp, and seeing as we love Node, wouldn't it be nice to manage all of our dependencies with npm?

The Engineer's Dilemma

If you've ever been in a similar situation, you may have felt the anxiety of working with 'outdated' tools. Welcome to the Developer's Dilemma; do you spend the time to adopt newer technology now, or continue with the current stack and make the conversion harder later on?

Evolve, don't fork

Every engineer will approach this situation multiple times over the course of their career. In rare circumstances I have seen forks (or complete rewrites) work wonderfully. More often than not, they fail drastically due to a gross underestimation of the effort, scope and time needed to complete the project. Couple that with the inability for engineers to cut their losses and the oven is already hot for your recipe of disaster.

As forking is hard to justify and likely to fail, we took the iterative approach to this project and decided to convert our project to browserify in stages. Our app has hundreds of files, each one with lots of dependencies and we also rely on some features of RequireJS like the 'text' plugin and using Require's r.js optimizer in our build chain.

AMD with the simplicity of CommonJS

There are some brilliant tools out there like browserify-ftw which will actually convert from a RequireJS project to a Browserify one. Unfortunately for us, we would first need to exercise all of our aforementioned RequireJS demons. As we didn't want a complete fork, browserify-ftw is was out, at least for now.

When discussing this refactor with a colleague, they pointed out that RequireJS has a little known syntax which very closely resembles CommonJS. Where you might normally write something like this:

define([  
  'jquery'
  'underscore',
  'i18n'
], function($, _, i18n){
  'use strict';

  return {
    // Your module
  };
});

You can instead write this:

define(function(require, exports, module){  
  'use strict';

  var $ = require('jquery')
  var _ = require('underscore');
  var i18n = require('i18n'); 

  module.exports = {
    // Your module
  };
});

The only thing separating this from CommonJS is the initial define function call - everything else is identical. With this syntax, we can start getting the simplicity and clarity of the CommonJS style whilst we work on removing our other RequireJS dependencies. As a bonus, this style also let our linter detect unused dependencies (there were a lot!) as they were now variables instead of function arguments.

A daunting task

The first thought was to manually update each file that we touched into this new style. That worked for a little while, but it was quite tedious and cumbersome. Taking inspiration from browserify-ftw, I decided to write a module to do this transformation for me and amd-to-common was born.

The library uses Esprima - an incredibly useful ECMAScript parser to analyse each of the AMD files in a project. Esprima takes in the code as a string and outputs a AST object which can be traversed and searched.

We first need a little bit of code to determine if a particular node from the AST is the AMD style we're looking for:

/**
 * Determine whether a node represents a requireJS 'define' call.
 * @param {Object} node AST node
 * @returns {Boolean} true if define call, false otherwise
 */
AMDNode.prototype.isDefine = function(){  
  var node = this.node;
  if(!node || !node.type || node.type !== 'ExpressionStatement'){
    return false;
  }
  if(node.expression.type !== 'CallExpression'){
    return false;
  }
  return Boolean(node.expression.callee.name === 'define');
};

/**
 * Determine whether a node is an AMD style define call
 * This detects code in the format:
 *    define(['req1', 'req2'], function(Req1, Req1) {})
 * But not:
 *    define(function(require, exports, module){})
 * @param {Object} node AST Node
 * @returns {boolean} true if AMD style, false otherwise
 */
AMDNode.prototype.isAMDStyle = function(){  
  if(!this.isDefine()){
    return false;
  }
  var defineArguments = this.node.expression.arguments;
  if(defineArguments[0].type !== 'ArrayExpression'){
    return false;
  }
  return Boolean(defineArguments[1].type === 'FunctionExpression');
};

If we find the AMD style that we're searching for, we then pass that node on to a module which will convert this into the CommonJS style.

Writing code with code

Once we've identified a node that we want to convert, we need to figure out how to convert that to the new style. My first thought was another fantastic library called
escodegen which will convert back from an AST to actual code. Unfortunately for us, in the process it also will write that code with it's own syntactical conventions. That's a great feature for easily 'cleaning code' but a bad one for converting a project. Ideally, we don't want to change every single line of the project.

This leaves us with the unfortunate process of slicing and dicing the code as a string. We have to take 4 things in to consideration for the conversion:

  1. Removing the array of string dependencies
  2. Converting those dependencies to variable declarations
  3. Changing return to module.exports = ...
  4. Moving the 'use strict' definition to the correct place.

As a sample, here's the code used to convert the return statement to module.exports:

var _ = require('underscore');

/**
 * Given the content and an AST node, convert the return statement
 * to be a module.export
 * @param {String} content The code
 * @param {Object} node The AST node
 * @returns {String} The converted content
 */
module.exports = function convert(content, node){  
  var defineFunction = node.body[0].expression.arguments[0].body;
  var functionBody = defineFunction.body;
  var returnStatement = _.find(functionBody, function(node){
    return node.type === 'ReturnStatement';
  });
  if(!returnStatement){
    return content;
  }

  var returnStart = returnStatement.range[0];
  var definitionStart = returnStatement.argument.range[0];
  var upToReturn = content.substring(0, returnStart);
  var afterReturn = content.substring(definitionStart, content.length);

  return upToReturn + 'module.exports = ' + afterReturn;
};

It's not exactly pretty, but it works. I can attest to that after running this thing on a pretty significant AMD project.

To install and run against your project:

> npm install -g amd-to-common`
> amd-to-common path/to/scripts

You can get the full source and read about the project over at GitHub