Node.js is a great technology that gives a lot of functionality out of the box, especially if you need to interact with a filesystem. See just how easy it is build an API to serve up content on disk with this open source Node project.

Using Node.js to Mock an API

I was recently working on a product that had a complicated set of external dependencies which weren’t practical to run locally. This made development dependent on an external server. And if this server was unavailable for any reason, development was halted. Less than ideal, for sure.

I began looking into options for stubbing out these services. We didn’t actually need the function they were performing. We just needed to be able to send a request to an endpoint and receive a response back. A lightweight stub of these services would be more than enough.

The challenge was finding a solution that would be flexible enough to manage the complex routes that this application needed while still being simple to set up and maintain.

While doing some research on this, I came across mockserver, an open source project that makes it easy to stub out APIs.

The basic idea of mockserver is to parse the http request, convert the URL to a file path, then search on disk to find a match. If a match is found, parse the file on disk and return the response, otherwise return a 404.

Technical Limitations

This library was great for a basic API, even including support for wildcards to emulate resource IDs ( /user/45678/ ). The problem that we ran into though was that the resource IDs had to be at the end of the URL. A URL like ( /user/45678/edit ) would fail.

This was a pretty important feature to be able to test the application we were working on and we hadn’t found anything else that was close to giving us the functionality we needed. So I decided to dig into the project to understand how it was currently supporting wildcards in the directory path and what it would take to extend this to support the functionality we needed.

I ended up doing a pull request to implement this feature. You can check that out here then read on below for more of an explanation of how it actually works.

How it Actually Works

Node.js gives you a server out of the box with http.createServer. It’s pretty handy. You just need to provide a function that accepts a request and response object. This function gets called when the server receives a request.

Note: In this code snippet and most of the others in this post, some of the code has been simplified to keep from getting too bogged down. The full source is available on github at https://github.com/namshi/mockserver

// nodeserver.js
http.createServer(mockserver(mocks, verbose))

// mockserver.js

module.exports = function(directory, silent) {
    mockserver.init(directory, silent);

    return mockserver.handle;
}

When Mockserver starts, it creates the Node server and passes the handle function exported from the mockserver module as the requestListener.

mockserver.handle is a function that accepts a request and response object, which is passed by the node server when the server receives an http request.

var mockserver = {
  ....
handle:  function(req, res) {
....
}
  ....
}

The implementation details don’t matter right now. We’ll get to that in a few minutes. For now, just take note of how everything is wired up.

We create the Node HTTP server, passing in a function that accepts two arguments - the request and the response.

Once we have the request, we can grab the URL and compare that against the file structure on disk. If there’s a match, we serve up the content. Otherwise, return a 404.

This happens in a function called getMockedContent

function getMockedContent(path, prefix, body, query) {
    var mockName =  prefix + '.mock';
    var mockFile = join(mockserver.directory, path, mockName);
    var content;

    try {
        content = fs.readFileSync(mockFile, {encoding: 'utf8'});
    } catch(err) {
        content = (body || query) && getMockedContent(path, prefix);
    }

    return content;
}

getMockedContent expects a path (the URL from the HTTP request), prefix (the REST method), and the body or query parameters from the request.

For example, if a GET request was sent to /posts/, mockName would be set to GET.mock, mockFile would be set to C:\...\posts\GET.mock, and Node would try to read the file at that path and return its content.

Back in our main handle function, we see this

if(matched.content) {
            var mock = parse(matched.content, join(mockserver.directory, path, matched.prefix));
            res.writeHead(mock.status, mock.headers);

            return res.end(mock.body);
        } else {
            res.writeHead(404);
            res.end('Not Mocked');
        }

If we were able to find a mock at that directory location, return it’s content. Otherwise, return a 404 with the message ‘Not Mocked’;

So far so good?

Everything is fairly straightforward when we’re looking for a path on disk that matches a URL path. It gets a little more complex when wildcards come into the mix.

Since wildcards can be anywhere in the URL, there’s no longer a one-to-one relationship between the URL and the directory structure. We now need to traverse the directory tree to determine if an appropriate match can be found.

If you want to jump ahead, take a look at my pull request here where I implemented this flexible wildcard structure.

Starting from where we left off above, if the URL doesn’t match a path on disk, we need to do more work to determine if there are any wildcard matches.

matched = getContentFromPermutations(path, method, body, query, permutations.slice(0));

if(!matched.content && (path = getWildcardPath(path))) {
            matched = getContentFromPermutations(path, method, body, query, permutations.slice(0));
        }

If we didn’t find a match on the first pass, try to reassign the path variable based on a wildcard match.

function getWildcardPath(dir) {

....

    var res = getDirectoriesRecursive(mockserver.directory).filter(dir => {
            var directories = dir.split(path.sep)
            return directories.indexOf('__') >= 0
        }).sort((a, b) => {
            var aLength = a.split(path.sep)
            var bLength = b.split(path.sep)

            if (aLength == bLength)
                return 0

            // Order from longest file path to shortest.
            return aLength > bLength ? -1 : 1
        }).map(dir => {
            var steps = dir.split(path.sep)
            var baseDir = mockserver.directory.split(path.sep)
            steps.splice(0, baseDir.length)
            return steps.join(path.sep)
        })

    steps = removeBlanks(dir.split('/'))
    var length = steps.length
    for (var index = 0; index < length; index++) {
        var dupeSteps = removeBlanks(dir.split('/'))
        dupeSteps[index] = '__'
        var testPath = dupeSteps.join(path.sep)
        var matchFound =  res.indexOf(testPath) >= 0;
        if (matchFound) {
            newPath = testPath
            break
        }
    }

    return newPath;
}

There’s a lot going on there, so let’s walk through it one step at a time.

The first thing we do is retrieve the entire directory tree. Check the source if you’re curious about the implementation of getDirectoriesRecursive but suffice it to say it’s a function that recurses through the directory tree and returns an array of every path under the root directory.

We then take that array and filter it to only include the paths which contain our wildcard character, the double underscore ‘__’.

Those results are then sorted from longest to shortest. Because in the event of a URL matching multiple disk paths, we want to match on the most specific first.

Finally, we do a little cleanup to remove the root directory and reconstruct the file paths. This should result in an array of all directory paths that contain a wildcard value. (e.g. user/__/edit)

The last step is to loop through this filtered array and test those paths against the URL originally passed in. If we find a match, that path is returned and assigned to the path variable. From there, we can read the mocked file from disk and return its contents!

if(!matched.content && (path = getWildcardPath(path))) {
            matched = getContentFromPermutations(path, method, body, query, permutations.slice(0));
        }

So in just a few lines of code we can fire up a server and serve files from disk. And, with a small amount of effort, we can add some extra pizazz like allowing for wildcards in URLs.

Like I mentioned above, these little code snippets don’t give all of the details so be sure to check out the github repo if you’re interested in how all of the pieces fit together!