Node.js performance 2017: v7.9.0 vs. Hapi, Express.js, Restify and Koa and more

Alex OgierLanguages, Node.js, Web Development8 Comments

Node.js performance 2017

For the past two years (2015 and 2016), Raygun has tested the Node.js framework against other popular frameworks including Hapi, Express.js, Restify and Koa. This year (2017), we’ve added some more frameworks due to popular request; Sails.js and Adonis.js.

The aim of these performance tests is to help you benchmark popular frameworks so you can see which one best suits your project.

As always, we’ve broken the results down and compared them to last year. We’ve also included instructions on how to reproduce the test.

Node.js performance tests were performed on the Ubuntu subsystem on Windows 10, along with a VM provisioned from Digital Ocean. The tests only utilize the most basic capabilities of the frameworks in question, therefore the main goal was to show the relative overhead these frameworks add to the handling of a request. This is not a test of the absolute performance as this will vary greatly depending on the environment and network conditions. This test also doesn’t cover the utility each framework provides and how this enables complex applications to be built with them.

Here’s a quick recap of what we found in our previous Node.js performance tests in 2015 and 2016:

Node.js performance 2015

In 2015, we tested tested Hapi, ExpressJS, Restify and naked Node with no middleware. The results were clear:

Node.js performance 2017 - results from 2015

Node.js performance June 2016

This year, we added one addition framework, the results of which you can see below:

Node.js performance 2017 - results from June 2016

Node.js performance December 2016

It turned out that total.js was the fastest of the frameworks tested, being only 15% slower than using the raw node.js http library. The other frameworks, koa, restify and express all performed similarly and hapi performed the worst. For this update, we introduced a secondary test environment:

Node.js performance 2017 - results from December 2016

Node.js performance 2017: the test

The original reasoning behind these posts was an exploration of Node.js frameworks for use in the development of a lightweight public facing ingestion API for possible use in developing Raygun’s public API.

For this round of testing, as requested by our readers, we added two new frameworks (Sails.js and Adonis.js) to the test. We also used the updated versions of the previously tested frameworks which included a major version bump for Koa to Koa 2.0 and minor updates for some of the other frameworks.

As with the previous tests, I used Apache Bench to fire GET requests at the endpoint for each sample app. We configured the applications to respond with a simple “Hello World!” string. Note that this does’t necessarily correlate with any desired real world behavior but is rather testing the theoretical maximum number of requests which can be handled.

Apache Bench was configured to make 100 requests concurrently until 50,000 requests had completed or 20 seconds had elapsed. The request responses per second were then recorded.

Both Apache Bench and the server were hosted within the same environment removes any confounding factors network stability could introduce.

All tests were repeated five times in each environment. The displayed figures refer to the averaged values produced by the test.

The rough code used in the test is available on Github here. Depending on the environment additional packages or permissions may be required to run the tests.

Software versions tested

Runtime:

Node.js v7.9.0

Frameworks:

Hapi 16.1.1

Express 4.15.2

Restify 4.3.0

Koa 2.0.0

Total.js 2.4.0

Sails 0.12.13

Adonis 3.0.12

Test environments

  • The Ubuntu subsystem environment running on Windows 10 PC – 32 GB RAM, i7-4790 CPU
  • A $20 Digital Ocean VM running Ubuntu 16.04 – 2GB Memory, 2 Cores

The Results

Ubuntu Subsystem on Windows 10:

Express 1745 requests/second
Hapi 1094 r/s
Raw Node 2291 r/s
Restify 1759 r/s
Koa2 1887 r/s
Total 2144 r/s
Sails 1554 r/s
Adonis 2177 r/s

Ubuntu VM (Digital Ocean):

Express 2875 r/s
Hapi 688 r/s
Raw Node 5092 r/s
Restify 2380 r/s
Koa2 3317 r/s
Total 3924 r/s
Sails 772 r/s
Adonis 962 r/s

 

Node.js performance 2017 results

What these results mean for you

The results of this test shouldn’t be taken as an indication of the framework’s merits or flaws as a whole, as this test is focused on a simplistic operation. If you are going to write a high performance API, we recommend having some idea about the base performance overhead. This will better inform whether or not it is suitable.

What the performance on the different systems mean

As the test was performed on environments which aren’t similar to a professionally hosted service’s environment, the raw numbers here mean little. The windows subsystem is mainly useful for emulating nix-like functionality within Windows with ease and a $20 VM is very much at the lower end of the scale for any professional webservice. Because of this the main takeaway should be the relative performance of each framework.

Comparing with prior tests

The results of the tests were similar to previous runs. There was very little variation in performance between the VM performance of the last test and the most recent one. This make sense as Node.js has not undergone any major updates since the last test was run. However, we wanted to monitor the performance against other frameworks as they have had minor updates.

New frameworks:

Both of the new frameworks were relatively opinionated and dependency heavy frameworks came with many out-of-the-box features. That they performed somewhat unfavorably and this micro-benchmark was not surprising. Both were leaning towards the creation of larger applications rather than simple API endpoints. Their performance was surprising good on the Subsystem environment. Having access to a larger amount of memory may have been a factor in both these frameworks performing better than on the relatively resource light Virtual Machine.

How to reproduce the test

These tests can be replicated by cloning this repository and running:

The first script should install all the required tooling to run the test as well as npm and the latest version of Node.js. The second script then runs all the servers and benchmarks in sequence outputting the results to a file. We highly recommended reviewing the scripts to ensure it will not interfere with any other processes on your machine before running it.

Overall

Overall these tests showed that Total.js remains the fastest framework with performance within 25% of a raw Node.js service. The large amount of built-in features  might make them unsuitable for use of an API project without significant modifications to their default behavior.

Related articles

Node.js for Visual Studio: what to expect

C# performance tips and tricks

Next level software intelligence across your entire stack. Get deeper analysis into how your applications are really performing. Learn more.

8 Comments on “Node.js performance 2017: v7.9.0 vs. Hapi, Express.js, Restify and Koa and more”

  1. Pingback: Node.js performance showdown v7.2.1 - a framework comparison

    1. Alex Ogier

      This prevents the connection from remaining open over many requests which is the default behavior for some cases. Closing the connection levels the playing field and makes the test closer to real world conditions.

  2. Trident

    Of course, it all depends of what you want with your setup. A more complete test is the TechEmpower test.

  3. Pingback: How to develop software ideas

Leave a Reply

Your email address will not be published. Required fields are marked *