Realtime database applications with Node.JS and HTML5 WebSockets

As you could’ve read in my first post, I’m doing the Honours programme at the college I’m at. This basically means that I get extra (more challenging) work to do next to my standard college assignments.

The first assignment we got came from Thialf Heerenveen, an ice skating stadium which is also used for national and international ice skating competitions. Thialf is using the Local Position Measurement (shorthand: LPM) system which was developed by InMotio. The system is basically some sort of GPS system, but MUCH more accurate. When ice skaters are training, they get a tracker, and as they do their training, data about their training is collected. The downside about this approach was that the coaches of these ice skaters could only see the data visualised after the training was complete, which made it still handy, but made immediate feedback impossible. Also the data they were given was mostly not visual enough and required thorough analysis before you could draw conclusions.

This is where we came in, our task was to develop applications for the major tablet brands. The applications should make it possible for the coaches to instantly, realtime, see how their pupils were performing and could immediately give them feedback based on the data they were presented with.

Seems like an easy enough task right? Making a few applications for Android, iOS and BlackBerry. That’s what we though initially, but nothing could be further away from the truth. This was because we initially thought we would have instant access to the data-stream InMotio was using to get their data from. But it appeared that this data would be inserted into a database, which in turn we had access to. Oh, and did I mention this application had to be realtime? Well, anyone who has ever worked with a database before knows that realtime data retrieval from a database is NOT an easy task. Oh, and the database we had to use was a MS SQL database. This was something I wasn’t really happy about in the beginning (because I had no knowledge of MS SQL) but it turned out to be a good choice, but I’ll get to that.

To sum things up, these were the problems we were faced with:

  • Realtime data access in a database environment.
  • Application development in multiple, major, mobile operating systems.

Realtime data access in a database environment

So, what is realtime data access? And how does this apply to the web environment we’re using? Wikipedia defines it as follows:

The real-time web is a set of technologies and practices that enable users to receive information as soon as it is published by its authors, rather than requiring that they or their software check a source periodically for updates.

To make that a bit more clear, let me take an example. When you’re using services such as Twitter or Facebook and someone posts a new update it does NOT automagically, and more importantly IMMEDIATELY, appear on your screen. Your browser polls the web server, asking it if it has any more updates available (in this case tweets/status updates). Most obviously this method has a lot of overhead, because sometimes there aren’t any updates and the browser still has to ask for them. And not to forget the overhead it costs to build up and tear down an HTTP request. This is where Node.JS comes in.

Node.JS is, as Wikipedia describes it:

Node.js is a software system designed for writing scalable internet applications, notably web servers.[1] Programs are written in JavaScript, using event-drivenasynchronous I/O to minimize overhead and maximize scalability.[2] Node.js consists of Google’s V8 JavaScript engine plus several built-in libraries.

In this project we used Node.JS to communicate from the database to our clients, somewhat as a middleman, passing on the messages it receives to the correct clients. But before I show you any code I’d like to talk about the MS SQL database and it’s CLR awesomeness.

The MS SQL database has some awesome functionality such as the possibility to add pieces of C# code to execute when a specified event has fired. In our instance there was a specific table in our database which we wanted the contents to be displayed as soon as it was inserted in the database. So we wrote a little C# program (a CLR-trigger) which ran every time the specific table was updated. This program set up a socket to the Node.JS server and passed to it any data which was inserted into the table. This way the database communication worked, according to our specification, realtime!

Once the database obstacle was blown away by the CLR-trigger awesomeness it was time to get the communication going between the Node.JS server and the clients. Keeping in mind that we wanted the platform to be realtime we decided to go with HTML5 WebSockets. This is a new technique used to get the statelessness out of HTML/HTTP. To make sure that there was a form of backward compatibility, a.k.a. the IE browser should work with it, we went with Socket.IO, a framework for Node.JS which makes it possible to fallback to different methods such as forever iFrames, polling, flashsockets, etc. if there is no WebSocket functionality available (which is the case in some mobile OS’s) Let me show you some code to clarify what’s going on here

 * @author Ruben Homs <>
 * @date 27-12-11
 * @version 1.0
// Require net for listening to the CLR-trigger.
var net = require('net');
    socket.on('data', function(data)

This piece of codes makes a raw TCP-server, waiting for messages from the database (which are in the form of JSON objects). Nothing real fancy, but that’s enough to get the database- Node.JS part working. Next up is a piece of code about the Node.JS – client communication. I can’t copy and paste all of our working code because of legal reasons but I’ll try to let you get the idea.
// Require as abstraction layer and listen on port 8080.
var io = require('').listen(8080);

// Require http for making requests to our REST-service.
var http = require('http');

// This array contains all the authenticated clients which are connected to the server.
var authenticatedClients = new Array();

// When someone connects request authentication.
io.sockets.on('connection', function (socket) {
    // Ask for authentication.

    // When a response is received, match it with the server to ensure authentic clients.
    socket.on('authRes', function(data) {
        // get the data and subscribe to the correct channel.

    socket.on('disconnect', function() {

 * Subscribes a user to the channel it belongs to. This is done by getting the user_info variable
 * which was received on connecting and linking it to a socket.
 * @param jsonObject The jsonObject received after the get on the webservice.
 * @param socketId The socketId of the user, used to send data to the correct socket.
function subscribe(jsonObject, socketId)
    authenticatedClients.push([jsonObject.user_info, socketId]);
 * This method takes a received value and publishes it to the socket which is associated
 * with the array value in authenticatedClients.
 * @param playerId The playerId received.
 * @param teamId The teamId received.
function publish(playerId, teamId, dataPackage)
    //iterate over the clients and send them the data packages.

 * This function deletes all values from the authenticatedClients array which have a given socketId associated
 * with it.
 * @param socketId The socketId to search for.
function deleteArrayValue(socketId)
    //Delete the array value which is associated with the given socketId

We’re using a subscribe/publish model. A user connects, we let him authenticate with our servers, if all is fine we subscribe him to the channels he’s interested in. Whenever data is received from the database we publish it to all the connections which are subscribed to that specific channel.

For a complete overview of the communication see this document.

Application development in multiple mobile OS’s

For this specific problem we found a framework called Phonegap. Phonegap, as it name already implies, bridges the gap between mobile OS applications and web applications. It acts as a wrapper for your HTML/JS/CSS documents. It crams it all in a neat little application so that it can be run as a native application on the platform at hand. It also let’s you use native smart phone functionality such as the camera from within JavaScript, how cool is that!?

So, needless to say, we used jQuery mobile and built ourselves a nice little website within a few days, wrapped it in an application and installed it on the tablets. There still is some work to be done in connection to the views (visualisation of the data), which is not that much work. We already made two views within a matter of hours, so further expansion should not be a problem as far as I’m concerned.

Behold the beauty of it!

Thialf live system(click to get a better picture)

It’s too bad that the picture is static and that I’m simply too lazy to make a GIF/video showing it in action but what I can do is describe it to you. The dot you see on the right side is the ice skater on the track (in this case Sven Kramer), and the graph on the right is the speed/time(in ms) plot. It all runs very smoothly without any lag whatsoever.

What I learned from this project is quite a lot, and it was very fun to do. For instance I learned that there are some awesome, new, and very exciting technologies out there which can solve problems very elegantly. I had the world of fun with WebSockets in combination with Node.JS and will use them whenever I can from now on. Man, how I love WebSockets.


  1. First I would like to thank you for sharing this article with the world. As you may know, I am a man of code. I love the Node.JS snippets, but could you also share some of the database-trigger code?
    I am a real Microsoft database fan. I consider the MsSQL as my favorite database systems.

  2. Eric says:

    Very cool – I like this case study of how Node.js/HTML5 are applicable to such a specific data set. Readers at DZone would really appreciate this – mind if I republish? I’ll include a link back to the original and attribution to you as the original author.

Leave a Reply