Data Wrangling on the Client with Javascript

Posted by Tom on 2011-10-24 21:20

Filtering, sorting and paging. These three operations are the bread and butter of data display. And it really is a display concern, so why not push it as far as possible into the presentation layer? When it comes to webapps that means the browser. So I wrote a data wrapper in Javascript as part of one of my side projects. It goes by the highly uninspired name of DataContainer, because I was having an unoriginal day.

Setting up

Everything is contained in the DataContainer object, and the constructor takes the data itself as an array of objects, and the number of objects on each page.

var data = [
    {
        color: "red",
        value: "#f00",
        prime: true,
        rand: 1
    },
    {
        color: "green",
        value: "#0f0",
        prime: true,
        rand: 3
    },
    {
        color: "blue",
        value: "#00f",
        prime: true,
        rand: 1
    },
    {
        color: "cyan",
        value: "#0ff",
        prime: false,
        rand: 2
    },
    {
        color: "magenta",
        value: "#f0f",
        prime: false,
        rand: 1
    },
    {
        color: "yellow",
        value: "#ff0",
        prime: false,
        rand: 1
    },
    {
        color: "black",
        value: "#000",
        prime: false,
        rand: 3
    }
];

var pageSize = 10;
var dataSet = new DataContainer(data, pageSize);

As intimated above, there are three types of operations available in DataContainer - filtering, sorting and paging - and they are always performed in the following order.

1. Filtering
2. Sorting
3. Paging

A call to getData takes the original dataset and then applies the filters, performs a sort and then splits the result onto the correct page and returns it. The most practical way of fetching data is through the use of callbacks. Whenever any operations are performed on the DataContainer which result in the resulting dataset changing the DataContainer's update function is called. If you want to update UI elements whenever the data is changed (and what else would you be using it for?) then simple pass the DataContainer a new update method with your own code. I'll be covering this in more detail in the next post - this is more about how to use the various DataContainer methods available.

Controls

Here's a brief rundown of the methods available on each DataContainer object.

Filtering

dataSet.addFilter(property, value);
dataSet.removeFilter(property);

Filtering is done based on the properties of each object in the initial array. Given the above data, if you make the call dataSet.addFilter("primary", true) then subsequent calls to getData will return on only those objects with the primary property set to true. If the initial data contains complex objects, then you can drill down into the object hierarchy using dot notation, the same way you would in code.

You can add multiple filters, allowing you filter the data more specifically. Also of note is the getPropertyValues method. When you pass this the name of a property it will apply the current filters to the dataset and then build a list of values of that property. The intent here is make building a drill-down UI nice and easy.

Sorting

dataSet.toggleSortDir();
dataSet.setSortProperty(property);

You can only sort by one property at a time. If you call the setSortProperty method a subsequent time with the same property name it will toggle the sort between ascending and descending.

Paging

dataSet.pageUp();
dataSet.pageDown();
dataSet.setPage(pageNum);

All pretty self explanatory. The pages are zero-indexed. You can also use the maxPage method to get the index of the final page.

Yay

There's real advantages to doing this stuff in the browser.

• It means that your backend infrastructure can be swapped out easily. Wish to port from ASP.NET to Rails? It's not exactly free, but it's less work than it would be otherwise
• No more sorting, filtering and paging gunk in your controllers

However . . .

• Doesn't scale so good. For large datasets or crappy hosts you're going to kill the browser
• If you want a non-Javascript fallback you'll need to implement everything ye olde way anyway

This is going to get long and boring so I'll move the second half of this over to second post in a couple of weeks. The next post will show how to create a dynamic UI over the top of DataContainer using Javascript templating.

Random burbling follows:

In my professional life we need to target everything when it comes to browsers. Sales ride on it and you can't just clam the site up because users are a bunch of dirty, IE6-using, tech-ignorant peasants. While we can create sites that rely heavily on Javascript we still need a Javascript-free fallback, and so we end up doing the work twice, in two different languages and on two different sides of the client-server divide. Usually this means we shy away from too much Javascript due to our time budget. Since this was part of an intranet project it was nice to be able to get down and write some real JS code, safe in the knowledge that if someone isn't using a supporting browser I can just walk over to the other side of the office and slap them. That said I did a trawl through some sites recently and a good proportion of them in our target market sector (fashion), simply don't work without Javascript. Maybe the days of non-Javascript fallbacks are past. Man, that would be great.