Tools from the Pros #3: Jan Willem Tulp on D3 and Protovis

by Enrico on October 13, 2011

in Guides,Interviews

Jan Willem Tulp When I saw for the first time a visualization developed by Jan, the Ghost Counties, I was totally fascinated. It’s brilliant. It took me a while to understand how it works, but once I got it I could not help but admiring the strange mix of complexity and simplicity it provides.

Despite he looks so serious in this picture on the left, he has a big smile and he is fun. I met him for the first time at Visualizing Europe and since then we exchanged many emails. Plus, he is a regular commenter here (and everywhere) and I love him for that.

I don’t know how much I have to add to convince you his advice is a valuable one. Just give a look to his portfolio and judge yourself. He is IMO one of the most interesting data visualization freelancers recently appeared on the scene.

I know, by talking with him, he is proficient with several technologies but he has a passion for D3.

How did you start using Protovis/D3?

I’ve always been someone interested in the latest technologies. So, since I follow the data viz community very closely, I was aware of Protovis very early on, and I was aware of the development of D3 even before it was released to the public. I have a software development background, so I don’t have too much trouble finding my way in new programming languages, and since it excites me to work with new technologies and frameworks, I just started playing with Protovis and D3 as soon as it became available.

What’s the best and worst aspect of Protovis/D3?

The best aspect of Protovis is that it is a domain specific declarative language, which means that is fairly easy to start writing code, using visualization related keywords and functions. The best aspects of D3 is it’s flexibility (more direct integration with SVG) and better performance. The worst aspect of both D3 and Protovis is that it’s hard or impossible to get it working on older browsers, and the learning curve for D3 may be somewhat harder than for Protovis.

Ok, I am a beginner and I want to learn Protovis/D3, where do I start?

I think Protovis is easier to start with than D3, but Protovis is no longer under development. I also see that the Protovis mailing list is not very active anymore, while the D3 mailing list is very active. But, I guess Protovis would be a very good way to start if you’re a beginner. Basic Javascript programming skills would be recommended, both for Protovis and D3. For some great Protovis tutorials you should check out Jerome’s blog: http://www.jeromecukier.net/blog/category/protovis/page/2/ (part 1 – 5: working with data in protovis). Also, the Protovis website has quite some examples that are pretty good. For D3, documentation, examples and tutorials are still under development: http://mbostock.github.com/d3/api/ so, with D3 you’re more on your own right now (and of course the mailing list). But things are improving rapidly.

How is the learning curve vs. return-on-investment of Protovis/D3?

Protovis is really a good way to learn visualization and programming at the same time. Protovis is a language that is really geared towards the visualization (and diagrams) domain, so it really makes sense to talk about axis, marks, lines, bars, pies, etc. Also, there are really quite some good examples on the Protovis website, so it’s fairly easy to get started. However, right now Protovis is not supported anymore, and people are really moving to D3 now, so getting support may become a little tricky. Also, Protovis does not perform as well as D3 with very complex graphs for instance, and also, compared to D3, in Protovis you’re a little bit limited to the animations you can achieve. So, overall, a good way to start and also good to make some nice standard diagrams and visualizations, but if you really want to do ‘heavy’ visualization stuff, you might consider moving on to D3.

D3 is more powerful, more flexible and seems to have more capabilities (and better performance) than Protovis. The flip side is that in order to have a more flexible programming language, the language is also more abtract. Though many concepts of Protovis are also implemented in D3, and there are also quite some predefined visualization layouts, it’s also more useful (compared to Protovis) to gain some knowledge of SVG, since it’s more likely that you might do some low level stuff. D3 does give you much better animation capabilities, better performance, more flexibility, so, once you get the hang of D3 and some SVG, you’re able to create some very compelling interactive visualizations.

What other tools would you recommend other than Protovis/D3?

The tools you mention are some of the best right now. I also think that Raphaël is a fairly good alternative if you want to do Javascript-based visualization that works in older browsers as well. Personally I don’t have much experience with Raphaël yet. Also Processing now has an Android mode, which is great if you want to create visualizations that run on Android phones, and the upcoming Processing 2.0 also has a Javascript mode, so you can easily create HTML5 canvas based visualizations with the Processing development tool.

A recommendation I’d like to add: when I work on Protovis or D3 visualizations, I use TextMate on the Mac. This allows you to open a preview window which renders your visualization near real-time when you are typing in your code. I’m sure that are similar tools that do this. This is really great for getting immediate feedback while you’re coding.

  • http://vallandingham.me Jim V

    Great interview ! I was wondering who the D3 interviewee would be.

    With Raphaël 2.0 out now, and the processing javascript output (plus paper.js and others), do you think one particular js framework will become the ‘leader of the pack’, or is there enough differentiation between these tools – and room in the visualization world so that they will all be popular for different groups?

    One thing about these interviews that I’ve been thinking about is that it might be a good idea to talk about the exploratory vs explanation capabilities of each of these tools.

    For example, D3 is wonderful for presenting a final explanatory visualization to be displayed on the web. However, I haven’t really gotten into the swing of using it for exploratory-style visualizations – when you are just trying to get a sense of what the data holds.

    It would seem to me that tools like R (and perhaps Tableau) are better suited for exploratory tasks. Would you agree? Or do you use D3/Protovis as a general tool for both exploring and explaining?

    Also, nice work on the WEF report! http://reports.weforum.org/global-agenda-survey-2011/#overlay/interactive-slides/top-trends

    It looks like you might have used Raphaël here, is that correct? Care to comment on the decisions that went into which tool to use for this visualization?

  • http://www.janwillemtulp.com/ Jan Willem Tulp

    Jim, thanks for your comment!

    I think that most frameworks have the potential to have their own place. For some there is overlap, but differences are mainly on technology (SVG or WebGL or Canvas), but also browser compatibility (will it run on the iPad, run on IE, or older versions of IE of just Chrome, mobile devices), performance (some are faster than others), and of course framework language which may be some personal preference as well (is it very verbose or not, clean syntax), and user base / community / support is also important (can I get help if I can’t figure it out, is the framework actively being developed on). So, in general I think all of them could have their place, but there are multiple factors that determine how successful a framework will be.

    I think all of these tools can be used as exploratory visualization as well, but compared to Tableau for instance, you do have to program all of it yourself, so often for basic analysis a solution like Tableau is often faster. However, quite often the final visualization you’re creating is something that may not be possible in a tool like Tableau (take a network diagram). So, even though you can do some preliminary analysis in Tableau, you’ll always run into some other insights, or things that do or don’t work. The process of creating a visualization is by itself a very exploratory and iterative process (which is also something I really enjoy about it). But I am curious how you see the difference between exploratory and explanation, as there seems to be some overlap in these terms…

    With regards to the WEF report, I’d like to quote Moritz which he has commented on Google+: “The client demanded for iPad as well as IE>=6 compatibility, so we used raphael.js for all the graphics, as it seems to be the only framework for browser-based interactive graphics which fully supports IE<9. The only exception is the network visualization on the last slide, which we decided to do in d3, as it was added later in the process and the d3 force layout works just great. It might be possible to render the d3 network with rafael. (Probably at low fps though – Rafael is not terribly fast when it comes to animations). Underscore proved very useful for array and data manipulations (grouping, sorting, filtering, mapping)."

  • Pingback: Interview on Fell In Love With Data | TULP interactive

  • http://vallandingham.me Jim V

    Thanks very much for the response.
    Certainly, as you say, there is overlap between exploration and explanation and you are always exploring, in a sense, till the visualization is done.

    I guess I was thinking of exploratory visualizations as those used to get an initial handle on your data – when starting from square one. From “Designing Data Visualizations”:

    “Exploratory data visualizations are appropriate when you have a whole bunch of data and you’re not sure what’s in it.”

    When starting from scratch with a data set, and you want to get a feel about that data, this is where I think tools like R could be better suited. In R, I can just say “plot(dataset)” and immediately get a matrix of small multiples of scatterplots comparing each dimension of the data. In D3, as you say, you would have to program it all yourself.

    However, after you get a sense of what variables, etc to display, this is where tools like D3 really start to shine – you can easily control every aspect of your visualization in a way that would be cumbersome – or even not possible with tools like Tableau and R. (Not to mention the interactive capabilities of D3).

    thanks again!

  • datagraphers

    Thanks Enrico for this post!
    To the points rightfully raised by Jim, I would concur with the idea that R is a very efficient to get a sense of what the data look like in the exploratory. However, as mentioned R has very poor capabilities, at the moment, to generate dynamic and/or interactive graphics. That’s where D3 has enormous potential I think.
    Other big differences are:
    1- R can generate static graphics in various format (png, pdf, etc…) which is very interesting as you can insert these plots in any documents you want. D3 (or Raphael or any JS-based programming environment) run on internet browsers ‘only’ – please correct me if I’m wrong here – which is not always an advantage.
    2- To create a visual with D3 or R, you develop a script, which AFAIK is not the case for Tableau (or Spotfire). In some case, having a script is not needed, but in some others, it is a necessity, eg, in highly regulated environment (say aeronautic, automobile, pharmaceutical industries for instance).
    Regards/DG

  • Pingback: D3 (Data-Driven Documents) | Craig's Musings

Previous post:

Next post: