The hidden legacy of Bertin and “The Semiology of Graphics”

by Enrico on December 11, 2010

in Guides

The Semiology of Graphics (SOG) is a kind mythical book. Everybody knows the title, but few actually know its content. I already suspected it since a long time but this was confirmed by Jean-Daniel Fekete in my interview in my last post. But this is not because people are lazy, it is more, I suspect, that it has always been hard to find a copy, especially in English.

And I am also guilty. Apart from reading the short abstract contained in Readings in Information Visualization, I never had the book in my hands until few weeks ago (a French copy of 1967! thanks to the wonderful library at the Univ. of Konstanz).

As I said, part of the problem resides in its limited availability but, as some of you might already know, Amazon is promising since some weeks to have the new English edition out around December. So, no more excuses. By the book and read it. (Update: the book is no longer listed in Amazon and I have no ideas whatsoever what happened to it I am sorry).

So, if Bertin is such a wealth of hidden information what can we do now? I plan to show more about Bertin’s work in the future. In the meantime I want to re-propose large sections of the slides Jean-Daniel presented at VisWeek in his tribute to Bertin as a way to give at least an initial impression of how big the legacy left by Bertin is. His work influenced, directly or indirectly, almost all the future developments of visualization starting from the foundations.

What is really surprising to my eyes is how many things we have been re-inventing when Bertin actually presented them in full and clear details in SOG. This stuff was written in the ’60! No computers with affordable graphic displays were around at that time. This was all done by hand and the results are stunning.

Retinal Properties

Retinal Properties

Retinal properties as defined by J. Bertin

Bertin identified early in his work that every visualization is made by a series of basic components that have different expressive power and that each one works best only in some conditions. He suggested 6 basic variables: size, value, texture, color, orientation, shape and for each one he pointed out in what cases they work best and how to use them.

This same idea was expanded and refined by new foundational work after 20 or 30 years. William Cleveland run experiments on basic retinal properties in the ’80s trying to rank them according to their effectiveness in carrying quantitative information1.

Lately, Jock Mackinlay in his PhD thesis developing the APT (A Presentation Tool) system, applied the same principles to automatically construct visualizations out of data. Note that we experience the impact of this research up to our days as this is the basic research that inspired Tableau few years ago.

You can find the same ranking on visual features almost in all books about visualization or graphic design and this is largely considered at part of its foundations …. from Bertin straight up to our days.

Taxonomy of Networks

networks-taxonomy

Taxonomy of Networks

What is surprising of SOG is that not only it contains excellent examples of clever visualizations but you can recognize the intent of Bertin to systematize the whole design space. The example of his taxonomy of networks is stunning. By looking at the figure (click on it for a larger version) you can appreciate the sophisticate classification and especially the number of solutions that have been rediscovered lately!

You can recognize a  treemaps! Yes, Bertin thought about treemaps already in the ’60. You can recognize arc trees, 3D treemaps, adjacancy matrices. I’ll tell you more in another page of the book I could spot parallel coordinates too! But this will maybe be a later post.

Cyclic Patterns in a Spiral

time-spiral

Cyclic patterns in a spiral

Bertin proposed the initial design of presenting temporal data in a spiral as a way to spot cyclic trends. The same idea was re-proposed many years later by several researchers.

Notably by Carlis et al. in a UIST paper in 19982

And later by Dragicevic et al. in a CHI paper in 20023

The Reorderable Matrix

reorderable-matrix

Reorderable Matrix

The reorderable matrix is maybe one of the most mythical of his innovations. The basic matrix representation used to represent categorical data, adjacency matrices or multidimensional data like in heatmaps, needs to be reordered in order to show interesting patterns. Bertin was fascinated by this problem and built some mechanical devices to experiment with different reordering techniques. The device was called Domino and here you can see a pictures of it.

reorderable-matrix

Lately the same idea was reworked in a very large number of research endeavors. Especially in bioinformatics, where heatmaps are the standard tool for the analysis of genomic data, researchers and practitioners tried to devise clever and fast ways to reorder an heatmap in a way that relevant patterns become apparent.

heatmap

Final words

This is probably only a small fraction of the ideas that influenced and will influence people in the construction of clever visual solutions. I really encourage you to get a copy of SOG in your hands if you can and get inspired by it. I will certainly do it in the future.

Finally, let me thank Jean-Daniel Fekete for giving me his presentation. The whole material used in this post come from it and it’s invaluable. Thanks Jean-Daniel!

  1. Cleveland, W. S. and McGill, R. Graphical perception: The visual decoding of quantitative information on graphical displays of data. Journal of the Royal Statistical Society. 150(3):192–229, 1987. []
  2. Carlis, J. V. and Konstan, J. A. 1998. Interactive visualization of serial periodic data. In Proceedings of the 11th Annual ACM Symposium on User interface Software and Technology. UIST ’98. ACM, New York, NY, 29-38. []
  3. Dragicevic, P. and Huot, S. 2002. SpiraClock: a continuous and non-intrusive display for upcoming events. In CHI ’02 Extended Abstracts on Human Factors in Computing Systems.
    ACM, New York, NY, 604-605. []
  • http://www.janwillemtulp.com Jan Willem Tulp

    This is really a great post. I have never had the chance to have a copy of Bertin’s book in my hands (I’d love to though). It seems that Bertin has categorized many aspects of encoding and visualizing data. Do you have a suggestion on books or websites where similar categorizations are available?

    • Enrico

      @Jan: thanks for your comment! I need this kind of encouragement. If you want to know more about Bertin and SOG check out the interview with JD Fekete in the previous post. There are indeed some other categorizations. You might want to give a look to the first chapter of Readings in Information Visualization. Even though the book is a bit old the first chapter is still the best introduction to information visualization I can think of. From time to time I go back to it to refresh my mind. On a more technical level you can give a look to William Cleveland mighty books: The Elements of Graphing Data and Visualizing Data. Another excellent sources which is also a lot more accessible is Stephen Few’s Show Me the Numbers. Please let me know if that helps.

      @Barney hey thanks for the information! But a couple of friends of mine ordered the book from ESRI as well but it never arrived. There was the same option from Amazon until few days ago and then it disappeared. Do you know anyone who got the book or pre-ordered it and hard more info? Thanks.

  • http://barneycarroll.com Barney

    Hiya,

    ESRI books UK lists the English translation as being (re-)published on the 15th of December, and available for pre-order:

    http://www.eurospanbookstore.com/display.asp?K=9781589482616&st_01=semiology&sf_01=keyword&sort=sort_date/d&x=0&y=0&m=1&dc=21

  • http://barneycarroll.com Barney

    @Enrico
    I’m afraid not; I’m only going by what it says on their site — although the fact they acknowledge a future release date may indicate something… Depending on what happens on the 15th, I may try to get a copy myself…

    @Jan Willem
    Edward Tufte [1] is my personal Jesus when it comes to visual information displays. The Visual Display of Quantitative Data investigates what it says on the cover, and Beautiful Evidence gives an excellent overview of the qualities of representation and credibility in visual explanations. His other books are all excellent, and the titles reflect their area of focus: his work tends to run through an explanation of the principles at work governed by the task at hand; critical historical case studies of the excellent and the abominable (often with examples of improvement by his own hand) in the given field; and wonderfully-written, beautifully-presented conclusions as to how one should tackle information management and rendition. I happen to be lucky enough to have all of them, so feel free to ask me more about them on twitter @barneycarroll if you’re considering a purchase.

    [1] http://www.edwardtufte.com/tufte/books_vdqi

    • Enrico

      I love Tufte’s books too, but to tell you the truth their use in my case is different: they are more for inspiration than for guiding me in design. Bertin has great examples too but he also tries to “sell” a system to the reader. The same thing is not evident from Tufte … especially in his latest books (The Visual Dispaly of Quantitative Information is the only exception I can think of).

  • http://www.janwillemtulp.com Jan Willem Tulp

    @Barney @Enrico

    Thank you for your suggestions. I’ve been lucky to order an English copy of Bertin’s Semiology of Graphics on Amazon, and it is shipping now! So just a few more days, and I’ll be able to enjoy Bertin’s wonderful insights.

    I have all the Stephen Few Books, and most of Tufte’s books. They are very good indeed!

  • Carol Briam

    Thank you for your posts. I love Jacques Bertin’s reorderable matrix (but I’m not a computer scientist or hard-core techie). Is there an Excel add-in or other accessible software that can be used to create a reorderable matrix from a given set of data? I created a reorderable matrix on several occasions using a Word table and rearranging cells (time-consuming, but it works–did it for my PhD dissertation in business/technical communication). Surely, there must be something out there available for the average creator of business graphics.
    Thanks for any guidance you can offer.

    • Enrico

      Carol unfortunately I don’t have a clear pointer for you. You can ask to the best Excel Vis techies I know from the blog arena: Jorge Camoes with his marvelous http://www.excelcharts.com/blog/ or Jon Peltier at http://peltiertech.com/Excel/Charts/. If you cannot find anything from them (you can try to ask too) there are few chances there is anything at all around.

      But you might also want to give a look to the work of Jean-Daniel Fekete http://www.lri.fr/~fekete/. He is one of the major experts on the topic I know from research.

  • Carol Briam

    Thanks, Enrico, for the suggestions.

  • Brandon
  • Pingback: DJ31033/2: Info on Jacques Bertin | Shaleph O'Neill

Previous post:

Next post: