InfoVis Course Diary: Flipping My Class and Other Innovations

flipped-classroomLast week I started a new edition of “Information Visualization“, the course I give at NYU Tandon, my university. With this blog post I am starting a new series, similar to the one I did a couple of years back, in which I write about the course and ideas that originate from my experience while teaching it.

In this first post I want to talk about new ideas I am implementing, in the hope this will be inspiring and helpful to other instructors like me. Last summer, I spent quite some time reviewing education research and reading books on the topic (while preparing my NSF CAREER), and I felt I really needed to change a few things in the way I am teaching. Here are some of the more most prominent ones.

Flipped classroom model. For the first time this semester I will be using a flipped classroom model. In this model students read assigned material and watch my recorded lectures (which I had recorded in previous semesters) at home. Time in class is entirely spent on feedback, discussions and exercises.

Why do I do that? First, because throughout the years of teaching I learned that I am the most valuable when I am giving feedback to my students, not when I am lecturing. The get the best out of me when I show them what is a better way to solve a given problem. Second, because lectures allow (encourage?) students to be totally passive and to disengage (a weird kind of luxury given how much it costs to be an NYU student). When this is then compounded with how pervasive digital distractions are in today’s classroom, this is a perfect recipe for disaster. Finally, I am just bored of spending two and a half hours talking and seeing everyone progressively “fade away” (together with my voice).

Last week I gave my first “flipped class” and I believe it went very well. There are a few things I’ll need to fix, but overall it seems to work. It’s work in progress and I expect to learn a lot.

Each week I ask my students to answer a few questions about the readings/videos I assigned them to read/watched and I use their answers to provide feedback to everyone at the beginning of each class. Three benefits: (1) I now have time to review all students’ responses before going in class (as opposed to preparing the lecture); (2) I can cluster them into major themes/issues and use this information to improve the course; and (3) since I discuss this in front of the class, everyone can hear what I have to say and still reacts and benefit from it. Next week, I’ll assign the first hands-on exercise in class. I am hopeful this also will work just right.

Decoupling grading and assessment. This is another no-brainer I figured out only a few months ago: assessment and grading do not need to coincide. The main goal of assessment is not to assign a grade to each student, but: (1) to inform the instructor about how much and what students are learning and (2) to inform the students about what aspects of the assigned material they still need to work on. When you see it this way, it transforms completely the way you think about assignments and exercises. I am relieved by the burden of turning everything into a grade and at the same time I now see this as my little tool I use to decide what to talk about in class and tweak it.

In my new model, assessment and grading are decoupled. I use assessment to guide myself and the students, and grading, through a test later on to give a final grade to a given set of modules.

Of course, I still need to check that students are actually doing the work I ask them to do at home. For this reason all my assignments are mandatory and graded only on a pass/fail scale. My students know that their final grade will be affected only slightly by these assignments but at the same time they know they must submit and take it seriously. In short, it’s hard to fail and your grade is rarely affected, unless you do not take these assignments seriously (in which case you fail badly).

Promoting self-awareness and ownership. This is a hard one. One I am still struggling with. What I noticed in the way we teach, is that we give too little responsibility to students to “own” the burden of becoming knowledgeable in a given topic. While structure, syllabi, grading, etc., have their role, they also have a dark side: they allow students to just blindly follow whatever they are asked to do as automaton; without self-reflection and criticism. This is not a good model. Life out of school does not work this way. It does not matter how good your grades are, what really matters is learning about how to be responsible for your own education and empowerment. In the real world, nobody is feeding you with a spoon, you have to figure things out on your own (jobs where this happens tend to be crap by the way) and this is a way bigger lesson than actually learning the very content of the course.

So now … How do we encourage self-awareness and ownership? It’s hard. And I can’t say I have figured it all out. But I am carefully inserting little exercises and questions to encourage students to reflect more on what they are learning an why. For instance I would ask in the reading response things like: “why do you think this is important?” or “how are you going to apply this in the future?” or “how does this fit in your current and future knowledge?“. This is an area where I want to learn more. For now, I am just moving baby steps. If you have some ideas to share with me, I would be more than happy to learn!

Putting even more emphasis on “vis basics” and UI design. As a visualization designer and researcher, what I have learned throughout the years is that: (1) it is very rare for an “unconventional” chart to beat a conventional one; (2) it’s much more effective to learn how to use (and tweak) a few basic charts, then to explore a very large set of new fancy solutions as a first step; (3) you always need to start from the basics to realize they don’t work, and then change it. With this in mind, I am putting even more emphasis than usual in teaching students how to use well the most basic charts that cover fundamental combinations of 2 or 3 attributes (most charts out there don’t go past beyond 3).

Other than basic charts such as bar charts, heat maps, line charts etc., I’ll spend some time on maps because they are pervasive and are used in virtually every single data visualization job out there. Whereas I am growing way more doubtful about networks and trees. The heretic side of me is willing to heavily reduce the amount of time I’ll spend on them or even cut them out completely. I know many will harshly protest about this, but that’s the way I am thinking right now. Apologies.

A second aspect of my course is my focus on “analytical interfaces“: I teach students how to build interactive visual applications to help people make sense of data. I am teaching CSE students and I want them to acquire a special skill very few people have.

What I noticed in past editions of my course however is that students inevitably end up in a situation where they know how to design a single good chart/view, but have no idea whatsoever how to tie multiple charts and interactive components together in one complex UI. When you think about it, it’s astounding how little information and guidance exists out there on how to do this properly for analytical interfaces! This semester therefore I’ll put even more emphasis on this aspect and I’ll design new material to address this gap.

Asynchronous real-time communication tools to use in class. I started using Slack to communicate with my students inside and outside the classroom. I was fed up with the jurassic tools my university provides to communicate with students and I felt I needed a more direct connection. Slack is just perfect. Now I have a specific team for the course and receiving messages from students does not clutter up my (sacred) email inbox anymore.

But the most surprising advantage of Slack is that I can use it to communicate with my students in class! I know … It looks counterintuitive. But the thing is that in class, especially now that everyone is very active, there is often the need for me to: (1) send a link or document of some sort to everyone and (2) ask students to provide information to me asynchronously but in real-time. This second use is particularly useful. I can for instance ask a question and tell students to write an answer in Slack. This way they have time to work on it and I have time to process the results and produce feedback of some sort while they are working.

I am doing something similar with Google Docs: I assign an exercise to groups of students and ask them to produce the results in a Google Doc. As they do that, I can monitor what they are doing and intervene when necessary. For instance, I can identify issues with one group and help them directly, or sometime I can simply write a comment in the document and they’ll see it coming from me. I will experiment much more with these tools. So far, my experience is extremely positive.

Book suggestion. There is much more I have learned about new/better ways of teaching over the summer. I could go on forever. I just want to conclude suggesting an amazing book I read which inspired many of the changes I am making in the course. The book is “McKeachie’s Teaching Tips“. There are many good books around, but this one covers a lot in a small space. It’s a fantastic starting point to get the basics on a given aspect related to teaching. It provides an overview and lots of references you can then use to dig deeper if you want. It’s highly recommended.

This concludes my first entry for this semester’s course diary. I’ll write again as soon as I have more to say.

Hope you’ll enjoy it!

Paper: How Deceptive Are Deceptive Visualizations?

Screen Shot 2015-02-25 at 2.33.35 PM

We all know by now that visualization, thanks to its amazing communication powers, can be used to communicate effectively and persuasively massages that stick into people’s mind. This same power, however, can also be used to mislead and misinform people very effectively! When techniques like non-zero baselines, scaling by area (quadratic change to represent linear changes), bad color maps, etc., are used, it is very easy to communicate the wrong message to your readers (being that done on purpose or for lack of better knowledge). But, how easy is it?

Continue reading

What do we do this VIS thing for? Towards a data visualization ethos.

I often find myself asking: “What do we do this Data Visualization thing for?”. Of course I do it mostly because it’s fun, and I bet it’s the same for you. Yet, is there a way we can find some deeper meaning in it? Are there some higher level purposes we can identify? Meaning often comes in relation to impact one can have on other people’s lives, so here is a tentative list off the top of my head of how vis can impact people’s lives (feel free to add yours in the comments below). Continue reading

Book: Statistics as Principled Argument

stats-principledI just started reading Statistics as Principled Argument and I could not resist to start writing something about it because, simply stated, it’s awesome.

The reason why I am so excited is because this is probably the first stats book I found that focusses exclusively on the narrative and rhetorical side of statistics.

Abelson makes explicit what most people don’t seem to see, or be willing to admit: it does not matter how rigorous your data collection and analysis is (and by the way it’s very hard to be rigorous in the first place), every conclusion you draw out of data is is full of rhetoric. Continue reading

How do you evaluate communication-oriented visualization?

Had a fantastic visit at ProPublica yesterday (thanks Alberto for inviting me and Scott for having me, you have an awesome team!) and we discussed about lots of interesting things at the intersection of data visualization, literacy, statistics, journalism, etc. But there is one thing that really caught my attention. Lena very patiently (thanks Lena!) showed me some of the nice visualizations she created and then asked:

How do you evaluate visualization?

How do you know if you have done things right?

Heck! This is the kind of question I should be able to answer. I did have some suggestion for her, yet I realize there are no established methodologies. This comes as a bit of a surprise to me as I have been organizing the BELIV Workshop on Visualization Evaluation for a long time and I have been running user studies myself for quite some time now. Continue reading

Mechanics for the Formula 1 of Science

I could not resist writing this short blog post after having a such a nice conversation with Scott Davidoff yesterday. Scott is a manager at the Human Interfaces Group at NASA JPL and he leads a group of people that takes care of big data problems at NASA (I mean big big data as those coming from telescopes and missions).

While on the phone he said:

You know Enrico … the way I see it is that we are mechanics for scientists … the same way Formula 1 has mechanics for their cars“.

What a brilliant metaphor! Irresistible. It matches perfectly my philosophy and at the same time, sorry to say, I think it does not match very well with the way most people see vis right now. Continue reading

Data Visualization or Data Interaction?

… or whatever we want to call it.

Yin Shanyang writes on twitter in response to my last post on vis as bidirectional channel:

Screen Shot 2014-05-08 at 11.18.17 AM

This comment really hits a nerve on me as I have been thinking about this issue quite a lot lately. I must confess I am no longer satisfied with the word “visualization”. And I am even less satisfied by all the other paraphernalia people like to use: data visualization, interactive visualization, information visualization, visual analytics, infographics, etc.

The reason is that I think all these words do not describe well the work I and many other people do. While visualization seems to be appropriate when the main purpose is data presentation, I don’t think it captures the value of visualization when it is used as a data sensemaking tool.

When used for this purpose interaction is crucial. Analysis looks more like a continuous loop between these steps:

  1. specify to the computer what you want to see and how (the specific visual representation)
  2. detect patterns, interpret the results and generate questions
  3. ask the computer to change the data and/or the visualization to accommodate the new question(s)
  4. assess the results … repeat …

Analytical discourse is a term I saw used in the visual analytics agenda a few years back and I think it captures very well this concept. This all interplay and discourse between the machine and the human. This is what many of us are after and I am not sure the term visualization is able to express this concept in its entirety. The value of these tools is not exclusively in the visual representation; interaction plays a major role.

This became even more apparent to me while teaching my InfoVis course this semester. I teach a lot of things about visual representation but when students come down to building software for their projects, what they are really working on is a fully-fledged user interface. They have multiple linked views, search boxes, dynamic query sliders and all the rest. It’s interactive user interface design they end up doing, not visualization. And user interface design carries a lot of additional challenges that go beyond visual representation. Sure, designing the appropriate representation is still very important but many other choices impact the final results.

For instance all my students’ projects have multiple interactive views, maybe sometime just a main visualization, a list of terms and a couple of query sliders for dynamic filtering, but how do you call that? I call that visualization but in practice it’s a complex user interface. Or a “data interface” as suggested by Yin.

One last note. While thinking about this whole idea I recalled that Jeff Heer‘s lab at UW is called Interactive Data Lab and I think he’s got it right. Interaction with the data is the main thing, visualization is the medium we use to create part of this interaction.

What do you think? Too heretic? To much of a hassle?

Visualization as a bidirectional channel

I am preparing a presentation for a talk I am giving next week and I have a slide I always use at the beginning that asks this question:

How do we get information from the computer into our heads?

This works as a motivation to introduce the idea that regardless the data crunching power we are going to produce in the future the real bottleneck, in many applications, will always be the human mind. Getting information across from what our computers accumulate and generate to our heads and being able to understand it is the real challenge. Visualization is the tool we use to deal with this problem. By using effective visual representations of data we tap into the power of the human brain with all its incredible powers we have not been able yet to reproduce and synthesize in a machine (I let the discussion of whether this is possible or even desirable to others).

When I present this slide I normally quote the great Fred Brooks’  The Computer Scientist as Toolsmith and add this image from the paper:

Screen Shot 2014-05-06 at 8.00.27 PM

But today for the first time I realized that when we talk about visualization we always talk about it as a one way channel, from the computer (or other media) to the human, when in fact there is a lot of knowledge flowing from the human to the machine.

When we use an interactive visualization tool we decide which data segments we want to attend to (think how Tableau works). This is derived from our knowledge and questions which we implicitly use to make choices about what to visualize next and how. When we use dynamic queries we use our knowledge to tell the computer that we are interested in a specific segment of the data and that we want to see it now.

There is a simple but effective function in Tableau that I love and is a good example of what I am trying to say here: the “exclude” function, which allows you to remove a data item from the visualization completely because not interesting or just annoying. When we do that, we are transferring our specific knowledge to the computer to tell it that we don’t need to see that data point anymore.

All in all it seems to boil down to interaction and how it is the only way to translate our intentions into instructions our computers can interpret. I think that what I really want to say is that we tend to forget how powerful this channel is and how limited it is to think about visualization exclusively as a 1-way communication tool. Sure, we can keep considering visualization this way but I think it’s much more exciting to think about it as a “visual thinking tool” where information flows in both directions.

And I think there is even more than that. While interaction in visualization is currently limited to giving instructions about what to see next, nothing prevents interaction to be used as a tool to transfer pieces of human knowledge directly to the computer. Classic examples where this has been attempted in the area of machine learning and related fields are relevance feedback mechanisms and active learning. Both technique rest on the idea of asking a human how to judge a decision made by the computer and use the result as a way to improve the computation. This is only one example but I think there are many unexplored ways to input our knowledge back into the computer to make it smarter and I think visualization should play a much larger role there.

That’s all for now. Thoughts?

My (stupid) fear we may, one day, become irrelevant

[Be warned: this is me in a somewhat depressive state after the deep stress I have endured by submitting too many papers at VIS’14 yesterday. I hope you will forgive me. In reality I could not be more excited about what I am doing and what WE are doing as a community. Yet, I feel the urge to share this with you. I will probably regret it in a few days :)]

I happen to click on one of the last links in one of the popular visualization blogs. I am excited. The title looks cool, the data looks cool and the design of the visualization looks super cool: sleek and clean, the way I like it. I give a look at the demo and you know what? There’s nothing there to see. Empty. No new knowledge, nothing to learn, nothing you can absorb. Nada.

This is not an isolated case. And that’s the reason why I am not happy to disclose which particular project I am talking about. First, because it would not be fair (I hate throwing shit at people). Second, because, as I said, this is not an isolated case. Third, because this particular project is only an expedient to talk about something much larger.

The way I see visualization is as a super powerful discovery tool. Stealing words to Fred Brooks, visualization for me is, ultimately, an intelligence amplification” tool: interactive user interfaces to observe the unobservable (or think the unthinkable?).

But many many visualizations out there show nothing. They are like modern food: empty calories. We, as a community, spent and still spend lots of energy debating whether one particular way of representing a given piece of information is better than another but we seem to forget that what is really important is what we decide to show in the first place. Ultimately, the yardstick should be: did you learn something watching this? Is there any kind of nutrient that enters your brain?

Let’s put it this way: if it was possible to observe exactly what kind of changes happen in the brain of a person when exposed to some new piece of information, through visualization, what would you like to see there? I would like to see a Pollock-like explosion of spreading activation followed by a difference. A delta. A sweet and tiny new brick of knowledge.

I see too much ambiguity out there. We talk about telling stories, about beautiful visualizations, and we talk a lot about wrong ways to visualize data. But what I would like to talk more is about: are we making a difference? Not a difference in the market or on twitter or whatever. A difference in people’s mind. In their brain actually.

I think the answer is mostly yes. I think … I believe … Or I like to believe. But sometime I fear we are not. The biggest fear I have, and this is the real sense of this post, is that if we will not be able to teach people how to create nutritious visualizations we may become irrelevant. Maybe it’s just a stupid thought, I don’t know, but that’s the way I feel when I get depressed by watching empty calories visualization (btw, maybe this should have been the real title of this post). The allure of pretty picture one day will end and I am not sure what will be left to see.

Creating visualizations to change people’s brain significantly is not an easy task but it’s also the only thing that really excites me about visualization  [Added note: Alberto and Gregor in the comments pointed out there is no way NOT to change your brain anyway when you are exposed to a visualization. They are right. So this is more of a colorful image than a good representation of what happens in reality. Yet, I like the concept anyway. Just don’t take to literally!]. And now that I think about it, maybe I am writing this post more for myself than for you. I want to remind myself that my ultimate goal is to help people do remarkable things with visualization. It’s so easy to forget it in the day-to-day. I want to be able to literally change those neurons and synapses and make a difference in people’s brain. That’s what counts for me. Isn’t that a more than worthy and magnificent goal?

And what is your goal by the way?


Take care,
Enrico.

Course Diary #3: Beyond Charts: Dynamic Visualization

This is the last lecture of the introductory part of my course where I give a very broad (and admittedly shallow) overview of some key visualization concepts I hope will stick in my students’ head. After talking about basic charts and high-information graphics I introduce dynamic visualization as visual representations that can change through user interaction.

Here are the lecture slides: Beyond Charts: Dynamic Visualization.

That’s the magic of computer graphics! The visual representation can respond and change according to our actions. Isn’t that great? Yes it is, but what is it for? This is what I asked to my students at the beginning of this class. I ask because I have the impression interaction in many visualizations comes as an afterthought: let’s put a little bit of hovering there and a nice animated zoom there. But interaction is an integral part of the well-reasoned choices a designers has to make in order to make a visualization effective, it’s not just an additional layer one can add there to add a couple of cool functions.

Interaction is the element of a visualization design that allows people to reason about data and that’s the way I presented it in class. It’s only through interaction that you can smoothly go through a long series of loops of: (1) detect something interesting in the data; (2) trigger a question; (3) change the representation in order to answer that question. Here is the (almost embarrassingly simplified) diagram I have used:

Interaction in visualization

Interaction is basically about reasoning with data though many of these intricate loops, not making it cool. Even though admittedly interaction does make visualization cool. But I guess you want to go past beyond the coolness factor, right? That’s almost too easy to achieve.

Next, I introduce Donald Norman’s 7 stages of action. The model describes the stages humans go through when they interact with the world to achieve a specific goal. Here is a sketch of the model:

Screen Shot 2014-03-06 at 11.09.45 PM

The model has been designed to describe things as simple as opening a door or turning on the volume of you speakers but it works equally well with complex user interfaces. The pedagogical value of the model in my opinion is that it make explicit the fact that interactive visualization is a lot about translation: (1) translating the goals we have in our head into actions and visual search tasks we perform with our hand and eyes and (2) translating (actually decoding and giving a meaning) to the changed visual representation we have in front of us after changing it through our actions. Our role as visualizations designers is to make these translations as smooth and natural as possible. Norman calls these critical points “gulf of execution” and “gulf of interpretation”. Easy and effective.

The comments I received after the lecture in our internal forum confirmed that the model does help students wrapping their head around the role of interaction in visualization so I am glad I included it. One student commented: “It is really interesting to see a process, which we all manage, unconsciously broken down to separate steps, where we can surprisingly easily relate those steps to our own experiences. ” Another one wrote: “I was really intrigued with Norman’s 7 Stages of Action. It seems like a really logical way to think holistically about interaction design.

During the rest of my lecture I described this paper: Yi, Ji Soo, et al. “Toward a deeper understanding of the role of interaction in information visualization.” Visualization and Computer Graphics, IEEE Transactions on 13.6 (2007): 1224-1231. This is a super useful paper if you want to learn more about the role of interaction in visualization. The thing I like the most about it is that it describes interaction techniques in terms if “intent” rather than how they are implemented. I like this approach because it abstract away from the technicalities of the technique and creates a more direct connection between interaction and reasoning. These are the categories:

Mark something as interesting (Select)
Show me something else (Explore)
Show me a different arrangement (Reconfigure)
Show me a different representation (Encode)
Show me more or less detail (Abstract/Elaborate)
Show me something conditionally (Filter)
Show me related items (Connect)

If you have never read this paper I suggest you to give it a look, it’s a very good read. Another very good read on the same topic is the more recent: Heer, Jeffrey, and Ben Shneiderman. “Interactive dynamics for visual analysis.” Queue 10.2 (2012): 30. That’s a very good one too.

One of my students in the forum raised a question about complexity: by introducing all this interaction don’t we risk to make visualization too hard to use and understand? Yes, I think there is a very high risk to make things too complex and more interaction does increase the need of users to learn how to use the system. It’s wise to adopt a parsimony principe when we talk about interaction in visualization. Cramming twenty different techniques in one system for the sake of it it’s not going to work. Interaction is a dangerous tool and it must be used with great care. The best is when it blends smoothly into the visual representation and makes important questions easy to answer.

Overall I think we still have to learn a lot about interaction. Most visualizations on the web are static, and most of the interactive ones are either not very well designed or very limited. While little interaction may be necessary for visual data presentations, more rich and well-integrated interaction is crucial for analytical reasoning. If we want to help people reason about data and derive useful insights we have to better understand how to support this complex process.

That’s all for now. Thanks for reading.