Category Archives: Teaching

InfoVis Course Diary: The Course Recap Exercise

The last day of my course this year I decided to try a new experiment: the “course recap exercise”. I asked each student (individually) to create a new google doc file in our class folder and to answer questions about what they have learned during the course.

It occurred to me that if I have done my job right, students should be able to remember the most important ideas and principles presented during the course.

I also wanted the students to have an opportunity to actively reflect on what knowledge and skills the course provided to them, with the hope that in so doing they would cement them even further.

The exercise turned out to be incredibly instructive, both for me and for my students.

Here is what I asked precisely:

  • Question 1: what are the top ten most important concepts/ideas you have learned in this course? I asked the students to avoid talking with their peers and looking into our study material but just try to recall concepts from their mind.
  • Question 2: What is the most important idea/concept you have learned about each of the following topics (focus on 1 only): Data Abstraction, Fundamental Charts, Visual Encoding, Color and Other Channels, Visualization Guidelines, Interaction and Multiple Views, Spatial Data. In this second step I asked the students to recall one specific idea or concept for each of the major modules we worked on (no, we did not cover networks and trees and the world did not collapse). This second step ensured me they would not try to recall at least one concept from each of the main topics we covered.
  • Question 3: What is the most important lesson you have learned in each of this type of activity: Projects, Data Analysis and Presentation, Chart Decomposition, Vis Design Workshops. In this final question I aimed at facilitating mental processing of lessons learned while doing practical work.

The results of this exercise went way better than I had imagined. I was really surprised by how many concepts, ideas and principles the students recalled. At the end of each question I also spent considerable time probing the students with more detailed questions trying to understand if they only understood the concepts superficially, parroting the things that I went repeating throughout the course, or they had internalized them in ways that allow them to reason productively. To my surprise, most of the probing went pretty well and confirmed me that understanding visualization concepts is not particularly hard.

On a negative note, however, all this knowledge does not translate into being particularly proficient in developing effective visualizations. As I will explain in future posts I did not see huge improvements in the way students designed and developed their projects. I keep seeing a big gap between visualization theory and practice.

On the value of nongraded assessments.

In any case, this little exercise was another opportunity for me to test a concept I started experimenting with this year: the idea that assessment and grading do not need to go together and that actually coupling them together can also be detrimental.

All students seemed to be pretty relaxed in answering the questions I posed and discussing their answers allowed us to discuss many details in much more depth.

I don’t know why it occurred to me to assign this exercise only at the end of the course but I believe the same structure can be used at regular intervals during the course to better understand what students are learning and how to fix potential gaps.

InfoVis Course Diary: Developing Visualization Design Workshops

During the last four/five weeks of the course I have been assigning visualization design exercises in class. The main idea is to assign practical design problems to students to solve in class during a workshop of about 2.5 hours. Here is a description of how I am organizing the workshops and what I have learned so far.

Philosophy

The reason why I decided to create and assign design exercise in class is because in past years I have always been frustrated with how little students learned by listening to my lectures and by observing how quickly they learned once we worked together on the design problems they had to solve for their group project. The philosophy behind the workshops is therefore to bring more of this experience in class, together with the advantage of having the whole class working on the same problem in a much more structured way (in group projects each team solves a different problem independently). Another advantage of workshops is that when I give feedback to one group of students all the other students can listen and relate what I am saying to their own solution.

Preparation (Reverse Engineering Vis Projects)

To prepare these exercises I decided to use the following strategy: reverse engineer existing visualization project I like. I start from an existing project and I effectively go from solution back to the original problem. Inspiration can come from many sources: papers published at IEEE VIS or ACM CHI; projects developed by some established visualization designer or newspaper; class projects developed during past editions of the course.

(Note: I recently discovered Shiqing He and Eytan Adar use the same strategy in their beautiful and admittedly much more advanced VizItCards method)

How do I do that? I focus on the following elements:

  • Problem Statement: What is the original problem they wanted to solve? Who has this problem? Why is it interesting and important?
  • Data Set: What data set did they use? Is the data set available? If not, can I use a similar one?
  • Questions: What are the driving questions they want to answer by looking at the visualization?

Execution

Once I get in class, I typically follow this sequence:

  1. I ask my students to form groups (basically the same every week).
  2. I read the whole text of the exercise and make sure everything is clear to everyone.
  3. I ask students to read everything on their own.
  4. I ask students to create individual solutions first.
  5. I ask students to discuss their solutions and create a group solution.
  6. Student put their solutions in a shared google doc, one for each team (I don’t care about possible cheating).
  7. I create a set of slides (on the fly) with students’ solutions and comment on them in front of the class.

What I noticed during these last few weeks is that the are many implementation details one needs and should work on to tune the execution of the workshop.

  • Group and individual work. I still have to find the right balance between individual and group work. But after some experimentation I believe both are highly needed.
  • Material and techniques for creating mockups. I am not particularly happy with the unstructured way I let students create their mock-ups. Some are very good, some are very bad. In the future I want to give more precise instructions and unify the way mockup are developed.
  • Using whiteboard and markers to create mockups. I discovered, almost by chance,  that using whiteboards is a much better way for groups to generate and discuss mockups. If you can afford having multiple whiteboards or, even better, you have the luxury of having “whiteboard walls” in your classroom you should try this out. The biggest difference is that with whiteboards everyone can see what is happening and participation and feedback happen much more naturally.
  • Deriving patterns and principles from the solutions. Ideally, at the end of each workshop we should be able to derive general principles students can apply to other cases, beyond the specifics of the exercise. I truly believe this is an important pedagogical step. I started collecting some of these ideas and patterns at the end of each workshop but I’d like to find a better and more systematic way to do it in the future. Ideally, these patterns and principles may be reused in future workshops as the material develops further.

Sharing workshop material

I plan to share all of the exercises I created for my course later on in 2017 and make it available for everyone to use. I just need to give it a more decent shape. I’d be more than happy to develop these exercises further together. Just stay tuned!

Developing a “Data Sensemaking” Course

As we approach the end of the Fall semester I start thinking about the new course I’ll have to teach in Spring. It turns out I decided to design and give a new course I am really excited about. It’s name is “Data Sensemaking” and I will be teaching my students how to derive knowledge out of data using interactive visualization and data processing tools. I am writing this blog posts to share some of the ideas I am “brewing” and hopefully to get some feedback from you to see if you can help me perfect my plans.

What is “Data Sensemaking”?

The main purpose of the course is to teach students how to use a variety of exploratory data analysis methods to extract information and hypotheses from data. Some people call this Exploratory Data Analysis (EDA), but I preferred to use Data Sensemaking because EDA carries a somewhat heavy baggage from years old statistical debates. I also like the name “data sensemaking” because it refers more to the outcome than the process.

Why a course on “Data Sensemaking?”

First, because after teaching Information Visualization for a number of years I realized that students lack the very basic knowledge needed to reason with data and this is what matters the most today. It does not matter how cool a given visualization tool or solution is, ultimately we need people to extract valuable and possibly actionable information out of data.

Second, because people are fixated with complex statistical models and machine learning (which are mostly applied blindly to data problems) when in fact what we need to perfect is our ability to reason with data. There is ready-made recipe for this and because of that I find teaching this topic fascinating and enriching. Our brain and rational thinking is the biggest asset we have and I am appalled at how little information and training programs exist to develop our ability to reason effectively with data.

Third, because I want to learn more myself and the best way to learn I know is to teach. While there are some skills in data sensemaking that I have been honing for several years, there are other I feel I need to perfect and develop much much further.

What will I teach in the course?

This is still work in progress but I have a tentative set of topics in mind I’d like to share with you to see what you think about it. Here is my current set of topics:

  1. Defining (worthwhile) data problems. When is a data problem a good problem? How does one go about defining its requirements and constraints?
  2. Asking good and effective questions. How can one systematically transform a foggy data problem into a set of well defined analytical questions? What is a good question?
  3. Finding, generating, and manipulating data. How does one find or generate data useful towards solving the stated problem? What kind of data manipulation and integration are needed? How does one know if the data are good enough?
  4. Exploring data to generate answers and questions. How does one explore data with interactive data visualizations to investigate the stated questions? How does one deal with the new questions/ideas inevitable arising during data analysis?
  5. Biases, lies and data malpractice. How does one avoid to be fooled by data and visualization? What are the major traps out there? What is the best mindset for “data investigators“?
  6. Creating effective data narratives. Once the analysis is ready, how do you organize and re-design the results to communicate them in a way that people are willing to listen/read and actually understand your message?

Major doubts/hurdles.

When looking at the list above there are a few elements I am still uncertain about:

  • What makes a data problem a good problem? How do I teach students what a good problem is? We are so much used to thinking about solutions and so little about problems. But, in the end generating worthwhile problems is the most important thing to do. Do you know of any resources on how to pursue and define good problems?
  • What statistical fallacies, biases, etc., should I focus on? There is an endless list of biases and fallacies humans are prone to commit. How do I prioritize them. What is the core set of concepts and examples I should focus on? How can I uniformly cover problems with thinking, statistical and visualization fallacies?
  • How do I teach the narrative side of visualization? I know very well how to teach which chart is “right” for a given communication task. But this is only a fraction of what one needs to learn to communicate effectively. The narrative style, the sequence and the interrelation between text and graphics is also very important. How do I teach this? Is there any book or resource I can use to cover these aspects?

What else?

What else should I teach? If you would be one of my students, what else would you like me to teach you? Note that I omitted the technical side of it but I will be teaching some practical tools to use. I plan to focus mostly on Rstat and Tableau. I also plan to have each student set up a blog in which they have to create a number of data sensemaking mini-projects.

Let me know what you think!!!