The last day of my course this year I decided to try a new experiment: the “course recap exercise”. I asked each student (individually) to create a new google doc file in our class folder and to answer questions about what they have learned during the course.
It occurred to me that if I have done my job right, students should be able to remember the most important ideas and principles presented during the course.
I also wanted the students to have an opportunity to actively reflect on what knowledge and skills the course provided to them, with the hope that in so doing they would cement them even further.
The exercise turned out to be incredibly instructive, both for me and for my students.
Here is what I asked precisely:
- Question 1: what are the top ten most important concepts/ideas you have learned in this course? I asked the students to avoid talking with their peers and looking into our study material but just try to recall concepts from their mind.
- Question 2: What is the most important idea/concept you have learned about each of the following topics (focus on 1 only): Data Abstraction, Fundamental Charts, Visual Encoding, Color and Other Channels, Visualization Guidelines, Interaction and Multiple Views, Spatial Data. In this second step I asked the students to recall one specific idea or concept for each of the major modules we worked on (no, we did not cover networks and trees and the world did not collapse). This second step ensured me they would not try to recall at least one concept from each of the main topics we covered.
- Question 3: What is the most important lesson you have learned in each of this type of activity: Projects, Data Analysis and Presentation, Chart Decomposition, Vis Design Workshops. In this final question I aimed at facilitating mental processing of lessons learned while doing practical work.
The results of this exercise went way better than I had imagined. I was really surprised by how many concepts, ideas and principles the students recalled. At the end of each question I also spent considerable time probing the students with more detailed questions trying to understand if they only understood the concepts superficially, parroting the things that I went repeating throughout the course, or they had internalized them in ways that allow them to reason productively. To my surprise, most of the probing went pretty well and confirmed me that understanding visualization concepts is not particularly hard.
On a negative note, however, all this knowledge does not translate into being particularly proficient in developing effective visualizations. As I will explain in future posts I did not see huge improvements in the way students designed and developed their projects. I keep seeing a big gap between visualization theory and practice.
On the value of nongraded assessments.
In any case, this little exercise was another opportunity for me to test a concept I started experimenting with this year: the idea that assessment and grading do not need to go together and that actually coupling them together can also be detrimental.
All students seemed to be pretty relaxed in answering the questions I posed and discussing their answers allowed us to discuss many details in much more depth.
I don’t know why it occurred to me to assign this exercise only at the end of the course but I believe the same structure can be used at regular intervals during the course to better understand what students are learning and how to fix potential gaps.