Charts: OECD Education at a Glance

by Enrico on October 29, 2008

in Experiments

Inspired by some recent government interventions on the Italian public school and the consequent large development of protests all around the country I have designed few charts to see if I can better understand the issue from the data and communicate some results.

Background

I have used the data from the Organization for Economic Co-operation and Development (OECD), which is often referred to as one of the main trusted authority for whatever concerns the education systems of a country. More precisely the data comes from the OECD report: “Education at a Glance“.
At the origin of the protest there is the reduction of the number of main teachers per class from 3 to 1, with a consequent reduction of the public personnel. The government says that having less teachers will not influence the quality of the studies and that quite a lot of public money will be saved. The protesters believe that the opposite is true and that the savings should not come from these cuts.
The goal of these charts is not to provide a solution to the debate, rather it is a very small and focused view on the problem. I just tried to find some hints on two related questions that came to my mind:

  • How efficiently does the Italian system spend its money?
  • Is proportion of students to teachers the cause of poor performance?

How efficiently does the Italian system spend its money?

The first chart replies to the first question. At least partially. The chart is a scatter plot of the OECD data on efficiency of school systems based on the following data:

Scientific performance: called PISA (Programme for International Student Assessment) and defined as “an international study conducted by the OECD which measures how well young adults, at age 15 and therefore approaching the end of compulsory schooling, are prepared to meet the challenges of today’s knowledge societies.” It is supposed to be a good indication of how well our schools do.
Expenditure per Student: It is defined as the equivalent US dollars expended per student.


chart1_s.png

I have drawn two lines to divide the space into 4 quadrants with respect to where Italy is placed. Of these quadrants I have highlighted the bottom right because it represents all countries who can perform better in terms of the PISA index and spend less. In other words all the countries in the quadrant not only are able to spend less but they also use this money more efficiently because they produce better students.
The sad truth is that my lovable country performs very bad. Greece and Portugal are valid companions but at least they spend less.
In oder to be sure that these results are not affected by the economic level of countries, I have also produced a second chart where the expenditure is normalized with respect to GDP (gross domestic product).

chart2_s.png

Unfortunately the result is even worse: Greece and Portugal perform worse but almost all the other countries are better. From the chart we can also see (in the bottom right) that Finland performs exceptionally well and that New Zealand, Netherlands and Australia performs very well too but spending less money.

Is proportion of students to teachers the cause of poor performance?

Since at the center of the debate there is the question of whether more or less teachers affect the quality of an education system, I created a bar chart comparing the ratio of students to teachers for the countries shown in the scatter plot.
Here are two bar charts, one for primary school and one for secondary school. Again I have highlighted Italy in the chart to make the comparison with it easy.

barchart1_s.png


barchart2_s.png

As you can see Italy has one of the lowest ratios both in primary and secondary school, meaning that there are quite a few students for each teacher or, in other word, that teachers are not very overloaded compared to other countries. The comparison with other countries is quite interesting. Finland, Netherlands and New Zealand (Australia is missing in the data) which are very efficient, as we have seen in the scatter plots above, have quite higher values compared to Italy. Can we say then that at the root of the poor Italian performance there is the number of teachers? Or can we say that a small number of students per teacher is necessary to produce a school of high quality? I don’t know … but at least the graphics instill some doubts.

Technical Notes

The charts have all been done with Excel. After all it is always the best and most readily available tool. There is always a bit of a hassle in doing certain things, especially the defaults are crazy (like strong dark backgrounds), but in the end it works great.
I have used the XY Chart Labeller to reduce label overlaps on the bar charts. This is also a bit cranky but in the end it does its job well.
The annotations on the charts have been done with the graphic tools in Excel and externally within SnagIt, which I use to screencapture the charts. Yes I’ve used screen capture! I know I could use VBScript stuff or similar things to save the charts into images but it’s always a kind of pain and less flexible than just press PrintScrn and edit the image.

Disclaimer

With these charts I don’t pretend to demonstrate anything, it’s more an interesting exercise for me to create data graphics and to show how easily we can reason about data that pertains to facts related to our social life.
The charts might show and evident bias towards judging the government interventions appropriate, but this is not my intent. Rather I would be very curious to see other charts that better clarify the issue and show with data and graphics arguments opposite to mine.

Final Reflection

In order to build these charts I have invested very very few time (I invested a lot more time to write this post though!). I was able in a few clicks to clarify to myself some things on an issue which is quite hot during these days in my home country and which I dare about. The same thing might be done by millions of citizens if only instructed appropriately. And that would mean having a population of informed people, able to ground their protests on hard data and to communicate their arguments with the vividness of well done data graphics.
Unfortunately this is very far to come. Simple techniques like these are never used by politicians or protesters, they prefer to use thousands and thousands of words in place of few well done charts. It’s a pity for us and it’s a pity for them.

  • http://charts.jorgecamoes.com Jorge Camoes

    Enrico, nice post. It is always interesting to see this discovery process at work.
    This is what I would do differently:
    I would remove Luxembourg from the analysis. This is a known outlier and we know why. Remove it and you’ll have a higher resolution chart.
    I would not use bar charts with the ratio of students. Instead, I would create three groups (low-average-high) and color-code the data points in the scatter plots.

  • http://www.joeparry.com/blog/ Joe Parry

    Hi Enrico,
    I agree, nice charts, but I’d like to suggest a couple of things.
    First, and this is really simple, I think scatterplots work best when the ‘potential cause’ is on the x-axis and the ‘potential effect’ is on the y. In your case this would put the education result (the goal) along the y-axis. This would probably be more intuitive for picking out the best results and the worst results.
    The other thing I’d say is that it might be worth plotting a scatter plot of teacher-pupil ratio on the x-axis with result on the y-axis. That way I’d be able to see if there is a correlation directly without having to switch between two representations (the bar chart and scatter plot in your posting).
    Good work though! Shame I didn’t bump into you at InfoVis this year…

  • Enrico Bertini

    Jorge, Joe, thanks so much for your comments and suggestions! They are very pertinent and much appreciated.
    I like the idea to get rid of bar charts and use the scatter plot to tell the whole story. I’ll try to implement a color coded scatter plot as soon as I can.
    Joe, you are also right with your cause-effect factors mapped respectively to the x-axis and y-axis. I like Jorge’s idea on color coding better than the one you suggest to see the correlation hwever.
    I want to point out that if the goal is to see if there is a real correlation, computing correlation coefficients would be a much more valid and robust approach than that (you can in fact find them in the OECD report somewhere). There is a subtle but relevant difference between what I am doing here and the work of a statistician or whoever. Here I don’t pretend to demonstrate anything at all, I just want to use these graphics to raise some questions. It is more a question raising device than and explanatory or confirmatory one. Which in turn is somewhat true for visualization in general to some extent.
    Joe, I could not attend InfoVis this year: no papers to present, no money to go :-( I really hope to meet you there next year.

  • http://dysplastic-brain.blogspot.com Bob Calder

    Enrico,
    If I recall the PISA data correctly, the causal conclusions track well with the results of the STEM study on science.
    If you plot the scores against the teacher degree – ie – does the teacher have a science degree or not – at least in high school it gives a good indication of PISA score. One of the things the PISA presenters made a big deal over was longitudinal data on low achieving populations within a country. In Canada, they track populations of immigrants by duration of residence as well as tracking native Inuit population.
    Finally, your blog reminded me that the data for my country at least are easy to misinterpret. The average class size in the US public school system is between 25 and 35 for science and math. The average is brought down by including low population courses. The expenditure per pupil in Florida is around 7,000 USD, but in New York, around 10,000. The private school Obama’s girls will attend charges 30,000 and the school Bush’s nieces attended charged 25,000 in Miami.
    At those schools, it is common for teachers to have doctoral level degrees in subject areas. But in public middle schools in the US, only around half have bachelor’s level science degrees.

  • Registered User

    Dear Bob,
    Thanks a lot for your clarification. It is very interesting indeed to know that students performance correlates with teachers’ degree!!! I would be very glad to have some pointers on the studies you mention. Do you maybe have a link?
    This make me thinks once again how we should take care of teachers and promote high quality. I don’t know well how it works in the US but in Italy this wonderful job called “teaching” has been mortified by crazy policies that I don’t have time to mention here.
    Regards.

Previous post:

Next post: