Graphing Ordinal Data
You shouldn’t analyze ordinal data like metric data. Should you graph ordinal data like metric data?
I am convinced that ordinal data—like Likert scale items commonly found in psychology or experimental philosophy—should not be analyzed using statistical models built for metric data. For example, Spearman’s rho is a more appropriate statistical test for assessing the correlation between responses to two Likert scales than Pearson’s r.
The best argument against analyzing ordinal data as if they were metric is that, most of the time, there is no theoretical reason to think that the points on a Likert scale are really spaced equally, even if they are represented to be. For example, there is no reason to assume the distance between, say, “agree” and “strongly agree” is the same as the distance between “strongly agree” and “very strongly agree”.
Liddell and Kruschke (2018) showed that even experienced researchers who might well recite textbook definitions still analyze ordinal data inappropriately. (“We surveyed all articles in the Journal of Personality and Social Psychology (JPSP), Psychological Science (PS), and the Journal of Experimental Psychology: General (JEP:G) that mentioned the term ‘Likert’, and found that 100% of the articles that analyzed ordinal data did so using a metric model.”)
My worry is that even researchers who know how to appropriately analyze ordinal data still visually represent them inappropriately. (Yes, I am definitely talking about my very recent past self.)
Many common visual representations for ordered data seem to be built for metric data, not ordinal data. For example, a standard scatterplot evenly spaces the numbers on the axes. So, if one were to use a scatterplot to visually inspect the relationship between two ordinal variables, one might be prone to misleading inferences.
And, unless one thinks of the visual representations as mere eye candy, this can be a serious problem for their uses in articles. Newman and Scholl (2012) found that people systematically misread bar graphs, such that they believe that “points that fall within the bar [are] more likely than points equidistant from the mean, but outside the bar”. I would not be shocked if similar perceptual biases existed for common visual representations of ordinal data.
For some visual representations, there exist good substitutes. For example, this R Bloggers post from 2012 recommends using, say, a diverged stacked bar instead of, say, a violin plot. But for others, such as a scatterplot (“the workhorse of data visualization in social science”), I cannot think of any good substitutes. That seems like a problem.
UPDATE: Thanks to the online community, I found out that Lauren Ackerman has written an amazing tutorial on visualizing ordinal data in terms of probabilities. Check it out!
I am an Associate Professor of Philosophy at University of Puget Sound. My academic website is liao.shen-yi.org.
