Data Literacy is often thought of as the ability to read meaningful information from data. Job descriptions for analytics roles emphasise the importance of mathematics and statistics skills to interpret and evaluate data. But just as literacy, in general, refers to the ability to both read and write, what about the ability to write/communicate data as well as read/interpret it?
Web analytics provides an analyst with a wealth of data which can be used for user experience (UX) improvements. Funnel analysis and a/b testing shows that a conversion rate can be greatly increased if the user is provided the right information at the right time. However, the very same analysts that draw these conclusions exclude vital information from the reports they create to communicate their findings.
It’s easy to find the piece of information you were searching for and then move on without considering how to lead others through the journey you took to get there. As an analyst, I’ve both fallen into and witnessed some familiar traps when trying to structure my analysis in a way that helps the user effectively draw conclusions:
Not explaining how your aggregations are calculated
This example was shown to me at a Tableau conference I went to recently. Say you’re a School Superintendent who has three schools in your district. Two of the schools are small and perform well but there’s one larger school that has a lower average grade. Someone asks you what the average grade is for your area. To make yourself look good you’d be tempted to take the average grade for each school and divide by 3 which would be 690.
While this isn’t technically wrong, if you presented it without explaining how you calculated it, a perfectly data literate person might interpret it incorrectly. They’d likely think it was the average grade per student in your district, which is actually 501.
| Gina’s Secondary School | Ally’s High School | Marty’s School | |
|---|---|---|---|
| Students | 75 | 100 | 1000 |
| Average Grade | 790 | 800 | 450 |
The mistake was caused by poor data literacy, but not because of how it was read, because of how it was communicated. Okay, someone with strong analytics skills might ask if it was the average per student, but as analysts we should do all we can to prevent this kind of mistake happening.
One way to do this is to avoid the term average where possible. When your creating aggregation, even if it is an average, be as explicit as you can. In the above example that might mean saying “Grade Points per Student” or “Grade Points per Student per School”, in the case of campaign analysis it might be labelling a field “Signups per Visit” or “Signups per Application Start” rather than “Signup Conversion Rate”.
Visualisation tools like Adobe Workspace and Tableau also allow you to add text to your visualisations to explain to users how you worked out your calculations. Tableau also allows you to create Stories as another way to explain your thinking when you created the visualisation.
Cherry Picking Data
An easy mistake to make as an analyst is trying to find data that fits your hypothesis. In economics it might be to say that UK economic growth is going great as we’ve seen growth in 23 of the last 32 quarters – forget that the months that saw decline meant that there was a contraction overall!
In digital analytics to might mean using a starting point further up the funnel than you should to make the conversion rate look better.
In general, the more stages you can add in the funnel towards a conversion the better. The exception is that multiple fallout fields on the same page might not work since you can’t account for customers who fill in the fields in reverse order, god knows why!
Just because the conversion rate is lower when using a starting point further up the funnel doesn’t mean your form/workflow is performing badly. One of our clients found that their conversion rate was much lower if they included visitors who landed on the form page but didn’t start it. This insight led them to the discovery that existing account holders were going to that page because they couldn’t work out how to log in. They were able to fix a UX issue they wouldn’t have known about otherwise.
Choosing the right visualisation
There have been plenty of times I’ve thought my point was very clear but still found myself with confused faces looking back at me during a presentation. Here’s a few tips that might help you avoid that awkward scenario. This is a topic that’s been covered in great depth by some experts and is too big for the scope of this post, so I’ll just be giving a couple of tips about how to keep your visualisations simple.
Don’t give TMI
Look at the below pie chart? It sucks doesn’t it… There’s too much information for a pie chart. Forget about getting anything quickly, you’d have to be there with a measuring tape to get any insights at all.
Checking whether your visualisation can answer basic questions of the data is a way to avoid this. In this case it might be to ask, for example, “Which year saw the highest sales?” or “How much more sales were there in 2011 compared to 2010?”. Neither can be answered quickly by the visualisation, in fact the raw data in a spreadsheet might actually be an improvement.
Do use consistent scales
Again, in our example pie chart, if someone did look at it for more than a few seconds, they might notice that 2018 has been broken down into quarters while the rest of the periods are yearly, this gives the impression that those periods did poorly compared to the rest when in fact it’s just that they were shorter.
Another common example of this mistake is to use bar charts where the Y axis doesn’t start at zero, or to have a dual axis with different scales. It’s not like these are rules set in stone, but if you’re going to have a Y axis start at 50 then communicate it clearly. Likewise, if we’re going to use a different scale for your second axis then make sure it’s not for the same metric.
% Change YOY is an example of a good place to use a second axis
Conclusion
The way we communicate our findings is key to achieving our goals as Analysts, but as we focus on making sense of data, we often forget about how we’re going to communicate it later. When we consider communication only after we have our findings it becomes an afterthought, the little eureka moments we have along the way get forgotten about.
The Dunning-Kruger Effect shows that people who are relatively new to a topic have a tendency to overestimate their own skill, while those with more experience tend to underestimate their knowledge compared to others. Keeping our visualisations more basic than we think they should be might actually mean they will be at the right level for the audience.
