There have been times (quite a few actually) when I have been looking at data in a report and what I was expecting is not what I was seeing. From my experience there are 2 possibilities to look at straight away as to what might be occurring.
These are:
1. Is my data being collected correctly?
2. Am I interpreting the data I am looking at correctly?
Let’s look at the first possible cause for the “unexpected” data.
Is my data being collected correctly?
This is an obvious place to start. If your data is not being collected correctly, then it certainly isn’t magically going to start displaying the results correctly in your reports. But how can you validate that your data is being collected correctly? Fortunately there are quite a few tools at your disposal that allow you to see what is being sent back to your analytics platform. Product specific tools such as DigitalPulse Debugger (which is also free to use) is great for tracking Adobe products such as Adobe’s Reports & Analytics (formerly SiteCatalyst) and Test & Target.
Another product that I use extensively is Web Analytics Solution Profiler (WASP), which is a Firefox and Chrome plugin. WASP allows you to view data being sent to different analytic tools such as Google Analytics and Reports & Analytics, as well as quite a few other analytics platforms. There is a small fee to purchase this plugin, but well worth the money in my opinion.
Both tools allow you to see the values of variables being set, as well as a range of other information, such as what report suite data is being sent to in Reports & Analytics.
Another tool that you might find useful, especially for mobile app data validation, is Charles. Charles allows you to view all of the HTTP and SSL / HTTPS traffic between your machine and the Internet, or setup an emulator to view data from a mobile app. This comes with a small price tag attached.
If the data isn’t being captured correctly, then it might be time to start looking at code to make sure it has been implemented correctly.
Am I interpreting the data I am looking at correctly?
Let’s assume that all the data is being collected correctly, but still the data in your reports just doesn’t make any sense. Interpreting the data incorrectly is an easy trap to fall into, it can be quite confusing and make absolutely no sense at all.
I’ll give 2 examples to highlight what I mean. If something isn’t making sense, it might be time to do a bit more research to make sure you fully understand what you are looking at.
Example 1 – Daily Engaged Users and Monthly Engaged Users on a mobile app
If you look at the 2 reports side by side and over the same period of time, the Daily Engaged Users report will have a larger number than the Monthly Engaged Users report.
Following is an example where the Daily Engaged Users is approximately 1 million and Monthly Engaged Users is approximately 130 000:
If you know how the counting for these 2 reports occurs then it all makes sense, if you don’t though then it is just plain confusing.
So how does the counting work?
For Daily Engaged Users the number for each day is accrued, then totalled for the date range selected. As an example let’s say there is 22 000 Daily Engaged Users per day for the month of July. When each day is totalled it comes to 682 000 for the month.
So if I used the app on the 6 July, it would count as 1 for the Daily Engaged Users for that day, and 1 for the Monthly Engaged Users. If I use the app again on July 8, the Daily Engaged Users would count 1 for that day, and increment to 2 for the month. The Monthly Engaged Users would not increment and would still be 1 as it only counts unique users for the month. So once you understand how the reports are counting, it all falls into place and makes sense.
Example 2 – Viewing data from a segment where the referring domain is facebook.com
This example also relates to data in Reports & Analytics.
The scenario follows:
A facebook referrals segment at the visit level is setup (i.e. include referring domain contains facebook.com) as we just want to see facebook.com as the referring domain.
Following is an example of the segment:
and then the Last Touch Channel Detail report was viewed. Please note the following is for example purposes only.
There were quite a few referrals coming back that did not have facebook.com in them though. Why would they still show up in this report as the segment created should only show facebook.com, right?
The key here is that the segment was created for visits. What is actually happening is that someone has come into the site from facebook.com, then exited the site and re-entered from notfacebook.com within the same visit. So there are 2 referring domains within the same visit for this segment showing (as long as one of the referring domains was facebook.com). It doesn’t matter if they came in a different order (i.e. entered from notfacebook.com first, exited then entered from facebook.com).
So if you just wanted the referring domain facebook.com showing you would create a segment with the same “include” as above, but you would add an “exclude” as well where for the visit, the referring domain did not contain facebook.com.
So the 2 key points to take away from this post are:
1. You can validate your data is being collected with various tools.
2. You need to really understand what you are looking at when you are trying to interpret your data.
