For data analyst freelance services contact me [email protected]
Introduction
Almost every data analyst at some point faces this question from managers – what are the reasons for the decline in KPI metrics?
The question has several variations: why is there a 7% decrease in sales volume? Why did the rate of churned customers increase? Why did the conversion rate of my campaigns decrease significantly?
In the next post, I will describe the things that a data analyst can investigate to identify the source of the declines and to answer such questions.
Note – usually, the question is directed to the data analyst when there is a negative change in the data, but the same analysis can be done in the case of positive changes in the KPI’s.
Segments analysis
Metrics (or KPIs) are aggregated data, and it is not possible to learn from them what is happening within the data. Therefore, the first thing a data analyst should do is to break down and segment the data in order to examine in which segment the decline occurred. Below is a possible list of segments to pay attention to:
- User characteristics – If users are characterized in a certain way, these characteristics can be used. For example, did the decrease occur in premium customers? Does the decrease occur only in new customers?
- Geographic segments – Does the decrease occur in users from a certain country or city?
- Marketing channels – Did the decrease occur in users that came from Google or Facebook campaigns? Or maybe there is a campaign that brought in bad users that caused a decrease in conversion rate?
Decline due to AB testing
In the A/B testing methodology, product managers test a new version of the product on a small number of users. The test results indicate whether the new version shows improvement in the metrics that they measure.
The problem with these tests is that the changes may actually have a negative impact on the metrics that the managers are testing. In this case, the group of users who were tested will affect the overall results and a decrease in metrics may be seen.
Another problem with A/B tests is that they can improve certain metrics but at the same time, they may harm other metrics that were not tested, which may lead to a decrease in these KPI’s.
Changes in the user interface (UI)
Sometimes a change in the user interface can confuse users and prevent them from performing actions that are measured by metrics. For example, users who can’t find the purchase button that has moved up.
For these reasons, it is recommended to perform A/B testing before UI changes. However, even when a successful A/B test has been conducted, there may still be a situation where users do not behave as expected after the change. Humans are unpredictable creatures, and despite all our efforts, it is difficult to predict what they will do.
Technical failures
Sometimes, downtimes are not caused by user behavior but by technical malfunctions.
Server outage – We all remember the day when Facebook, WhatsApp, and Instagram went down due to a technical malfunction. Technical malfunctions happen and affect user activity, which can cause declines in metrics.
Incorrect data in DWH – There are cases (very annoying ones) where the product works great, but the data in the DWH is recorded incorrectly. This can happen because one of the ETL processes did not work well or because there was a malfunction in the DWH system. In stable systems, these cases are rare, but if no other reason is found for declines in metrics, it is always worth suspecting technical failures as the source of changes.
Seasonality
There are situations in which the reason for a decrease in KPIs is due to different usage patterns of users during certain periods. For example, in the summer, more vacations are booked compared to the winter. This phenomenon is called seasonality, and in order to understand whether it is the cause of a decrease in KPIs, the analyst must be familiar with the business world in which the company operates.
The effects of social phenomena on product usage
The activity of product users is not disconnected from events happening in the world, and therefore a data analyst must always be aware of local and global social trends that may affect users. For example, the Russia-Ukraine war could lead to a decrease in demand for products originating from Russian companies.
There may also be a situation in which there is negative sentiment against the company’s activity – for example, companies involved in gambling may experience a decline in certain countries if there is a campaign against gambling. Sometimes the sentiment can be directed against a specific company – for example, consumer boycott of a particular company because it raises prices.
This article was written by Yuval Marnin.
If you have a need to hire a freelancer data analyst you may contact me at: [email protected]
You may also hire me through upwork platform on that link:
https://www.upwork.com/freelancers/~018940225ce48244f0\
Further reading
The advantage of hiring a freelance data analyst.
What does a data analyst is doing and how it can help your company.
