“You can’t manage what you can’t measure,” they say. What if you measure something, but eventually can’t manage it to achieve customer satisfaction? Let’s talk about the reliability of KPIs in IT service management.
Remember the last time you gave a taxi driver 5 stars, even though they were kind of a jerk? We often do this to save time or avoid causing trouble. This is an example of the watermelon effect: the rating looks good (a 5-star review), but the customer experience is not truly reflected (the rating should be red). Congratulations, you’ve just given the company management a false impression of your satisfaction level!
This is how it looks in IT service management:
The inconsistency is dangerous because it hides issues that can lead to major disasters. Let’s learn how to prevent this effect and recognize if it’s already impacting your team.
Table of contents:
The watermelon effect refers to a phenomenon where performance indicators appear favorable on the surface but reveal issues when examined more closely.
This term is often used to describe a situation where external measurements, like service level agreements (SLA) or performance reports, show a “green” status, indicating everything is fine, while the reality is “red,” indicating problems and dissatisfaction. Similarly, a watermelon is green on the outside but red on the inside.
While the effect’s name is primarily associated with IT service providers, it is relevant to any field where management relies on reports to evaluate the performance of their employees.
In IT, this term is frequently used in areas like IT outsourcing, where service providers may report achieving SLAs on all fronts to their clients, even though the actual work or end-user satisfaction is poor.
Often, you don’t need special knowledge to recognize that your numbers are misleading; negative feedback from end-users can be a clear sign. However, here are some less obvious signs that IT support might not be delivering results to users despite green numbers:
People aren’t perfect, and KPIs aren’t perfect either. The fact that dashboards don’t show the real picture isn’t always a problem.
For example, in a small team, even if there is a formal service request procedure, the service requester and the technician might reach a verbal agreement and not create a ticket. Since they work on the same floor and are both too busy, they might skip filling out another form. As a result, the technician’s manager won’t see the details of this case in the reports.
If the manager knows that the numbers he sees don’t cover every case and doesn’t mind it, then the mismatch isn’t a problem.
However, this effect becomes a problem when managers rely on measurements that don’t reflect reality to inform their decisions. The watermelon effect might then prevent management from addressing real problems, misguide decision-making, and leave the company unprepared for emergencies.
Alloy’s comprehensive IT service management and IT asset management solution allow you to measure the performance of your IT support team and track key metrics and trends in your IT processes.
Connect with our sales team and start using Alloy Navigator in a free trial!
Here are some reasons why the mismatch between reported performance and the real performance might happen.
Since people naturally tend to highlight positive aspects and hide negative information, special effort is needed to discover and monitor areas that present risks.
If you’re a manager reviewing reports, assume you’re seeing the best metrics while failures are hidden. If you’re an employee preparing dashboards, intentionally focus on weaker parts of your processes.
Here are some practical tips for focusing on gray zones:
Fostering transparency involves listening to employees. Technicians should feel encouraged to share both positive and negative results, knowing they won’t face backlash.
The key advice to those fighting the consequences of the watermelon effect is to shift focus to the end-users’ real expectations, requirements, and wishes.
One of the tools is the so-called XLA—experience service agreement. XLA is a framework that helps service providers deliver better end-user experience by collecting user experience data.
Within SLAs, the most popular metrics are service uptime, availability, and response time. XLAs, on the other hand, prioritize metrics such as customer satisfaction and overall service perception and use qualitative data, like survey results and user interviews.
XLAs help the service provider understand what the user feels when using the service.
As always, there’s no ideal solution.
Shifting to an experience-centered approach like XLAs comes with its own limitations.
Start your trial with Alloy Software today
