Why Employee Experience demands new forms of measurement
Over the last 2 years we’ve seen a big shift in HR conversations from a narrow focus on employee engagement to a wider one about employee experience. Whilst this change has been widely accepted the implication for what this means in terms of measurement hasn’t been considered in such detail.
More qualitative data is needed
Unlike employee engagement, employee experience isn’t grounded around such a formal, psychology-based concept. Understanding experience demands a much more exploratory-based approach which of course demands much more qualitatively based measurement.
When I first got involved in what would now be called ‘employee experience’ almost 15 years ago as a quantitive analyst I was initially surprised not only about how my colleagues in customer experience used a much-more qualitative based research approach but also how effective this qualitative data was at driving effective action.
A simple example were the user tests we’d conduct on the careers site and tool that we designed / implemented. I was advised by our researchers that large numbers of different individuals weren’t necessary to run an effective study. Of course we ensured we recruited well and tried to reduce bias but I was surprise at how quickly the results started to converge. In terms of identifying key areas to change we probably had found most before the 10th person.
Another thing that I rapidly realised was that the issues that we identified were rarely the ones which we had expected. We went in to the testing with a set of ideas to test, often building multiple versions to compare, then found that something completely unexpected needed fixing more urgently.
The data that we captured was typically much richer than a traditional quantitive approach. Quantitive analysis can be very good at identify what, when and how but it’s very poor at helping understanding why - i.e. the underlying causes. From the perspective of improving experience the critical part is to understand ‘why’.
Measurement and service design
Within service design, which we believe employee experience aligns closely with, an approach of going in with an open mind and asking simple, open questions will typically outperform analysing more structured, quantitive data.
We’re trying to understand the events - often called ‘moments that matter’ - that create the positive or negative experiences so that we can systematically improve these touch-points (or reinforce the good ones). Understanding experiences is inherently an exploratory approach. We need to start with an open-mind to understand the experiences from the users’ perspectives.
Traditional surveys are ineffective
Traditional surveys used in HR (and old-school customer surveys) take a constrained view of what could have happened and then offer each possibility as an option.
We all know not only how ineffective these can be and how frustrating. I guess everyone has seen a customer survey which asks question after question about an experience (eg a hotel stay) but fails to ask about the one event that you want to comment on.
Not only do these designs fail to capture the relevant information, they also provide a poor usability and make a poor experience feel worse.
Over the last 15 years these types of surveys have become less common as firms have transitioned to simpler feedback methods, usually with a small number of scale questions and the opportunity to provide open feedback. Think NPS-type surveys or online rating platforms such as Expedia or Glassdoor. The common part of all these is that they use open-text instead of closed questions to capture the majority of the information.
Quickly addressing issues
If we look at mature customer experience efforts we see a strong (often company-wide) desire to close the loop quickly with raised issues, regardless of the channel they come in from.
In the hotel industry a range of platforms have been developed to enable managers to capture feedback regardless of where it is left (guest surveys, social media or one of the many hotel-review sites) and ensure that the guest is acknowledged quickly and any issue addressed. Many hotels will have daily ’meetings for management to review issues and identify solutions.
This approach has two key benefits, both of which apply to employee experience. First, the issue can be addressed in a timely and effective manner. Second, responding quickly minimises the chance that issue becomes bigger than necessary via word-of-mouth.
Multiple levels of action
With employees action often needs to come from various sources. Some interventions can be dealt with at the manager-level. Others might require escalation to various internal teams, eg IT. Finally there are a class of issues which are most effectively dealt with at a senior level.
If HR want to effectively manage employee experience I believe that they need to build an approach similar to the management meetings in hotels. They need to build cross-organization groups to ensure issues are addressed quickly and effectively.
When I managed the new employee experience program at UBS we created governance groups to ensure a multi-stakeholder approach. IT created a special team to manage global technology provision and we co-designed processes to bridge the gap between HR and IT and ensure that even though managers were still responsible for ordering, if IT hadn’t received the order in sufficient time they could proactively contact the manager. We also created technology that could ensure that different members of our cross-business governance group could be responsible for their content whilst providing one simple coordinated communication.
With employees it’s not always a good idea to devolve action to managers. Employees will often choose to remain silent rather than providing voice if they don’t feel safe. With the manager so important in creating, and destroying, employee experience there is an opportunity for a employee experience team to act as the filter, viewing the feedback and allocating it to the correct individual, whilst preserving confidentiality if appropriate.
On a regular basis there is a need to take a top-down view of all feedback. Statistical analysis and ‘AI’ can provide badly needed objectivity to understanding how themes are distributed, where outliers lie and how feedback is linked. This objectivity is vital to ensure that decisions are effective. The technology ensures that such work can be done in a timely and cost-effective manner.
Ask for ideas and issues
Many in the customer experience space suggest asking a simple ‘why did you provide that answer’ type question as the open question. After studying millions of answers to employee questions we believe that there is a better way.
Some of your most engaged employees will be the ones who identify the most important things to change. Conversely it’s important to understand what those who are disengaged or unimpressed value.
Our experience is, just like how good performance feedback can be given, it’s best to ask everyone ‘what is good about x?’ and ‘what could we do to improve x in the future?’
This effectively doubles the amount of feedback you receive whilst reducing bias in the data. It makes analysis easier and adds little additional burden on the user.
The advantages of asking many people open questions
At the beginning I suggested that qualitative studies typically required less participants. The key reason was that the ideas tend to converge quite quickly.
The main reason for not asking lots of people is the cost of doing so. Manually themed text is time and resource-intensive so it made sense to do as little as is needed to get good results.
However there are advantages of getting more data. Themes and ideas have a long-tail distribution and the more data you have the more you can look down that tail.
At the same time if you want to understand the distribution or relationship between feedback and ideas you can extract more insight with more comments.
Modern text algorithms dramatically reduce the time and resources needed to get human-level accuracy. They therefore offer more opportunities to collect and comprehend ideas at scale.