New ways to run engagement surveys

A version of this post originally appeared in EFMD’s ‘Global Focus’ Magazine.

“A new market has emerged: Employee feedback apps for the corporate marketplace. These tools are powerful and disruptive, and they have the potential to redefine how we manage our organizations.” Josh Bersin, Forbes August 26 2015

The measurement of employee engagement is changing. Businesses have been measuring engagement for about 15 years, the market is currently worth USD1bn per annum yet most reports suggest engagement is trending flat if not actually decreasing. Something is obviously not working.

There are many reasons businesses are growing frustrated with current methods. Slow, expensive and resource-intensive are some of the more common ones we hear. In many businesses the only things that are now measured on an annual cycle are engagement & performance management – both run by HR. Business leaders are demanding more real-time insight.

During the same time HR has been emphasising understanding employee engagement the measurement of customer engagement and feedback has been changing remarkably. Today many firms are capturing an always-on stream of customer data from a wide variety of channels from short surveys to social media.

Often called Voice of the Customer, the emphasis has switched to continual listening, rapid resolution and bringing deep insights of customers needs into everything from product development to service provision.

Business leaders are asking why there is a disconnect. Why have customer teams adapted whilst HR has stood still? Many of the trends that we’re seeing in engagement measurement could be viewed as an application of a Voice of the Customer philosophy to employees.

At the same time that there is this shift in the demand side we’re also seeing a shift in the supply caused by technology.

The technology changes can be classified into four categories.

Automation

Technology-led automation is something that is happening across society and it should be no surprise that it’s surfacing in our area.

Firms, especially new entrants, are automating parts of the engagement measurement & analysis process that typically was done by analysts. Whilst it started with relatively simple report automation – the production of thousands of template-based pdf reports moments after a survey closed – we’re seeing the level of sophistication increasing.

Whilst this might, and should, have been utilised by the traditional firms for some time their incentive was to increase margin. Most of the new entrants are using it to radically reduce prices and complexity. The new business models are disruptive.

Real time reporting via dashboards is becoming the norm. Production of large numbers of PDFs is possible. We’re seeing a shift from the multi-page result presentation to one page infographic style reports. Ultimately there is a shift from seeing the provision of a large numbers of individual reports as complexity to seeing it as a commodity solution.

The ability to automate however can blind the user to really question whether they’re addressing the real issue, or merely creating a faster, cheaper broken process.

Rethinking the end-to-end feedback process

For Workometry we took the full end-to-end feedback process back to first principles. At the beginning and end of this process there are likely to be two time consuming and expensive periods of qualitative research – designing a great survey at the beginning and running workshops to understand context regarding the issues at the end. Only by addressing these long, expensive activities can you make feedback truly agile whilst preserving the richness.

Mobile & User Interface changes

The second technology-led shift is to do with the way that employees are able to take surveys.

For several of our clients mobile has become the dominant channel for employees to take surveys. We see respondents taking surveys just before the working day, during lunchtime and even in the evening. They’re interacting on the edges of their working days and grabbing a mobile device to do it.

Consumer web technologies have changed the way we expect to interact with our devices & engagement surveys can’t escape this trend. Many of the question types we used were the same as we used on paper but digitalised. We used these methods often because they were easy to score. Digital-only surveys aren’t bound by these constraints.

Research in user interfaces is reinforcing these methods. In a world where people expect to touch, slide and scroll through long-form sites surveys have needed to adapt.

Big data technology

The majority of the new entrants are focussing on the previous two technologies. Whilst this is right for medium sized businesses, enterprise organizations typically have a set of needs that extend these simple use cases.

One shift that has occured during the last few years in a number of firms has been the building of sophisticated People Analytics capability. Firms in this position are increasingly wanting to combine and analyse employees’ demographic, behaviour and perception data to answer key, strategic business questions.

Whereas employee survey data has historically been treated as an island – analysed with the context of the perception data or a predefined limited set of demographic information – survey data is now used to give critical insight into the reasons why.

To do this type of analysis requires that the survey data can be linked on an individual basis to both an extended set of demographic data, and to behaviour data, either from HR or business systems.

Furthermore it’s often useful not just to analyse the result of one survey with the extended data set, but to also include all other survey data belonging to an individual. Such requirements quickly dictate the sizes of data processing systems.

As well as the ability to handle large data sets increasingly analysts are using non-table data structures to better answer questions. One alternative that offers great potential are so called ‘graph databases’ where data is stored in a network. Such data structures allow us to ask very different questions.

With network data we can more easily answer questions not only about the individual employees but also the relationships between the employees. We see early promise in a network perspective at looking at contagion of engagement – ie how changes in employee engagement can spread across an organization.

Network survey technology such as that produced by Australian start-up Polinode allow businesses to capture not only traditional survey questions but also ask questions about an individual’s working relationships. Alternatively it is possible to understand communication patterns through the data trail left by emails, telephone calls or participation in internal social channels.

Machine learning

The final technology trend which is starting to disrupt the survey world is the application of machine learning – the use of algorithms to search for and learn patterns in large quantities of data. Machine learning is also the basis of much so called ‘predictive analytics’.

With employee survey data we’re seeing great success with three applications of machine learning: using text analytics to make sense of vast amounts of open text answers, using pattern-spotting techniques to make probabilistic assessments of which populations are most likely to raise certain topics and finally to use survey data to answer business questions.

Historically it’s often been acknowledged that open text is the most valuable part of a survey, however it’s been very difficult and resource-consuming to deal with it at scale. Text analytics can solve this problem and therefore provide new opportunities to capture this richer form of information.

Our experience is that with these techniques we’re able to analyse open text responses in almost any language, categorise a comment against a continually evolving set of categories & score against things such as sentiment and to do so in near real time. With this capability it’s possible to radically rethink how and what data is captured.

The second use of machine learning is to identify groups most likely to be discussing certain topics. Whereas traditional surveys might show differences between one function’s scores against their peers, with machine learning it’s possible to segment the population a much more granular manner. For example you might discover that those complaining about a shortage of career development opportunities are much more likely to be women, gen ’Y’ers who are in the upper performance grades.

Finally survey data is increasingly important to answer strategically important business questions that involve the workforce. For example you might link the survey data to sales data from a CRM system to try and optimise sales performance. In some cases it’s possible to use existing survey data. In others surveys need to be used to collect new data.

So with all these opportunities where to get started? We typically advise our clients to do three things:

1) Make sure that the legal and other agreements are in place to use data in this new manner. Be open with your employees about how their data is being used and how the new approaches don’t need to mean lower levels of confidentiality

2) Pilot some approaches with new use cases or in discrete populations.

3) Consider those pilots as supplementing existing work. From our experience you might replace old approaches but there is often significant political capital invested in the established approaches.