Thoughts on winning the ‘Emerging Game Changer’ award at People Analytics 2016

Last week, at People Analytics 2016 – probably the largest European Workforce Analytics conference – we won the inaugural ‘Emerging Game Changer’ award.

For the pitch presentation I decided to show 5 reasons in 5 minutes why we think Workometry is a game changer for Employee Feedback. For those not present I thought I’d share the deck and my commentary and in doing so hopefully highlight what the judges (Eugene Burke, Max Blumberg, Emma Parry and Richard Phelps) saw.

According to Josh Bersin, the employee engagement market is worth $1bn a year. Businesses have been focused on engagement for about 15 years yet even with this massive spend engagement has remained stubbornly flat, or even declining. Something can’t be working.

At the same time all areas of business see a demand for more, better and faster measurement. A large proportion of firms want to include more regular engagement measurement.

The trouble is to increase frequency yet maintain participation you need to reduce employee burden. The easiest ways of doing this are to reduce the number of questions and improve UI. The side effect of short is less rich data.

Listen to your employees’ views in their own words

As analysts, when we presented traditional survey data to executives we were surprised that what really drove action wasn’t our wonderful data analysis but instead the open comments. The reason was that the comments provide context, which adds clarity and a sense of knowing what to do.

So we set ourselves a bold target. Could we get good enough at understanding text that we could drop most of the traditional, scale questions? Can we do that with the multiple language surveys our clients want to run?

It took 2 years of experimentation before we cracked it. In the end we found our saviour in the research lab of a prominent European university’s computer science department. Originally developed for the consumer market we’ve worked with them to adapt it to understanding employees. It’s using an approach that uses the semantic meaning of the statements instead of keywords. We tune the algorithms to each question a client asks to ensure maximum accuracy.

In the presentation I showed our 4 question engagement survey. We believe that the data we’re getting from this – typically around 80 topics – is age least as rich as a traditional 80 question survey yet typically takes 3 minutes to complete.

Most importantly, the comments quickly add context. The hardest part is to identify reliable patterns across large volumes of open text. This is where the following can help….

Different data = Different (better) questions.

The second differentiator may seem a bit geeky at first but is at least as important as the ability to master text understanding. Workometry is based on two new data storage technologies.

No analyst enjoys data munging but typically it’s about 80% of the time of a typical predictive modelling project. We wanted to capture data in a format which would be analysis-ready.

There are two structures we use. The first is a very fast single datastore which enables us to use one store for all surveys a client runs at any time. Surveys are confidential but we can do individual-level longitudinal analysis to spot patterns how perceptions for individuals develops over time. We can compare the results of a particular question based on what employees said at another time. Does the experience of a New Joiner determine behaviours and attitudes throughout their employment?

The second, and one I’ve written about here, is to use graph data to understand how perceptions are influenced by an employees relationships. We use graph-data both for understanding things such as contagion of perceptions and to build more complex reports based on working relationships. (e.g. show people who report into Jane or those who worked on one of her projects in the last 6 months).

Great data visualization

Our approach to data visualisation is to develop designs to answer specific questions that the user is likely to have rather than just show the data. For text-based data visualisation, Ben Schneidermann’s famous mantra of “Overview first, zoom and filter, then details-on-demand” particularly holds. We’re trying to help users guide and find important insight, spotting patterns and relationships across tens of thousands of individual comments.

Of course like most new providers we can do live reporting, even on text. However we can quickly design and deploy new visualisations to answer new client questions quickly. Need to understand the last year’s answers by sickness? Just let us know sickness levels and we can provide answers quickly.

For communication where interaction isn’t necessary we work with graphic designers to create info-graphic reports and then automate the production of hundreds of personalised views. The most important part of data communication is the communication, not the data.

Machine learning to aid comprehension and find patterns

ML approaches are baked deep into Workometry but we like them to be almost unnoticeable to the user. We want to bring them into visualisation to help answer questions in a natural way by guiding the user.

I showed two examples. The first is use of clustering algorithms to simplify groups of topics which are frequently used by the same people. This can be within the same question – cooccurrence – and between questions.

The second is our use of probabilistic Modelling to answer questions such as ’what’s the probability that people who talk about ‘great pay’ when asked about the best things about working for the firm are engaged (they’re not, they’re mostly disengaged).

Using perception data as part of a larger analytics ecosystem

Workometry links closely with other data about employees and as such has been set up to enable easy integration with advanced People Analytics.

For example we’ve recently finished an employee attrition project where survey data was included. Not only did we identify issues which were associated with employee turnover but we could identify those issues most critical to high performing leavers.

Workometry is being used outside HR. One internal communication client is using it to understand resonance of key messages across the organization and to various segments of their population. We’re even working with one IT department to understand help-desk issues to predict which employees are most likely to suffer certain issues and to ‘push’ support and training in advance. This combination of perception, behaviour and demographic data is really powerful.

Finally, for those clients who are at a more capable level of analytics, providing their legal teams allow, we can provide certain groups direct access to the datastore enabling teams to bring live perception data into their models.