Wharton People Analytics Conference

Wharton People Analytics 16-9_Page_05.jpg

OrganizationView was a runner-up in the Startup competition at the 2019 Wharton People Analytics Conference. This was our presentation


Good Morning.

I’m Andrew Marritt, CEO and Founder of OrganizationView, a specialist People Analytics practice founded in 2010.

Since 2015 we’ve focussed almost exclusively on helping clients use workforce sourced text data in their models and decision-making. As analysts we saw that the best quantitive models could identify who, when and what but failed to explain ‘why’ a behaviour occurred. We understood that to improve we didn’t need better algorithms but instead better data.

Understanding ‘why’ is critical to design policy and process changes that are at the heart of realising value from data-driven changes. We saw that rich, contextual text data was critical to driving executive action and therefore positive change.

OrganizationView, through our Workometry service provides large, global clients with an effective and efficient way of generating insight from employee feedback quickly and flexibly.

Alternatively structured, not unstructured data

Wharton People Analytics 16-9_Page_03.jpg

Text is often classified as unstructured data. We disagree with that view.

Employee text is structured by its grammar, it’s structured by the context of the question that was asked, the context of the unique language that builds-up in any large organisation.

Any effective analysis of feedback has to be able to incorporate all these aspects. With Workometry therefore we code at the organisation and question level, using the grammar of the answer to inform the understanding of the text. We use an inductive approach which learns the themes from the clients’ own data.

Themes

Our themes are context rich groupings where the label should be a legitimate answer to the question. So instead of ‘communication’ we might state that employees want more transparent leadership communication, better interdepartmental communication or less communication at weekend. Only when you summarise at this level can you close the distance from raw data to effective decisions.

Clients constantly tell us that our specific models comfortably out-perform the best generic models. We’ve yet to be beaten on a competitive assessment.

Humans + Algorithms beat Humans or Algorithms

A key differentiator of Workometry is that we use a human-in-the-loop approach.

The inspiration for this came from Gary Kasparov who discusses how decent but not exceptional chess players plus a chess computer can beat Grandmasters or the best chess computers.

With Workometry we use domain-experts to support the algorithm when it’s unsure. We’ve built a pipeline that asks experts for interpretation when needed, but where the algorithm handles routine decisions. The experts’ coding constantly builds and refines the training data ensuring rapid model convergence.

Ask any question

Wharton People Analytics 16-9_Page_06.jpg

Another aspect of our specific modelling approach is that we can interpret the answers to any question.

This has enabled our clients to increasingly use Workometry to inform key business challenges. 

Ask any executive of a major global firm and they’ll tell you that the organisational distance between them and those closest to their customers and key processes is large. Communication typically is done via Chinese Whispers. We give them the ability to go directly to where the knowledge and ideas originate and opportunity to use machine learning to synthesise massive volumes of multi-lingual feedback to the key insight.

Let me give you a couple of examples:

What is getting in the way of you doing your job?

Many firms find that technologies and processes can stop employees operating to the best of their abilities. High Performers can be especially frustrated by these inefficiencies.

One of our global clients wanted to identify these frustrations. The question they asked all the employees globally was ‘What processes and technologies are preventing you from working to your best ability?’ They received 7.5 million words from 50,000 thousand employees. For this instance we built two models - one to identify the processes and technologies (over 300 key ones) and another to identify the issues and causes. 

Optimising retailers

Several of our clients are large, global retailers. In this highly competitive, low margin sector capturing and responding quickly to what is happening on the ground is vital, these days even helping ensure the survival of the business.

In some instances our retails clients ask business-specific questions such as ‘what can we do to improve customer experience in our stores?’ or ‘What do we need to change to ensure we have the best ever Christmas?’.

In other instances employees mention business specific changes in engagement type questions. The quotes showing came from one retailer who had changed the music in store to reduce broadcast licensing charges. We quickly identified that the new music was causing significant numbers of complaints from employees, that these employees also linked it to customer problems. Those employees were typically the most engaged yet felt that leadership was out-of-touch and didn’t listen.

By surfacing the issue this, like many of the changes we identify could easily be fixed. In fact not only could we reverse the issue with the music but leadership could also demonstrate that they listened and acted quickly.

Listening to employees

I’ve given you just a few of the many examples where our clients have used Workometry to make positive changes to the way they run their organizations. Whilst in many times the data comes from surveys we have helped clients understand feedback from performance management systems, help-desk systems… anywhere where employees provide answers to questions in large volumes.

Our clients typically fall into two groups. Many are the People Analytics functions of some of the largest global firms. We find that the best data scientists become some of our biggest supporters. They typically understood the difficulty of achieving the quality level we do and are glad that they use Workometry flexibly to incorporate text into their work.

The other big set of clients that uses Workometry are the large, global management consultancies and survey organisations. We let them white-label Workometry to extend their services with their clients. With this group we often build hybrid models - starting with their proprietary classifications and adding new themes for the comments outside their model. We can provide the service via an API.

As someone who has been applying machine learning to employee data for 15 years I’ve always passionately believed that the real value comes from combining business and workforce information to solve business problems. 

With Workometry we believe that we enable our clients to use the best possible data to solve these problems, not just the data that is easiest to analyse.