How do you draw the right conclusions from open-ended responses from employees?
This insurer conducted a qualitative study in which employees were asked questions about various themes such as communication, workload, engagement and enthusiasm.
Within the company, critical job families have been defined. These are roles that are critical to the continuation of the business operations. The organization is interested in the topics and needs that arise among employees within these critical job families, compared to non-critical job families. Another question is whether there is a difference between the various business units, so they can actively implement policies accordingly. The goal is to gain insight into the current situation and zoom in on improvement points.
The study consists of a quantitative part, supplemented with qualitative results. The questions are asked based on a 0-10 Likert scale, after which employees can provide an explanation for the score they give.
What is the Likert scale?
The Likert scale is a response scale commonly used in surveys, psychological and social science research, and market research. Respondents indicate the extent to which they agree or disagree with a statement. The scale consists of a number of response options, ranging from “strongly disagree” to “strongly agree,” and is therefore also called a rating scale or forced-choice scale. It is named after the American social psychologist Rensis Likert, who developed this method in the 1930s.
Analyzing the results proved to be a significant challenge for this organization. Like many other companies, they struggled to process and interpret open-ended responses from employees in the questionnaires. Reading and analyzing each response manually is not an option given the large numbers. Therefore, this valuable source of information often remains underutilized. This is where Highberg steps in to help.
The structured analysis approach of Highberg
To interpret the results, Highberg performed several steps and analyses. The goal was to compare and cluster topics that emerged based on qualitative input and then analyze them in detail. We naturally choose the most suitable analysis methods for this purpose.
- Distinguishing between positive and negative responses:
We looked at differences between the comments of employees who scored questions positively (a 9 or 10) and negatively (a 6 or lower)
- Descriptive analysis:
A descriptive analysis is a relatively high-level investigation into the number of comments, such as quantitative response and qualitative response per question
- Cluster analysis:
We applied the K-means clustering algorithm to extract topics from the open responses. This characterizes and clusters themes and topics that are relevant to the participants in the study
- Word count analysis:
Words from the responses per question and theme were combined. Subsequently, we analyzed how often a word or combination of words appeared in a positive or negative context
What is K-means clustering?
K-means clustering is a commonly used technique to find groups or clusters in a dataset based on their similarities. The goal is to divide the data points into a predetermined number of clusters, where points within a cluster are similar to each other and differ as much as possible between different clusters. This allows you to identify themes, for example.
Analytical techniques help determine next steps
The analysis was very valuable for this organization. Not only because it is now clear where the company should focus to increase employee satisfaction, but also because this is supported by numerical data. Without these extensive and varied analytical techniques, this would not have been the case.
Insight
For example, by analyzing the open responses of employees, it is known what satisfied employees are specifically satisfied with and on which topics the organization needs to improve. One of those themes is self-development. Based on the findings, the organization has established new policies to facilitate self-development more effectively. The same applies to themes such as workload and collaboration with managers. Aspects that can be extrapolated from open responses often remain underemphasized. With Highberg’s approach, this valuable source of information is well utilized.
Continuous improvement
By conducting ongoing Voice of the Employee Surveys, it quickly becomes apparent which interventions have achieved the desired results. This allows for continuous improvement, and any issues and challenges are identified quickly.
