Measuring The Quality of Hire
And Why You Really Can't
What do you mean when you say Quality of Hire? How do you measure it?
One of the most difficult aspects of creating a metric of any kind is having a precise, clear, and measurable definition. And here is where the challenge lies when talking about the quality of a candidate or a hire.
Quality defined by manufacturing standards is the elimination of variance. In other words, the more an object or a process is exactly like a similar product or process, the higher the quality. Variance is measured in thousands of percentages with a goal of at least 99.999%, realizing that 100% is never attainable. This is called Six Sigma and means only 3.4 defects for every one million opportunities.
Can we define human performance in this way? Perhaps in some cases. We can say that high-quality coders, for example, are those that write N lines of code in an hour or a day with fewer than 3.4 errors for every one million lines of code. But how do we define such a measure for leadership? For creative writing? For marketing success? For human resources? For building relationships with candidates? In some cases, we can count how many of something (e.g., novels or stories written, relationships formed on LinkedIn), but we cannot understand the value or efficacy of these numbers.
Once we have a definition and a target measure, there needs to be a data set large enough to be objective and reliable. For example, when we test a vaccine, it requires thousands of people to take the vaccine to determine if it is effective. The sample size needs to be quite large. One of the fallacies in statistics is called the “statistics of small numbers,” where we use a small number of instances and extrapolate those to an entire group. This is often what hiring managers do by basing their ideal hire on one or two others they have hired.
If we adopt the definition of quality that is the hallmark of manufacturing, quality is not about perfection or opinion; it is about reducing variations. In the context of recruiting, this would mean reducing the variations or differences in skills between candidates as much as possible for a similar position. We would need hundreds or thousands of exactly the same type of hire even to begin to judge quality.
Taken to the extreme, this means hiring people who are almost clones of each other. This is the inherent danger of only using artificial intelligence to determine who is the best candidate. As it is today, A.I. uses specific data such as pedigree, skills, or previous experience to determine fit for a job. It matches a candidate’s pedigree, skills, and experience to those asked for in a job description, but these have proven to not be the best indicators of quality or performance.
Trying to remove the differences between candidates works against us because it reduces diversity, potentially leads to groupthink, and inhibits innovation.
If a job is completely defined and quantifiable, a robot would be a better choice than a human. This is often the case in jobs where rules and procedures apply and there is little room for judgment.
Without a well-defined, objective measure, we are left with subjective judgment. We all have been in situations when one manager thinks an employee is great, and another manager feels the opposite. Virtually every way that recruiters define quality has flaws that make the measure useless.
Flaws In Typical Measures of Quality of Hire
#1. The first flaw is measuring how quickly a new hire is productive. Why is speed important, and how do you determine the standard? Is productivity more important than judgment or innovation? In a handful of repetitive jobs measuring speed could be used fairly, but for most jobs, it does not provide useful or valid information. Perhaps an in-depth understanding of the work by observing and learning and getting a thorough grasp of details would be more useful critical to long-term success than speed.
#2. The second flaw is not defining precisely what productive means. This is very hard to do for positions where the output is not tangible or varies from time to time. Is it possible or fair to tie the level of an employee’s productivity back to the hiring process? The variables that influence productivity include training, the work environment, the new employee’s teammates, and the corporate culture. There are too many variables to say that the recruiting process alone causes or even has any real impact on performance.
#3 The third flaw is measuring quality by looking at a new employee’s performance rating (assuming your firm still has performance ratings). Once more, this is a post hoc ergo propter hoc fallacy. It does not follow that recruitment leads to poor or good performance. Once again, there are too many other variables that affect performance far more than recruitment. Performance ratings are subjective and depend on personality, and the relationship the new hire and the manager have with each other as on actual performance. Many organizations have recently abandoned performance ratings because they are not objective, increase employee dissatisfaction, and do not improve performance.
#4. And finally, the fourth flaw is using turnover rates as a measure of quality. Whether someone leaves in a short period of time or a longer one is much more likely to be caused by the manager, the economy, the organization’s product vision, economics, the corporate culture, or some other factor than because of the type of person recruited.
Quality of hire remains elusive and very hard to measure validly and objectively. We need to either find measures that we can all agree are fair or stop trying to measure quality in ways that lead to biased and unjustifiable exclusions.
NOTE: I don’t agree with these articles, but I have included them so that you get a balanced view.
If you insist on measuring QoH, Lou Adler offers the best way to think about it.
If you enjoyed this article, please encourage your mates, colleagues, and friends also to subscribe. Just send them the link below. For Paid Subscribers: Two white papers will be published soon, along with recent interviews with key influencers. Don’t miss these.