Just about every consulting firm, CEO, hiring manager, and recruiter believes there is a significant shortage of qualified people, especially for positions requiring artificial intelligence experts, machine learning gurus, data scientists, and some engineering categories.
If they look only at the local, experienced, and mostly white ivy-league educated candidates, they are correct.
The reality is that there is a bigger shortage of realism about the job market and how we define needed skills, how deep or comprehensive they need to be, what schools and education are required, and until the pandemic, where the person is located.
We have created our own perceived talent shortage through short-sighted and often biased thinking based on an over-reliance on data that provides a distorted and inadequate view of the talent market. The pandemic has illustrated that we need to rethink old assumptions and tap into a more diverse talent pool no matter where people are located.
Here are some of the factors that have led to our perceived talent shortage.
1. Credential and Experience Creep
Employers have raised minimum job requirements to very high levels. Jobs that a decade ago required a certificate or an Associate Degrees or less are now being filled by those with bachelor’s or even master’s degrees. For example, a recent study by Burning Glass reports that only 39% of those currently holding computer or network support jobs have a bachelor’s degree or higher, but 60% of all similar jobs posted now require one. A similar pattern emerges for production supervisors and managers. About 42% of incumbents hold bachelor's degrees or higher, but 68% of all posted jobs require one.
Many professions require degrees and, in some cases, an additional certification that historically required only an apprenticeship or the passing of an exam. The evidence that these credentials are necessary is sketchy. Many jobs can be taught on-the-job, learned through apprenticeship, or by serving as an assistant to someone with deeper knowledge. In many areas, job content has not significantly changed or can be learned without formal academic training.
Furthermore, a U.S. Census Bureau report from 2014 shows that 74 percent of those who do have a bachelor's degree in science, technology, engineering, and math are not employed in STEM jobs. Some go on to pursue graduate study, but many cannot find employment in their chosen field. This is often the result of bias, which we will discuss later.
Often they seek work but cannot find it because employers only want experienced people and are not willing to invest in training or apprenticeships.
2. Not Investing in employee development
According to the U.S. Census Bureau, in 1996, almost 20% of workers received employer-paid training. By 2008, that had dropped to 11%.
During the 1980s, companies laid off employees for the first time to cope with rising inflation and retirement costs. They removed layers of management and mid-level workers and cut training and other programs that were considered more costly than they were worth. However, by removing a generation of future workers who would be ready and able to fill the new jobs, they inadvertently contributed to the shortage. Eliminating most employee development programs, a standard feature in firms of all sizes in the past, resulted in no immediate impact but reduced the pipeline of skilled employees ready to move into open positions. Firms are now forced to compete in the open market for whatever talent is available. Development programs that remain are usually focused on senior leadership levels or improving management skills rather than developing technical skills.
3. Not understanding supply & demand
Technically there is no shortage of STEM graduates in the United States. We graduate thousands of computer, science, math, and technology students every year. The Bureau of Labor Statistics projects that the total number of STEM job openings between 2010-2020 will be slightly over 2.5 million. Still, the number of STEM graduates will exceed this number by over 1.3 million graduates. In other words, there is no shortage of raw supply. Granted, many of these graduates will pursue graduate degrees or choose to enter some other profession, but there is still a healthy supply. Add to this the large number of foreign graduates hired on a variety of work visas, and it is very hard to make the case that there is a numerical shortage of qualified workers.
At the same time, demand for various STEM professions changes rapidly and often. In the recent past, there was a spike in demand for software coders. Coding schools sprang up, and many high schools and colleges offer coding courses. But concurrently, the demand ebbed as A.I., and other tools made coding much easier and faster. Frequently demand grows suddenly for a particular role, people self-study or go to school to learn the newly desired skill, and then the demand drops quickly. In a fast-changing environment, it is tough to know which specific skill will endure.
4. Limiting the scope
College recruiting practices have changed very little over the past twenty years or more. Most large companies recruit only at a handful of highly regarded universities, often called key schools, and even at those schools limit their recruiting to a small subset of graduates. Firms require very high GPAs and majors that exactly mirror what they are doing in the company. There is no room for people who have studied different subjects or who have lower GPAs. Hiring is often biased toward certain graduates – typically male, white, and from higher-level socio-economic backgrounds. There is no budget for relocating graduates or other services to transition from school to work less stressful in many cases.
Small schools, less prestigious schools, and minority schools are often forgotten or ignored in the college recruiting process. A few companies may go to schools with large minority populations, but this is not mainstream.
Smaller firms often do not do any college recruiting and rely on finding and recruiting experienced workers in the open job market.
Because the supply chain is so limited, there appears to be a shortage of graduates. Yet as we saw above, there is a surplus of STEM graduates – many of whom never receive an interview or a job offer.
5. Not embracing or understanding technology
Robots will augment many jobs within the next 20 years. Automation has already augmented and, in some cases, replaced engineers. Automation is entering the hospital, and telemedicine allows one doctor to see many more patients virtually than they can in-person. The gig workforce can expand the skills needed for a project without the burden of permanent hiring. Crowdsourcing a team of people can also lessen the need for more hires. It would be a mistake to assume that the growth in STEM jobs will be continuous and linear. Demand is very likely to fall off precipitously within our lifetimes.
6. Biased, intuition-based hiring decisions
Bias runs rampant in recruiting. Hiring managers and senior leaders have stereotypes about what good candidates should be like. They have beliefs about cultural background, education levels, schools people attended, grades they achieved, and their activities. They may like people from one organization but not from another. Recruiters, too, tend to weed out candidates that do not fit their own biases. There is little quantitative evidence to support most of these stereotypes. Google has done extensive analysis of its past recruitment criteria and compared those criteria to success on the job, tenure, and productivity. They found no correlation and have now changed their criteria to be more inclusive. We have data showing that women and minorities are underrepresented in almost all workplaces, especially in the high tech world.
Google released a report on its minority employees that shows only 1% of its technical employees are black, and two percent are Hispanic. Yet Asians make up 34% of the company's tech workers. Eighty-three percent of Google's tech workers internationally are male. For non-tech jobs, the number is 52%. And this is mirrored throughout the tech world.
The available qualified workforce is then reduced again by excluding women and minorities.
7. Ageism
Anyone over 45 looking for a job has almost certainly woken up to the reality of age bias in hiring. Firms like to hire young people for various reasons, some of them with some legitimacy but most based on prejudice and misunderstanding. The reasons for excluding older people include lack of energy, limited technical knowledge, limited computer skills, unwillingness to put in long hours, an expectation for higher salaries, and simply because they don’t fit into many high tech youth cultures.
As our population ages, we will need to focus more on using older workers and finding how to use their experience and skills best. Older workers can act as coaches or mentors to younger employees, and many of them are able and willing to learn new skills, including coding. Some older workers are looking for part-time employment or flexible employment and could be hired to augment the current staff. They could temporarily replace employees on vacation, and they could pick up extra work during peak times.
To exclude older workers is to cut down on the available talent and lose out on years of experience and knowledge.
Combine the requirements for higher levels of education, credential creep, and experience with the lack of internal training programs, and you have a self-imposed talent shortage that could easily be corrected by a change in hiring requirements and by an increase in internal training.
It is hard to support the claim that there is a shortage of talent. If we were to tap into the diverse pool of talent that includes women, underrepresented ethnic groups, and older people, whether they went to our favorite schools or met all our requirements, we would likely end the anguish about a shortage. If we add new graduates and those seeking a career change while we spent some of the money on training that we currently spend on recruiting, we will create a vibrant and creative workforce ready to tackle future demands. Our only shortage is in the narrow definitions we hold.
Why Upgrade Your Subscription?
We are working on several white papers, introductions to hot emerging products, book reviews, video interviews with cool people, and podcasts.
Please consider upgrading to a paid subscription or recommend this blog to your friends. By doing so, you help me make these possible.
The subscription is only $5 per month or $30 for a full year (only $2.50 per month). Thanks.
Kevin - This is such a great summary. I know you're a holistic thinker, and would emphasize your point about lack of development...there's such a focus (especially in the US) on hiring from the outside vs developing the inside. And given system access issues, I fear the impact that'd had and will have on people from under--represented groups. I'd love to see you share even more on why companies insist on hiring from the outside/buy vs developing/build, and what kind of cultural issues and economic incentives create differences in thinking around this between west and east, rich/high margin industries and low, tech and non-tech. Anyway, thank you, as always, for sharing. Great stuff!
Kevin Wheeler's post "7 Ways We Created the Talent Shortage" is an excellent summation of what talent management professionals and senior leadership are overlooking, consciously or unconsciously, with systemic effects. He's expressed these ideas before, explored a few in more detail elsewhere, but the reified minds of too many corporate leaders and business school academics suggest these seven points are not yet redundant to them. They are left encouraging a short-money game, relying on worn-out notions and unoriginal techniques that waste and exploit talented people.