The article discloses key risks in hiring candidates through interviews with the help of AI and provides judgment points to consider for decision-making. The article’s main goal is to remove AI-related fears for hiring managers and provide mechanisms to ensure the quality of hires with each decision.
Background
Large language models (LLM), as good as AI-backed tools and agents, continue integrating into people’s day-to-day lives and careers. The variety of questions and problems we delegate to AI is growing daily. It is no longer limited to questions like “How do I cook my favorite dish?” and “What is the distance from me to the nearest planet where I can drive my car without traffic jams?”. General purpose LLMs answer our questions about taxes, laws and even become aware of our health problems faster than specialists.
Existing general-purpose models provide reasonable answers to questions that require basic and intermediate knowledge in almost any area. Purpose-built models can solve fractions of advanced problems. But all of them become unstable when it comes to an expert level.
From a business perspective, research and articles state that “using AI saves approximately 10% of the time by speeding up routine processes” (check this and this, for example).
But what about hiring and interviewing?
Since the previous century, people have been used to face-to-face interviews. However, COVID-19 proved the effectiveness of remote work and virtual interviews. After the LLM boom, it was ridiculous not to give it a try and use AI during your virtual interview session. And this is our reality: AI is a part of people’s professional lives, but companies still insist on face-to-face setup and clearly state that the use of AI is not acceptable during interviews.
Problem
AI is not a reliable source of information, and it is a significant risk for a business owner to rely on something untrustworthy. LLMs can hallucinate and provide information that looks correct when it isn’t. Even more, they can provide a faulty conclusion wrapped with well-established facts, and the person aware of the facts, can be biased to trust the conclusion.
Only specialists with a deep enough understanding of the problem can identify the issue and reject AI’s answers when needed.
So, from a business perspective, nothing has changed in hiring from the previous century: everyone needs to hire good specialists. And if the goal is the same, why should we change the approach?
Opportunity
The ban on AI usage during interviews has the following key questions to answer:
- We need to hire a specialist. So, how do we evaluate the expertness of a candidate and separate it from AI capabilities?
- How do we ensure equal opportunities for all interviewees?
- How do we assess each person’s behavioral characteristics?
For the first question, the answer will be straightforward: we don’t have to. If humans and AI can solve problems effectively and efficiently, it would be an excellent setup for the company to hire both. For example, many IT companies ask candidates to solve an algorithmic problem with the goal of finding the most effective and efficient solution at the end of the allocated time. If the candidate will use a solution provided by AI and can explain it, discuss trade-offs of different approaches, and clarify why this specific approach is the right one, how is it different from writing the same solution themselves? To be extra sure of this candidate, the interviewer can share updates to the original task that increase the complexity of the problem or will require a rethinking of the approach. If the candidate can accomplish this task, it is an even more obvious factor of a good fit. Even more, AI usage saves the time needed to code the solution, and the interviewer will be able to check the in-depth knowledge of a candidate in different areas.
In summary, interviews should focus on evaluating the candidate’s deliverables and expertise instead of criticizing the approach.
Regarding equal opportunities for all interviewees in the context of AI usage, the world has numerous AI tools, models, and agents, whether specialized or general purpose. Companies also invest in different AI subscriptions for their employees. So, the problem is to align expectations. The worst-case scenario is if a candidate is used to a specific AI agent, and a company will have to buy it for them in case of a hire. To address this problem, the company should communicate a set of acceptable AI tools, and the candidate should share the AI tool they will use BEFORE the interview starts.
- If the tool used by the candidate is on the company’s list, that would be perfect.
- If the tool is not in the list, the company should consider a subscription to the tool as an employment requirement. If it’s impossible to establish such a subscription for the candidate, alternatives should be discussed. The main goal: ensure each candidate uses a tool they are the most familiar and comfortable with.
Moving to the third question, the behavioral aspect is never easy to evaluate. One of the most known approaches to get data points during the interview is asking about different “tricky” situations a candidate had previously faced. Based on the response, an interviewer can analyze how a person’s behavior fits into an employer’s culture.
Knowing the employer’s cultural tenets, interviewees can generate various stories that perfectly fit the interviewers’ expectations using any popular LLM. However, this approach is limited, and each story can be identified by probing different levels of detail. Communication between LLM and a human is not fast enough to extend the story on the fly (LLM can generate additional context of the story fast enough, but a human needs to read and understand it before serving).
Let’s take a look at the other side of the situation: the interviewee has a story they are proud of, and they want to share it the way the interviewer finds it useful. They can provide LLM with the story, add the employer’s cultural tenets as a context, and ask to re-phrase it the way storytelling highlights data points the interviewer will be looking for. It is a clear win-win case: the interviewee will have more confidence during the session and will be able to respond to any follow-up questions (it is still their own story), and the interviewer will have a story that demonstrates what they are looking for and will have more time to dive into details.
Conclusion
Blindly rejecting the achievements of science was never a good idea for business. Modern AI solutions allow specialists to focus on what they can do best—solving complex problems. Universities graduate specialists with several years of experience using LLMs and neural networks. Tomorrow, AI will become a mandatory part of everyone’s workspace. Do you still have excuses to strictly ban AI usage for interviews instead of training your interviewers on how to benefit from it?
