back

Development & Retention / Hiring

AI in Assessment - 8 Questions Talent Professionals Must Answer

Artificial intelligence, or AI, is already embedded in our work lives. Automation minimizes some of our repetitive tasks, and chatbots give us extra support when we need it. But with more companies adopting AI in workforce technology, there’s also more debate about how AI affects hiring decisions.
 
AI-based assessments are designed to help you identify the best candidate based on objective performance criteria. Ideally, AI in assessment is used to prevent human bias having an undue impact on hiring decisions.
 
The use of AI in assessment isn’t a new trend. For years, personality questionnaires have been scored and “interpreted” by algorithms developed by industrial/organizational psychologists. But AI in assessment is growing at a rapid pace, and more companies are integrating the technology into their hiring process.
 
You’re likely to encounter questions from stakeholders regarding the usefulness, objectivity and safety of AI in assessments. Here are eight fundamental questions you must be able to answer to dissuade fears and ensure confidence in AI-driven assessments.

How Is AI Used in Assessment?

AI is a part of many candidate and employee psychometric assessments. The use of AI in assessment now regularly informs HR and talent decisions. AI scores a candidate based on their responses to a test and interprets their score to predict behavior on the job.
 
AI in assessments can be used to administer situational judgment tests using realistic, chatbot-style conversations with candidates, for example. The AI program collects, scores and assesses candidate responses to predict how they might react in similar situations.
 
AI can also analyze a candidate's responses to test questions, then use proven algorithms to make predictions based on that data. Hiring managers use AI-driven predictions to make objective hiring decisions regarding the best-fit candidate for the role.

Is AI Already Used in Our Talent Assessment?

AI has been used in talent assessments for decades but has ramped up significantly in recent years.
AI was first implemented in assessments in the 1990s. Paper-based versions of tests moved to computers, which automatically scored the tests and generated interpretive reports. For the first time, technology was taking on some routine tasks and using algorithms to produce a candidate report.
 
AI in assessment has come a long way since those days. Linear on the fly testing, or LOFT, and adaptive testing are recent innovations that improve the testing experience. LOFT generates unique questions pulled from a robust question bank. No two candidates will see the same test.
 
Adaptive testing adapts to candidate responses to generate questions at their level. This prevents candidates from disengaging because the questions are too easy.

What Do All of the Different AI Terms Mean?

There are dozens of technical terms being used to identify specific uses of AI in assessments, which can be overwhelming. Here are a few key terms that you need to know.
 
  • Robotic process automation: RPA is achieved by gathering and transferring expert knowledge. RPA tools are programmed to follow an “if/then” rule-based approach. Unlike more advanced forms of AI, rule-based systems aren’t capable of learning and improving without explicit instructions. In talent assessment, computer-generated interpretative reports make use of RPA technology.
 
  • Machine learning: Even though a computer can’t think for itself, statistical tools enable a system to model predictions from data. You can add to the model to improve its predictions over time. Machine learning is used in data analysis to create predictive people analytics, which helps employers make better talent decisions.
 
  • Pattern matching: This AI technique uses a computer to check the sequence of responses to determine whether there’s a pattern. It can be used to carry out some “human” tasks, such as recognizing faces or identifying emotions.
 
  • Natural language processing: This AI technique uses text and speech analytics to extract underlying meaning from spoken language. Natural language processing is most often applied to analyze speech in video interview assessments.
 
AI in assessments plays a key role in analyzing and interpreting vast amounts of candidate data.

How Can AI Improve Candidate and Employee Assessment?

There are five key benefits of using AI in assessment.
 

Precision

AI can analyze massive amounts of data — far more than any human. The increased power of today’s computers means more candidate data can be precisely evaluated to help make better selection decisions.

Efficiency

AI enables talent teams to consistently and objectively assess candidates for job-relevant skills and traits at an early stage. For example, AI lets you use video interviews far earlier in the selection process than in-person interviews.

Reducing Bias

Human biases and stereotypes are often to blame for poor selection decisions. In theory, AI’s objectivity helps recruiters eliminate conscious and unconscious bias. However, in reality, you have to be careful about how your AI system is programmed.
 
An algorithm is only as good as the data that’s fed into it. Your AI design shouldn’t mimic just one assessor and all of its biases. It has to draw from several assessors to reduce subjectivity.

Legally Defensible

Your assessment AI must be transparent and open to challenge. Complex “black box” algorithms make selection decisions difficult to justify. It’s almost impossible to understand how their conclusions are reached. And if your selection decisions can’t be easily explained, they can be challenged in court.
 
Allowing AI to continue learning by “observing” the best practices of human raters offers the best and most legally defensible approach.
 

Engagement

AI can significantly improve the candidate experience. Interactive chatbots, for instance, can answer queries about the selection process or about specific assessments.
 
AI can also improve the experience for candidates by allowing open-ended responses in personality questionnaires and situational judgment tests. Faster decisions, reduced bias and enhanced assessments all are ways that AI can improve the experience for job-seekers.

How Does AI Support Video interviewing?

Video interviewing typically involves candidates recording themselves responding to competency-based interview questions. There are several benefits to using video interviews:
 
Candidates no longer need to travel for interviews. Since you can conduct a video interview from anywhere, you can save on travel costs. Video interviewing also allows you to reach more candidates, especially those with disabilities or who can’t travel for an interview.
 
Interviewers get to rewatch and share candidate responses. Since responses are recorded, more interview panelists and hiring managers can participate in the selection process. This enables you to ensure that more diverse panelists are involved in the hiring decision, which can disrupt bias.
 
Less time is spent on the interview itself. A pre-recorded video interview is directed by the candidate and takes much less time than a standard in-person interview. Traditionally, however, hiring managers have spent a considerable amount of time analyzing video interview responses.
 
AI in assessments enables recruiters and hiring managers to quickly and objectively assess candidate responses to a video interview. The AI program transcribes the candidate’s recorded responses and analyzes them for clarity of speech and language proficiency. The AI program scores candidates transparently to ensure legal defensibility.

What Are the Challenges with Using AI in Assessment?

Beyond the technological considerations, there are four challenges that must be addressed:
 

Defensibility

 
Be careful which AI in assessment you use. There are several standardized “plug-and-play” AI systems on the market. These systems use “deep learning networks” that learn as they’re exposed to new data inputs.
 
This sounds promising, but such a system can create difficulty in explaining why a candidate was accepted or rejected for a role. Standardized systems can lead to selection decisions you can’t defend, which leaves you vulnerable to litigation.
 
Only custom AI systems offer the ability to make transparent and defensible selection decisions.
 

Time

Custom AI systems mirror human behavior and replicate the best practices of your assessors and raters to minimize bias. To achieve this, though, you have to pre-feed the system relevant information. It can take up to six months to “train” an AI system to assess candidates like your assessors would. Managing this lead time can be a major challenge but will result in better hiring decisions in the long term.
 

Ethics

Using AI in assessments raises ethical concerns, especially when determining what safeguards and controls to implement in the process. For example, should you allow an AI system to reject candidates? Or should it flag unsuitable candidates so you can review and check their details? Learning how to use AI ethically is a key consideration for many employers.
 
AI’s role should be restricted to providing additional information and enhancing efficiency, not making decisions. Recruiters should always set the objectives when hiring. AI can then deliver useful information at various stages of the selection process to support a final decision.
 

Data Handling

AI excels at analyzing massive amounts of data, but the results can be misinterpreted or even deliberately abused. Good data-handling practices will be essential not only for maintaining confidentiality but also for preserving your company’s reputation. Use AI carefully and transparently to minimize potential misuse of candidate data.

How Can We Make Sure AI in Assessment is Legally Defensible?

The process of selecting candidates — whether for entry into the organization or promotion within — must always be legally defensible. AI systems shouldn’t discriminate. Your assessment processes shouldn’t favor a particular candidate based on gender, race or any other protected aspect of identity.
 
Confidentiality is a legal concern, too. Candidates have the right to be informed of how their personal data and assessment information will be used. Under the EU’s General Data Protection Regulation, for example, you need to ensure transparent use of data.
 
Candidates need to know how and why the assessment is being used. If your assessment constructs a personality profile, candidates should be informed how that will be used to make hiring decisions.
 
Although the U.S. isn’t governed by a federal data protection law, implementing transparency can ensure data privacy and legal defensibility.

How Is AI in Assessment Likely to Develop?

We’ve come a long way in applying AI in assessment. The process will only speed up and become more refined as further advances are made.
 
Assessment developers are already looking at how AI can help interpret responses to open-ended questions. These might be in traditional personality questionnaires or other progressive assessments, like those based around messaging apps. Soon real-time interviews could be carried out by an avatar or with the avatar observing a hiring manager’s interview.
 
AI in assessment minimizes administrative tasks and reduces hassles for HR decision makers while enabling humans to control the process. AI in assessment enables candidates to respond in a more natural way instead of being restricted to written tests.
Aon

Aon | Assessment Solutions

Aon's Assessment Solutions provides clients with powerful tools and insights to help them make better talent decisions at every stage of the employee lifecycle. This includes pre-hire assessments, identifying future leaders, screening for digital skills and agility, and AI-enabled solutions.

Share article:

 

General Disclaimer
The information contained herein and the statements expressed are of a general nature and are not intended to address the circumstances of any particular individual or entity. Although we endeavor to provide accurate and timely information and use sources we consider reliable, there can be no guarantee that such information is accurate as of the date it is received or that it will continue to be accurate in the future. No one should act on such information without appropriate professional advice after a thorough examination of the particular situation.

Terms of Use
The contents herein may not be reproduced, reused, reprinted or redistributed without the expressed written consent of Aon, unless otherwise authorized by Aon. To use information contained herein, please write to our team.