How to Choose
a Robust Psychometric
Talent Assessment Provider

How to select the best aptitude
and personality tests for your needs

What to Look for in Valuable and Robust Psychometric Assessments

When faced with a variety of talent assessments, it can be difficult to decide which one will provide the most accurate information, deliver the most value for your organization, and result in a great experience for your candidates and employees.

This article outlines criteria for evaluating the quality of a psychometric instrument and considerations for selecting an assessment provider. It also includes a free checklist you can download to help you choose the most effective talent assessments for your needs. Using this information as your guide will help ensure that the instrument you choose, whether delivered online or offline, offers the accuracy, value and fairness needed to best inform your talent decisions.

Across the world, professional organizations such as the International Test Commission (ITC), the Society for Industrial and Organizational Psychology (SIOP) and the European Federation of Psychologists’ Associations (EFPA) have developed and defined consistent standards for psychometric assessments. The guidelines set minimum standards a tool must reach and describe how these tools can be used and by whom.

Looking for the right Psychometric Assessment?
Feel free to contact us and learn more about Aon's Assessment Solutions.

Which Type of Psychometric Test to Use

The main types of psychometric tests include aptitude tests, situational judgment tests, skills tests and personality questionnaires. The type of test or combination of tests you need depends on your organization’s goals and the job-relevant selection criteria critical for success in the role. Aptitude tests add immense value to the application process as they are powerful predictors of long-term job success as they measure ability in specific areas or overall ability. Types of cognitive ability tests include numerical reasoning tests, verbal reasoning tests, inductive reasoning tests, and logical reasoning tests. Situational judgment tests assess how a candidate is likely to behave in real-world job scenarios as well as give the candidate a preview of what the job entails. Skills tests measure a job applicant’s ability to perform or learn specific job-related skills. Personality assessments evaluate candidates’ job-relevant competencies, workstyles and likely behaviors to determine their fit in the organization and role requirements. Data from a job analysis and validated competency model can help determine which assessment or combination of assessments will give you the insights you need to make the best possible talent decisions.

The following sections will help you determine the quality of individual assessments as well as the overall recruitment process.

What makes a psychometric assessment objective?

What it is:
Objectivity refers to the extent to which the test or questionnaire results are unable to be influenced or manipulated by the person scoring the assessment. There are three main points where objectivity comes into play during the assessment process: test administration, test scoring and results interpretation.

Why it’s important:
Every candidate needs to be given a fair shot. An objective and standardized assessment helps ensure test results can’t be manipulated or misinterpreted and are a fair gauge of the candidate’s abilities.

What to look for:

  • Standardized Administration
    The assessment must be administered using standardized instructions and a regulated testing situation.
  • Standardized Scoring
    There should be no way for a candidate’s score to be manipulated or influenced during the scoring process, either manually or by altering the scoring algorithm.
  • Standardized Interpretation
    The same conclusions should always be drawn from any one specific result. And ideally, the score should be compared with the results of a norm group.
Icon Digital Interview Quick Set Up

A test can only be interpreted unambigously if objectivity in all three stages is assured.

What makes a psychometric test reliable?

Reliability, or accuracy, refers to how consistently a psychometric tool measures what it is intended to measure. In other words, does it measure the same aspect consistently again and again? Are the results dependable?

There are different ways of defining reliability, including:

  • Re-test Reliability
    This is calculated when the results from an initial application of the test are correlated with the results of a subsequent application.
  • Alternate Form Reliability
    This is calculated when two versions of the same test are completed by the same people and their scores are compared.
  • Internal Consistency
    Internal consistency (Cronbach’s alpha) measures the correlations between different items of the assessment. It shows whether the items intended to measure the same general construct result in similar scores.

A perfectly reliable test provides a correlation coefficient of 1.00. There are no perfectly reliable instruments, however only reliability values over 0.7 should be accepted. For questionnaires, the ideal value lies between 0.75 and 0.85. For ability tests, it is between 0.8 and 0.9.
 

Why it’s important:
A reliable assessment dependably measures what it’s meant to measure, meaning you can be confident participants would get the same or very similar results if retested or given an alternate version of the test.

What to look for:
You can find information on a psychometric test’s reliability in its technical report. Check to ensure the assessment meets guidelines for statistical reliability according to best practices.

Icon Digital Interview Quick Set Up

A reliable instrument is robust against interference and sensitive to underlying differences in characteristic values.

Is the assessment valid?

What it is:
Validity refers to how well the assessment measures what it is intended to measure. To what degree of certainty can we draw conclusions about how someone will perform on the job based on how well they perform on the assessment?

There are different types of validity, including:

Criterion-related Validity

Criterion-related validity measures the correlation or statistical relationship between test results and job performance. Criterion-related validity ensures the right characteristics, those predictive of performance on the job, are what is being measured.

Concurrent Validity

Concurrent validity refers to the extent that the assessment results correspond to the results of existing assessments or tests that measure the same constructs and have already been found to be valid.

Predictive Validity

Predictive validity is the extent to which a score on a psychometric scale or test predicts scores on some criterion measure. For example, you may look at the predictive validity of a cognitive test for job performance. This is calculated as the correlation between test scores and supervisor performance ratings. With a high level of predictive validity, it is possible to draw conclusions about a candidate’s likely success in a role and organization based on their test results.

Construct Validity

Construct validity is the extent to which the instrument measures what it sets out to measure. It looks at the test results and how these are related to the results of other tests that have proven to be valid indicators of the construct, which could include measures such as external ratings, behavior measurements or experimental results. The construct validation does not end in a validity coefficient but provides an overall picture of validity. In theory, a test should correlate higher with a test that measures approximately the same thing than with a test that measures something else.

Face Validity

Face validity is the extent to which the assessment and its content appear to cover what is intended to be measured, and the extent to which it is perceived as doing so by test takers and stakeholders.

Internal and External Validity

Internal validity refers to the degree of certainty that the test results are trustworthy and are not the result of outside factors. Moreover, an instrument is externally valid if its results can be generalized across different situations. This means that external validity is heavily dependent on the testing situation being as realistic as possible and the representativeness of the tested sample.

Why it’s important:
Validity is the most important factor to consider when selecting a psychometric test, it’s the purpose of implementing an assessment. Validity tells us to what degree of certainty we can use the assessment results to predict performance on the job. It ensures what’s being measured relates to the job qualifications and requirements.

What to look for:
You can find information on a psychometric test’s validity in its technical report. Check to ensure the assessment meets guidelines for statistical validity according to best practices.* If the technical report does not provide evidence that the assessment predicts job performance, it isn’t a valuable assessment for your organization.
 

O*net’s Testing and Assessment: An Employer’s Guide to Good Practices - Click here!
 
Icon Digital Interview Quick Set Up

A valid ability test provides a measure for the general ability of a person and, therefore, helps predict performance in situations in which general ability is important.

What makes a psychometric test valuable?

What it is:
It takes time and money to acquire, license and use assessment systems, test materials and scoring or interpretation tools. To justify the use of a psychometric assessment, you can develop a business case to ensure the value an assessment will bring to your company will exceed the costs.

Why it’s important:
Psychometric tests are an investment of organizational resources. It’s important to have a clear and accurate understanding of the costs and return on investment (ROI) to be confident the assessment will deliver the value and outcomes you are targeting.

What to look for:
There are many factors to consider when appraising the value of an assessment. Make sure the pricing structure is easy to understand and makes sense for your company’s needs. Also make sure you choose an assessment partner who is invested in your success. Look for a partner who will provide ongoing account management and support and will help with change management within your organization. It’s also important to make sure criterion-related validation studies are included in the license to assure the tool is performing for you as expected. Request client case studies to get a clear picture of an assessment’s success. Although there are many aspects to consider, most importantly – make sure the insights you gain will enable you to make better talent decisions.

The benefits an assessment brings to the business should always surpass the costs.

Do assessments contribute to a positive candidate experience?

What it is:
Candidate experience is the perception candidates have about your company based on their interactions during the recruitment and hiring process.

Why it’s important:
A great candidate experience can make your organization a top recruit’s preferred employer – rather than a fallback option. This means organizations can build a lasting competitive advantage by refining their assessment and selection processes. Candidates who have a positive experience are likely to recommend the company to others, regardless if they are offered a job. However, that likelihood drops dramatically when a candidate has an unsatisfactory experience. For many companies the damage of a poor candidate experience isn’t confined to employer brand. Their candidates are their customers and a subpar process can translate to lost sales.

What to look for:
A good assessment process should boost your employer brand and provide a give-and-take between the candidate and your organization, where both walk away with valuable insight and information. Make sure the assessment experience is consistent with your employer brand and is not a barrier for good candidates. Research suggests psychometric tests designed for mobile-first delivery are more accessible for all candidates and result in higher completion rates, especially among minority groups, so look for an assessment experience that is consistent across devices (i.e. on a desktop computer versus a mobile device).

Is the psychometric test fair?

What it is:
A fair psychometric assessment is unbiased and fair to all candidates.

Why it’s important:
Everyone needs to be given a fair shot. Every assessment needs to be the same experience for each candidate, and it is vital that tests are not discriminatory against any group based on race, ethnicity, gender or age.

What to look for:
Check the technical report for evidence that the assessment does not discriminate against any group of people and avoids potential to cause adverse impact. Accessibility guidelines also state that everyone should have the same opportunity to complete psychometric tools. This means the assessment needs to be accessible to people with disabilities – even if this requires additional support. Also check to make sure the assessment provides results that cannot be manipulated or misinterpreted. All applicants must be treated equally and fairly.

Looking for the right Psychometric Assessment?
Download the checklist to help you gauge the quality of psychometric tools!