From the Journals: Multimedia-Based Simulations in Your Pre-Hire Assessments

September 4, 2015

In the September 2015 issue of the International Journal of Selection and Assessment, the value of job simulation was reviewed in an in-depth study by five researchers from the US Customs and Border Patrol. This study examined the effectiveness of how simulations can have a positive impact on an organization’s talent acquisition strategy. The study found that simulations better predicted performance in training and on the job, especially when it related to social and interpersonal traits. This research study argues that a job simulation exercise in the hiring process is an incredibly powerful and accurate pre-hire assessment tool that should be utilized for positions such as contact center agents and retail employees.

Key Takeaways

talent acquisition pre-hire assessments job candidates job simulations video based testing video simulationsThe study divides pre-hire assessment simulations into three categories:

  • Low Fidelity: Text-based simulations that are only written or spoken descriptions of the task and the candidate’s response.
  • Medium Fidelity: Multimedia-style simulations that have visual situations presented to the candidate.
  • High Fidelity: The use of assessment centers to fully simulate job situations, oftentimes using role-playing actors that would demand candidates produce the same steps and actions that they would on the job.

Of these three categories, the study focuses on medium fidelity simulations. This is important for you because the medium fidelity simulation offers the best cost/benefit trade-off for high volume hiring. The study focused on four advantages of using a simulation tool in the pre-hire assessment process, namely:

  1. A reduction in cognitive load: Simulations eliminate the need for applicants to “read scenarios and then imagine the scenario in their mind…[and] the use of vocabulary are reading comprehension, which are cognitively loaded constructs.”
  2. Increased content validity: By having an applicant’s responses be more free form (as opposed to limited to a pre-listed set of options the job candidate must pick from), multimedia simulations are able to provide more content validity, as they observe “an applicant’s verbal and nonverbal response to the [depicted] scenario.”
  3. Vivid depictions and measurement of emotions in interpersonal situations: Through the use of simulations, organizations are able to “better [depict] vivid emotional situations.”
  4. Measuring demonstrated performance versus indicated performance: Demonstrated performance (“a sampling of an applicant’s behavioral performance in a job-related situation”) are more accurately measured through simulations since they measure what a candidate will “actually say or do in a situation and, thus, are measures of demonstrated performance.”

Based on these benefits, the study formed three hypotheses:

  • First, that multimedia simulations “will demonstrate high inter-rater agreement”
  • Second, that “criterion-related validity for both training and performance” will be more accurately measured
  • Finally, that a simulation will “demonstrate significant utility, despite the higher development costs and the investment required for equipment and supplies.”

So, in other words, the study set out to prove that the use of simulations were more accurate to the candidate’s ability to perform the job, had a more accurate depiction of the candidate, and – despite being more expensive – was a better investment in the long term for an organization’s hiring assessment strategy.

Through the study of 439 participants in the criterion-related validity study, the research study was able to demonstrate that all three of their hypotheses were supported. By having all three hypotheses supported, the article concluded that “[multimedia simulations] demonstrated impressive utility” as well as encouraging “raters to provide more accurate ratings through enhanced perceptions of accountability.”

What it Means For You – Talent Acquisition, Job Candidates, and Pre-Hire Assessments

Utilizing a job simulation is one of the best assessments you can leverage when evaluating a candidate’s ability to perform the job they’ve applied for. By challenging the candidate with situations they’ll be facing in the position they’ve applied for, you’re giving the candidate the chance to directly demonstrate whether or not they have the capabilities to do the job at hand, while providing your organization with qualitative data that can be analyzed to help make an informed decision on whether or not the candidate should be hired.

Through the use of multimedia job simulations, you’ll be able to get a more accurate depiction of the candidate’s ability to perform than if you were to simply administer a written exam with multiple choice options. This is because simulations are proven to show demonstrative performance capabilities, as well as provide insights that written simulations cannot, such as emotional and physical reactions to various situations. Simulations can be expensive to develop on your own but other options do exist that will allow you to achieve much better results in your hiring decisions and get a better ROI on your candidates, since the tools you’ll be using are much more insightful with more in-depth feedback.


Business & Finance Articles on Business 2 Community


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.