What is a job simulation? Types, examples and what it shows
A job simulation is an assessment that puts an applicant in a realistic piece of the job, recreated or taken from real work, and records what they do instead of what they say they would do. Formats range from scenario questions to full assessment-centre exercises, and they differ in how closely they resemble the work.
What counts as a job simulation?
“Job simulation” is an umbrella term. It covers any exercise that asks an applicant to respond to a job-like situation. The same word is used for a ten-question scenario quiz and for a half-day exercise with role-players, so it helps to name the format.
- Situational judgement test. The applicant reads a described scenario and chooses or ranks responses. This is the lower-fidelity end of the family (Lievens & Patterson, 2011).
- Work-sample test. The applicant performs tasks or work activities that mirror the tasks employees perform on the job. This is how the US Office of Personnel Management (OPM) describes work samples.
- Assessment-centre exercise. A staged exercise, often with role-players or observers. This is the higher-fidelity end of the family (Lievens & Patterson, 2011).
- Interactive environment built from real work. Calavion’s proposed term for a prepared environment, built from work a company has handled, in which the state changes as the applicant acts. This is a product category description, not a scientific classification, and it has no validation evidence of its own.
How do the formats compare?
| Format | What the applicant does | Fidelity to the work | What the employer reads |
|---|---|---|---|
| Situational judgement test | Picks or ranks responses to a described scenario | Lower | Chosen responses |
| Work-sample test | Performs a bounded task that mirrors the job | Medium to high | The finished output |
| Assessment-centre exercise | Works through a staged scenario, often with role-players | Higher | Observer notes and ratings |
| Interactive environment from real work | Reads records, writes messages and takes actions that change the situation | Intended to be high; untested | A record of what the applicant did and wrote |
How is a job simulation different from a work sample, a take-home or a realistic job preview?
A work sample can be delivered inside a simulation, and the two terms overlap. A take-home assignment is a work sample done independently over a stated period. A realistic job preview has a different purpose: it gives applicants an accurate picture of the job so they can decide whether to take it. A simulation can double as a preview, but the two are not the same thing. See work samples, job simulations, take-home assignments and real-work hiring for the full comparison and realistic job preview vs job simulation.
What does the research say about job simulations?
Lievens and Patterson (2011) studied 196 applicants in an advanced-level, high-stakes selection setting. A knowledge test, a lower-fidelity situational judgement test and a higher-fidelity assessment centre were all significantly related to job performance. The situational judgement test and the assessment centre each added predictive value beyond the knowledge test, and the assessment centre added value beyond the situational judgement test. That is a single study in one setting.
Arthur, Day, McNelly and Edens (2003) meta-analysed 34 articles on assessment-centre dimensions and reported estimated true validities from .25 to .39 across six dimension categories. Sackett and colleagues (2022) re-estimated validity across selection methods and put work-sample tests at .33, assessment centres at .29 and situational judgement tests at .26, with most work-sample evidence coming from studies of current employees rather than applicants. Moderate validity is not the same as high validity, and it does not tell you how a particular exercise will perform for a particular employer. The full figures and their limits are in what research says about seeing applicants perform real work.
Where does Calavion fit?
Calavion prepares an exercise with a company from work the company has already handled. Applicants work inside it, and the employer reads a Replay of what the applicant did and wrote. Calavion does not currently produce automatic scores, applicant rankings or hiring decisions. Employers inspect the work and make the decision. Calavion has not demonstrated predictive validity, fairness or improved hiring outcomes, and the only exercise it has run end to end is a customer service cancellation case.
What should you ask before using a job simulation?
- What exactly does the exercise ask the applicant to do, and does that match the job?
- Will every applicant get the same brief, materials and conditions?
- How will the work be reviewed, and by whom, before you invite anyone?
- What is recorded, and have applicants been told?
- What does the vendor claim about validity, and is it backed by a study you can read?
Sources
- Lievens, F., & Patterson, F. (2011). The validity and incremental validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations for predicting job performance in advanced-level high-stakes selection. Journal of Applied Psychology, 96(5), 927–940. doi:10.1037/a0023496
- Arthur, W., Day, E. A., McNelly, T. L., & Edens, P. S. (2003). A meta-analysis of the criterion-related validity of assessment center dimensions. Personnel Psychology, 56(1), 125–153. doi:10.1111/j.1744-6570.2003.tb00146.x
- Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040–2068. doi:10.1037/apl0000994