How to discuss applicant work without reducing it to a score
A practical protocol for an employer who has an applicant’s Replay in front of them and needs to prepare, run and record a follow-up conversation. It has eight steps, a checklist you can copy or print, and a note template. It adds no scores, ratings, weighted rubrics or recommendations.
Summary
- Work through what was available, what the applicant did, what changed and what was left open, before forming a view.
- Pick a small number of specific moments and ask the applicant what they were trying to establish.
- Write your interpretation separately from the evidence, and label it as an interpretation.
- The hiring decision stays with you. Calavion does not currently produce automatic scores, applicant rankings or hiring decisions.
Method and scope
Status: this is a protocol Calavion proposes. It has not been tested with employers and its effect on hiring decisions has not been measured. It is not legal advice.
Basis: what Replay contains, checked against Calavion’s application and product records on 25 September 2026, and general findings on interview structure summarized in the evidence review. Steps 1 to 8 are Calavion’s own design, not a published research method.
What you are reading
Replay shows the applicant’s exact messages, the records they opened, the actions they took, delayed messages and consequences, the recorded state of the case after events, and outstanding work. It is a record for a person to read. It is not a video, a screen capture or an AI judgment, and it does not show private reasoning. The recorded state after an event can include details the applicant never opened, so check what they actually opened before assuming they knew something.
The eight steps
- Review what information was available. Before judging any action, read the starting records and guidance the applicant had, and note which records they opened and in what order. Ask what a reasonable person could have known at each point.
- Review the applicant’s actions and messages. Read their messages as written, in order. Note what was said to whom, and what was left unsaid. Stay with what the record shows.
- Examine what changed. Follow the consequences: what each action changed in the case, and which delayed messages or events arrived because of time passing rather than anything the applicant did.
- Identify unfinished obligations. Note what was left open at the end and whether the applicant left it visible and owned. An open item is not automatically a failure. It may be the responsible outcome, and the conversation can find out.
- Select specific moments for discussion. Choose a small number of moments that raise a real question for you, such as a message, a decision, a record opened or missed, or a point where the case changed. Write each one down with where it appears in Replay.
- Ask the applicant what they were trying to establish. Use open questions about that moment: what they were trying to find out, what they knew and what they were assuming, what they considered doing instead, and what they would do next. Ask every applicant the same core questions.
- Record your interpretation separately from the evidence. Keep two columns or two notes: what the record shows, and what you conclude from it. Label the second as your interpretation. Add what the applicant told you as a third item, attributed to them.
- Keep the final decision with you. Calavion produces no automatic scores, applicant rankings or hiring decisions. Decide using your own process, alongside anything else you are entitled to consider.
Copyable checklist
Copy this into your notes, or use your browser’s print command. The page has a print layout that hides the site header and footer.
Replay discussion checklist
- I read the starting records and guidance the applicant had.
- I noted which records they opened, and in what order.
- I read their messages and actions in order, as written.
- I followed what changed, including delayed events.
- I noted anything left open, and whether it was owned.
- I picked a small number of specific moments and noted where each appears.
- I prepared the same core questions for every applicant.
- I asked what they were trying to establish at each moment.
- I wrote my interpretation apart from the evidence and labelled it.
- I noted what the applicant told me, attributed to them.
- I did not turn notes into numbers, rankings or labels.
- I made the decision myself, using my own process.
Note template
One block per moment discussed
- Moment: where it appears in Replay
- What the record shows: facts only
- What was available at that point: records opened, guidance
- Question I asked: in the applicant’s terms
- What the applicant said: attributed to them
- My interpretation: labelled as interpretation
- What I would still want to know: open questions
How reducing to a score happens by accident
- Counting things, such as messages sent or records opened, and treating the count as quality.
- Treating speed as judgment. A fast response and a careful response are different things.
- Turning a single moment into a label, such as “proactive” or “careless”, and defending the label instead of the evidence.
- Ordering applicants by how comfortable the conversation felt.
- Assuming the historical response, or the one you would have given, is the answer key.
Limitations
- Not validated. Nothing here shows that this protocol improves hiring decisions, or that it treats applicant groups equitably. Calavion has not demonstrated predictive validity, fairness, reduced bad hires or improved outcomes.
- Structure and evidence. Research on interviews associates structure, such as standard questions and consistent evaluation, with higher reliability and validity than unstructured discussion. This protocol is structured in its steps and core questions, but it deliberately has no numeric ratings, so it does not inherit the evidence for scored structured interviews. See the evidence review.
- Reviewer bias. Reading a record does not remove reviewer bias. Using the same steps and questions for every applicant helps consistency but does not guarantee fairness.
- Time. Calavion has no measured figure for how long a Replay review takes and claims none.
- Your obligations. Selection procedures can be subject to equal-opportunity law. Check how adverse impact is assessed in your setting and seek legal advice where needed.