Test Health Dashboard

How many of your invite emails reach their destination?

The first step in assessing your candidates is making sure they receive their test invitation. The delivery rate shows the percentage of candidates who received the invite in their inbox, without it bouncing or getting marked as spam. Make sure your emails are reaching candidates to keep your candidate pipeline healthy.

Measure = Invite emails delivered to candidates* / Total invites sent.

• Double-check your contact list to be sure you are sending your invites to the right email addresses.
• Avoid using words common in spam messages, including exclamation points and excessive capitalization.
• Try using a different email address or domain to send your invites from.

The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD.

*This can be lower than the actual number because this is dependent on the external email relay server.

How many of your invite emails do candidates open?

Before candidates can take your test, they need to notice the invitation email in their inbox and feel compelled to read it. Your invite open rate shows the percentage of candidates who receive your invite and open it, as opposed to ignoring or deleting it. For a healthy open rate, ensure that you have caught the candidate’s attention at an earlier stage of the recruiting process so that they are excited to receive your email.

Measure = Invite emails opened by candidates* / Total invites delivered.

• Catch the candidate’s attention with an intriguing or personalized subject line in your invite email.

• Improve your sourcing to find candidates who are invested in your company and therefore more likely to open and read your invite emails.
  
  

  The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD.

*This can be lower than the actual number because of the Adblocker installed on candidates' computer.

How many of the candidates who opened your email invite click the link to take your test?

The click rate shows the percentage of candidates who clicked the link to take your test after opening the invite email, indicating that you have prepared your candidates well.

Once a candidate opens your invite email, you want them to feel ready to take the test. You can do that by discussing the test and your expectations for the candidate in a preparatory phone call, or personalizing the email based on the candidate’s background. 

Measure = Test links clicked by candidates/Total Invites opened.

• Reach out to candidates before sending the invite and keep them informed.
• Make your invite email sound compelling.
• Ensure your email template contains your unique company branding and key messaging.

The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD.

Test Start Rate

How many candidates who visited the test landing page started the test?

For all of the candidates who open your invite email and click the link to take your test, how many of them take that final step and start the test? The candidates you lose at this step may have been intimidated by the test landing page or are unsure of what to expect from the test.

Help candidates get over their fear by ensuring that they feel comfortable with taking the test.

Measure = Tests started by candidates / Total test links clicked.

• Add detailed instructions, clear expectations, and inviting content to your test’s landing page so that candidates are eager to start the test.
• Encourage recruiters to talk with candidates beforehand about what the test will be like, including how long it will take, what subjects will be covered, and what you are looking for.

The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD.

Candidate Response Health (CRH)

How many of the candidates who received your invitation email end up taking the test?

This metric shows the percentage of all of your invited candidates who started the test, whether or not there was email tracking for that candidate. It helps you see a more comprehensive picture of the overall health of your invites

Measure = Invited candidates who start the test / Total invites delivered

Look at each of the metrics that lead to this one - including open rate, click rate - to see where there is room for improvement.

The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD.

Test Score Distribution

How well does your test separate qualified and unqualified candidates?

This metric is based on the distribution of scores candidates receive on the test—is it a bell curve, a U-shape, or something in between? Tests with a wide score spread, that are not too easy or too hard, are more likely to lead to a positive hiring outcome. This metric rewards tests with a wide range of scores and are heavy on tests where candidates tend to receive mostly high or mostly low scores.

Measure = Spread of test scores

A good test is a combination of the spread of the scores and how efficiently it can identify the right candidates.

• If candidates tend to receive similar scores on your test, the test is not doing a good job of differentiating qualified and unqualified candidates. Add more questions or exchange the existing questions for ones better suited to the role to get a better idea of how candidates will do on the job.
• If most candidates are getting high scores on your test, the test is too easy, and it will be difficult to efficiently identify the right candidates. Include more questions, or use questions that are more difficult in order to identify the most skilled candidates.

Optimum distribution should look like one of the charts below.

Although the bar is high, it is efficiently able to identify the strong candidates

Efficiently separates the candidates

Efficiently separates the candidates

Not only does it efficiently separates the candidates, but it is also able to separate the different performance groups.

What is the average star rating that candidates give your test?

This metric gives the average rating across all candidates who chose to provide feedback. You can use a test’s average feedback rating to compare it to your other tests, identify which tests are least successful, and target those tests for improvement.

Candidate feedback is a useful metric for gauging the overall candidate experience of your test. After taking a test, candidates can provide a star rating on a scale of 1 to 5. 

Measure = Average test feedback star rating out of 5

• Consider whether your test is well-suited to the particular candidates you invited. Even well-designed tests can have low feedback ratings if they are not what a candidate was expecting.

The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD.

Attempt Duration Distribution

How long do candidates take to finish your test?

Deciding how long you should give candidates to finish your test is a tricky problem. If your test is too long, you slow down your hiring process, and some candidates will feel that they have to devote too much time to your test. If your test is too short, candidates will have a stressful experience and feel that their performance does not represent their true ability. The attempt duration metric helps you determine whether your test’s length is well-calibrate, not too long, and not too short.

Measure = Spread of attempt duration

• If candidates are finishing your test too quickly, it might be too easy. Consider adding questions, increasing the difficulty of the questions, or using questions better suited to more experienced candidates such as HackerRank’s Role-Based Assessments.
• If candidates are spending close to the full allotted time on your test, it is likely that they are not finishing because of the time limit rather than because they do not know the material. You may be missing out on qualified candidates. Consider including fewer questions, or switching out a question for one that is less time-intensive.

Good: Not in Action Needed

Action Needed: 100% of the candidates submitted in less than 25% of the total time.

Good: Not in Action Needed

Action Needed: 40% of the candidates submitted in less than 50% of the total time or more than 85% of the candidates submitted at the last minute.

Good: Not in Action Needed

Action Needed: 50% of the candidates submitted in less than 50% of the total time or more than 80% of the candidates submitted at the last minute.

Good: Not in Action Needed

Action Needed: 60% of the candidates submitted in less than 50% of the total time or more than 80% of the candidates submitted at the last minute.

Good: Not in Action Needed

Action Needed: 70% of the candidates submitted in less than 50% of the total time or more than 75% of the candidates submitted at the last minute.

Test Attempt Rate

Do your candidates give the test a try, or do they give up without answering any questions?

A test’s attempt rate shows how many candidates started the test and submitted an answer to at least one question.

This metric measures the initial impression your test gives to candidates when they first see it: does it seem reasonable, or does it overwhelm candidates?

Another way of thinking about a test’s attempt rate is by considering its inverse, the “abandonment rate”—how many candidates gave up the test even before answering a single question?.

Measure = Candidates who answer at least one question/candidates who started the test.

• Consider whether your test is too long, difficult, confusing, or intimidating. Check the number of questions, the complexity of the question descriptions, and their overall difficulty.
• Ensure that the test is relevant to your candidates’ background in terms of subject matter, experience level, and skills tested. For example, experienced developers may be less inclined to answer simple algorithm questions, so you could use HackerRank’s projects to give them a more appropriate challenge.

The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD.

Completion Rate

How many of the candidates who start the test answer every question?

A test’s completion rate shows how many candidates are willing to see it through to the end, submitting an answer to every question on the test. High completion rates mean that your test is well-calibrated to your candidates’ abilities, while low completion rates can signal that candidates got stuck or frustrated midway through the test-taking process.

Measure = Candidates who answer every question/candidates who started the test

• One or more of your questions may be too time-consuming, confusing, or difficult for your particular pool of candidates. Consider removing or replacing any questions that candidates tend to skip.
• Consider altering your sourcing pipeline to target candidates with the particular skills that the questions of your test evaluate.

The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD.

Assessment Quality Health (AQH)

How well do your tests do their job of evaluating candidates’ strengths and weaknesses?

It is based on a variety of inputs, from the design of your test to candidates’ experiences taking it.

Increase the quality of your assessments by re-evaluating your tests: Are they screening for the right skills? Are the test questions relevant to the work that the developer will actually be doing in the role? Are they engaging and appropriately challenging for developers? Are you providing a good candidate experience? Use the Assessment Quality Health to consistently measure the effectiveness of your recruiting workflow, and identify ways to streamline it.

Measure = Combination of candidate feedback and score distribution