Overview
With 1 assessment taken every 8 seconds on the HackerRank platform, we have an unmatched database of over 150 million assessment and candidate data points. We use this data, with the help of machine learning and in-depth data analysis, to identify the metrics of an effective test.
The Test Health Dashboard takes these learnings and gives you the power to make meaningful changes to your assessment process:
- Visualize funnel performance from test invite to candidate offers.
- Track a variety of metrics to gain insight into the health of candidate experience and assessment quality for every HackerRank test.
- Understand the underlying drivers of test health and make improvements to attract better, evaluate, and hire talent.
How to view the Test Health Dashboard inside a test
Here is a quick video to get you started with the Test Health Dashboard workflow.
Or, if you prefer to follow directions, here are the steps for your ready reference.
Prerequisites:
- You must own a Recruiter or Developer license for HackerRank for Work.
- You must have a user account to log in to HackerRank for Work.
- It is recommended that your HackerRank account has an active integration with any of the supported Applicant Tracking Systems (ATS).
Note: Without an active ATS integration, your funnel will cover from ‘test invite’ to ‘attempt’ data on the candidate onsite. Analysis of Candidate offers relies on data from the ATS.
Steps:
- Login to HackerRank for Work, and click Tests on the home page.
- Select the relevant Test.
- Click the Health tab to view the “Test Health Dashboard”.
The performance funnel gives you a perspective of how candidates taking the test are progressing through the funnel from the test invite stage to the candidates offered stage. You can also see the contributing factors in a candidate’s response health details.
For example, the Dashboard above shows that the test has a good response rate with Candidates with the invite open rate, and test start rate showing 100%.
Note: By default, the Dashboard displays the “Test Health” for invites sent to Candidates only through email or API requests with email. Change the selection to “All Candidates” if you want to view the Test Health based on invites sent through all communication channels such as inviting Candidates through a public test URL or from the API invitations without email.
The Candidate Response Health (CRH) and the Assessment Quality Health (AQH) will only have data for the last 90 days. The health indicator shows either Red, Green or Dark Green based on whether the health is bad, good or excellent respectively. These category ranges are driven by analysis of the millions of tests on the HackerRank platform.
You can view further details for CRH and AQH to understand the different factors driving overall health.
Candidate Response Health (CRH)
The Candidate Response measures the performance of your candidate outreach by tracking candidate journeys from the stage they are invited to take your technical assessments. It measures the conversion of those delivered invitations to test attempts. We track candidates' response to your invitations so that we can detect where candidates are dropping off - are your emails getting caught in a spam filter? Are candidates reading the email but not motivated to start the test?
Work with your customer success manager to diagnose any problems in your recruiting pipeline and plan how to improve your Candidate Response Health. Focus on increasing developers' interest in your company by investing more in your company's tech talent brand and improving candidate communication. Using Candidate Response health, you can consistently measure the success of your team's efforts and use that data to inform future decisions.
Click View more details to see the rating for different parameters which contribute to the overall Candidate Response Health. On the right, you can also see charts depicting data insights.
Assessment Quality Health (AQH)
The Assessment Quality Health measures how well your tests do their job of evaluating candidates’ strengths and weaknesses. It’s based on a variety of inputs, from the design of your test to candidates’ experiences taking it. Increase the quality of your assessments by re-evaluating your tests: Are they screening for the right skills? Are the test questions relevant to the work that the developer will actually be doing in the role? Are they engaging and appropriately challenging for developers? Are you providing a good candidate experience?
Use the Assessment Quality Health to consistently measure the effectiveness of your recruiting workflow, and identify ways to streamline it.
Click View more details in this section to see the rating of the factors which contribute to the overall AQH. On the right, you can also see charts depicting data insights. Review this data to determine whether the test questions are fit for the candidates’ experience level and the job role.
When you integrate HackerRank with a supported ATS platform, the Test Health Dashboard lets you also see how your tests convert to on-sites, offers, and hires. (Note: This feature is currently in private beta. Share your outcome data to unlock extra analytics capabilities! Reach out to support@hackerrank.com if interested in early access.)
Best Practices and Glossary
How do we calculate Test Health?
Our unique dataset and machine learning algorithms collect multiple inputs from your test as well as other similar data points to estimate the health of each one of the scores below and indicate the health status as Red, Green, and Dark Green.
Usually, there is a delay in updating the health of the metrics (Lagging Window) and this is done to prevent any false alarms. For e.g., the open rate metric will not update the health status to red (action needed) if it’s below a threshold for 3 days because the 90th percentile of the candidate takes at least 3 days to open the invitation email.
Metrics |
Lagging Window |
Delivery Rate |
None |
Open Rate |
3 days after the invitation |
Click Rate |
7 days after the invitation |
Start Rate |
7 days after the invitation |
CRH |
7 days after the invitation |
The sections below include a description of the CRH and AQH inputs and some guidelines on actions to take when specific metrics are underperforming.
Inputs to Candidate Response Health (CRH)
Delivery Rate |
|
Description | Measure and actions to optimize |
How many of your invite emails reach their destination? The first step in assessing your candidates is making sure they receive their test invitation. The delivery rate shows the percentage of candidates who received the invite in their inbox, without it bouncing or getting marked as spam. Make sure your emails are reaching candidates to keep your candidate pipeline healthy. |
Measure = Invite emails delivered to candidates* / Total invites sent.
The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD. *This can be lower than the actual number because this is dependent on the external email relay server. |
Open Rate |
|
Description | Measure and actions to optimize |
How many of your invite emails do candidates open? Before candidates can take your test, they need to notice the invitation email in their inbox and feel compelled to read it. Your invite open rate shows the percentage of candidates who receive your invite and open it, as opposed to ignoring or deleting it. For a healthy open rate, ensure that you’ve caught the candidate’s attention at an earlier stage of the recruiting process so that they are excited to receive your email.
|
Measure = Invite emails opened by candidates* / Total invites delivered.
*This can be lower than the actual number because of the Adblocker installed on candidates' computer. |
Click Rate |
|
Description | Measure and actions to optimize |
How many of the candidates who opened your email invite click the link to take your test? The click rate shows the percentage of candidates who clicked the link to take your test after opening the invite email, indicating that you’ve prepared your candidates well. Once a candidate opens your invite email, you want them to feel ready to take the test. You can do that by discussing the test and your expectations for the candidate in a preparatory phone call, or personalizing the email based on the candidate’s background. |
Measure = Test links clicked by candidates/Total Invites opened.
The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD. |
Test Start Rate |
|
Description | Measure and actions to optimize |
How many candidates who visited the test landing page started the test? For all of the candidates who open your invite email and click the link to take your test, how many of them take that final step and start the test? The candidates you lose at this step may have been intimidated by the test landing page or are unsure of what to expect from the test. Help candidates get over their fear by ensuring that they feel comfortable with taking the test. |
Measure = Tests started by candidates / Total test links clicked.
The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD. |
Candidate Response Health (CRH) |
|
Description | Measure and actions to optimize |
How many of the candidates who received your invitation email end up taking the test? This metric shows the percentage of all of your invited candidates who started the test, whether or not there was email tracking for that candidate. It helps you see a more comprehensive picture of the overall health of your invites |
Measure = Invited candidates who start the test / Total invites delivered Look at each of the metrics that lead to this one - including open rate, click rate - to see where there is room for improvement. The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD. |
Inputs to Assessment Quality Health (AQH)
Test Score Distribution |
|
Description | Measure and actions to optimize |
How well does your test separate qualified and unqualified candidates? This metric is based on the distribution of scores candidates receive on the test—is it a bell curve, a U-shape, or something in between? Tests with a wide score spread, that are not too easy or too hard, are more likely to lead to a positive hiring outcome. This metric rewards tests with a wide range of scores and are heavy on tests where candidates tend to receive mostly high or mostly low scores. |
Measure = Spread of test scores
A good test is a combination of the spread of the scores and how efficiently it can identify the right candidates.
Optimum distribution should look like one of the charts below. Although the bar is high, it is efficiently able to identify the strong candidates Efficiently separates the candidates Efficiently separates the candidates Not only does it efficiently separates the candidates, but it is also able to separate the different performance groups. |
Candidate Feedback |
|
Description | Measure and actions to optimize |
What is the average star rating that candidates give your test? This metric gives the average rating across all candidates who chose to provide feedback. You can use a test’s average feedback rating to compare it to your other tests, identify which tests are least successful, and target those tests for improvement. Candidate feedback is a useful metric for gauging the overall candidate experience of your test. After taking a test, candidates can provide a star rating on a scale of 1 to 5. |
Measure = Average test feedback star rating out of 5
The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD. |
Attempt Duration Distribution |
|
Description | Measure and actions to optimize |
How long do candidates take to finish your test?
Deciding how long you should give candidates to finish your test is a tricky problem. If your test is too long, you slow down your hiring process, and some candidates will feel that they have to devote too much time to your test. If your test is too short, candidates will have a stressful experience and feel that their performance doesn’t represent their true ability. The attempt duration metric helps you determine whether your test’s length is well-calibrate, not too long, and not too short.
|
Measure = Spread of attempt duration
|
No of Attempts | Attempts Duration Distribution |
1-5
|
Good: Not in Action Needed Action Needed: 100% of the candidates submitted in less than 25% of the total time. |
6-10 |
Good: Not in Action Needed Action Needed: 40% of the candidates submitted in less than 50% of the total time or more than 85% of the candidates submitted at the last minute. |
11-50 |
Good: Not in Action Needed Action Needed: 50% of the candidates submitted in less than 50% of the total time or more than 80% of the candidates submitted at the last minute. |
51-200 |
Good: Not in Action Needed Action Needed: 60% of the candidates submitted in less than 50% of the total time or more than 80% of the candidates submitted at the last minute. |
200+ |
Good: Not in Action Needed Action Needed: 70% of the candidates submitted in less than 50% of the total time or more than 75% of the candidates submitted at the last minute. |
Test Attempt Rate |
|
Description | Measure and actions to optimize |
Do your candidates give the test a try, or do they give up without answering any questions?
A test’s attempt rate shows how many candidates started the test and submitted an answer to at least one question. This metric measures the initial impression your test gives to candidates when they first see it: does it seem reasonable, or does it overwhelm candidates? Another way of thinking about a test’s attempt rate is by considering its inverse, the “abandonment rate”—how many candidates gave up the test even before answering a single question?. |
Measure = Candidates who answer at least one question/candidates who started the test.
The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD. |
Completion Rate |
|
Description | Measure and actions to optimize |
How many of the candidates who start the test answer every question? A test’s completion rate shows how many candidates are willing to see it through to the end, submitting an answer to every question on the test. High completion rates mean that your test is well-calibrated to your candidates’ abilities, while low completion rates can signal that candidates got stuck or frustrated midway through the test-taking process. |
Measure = Candidates who answer every question/candidates who started the test
The optimum score range will be calculated dynamically based on the number of invites and will be available under the test in THD. |
Assessment Quality Health (AQH) |
|
Description | Measure and actions to optimize |
How well do your tests do their job of evaluating candidates’ strengths and weaknesses? It’s based on a variety of inputs, from the design of your test to candidates’ experiences taking it. Increase the quality of your assessments by re-evaluating your tests: Are they screening for the right skills? Are the test questions relevant to the work that the developer will actually be doing in the role? Are they engaging and appropriately challenging for developers? Are you providing a good candidate experience? Use the Assessment Quality Health to consistently measure the effectiveness of your recruiting workflow, and identify ways to streamline it. |
Measure = Combination of candidate feedback and score distribution |
Comments
0 comments
Article is closed for comments.