Overview
Test cases help in validating candidates' code. They form the basis for automated evaluation of a candidate's code. A test case consists of an input to the code and an expected output. Once candidates submit the code, it is run against all the test cases. The output from the candidate's code is compared with the expected output to see whether the test case has passed or failed.
A sample test case with an explanation can also aid in explaining the problem better to the candidates. Ideally, you should use 2 to 3 such sample test cases to help the programmer understand the problem. We recommend having a total of 8 to 15 (ideally a total of 10) test cases that cover all the scenarios. The test cases should be of different sizes that cover different levels of complexity. No two test cases should test the same concept.
Note: Please refer to the Test Cases in your Coding Question to understand the usage of Test cases while creating coding questions for HackerRank Tests.
Types of Test Cases
There are two types of test cases for each problem, a Sample Test Case and a Hidden Test Case.
Sample Test Cases
Sample test cases are the test cases that the candidates can view and use to validate their code. They see both input and expected output values for these test cases. While creating the test cases, you can mark certain test cases as sample test cases.
For example, see the illustration below of a sample test case defined for a coding question. The question requires the candidate to write code to find the first non-repeated character in a given string.
In the test, when the candidate writes and compiles the code for the question, the sample test case is validated against the candidate's code. The candidate can view the sample test case execution results along with the input and expected output values as defined by you. Viewing the sample test case execution results helps the candidates understand their code's expected results. Candidates can also download their sample test cases.
The following image displays the candidate's view of sample test cases execution status after compiling the code in the test.
Hidden Test Cases
While adding a test case to your coding question, if you do not mark it as a sample test case, it is considered hidden. Hidden test cases have evolved from the era of competitive programming.
You can include various corner cases as hidden test cases to evaluate candidates against different scenarios. When a candidate runs their code, the hidden test cases are indicated by the lock icon.
Unlike sample test cases, the candidates cannot see the input and expected output values for these hidden test cases. Although, they can see their output and debuggable output when they click on any of the hidden test case options. This feature is provided to candidates to give them a better debugging experience.
Consider a coding question where a candidate is required to write code to find the first non-repeated character in a given string. 
In the test, when a candidate writes and executes the code for this question, the hidden test case is validated against the candidate's code to return the test case execution result. If the candidate's code successfully executes the hidden test cases and returns the expected output, then the candidate achieves the score assigned for this test case.
Disabling the Output for Hidden Test Cases
You can also choose not to show the output of the hidden test cases for the candidates. To disable the output for hidden test cases, follow the below steps.
- On the Tests home page, click on the desired test.
- After clicking on the question, click on the Settings tab and then the Questions option from the left.
- After clicking on the Questions tab, scroll down, you will find the option to disable the output for the hidden test cases as shown below. However, this is not recommended since showing the output helps the candidates to debug their code effectively. The option to show the output is selected by default. If you want to hide your output, you can deselect the Show Output for the hidden test cases checkbox.
Note: Our reports suggest that candidates tend to lose a lot of their test time while finding a solution or debugging because of hidden test cases. To reduce this challenge and give candidates a better debugging experience, HackerRank looks forward to moving away from hidden test cases in the future and introducing a breakpoint-based debuggable experience to candidates.
Scoring
Not all test cases have the same difficulty level. While creating the test cases, you must ensure that more points are assigned to the difficult test cases as compared to the easy ones. This leads to a better distinction between outstanding, good, and average programmers. The sum of scores of all test cases is the total score assigned to a particular question. You can assign zero points to sample test cases if required. The overall score for a coding question is the sum of the scores of all the test cases which are successfully passed.
In the following illustration, you can see the candidate's detailed test report in HackerRank displaying the score achieved for the question based on the test execution status.
Sync Test Cases with Code Stubs
When you are using autogenerated code stubs, ensure that the input and expected output are in sync with the code stubs. You can refer to the tail portion to understand this.
For example:
-
If the input is an array, you will always read in "n" (denotes array_size) first and then read "n" array elements in "n" separate lines. This is required to ensure consistency in reading input across different languages.
-
If the input is a linked list, you will always read in "n" (denotes the number of elements to be added to the input linked list) and then read "n" elements in "n" separate lines.
Refer to the topic How to add a New Test Case for information about creating and adding Test cases to your Questions.