Test cases help in validating candidates' code. Test cases form the basis for automated evaluation of a candidate’s code. A test case consists of an input to the code and an expected output. Once candidates submit the code, it is run against all the test cases. The output from the candidate’s code is compared with the expected output to see whether the test case has passed or failed.
A sample test case with an explanation can also aid in explaining the problem better to the candidates. Ideally, you should use 2 to 3 such sample test cases that help the programmer to understand the problem. We recommend having a total of 8 to 15 (ideally a total of 10) test cases that cover all the scenarios. The test cases should be of different sizes that cover different levels of complexity. No two test cases should test the same concept.
Types of test cases
Sample test cases are the test cases that the candidates can view and use it to validate their code. They see both input and expected output values for these test cases. While creating the test cases, you can mark certain test cases as sample test cases.
For example, see the illustration below of a sample Test case defined for a coding Question. The Question requires the Candidate to write code to find the first non-repeated character in a given string. You can define a sample test case for this Question as shown below:
In the Test, when the Candidate writes and compiles the code for the Question, the sample Test case is validated against the Candidate's code. The Candidate can view the sample Test case execution results along with the input and expected output values as defined by you. Viewing the sample test case execution results help the Candidates to understand the expected results from their code. Candidates can also download your sample test cases.
The following image displays the Candidate's view of sample test cases execution status after compiling the code in the Test.
All other test cases which you define are hidden from the candidates. Candidates cannot see the input and expected output values for these test cases. They can only see the number of hidden test cases that passed or failed with their code.
The hidden test cases often include corner cases to ensure that the code submitted by the candidates also account for these. The hidden test cases also ensure that candidates write a logical code to solve the problem and not focus on writing a code that generates the expected output if all test cases are exposed.
- A test case where only a single space is present.
- A test case where only special characters are present in the string
Consider a coding Question where a Candidate is required to write code to find the first non-repeated character in a given string. You can define a hidden test case for this Question as shown below.
In the Test, when a Candidate writes and executes the code for this Question, the hidden test case is validated against the Candidate's code to return the test case execution result. If the Candidate's code successfully executes the hidden Test cases and returns the expected output, then Candidate achieves the score assigned for this Test case.
Note: For hidden test cases, the Candidate can only see the test cases execution results and not the input and expected values of the test case.
Not all test cases have the same difficulty level. While creating the test cases, you must ensure that more points are assigned to the difficult test cases as compared to the easy ones. This leads to a better distinction between outstanding, good, and average programmers. The sum of scores of all test cases is the total score assigned to a particular question. You can assign zero points to sample test cases if required. The overall score for a coding question is the sum of the scores of all the test cases which are successfully passed.
In the following illustration, you can see the Candidate's detailed Test Report in HackerRank displaying the score achieved for the Question based on the Test execution status.
Sync Test Cases with Code Stubs
When you are using autogenerated code stubs, ensure that input and expected output are in sync with the code stubs. You can refer to the tail portion to understand this.
If the input is an array, you will always read in "n" (denotes array_size) first and then read "n" array elements in "n" separate lines. This is required to ensure consistency in reading input across different languages.
If the input is a linked list, you will always read in "n" (denotes the number of elements to be added to the input linked list) and then read "n" elements in "n" separate lines.
Refer the topic How to add a New Test Case for information about creating and adding Test cases to your Questions.