Test cases help in validating candidates' code. Test cases form the basis for automated evaluation of a candidate’s code. A test case consists of an input to the code and an expected output. Once candidates submit the code, it is run against all the test cases. The output from the candidate’s code is compared with the expected output to see whether the test case has passed or failed.
A sample test case with an explanation can also aid in explaining the problem better to the candidates. Ideally, you should use 2 to 3 such sample test cases that help the programmer to understand the problem. We recommend having a total of 8 to 15 (ideally a total of 10) test cases that cover all the scenarios. The test cases should be of different sizes that cover different levels of complexity. No two test cases should test the same concept.
Types of test cases
There are two types of test cases for each problem.
1. Sample test cases:
Sample test cases are the test cases that the candidates can view and use it to validate their code. They see both input and expected output values for these test cases. While creating the test cases, you can mark certain test cases as sample test cases.
2. Hidden test cases:
All other test cases are hidden from the candidates. Candidates cannot see the input and expected output values for these test cases. They can only see the number of hidden test cases that passed or failed with their code.
The hidden test cases often include corner cases to ensure that the code submitted by the candidates also account for these. The hidden test cases also ensure that candidates write a logical code to solve the problem and not focus on writing a code that generates the expected output, if all test cases are exposed.
- A test case where only a single space is present.
- A test case where only special characters are present in the string
Not all test cases have the same difficulty level. While creating the test cases, you must ensure that more points are assigned to the difficult test cases as compared to the easy ones. This leads to a better distinction between outstanding, good, and average programmers. The sum of scores of all test cases is the total score assigned to a particular question. You can assign zero points to sample test cases, if required. The overall score for a coding question is the sum of the scores of all the test cases which are successfully passed.
Sync Test Cases with Code Stubs
When you are using autogenerated code stubs, ensure that input and expected output are in sync with the code stubs. You can refer to the tail portion to understand this.
If the input is an array, you will always read in "n" (denotes array_size) first and then read "n" array elements in "n" separate lines. This is required to ensure consistency in reading input across different languages.
If the input is a linked list, you will always read in "n" (denotes the number of elements to be added to the input linked list) and then read "n" elements in "n" separate lines.