Test cases are crucial to assessing a candidate's code. They consist of specific inputs and expected outputs that verify the code’s correctness and functionality. By covering various scenarios, including edge cases and typical use cases, test cases help ensure the code behaves as intended.
When a candidate compiles their code, each test case checks the code's execution status and results. The score is based on how many test cases the code passes successfully.
Types of Test Cases in Coding Questions
Sample Test Cases
Sample test cases are visible to candidates. They display the input and expected output values, allowing candidates to validate and debug their code. Marking test cases as sample test cases during creation enables candidates to view execution results and download them if needed.
Example Use Case: For a problem requiring the identification of the first non-repeated character in a string:
- The sample test case helps candidates understand the expected solution approach.
- Candidates see the input, expected output, and their code’s execution results.
Importance of Sample Test Cases
Sample test cases enhance the clarity of problem statements by illustrating specific examples. Providing 2–3 sample test cases helps candidates grasp the problem requirements effectively. We recommend including 8–15 test cases per coding question to ensure comprehensive evaluation, with 10 being the ideal number. These test cases should:
- Cover varied scenarios and complexities.
- Avoid redundancy by testing distinct concepts.
Hidden Test Cases
Hidden test cases evaluate candidates on diverse edge cases without revealing input or expected output. Not marking test cases as a sample test case will hide the test cases from the candidates and will be indicated by a lock icon. These cases:
- Challenge candidates to write robust code capable of handling edge scenarios.
- Display only the candidate's output and debug messages during execution.
Example Use Case: For the same problem of identifying the first non-repeated character:
- Hidden test cases validate the solution’s correctness under corner conditions, such as empty or long strings.
Tip: You can disable the output visibility for hidden test cases via the Questions tab in the test settings. However, enabling output is recommended for an improved debugging experience.
Creating a New Test Case
1. When creating a new coding question or editing an existing one, you will need to follow the steps for creating a new question and eventually come to step 4, which is adding the Testcases.
2. In the Testcases section, you can Upload Solutions and Add test cases one by one or in bulk by uploading a Zip File.
3. While adding a test case, enter the mandatory details such as name, difficulty, score, input, and output.
4. Select Mark as a sample test case only if you want to identify a test case as a sample type. You can add the required number of sample test cases to a question and choose not to assign scores for them.
5. Similarly, you can add the required number of hidden test cases to the question (uncheck "Mark as a sample test case"). Hidden test cases should contain edge case scenarios to validate the candidates' code solution. When the candidate compiles the code, the hidden test cases will be executed to show the status and help the candidate to know if their code returned the expected results.
5. To avoid EOF issues, we recommend all test cases that require a multi-value input to define the input in the format suggested below:
|
number of lines (ex. 4) value of line 1 (ex. 100) value of line 2 (ex. 500) value of line 3 (ex. 40) value of line 4 (ex. 200) |
6. Leverage our code stub generator to manage the input parsing for candidates. The test case input format for our code stubs is as below:
|
5 separate input parameters e.g. foobar(a, b, c, d, e) - separate parameters by newline: 2 3 4 5 6 |
|
Int / String Array - the size of the array followed by the list of values in the array 3 100 200 300 |
The following image illustrates the candidate's view of the test case execution status for the compiled code in the test. Note that the candidate can only view the sample test case's input and expected output values. Only the test execution results are displayed for the hidden test cases.
Refer to Key Features of a Detailed Report to know how the test cases show up in a test report.
Scoring Test Cases
Assign varying scores to test cases based on difficulty, ensuring a fair distinction between candidates' skill levels:
- Assign more points to complex test cases.
- Sample test cases can be assigned zero points if desired.
- The sum of all test case scores constitutes the total question score.
Best Practices for Test Cases
Syncing with Code Stubs
Ensure input and output structures align with the code stubs to maintain language consistency:
- For arrays, read the size (
n) first, followed bynelements. - For linked lists, input includes the number of elements and their respective values.
Key Recommendations:
- Diversify test case complexity and sizes.
- Avoid duplicating test objectives across multiple test cases.