Best practices for Test creation

Best Practices for Coding Challenges

1. Each test should have at least two coding questions: one simple question and one advanced question.
   • Simple question: 80% of candidates should be able to solve this question.
   • Advanced question: only 20% of candidates should be able to solve this question. 
2. A test should take between 60 and 90 minutes for a candidate to complete.
3. You should include at least two sample test cases in the problem description along with an explanation to help candidates understand the format, as well as the problem
4. Problem statements should very clearly state if the candidate is expected to write full code or only fill out the stub. Use code stub programming questions as much as possible
5. If you refer to a variable in the problem statement as "N", then it is best to use the same notation in the code stubs as well. For example, it is best not to use "N" in the description and "n" in the code stub (or vice versa).
6. All variable names, constants, etc. should be italicized when referred to in problem statements. 

Be Prepared

1. Before sending your test to a candidate, ask a fellow employee to take the test first to ensure it is easy to understand and error- free.
2. Be mindful of the problem statement. It should be clear and include written examples of inputs and outputs.
3. Make the experience worth the candidate’s time: high scores should move the hiring process forward faster.
4. Not every candidate will take your challenge, however, the ones that do are far more qualified than before.

Test Cases

1. Test cases form the basis for your evaluation of a candidate’s code.
2. The total score given to a candidate is the sum of the scores of all test cases that pass
3. There should be at least three test cases per question (sample, easy, edge).
4. To the extent possible, ensure that no two test cases have the same output or return value
5. Create challenging test cases at a difficulty level not every candidate will pass.
6. You should have at least 2 test cases marked as Sample Testcases, and ensure they are the same as the ones you have explained in the problem description

Remember: Be Creative

1. Take the time to make your challenges as compelling as possible.
2. Challenges that relate to your industry or brand average better click-through and completion rates than generic versions.
3. Do not be afraid to reach for new ideas:
   • “How much fuel does the rocket ship need to reach mars?”
   • “James Bond must decrypt code to save the world...”
   • “A city is planning a new rail line, determine the route that maximizes most residents served in the fewest stops.”