Skip to main content
↑

Best Practices for Test Creation

  • Updated
Hide
In this article

Best Practices for Coding Challenges

  1. Each test should have at least two coding questions: a straightforward question and one advanced question.
    • Simple question: 80% of candidates should be able to solve this question.
    • Advanced question: only 20% of candidates should be able to solve this question. 
  2. A test should take between 60 and 90 minutes for a candidate to complete.
  3. You should include at least two sample test cases in the problem description along with an explanation to help candidates understand the format as well as the problem.
  4. Problem statements should clearly state if the candidate is expected to write full code or only fill out the stub. Use code stub programming questions as much as possible.
  5. If you refer to a variable in the problem statement as "N," then it is best to use the same notation in the code stubs. For example, it is best not to use "N" in the description and "n" in the code stub (or vice versa).
  6. All variable names, constants, etc., should be italicized when referred to in problem statements. 

Be Prepared

  1. Before sending your test to a candidate, ask a fellow employee to take it first to ensure it is easy to understand and error-free.
  2. Be mindful of the problem statement. It should be clear and include written examples of inputs and outputs.
  3. Make the experience worth the candidate's time: high scores should faster move the hiring process.
  4. Not every candidate will take your challenge. However, the ones that do are far more qualified than before.

Test Cases

  1. Test cases form the basis for your evaluation of a candidate's code.
  2. The total score given to a candidate is the sum of the scores of all test cases that pass.
  3. There should be at least three test cases per question (sample, easy, edge).
  4. To the extent possible, ensure that no two test cases have the same output or return value.
  5. Create challenging test cases at a difficulty level. Not every candidate will pass.
  6. You should have at least 2 test cases marked as Sample Testcases, and ensure they are the same as the ones you have explained in the problem description.

Remember: Be Creative

  1. Take the time to make your challenges as compelling as possible.
  2. Challenges related to your industry or brand average better click-through and completion rates than generic versions.
  3. Do not be afraid to reach for new ideas:
    • "How much fuel does the rocket ship need to reach mars?"
    • "James Bond must decrypt code to save the world..."
    • "A city is planning a new rail line. Determine the route that maximizes most residents served in the fewest stops."

Related articles