Code Review questions have been designed to give candidates a real-world task - review the code written by someone else and provide feedback on it. This task is ideal for more senior-level candidates who are expected to have existing industry experience. They may be managers, mentors, or simply contributing members of a high performing team.
Code Review questions are manually graded, as there is no automated way to read the feedback or judge their seniority. However, written comments from the candidate can provide a really good signal of their seniority and experience in a relatively short amount of time for the candidate.
How to add Code Review questions to a test
Follow the below steps to add a code review question from the library to your test.
- Create a new test or open an existing test
- Add a question from the library
- Select Code Review in the Type search area to display existing code review questions from the library. Click on the + icon to add a question to your test.
How to create a custom question
To create a custom Code review question in your library, follow these steps:
- Open Library and click Create Question
- On the Select Question Type screen, click Code Review
- Fill out the Name, Score, Recommended Time, and the Problem Description
Note: In the problem description, you should explain to the candidate that they are reviewing code and are expected to leave comments.
- Save this page to go to the Upload Code step
- In the Upload Code step, you can set up the code that needs to be reviewed
- Note: The platform supports either reviewing a set of new files or Creating a Diff from 2 sets of files. This is useful for simulating a pull request where only a few changes were made, but the rest of the code should be there for context.
- The system supports multiple changed files, so you must create 2 .zip files. The first .zip file should be the initial code files. The second .zip package should be the changed version. The platform will automatically create the diff between the two packages. Make sure that the folder path and filenames are the same inside of each zip file. That way the patch will look like a set of changed files instead of separate deleted and new files.
- After Saving, you can set up the Grading Rubric
In the Gradic Rubric, you can add comments that will serve as a reference at the time of evaluation. These comments are private and won’t be visible to the candidates. Click on the icon beside each line of code to add your comment against that line.
The Candidate Experience
You can view the candidate experience with this publicly available test here.
Candidates will be able to leave comments on the code diff as shown in the below image.
The candidate can change their view to Unified or Split. Unified is the default. Split is helpful if an original set of files were provided.
When they are finished adding comments, they can click the Submit Review button to complete the question.
The Reviewing Experience
All of the Code Review questions are manually graded. The evaluator will need to open up the question in the Detailed tab and set the score on the left side of the screen.