The Automated Code Review Scoring feature automatically evaluates a candidate’s code review submission against expert examples. It measures how effectively the candidate identifies issues, explains reasoning, and provides actionable feedback. This helps hiring teams assess review skills at scale with consistency and efficiency.

Key benefits

The Automated Code Review Scoring feature offers the following benefits:

• Scalable evaluation: Automatically grades multiple candidates.

• Consistent scoring: Reduces subjectivity in evaluations.

• Transparent results: Provides detailed reasoning for every automated score.

• Efficient process: Saves time while maintaining evaluation quality.

How it works

When you enable Advanced Evaluation at the company level, Automated Code Review Scoring automatically applies to all Code Review questions.

Step 1: Define key evaluation areas

You can define specific areas where candidates are expected to leave comments. within the grading rubric.

To configure these areas:

1. Open the relevant Code Review question.

2. Click Edit.

3. In the Grading Rubric section, define the key areas where candidates are expected to leave comments.

   

4. Click Save question to finalize the rubric.

Note: HackerRank provides predefined comment locations that you can customize based on your evaluation criteria.

Step 2: Candidate review process

During the test, candidates leave inline comments directly in the provided code to identify issues, suggest improvements, or justify their reasoning. The system uses these comments as the basis for automated evaluation.

Step 3: Automated evaluation

During evaluation, the AI system compares the candidate’s comments with the defined rubric. It analyzes each comment based on the following factors:

• Contextual relevance: Whether the comment applies to the correct code section.

• Semantic similarity: Whether the comment meaningfully aligns with the expected feedback.

Each relevant and accurate comment contributes to the candidate’s overall score.

Viewing evaluation results

Recruiters can view automated evaluation details in the Summary Report, including:

• Candidate comments compared with expected comments.

• Automatic scoring (For example, 30 points for three relevant comments).

• AI-generated reasoning explaining how the score was determined.

The Candidate Evaluation tab in the detailed report displays:

• Expected comments defined by the recruiter.

• Comments submitted by the candidate.

• AI justification for each scoring decision.