How to Check Suspicious Activity Using the AI-Powered Plagiarism Detection Solution

Overview

In a world where humans and machines alike can write code, plagiarism detection must keep pace with advances in AI tools. HackerRank’s AI-powered model for plagiarism detection will help you identify suspicious cases more precisely.

HackerRank for Work provides detailed Test Reports which describe a candidate's performance in a Test. The Test Reports are generated after the Candidates have submitted a Test. The topics below describe how to use AI-powered plagiarism detection when reviewing candidate test reports.

For more information on how HackerRank's AI-power plagiarism detection works, see the support article AI-powered Plagiarism Detection.

Prerequisites

• It is recommended to have a HackerRank for Work account.
• You must have at least one test attempted by Candidates and their submissions pending for further evaluation.

Verifying Plagiarism Activity on Candidate Reports

1. Navigate to the Tests page and select the required Test.
2. Click the Candidates tab on the right. If a candidate is marked for Plagiarism, then a flag appears beside the candidate's name.
3. Select a Candidate entry flagged with plagiarism. Note that based on the level of suspicious activity, the candidates are flagged with the appropriate flag.
4. The Candidate Summary page will display the Suspicious activity detected, including indications of plagiarism.
5. You can expand this section to get a brief view about the plagiarism pattern detected for a question.
6. To view the plagiarism, click the question in the Questions tab.
7. The Attempt Activity Tab on the report will show if the candidate has submitted code that has been flagged as plagiarized. Plagiarism is detected in the example shown above as an image. You will review the flagged behaviors to confirm or override the plagiarism flag.
8. Click Yes or No based on the suspicious activity of the candidate. If you click No, the plagiarism flag for that candidate will be removed. Clicking Yes retains the plagiarism flag for that question.
9. For AI-powered plagiarism, the Attempt Activity tab shows the Coding Patterns and Code Similarity under Reasons for flagging a candidate for suspicious activity.
10. Click View in Codeplayer to check a keystroke-by-keystroke playback from the last attempt of the suspicious activity of a candidate.
11. To check for suspicious activity based on Code Similarity, click View Matches. The plagiarism matches are displayed for the candidates.
12. Click View in the Output Diff column to check the candidate's suspicious activity.
13. The code is displayed as follows:

Understanding Plagiarism Activity on the Candidate Report

HackerRank's plagiarism detection tool also shows matches with other candidates across HackerRank for Work who might have submitted the same or similar code to the question in any test.

For information on the additional indicators of plagiarism detection and how to use them, see the

Plagiarism Best Practices guide.

We match for a similar code with the following:

• All candidates attempted any test across any company on the HackerRank for Work platform.

The code matching happens as follows:

• If a matched candidate is from the same test as the current one, you can see the code match percentage, the date, similar code submission, and the candidate's email.
• If a matched candidate is from a different test, but you have access to the test, you will still be able to see the code match percentage, date of identical or similar code submission, name of the test, and the candidate's email.

Note: If a matched candidate is from a different test you do not have access to; you will still be able to see the code match percentage date of duplicate or similar code submission but not the test name or candidate email. 

• Code Match Percentage and View Output Diff

The Match Percentage shows the percentage of matches between the current candidate's code and matching codes. To the far right of the match, the percentage is the option to 'view diff,' clicking displaying matching codes with highlighted differences. Based on this possible, partial, or no-match scenario, we recommend the match percentage.

Based on the result, the user can take a call as to how to proceed with the candidature.

To review the suspicious activity of a candidate if they have been flagged for ‘Coding Patterns’ 

• You can make use of the Codeplayer functionality to review the candidate attempt keystroke-by-keystroke as shown below:

Override the Suspicious Activity Detection

The system is not fully automated and is a decision support system for Hiring Managers or TAs. They can override the decision by providing their feedback at a candidate level after they have reviewed the code playback.

1. Click the Detailed Report view.
2. In the Attempt Activity tab, select Yes or No based on the plagiarism detected.
3. Clicking No will remove the plagiarism for the question for which a candidate has been flagged.

Additional Plagiarism Indicators With Generated CSV Reports

The plagiarism information can also be verified on the report in CSV format you can download from the platform. The Report includes the performance report of all candidates in a single file. Through the Report Preferences settings in HackerRank, you can choose the candidate performance data you want to download in your file.

For information on the additional indicators of plagiarism detection and how to use them, see the Plagiarism Best Practices guide.

Once an assessment is completed, go to the Candidates tab of the test and select the status Completed. You will get an option to download the report.

After you download the report for AI-powered plagiarism, you can check the following columns (along with the other columns) of the report:

How to Filter Candidates Using the CSV Reports?

Your tolerance for the suspicious activity should be determined by your business demand for the throughput of candidates, subsequent evaluation activities in the hiring process, the need for new employees to work without external resources, the severity of the behavior, and other business necessities.

Example 1: If you are concerned only about highly suspicious candidates, use the Attempt activity Severity column and filter out candidates marked as High.

Attempt Activity Suspicion Severity	Attempt Activity Suspicion Score	Attempt Activity Reasons
Medium	7	Coding Patterns
High	10	Coding Patterns, Code Similarity
Medium	5	Coding Patterns

Note: Suspicion scores between 8-10 are marked as High, 5-7 are marked as Medium, and <5 are marked as No.

Suspicion scores between 8-10 are marked as High, 5-7 are marked as Medium, and <5 are marked as No.

Example 2: If you are fine with moving ahead with candidates flagged for reasonable suspicious activity because you do not have enough candidates, use the Attempt activity suspicion score column to filter candidates appropriately.

Attempt Activity Suspicion Severity	Attempt Activity Suspicion Score	Attempt Activity Reasons
Medium	7	Coding Patterns
High	10	Coding Patterns, Code Similarity
Medium	5	Coding Patterns

 

For example, you may consider progressing candidates on the lower end of the suspicion scale. As Medium ranges from 5-7, you can use the Attempt activity suspicion score column and choose candidates marked as 5 (5 being the lowest on the Medium scale).

 

Recommendation: When you progress candidates marked as a Medium to the next stage, it is recommended that you assess the candidates further to ensure their skills are a good match and that the candidate did not cheat in the previous round. 

Therefore, these columns can be used in conjunction with one another to filter out candidates based on your criteria.  Also, these columns show only attempt-level data and do not provide question-level information.

Additional notations:

• HackerRank has moved away from a narrow approach of flagging suspicious behavior to a multi-category approach of flagging candidates and thus being Candidate-first.
• You can review the Code-Similarity-based information, which will be available in the CSV report. For more information, refer to the Plagiarism Detection (Measure of Software Similarity) support article.
• For questions that do not have plagiarism detection capability, you can also leverage the proctoring capabilities such as Tab Proctoring and Copy-paste proctoring to ensure the integrity of the results.

You can do this by referring to the following columns in the CSV report:

• For tab proctoring:
  • Out-of-window duration: Show the duration (in seconds) for which the candidate was out of the Test tab.
  • The number of window exits: Show the number of times a candidate moved out of the tab.
• For Copy-paste proctoring:
  • Copy-paste frequency: Show the number of times a candidate tried to copy-paste their code into the coding editor.