HackerRank automatically flags coding questions that artificial intelligence (AI) tools, such as ChatGPT, Gemini, etc. can solve in its library. To ensure transparency, HackerRank performs monthly automated analyses of its question library to simulate how AI tools would perform on these questions. This feature aims to inform users about AI-solvable questions, helping them design tests effectively, particularly where plagiarism might be a concern.
Key Benefits
- Awareness of AI-solvable questions.
- Ability to exclude such questions when plagiarism is common, such as high-volume or university hiring.
How to Identify AI-Solvable Questions
HackerRank marks AI-solvable questions with an AI Solvable = Yes indicator. The following steps can help you use this feature:
- Locate flagged questions in the Question Library.
- As shown in the example below, you can curate your test using the Hide AI-Solvable Coding Questions filter in the left pane of the library.
Why This Feature Matters
AI tools have evolved to solve certain coding problems efficiently. While their capabilities are improving, simpler questions with concise answers are more vulnerable to being solved correctly by these tools. HackerRank protects its customers by offering filters to avoid AI-solvable questions.
When curating tests where plagiarism is a concern, you can additionally leverage HackerRank’s AI-Powered Plagiarism Detection features for further protection.
Recommendations for Question Usage
| Question Type | Susceptibility to AI | Indicator & Filter Availability | Recommendation |
|---|---|---|---|
| MCQs | High | No | Use to assess tertiary skills and basic knowledge and minimize evaluation time. |
| Sentence Completion | Varies | No | Use for tertiary skills and basic knowledge. Limit usage for primary skill testing. |
| Coding, Database | Medium | Yes | Ideal for testing coding proficiency. Avoid flagged questions in tests that are prone to plagiarism. |
| Backend, Frontend, Fullstack | Low | No | Best for evaluating primary skills. These require extensive customization, making them less AI-solvable. |
Scope of the AI-Solvability Filter
This filter applies to Coding and Database questions, the most common test types. While also susceptible to AI, questions like MCQs and sentence completion are not covered by the filter. HackerRank recommends prioritizing hands-on exercises (e.g., coding, database, full-stack tasks) for primary skill evaluation.