Overview
HackerRank Question Library automatically flags coding questions that can be solved using artificial intelligence (AI) tools such as ChatGPT and Bard and can not be caught using HackerRank’s industry-leading AI-Powered Plagiarism Detection tool. We run an automated monthly analysis on our entire library of questions, simulating how AI Tools like ChatGPT would fare if they attempted to solve these questions. This feature aims to help users be aware of questions AI tools can solve. You may avoid using such questions in high-volume/university hiring where plagiarism is likely or common.
How to Use
HackerRank highlights the question with “AI Solvable = Yes,” as shown in the below image.
You can use the "Hide AI-Solvable Coding Questions" filter on the left pane of the library to hide such questions while curating your test.
Why Do I Need This?
Large language models have progressed the power of artificial intelligence tools such as ChatGPT and Bard to a point where they are able to write somewhat like humans. These tools are also capable of solving written problems, including coding problems. Their capability is limited, though evolving. HackerRank protects customers from exposure to such capabilities by helping you design robust tests by using the AI Solvability Filter & Indicator where you feel plagiarism is likely.
When Should I Use This?
All questions across the internet (not just HackerRank) are to some degree exposed to being solvable by AI. Easier problems with small answers are likely to be solved correctly and fully, and tougher problems with large/complex answers are much less susceptible. Meanwhile, HackerRank’s industry-leading AI-Powered Plagiarism Detection tool can detect when an AI tool might have been used to answer a question in a HackerRank test.
Our plagiarism detection tool is always evolving to shield as many coding questions as possible in HackerRank Library. That said, simple questions with small answers may go undetected if plagiarized using AI tools like ChatGPT and Bard. We flag such questions as AI-solvable so you can avoid using them in tests catered to high-volume/university hiring or where you feel plagiarism is likely or common.
Does This Filter Apply to All Question Types?
All questions across the internet (not just HackerRank) are to some degree exposed to being solvable by AI. This filter is available for the most common test question types, i.e., Coding & Database questions.
Note that MCQs and sentence completion questions can also be solved by AI but are not covered in the filter as we recommend using hands-on (Coding, Database, Full-stack, etc) questions to test developers on primary skills. You may still use MCQs and sentence completion questions to your advantage, but we recommend limiting their use to testing for tertiary skills.
The following table outlines recommendations for using different question types:
| Question Type | Susceptibility to AI | AI Solvability Indicator & Filter Availability in Library | Recommendation |
| MCQs | High | No | Best for testing tertiary skills, to screen for basic knowledge, and to limit the time taken to assess for such skill |
| Sentence Completion | Varies | No | Best for testing tertiary skills, to screen for basic knowledge, and to limit the time taken to assess for such skill |
| Coding, Database | Medium | Yes | Best for testing basic coding knowledge. Avoid using AI-solvable coding questions in tests catered to high-volume/university hiring or where you feel plagiarism is likely or common. |
| Backend, Frontend, Fullstack | Low | No | Best for testing primary skills. Least susceptible to AI as it requires heavy prompt engineering to be solved. |
Learn more about AI-Powered Plagiarism Detection.