Overview
HackerRank Projects for Data Science allows you to set up automatic scoring for your custom project-based real-world questions to assess Data Scientists.
This is an optional step under the Setup Project step of the Data Science question creation workflow that allows you to set up automatic scoring. You can add evaluation scripts, submission files, solution notebooks, and so on in the section Add evaluation files. All files uploaded in this field will be hidden and not be available to the candidate. During scoring and evaluation, these files will be downloaded in the Juypter session inside the "evaluation_files" folder. The configuration file Hackerrank.yml needs to have the scoring command. The scoring command would be executed automatically after a candidate submits the solution.
In this article, we will walk you through the process of setting up automatic scoring while creating your own Data Science Question on HackerRank Projects.
Setting Up Automatic Scoring of Data Science Questions
- Create a program to evaluate candidate submission.
(Example: a python program that will compare the candidate's submission.csv with the uploaded actual_output.csv and calculate the desired model performance metric such as accuracy, recall, precision, f-score, etc., convert that into a % and return FS_SCORE: X% as output)
- Ensure the scoring program returns in the format “FS_SCORE: X%”
- Upload all the necessary files (actual_output.csv) needed to run the evaluation program successfully along with the scoring program(score.py, score.sh) in the evaluation section.
- Edit the hackerrank.yml configuration file and add the scoring key with the scoring command. The example configuration is below:
Scoring
command: python evaluation_files/score.py evaluation_files/actual_output.csv
- Ensure that your evaluation script can handle edge cases and will return a score in all possible scenarios.
- An erroneous response will result in a no score and you will be required to manually review the submission.
- Click on the validate and save button to validate the changes. In case of errors, follow error logs and messages to debug.
Note: To update the evaluation files, you need to re-upload them and delete the previous version.
HackerRank also offers manual scoring for the Data Science questions, learn more about it here.