Overview
You can create DevOps questions that help assess the Linux skills of a candidate. When a candidate takes a test consisting of DevOps questions, we set up their sandbox Linux virtual machine to log in remotely. You can ask them to perform whatever tasks you like: install a package, create a file with particular contents, or act with a specific tool like grep.
You can write a script to score the performance of candidates or rely on the session playback of the session to grade the test subjectively. The video playback feature lets you view if a candidate checked a manpage or knew how a tool worked from memory.
When a candidate views a DevOps problem, this problem statement is displayed alongside the login credentials and an SSH command for logging into the Linux virtual machine. The in-browser code editor does not exist for DevOps questions, which differs from other question types where the candidate views the problem statement and enters their solution on the same page. This article will take you through the steps in creating a DevOps question.
Note: Even if there is more than one DevOps question in a test, a candidate is supplied with only one virtual machine. The candidate logs on to the same machine to perform each task and run all the post-submission scripts on the same machine in succession.
Steps to Create a DevOps Question
- Click on the Library tab on the home page and click on the Create Question button.
- In the Select Question Type dialog box, select the DevOps question type.
The question creation wizard for the DevOps type questions comprises two tabs. The functionality of the two tabs is explained in the below steps.
Settings
Operating System
Inside the question creation wizard, the first option under the Settings tab will let you choose the operating system and its root access.
Note: Root access is provided via the 'Sudo' command. Even if one question in the test requires root access, the candidate is provided with root access privilege on the machine.
Scripts
On the Scripts option, set up the following scripts:
- Configure the Setup script to enable the environment for the candidate to log in.
This script is run at server startup time before the candidate logs in. Here, you can install any packages for the candidates or create some sample input files for them to interact with. The Setup script timeout limit is 5 minutes. - Configure the Scoring script to score and evaluate the performance of the candidate. This script is run once the candidate has submitted the code. You can create a custom scoring logic that suits your requirements. You can award partial credit for the questions. This is a mandatory script. The Check script timeout limit is 20 minutes. Refer to the Scoring a DevOps question topic to understand the required output format of the Scoring script.
- Configure the Evaluator solution to store the correct answer to the problem.
You can add this script to solve the question. This is only for your reference, and we do not run this script. This script is placed in the /tmp folder. You can access this script for easy debugging and testing when you try the test as a recruiter or admin. This is an optional script.
Note: All these scripts are written in Bash.
Once you are done with the scripts, click on Next to move forward in the process. If there are any errors in your Setup, Scoring, or Evaluator scripts, you will be prompted to fix these errors before proceeding to save the question.
Details
Upon clicking on Next, you will be taken to the Details tab, where you can fill in the details explained below:
Problem Statement
On the Problem Details page, specify
A. Problem Name
- The problem name should be short and topical. It should not give any hints to solve the problem.
B. Score
- While you can assign any score you want for the questions you create, we use certain standards for assigning scores to the questions we create that you might find useful. Refer to the Scoring a DevOps Question topic for more information.
|
Score |
Question Type |
|
50 points |
For an easy question that can be solved in 15 minutes |
|
75 points |
For a medium question that can be solved in 30 minutes |
|
100 points |
For a difficult question that can be solved in 45 to 60 minutes |
C. Recommended Time
- Specifies the time recommended for finishing the problem.
D. Tags
- Tags are words or phrases that help with the searchability and organization of your questions. You can add the existing tags or create new tags and set the difficulty level for your question by associating the tags as Easy, Medium, or Hard. Alternatively, associate custom Tags to identify your question by its complexity or levels.
- Click the Tags drop-down, and type a keyword—example: Easy.
- Click Add. The “Easy” tag is associated with the question.
- When you view your Questions in the Library, the associated tags will be indicated for every question. You can use these Tags to generate candidate reports and performance graphs.
Refer to Associating the Tags for Questions for more information.
E. Problem Statement
- A good problem statement should comprise a clear and detailed problem description.
F. Interviewer Guidelines
- Interviewer Guidelines are for later reference. You can include a rubric about scoring the questions or write solutions to the problems in this section. These can be used by your team while evaluating the test.
Note: Interviewer Guidelines are only visible to your team and you; however, candidates cannot view these notes.
You can support your problem statement and internal notes by attaching a relevant reference file in these sections. Refer to the How to Attach a File to a Problem Statement topic for more information.
- You can click on the Try Question to test your Question by launching the terminal. This will launch a machine with a terminal like a candidate would while doing the test.
- Click on the Save button to save the question.
The Questions you create are stored in the HackerRank Library under the “My company questions” section. The Question type and the associated tags are indicated below every Question title.