In the workplace, developers are frequently required to reuse and refactor their code in order to meet changing requirements. With progressive questions, you can mimic this real-world scenario with multilevel questions that start with basic requirements and gradually increase in complexity, and require candidates to write, test, and refactor code. A candidate’s code and test cases will automatically carry over to the next level so they can continue to build on the work they’ve done previously. 

There are several question formats that can be made progressive:

• Creating progressive free coding questions
• Creating progressive single function questions: described below; used for defined coding scenarios with test cases that can be automatically scored. Available in CodeSignal Assessments or CodeSignal Interview. Will also support advanced assessment functionality soon, too! 

Single function questions give candidates a defined scenario, where a given output is expected. Candidates can be automatically scored via (visible and/or hidden) test cases that are run against the candidate’s solution. 

When creating each level, you build the question as you would normally for a single function question; with progressive questions you just now have the ability to build on the question with multiple levels. 

How to create a new progressive single function question

To create a progressive free coding question, navigate to the Questions. Then, click Create Question.

Select free coding from the options that appear, then click Next.

When prompted, click the Make Progressive toggle to turn on the feature. The toggle will be green when enabled. 

In the SETTINGS tab, add a title, description, and a label if needed for your question. You can also choose to enable the network access if desired, which allows the candidate’s code to interact with an external API. 

Go to TIMEOUT tab and edit the execution time limit for all languages as needed. 

In Languages & Code, choose the languages you want. Once you activate the limitations, you can select your preferred languages.

From New level + tab you can add a level.

Reviewing your question

To have peers review your question, you can use Review Mode. Simply navigate to the Question Library, search for your question and select it. 

Upon opening the question, reviewers will see what the candidate sees. 

Reviewers can use the Question Notes tab, as noted by three lines on the left hand navigation,  to leave comments in each level. These notes will not be visible to candidates. 

To progress to the next level, click the Next (with code carry) button in the upper right of the settings panel. To navigate between levels, click the karat icon next to the current level name and a drop down will appear. Click the level you’d like to jump to and you’ll be navigated there directly. Interviewers can navigate both forward and backward. 

Note that hidden test cases will be noted with gray text and visible to reviewers, but they will not be visible to candidates.

Leveraging Progressive Single Function Questions in Assessment or Interview

Visit the article here to learn how to create an assessment.

Visit the article here to learn how you can leverage your progressive question in an interview. 

Notes & Limitations

When a progressive question is created, you can modify existing levels in the question, but will not be able to add or remove levels from the question later.

This is intentional: adding/removing levels from existing progressive questions causes data inconsistency issues, wherein candidates could have different results based on what version of the progressive question they solved.

If you’d like to change the levels of an already existing progressive question, you can duplicate the existing question and create an entirely new question, with levels added/removed.