With CodeSignal filesystem questions, candidates and interviewers can create, edit, and remove their own files using a testing framework. Filesystem questions support multiple languages, which the user can limit and configure in the question set-up. This article will cover:
- Setting up a new filesystem question
- Adding files and folders
- Adding unit tests to a filesystem question
- How scoring works in filesystem questions
- Configuring unit tests in each environment
Setting up a new filesystem question
1. After selecting Filesystem from the Choose question type modal, you will be prompted to choose whether you'd like to toggle on Automated Tests for the question. This is toggled OFF by default.
Make your selection and click CREATE in the bottom right.
2. Next, you will be taken to the Global Settings tab, where you will need to:
a. Give a name for your question
b. Add label(s) to your questions (optional). When you click in this field, you'll see your existing labels. You can also type in a new label.
3. If you enabled automated tests in question set-up, skip to the next section:
If you did NOT enable automated tests, choose which language(s) you'd like to allow your candidates to use for the question by clicking the +ADD AN ENVIRONMENT button above Global Settings.
4. Select a language you'd like to add to your question from the drop-down list. Then, select the Environment Settings icon on the left navigation bar, below the Global Settings icon.
5. Here, you will configure the settings for each language in 3 tabs:
a. SETTINGS: Add your question description to the Description field
b. TIMEOUT: Edit the default Execution time limit for the language (optional)
c. PREVIEW: View a preview of the starter code that will be shown to the candidate. You can edit the starter code on the right side of your screen by navigating between the tabs for each file.
6. To select another language to add to your question, click the Environments + button near the top of your screen.
Repeat the previous step (Step 5) to configure the environment settings for each language you select.
To remove the environment, simply click on it.
7. To navigate between the languages, click the name of the language in the top left.
8. When you're done configuring the question, click SAVE ALL in the bottom right.
Adding unit tests to a filesystem question
With Filesystem question types, if you enable Automated Tests during the question creation, you can add unit tests that will be automatically run and scored by CodeSignal each time a candidate completes the question. The unit tests can be hidden from the candidate or exposed to serve as guidelines.
1. Follow steps 1 and 2 in the previous section for Setting up a new filesystem question
2. Click + ADD AN ENVIRONMENT above Global Settings to choose an environment for your unit test. (You can select multiple by adding them one at a time.)
3. Click the Environment Settings icon in the left navigation bar, below Global Settings. Here, you can configure settings specific to the environment you chose:
a. SETTINGS: Add your question description to the Description field
b. TIMEOUT: Edit the default Execution time limit for the language (optional)
c. PREVIEW: View a preview of the starter code that will be shown to the candidate. You can edit the starter code on the right side of your screen by navigating between the tabs for each file.
d. TEST WEIGHTS: Assign weights to each test case. You can do this manually with the ADD A TEST CASE WEIGHT button, or have CodeSignal automatically detect test cases in your code with the AUTO-DETECT & REVIEW TEST CASES button, and then assign weights.
4. To select another environment to add to your question, click the Environments + button near the top of your screen.
Repeat the previous step (Step 3) to configure the environment settings for each environment you select.
5. To navigate between the environment, click the name of the environment in the top left.
6. When you're done configuring the question, click SAVE ALL in the bottom right.
Adding files and folders
Whether or not you are using unit tests in your filesystem question, you can add an unlimited number of files and folders to the question.
Note: You will need to add files separately for each environment you include in your question. The max file size of individual files is 100 KB, up to a total of 1 MB for the entire filesystem. You can also provide links to files and ask candidates to download them in runtime. Any files that may become part of the source code should be < 32KB for optimal performance.
1. Navigate to the Files tab next to the Environment Settings. You will see the default starter code files listed next to Files.
2. Add a new file or folder by clicking the appropriate icon on the top right, under the Environments button.
3. To Upload or drag-and-drop files click on the arrow icon and select files.
4. Edit preferences for each file by hovering over the file name. From left to right, the icons allow you to toggle or edit the following settings: start file open in editor, hide/unhide, make read-only/editable, rename, and delete.
5. Remember to click SAVE ALL in the bottom right when you're done configuring the question.
How scoring works in filesystem question
- Each unit test that is added with the correct configuration will be incorporated into the overall score. (See more about configuring unit tests in the next section.)
- We recommend making your unit tests read-only and/or hidden from the candidate so that they cannot be modified.
Note: Hidden files are only included on a Submit action, not a Run action. Pressing Run will only include visible files and unit test files. Submit is only available during a test or interview session.
Configuring unit tests in each environment
Java + JUnit (Maven or Gradle)
- Following convention, the path containing unit tests should be: src/test/java
- Unit tests must have the appropriate annotations.
JavaScript or Typescript + Mocha
- The path containing unit tests must be: specs/
- Learn more about writing tests with Mocha here.
C# + NUnit
- All NUnit attributes are contained in the NUnit.Framework namespace. Each source file that contains tests must include a using statement for that namespace.
- See here for more documentation.
C# + xUnit
- The tests must be part of a test project, and that project must be included in the solution file.
- For more information on writing tests in .xUnit, see the documentation here.
Go + go test
- No tests should have matching names—even for different packages.
- Go testing documentation can be found here.
- You can add external libraries if needed by modifying go.mod and adding go.sum accordingly. See here for more details.
- You can modify the structure of the project as you wish, including the name of the module.
PHP + PHPUnit
- The path containing unit tests must be: tests/
- The file(s) being tested must be included, e.g.:
include(dirname(__FILE__)."/../sumNumbers.php");
- See information on writing tests in the documentation for PHPUnit here.
Python (2 or 3) + unittest
- The path containing unit tests must be: tests/
- The following code, which lets you import Python files from the root directory, must be included:
import os, sys, inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)
import unittest
- More information on the unittest module can be found in the documentation here.
Ruby + RSpec
- The path containing unit tests must be: specs/
- The filenames must conform to the following conventions (as defined here):
- ..._spec.rb
- The file(s) being tested must be included, e.g.:
- require_relative '../sum_numbers'
- Documentation can be found here.
Questions? Contact support@codesignal.com