Configure automatic scoring for filesystem tasks

With Filesystem task types, if you enable Automated Tests during the task creation, you can add unit tests that will be automatically run and scored by CodeSignal each time a candidate completes the task. The unit tests can be hidden from the candidate, or exposed to serve as guidelines.

This article will cover:

• Adding an environment
• How scoring works 
• Configuring unit tests in each environment

Adding an environment

After creating the new filesystem task, click the ADD AN ENVIRONMENT button to select an environment for your task.

Don’t see the environment that you would like to use? Fill out this form to indicate that you would like it to be supported.

How scoring works 

• Each unit test that is added with the correct configuration will be incorporated into the overall score. See more about configuring unit tests below.
• We recommend making your unit tests read-only and/or hidden from the candidate so that they cannot be modified.
• In custom Filesystem tasks that you create, each unit test counts equally towards the overall score by default. If you want to adjust the weights in the backend, our content team can help you with this. Please contact customer_success@codesignal.com. 

Configuring unit tests in each environment

Java + JUnit (Maven or Gradle)

• Following convention, the path containing unit tests should be: src/test/java
• Unit tests must have the appropriate annotations.

JavaScript or Typescript + Mocha

• The path containing unit tests must be: specs/
• Learn more about writing tests with Mocha here.

C# + NUnit

• All NUnit attributes are contained in the NUnit.Framework namespace. Each source file that contains tests must include a using statement for that namespace.
• See here for more documentation.

C# + xUnit

• The tests must be part of a test project, and that project must be included in the solution file.
• For more information on writing tests in .xUnit, see the documentation here.

Go + go test

• No tests should have matching names—even for different packages.
• Go testing documentation can be found here.
• You can add external libraries if needed by modifying go.mod and adding go.sum accordingly. See here for more details.
• You can modify the structure of the project as you wish, including the name of the module.

PHP + PHPUnit

• The path containing unit tests must be: tests/ 
• The file(s) being tested must be included, e.g.:

include(dirname(__FILE__)."/../sumNumbers.php");

• See information on writing tests in the documentation for PHPUnit here. 

Python (2 or 3) + unittest

• The path containing unit tests must be: tests/ 
• The following code, which lets you import Python files from the root directory, must be included:

import os, sys, inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)

import unittest

• More information on the unittest module can be found in the documentation here.

Ruby + RSpec

• The path containing unit tests must be: specs/
• The filenames must conform to the following conventions (as defined here):
  • ..._spec.rb
• The file(s) being tested must be included, e.g.:
  • require_relative '../sum_numbers'
• Documentation can be found here.

Questions? Contact support@codesignal.com