Please note: Coding Score 2023 is set to go live in the CodeSignal product in Spring 2023.
Coding Score 2023 is a recalibrated scoring system for all CodeSignal Skills Evaluation Frameworks. This new system is designed to maintain scoring consistency across Skills Evaluation Frameworks, increase the precision of measuring candidate skills, and improve fairness of evaluation results.
This article shares more information on how to interpret the results of Coding Score 2023, and how it is calculated.
Please note that when the transition to Coding Score 2023 goes live, historical coding scores will be converted to Coding Score 2023, and candidates will be eligible to re-share the converted score (as usual). For more information on score conversion, please see the article here.
Understanding Coding Score 2023
There are two types of insights delivered by Coding Score 2023:
- Overall Coding Score: Each Skills Evaluation Framework completion will result in a single Coding Score that is an overarching metric for quantifying a candidate’s overall skills. This is the data point you can use to determine if they have the core skills to do the job. This number will range from 200-600, with higher scores indicating that a candidate successfully completed more questions in their assessment. For more information about converting historical coding score thresholds to an equivalent Coding Score 2023 value, see the article here.
- Skill area proficiency: Coding Score 2023 also gives you and your candidates insight into a candidate’s level of proficiency across the various skills each Skills Evaluation Framework measures. This data is not meant to be used to make hiring decisions, but it can give you directional information on candidate differences, and inform candidates of areas where they may need more development.
How Coding Score 2023 is Calculated
The Coding Score 2023 includes a two-tiered scoring system contributing to the overall score and a detailed breakdown of candidate skill proficiencies based on their performance within the respective assessment. Questions within a Skills Evaluation Framework are organized and grouped into modules. Within the two-tiered system used for calculating the overall score, the first tier (i.e., the base points) treats all modules or question groups equally; the second tier (i.e., the bonus points) is awarded to candidates upon full completion of a module.
Coding Score 2023 uses a two-tiered scoring system for each module (a set of 1 or more questions). Note: Bonuses are derived from normative difficulty and discrimination parameters for all questions within the module.
First tier: The first tier refers to the base points a candidate can earn by successfully completing all questions within a module within the Skills Evaluation Framework. As a candidate successfully completes more questions within a module, they can earn up to the module’s base point value. Weighing all modules relatively equally will help motivate candidates to complete as many questions as possible. Since all content within a Skills Evaluation Framework is carefully designed to capture meaningful signals from candidates, this scoring approach maximizes the precision in capturing candidate skill levels by expanding the amount of signals captured by the assessment.
Second tier: The second tier refers to the bonus points a candidate may be awarded based on their performance. Candidates only earn bonus points for a module if they successfully solve 100% of the questions within the module. The allocation of bonus points across modules is determined by how well they differentiate between candidates of differing skill levels and their level of difficulty. More bonus points are awarded for completing modules that are both better at differentiating between candidates and higher in difficulty. This ensures that a higher overall score on the assessment represents stronger role-related skills.
Final Score: Candidates earn a final score ranging from 200 to 600. The score range was designed to avoid potential overlap with 1) scores from other standardized assessments (SAT, GRE, etc.), and 2) percentiles that may be closely associated with the standard U.S. grading system (such as a 0 to 100). Overlap with those numeric ranges may influence how scores are interpreted by end users, so the numbers associated with low and high values within the Coding Score 2023 range are designed to have no inherent significance that may influence interpretation.
CodeSignal also offers qualitative feedback on the individual skills each Skills Evaluation Framework measures, separate from the Coding Score 2023 overall score. The feedback system was designed to: 1) provide companies and candidates with greater insights into opportunities for growth and development once hired, and 2) improve the candidate experience by providing individualized feedback on specific skill areas where candidates excel or have the potential to develop their skills further. The feedback system includes qualitative descriptions of skill proficiency levels ranging from “Developing” to “Expert,” and a visualization of candidate skill proficiency levels, for the skills areas measured in each Skills Evaluation Framework. For example, skill areas of the General Coding Framework are Basic Coding, Data Manipulation, Ease of Implementation, and Problem-Solving.
Skill proficiency levels for each Skills Evaluation Framework are determined based on input from subject matter experts (SMEs). Given that a specific skill may be relevant to multiple modules, candidates have increased opportunities to demonstrate skill proficiency by successfully completing more than one module. The level assigned to a candidate depends on how well the candidate successfully completes the modules linked to each skill area. Poorer performance results in lower skill proficiency labels, whereas someone who demonstrates full competency will be assigned a level of "Expert". The four possible skill proficiency levels are:
- Developing, candidate displays minimal skill proficiency
- Intermediate, candidate displays an average degree of skill proficiency
- Advanced, candidate displays an above average degree of skill proficiency
- Expert, candidate displays the highest level of skill proficiency
Although the skill proficiency feedback is related to the Coding Score 2023 overall score (i.e., a candidate that earns more “Expert” level classifications will have a higher overall score than a candidate that earns primarily “Developing” level classifications), their intended purpose is developmental for candidates. Skill proficiency levels are calculated independently from Coding Score 2023, not directly connected to the overall score, and have not been validated for use in making hiring decisions. Because of this, CodeSignal strongly recommends all selection or administrative decisions to be based on candidates’ Coding Score 2023, without consideration of individual skill proficiency feedback.