Getting Started With Pre-Screen Analytics
Pre-Screen Analytics highlights key performance metrics and benchmarks for each Pre-Screen Assessment, and is designed to help you optimize your technical hiring process. This deep dive view is available for every Pre-Screen, whether that's a Custom Pre-Screen or a Certified Evaluation backed by a Skills Evaluation Framework.
Here, you'll see data about your candidates, including their progression through the assessment funnel, average scores, skill areas, feedback, and language preference. These are quantitative metrics that describe how each Pre-Screen is performing, and help drive discussions with stakeholders and with your Customer Success Manager about process improvements that can help you acquire and retain quality candidates.
We'll go into each section in detail, so follow along to see how Pre-Screen Analytics can enhance your understanding of your Pre-Screens' impact. Please note that Pre-Screen Analytics is only available to users with certain account permissions, so contact your company admin if you have questions about your access.
Where to find Pre-Screen Analytics
There are two main ways to access Pre-Screen Analytics. For both methods, first select Pre-Screen in the top toolbar, then click on the Pre-Screens tab.
From here, you can select Analytics on the right side of any Pre-Screen:
Or click in to any Pre-Screen and select All Analytics in the analytics bar:
Remember that neither of these options will be available if your user account does not have the appropriate permissions.
Navigating Pre-Screen Analytics
Pre-Screen Analytics is made up of the following sections:
- Assessment Funnel Breakdown
- Scoring Overview
- Skill Area Distribution Overview and Skill Area Analysis*
- Candidate Feedback Results
- Submission Attempts Per Language
*Please note that some metrics, including the Skill Area Distribution Overview and Skill Area Analysis modules, will not be populated for Custom Pre-Screens. These metrics rely on the comparability and validation of Skills Evaluation Frameworks, and are only available for Certified Evaluations.
We calculate these metrics for the individual Pre-Screen selected. Use the date picker in the top right of the screen to change the period for which the analytics are measured.
Selecting Custom in the date picker will give you the option to select any date range between when you first started using CodeSignal and today. All of the metrics in Pre-Screen Analytics will automatically update to reflect the time period you choose, and will be saved for you to come back to, for that individual Pre-Screen only.
Assessment Funnel Breakdown
Are you retaining candidates effectively throughout the Pre-Screen process? The Assessment Funnel Breakdown is a great place to start thinking about how efficient your funnel is. These metrics show you how many candidates have been invited, how many have responded, and of those, how many have made an attempt. Statistics like these highlight the location of candidate engagement bottlenecks—or potential drop-off.
The first section of the Assessment Funnel Breakdown shows how many private invitations have been sent for that individual Pre-Screen. These are invitations tied to a specific email, for a specific candidate. Note that we are not able to track invites for shared links, as these links are publicly available rather than tied to a single candidate.
Next, we see how many responses this Pre-Screen has received. A response is the candidate opening the Pre-Screen, rather than ignoring or actively declining the invitation. Here, we can measure the response rate for private invitations, or how many candidates who received a private invitation have actually clicked into it and opened the Pre-Screen. We can track how many candidates open a shared link, and that number is included as well.
For Certified Evaluations backed by a Skills Evaluation Framework, we can also give a comparison between your candidates' response rate and the response rate we see across every Pre-Screen backed by that Skills Evaluation Framework. If your Pre-Screen is performing under the benchmark, it may be time to look at how you select which candidates receive invitations, the messaging associated with those invitations, or how long candidates have to complete the Pre-Screen. Since Custom Pre-Screens are not associated with a Skills Evaluation Framework, you won't see this comparison metric for Custom Pre-Screens.
Finally, the Assessment Funnel Breakdown shows your candidates' attempt rate for this Pre-Screen. Of all of the candidates who open the assessment, regardless of how they got there, this percentage of them have actually attempted the Pre-Screen. As with Private Invitation Responses, we can provide a comparison against the CodeSignal benchmark for Certified Evaluations. If your attempt rate is low, that might mean candidates decide it's not worth attempting the Pre-Screen. Perhaps that Pre-Screen is too long or too complicated for the candidate pool that is seeing it, or perhaps the Pre-Screen is not aligned with what candidates expect to be asked about given the position for which they're applying.
Is your Pre-Screen attracting quality candidates? Scoring Overview helps you recognize when your Pre-Screens might be targeting the wrong candidate pool. For Certified Evaluations, we can also compare your score distribution with a benchmark, helping you see whether there might be untapped potential in the market. The scoring threshold tool gives you insights to help manage your interview volume by selecting an appropriate threshold for advancing candidates to the next stage.
The Scoring Overview graph shows the distribution of scores for your Pre-Screen. Because we can directly compare scores for Certified Evaluations backed by the same Skills Evaluation Framework, those Pre-Screens have benchmark comparison bars. This information is not available for Custom Pre-Screens.
Another key feature in the Scoring Overview module is the scoring threshold tool. If you use scoring thresholds to progress candidates to the next phase, you can now see how changing the threshold for any given Pre-Screen will affect your volume at the next stage. Click the pencil icon to modify the score threshold, and we'll automatically update the percentage of candidates who score at or above that threshold. For Certified Evaluations, we'll also calculate a benchmark to compare against the industry average.
Skill Area Distribution Overview and Skill Area Analysis
What are your candidates' strengths, and where might you need development programs for this role? The Skill Area Distribution Overview and Skill Area Analysis modules offer a well-rounded view of your candidates' skill proficiency. With two ways of visualizing skill area proficiency, this section dives into deeper detail than candidate scores alone.
The Skill Area Distribution Overview and Skill Area Analysis metrics rely on skill area information from validated Skills Evaluation Frameworks. Because this data is not available for Custom Pre-Screens, this module will appear blurred out when visiting Pre-Screen Analytics for a Custom Pre-Screen.
For Certified Evaluations, these modules display candidate proficiency for each skill area measured by the Pre-Screen. These visuals give you a sense of where your candidates' strengths and areas of improvement lie. Candidates are ranked from Developing to Expert depending on their proficiency. Each Skills Evaluation Framework covers different skill areas, and some cover a larger number of skill areas than others. Where available, click View All below the Skills Area Distribution Overview graph to uncover the full analysis.
Candidate Feedback Results
Are candidates who take your Pre-Screen satisfied with their experience? We call out three specific feedback metrics in the Candidate Feedback Results module, placing candidate experience at the forefront for discussions about Pre-Screen positioning. Monitoring these metrics can give you a sense of where candidates may be encountering hidden friction.
The three Candidate Feedback Results metrics are user experience, fairness, and relevance. All candidates completing Pre-Screens, whether those are Custom Pre-Screens or Certified Evaluations, are invited to rate their experience after completing a Pre-Screen. Candidates evaluate three statements to generate these feedback metrics. The statement candidates rate for UX is: "The platform provided a good user experience." For fairness: "Given the purpose of this evaluation, the questions seemed fair." And for relevance: "The questions represented what I would expect to do on the job."
For Certified Evaluations, we can also compare these scores with average scores across all candidates taking a Pre-Screen backed by that same Skills Evaluation Framework. Comparing your candidates' feedback with the benchmark is a good starting point for evaluating your recruitment messaging and candidate targeting strategies.
Submission Attempts Per Language
Are the programming languages your candidates are most comfortable with aligned with your team's work? Submission Attempts Per Language shows the coding languages that your candidates prefer to use for that Pre-Screen. This information can indicate whether you might expect new hires to need some more support when settling into the team.
The Submission Attempts Per Language graph draws its data from candidate code executions, so the languages here represent what candidates are actually using, not just what they say they prefer. Although most Pre-Screens tend to see four or five common preferred languages, you can click View All to see a full breakdown across all available languages for that Pre-Screen. Note that some Certified Evaluations are backed by frameworks that only accept certain languages, so some Pre-Screens may naturally have a less diverse language preference graph.
Submission Attempts Per Language statistics are calculated across both custom Pre-Screens and Certified Evaluations. However, users looking at Certified Evaluations will also see a benchmark comparison in this module. This benchmark comparison gives you a sense of where the broader market is trending.