One way that companies address these skills gaps is through hiring new talent. Since 2015, CodeSignal has helped hundreds of top companies find and hire technical talent with the right skills for their teams. Hiring is just one piece of the puzzle, however. To keep up with the rapid pace of tech innovation, companies also need to upskill the talent they already have. That’s where CodeSignal Develop comes in.

CodeSignal Develop is how leading employers have been upskilling their technical teams, and we’re excited to announce that it’s now generally available to the public. 

Here’s what sets CodeSignal Develop apart from other technical learning platforms:

1. Practice-based learning. Humans learn skills through practice. Practice keeps learners engaged in what they’re learning—and it’s how they actually develop skills. (Imagine trying to learn how to ride a bike by watching a video!) Develop takes learners through hundreds of bite-sized practice exercises in each learning path, allowing them to apply what they’ve learned to solve realistic technical problems.

2. One-on-one guidance and tutoring. It’s just a fact of learning: everyone gets stuck. Getting stuck for too long, though, can lead to frustration—this is when many learners give up on building new skills. A proven way to help learners get past feeling stuck is with one-on-one tutoring, a technique that’s been proven by educational psychologists to be highly effective. 

With personalized support from a tutor, learners can get back to a more satisfying and engaging learning experience: what many researchers call “flow.” Develop has made one-on-one tutoring scalable and accessible with Cosmo, our built-in AI-powered guide who helps learners build skills and stay motivated with personalized and contextually-aware feedback. 

Learners interact with Cosmo while working through practice exercises in CodeSignal Develop.

3. Focus on outcomes. Skill-building should be about outcomes. When teams invest in upskilling their employees, what matters isn’t whether each team member has completed the training—it’s whether they’ve mastered the skills your team needs. 

Develop is designed to advance learners through a learning path only when they’ve demonstrated mastery of the skills they need to level up. It also provides team managers insights into what skills each of their team members are learning, how much time they’re spending on learning, and their level of proficiency in specific skills.

Gain deeper insights into team members’ skill proficiencies in CodeSignal Develop’s Analytics.

Want to learn more about how companies are using CodeSignal Develop to close skill gaps on their teams? Check out a recording of our Innovation Spotlight webinar, which showcases how Google and others are using Develop to level up their teams’ skills.

The post Product spotlight: CodeSignal Develop appeared first on CodeSignal.

CodeSignal Learn is for anyone launching a technical career, pivoting into a new role, building competitive and in-demand skills for a job search, or wanting to level up in their current role. 

Here’s how it works

• Follow personalized skills paths. Every individual’s learning journey is unique. Our approach begins with getting to know your current experience and skills to ensure you don’t waste time rehashing the basics, or face advanced concepts ahead of schedule. As you progress, the platform steers your journey in alignment with your career goals.

• Learn with Cosmo: The smartest AI guide in the universe. Cosmo, our friendly AI guide built into the Learn platform, creates a learning journey built just for you. Cosmo prompts you with personalized challenges and unblocks you when you get stuck. He’s designed to create a one-on-one learning experience that is both challenging and supportive.

• Engage with rewarding, practice-based learning. Research shows that the most satisfying learning experiences happen when you’re totally engrossed in solving challenging problems. CodeSignal Learn follows this principle by replicating on-the-job experiences where you’ll solve problems designed to push the edge of your abilities, broken down into short lessons that fit into your busy schedule. 

• Build in-demand technical skills. CodeSignal has helped hundreds of top tech companies build great hiring processes, so we know what skills matter to employers today. CodeSignal Learn hones your learning on these core skills with learning paths for a wide range of in-demand skill areas such as AI, Data Science, Full-Stack Engineering, Front-end Engineering, and more. 

• Prove your hire readiness with real mastery, not just a certificate. Achieve skill mastery through real-world coding challenges and adaptive learning paths, rather than passively clicking through content. You advance through levels only after achieving hire-ready competency. Each course you complete is then added to a skills profile that employers trust.

What developers are saying about CodeSignal Learn

“I find the interaction with Cosmo highly valuable. Having a ‘live’ code reviewer and mentor is a great feature. It allows me to receive immediate feedback on my code and gain a deeper understanding of the concepts.”

“Bite sized. Easy to follow. Concise. Really good for maintaining pace of study.”

“Cosmo is an awesome tool for learning because you don’t have to look back on the lessons or look up any syntax. When you’re stuck he’s very good at pointing you in the right direction… It’s really cool how he can look at your code and tell you about it.”

“For the Lessons, my favorite part is definitely the examples of where the concept could be seen in real life, as well as solutions in how that might look in Python.”

“I like that the lessons are broken up into bite sized pieces that make learning a new concept seem very achievable.”

“Chatting with Cosmo was extremely enjoyable. Cosmo answered questions very well even when they were ambiguously worded.”

“I really feel like this kind of thing is the future for learners like me.”

Ready to get started?

Sign up to get started for free.

The post Introducing: CodeSignal Learn appeared first on CodeSignal.

This all may make you wonder: Are we still assessing our engineering candidates in the best way possible? And how can I know if we are?

There are a number of questions you need to answer in order to improve your hiring strategy:

• What’s the impact of our hiring process on the bottom line? 
• Is our technical hiring process efficient and effective, especially when compared to other companies in our industry?
• How can we tell if a candidate used generative AI to complete an assessment? Can we trust our assessment results? 
• At the same time, should we enable candidates to use generative AI, since it’s becoming commonplace on our teams?
• How can we gain a deeper, more well-rounded view of each candidate in order to make better hiring decisions?

That’s where CodeSignal can help. Our latest product enhancements deliver the insights and features you need to thrive in today’s quickly-evolving talent landscape. 

These enhancements are..

Executive Analytics dashboard

Our new Executive Analytics dashboard connects the performance of your hiring process to cost savings, time reduction, and candidate quality metrics. This allows you to:

• Quantify savings: See the amount of time and money your organization is saving, substantiating your ROI.
• Refine your hiring metrics: Track and improve your hiring process with the ability to measure key metrics like time-to-hire and cost-per-hire.
• Ensure candidate quality: Compare your assessment score distribution and more with industry peers, as well as gain insight into overall skill proficiency.
• Dive deeper into your candidate pool: See a geographical distribution of your candidate pool, learn about candidate preferences, and dig into the high-level candidate experience sentiment, all of which can inform your sourcing strategy and hiring process.

Pre-Screen Analytics

Zoom in even closer on metrics with CodeSignal’s Pre-Screen Analytics, which allow you to build an industry-leading technical hiring process with key performance metrics and benchmarks for each assessment. Use Pre-Screen Analytics to:

• Drive process efficiency: With critical funnel metrics, you’ll be empowered to identify and address bottlenecks where candidate drop-off exceeds benchmark.
• Ensure candidate quality: Compare your score distribution and more with industry peers.
• Manage interview volume: Regulate your interview volume effectively with insights into how scoring thresholds impact your candidate pass-through rate.
• Get a well-rounded view of your candidate pool: Gain insight into skill proficiency, candidate preferences, and overall candidate experience sentiment to deliver a better process for candidates and hiring teams alike.

Suspicion Score

We’ve leveled up our Similarity Score with product improvements that enhance our platform’s ability to flag suspicious activity during a CodeSignal evaluation. With our new Suspicion Score, you can maintain the integrity of your hiring process by flagging potentially suspicious activity from a variety of sources, including AI-aided plagiarism, previous candidate submissions, and more. The Suspicion Score:

• Dissects each solution to identify patterns across the millions of interviews and assessments conducted on the CodeSignal platform
• Looks beyond code to reliably identify activities correlated with the use of AI coding assistance or unauthorized resources
• Synthesizes these factors to assign a trust level to each result, giving you confidence in your candidates’ results while flagging those that require further review

Deeper skill insights

Trusting the integrity of a candidate’s solution is the first step toward making a great hire. The next one is digging deeper into their skills. With our new Code Quality Score and AI Insights, you get detailed insights into a candidate’s performance on a CodeSignal evaluation.

• Code Quality Score: Get directional insights on the quality of a candidate’s code—not just the correctness of their solution.
• AI Insights: Streamline decision-making by summarizing a candidate’s assessment performance with easy-to-understand takeaways for technical and non-technical audiences.

AI-powered coding assistant, Cosmo

Last but certainly not least, we’re proud to introduce the newest member of the CodeSignal community: Cosmo! We built Cosmo, an AI-powered coding assistant integrated into our IDE, to allow candidates to leverage the power of AI while taking a CodeSignal assessment. With Cosmo, candidates can get instant support for navigating the CodeSignal platform—and take advantage of the AI revolution, just as they would on the job, with the ability to write code, debug solutions, and more.

Plus, the candidate’s conversation with Cosmo can be recorded and made available to you, providing insight into how well the candidate can leverage generative AI when coding.

Learn more

Want to see these exciting product enhancements in action? Contact or CSM or schedule a call with one of our experts.

The post AI and analytics: We’ve got answers to your burning questions appeared first on CodeSignal.

But this year, our Talent Science team wanted to go deeper than just platform insights. They wanted to know more about the candidates who take assessments on CodeSignal: how much coding experience and education they have, how they feel about the tech hiring process, what they’re currently doing in their careers, and more. 

Our 2023 State of Engineer Hiring Survey showcases the key findings from our survey of thousands of candidates who took a Certified Evaluation on CodeSignal. Here are three of the most exciting things we learned:

Back-end engineers earn the most

Among our survey respondents, back-end engineers reported the highest salaries, with a median annual salary of $120,000 to $139,999. 

They are followed by full-stack and front-end developers, who had median annual salaries of $100,000 to $119,999. 

Python 3 is the most commonly-used coding language 

An overwhelming majority—78%—of survey respondents reported that they frequently use Python 3. This was followed by Java (62% of respondents), JavaScript and NodeJS (40%), and C++ (37%).

We additionally found that Python 3 dominates as the most popular coding language across specific engineering roles, with the exception of front-end development (their most frequently used coding language is JavaScript/NodeJS).

Experience, not education, is associated with coding skill

In addition to learning what coding languages engineers are using, what roles they hold, and how much they’re paid, we wanted to know more about how a candidate’s background (their experience and education) correlated to their level of coding skill, measured by their score on a CodeSignal Certified Evaluation.

What our Talent Science researchers found was that years of coding experience—not education level—predicted a candidate’s coding score. On average, the more years of experience candidates had, the higher they scored on a CodeSignal assessment. This included experience from any source: work, school, or learning as a hobby.

Learn more

Interested in digging deeper into these findings and our methodology? Read our 2023 State of Engineer Hiring Survey. 

The post Top-paying tech roles in 2023—and other key findings from our State of Engineer Hiring Survey appeared first on CodeSignal.

Updated April 2024

What generative AI means for technical hiring, now and in the future

ChatGPT, OpenAI’s artificial intelligence-powered chatbot, has inspired both awe and fear in the tech industry and beyond. Since its public launch in November 2022, ChatGPT has proven to be adept at drafting college essays, analyzing data, and even writing code—with the latter having obvious implications for technical hiring. 

Working in partnership with OpenAI, CodeSignal’s data and engineering teams have analyzed the impact of ChatGPT on our live and asynchronous technical assessments, now and in the future. In this blog post, we’ll address a few of the most common questions about technical hiring and AI-powered tools like ChatGPT:

• • What is ChatGPT, and how good is it at writing code?

• • Is it possible to tell if a candidate used ChatGPT or another generative AI solution to solve a CodeSignal question?

• • How does CodeSignal make use of generative AI to enhance our evaluations?

Overview of ChatGPT as a code assist tool

ChatGPT, short for Chat Generative Pre-trained Transformer, is a browser-based chatbot that excels at mimicking human speech. It works a bit like a search engine—but rather than producing a list of search results in response to a query, it generates an original and coherently worded (though sometimes factually incorrect) answer. 

If prompted, it can also generate and improve code in a number of popular coding languages, including Python, JavaScript, and C++. Researchers recently tested its skill as a debugger, and found ChatGPT to be on par with or even beat out other automated code repair tools. 

The power of ChatGPT to do complex tasks (like writing and debugging code) lies in the innovative training of its AI model, which included billions of data points and human trainers. In fact, CodeSignal worked in close collaboration with OpenAI as they developed and trained the GPT-3 and Codex models that power ChatGPT. Some of our anonymized coding data and questions have even been used (with our consent!) in the training of ChatGPT models. 

Understandably, some engineer hiring teams are concerned about how ChatGPT and similar technologies will impact their technical evaluations. Are candidates using ChatGPT to cheat on their assessments? And if so, can it be detected?

Here’s what we at CodeSignal have done to address the use of AI tools in our evaluations.

There are clear tells of ChatGPT-generated code

The use of generative AI tools, including ChatGPT, often leaves footprints of detectable indicators that CodeSignal currently tracks using our proprietary technology. We communicate these to company users with our Suspicion Score. 

The Suspicion Score flags potentially suspicious activity from a variety of sources, including AI-aided submissions, previous candidate submissions, and more, by:

• Dissecting each solution to identify patterns across the millions of interviews and assessments conducted on the CodeSignal platform
• Looking beyond code to reliably identify activities correlated with the use of AI coding assistance or unauthorized resources (for instance, copying and pasting large blocks of code into CodeSignal IDE)
• Synthesizing these factors to assign a trust level to each result, giving you confidence in your candidates’ results while flagging those that require further review

We also recommend companies use proctoring in their assessments, which provides an additional layer of protection against cheating via ChatGPT. 

The future of technical hiring must embrace, rather than shun, AI tools

At CodeSignal, we believe that AI-assisted coding solutions like ChatGPT and Github Copilot will eventually become the norm in software development. We developed an AI-powered coding assistant, Cosmo, that is built into our IDE.

With Cosmo, candidates can get instant support for navigating the CodeSignal platform—and take advantage of the AI revolution, just as they would on the job, with the ability to write code, debug solutions, and more. Plus, the candidate’s conversation with Cosmo can be recorded and made available to you, providing insight into how well the candidate can leverage generative AI when coding.

Looking to the future, we plan to regularly adapt our Certified Evaluations to continue mimicking what real-world developers do in their respective job categories.

We believe ChatGPT is here to stay and will become a tool that software developers and engineers use in their everyday work. The future of technical hiring requires learning how to embrace this tool and innovate new ways of evaluating core engineering skills that incorporate ChatGPT and other AI resources.

The post Can ChatGPT pass a CodeSignal assessment? appeared first on CodeSignal.

Buying or renewing CodeSignal on Marketplace

As of November 2022, AWS users can use their committed cloud spend to adopt CodeSignal’s Pre-Screen and Interview solutions, which both provide advanced job simulation technology to hire high-quality and diverse technical talent. 

Prospective and current CodeSignal customers can take advantage of several benefits that come with purchasing software through AWS Marketplace. These include:

• Simplifying the procurement process
• Consolidated billing and renewal management
• Leveraging a budgeting source via AWS purchase power agreements (PPA)
• Meeting AWS spends minimum to qualify for larger discounts via the enterprise discount program (EDP) 

What is AWS Marketplace?

AWS Marketplace is a digital catalog with thousands of software listings from companies including CodeSignal. The Marketplace makes it easy for AWS customers to find, test, buy, and deploy software.

When you want to buy CodeSignal via AWS Marketplace, reach out to our sales team directly or via the public listing, and we will work with you to develop a quote that reflects your specific needs. We’ll then send you a Marketplace Private Offer to accept. With a Private Offer, you get exactly what you need and still enjoy all the benefits of buying through Marketplace.

Important note: Any existing CodeSignal customers that are looking to do a renewal or expansion must use a Private Offer. Please talk to your Customer Success Manager (CSM), who will be happy to help you.

What are the benefits of using AWS Marketplace?

As we touched on earlier, there are a number of benefits that come with purchasing software through AWS Marketplace. Let’s look at these in some more detail:

• Ease of procurement: The charges for Marketplace purchases show up on your next AWS invoice. There’s no need to set up a billing relationship with CodeSignal—you pay for your CodeSignal contract simply by paying your AWS bill.
• Sourcing budget: Many organizations make an upfront commitment to spend a certain amount of money with AWS in exchange for discounts. This is known as a purchase power agreement (PPA) or enterprise discount program (EDP). Most companies will allocate budget to cover this commitment. Since AWS considers 50 percent of any Marketplace purchase as native AWS spend, your finance team might be willing to let you pay for half of a Marketplace purchase using funds drawn from your AWS EDP budget. Talk to your finance team to learn what’s possible.
• Increased AWS spend: AWS has minimum spend levels an organization must reach to qualify for the discounted pricing of a PPA or EDP. Since AWS counts 50 percent of any Marketplace purchase as native AWS spend, organizations that are close to qualifying for discounts often use Marketplace to get their spend level over the line. In addition, many organizations that already have an EDP use Marketplace to increase their AWS spend so they can qualify for larger discounts in the future.

To learn more about CodeSignal’s leading technical interview and assessment solutions and purchasing them through AWS Marketplace, please request a call with our team.

The post Maximize your AWS budget: Buy or renew CodeSignal on AWS Marketplace appeared first on CodeSignal.

• #1 for Overall Results
• #1 for Implementation
• #1 for Usability

One G2 reviewer who used CodeSignal to hire technical candidates wrote, “We really value that all candidates have the same assessment experience and that we can use data to identify the strongest candidates to interview, thus saving hundreds of hours of our engineer’s time.”

See how CodeSignal stacks up against the competition

G2’s Grid reports show how companies compare in terms of market presence and user satisfaction. We’re proud to see CodeSignal recognized as a leader in both of these!

Learn more

Want to see for yourself how CodeSignal can help you hire top candidates while reducing engineering time spent interviewing? Let’s talk.

The post CodeSignal brings the heat in G2’s Summer 2022 reports appeared first on CodeSignal.

Here at CodeSignal, we asked: what are the top schools for engineering if you look at students’ actual coding skills—rather than the US News report’s obscure methodology?

The second annual CodeSignal University Ranking Report does just that. This report analyzes the results of the General Coding Framework, a widely-adopted validated skills assessment for evaluating core programming and computer science knowledge taken by more than 50 percent of graduating computer science majors in the US. 

Based on this data, the report ranks universities based on the percentage of students who took the GCA that scored over an 800 (equivalent to the 84th percentile).

Here are some of the key findings of our 2022 report:

• 12 of the top 30 schools in our skill-based ranking did not make the US News & World Report’s top 30 undergraduate engineering programs.

• Even the #1 school in our skill-based ranking, University of Virginia, Charlottesville, didn’t make the US News top 30.

• UC Berkeley and Stanford, tied for #2 in the US News ranking, don’t make our top 10.

• 24 of the top 50 schools in our skill-based ranking are Hispanic-Serving Institutions (HSIs) or Emerging HSIs.

To see our top 50 ranked universities in the US for 2022, you can download the full report for free.

The post CodeSignal’s 2nd annual University Ranking Report redefines what it means to be a top engineering school appeared first on CodeSignal.

• Most Implementable
• Highest User Adoption
• Best Results
• Best Usability
• Easiest Admin
• Best Relationship for Enterprise
• Easiest To Do Business With for Enterprise
• Overall Leader for Small Business

G2 collects and publishes authentic peer reviews of business software. More than 9 out of 10 reviewers have given CodeSignal a ranking of 4 stars or above, year after year.

What our users have to say

In recent reviews of CodeSignal on G2, engineering hiring managers, recruiters, and candidates themselves remark that they find the platform intuitive to use and an effective tool for evaluating candidates’ coding skills.

Here is a sampling of a few of these reviews:

Learn more

Want to see for yourself how CodeSignal can help you meet your technical hiring goals this year? Schedule a demo to find out what all the hype is about.

The post CodeSignal springs forward in G2 reports appeared first on CodeSignal.

Luckily, a better way is possible. CodeSignal’s new guide digs into how to revolutionize your company’s technical hiring processes to hire better candidates, faster—whether your candidate pipeline is high or low volume. 

This guide will take you through optimizing your recruiting funnel to:

• Improve candidate experience
• Dramatically reduce engineering time spent on recruiting
• Increase pass-through rates for quality candidates with an objective, unbiased process
• And much more 

Download the free guide here to get started.

The post CodeSignal publishes guide to optimizing your recruiting funnel appeared first on CodeSignal.

To discuss the challenges and best practices for capacity planning, we held a one-hour webinar with CodeSignal’s Co-Founder and CEO, Tigran Sloyan, and VP of Marketing, Brenna Lenoir, along with Robinhood’s Head of Tech Recruiting, Craig Campbell. Craig shared the best practices he’s learned from leading tech recruiting at high-growth startups like Robinhood and Uber. The conversation also touched on what TA can borrow from the marketing side of the business to help with planning under pressure. 

Here are the top takeaways from that conversation, which you can watch in full here.  

1. Involve engineering in your capacity planning

“One of the things we partner with our engineering stakeholders on is the capacity planning for interviewing,” Craig said. “We have the ability to articulate: In order to deliver X, we forecast that we’re going to need X interview hours per week.” 

You’ll need some kind of visualization software — everything from a well-designed and reliable Google Sheet to a more complex platform like Tableau can work. What you’re looking for is a way to articulate the workflow, the owners at each stage of the workflow, and the productivity associated. 

Your stakeholders will then do the math, and understand how exactly they’re connected and accountable to the process. As Brenna said, “You’re having a conversation with your hiring managers or your heads of department to say, ‘our model is showing me this many hours from you in interview time. Can you commit to that? Because if you can’t, we’re all going to fail in this.’” 

2. Watch out for “watermelon metrics”

Tracking the right data is essential to be able to make adjustments, since no plan is perfect from the start. Robinhood looks at a variety of metrics: “At the highest level, we look at offer acceptance, NPS rating (which is giving us a sense of our candidate sentiment), time to hire, and the pace in which we deliver on that,” Craig said.

Keep in mind that some high-level metrics could be so-called watermelon metrics: green on the outside, red on the inside. When you doubleclick into these metrics, you might find places where you can still improve. As Craig explained, “You might be able to zero in on one component that you need to solve for — kind of the ‘smoking gun’ — that will unlock exponential performance in your pipeline.”  

It can be tempting to go more granular on everything, but where should you be investing your (limited) time? One area, Brenna suggested, could be the specialized roles that you’re trying to hire for: “They may not follow the same metrics that you have for your overall funnel, so that may be an area where it does make sense to dig a level deeper.” 

3. Understand your talent like you understand your customers

“The name of the game today is definitely engagement,” Craig said. Engagement is what will make or break your efforts to recruit in a hard-to-hire market like engineering, especially with communities that are traditionally underrepresented in tech. And yet there’s a problem. Tech companies are great at collecting intelligence on their customers, but as Craig put it, “we lack intelligence about our talent communities in general.” 

To understand talent communities, recruiting can learn from marketing, where there are customer personas. As Brenna explained, “Persona marketing is ‘how well do you understand your target audience’? Do you know where they get their information, what they care about, what keeps them up at night?” 

Perhaps most importantly, what opinions do prospective candidates have about your company? Getting this understanding is especially important to recruit engineers who may come from traditionally underrepresented backgrounds.  

4. Use technology to free up resources

At a certain point, scaling is inevitably going to require some input of technology. “If you’re really still relying 100 percent on people to get to scale, that might be a losing proposition,” Craig said. “Humans are only going to be able to do so much. You need these enablement platforms of technology to relieve the humans to do what they do best.”

These might be tools that help you get more intelligence about your candidates. Increasingly, talent acquisition teams are borrowing from other departments: “Tools that have been commonplace in marketing are starting to show up on the recruiting side,” Tigran observed. 

Or, these could be tools to help you streamline the hiring process. At Robinhood, Craig’s team uses CodeSignal as part of their tooling stack to help democratize tech hiring, scale effectively, and get a high ROI on engineering hours. “Everybody experiences a core-in-common assessment stage,” Craig explained, “so engineers feel like most of the conversations they’re having, there’s not a waste of time. They’re talking to the right people as the result of the output of this assessment stage.”

Could CodeSignal help you optimize your tech hiring? Click here for a free demo. 

The post Tips from Robinhood’s Head of Tech Recruiting on how to confidently plan for your recruiting needs in 2022 appeared first on CodeSignal.

If you feel uncertain about any of these questions, you’re not alone. The start of the year can be especially stressful for Talent Acquisition leaders. But don’t worry — we’re hosting a FREE webinar where you can learn how to create a strategic plan and get a jump start on hitting your recruiting goals in 2022. 

Join us on February 3, 2022 at 10 AM PT / 1 PM ET for a conversation with Craig Campbell, Head of Technical Recruiting at Robinhood, along with CodeSignal’s CEO, Tigran Sloyan, and VP of Marketing, Brenna Lenoir. In this one-hour webinar, you’ll learn:

• How to calculate funnel metrics that predict applicant volume and recruiting team headcount needs 
• How to track and manage data-drive progress towards your hiring goals
• How to ask for more resources with confidence

Interested? Reserve your spot for FREE today.

The post Feb 3: How to confidently plan for your 2022 recruiting needs appeared first on CodeSignal.