Section Three: How is Competence Measured and Communicated?

As any hiring manager can tell you, approaching a stack of resumes without a clear idea of the specific qualifications needed for the job is a recipe for inconsistent, inefficient, and potentially biased hiring practices. But when standard indicators of learning (like university degrees) often do not correlate well with the knowledge, skills, and abilities most directly relevant to the workplace, what should hiring managers be looking for when they sort through a stack of applicant resumes? The answer depends on context. The following section will explore how employers and jobseekers use industry certifications and other tools to communicate competence, highlighting that each method’s overall utility depends on the way in which they are used.

Certifications provide a theoretical framework that could be used to create an entry point into the field, but hiring patterns suggest that the certifications are used as a proxy for work experience more than an indicator of competence. Coupled with the expense of training and testing, this limits the effectiveness of certifications in creating additional entry points into cybersecurity jobs.

Industry certifications are one alternative measure of competence already widely in use in cybersecurity. In fact, 35 percent of job postings call for industry certifications to demonstrate competence.¹ Cybersecurity has a wealth of certifications denoting different levels of technical ability, so much so that knowing which to pursue can be a challenge for cybersecurity experts (and soon-to-be-experts). Industry associations like CompTIA, training providers like the SANS Institute, and professional organizations like the International Information System Security Certification Consortium (more widely known as (ISC)2), ISACA, and the International Council of Electronic Commerce Consultants (EC-Council) each have their own suite of credentials with different certifications for different levels of expertise, specific skill sets, or particular vendors or products.

The CyberSeek website jointly developed by NICE, CompTIA, and Burning Glass monitors popular certifications as a ratio between holders of certifications and number of job postings requesting those certifications.² As of this writing, there are 78,523 open positions in the United States requesting candidates with a CISSP certification, and 76,413 current certification holders.³ There are more open positions seeking CISSP holders than there are CISSP holders in total—the vast majority of whom, of course, are already employed.

Costs to take the tests necessary to obtain these certifications vary widely; CompTIA’s Security+ is $330,⁴ whereas attempting the GIAC test runs $1,899.⁵ Additionally, many providers offer study guides or classroom trainings designed to prepare prospective test-takers for the exam. For example, CompTIA offers a study book for the Security+ exam for $149, or a bundle of resources—including the exam fee and a retake fee—for $1,049.⁶ Given this, potential test-takers must consider the expected return on investment for pursuing the certifications, but data on this is limited.

Statistics from the organizations that operate these tests present a compelling case for the return on investment of certification. (ISC)2’s 2015 Global Information Security Workforce Study compares average salaries of security professionals who hold an (ISC)2 certification⁷ with those who do not: $103,117 for certification holders and $76,363 for the non-certified,⁸ which makes the test an easy sell for those candidates that are qualified to take it. (That data correlating earnings and certification almost certainly reflects additional factors. For example, professionals further along in their careers are both more likely to earn a higher salary and to have had an opportunity to take the test.)

The cybersecurity community would benefit from more in-depth, third-party research on the economics and expected outcomes of certifications in cybersecurity. (ISC)2’s numbers do generally track with non-industry-specific expectations for the returns on industry certifications. The Bureau of Labor Statistics collects data on the impacts of certifications on salary via the Current Population Survey, and has found that the median earnings of a worker are $793 per week without a certification, and $1,145 per week with a certification.⁹ Furthermore, research into certifications in other industries shows that they can enable career transitions across industries. Because certifications can demonstrate competence in core or transferrable skills, if an employer or section of the market becomes obsolete or unprofitable, workers with certain certifications may be more able to parlay their experience into a career transition.¹⁰ All other things being equal, there are good economic reasons for an individual to pursue industry certification.

While earning a certification does not guarantee a job or higher earning potential—especially in cybersecurity where the impact depends heavily on which certification one pursues—it does offer a way for jobseekers to distinguish themselves and demonstrate their workplace-relevant capabilities. Unfortunately, patterns in hiring limit the extent to which these certifications are useful as a means for offering a clear entry point to jobseekers trying to enter the cybersecurity industry. The most in-demand certifications often require years of work experience, whereas the more commonly held early-career certifications see much less demand. Analytics company Burning Glass breaks certifications into two broad categories: “Door Openers, which help new labor market entrants enter a field; and Career Escalators, which pave the path for experienced workers’ upward mobility.”¹¹

There are more open positions seeking CISSP holders than there are CISSP holders in total.

Using this framework, cybersecurity’s most in-demand certifications are Career Escalators, rather than Door Openers, which limits their utility as an alternative means for breaking into the industry. For example, the Certified Information Systems Security Professional (CISSP) is the most frequently requested certification,¹² and the certification is only granted to applicants who pass a test and can demonstrate five years of work experience. There are variations on the CISSP certification for applicants who can pass the test but do not have the required work experience, and a year of experience can be waived based on education or prior certifications,¹³ but these adaptations are only useful if employers are willing to hire applicants who have taken these routes.¹⁴ Not all certifications have the same requirements for work experience, but applicants with more easily obtained certifications are generally much more common.

While industry certifications could offer an alternative point of entry into a cybersecurity career, and Door Opener-type certifications do exist in cybersecurity, it does not appear that they are often being used for this purpose. As long as employers are predominantly relying on the Career Escalator-type certifications that come with years of work experience, these certification regimes will not creating an alternative entry point into the industry on a meaningful scale. Having a means of demonstrating a sound grasp of the fundamentals does very little good if employers are only willing to hire employees with advanced knowledge and experience.

Forums like online platforms and cybersecurity competitions could add to the number and variety of mechanisms for demonstrating competence, but they are not yet to the point of being effectively scalable. Carefully designed hiring policies and funding mechanisms could incentivize growth and best utility of these mechanisms.

Certifications are not the only means of demonstrating competence. Another promising means for communicating this follows the model of the TechHire initiative that began with the support of the White House in 2015.¹⁵ The program is a combination of training, evaluation, and community outreach designed to provide a means for candidates to demonstrate their skills, to supplement their knowledge as needed and to connect them with employers.¹⁶ Programs like this generally require external support, which limits their growth. Moreover, while the information technology sector more generally has some successful examples of this approach, cybersecurity does not. A valuable proof of concept would be the creation of a low- or no-cost indicator of competence, ideally coupled with a teaching platform, that carried the credibility of a well-respected organization. Carefully crafted funding mechanisms could incentivize the development and operation of such an indicator. More generally, with appropriate support, organizers and developers could provide meaningful pathways into the cybersecurity workforce.

Hiring models that reward self-taught talent offer particular promise in cybersecurity, a field with a culture that prizes hackers and curiosity. Information security is a community known for its particularly individualistic ethos, and there is already a legacy of hiring based on demonstration of individual skill, rather than formal education. Capture the flag competitions (CTFs) are a cornerstone of many major information security conferences, and, anecdotally speaking, many organizations use these competitions to identify potential future hires.¹⁷

Similarly gamified cybersecurity exercises are increasingly popular as clubs for students. The CyberPatriot National Youth Cyber Education Program, for example, is a nonprofit program that runs the annual National Youth Cyber Defense Competition. Teams are organized around schools, girl scout and boy scout troops, and other youth organizations. These teams compete remotely at intervals throughout the school year to practice and demonstrate students’ ability to secure computer systems.¹⁸ Having just completed its tenth year, the program boasted 5,584 teams registered to compete in the 2017-2018 competition.¹⁹ Similarly, the National Collegiate Cyber Defense Competition offers college students an opportunity to put theory to practice by competing with other schools to defend a simulated company network.²⁰ Programs like these reinforce the ethos that cybersecurity skill is something to be demonstrated in practice while also giving students hands-on experience to inform their education.

Because there is a strong tradition of demonstrating competence through competitions rather than through degrees or certifications, this community could be particularly accepting of hiring practices that use competitions to identify qualified candidates. However, data is scarce on just how widespread such hiring is. Gaining a more thorough sense of this hiring dynamic would be an excellent area for future research.

One of the challenges to using cybersecurity competitions as a hiring mechanism is that such hiring pathways are only useful to employers that specialize in cybersecurity itself. There are a wide range of jobs that require cybersecurity skill, but that cross over into other sectors like healthcare, finance, or transportation. Employers in these hybrid jobs are not likely to have the contextual knowledge of the cybersecurity community to know how or where to recruit competition winners. Recognition from a capture the flag event is not likely to mean much to a hospital administrator or port authority manager who may not understand which specific technical skills that particular competition demonstrates. While some employers may be able to use competitions as a hiring tool, the benefit is limited to those employers already familiar with the CTF scene.

A second challenge with using competitions to address the cybersecurity workforce shortage is identifying how it could be scaled beyond its present capacity. If this kind of hiring is already occurring, what can be done to widen this onramp into a cybersecurity career? Increasing the number of competitions does not necessarily increase the number of qualified participants. What can be done to encourage potential participants to pursue the skills that would distinguish them as a competitor? Such demonstrations of skill provide an alternative way for jobseekers to communicate competence, but in order to have a meaningful impact on the workforce shortage, they must be scalable.