Show Me The Numbers: Reflections from the Global Forum on Cyber Expertise’s Annual Meeting

Often absent from the capacity building conversation are numbers and other metrics that the community could use to measure success or failure. Some groups, like CyberGreen, have begun to take steps to shift the conversation towards more rigor and evaluation of practices, but more work in this space is needed. Improved empirical evaluation of practices will have the obvious reward of making capacity building more effective. However, it’s reasonable to expect ancillary benefits as impact and effectiveness gets more measurable. Chief among these may be the movement of more funders—like those that traditionally fund sustainable development projects and therefore rely on metrics to discern where and how to invest—into the field of cybersecurity capacity building.

With that context, the Global Forum on Cyber Expertise (GFCE), launched by the Dutch Government at the 2015 iteration of the Global Conference on Cyber Space, is onto something. It is undeniable that there are a wide range of cybersecurity capacity building efforts going on out there, and someone should probably coordinate them. But a lot more could be done to help the GFCE reach its full potential.

Earlier this month I was in Brussels, Belgium for the GFCE’s Annual Meeting. What I witnessed there was the beginning of a much-needed sea-change in the mindset of global capacity builders and their beneficiaries. For too long, cybersecurity capacity building has relied on “best practices” underpinned by little more than anecdotes, gut feelings, and the notion that “someone else is doing it, so it must be effective.” For the first day and a half of proceedings, I and several of those around me voiced concern that the GFCE, which had committed to producing a list of “global good practices” for international capacity building by the fall, had fallen into the same mindset.

However, around mid-day on the second day of the conference, Vladimir Radunovic of the DiploFoundation presented on the GFCE effort to produce this set of global good practices. In doing so, Radunovic chose to focus on the “good” portion of global good practices, expounding on the need for more rigor and empiricism in the identification of what works and what doesn’t.

To concretize the point, I’ll give you an example. Conventional wisdom suggests that one of the first and most impactful steps a developing country can take to bolster its cybersecurity is to develop a national CSIRT. I’ll be among the first to concede that it is probably the case that national CSIRTs are a good way to bolster security. However, very little by way of empirical examination has been done to discern what a national CSIRT can and should do, how it should do it, and what cybersecurity gaps is fills and leaves unfilled. In short, not much has been done to validate the notion of the development of a national CSIRT as a best practice. This type of evaluation could weed out practices that have seen much investment but made little impact and further bolster practices—like the development of national CSIRTs—that do have material impacts.