Meeting Summaries

A Field of Silos

In the opening session, participants identified the key grid security concerns they hoped to discuss at the retreat. A consistent theme was the fragmentation of policy, planning, technology implementation, and even knowledge, with participants calling for proactive prioritization, harmonization, and integration across the electricity enterprise, from rural co-ops to large investor-owned utilities to government officials and researchers. That continuum of industry players has robust and mature partnerships when it comes to dealing with natural events, but dealing with direct national security threats is new, and those relationships still need to develop. Indeed, most participants said there is a lack of clarity about threats to the grid and inadequate preparedness, particularly when it comes to state-sponsored cyber attacks. “We need a better understanding of our own threat landscape and vulnerabilities,” one participant noted, “there’s a lot of hype.” There was a sense of missed opportunities “to mitigate fear” in public opinion, born of bad information, or to incorporate security intrinsically in new technologies, including renewable energy. A number of attendees pointed to a “compliance mentality” as a barrier to progress. One participant warned that without taking a holistic approach that accounts for interdependencies, we are only building “fake resilience,” which left others asking for a better, actionable definition of resilience. Money was a leitmotif throughout this session, with wide agreement that however we define resilience, it will take investment to achieve it — but who will pay? Everyone expected important lessons out of Puerto Rico.

Not So New Directions?

“Critical infrastructure is an afterthought until it doesn’t work,” one expert said, which sums up this session’s discussion about the (bipartisan) tendency toward reactionary policy, politics, and legislation when it comes to the electric grid. Indeed, one participant pointed out that the U.S. Congress actually tried to outlaw blackouts after the 2003 power outage. In that regard, no one at the meeting expected the Trump Administration or 115th Congress to be radical departures: there is no sign at this time of a coherent national electricity policy. Not everyone at the meeting thought the lack of an overarching policy was a bad thing, however; some cautioned against overly prescriptive or panacean policies that lag the marketplace, the threats, and the technology. It’s hard to legislate adaptability.

At the same time, the group saw an important role for government in grid security, which everyone agreed is a non-partisan concern. As one expert noted, squirrels are a utility responsibility, but nation-state threats are a federal government responsibility. Some participants said that current grid security-related regulations are inadequate or accepted with dangerous complacency. Others cited the difficulty of private sector information-sharing on threats and vulnerabilities. But no one was certain who in the U.S. Government is the “mission owner” for grid security, particularly for cyber-related threats and multi-sectoral vulnerabilities (i.e., natural gas, communications). Solutions offered to the political and policy shortfalls ranged from prioritization of critical nodes and threats, to a better planning regime, to faith in decentralization. “One should not be so quick to think that 50 states should be homogenized,” one expert warned.

The Threat Spectrum: Squirrels, Storms, and Nation States

*Please note that although we are sharing a summary of these remarks, the speaker was off the record and should not be cited by name.

The keynote speaker, a former government leader, described a “threat spectrum,” which ranges from routine natural hazards, such as severe storms, to sophisticated, deliberate attacks. This spectrum is not static: events that used to be unlikely and manageable are becoming more likely and unmanageable, from the heavy precipitation in Houston to wildfires in California to the cyber attack in Ukraine.

Russian interference in U.S. elections should be a wake up call: with a clear doctrine of early escalation in order to deter adversaries, the Russians will and can go after infrastructure, nor are they the only nation gaining such capabilities. At the same time, we have to balance that growing threat against the considerable misinformation or disinformation about threats to the grid. How do we know what’s true?

The complexity and interdependencies of critical infrastructure are part of the challenge, which means that any one element of the grid system is only as secure and resilient as its weakest link. Communications, for example, were a weak link in the 2013 Metcalf attack, in Ukraine, and in Puerto Rico. The supply chain itself can be a weak link.

The speaker cautioned that whatever the benefits of digitization, there are also risks in “engineering out the human element.” In Ukraine, the operators really knew their system and were able to troubleshoot and mitigate the damage manually. In contrast, there’s a coming scarcity of experts in the United States, with a wave of retirements. Given this picture, the speaker wondered why security and resilience are “bolted on” as an afterthought, especially given increased interest in and momentum for smart technologies. The stakes are high, because while utilities and the public sector are good at dealing with hurricanes, we’re not ready for truly high consequence events with effects across sectors. It’s time to use a public-private partnership to create a framework for agility.

New Threats, Hazards & Risks for the U.S. Electric Grid

The session started with a summary of remarks from the previous evening, largely that the electric grid is a complex, highly interdependent ecosystem in a state of change and that the country is not ready for a major event.

Participants agreed about the difficulty of charting the new threat landscape, ranging from lack of hard science, to knowledge gaps, to interest groups with a specific focus that catalyze Congressional action. There should be a standard way of assessing threats, potential system impacts, and factual, data-driven assessments of response strategies. One expert shared the draft Electricity Subsector Coordinating Council (ESCC) threat matrix, which took about a year to develop and will ideally incorporate cascading effects and multiple sectors. Participants also thought there should be a gap analysis. Another expert quickly countered that the biggest threats are still squirrels and trees, and that misinformation or even “weaponized” information remains a significant concern. One participant underscored the need for a common operating picture and threat identification before contingencies occur, suggesting that some brand or seal of approval, such as a “Cyberstar” akin to an “Energy Star” is worth considering. There was discussion about GridEx as a diagnostic tool for gaps, with wide agreement that this public-private exercise is very helpful, and a few observations that it could be improved (e.g., the event is generally in November, using regional, real-time weather data, which is not necessarily the most stressing case).

Several participants noted that top industry executives are committed to being more proactive, often with access to classified information, which helps the industry understand how to target investment in research and risk mitigation. That may not extend to all parts of the industry, however, and there were concerns that Investor Owned Utilities (IOUs) may have better threat information than municipal or cooperative utilities.

There was both discussion and disagreement about the role of the Department of Defense in grid security. Many participants expressed confidence in DoD; others cautioned that DoD should not be in charge of restoration or threat information sharing. Defense missions may actually compete with restoration priorities for transportation nodes and other resources during a crisis.

New Technologies, from Smart Grids to Generation

Experts from four national laboratories presented on innovations to improve grid security, followed by a group discussion. Resilience was a theme throughout the session, though one participant noted that the term needs a more tangible, actionable definition. Keeping up with determined adversaries is difficult, if not impossible, so inherent resilience is important, but not common in the legacy system. A utility executive noted that nuclear generating stations tend to be an exception, with extensive systems-level resilience built in.

One key area of research is in better diagnostics. With cyber threats, for example, when a threat or anomaly is detected, the current practice is generally to slow traffic flows. That can actually exacerbate the consequences, such as denial of service. Approaches to improved diagnostics include hardware-in-the-loop, rapid prototyping, and modeling virtual “playgrounds” or “walled gardens” to simulate power systems, power electronics, various perspectives on threat, and interdependent systems, without actually “flipping the switch.” One researcher highlighted the promise of machine-to-machine automated response, when microseconds matter.

There was agreement about the importance of collaboration when it comes to innovation. The Grid Modernization Lab Consortium (GMLC), for example, streamlines and pools the resources of 13 national laboratories, looking at areas such as reducing the time and cost to integrate new technologies, “plug and play” technologies, standards that will improve recovery times after an outage, and security resilience metrics. Utility executives also hailed the ESCC and NERC’s Electricity Information Sharing and Analysis Center as key private sector cooperation mechanisms and proposed community reliability assessments.

Throughout this session and the entire event, there was a great deal of discussion about decentralization in the grid system, and microgrids, in particular. “Whether or not we decide microgrids are the solution, that is where the momentum is,” one expert commented, with others noting that technology companies and the U.S. Air Force, in particular, are moving to microgrids. Shifting from centralized systems to decentralized systems could improve grid security in some ways and threaten it in others (e.g., a decentralized grid may provide an adversary with more discrete targets for a cyber attack, but disperse the effects of any one attack; it may eliminate a single point of failure of a centralized system and increase reliability, or it may introduce multiple points of failure with more reliability problems for smaller groups of customers). There was recognition that there needs to be more R&D focused on networking assets to provide resilience across the system, especially with distributed energy resources. Puerto Rico is a tragedy, but also an opportunity for this kind of innovation.

Another key theme for this discussion was the importance of a systems approach. The system complexity of electricity is increasing every day, with steady shifts in variability and supply, which often requires, “coordinating flexibility to offset vulnerability” according to one expert. Participants acknowledged that security is not necessarily a technology challenge: there’s a need to innovate and integrate policy, economics, and regulations as well. At the same time, one executive cautioned that it’s very important to understand those needs from “an operational perspective.” The current tendency is to focus on components, not the system, but one cannot make meaningful change at the device level. Unfortunately, the industry currently lacks the capability or capacity for comprehensive system analysis, which would better identify gaps and coordination of investment decisions.

In the discussion of the role of consumers in driving or accepting innovation, there was much discussion about the Department of Defense, as a particularly big customer and big target. This is also a customer with poor visibility of its own needs, costs, and status of reliability (i.e., all bases are required to have backup generators, but any one base may not know how many they have or may not keep them well maintained), and that’s a barrier to innovation. The FAST (Fixing America’s Surface Transportation) Act required understanding of the electricity vulnerabilities of key military installations and other national security infrastructure, a goal as yet unrealized. One executive noted that DoD’s challenges are not pervasive and the agency is a special case; all utilities work directly with customers and know this type of information.

From Hacks to Hurricanes: the View from the Private Sector

*Please note that although we are sharing a summary of these remarks, the speaker was off the record and should not be cited by name.

The speaker, the CEO and President of a large public power organization, offered a complementary perspective (there were no other public power representatives in the meeting).

He was extensively involved in the response to Superstorm Sandy, which offered many lessons, such as the importance of prepositioned equipment and good data analytics. Given that there was an extended power outage, they had to deal with considerable political pressure to get the lights back on, as well. All of those lessons are now being applied in Puerto Rico. At the request of his Governor, the CEO sent a score of the company’s experts to the island in the immediate aftermath of the storm, collaborating with the U.S. Army Corps of Engineers on the Initial Damage Assessment. Although Hurricane Maria made landfall on September 20th, the request for mutual aid didn’t surface until October 31st, and the executive actually drafted the request himself. Puerto Rico undoubtedly needed the help: in addition to the age of equipment and the utility’s debt, there were no rights of way, no access, and no vegetation management. And it would be hard to overstate the damage — he thought there might have been micro tornadoes associated with the storm, based on what he personally saw.

The speaker talked about the plan for the recovery of Puerto Rico’s grid, a $17 billion request. 50 percent of the plan is non-traditional, including microgrids and renewables. Indeed, he thought that solar panels with storage always would have been the best way to provide power for remote areas of the island. The rest of the funding would be for hardening measures.

As for grid security more generally, the speaker said it becomes a matter of risk management. He talked about ground-based sensors that alert an operations center if someone is approaching equipment and called his strategy a “beehive defense — crunchy outside and sectional inside.” He did stress that cyber “is the big one,” and that cyber-security should be built in whenever possible, as “it’s so much harder to bolt on later.” He also is very concerned about supply chain vulnerabilities. “But it’s the insider threat that keeps me up at night,” he asserted. This is both the case for deliberate acts and for workforce attrition. 35 percent of his personnel are eligible for retirement in the next two years, for example.

As for new technologies, the speaker highlighted automated distribution and wearables, including Google glasses, which give his engineers an “augmented reality.” He noted that drones have been very helpful in Puerto Rico, and that they are experimenting with submersible equipment (including submersible transformers) that can withstand flooding or other exposure to water. He considered his company lucky because so much of their baseload generation is hydropower, which is especially resilient and critical for black starts (he noted that hydro did not go down in the cascading regional blackout of 2003).

Finally, the speaker offered that a missing piece of grid resilience is coordination across sectors. Telecommunications companies, in particular, are less regulated than the electricity sector and technology companies are not regulated at all, and it’s hard to bring them to the table.