This report explores the relationships between civil society organizations and entities that conduct oversight of surveillance activities in eight countries: Australia, Canada, France, Germany, the Netherlands, New Zealand, the United Kingdom, and the United States. This represents the countries of the Five Eyes alliance plus France, Germany, and the Netherlands. To a limited extent, the report also considers oversight in Norway. Our examination starts from the premise that oversight of surveillance can provide a critical check on surveillance powers and activities, and that oversight bodies and civil society organizations share the goal of ensuring that surveillance is conducted in accordance with the rule of law and with respect for human rights.

Our review has demonstrated that engagement between civil society and oversight bodies can be a productive tool for limiting the risks that government surveillance poses to privacy, civil liberties, and human rights. In many countries, the opportunities for civil society organizations to engage with oversight bodies are increasing. Government oversight can provide meaningful accountability for surveillance agencies and their activities, and can enforce safeguards to protect individual rights and the rule of law. In addition, because oversight bodies typically have access to classified information, they can conduct reviews and make findings in many settings that are inaccessible to public litigation or public debate in legislatures.

This report explores the ways in which civil society and oversight bodies can work cooperatively to promote robust oversight of surveillance. The report identifies and analyzes a variety of models of engagement between civil society and oversight that can serve this goal.

Engagement between civil society and oversight bodies can be a productive tool for limiting the risks that government surveillance poses to privacy, civil liberties, and human rights.

Our research for this report principally involved conducting interviews. In each of the eight countries we have explored, we conducted interviews of civil society representatives and of current and former staff for government oversight bodies. We interviewed representatives from almost all of the oversight bodies in these eight countries that both conduct oversight of surveillance activities and have engaged with civil society organizations. On the civil society side, we interviewed representatives from a variety of privacy, civil liberties, and human rights organizations that have engaged with these oversight bodies.

We first set out to examine the current state of engagement between civil society and oversight bodies, exploring the different models for engagement and the strategies that have been productive. Since the report explores engagement between civil society and oversight bodies, rather than litigation brought by civil society to challenge government activities, this report does not include a discussion of the type of oversight conducted by the courts through judicial review. Nor does it include the oversight conducted by specialized courts like the Foreign Intelligence Surveillance Court (FISA Court) in the United States or the Investigatory Powers Tribunal (IPT) in the U.K. We next identify barriers to productive engagement between oversight and civil society, possible strategies for mitigating these barriers, and instances where engagement has not been productive. Finally, we outline specific practical recommendations for civil society organizations and for oversight bodies.

Oversight bodies play a key role in ensuring that intelligence agencies exercise their powers lawfully and are accountable to the public whose security the agencies seek to protect. As an intermediary between the citizens and intelligence agencies, oversight bodies occupy a unique position and democratic function. They commonly have authority to access classified material, conduct fact finding missions, make recommendations, and educate and inform the public about intelligence practices.

In conducting oversight, these institutions serve two different and essential roles. The first is to assess legal compliance: are the intelligence agencies carrying out their activities in compliance with all applicable legal requirements? Almost all entities that play a role in overseeing intelligence surveillance serve this compliance function. The second role, which is not served by all oversight entities, is a policy or governance role, where entities explore whether intelligence agencies should be conducting these activities and whether the program rules make sense from a policy perspective. This function helps ensure that intelligence activities are not only lawful, but also legitimate in the eyes of the public.¹

Although civil society may, and often successfully does, play an important role in calling on oversight bodies to fulfill their mandates to ensure compliance, most engagement between oversight bodies and civil society organizations is actually focused on the oversight bodies’ policy or governance roles. In addition, it is helpful if the oversight bodies meet certain criteria. These criteria include that the body has clear structural independence, that it acts in as transparent a way as possible, and that the government representatives demonstrate a willingness to take input from civil society.

It is important to note in this regard that even where oversight bodies play a policy or governance role, they may not have the power to change the governing rules themselves, and may only be able to make recommendations for such policy changes. Indeed, other than in the United States, where Congress both conducts oversight and enacts legislation setting the rules governing what intelligence activities are permissible, in most countries reviewed for this report, there is a separation between the oversight bodies and the entities than can change the law regarding what intelligence activities are permissible. And even in the United States, other oversight bodies that play a role in recommending policy changes have no authority to require that such changes be made.

At a high level, oversight bodies and civil society organizations share the same goals: to ensure that intelligence collection is conducted in accordance with the rule of law and with respect for individual rights. In recognition of this fact, all the subjects interviewed for this report—both government oversight representatives and the representatives from civil society—agreed that engagement between oversight bodies and civil society organizations is a valuable goal that should serve the interests of both.

Among the different countries reviewed for this report, the maturity of the oversight bodies varies, and the relationships between these bodies and civil society groups are at very different stages. In countries like the United States, the United Kingdom, and the Netherlands, there is regular engagement between civil society and oversight bodies. In other countries, like Canada, New Zealand, Australia, Germany, and Norway, while oversight is developed, there is considerably less, and in some cases no external engagement at all. In France, the intelligence oversight body has only just been created, and in countries outside the scope of this report, there are no oversight bodies at all. Where engagement between civil society organizations and oversight bodies does exist, it can vary by the type and role of the oversight body, by the type and focus of the civil society organization, and by the nature of the surveillance and individual rights issues being discussed. We have found a variety of models for productive engagement. Given the more frequent engagements that occur in the United States and the U.K., most of our examples relate to those countries.

What counts as a “productive” engagement also varies. It can range from enactment of legislation that, at least in part, achieves the goals of both oversight officials and civil society representatives, to engagement that simply serves to educate both sides, so that civil society organizations can better understand how surveillance programs are operated and agency officials are better informed regarding civil society concerns.

Structural features

A key variable that can promote productive engagement between civil society and oversight bodies is the existence of a statutory “sunset” provision or other external action-forcing deadline.

In the United States, by design, a sunset provision induces Congress to reconsider legislation in the months prior to the statutory expiration date. In at least some instances, the threat that a program could be allowed to expire has provided an opportunity for enactment of reform amendments that strengthen safeguards for privacy and civil liberties, and a structural opportunity for meaningful engagement between civil society organizations and oversight bodies. Perhaps the best example was the June 1, 2015 sunset date for several key provisions of the USA Patriot Act, including Section 215. After the unauthorized disclosures by Edward Snowden in June 2013 that revealed the Section 215 bulk telephone records program to the public, there followed a two-year period of extensive engagement between civil society and both executive branch and legislative oversight bodies. Ultimately, after allowing Section 215 to expire for one day, Congress enacted the USA Freedom Act on June 2, 2015,² which reauthorized but limited the expiring Patriot Act provisions, and included a variety of additional provisions reforming the conduct of U.S. government surveillance.

Sunsets were also used as a catalyst for significant change in the U.K. intelligence law landscape and provided an opportunity for engagement between actors that previously had enjoyed very little contact. After the decision in Digital Rights Ireland, striking down the EU Data Retention Directive, the U.K. passed an emergency piece of legislation entitled the Data Retention and Investigatory Powers Act 2014 (DRIPA), which passed in 10 days. The Liberal Democrats, who were in coalition government with the Conservatives, succeeded in securing a number of concessions. Central to this was the inclusion of a sunset clause, and the creation of a number of independent reviews into the existing legal framework governing investigatory powers and their practice.³ This led to the public disclosure of more material about the policy and practice of intelligence collection than had been seen in decades. An impetus was created where civil society, business, law enforcement and oversight all had a reason to engage. New relationships between civil society and oversight were built, which became the foundation for future engagement.

The threat that a program could be allowed to expire has provided an opportunity for enactment of reform amendments that strengthen safeguards for privacy and civil liberties.

However, a sunset date alone will not ensure an in-depth reevaluation of programs, and the unique dynamics of the immediate post-Snowden period in the United States may not soon be repeated. Thus, although the December 31, 2017 sunset date for Section 702 of the Foreign Intelligence Surveillance Act (FISA) did prompt a great deal of engagement between civil society organizations and the congressional judiciary and intelligence committees, Congress ultimately reauthorized Section 702 without including any significant reforms to that authority.

Another structural feature that can serve as an action-forcing deadline is an upcoming change of political leadership. In the United States, this typically occurs at the end of a presidential administration, and can also occur with upcoming congressional elections and an anticipated change in the membership of Congress. An oversight body may press to complete certain projects while current leadership is in place, and the agendas of congressional oversight committees are frequently timed based on the election cycle.

These election-related deadlines can be less productive in inducing engagement between oversight and civil society than sunsets, because the outcomes of elections are difficult to predict, and thus the impetus to get something done will vary. In addition, with executive branch oversight bodies, unless they are transparent about their oversight agendas (as recommended below), the deadline for a particular oversight project may not provide an entry point for civil society engagement. For example, during the final year of the Obama administration, internal agency oversight officials at several intelligence agencies worked to finalize new Attorney General-approved guidelines governing intelligence collection, use, and dissemination under Executive Order 12333.⁴ Prior to the end of the administration, the Attorney General approved and released new guidelines for the Department of Defense,⁵ rules for sharing of raw signals intelligence by the National Security Agency (NSA),⁶ and new guidelines for the Central Intelligence Agency (CIA).⁷ However, civil society organizations generally were not aware that these efforts were underway, and they were not able to provide input and advice prior to finalization of the new guidelines. Rather, the agencies have only invited civil society groups in to hear explanations of the new policies after they were adopted.

Types of engagement

As noted above, most engagement between civil society and oversight focuses on the governance or policy role of those bodies. Where engagement between oversight and civil society does occur, we have identified three categories that can lead to productive results:

1. Practices that facilitate cooperation between civil society and oversight to accomplish a shared goal;
2. Practices that have enabled civil society to “activate” oversight, either by pressing for the creation or expansion of oversight bodies or by advocating for oversight bodies to address particular programs or issues; and
3. Practices which promote better understanding between civil society and oversight so that civil society is better informed and oversight bodies can benefit from the views of civil society—even where classified information or other rules prevent oversight bodies from explaining how civil society’s concerns were taken into account.

These types of engagements are productive in different ways, as explained below.

Cooperating toward a shared goal

One of the most productive models for engagement between civil society and oversight bodies exists when civil society organizations have been able to offer their expertise, insights, time, and resources to assist oversight bodies in pursuit of a shared goal. In this context, a “shared” goal may mean robust oversight that explains how certain surveillance activities are conducted and holds intelligence agencies accountable, or it might involve legislative reform that incorporates new privacy and civil liberties safeguards into the rules governing a surveillance program. It is not necessary (and probably unlikely) that oversight and civil society personnel agree on all particulars when defining the goal of robust oversight or meaningful reform legislation.

One United States oversight representative described such cooperation as civil society representatives serving “as staff to our staff.” Civil society organizations will present research or white papers, talking points, or other expert writings on an issue to help educate oversight body staff. Civil society can also be helpful in working to develop and propose actual solutions to the problems they identify. Oversight representatives noted that it is far preferable for civil society to point out solutions as well as problems. On occasion, civil society’s assistance essentially amounts to the provision of more personnel to overstrapped oversight bodies, but more commonly, civil society simply provide insights from a different perspective, to assist oversight bodies in thinking through issues and asking all the relevant questions.

It is not necessary that oversight and civil society personnel agree on all particulars when defining the goal of robust oversight or meaningful reform legislation.

The expertise and assistance offered by civil society ranges from conducting legal research and presenting legal analyses, to providing technologists to explain how a government surveillance tool actually works, to helping to draft proposed legislative language. Civil society groups also help educate oversight bodies on the practical implications of certain programs or proposals, and assist them in thinking through the potential impact of proposed reform recommendations, and groups with grassroots operations advise on public opinion. As one civil society representative noted, groups can help oversight bodies think through the “evil genie” test to assess potential reform recommendations, by analyzing how an intelligence agency might be able to interpret an authority more broadly than, or otherwise at odds with, the legislature’s intent in enacting the surveillance authority.

Civil society groups also work with oversight bodies to help them create a record on an issue. This includes serving as hearing witnesses, suggesting other individuals who should serve as hearing witnesses, providing questions that the oversight body should ask to witnesses, and submitting statements and reports for the hearing record. In this role, civil society organizations can help ensure that the record will include insights on privacy, civil liberties, and human rights concerns, and can help challenge the testimony of government witnesses and hold them accountable.

On occasion, the provision of assistance in these cooperative relationships can work in the opposite direction. Specifically, when civil society groups have had difficulty obtaining information from intelligence agencies about the operation of their programs, oversight bodies can provide assistance by making the information request themselves. Such requests for information by oversight bodies are frequently effective, even where the oversight body lacks the authority to compel action by the agency.

Activating oversight

In some cases, civil society organizations have been able to activate oversight, either by advocating for the creation of new oversight entities or by pressing for existing entities to examine particular programs or issues.

One example of work toward activating oversight was the civil society campaign in the United States to stand up the Privacy and Civil Liberties Oversight Board (PCLOB), after Congress enacted legislation creating this independent agency in 2007. It took five years, from 2007 through 2012, until the PCLOB actually came into existence, but during that time, a coalition of civil society organizations continued to advocate for this agency, headed by a five-member bipartisan Board, to become a reality. Civil society groups wrote numerous coalition letters, published opinion pieces, and issued papers pressing President Bush, and later President Obama, to nominate a full bipartisan slate of five members to serve on the Board, and then pressing the Senate to confirm the slate. Ultimately, in August 2012, the first four Board Members were confirmed, giving the PCLOB a quorum and allowing it to begin operations as an independent agency.

Civil society groups have also been able to activate oversight on particular topics. For example, in the United Kingdom, one of the central concerns was whether U.K. intelligence agencies were able to receive bulk raw signals intelligence (SIGINT) from foreign partners—most prominently the NSA—without necessarily requiring U.K. warranty to be in place and without the normal safeguards and oversight. The U.K. government’s position was that the routine sharing of raw, unanalyzed intercepted material was governed by “detailed internal guidance … and by a culture of compliance.”⁸ But the detail behind the sharing taking place between agencies was not visible in either statute, code of practice, public statement or policy. Nor was there was any reference in any public reports to raw intelligence sharing being reviewed by oversight bodies.

A number of U.K. civil society groups undertook advocacy trying to highlight this concern. Ultimately, the Interception of Communications Commissioner's Office (IOCCO), decided to undertake a full review of intelligence sharing between the U.K. and foreign partners. In their 2016 annual report, they explain their decision to commission an in-depth investigation, and their intention to explore the issue with foreign oversight bodies to assess whether they could jointly undertake inspections to ensure oversight was not being prevented simply because the matter had jurisdictional implications. The issue remains under review by U.K. oversight bodies with their report due in 2018.

Promoting better understanding between civil society and oversight

There are a variety of practices that have been successful in bringing together civil society and oversight representatives to promote better understanding. As noted above, this in turn can both help civil society groups to be better informed in their advocacy, and help oversight bodies to understand civil society concerns and take them into account.

A critical practice for promoting understanding between civil society and oversight is to convene public forums at which representatives of both participate. When oversight representatives go out in public and explain their role, and how they oversee the agencies they are responsible for, this helps ensure that oversight is not just done, but that it is seen to be done. This can be a panel at a public hearing of an oversight body or it can be a public education panel sponsored by civil society that includes speakers representing oversight bodies. Public forums enable oversight bodies to hear a wide range of perspectives on issues, and can test the arguments presented by all speakers. Such public forums also serve the important role of educating the public at large.

Another practice that has worked well for several oversight entities is to hold meetings using the “Chatham House rule.” This means that government and civil society representatives are free to disclose the fact of the meeting and the topics discussed, but they may not attribute particular remarks or positions to any participant. This practice has led to meetings at which attendees feel free to have candid conversations and where participants can get answers to questions and engage in vigorous debate. It is also important that such meetings are held regularly, to promote trust among participants and to ensure that the dialogue continues. In most countries included in this report, many meetings between civil society organizations and oversight offices have been conducted under Chatham House rule procedures. Some civil society representatives have complained that the downside to holding Chatham House rule discussions is that they are then unable to hold the government representatives accountable for promises made during the conversations. However, to the extent that these are promises to provide answers or further information on particular topics, the participants have generally been able to follow up productively by email and without violating meeting rules.

Public forums enable oversight bodies to hear a wide range of perspectives on issues, and can test the arguments presented by all speakers.

In the United States, meetings under Chatham House rule procedures have been the principal form of engagement between civil society organizations and the internal privacy and civil liberties offices at the intelligence agencies. Civil society representatives have noted that it is difficult to measure the impact of these engagements because the agency privacy officers are not permitted to disclose what policy changes or other decisions may have been made based on civil society input. However, government representatives noted in interviews that civil society input has influenced policy decisions, that advocacy groups have educated them on potential policy impacts and other considerations that they had not previously understood, and that “bringing back” civil society input has empowered them as privacy advocates within the agencies. Government representatives also noted that civil society organizations can present questions they did not even contemplate, and help them consider and address a variety of privacy issues. Meetings between civil society groups and officials within intelligence agencies, other than in the United States, remain a very rare occurrence, if it has happened at all.

Another positive Chatham House rule effort is the newly formed European Intelligence Oversight Network, which offers European intelligence oversight officials and civil society experts space for regular and structured exchange. It was created by the German think tank Stiftung Neue Verantwortung (SNV), with a goal to build on work already done by the European Network of National Intelligence Reviewers. The project connects European intelligence review bodies to share good practice, explore opportunities for joint work, and try and jointly solve shared challenges oversight bodies have been facing. By acting as a secretariat and hub, SNV hopes to promote discussion and dialogue and help oversight bodies innovate in their work.

In general, civil society representatives have agreed that such meetings are worthwhile even though they lack metrics to assess their success. In addition, with more transparency and better understanding between oversight bodies and civil society organizations, everyone can ensure that the public conversation focuses on real issues and parties can debate using a shared set of facts.

In addition, both oversight and civil society representatives from various countries have stated that at least on some occasions, it is productive is to seek smaller meetings—either one-on-one sessions or meetings limited to a small group of participants. Interviewees have stated that this allows in-depth exploration of a topic, permits greater candor in the conversation, and allows both sides to be sure they understand one another. Ultimately, a good mix of public and private, and large and small events is needed to ensure there are an appropriate number of forums to benefit from all types of engagement. It was highlighted by a number of interviewees how important it was for oversight to speak to a variety of different actors from within civil society when consulting on issues. In part, this is to ensure that everyone who has a stake in the issue has their voice heard.

More interesting was the feedback that if civil society groups know a number of them are being consulted, they feel freer to provide their own take on an issue and more nuanced comments than they might otherwise share. This was attributed to the sense that they have to “speak for the community” if they are the only ones being consulted, rather than “speaking for themselves.” In addition, some civil society interviewees noted that there are groups with different and important perspectives who are not typically included in dialogues with oversight. For example, in many countries, the perspectives of groups that focus on immigrant, Arab, and Muslim communities are often not represented in conversations with oversight regarding surveillance issues, despite the concerns in these communities about the impact of surveillance. As such, by speaking to a wider variety of actors individually, and letting them know that engagement is taking place with a large number of actors, it is more likely that a richer, more nuanced picture will be obtained.

As discussed in more detail below, a significant barrier to engagement between oversight and civil society is the extent of classified information. This barrier works in both directions—it prevents oversight representatives from being able to fully explain the operation of government surveillance programs, and it prevents civil society organizations from being able to provide fully informed input and advice. Although oversight and civil society representatives all recognize that much information regarding surveillance programs is legitimately classified and must remain so, they also agree that information about surveillance programs is subject to overclassification and this is a problem that should be addressed. In addition, some information about surveillance activities that has been legitimately marked as classified should be declassified in the public interest.⁹ It is important for oversight bodies to push for transparency where possible, and a fair amount of civil society engagement with oversight bodies has focused directly on promoting greater transparency.

There have been two principal methods by which oversight bodies have worked to increase transparency. The first is by releasing unclassified information. In some cases oversight entities release unclassified versions of their reports, pushing for “public interest” declassification of further information about the operation of surveillance programs to better educate the public.¹⁰ In addition, in the aftermath of the Snowden leaks, the U.S. Office of the Director of National Intelligence (ODNI) established a new website on which to publish declassified Intelligence Community (IC) documents, called IC on the Record.¹¹ Although civil society representatives have criticized the site as difficult to search and not particularly user friendly, most have also commended ODNI for publishing vast numbers of documents, such as decisions of the FISA Court, compliance reports, and internal IC agency policies, many of which had not previously been declassified.

A significant barrier to engagement between oversight and civil society is the extent of classified information.

In other countries, different approaches have been applied to the challenge of declassification in annual reports. The now defunct U.K. oversight body, IOCCO, decided that they would change their earlier practice under which they first drafted a classified annual report for the prime minister, followed by a version of the annual report that went through the appropriate declassification measures. Instead they decided they would write the annual report for the prime minister, excluding anything classified so that the report could be declassified immediately. Additionally, the staff at IOCCO kept an informal “wish list” of transparency requests raised by civil society organizations and other interested stakeholders. Where possible and where appropriate, more information relating to items on this wish list were included in the oversight body’s annual report. In many circumstances items on the wish list related to matters that did not need to be secret, but had never been included in public reports because it had not occurred to staff that there was confusion relating to them, or that they were of any interest or import.

In Germany, the parliamentary inquiry committee that was set up by the Bundestag in the wake of the Snowden disclosures went further than any oversight body had before in interrogating detailed technical matters relating to the intelligence agencies; and in providing information to the public. Their multi-year investigation into the Snowden disclosures was marked by a number of high profile events, including the refusal to hear evidence from Edward Snowden, allegations that the United States was spying on four members of the committee, and the leaking of internal documents from the committee’s work to Wikileaks.¹² Their report was over 2,000 pages in length, and is the most detailed of any parliamentary inquiry into signals intelligence¹³ and includes exhaustive descriptions of practices such as undersea fibre optic cable interception and selection.

The second method by which oversight bodies have promoted transparency is by creating requirements for future disclosures of information by intelligence agencies or permitting greater transparency by private actors. For example, in the United States, the PCLOB recommended increasing such transparency requirements and permissions in both its Report on Section 215 and its Report on Section 702, and a variety of provisions promoting transparency were enacted into law with the USA Freedom Act. Title VI of the act requires regular periodic transparency reporting by intelligence agencies to provide certain statistics regarding the use of various FISA authorities and requires that with certain exceptions, the Director of National Intelligence shall release unclassified versions of all significant decisions by the FISA Court. In addition, Title VI permits the recipients of FISA orders, such as technology companies, to disclose certain statistics showing the extent of production demands they receive from government agencies and the number of their customers that are affected by these demands. The FISA Amendments Reauthorization Act, enacted in January 2018, further expanded the government’s reporting requirements.

The CTIVD regards the Knowledge Network as “highly valuable” and “contributing to the effectiveness of their oversight.”

The creation of the “Knowledge Network,” comprised of external experts by the Dutch Review Committee on the Intelligence and Security (CTIVD) in 2014, is also a positive transparency initiative that could be replicated in other countries. The panel of external experts had a goal to explore current and further themes prevalent in the academic world and insights they might provide to assist CTIVD’s work. Additionally they act as a sounding board allowing CTIVD to gain insight to critical and different views on a range of issues, including specific investigations. The CTIVD regards the Knowledge Network as “highly valuable” and “contributing to the effectiveness of their oversight.”¹⁴ The network additionally ensures strong links between academic communities and oversight, helping oversight promulgate information about their work to other communities. Such an approach was adopted by the New Zealand Inspector General of Intelligence and Security who set up a “Reference Group” of external experts,¹⁵ crediting the Dutch Knowledge Network as a model.¹⁶

Civil society engagement with oversight bodies relates to both methods of promoting transparency. First, civil society organizations have worked with oversight bodies to advocate for public reporting and to help develop rules that will require agencies to issue reports and permit companies to do so. Second, when oversight bodies issue public reports explaining the operation of surveillance programs, or intelligence agencies and companies release statistical reporting showing the extent of use of these authorities, such public reporting helps to better inform civil society organizations and the public at large. This in turn promotes further engagement between civil society and oversight based on a common understanding of how programs operate.

In some circumstances, however, oversight bodies felt that they had no role to play at all in promoting transparency. In these circumstances the view was that it is the intelligence agencies themselves who should have that role, as it is their practices in the frame. As such, these oversight bodies saw themselves as custodians of the intelligence agencies’ data. Even in circumstances where they, as an oversight body, might generate statistics related to warrantry or other intelligence agency practices, this view remained, which necessarily limited the transparency that could be provided by their office. In some circumstances, there were strict limits in legislation which ended any debate about where the line should be drawn, but in most circumstances there was a significant degree of latitude given to the oversight body to decide what should or should not be public in their reports. While oversight bodies in all countries must submit their reports back to intelligence agencies for comments, and have the report go through a declassification procedure, it seems a question of culture as to how transparent those reports end up being and the extent to which oversight bodies feel it is part of their function to push for greater transparency.

In some of the countries mapped as part of this project there were very few examples of engagement (productive or otherwise) between oversight bodies and civil society groups. In these countries, a variety of barriers have prevented engagement. In other countries, such as the United States and the U.K., there has been significant engagement, but barriers to engagement still remain. The discussion below focuses first on barriers that have prevented engagement, and then on barriers that have challenged, but not prevented, engagement.

Barriers that prevent meetings

In some countries there is no engagement that either civil society groups or oversight bodies could point to at all, despite the fact that during interviews, both parties described engagement as positive and a worthwhile activity to undertake. When probed as to why no engagement had taken place given it appeared both civil society groups and oversight felt it would be beneficial, the response was mostly that they assumed the other party would not be interested, resulting in no-one initiating first meetings.

Following interviews, a number of interviewees decided to try and remedy the situation, with positive results. Afterward, one oversight interviewee shared advice: “Don’t be reluctant to make contact. Don’t assume they will have radically different views. You might be pleasantly surprised.” Similarly, one civil society representative who reached out to their oversight body for a meeting after being interviewed for this project reported back that, “It went well overall. The start was tense but it got more relaxed over time…But it was totally worth it: it decreases suspicion and increases the likelihood that we can have constructive exchanges in the future.”

Beyond a lack of initiative, many oversight bodies lack resources to dedicate to engagement with civil society groups. Regrettably, in some countries, oversight bodies also lack any appetite for external engagement and staff do not see engagement or promoting transparency as a necessary part of the role. Over the course of several interviews, civil society groups highlighted a number of different oversight bodies who had never reached out, and in other circumstances had refused engagement when requested.

In some cases this is due to having a small staff and a large number of challenging and time consuming inspections to undertake. The Canadian CSE Commissioner has gone on record¹⁷ raising concerns about the constriction of funds available to his office at a time when the agency he is responsible for overseeing is having its budget increased. Similarly, the New Zealand Inspector General of Intelligence and Security has just eight staff members¹⁸ overseeing the intelligence community, although pro rata, this number may compare favorably to other jurisdictions.¹⁹ Moreover, only the United States has staff working for oversight bodies who consider engagement with civil society as part of their jobs, although in the United Kingdom, the Investigatory Powers Commissioner's Office (IPCO) is in the course of establishing an engagement team.

“Don’t be reluctant to make contact. Don’t assume they will have radically different views. You might be pleasantly surprised.”

Some oversight interviewees also referenced pressures to reduce expenditure and costs in line with cuts to spending across the public sector as a reason for not engaging. However there are cases where oversight bodies have also refused additional resourcing, whether it be legal or technical, which would free up time for engagement. In the United Kingdom, then-Commissioner Sir Mark Waller of the now defunct Intelligence Services Commissioners Office “remained wholly resistant to acquiring any inspector resources save for secretarial support” and performed the role alone despite recommendations from other oversight bodies to secure such resources.²⁰

Similar resource constraints affect civil society groups. In some countries, the few existing civil society organizations have very few staff members. For example, the Electronic Frontiers Australia is run by a unpaid board and volunteer team.²¹ In some situations, the engaged civil society bodies are made up entirely of volunteers who are contributing expertise and their time.

Civil society groups in some countries have also been thwarted in setting up meetings with oversight bodies because there are simply no publicly available means to contact the bodies. Some oversight bodies have not yet developed their own websites. The French National Commission for the Control of Intelligence Techniques only has a listing in the government administrative directory²² providing a postal address as the means to contact them. Others such as the U.K. Intelligence and Security Committee only provide a postal address on their website, or a Google-powered web form.²³

The lack of easily accessible options to contact some oversight bodies creates a real barrier to engagement. Civil society interviewees explained that as well as occasionally being a logistical barrier (i.e. having to write a letter rather than being able to phone to ask a quick or urgent query) it leaves an impression that the oversight bodies may not wish to be contacted. A number of oversight interviewees said that their primary form of engagement with civil society stakeholders was by monitoring and engaging with content on social media, a tactic that is not generally viewed as significant engagement by civil society.

Another barrier that can prevent the initiation of meetings in some countries is geography. While not an issue in the United States, since most civil society organizations that focus on intelligence surveillance have a presence in Washington, D.C., where the oversight bodies are located, or in physically smaller countries like the United Kingdom where oversight bodies and the majority of civil society groups are based in London, geography can be a real barrier in other countries. In Canada for example, while oversight bodies are headquartered in Ottawa, the civil society and academic groups that are active on these issues are distributed throughout the country in cities like Vancouver, Toronto, and Montreal. With a train between even the closest of those cities taking more than five hours, the sort of organic connections that can be made bumping into each other at panel discussions or evening events does not happen in Canada. A similar issue was raised in Australia with oversight bodies working out of Canberra and civil society groups and academics often based in Sydney and Melbourne.

The end result of being geographically further away from each other is that meetings between groups happen far less frequently, and in many cases, simply not at all. While one would hope that technology might be able to bridge the difficulty this geographical distance creates, due to oversight bodies often being physically housed in classified environments, there are rarely video conferencing facilities or similar tools available.

Finally, in some cases civil society groups expressed the view that they are unlikely to engage with any oversight body where the body is regarded as toothless or captured. These examples are mostly tied to situations where legislation limits what oversight can meaningfully achieve, or where the senior leadership of an oversight body is viewed by civil society as a public relations arm of intelligence agencies. In countries where this issue exists, when coupled with barriers such as lack of resources and a lack of trust, it almost always results in no, or limited, engagements between oversight and civil society groups.

The result of being geographically further away from each other is that meetings between groups happen far less frequently, or simply not at all.

A common concern expressed in interviews with civil society groups in countries other than in the United States was the fear that their engagement would somehow backfire or make them complicit. This was expressed in a number of different ways, with slightly different scenarios in mind. One scenario involves the concern that comments made in private meetings might not be treated confidentially and would be repeated back to them in a public setting such as a panel discussion or similar event. In another scenario, there was the concern that a meeting they thought they were attending on an informal basis ended up being parlayed by the oversight body into some kind of formal consultation, and referred to in public reports as evidence of their outreach. An example proffered by some civil society groups in the U.K. related to recent meetings with the Law Commission which were then cited as evidence for wide ranging consultation undertaken as part of a reform of the Official Secrets Act, despite the meeting never being represented to civil society groups as forming part of any official formal consultation at the time.²⁴

These concerns about whether to engage with oversight can change over time, either when new legislation remedies structural defects in previous incarnations of oversight bodies, or when there is a change of senior leadership in oversight bodies. In the U.K., a number of civil society groups described the positive change that had occurred when there was a change of leadership at IOCCO when Joanna Cavan became head of the office and began a proactive outreach effort. Additionally in the U.K., many interviewees expressed hope that the previously fragmented and disjointed oversight landscape in the country has been appropriately fixed with the passing of the Investigatory Powers Act 2016 and the creation of the Investigatory Powers Commissioners Office.

Conversely, many oversight bodies outside the United States felt constrained not to engage too extensively with civil society. Some who had undertaken more engagement with civil society groups shared the view that it was difficult for them to maintain a good balance between different stakeholders, and that they felt they needed to remain independent not just from the intelligence agencies they oversee, but also from any other actor seeking to influence their practice. Oversight must not be seen to be “in anyone's pocket,” said one oversight body representative. Another described the practice as “walking a tightrope,” managing a variety of actors’ expectations and balancing their input and views. As a result, these officials felt that there is a limit to the amount of engagement and cooperation that could and should ever be undertaken between oversight and civil society bodies. No oversight body interviewed felt that they had gone too far in their engagement with civil society groups, but a couple did mention occasions where they had supported statements by civil society groups, particularly during legislative change, and received push back from intelligence agencies for doing so.

Classified and sensitive information

When engagement does occur, the principal barrier to productive engagement has been the fact that significant amounts of information about surveillance programs are classified. As noted above, this obstacle limits what oversight staff can say and explain, and inhibits civil society from fully understanding the operation of surveillance programs. As a result, it also prevents civil society representatives from being able to provide as accurate, detailed, and effective input as might otherwise be possible. As one civil society representative put it, reading redacted documents is like watching a movie and only seeing every tenth minute, and then trying to piece together the whole movie.

When asked for their views on how to mitigate the barrier of classified information, many oversight and civil society representatives noted the need to rein in the problem of overclassification of information. All recognize that certain information about surveillance activities must be classified—such as the facts describing particular surveillance operations or details that would reveal sources and methods—but the rules governing surveillance programs should be unclassified. The IC in the United States has made some progress in tackling this problem by declassifying information in the aftermath of the Snowden leaks. As described above, the U.S. ODNI has declassified and posted vast numbers of documents regarding surveillance programs. Nonetheless, overclassification remains as a significant problem.

Beyond tackling the problem of overclassification, there are several strategies that can help mitigate this barrier. First, some oversight and civil society representatives have noted that it can be helpful for civil society groups to send written questions in advance of a meeting. This permits the government representatives to research what information is already declassified and to prepare answers that are as comprehensive as possible based on available unclassified information. By contrast, when government representatives receive questions on the spot and they do not have clear recollections of the extent of information that remains classified, they will err on the side of caution, and may not provide the most comprehensive responses possible. Another strategy recommended by an oversight representative is for civil society organizations to develop hypothetical scenarios that government officials could address. Although intelligence agencies would not be able to either confirm or deny that they engage in certain activities, at least for some hypotheticals, the agencies could explain what tactics they would be permitted to employ and what the rules would require.

In addition to challenges posed by the barrier of classified information, engagement can also be hampered by oversight bodies’ tendency to consider information about their reviews to be sensitive, and not subject to disclosure.

Additional challenges to meaningful engagements

Beyond the obstacles posed by classified information, there are several additional barriers that challenge the ability of civil society and oversight to have meaningful engagements, even where meetings are possible. One such challenge has been the lack of publicly available professional biographic information regarding oversight staff. Outside of publicly appointed officials like inspectors general, commissioners, or committee members, the core staff working for oversight bodies typically have no public profile. The only examples of public profiles for staff from the countries examined were a small number in the United States. In many cases this appears to be due to the fact that the majority of staff working for oversight bodies are drawn from the intelligence community or other sensitive areas and no professional biographic information can easily be provided. This is partly to protect the sensitivity of any previous work they have undertaken, but also because in some circumstances staff members may seek to return to work in the intelligence community and do not wish to have public profiles tied to them, or because the ingrained intelligence culture leads them to be reluctant to post personal information on public websites.²⁵

Outside of publicly appointed officials like inspectors general, commissioners, or committee members, the core staff working for oversight bodies typically have no public profile.

The difficulty expressed by civil society groups is that when meetings do occur they do not know the educational background, the previous institutional background, or subject matter expertise of the oversight official. As a result, civil society groups may not be able to correctly pitch the discussion at the right level. For example, when attempting to have a productive conversation about a complex technical topic like computer network exploitation, it would be helpful to know whether the person you are speaking to is a seconded lawyer specializing in international law, a non-technical former counterterrorism police officer, or an engineer expert in telecoms networks and cybersecurity.

One option suggested by an interviewee could be to provide high level statistical information on the background of oversight staff to ensure there is some information in the public domain without disclosing potentially sensitive information about previous work. One model for this could be drawn from the U.K. Independent Police Complaints Commission who provide statistics on the number of staff with police or civilian backgrounds, among a range of other criteria.²⁶

A related but even greater barrier to productive engagement between civil society and oversight occurs where there is a lack of trust. Civil society and oversight staff typically have different professional backgrounds, and it takes time and effort to develop productive trust relationships. In some instances, this problem is exacerbated by frequent staff turnover, particularly at oversight bodies.

To overcome this lack of trust, communication is key. When discussing sensitive issues, to avoid misunderstanding, it is of critical importance that precise language is used which is understood by all parties. Many interviewees from both civil society and oversight bodies highlighted the lack of a shared lexicon between groups as a barrier to productive discussion and dialogue. The difficulties raised range from an actual inability to understand each other to a risk of causing offense or otherwise shutting down dialogue.

The most serious barriers to engagement arise when the difference in language used by civil society groups and oversight bodies reflects the lack of a common understanding about the use of certain phrases and terminology. For example, in the U.K., what civil society groups have long called “hacking,” is now codified in legislation as “Equipment Interference.” The intelligence agencies refer to such activities variously as “Computer Network Exploitation,” “Computer Network Operations,” or “Computer Network Attacks.” Previous oversight reports by different oversight bodies have referred to such activity as “IT Attacks” as well as “remote intrusion,” amongst other terms. Various government and oversight documents make different use of these terms. Some of these different phrases are synonyms, but there are often subtle differences between them that not all parties are aware of.

Some differences of language to describe the same act are intentional, however, such as whether to refer to the large scale interception, analysis and storage of communications as “bulk interception” or “mass surveillance.” In such cases, it can be a barrier when one side or the other takes offense at the chosen terminology, or responds by shutting down the dialogue. Some civil society representatives have complained that some government terms seemed designed to hide the intrusive nature of certain government practices. For example, the term “incidental collection,” suggests that the collection is minimal in scope and unexpected, whereas incidental collection of information about non-targets can be extensive and fully anticipated. Conversely, some oversight officials have noted that while they understand civil society’s need to use certain terms to communicate with the general public, they find it off-putting and counterproductive if, during closed-door meetings with government, civil society representatives use terminology that they consider inflammatory.

Cooperative relationships can also be thwarted in many countries because oversight bodies and civil society groups fail to share information about their priorities and planning. While neither party has a duty to share what their current priorities are, or to consider the priorities of another party in the creation of strategic plans, both oversight and civil society could benefit from knowing how other stakeholders working on similar issues are prioritizing their work. Indeed when priorities are not aligned, a barrier is created limiting engagement, as each party's focus is elsewhere and resource constraints may prevent each party from inputting or collaborating with the other.

Given that resource constraint is likely to be an ongoing issue for both oversight and civil society, one solution is for there to be greater contact between parties on future plans and priorities, and for them to explore where priorities could be aligned to ensure both parties are achieving the fullest impact of their work. At a minimum, it can be helpful if oversight bodies publicize their agendas on their websites, to enable civil society groups to plan accordingly. For example, should an oversight body decide to undertake a thematic review on a particular topic such as the use of social media monitoring, announcing that decision in advance of the start date of the project might impact where civil society groups decide to place energies and priorities for the coming year, allowing them to plan appropriately and dedicate resources to developing their views and submitting them for consideration to the oversight body.

It can be helpful if oversight bodies publicize their agendas on their websites to enable civil society groups to plan accordingly.

Perhaps one model to follow is the proactive publishing of work plans, as the New Zealand Inspector-General of Intelligence and Security does, to provide insight into priorities, where resources are being allocated, and timeframes.²⁷ The Dutch CTIVD undertake a similar action, publishing in their annual report any planned thematic investigations taking place for the coming year.²⁸ The U.S. PCLOB originally followed this same model, announcing its reviews of the Section 215 and Section 702 programs, and then announcing its short term agenda in July 2014.²⁹ However, since that time, the PCLOB has not publicized the topics of its ongoing oversight reviews.

Finally, in countries that have enjoyed at least some contact between civil society groups and oversight, one of the issues raised by civil society groups is the lack of any visible impact from their engagement. Despite having somewhat regular meetings, civil society groups are often not able to discern whether oversight subsequently acted on the specific concerns that civil society groups have raised. One oversight official explained “it’s very frustrating, as important issues are sometimes brought to your attention by civil society groups, or even members of parliament, but the public response we are forced to provide often seems like we are ignoring or dismissing the issue.”

One oversight staffer who had a background working at an Ombudsman office reflected that practice responding to concerns raised by civil society groups could be handled similarly to how Ombudsman offices dealt with complaints submitted by members of the public. For example, concerns or complaints should be welcomed, concerns or complaints should be treated seriously, civil society groups should be provided information about what was being done with the concern raised, there should be a resolution to the concern raised, and any actions taken should be reported back to the civil society group.

There were very few examples that could be identified from across the interviews where engagement between oversight and civil society has been counterproductive. Probably the main context in which there have been negative results from engagement or where something has gone wrong is where emerging trust has been breached. In the United States, oversight and civil society representatives provided examples of situations in which they thought a trust relationship had been established, but subsequent actions by their counterparts breached that trust, resulting in a damaged relationship going forward. Two examples—one involving a breach by oversight officials and the other a breach by civil society—illustrate the problem.

The engagements between U.S. civil society organizations and internal agency privacy and civil liberties offices in 2016 to 2017 provide an example of a breach of trust by government officials. In multiple meetings over time, participants discussed establishing some metrics to count the number of U.S. persons whose communications have been “incidentally” collected under the surveillance program conducted under Section 702 of FISA.³⁰ U.S. persons and people inside the United States may have their communications “incidentally” collected when they are in communication with Section 702 targets, who must be non-U.S. persons located abroad. The government has stated that it cannot determine the number of communications of U.S. persons and those of people inside the United States that are being collected, because it is difficult to determine from a communication the nationality of non-target participants. However, in response to pressure from the public and oversight, the NSA agreed to work to develop and implement some metrics that would provide insight into the extent to which the NSA collects and uses communications of U.S. persons and people inside the United States.³¹ The privacy and civil liberties officers for NSA and ODNI coordinated civil society engagement on this issue.

Government and civil society representatives have stated that they felt at the time that these meetings over the course of 2016 were productive, and everyone was moving forward in good faith. However, in the spring of 2017, new Director of National Intelligence Daniel Coats testified in Congress that the NSA would be unable to develop an accurate and meaningful methodology for measuring the collection of U.S. person communications under Section 702. The civil society representatives who had been involved in the earlier engagements felt betrayed by what they saw as the new DNI reneging on an earlier commitment.

Similarly, oversight representatives have sometimes felt that civil society organizations have breached their trust. In particular, some oversight staff pointed to situations in which a civil society group had stated that the organization would remain neutral on a particular legislative proposal related to surveillance, but the organization subsequently made public statements condemning the bill. The oversight representatives explained that they understood that sometimes civil society organizations will conclude that they need to oppose a particular bill, even if it includes certain reforms, because the bill does not go far enough. The concern they cited regarded trust and communication—because the civil society representatives said one thing to them and something else publicly, rather than warning the oversight representatives that they would be publicly condemning the bill.

Beyond such breaches of trust, the second principal context in which engagement has not been viewed as productive is where there is a misalignment of goals. Many oversight representatives stated that they felt that it could be counterproductive when civil society organizations fail to understand “the art of the possible,” or miss opportunities to work for meaningful reform because they are unrealistic in their goals. Their advice was essentially that the perfect can be the enemy of the good, and it can be counterproductive to press for the perfect if the result is to defeat a proposal that represents the good. Civil society representatives differed in their views on whether a refusal to compromise can backfire. Some felt that if oversight results in changes that are merely “fig leaves,” this is worse than no reform at all because it can undercut the argument that vigorous oversight and reform are still needed.

There was one situation in the U.K. where engagement resulted in more hostile relations than existed before. In the United Kingdom, the Intelligence and Security Committee misrepresented evidence given by NGOs to attack them and suggest they said that that terror attacks “are a price worth paying.”³² This led to stories in major national newspapers attacking civil society groups claiming that NGOs advocate and condone terrorism as “the price of freedom” or as a “price worth paying.” The U.K. NGO Liberty was the focus point for most of the media attention, with pictures of individual staff members who had given evidence appearing in the press alongside their misrepresented evidence. The context of the evidence and the responses were fiercely contested with Liberty stating this was “‘an attempt to undermine, discredit and damage our organisation’s reputation.”

The landscape of oversight bodies in different countries is not uniform. Different countries have had very divergent experiences, and the maturity of the oversight bodies that exist and their relationships with external actors are at very dissimilar stages. Thus, the regular engagement between civil society and oversight in countries like the United States, the United Kingdom, and the Netherlands contrasts sharply with the experience in countries like Canada, New Zealand, Australia, Germany, and Norway, where there is considerably less, and in some cases no external engagement at all. Nonetheless, there are a variety of strategies and approaches that could be productively be adopted in most, if not all, of the countries examined.

Increase technological expertise of oversight bodies

Technology is permeating every aspect of how intelligence and security agencies go about their work. Many agencies are working with data at a scale unimaginable a decade ago, and processing it using novel techniques that are more complex than ever before. Oversight bodies are charged with ensuring the deployment and use of these technologies is lawful and not a disproportionate intrusion. However, few oversight bodies have dedicated full time technical staff assisting them in their work. In many oversight bodies, there are more lawyers on staff than technologists. Almost all oversight bodies interviewed expressed concern about the pace of technological change and the challenge they faced in keeping up. The Norwegian EOS Committee has commented in its 2016 Annual Report³³ that it must “adapt its oversight activities to the technological development” of the intelligence agencies it oversees and cautions that there “is reason to believe that we have only seen the beginning of this [technological] development.”

Likewise the Dutch CTIVD acknowledged in its 2016 Annual Report that the criticism they have faced regarding lack of technical know how is a “justifiable” critique. In 2016 they set up an “IT expert unit that will, inter alia, delve into the operations of the services’ increasingly automatised data processing systems, from the initial collection of data, through its analysis, selection and use, to its destruction.”³⁴ Similarly, in the United States, the PCLOB Board and staff have been comprised mostly of lawyers, but in 2016, the agency brought on a Technology Scholar to assist in analyzing the technical operation of the counterterrorism activities being examined, and one of the new members recently confirmed to serve on the Board is a technologist.³⁵ Additionally, in the United Kingdom IPCO has just constituted a Technical Advisory Panel of external technical experts from academia and industry. Many oversight bodies made reference in interviews to technical assistance that previously has been, or could be provided by civil society groups in this respect.

In many oversight bodies, there are more lawyers on staff than technologists.

It is critical that oversight bodies not rely solely on technologists within the agencies they oversee in assessing surveillance activities. Particularly where oversight bodies have not hired many technologists on staff, civil society organizations can play an important role in assisting oversight bodies to understand the capabilities of, and privacy risks posed by, new technologies. In recent years, some technical information about government surveillance activities has been declassified, which has enabled civil society technologists who lack clearances to analyze the operation and intrusiveness of programs in ways that were not previously possible. There are also models for bringing technologists in to work with oversight bodies, such as the TechCongress program in the United States.³⁶ This program funds technologists to work for a year for a Member of Congress, to assist on technology-related issues.

Further, many technologies in use by signals intelligence agencies have some parallel application in the private sector. This means that there is often publicly available information that allows civil society groups to have a robust understanding of the applicable principles or concepts. This provides a far greater shared information pool between civil society groups and oversight bodies than might otherwise be the case. As one civil society representative noted, there are a variety of technical questions that can be addressed “in the abstract” without the need for civil society to have access to classified details about programs’ operations. For example, civil society technical experts can help educate oversight staff on such topics as the types of metadata created by different forms of electronic communications and what information can be gleaned from each type. Oversight staff might also be able to solicit civil society technical expertise by presenting potential recommendations at a high level or asking sufficiently generalized questions to avoid the barriers of classified information.

Such unclassified consultations should be possible even for capabilities as sensitive as computer network exploitation, which countries are only now beginning to avow the use of publicly. Plainly the specifics of how agencies use such techniques, the vulnerabilities they exploit, or the targets they infect cannot be discussed, but neither do they need to be for civil society and oversight to discuss a number of issues relating to the use of the technique. Both civil society groups and oversight bodies in some countries highlighted common issues that could be examined regarding computer network exploitation including: developing post-hoc auditing standards so oversight can ensure that exploit tools used by agencies do not deploy further, or collect more information than is permitted by warrants; and exchanging views on the broader cybersecurity impact caused when errors occur with the use of computer network exploitation techniques.

Promote international cooperation

Historically, oversight bodies have been focused solely on the actions taken by the intelligence agencies they oversee, and on national issues, with less attention on what the intelligence services do in cooperation with international partners. While the intelligence agencies themselves engage in cross-border sharing, particularly within the Five Eyes alliance, there has not been a similar level of cooperation among oversight bodies nor a focus by oversight on the international sharing arrangements.³⁷ Sometimes this is due to statutory limitations where the oversight bodies are not permitted to look at what intelligence agencies do in other countries. Other times it results from a lack of ability to compel the material from other agencies that would be necessary to review the practices. On yet other occasions, this results from the “third party rule” being applied that prevents the intelligence agency from being able to share material it received from foreign partners with its own oversight body, unless it receives permission from that foreign partner. Further, there is no supra, trans, or inter-national intelligence oversight body. In the course of interviews, many oversight bodies said they monitor what is happening in other countries, but that this was mostly via news articles and social media.

Coordinated action between oversight bodies in different countries could assist in plugging that accountability gap. While the issue has long been raised by civil society groups, and has been the subject of much academic writing, it is only recently that a formal grouping of oversight bodies from the Five Eyes countries has formed, named the Five Eyes Intelligence Oversight and Review Council. Another grouping of the Dutch, Belgian, Danish, Norwegian, and Swiss oversight bodies has published a joint statement discussing the risk of an oversight gap and ways to tackle this risk, when overseeing international data exchange by intelligence and security services. The grouping was created in 2016, and is conducting an investigation into the exchange of data on “(alleged) jihadists,” each from its own national context and within the framework of its own mandate, with the goal of comparing investigation methods, interpreting legal problems, and collating non-classified findings. ³⁸

Some countries have also undertaken thematic reviews looking into how agencies share and cooperate with foreign partners. The Dutch CTIVD have undertaken reviews on “the implementation of cooperation criteria” which assessed the Dutch intelligence and security agencies and their cooperation with foreign services. A further review has taken place on “the exchange of unevaluated data with foreign services.” They concluded by recommending that “written policy is drawn up, that each instance of data provision is recorded and that the ministerial authorisation for the provision of data to a foreign service is granted for a specific period, for instance one year.”³⁹ In their response to the report, the Ministers stated that they will no longer grant authorization for an indeterminate period, but will instead only issue authorizations for a period of no more than one year.

These sort of reviews and recommendations can plug gaps in accountability. These are positive reforms, and if similar reviews could be replicated in other countries, progress might be able to be made on an issue that has so far failed to gain traction in the courts, or in parliaments.

In the past there have also been efforts made to link together various national oversight bodies, whether it be the International Intelligence Agency Review conference series or events such as the International Intelligence Oversight Forum.⁴⁰ These conferences were well regarded by all who attended, but were infrequent or one-offs. Some efforts have been made to create formal networks of intelligence oversight reviewers such as the European Network of National Intelligence Reviewers⁴¹ which was supported by many oversight bodies, but which is now inoperative due to lack of support and funds. Today, although oversight bodies enjoy some relationships with counterpart organizations from other countries, these interactions are limited and reasonably rare.

Coordinated action between oversight bodies in different countries could assist in plugging accountability gaps.

In comparison, civil society groups’ relationships with their counterpart organizations are highly developed and frequent. Many of those civil society groups interviewed work weekly in partnerships with civil society groups from other countries. There is frequent pooling of knowledge and resources, and groups will often collaborate together on activities. Regular meetings are more varied between civil society groups, but some civil society representatives described working with counterpart organizations from other countries at workshops or conferences as frequently as once a month.

Due to this regular cross-border interaction, civil society groups could well have a more international perspective on issues they are working on than do oversight bodies. They are more likely to be exposed to international trends, and potentially have greater insight into issues that have arisen in foreign jurisdictions. There are also networks of civil society groups like the European Digital Rights Initiative (EDRi) or the International Network of Civil Liberties Organisations (INCLO) which are designed to facilitate knowledge sharing and develop joint work products. Of course, some civil society groups’ missions are directly international in scope. Organizations like Human Rights Watch, Amnesty International, Privacy International, and Access Now all have international networks or chapters, and work across many jurisdictions.

The recently launched Five Eyes Intelligence Oversight and Review Council could provide an opportunity for civil society cooperation in a parallel track, and for engagement between the two international collaborative organizations. Discussion between national civil society groups and their national oversight bodies on developments in other countries could be a productive exchange.

The Five Eyes Intelligence Oversight and Review Council was set up in September 2016 “to facilitate the sharing of experiences and best practice in oversight and review.”⁴² It will meet annually in person and quarterly by secure electronic communication. The New Zealand Inspector General describes the purposes of the forum as to include the “exchange of view[s] in subjects of mutual interest and concern, compare best practices in review and oversight methodology, explore areas where cooperation on reviews and the sharing of results can occur and to encourage transparency to the largest extent possible.”⁴³ IPCO has said that discussions at the the Five Eyes Intelligence Oversight and Review Council included “the oversight of intelligence sharing” and “exploring areas where cooperation on reviews and the sharing of results is appropriate.”⁴⁴ The CSE Commissioner has said that these “meetings will become more important, for learning not only about best practices in review and oversight, but also how the Five Eyes review bodies can more effectively examine the relationships among their intelligence agencies to strengthen public trust in their respective countries.”⁴⁵

Examine the efficacy of programs

A critical question that oversight bodies should examine when reviewing intelligence activities is whether the surveillance program is effective. Measuring the efficacy of surveillance programs and other intelligence activities can be extremely challenging. It is usually difficult to attribute any specific number of terrorist plots thwarted, or other specific metric, to any particular intelligence program, because individual surveillance programs are not used in isolation and intelligence agency successes will be based on a variety of sources. However, it is critical for oversight to assess the efficacy of programs because otherwise “policymakers and courts cannot effectively weigh the interests of the government in conducting a program against the intrusions on privacy and civil liberties that it may cause…[and] if a program is not working, [government] resources should be redirected to programs that are more effective.”⁴⁶

Oversight bodies’ assessments of efficacy can play an important role in their assessments of surveillance programs. If a program is not effective, there can be no justification for even minor intrusions on individual privacy. While various oversight bodies do have powers to conduct such reviews, there is little published research on how best to assess whether such programs are indeed effective. For example, the Belgian oversight body has an express mandate to consider the efficacy of intelligence programs, but in practice has not probed this area due to resource constraints and the challenges faced with trying to independently quantify how effective a certain intelligence program is. In the United States, the ODNI submitted a report in which it noted that it is challenging to measure efficacy, but the U.S. Intelligence Community uses a range of both quantitative and qualitative measures, and that more work remains to be done to develop metrics.⁴⁷ Although actual assessments will likely involve reviewing classified data, it may be possible for civil society to assist in developing standards and tools for conducting such reviews. Additional research could be commissioned that would arm overseers with a framework to conduct such assessments, and so when they are undertaken in the future, greater confidence can be had in their findings.

Act as a partner in securing legislative reforms

As discussed above, the interests and priorities of both civil society groups and oversight bodies can often align when it comes to intelligence reform. Such alignment can occur on a number of cross cutting issues. One key example is the report of the former Independent Reviewer of Terrorism Legislation in the U.K., David Anderson QC, which was instrumental in laying the foundations for the subsequent Investigatory Powers Act. His report diverged from civil society asks in several key places (such as his support for bulk collection warrants, which are opposed by civil society groups), but a number of vital long term civil society positions were adopted in his report. These areas included the introduction of judges into the warrant authorization process, creating a more robust oversight system, and finding there was no operational case for bulk data retention of third party web activity. All of these findings were consistent with civil society positions that had not found favor in the Home Office, but once supported by David Anderson, were adopted by government.

Thus, the final oversight report demonstrated the alignment between oversight and civil society, and helped lead to legislative reform that civil society had previously been unable to secure. Moreover, all parties have an interest in ensuring that oversight can be undertaken in the most fulsome way possible, which may include ensuring that oversight itself has a broad mandate or powers to obtain the information it needs to fulfill its function.

The interests and priorities of both civil society groups and oversight bodies can often align when it comes to intelligence reform.

In the United Kingdom, during the passing of the Investigatory Powers Bill, for the first time civil society and oversight bodies met together and shared proposals for oversight reform. This process culminated in a “wish list”⁴⁸ which was jointly supported by the IOCCO and the NGO Coalition Don’t Spy On Us. The list included novel initiatives such as ensuring oversight had “full access to technical systems,” “relaxation on secrecy provisions to aid transparency” and new independent expert resources for oversight including “technical, legal privacy advocates and academics.” Many of these issues had been legislative reform goals of civil society groups for years, but despite campaigning, not much traction had been made. However after the publication of the “wish list,” a number of items were secured as amendments to the act.

Similar situations have occurred in the Netherlands where Dutch NGOs and the CTIVD have met reasonably regularly and shared concerns and proposals about the new Wiv 2.0 intelligence law currently going through parliament. In Canada, where intelligence reform is imminent, but where there are no established relationships between civil society and oversight, there is a great opportunity that should be seized, if each party is able to reach out successfully.

A more detailed study is needed than has been able to be achieved in this report assessing the impact of successful oversight and civil society agreements on legislative reform. But it seems that there is potential for such partnerships between oversight and civil society to be productive.

Undertake future focused discussion

The Dutch oversight body CTIVD began a project⁴⁹ called “Supervision 3.0 project” aimed at thinking through what oversight should or could look like in the future. Its goal was to explore how intelligence agencies might work in the future, how they might use data and data analytics, and what oversight could do now to prepare for that, or a similar future.

This was done by creating various sub-projects, looking at a range of issues such as considering the future of bulk interception and possibilities to delete non-relevant data, as well as the use of automated data analysis by agencies and how oversight might be able to design systems to ensure internal control and external supervision. A number of workshops were run internally. All participants who were interviewed spoke very highly of the project. Civil society participants found it very encouraging to hear such an effort was taking place.

Replicating such an initiative, perhaps with multi-stakeholder input, could help build up common ground and understanding. Such an experience could positively impact the relationship between parties, as it allows them to engage together as part of a team, which benefits them both when discussing topics in other forums.

Foster network-building

Civil society groups and oversight bodies referenced the important role the other plays in developing each other's professional networks. From an oversight perspective, the networks civil society groups have with other community groups or grassroots organizations far exceed their own, and are a rich source of experience that can be tapped. In some circumstances, the civil society group might speak on behalf of particular communities affected by surveillance.

For example, several oversight representatives referenced the vital role a range of civil society groups played in introducing members of oversight bodies to representatives of Muslim and other minority communities. This allowed oversight to gain greater insight into the concerns of communities they might not otherwise have had an opportunity to engage with. In other circumstances civil society groups have played roles in connecting oversight bodies with representatives of journalist trade unions who were concerned about the protection of sources, as well as with membership bodies for the legal community who were concerned about the protection of legal professional privilege between clients and their representatives. From a civil society perspective, oversight bodies are ideally placed to act as interlocutors connecting government, intelligence agency and law enforcement staffers with civil society groups.

In some circumstances, the civil society group might speak on behalf of particular communities affected by surveillance

In countries where there are positive relationships between oversight and civil society bodies, there are also usually more productive relationships between civil society groups and government, intelligence agencies, and law enforcement. Although not exclusively due to connections made by oversight bodies, a number of civil society groups in those countries made express reference to the important and ongoing network building role that oversight bodies play.

Learn and apply lessons from other areas to assist oversight

Oversight bodies have strong competencies and experience as they relate to their function of overseeing intelligence agencies and law enforcement bodies. There are regularly challenging, novel and contentious issues that appear in that domain. Yet historically, due to the reasonably unique nature of the work intelligence agencies do, most of the challenging new topics affecting the intelligence community have originated from within that community.

However, with the increasing digitization of agency tradecraft, a greater number of potential techniques, opportunities, or issues can originate outside of the Intelligence Community. An example mentioned by a number of participants was the automated processing of personal data and the application of artificial intelligence and machine learning. Such techniques are being developed with far greater regularity and may be employed to assist intelligence agencies in the processing of the large quantities of data they collect, finding new targets of interest, as well as assisting intelligence analysts in their tasks.

There is a burgeoning field of academic study as well as research by civil society groups looking at a variety of issues associated with artificial intelligence (AI) tools. These include algorithmic accountability, the ethics of the use of machine learning, and potential discrimination in these contexts as neural networks are trained on data full of human biases and flaws. As civil society groups explore these techniques and their implications in fields unrelated to intelligence gathering, they may be able to share lessons learned with oversight bodies. Such lessons might include methods for finding hidden bias in AI algorithms or how to develop safeguards to minimize the privacy intrusions that AI may present.

Recommendations for Both Oversight and Civil Society

Continue to engage and expand engagements

All interviewees agreed that government and civil society representatives should continue to engage, work to expand opportunities for engagements, and initiate engagements where there are not any currently. Oversight and civil society organizations should meet regularly, both to build relationships grounded in trust and to address surveillance issues that arise. While both parties can and should initiate engagements, we recommend that oversight should consider itself as holding the primary responsibility to reach out. To the extent possible given the barrier of classified information, oversight should affirmatively solicit civil society input regarding its oversight reviews.

Communication is critical both during and between engagements

To build trust and facilitate productive working relationships, regular continued communication is critical. When civil society and oversight meet, it is helpful to have a clear agreed-upon agenda in advance. This will help to focus the discussion, to enable all participants to prepare in advance, and to avoid miscommunication. Communication following up after engagements is also important. For example, oversight bodies should make sure to report back to civil society when they have actually acted to respond to a civil society concern. Oversight and civil society should also endeavor to provide a heads up to the other in advance, such as when oversight bodies are about to release a new report or begin a new investigation, or where civil society groups are preparing to release a report.

Hold offsite conferences to bring together representatives of oversight and civil society

Some government and civil society representatives suggested that one way to promote productive engagement between oversight bodies and civil society organizations would be to hold conferences or retreats that brought people together for an extended period of dialogue. Such sessions would need to occur offsite for all participants so that everyone was removed from their daily work responsibilities, and they would need to last at least one full day. Some oversight representatives noted it would be helpful if such events were convened by a think tank or entity that is viewed as more neutral, rather than by a civil society group viewed as having a clear agenda.

Such sessions would serve two important goals. First, they would allow oversight and civil society representatives to get to know each other and build bonds of trust in a way that is not generally possible (or takes much longer) when participants are limited to individual office meetings. Second, such sessions could provide an opportunity to dig into particular issues and work toward a consensus solution.

Endeavor to use a common language

Even where communication does occur, the language everyone uses matters. One barrier to trust has been the tendency of both oversight and civil society representatives to talk past each other or to use terms that may be viewed as loaded or inflammatory which causes dialogue to be unproductive. When meeting behind closed doors, both sides should endeavor to avoid charged language and speak respectfully. In many instances, civil society organizations have developed terms that enable them to better explain government programs to the public, including laypersons unfamiliar with legal phrases or government terminology. However, when civil society representatives use such terms in conversations with government representatives it can be off-putting and cause a barrier to productive dialogue. The converse is true as well—conversations will be more productive and it will be easier to build trust relationships if government officials speak in language that recognizes the privacy and human rights risks posed by surveillance and avoid terminology that appears insensitive to civil society concerns. Everyone in the room should treat each other with respect and recognize that all participants contribute important perspectives.

Use hiring to promote understanding

Some interviewees from both oversight bodies and civil society organizations noted that it would be helpful if their counterparts hired at least some staff members with experience in the other arena. Thus, some oversight representatives noted that it would be helpful for civil society organizations to hire more people with government backgrounds and some civil society representatives stated that oversight bodies should hire more staff with experience working for civil society organizations. All these interviewees felt that such cross-pollination in hiring would facilitate better understanding and collaboration. Relatedly, one U.S. oversight representative noted that it would be helpful for intelligence agency personnel to gain experience working for oversight bodies and interacting with civil society as part of that job—and to gain the change in perspective that such work would bring.

Recommendations for Civil Society

In countries with less mature oversight systems, advocate for creation of additional or more robust oversight bodies

In some jurisdictions there are no expert oversight bodies looking at surveillance issues at all. In others, like France, the oversight body has only recently been established and their practices are still being developed. In these countries, civil society should begin by pressing for the creation or expansion of oversight bodies.

Press oversight bodies to conduct specific reviews of particular authorities and programs

Even in countries with mature oversight systems, not all surveillance activities are subject to rigorous oversight, and civil society organizations should press for increased oversight of surveillance programs that have been neglected by oversight bodies. Civil society may be able to assist oversight in identifying programs or policies that raise concerns due to their impact on individual privacy and human rights, and that should be the subject of oversight.

Urge oversight to conduct reviews even where civil society and public cannot know what is being reviewed

Many intelligence agency activities are completely classified such that even the existence of the type of surveillance cannot be revealed to the public. Thus, even where civil society cannot identify particular intelligence activities that should be subject to oversight, organizations can and should press for oversight bodies to explore the landscape of intelligence activities and ensure that the scope of oversight is as extensive as possible.

Clarify shared goals

As noted in the introduction to this report, at least at a high level, civil society and oversight share the same goal: to ensure that surveillance is conducted in accordance with the rule of law and with respect for human rights. In seeking to overcome barriers of trust with oversight bodies, civil society should emphasize these common goals and point out the extent to which civil society can assist oversight in conducting robust reviews. Whether by providing research and white papers or key questions for oversight to ask in its reviews, civil society can assist oversight bodies, and civil society representatives should emphasize this role in reaching out to oversight staff.

Provide positive reinforcement

When oversight bodies provide welcome transparency or otherwise act in ways that promote accountability, compliance with the rule of law, and respect for human rights, it is helpful for civil society to provide public positive reinforcement. Public praise can help empower these offices to continue their efforts.

Recommendations for Oversight

Seek civil society input before finalizing recommendations or policies

Oversight bodies developing recommendations could benefit from seeking civil society input before reports or policies are finalized. Executive branch oversight bodies in the United States often avoid such consultations due to a desire to protect the “deliberative process privilege,” which protects deliberative, non-final documents from disclosure. However, waiving the privilege in a given instance does not prevent the executive branch from claiming the privilege in another case. Further, executive branch oversight bodies may be able to develop strategies such as presenting several options under consideration for civil society input. Some oversight representatives spoke of the possibility of holding more generalized or “blue sky” conversations about particular issues, such as the appropriate definition of “foreign intelligence information” that can be sought through surveillance. Such practices would enable the oversight bodies to benefit from civil society’s perspective, to spot issues the oversight bodies may have missed, and to ensure that the resulting reports or policies are comprehensive and include appropriate safeguards for privacy, civil liberties and human rights.

Seek technological expertise from sources other than the agencies the oversight body is overseeing, including civil society

To understand the technology being employed in intelligence surveillance and to conduct effective reviews, oversight bodies need to have access to skilled technologists. Oversight bodies should not rely exclusively on the technologists who work at the agencies being overseen. To a large extent, oversight bodies will need to hire their own technologists or otherwise make arrangements to rely on technologists with clearances who do not work for the intelligence agencies. But oversight bodies should also tap into technological expertise in civil society. As one civil society representative explained, computer scientists and other expert technologists are familiar with most of the available technologies. So, even if they must speak about hypothetical scenarios due to their inability to access classified information, such technologists can still assist oversight bodies to understand how surveillance may be conducted and how to ask the right questions when conducting oversight.

Be transparent about how you conduct oversight and about your oversight agenda and press for greater transparency of information about intelligence activities.

Oversight bodies can promote public trust by being transparent about their own operations and how they conduct oversight reviews. Operating in public, such as through public hearings, is particularly helpful. But even where oversight bodies must conduct reviews in secrecy to protect still-classified information or the confidentiality of witnesses, they can still provide transparency about their operations. For example, oversight bodies can include a methodology section at the beginning of their reports.

In addition, to the extent possible consistent with the protection of classified information, oversight bodies should also publicize their oversight agendas to facilitate the ability of civil society and the general public to provide input. Further, as part of oversight reviews, oversight bodies should press for public interest declassification of information about the programs the bodies are reviewing. While this will not be possible for all oversight reviews or for all information regarding any given intelligence activity, oversight bodies should press to re-draw the line between information that must remain classified and information that should be available to the public. Finally, when oversight bodies do provide transparency, they should endeavor to make the information as accessible to the public as possible.

Engage with multiple stakeholders

When conducting outreach to civil society, oversight bodies should solicit input from a wide variety of stakeholders. Where there are numerous civil society groups, such as in the United States and the U.K., oversight bodies should work to engage with organizations from a range of perspectives and with a range of interests. For example, in the United States, although there are currently regular engagements between many oversight bodies and civil society groups that focus on privacy, civil liberties and human rights, oversight has far fewer meetings with groups representing the interests of minority communities, even though these communities have been directly and disparately impacted by surveillance policies.

In limited contexts, oversight bodies should sponsor civil society representatives for security clearances and bring them in for consultations

This model has been followed to a limited extent by the U.S. Department of Homeland Security, which sponsored several civil society representatives for TS/SCI clearances so they could serve on the Cybersecurity Subcommittee of DHS’s Data Privacy and Integrity Advisory Committee (DPIAC). This tactic permits civil society representatives to overcome the barrier presented by classified information, since their clearances will permit them access to classified information. However, such cleared civil society representatives will need to be bound by non-disclosure agreements (NDAs), which can make it challenging to engage in advocacy or academic writing on issues related to the classified topics on which they were consulted. As a result, many civil society representatives would prefer not to be sponsored for clearances. Thus, this strategy will likely only be appropriate in limited contexts where the consultation covers discrete topics.

The level and nature of engagement between oversight bodies and civil society organizations varies widely among the countries examined for this report. Yet, the models of engagement we explored confirmed that oversight and civil society share many of the same goals for holding intelligence agencies accountable, ensuring compliance with the rule of law, and promoting policies that respect human rights. There are many strategies and opportunities for increased cooperation between oversight and civil society that can help promote achievement of these shared goals.

CIA – U.S. Central Intelligence Agency

CSE – Canadian Communications Security Establishment

CTIVD – Netherlands Review Committee on the Intelligence and Security

DoD – U.S. Department of Defense

DRIPA – U.K. Data Retention and Investigatory Powers Act

EDRi – European Digital Rights Initiative

FISA – U.S. Foreign Intelligence Surveillance Act

FISA Court – U.S. Foreign Intelligence Surveillance Court

IC – Intelligence Community

IGIS – Australian Inspector-General of Intelligence and Security

INCLO – International Network of Civil Liberties Organisations

IOCCO – U.K. Interception of Communications Commissioner's Office

IPCO – U.K. Investigatory Powers Commissioner's Office

IPT – U.K. Investigatory Powers Tribunal

NSA – U.S. National Security Agency

ODNI – U.S. Office of the Director of National Intelligence

PCLOB – U.S. Privacy and Civil Liberties Oversight Board

SIGINT – Signals intelligence