Over the past decade, internet platforms have increasingly adopted artificial intelligence and machine-learning tools to shape the content we see and engage with online. These include automated tools for content moderation, ranking of content in news feed and search results, targeting and delivery of digital advertisements, and recommendation systems.¹ These systems are largely invisible to the public. But they are pervasive in our online interactions, and hold significant influence over how we view and interact with the world, determining everything from what news we encounter and what items we purchase to whose voices we see and engage with the most.

Many technology companies assert that algorithmic content shaping systems are valuable because they provide users with a personalized experience on a platform, and enable users to access content the platform deems “relevant” or “useful.” However, by delivering these personalized, algorithmically-curated experiences to users, companies also aim to retain user attention on their services. This translates into significant financial benefits for these companies, as they can target users with advertisements and provide further recommendations for content and purchases. In this way, algorithmically-tailored platform experiences are an important revenue generator and a critical component of platforms’ business models.

Further, as outlined in our report series exploring algorithmic content shaping practices in detail, there is a significant lack of transparency and accountability around how these automated tools are created, trained, refined, and deployed. This raises concerns around how internet platforms are safeguarding user rights to freedom of expression and privacy online when using these systems. Our report series includes case studies demonstrating that algorithmic systems can yield harmful, biased, and discriminatory results, which disproportionately impact marginalized and already vulnerable users and communities. For example, researchers have found that digital advertising algorithms can optimize ad delivery processes in a manner that prevents certain groups of individuals, such as African Americans and women, from receiving employment and housing ads. In addition, researchers have also found that engagement-driven recommendation algorithms can promote troubling content, including hate speech, conspiracy theories, and extremist content.²

This is the final report in our series Holding Platforms Accountable: Online Speech in the Age of Algorithms. It summarizes key themes from the four in-depth reports in the series and from a series of events that OTI hosted on these topics that merit consideration as internet platforms, civil society, researchers, and policymakers continue to explore how to promote greater fairness, accountability, and transparency around these algorithmic decision-making practices.

Approaches to promoting fairness, accountability, and transparency in the context of machine-learning and algorithmic decision-making need to be dynamic and responsive to the fact that today’s internet platforms offer a variety of services, and their algorithmic systems are built to cater to specific goals and objectives.³ As a result, it is important that stakeholders consider the audience, intent, and scope of the platform.⁴ In addition, it is also imperative to note that the goals of algorithmic systems, such as ranking and recommendation systems, vary from platform to platform based on a diverse set of technical and economic goals. According to Daphne Keller, director of platform regulation and Stanford University’s Cyber Policy Center, the technical goal of these systems is to translate human values—such as quality or authoritativeness in content or results—into mathematical and technical formulas. The economic goals of these systems, on the other hand, include maximizing ad revenue, although as Keller notes this is only one foundational piece of the puzzle.⁵ It is therefore vital that as experts work to decipher what meaningful fairness, accountability, and transparency around the use of these algorithmic systems means, they account for the differences in how platforms create and calibrate their systems at different points of the product life cycle.

In addition, it is important for stakeholders in this space to examine how meaningful transparency and accountability can be delivered to different audiences, such as users, researchers and journalists, policymakers, and watchdogs. Each of these audiences has a different level of knowledge and understanding of how algorithmic systems work, a different set of goals for transparency, and desires a different level of granularity. Consequently, any transparency efforts must account for these differences in goals and understanding to appropriately frame any insights.⁶

For example, in order for transparency efforts geared toward users to be meaningful, they must be accessible to the average user who may not have a strong technical background or a strong interest in understanding the granular components of how these systems work. As a result, such user-focused transparency efforts should aim to explain how these systems impact users and their experiences, as well as what level of control users have over how they interact with these systems, in a straightforward way.

Researchers and journalists, on the other hand, often have specific questions or concerns about certain components of these systems, and typically seek out more granular and technical information through processes such as audits. As a result, experts such as Daphne Keller have recommended that these groups should have access to more technical tools such as application programming interfaces (APIs) that enable them to submit queries to understand if there are differences in how an algorithm responds to different users.⁷ Some companies have expressed concerns that offering researchers broad access to such tools (and to their systems in general) could threaten the competitiveness of these companies,⁸ and could have significant privacy consequences, as demonstrated by the Cambridge Analytica scandal.⁹ However, companies can help mitigate these risks by establishing programs that enable a limited group of vetted researchers to access more granular and sensitive data about how these systems work.

Finally, as policymakers, watchdogs, and regulators around the world consider how to regulate and oversee internet platform use of these algorithmic systems, they have begun pressing companies to provide greater transparency and accountability. The type and granularity of these disclosures will similarly vary based on the needs of different regulations and legal obligations. For example, in the European Union, the conversation around the forthcoming Digital Services Act has featured numerous granular transparency-related proposals related to the use of a range of algorithmic systems, including those for content moderation and digital advertising.¹⁰ Similarly, legislative proposals such as the Platform Accountability and Consumer Transparency Act (PACT Act) in the United States have included content moderation-related transparency reporting requirements for internet platforms.¹¹

Stakeholders have made some progress toward achieving consensus around definitions of meaningful fairness, accountability, and transparency, but the space is continuously evolving. However, because the definitions vary across stakeholder groups, and because there is currently no set of standards to guide these frameworks, consensus has not yet been reached.¹²

Some civil society organizations and researchers, for example, have stated that in order for companies to promote greater fairness and demonstrate accountability around their algorithmic systems, they need to de-prioritize signals such as engagement and click-worthiness, which have been found to amplify and exacerbate many of the harms associated with these systems. These experts recommend that companies design automated tools to emphasize truthful and authentic information. Other experts, however, have argued the need for these systems to prioritize interests such as preventing racial discrimination, maximizing diversity of opinion and sources, and promoting competition.¹³ It is challenging for one system to achieve all of these goals. Therefore, as stakeholders continue exploring what meaningful fairness, accountability, and transparency looks like, experts have outlined that it would be helpful to reconcile these differing demands and determine which values must be prioritized in these content shaping algorithms.¹⁴ In addition, it is important to recognize that not all human values can easily be codified technically. As a result, how an algorithmic system understands such values, such as factual accuracy or societal benefit, will to an extent always be limited.¹⁵

Some of the potential new mechanisms for promoting greater fairness, accountability, and transparency around the use of these algorithmic content-shaping systems include human rights impact assessments, algorithmic audits, enhanced transparency reports, and political ad libraries. None of these mechanisms are blanket approaches, and they all need to be contextualized in their application. Further, these approaches are not mutually exclusive. Some target different aspects and effects of these algorithmic systems, and they could therefore be applied in a complementary manner.¹⁶

Joe Westby, technology and human rights researcher at Amnesty International, noted during one of our events on algorithmic content shaping that there has been a longstanding push for comprehensive privacy legislation in the United States.¹⁷ Despite this, the United States has still not enacted comprehensive federal privacy legislation, and the U.S. Congress has not yet even begun debating proposals. As a result, algorithmic content-shaping systems that rely on the collection of vast amounts of personal and behavioral data have become ubiquitous in today’s digital world. These data collection practices are part of what has been termed the “surveillance capitalism” economy, where companies monetize data on user behaviors and interests.¹⁸ This can lead to further data collection practices because the companies—which are profiting financially from collecting and harnessing user data—may be incentivized to create vast datasets that can be used to both further train algorithmic systems and increase revenue.¹⁹Unless individuals are prepared to forego using these services, which are often integral to daily life,²⁰ they may feel they have little choice but to accept that companies will continue to collect and monetize their personal data. Additionally, these expansive data collection practices are often not rights-respecting, and there is little transparency and accountability around how user data is collected, used, shared, and retained.²¹

For example, internet platforms collect vast amounts of data on users in order to compile their datasets and train their algorithms. However, users have little insight into this process and how their data is being used, and have little control over whether their data is used for these purposes. In addition, these algorithmic systems construct profiles about each user with little transparency or accountability. There is also little visibility into what data is being used as inputs into these systems, how these systems are processing this information, and what outputs they are generating. In this way, these systems assign an identity to users, and users have little agency to control or change the assumptions that are made about them. This is particularly concerning given that researchers have found that algorithmic systems—such as recommendation engines and ad targeting and delivery systems—can yield discriminatory, biased, and harmful results that disproportionately impact communities of color and other marginalized groups.

As companies have begun thinking through how to prevent further harmful and discriminatory outcomes and increase fairness in their systems, algorithmic audits have emerged as a method for assessing potential negative impacts. Researchers considering how these audits should be structured have raised questions related to what kind of data could be used to audit and better test existing algorithmic systems, and whether this would require collection of (and access to) sensitive information. Currently, many technology companies do not collect explicit data about race. However, their algorithms can infer race through certain data points. For example, an algorithm could assume an individual’s race based on whether they live in a neighborhood or ZIP code that has historically been associated with a specific racial group. Similarly, an algorithm could infer an individual’s race based on their interest in particular affinity groups or products, such as hair care products designed for Black women. These inferences are not always accurate, however, and thus using this data for testing purposes would not be as valuable. This has raised questions around whether companies should begin collecting or purchasing sensitive demographic data, such as race, in order to enable fairness testing, as well as what limits must be placed on this data if it is obtained. This raises additional questions around whether the benefits associated with having such data outweigh the potential harms that could result.

In addition to collecting a troubling amount of user data, internet platforms also offer users only a limited set of controls with which they can understand and determine how their experiences are being personalized.²² These tools are continuously evolving, and feedback from civil society, researchers, and other stakeholders is vital to driving this development process forward. As companies introduce these controls, it is important that they make the controls accessible and digestible, and do not require users to search through multiple pages or drop-down menus in order to acquire a certain piece of information or change a specific setting.²³ Companies should also enable data portability, which would give users the ability to extract an archive of the data that they have shared with an internet platform (or that the internet platform has collected about them) in a format that is structured, machine-readable, and that allows transfer of this data to a different service.²⁴ By enabling users to transfer their data from one service to another, and to use the data for their own purposes, data portability provides users with more agency and control over which companies have access to their sensitive data and how it is used.²⁵ The cross-industry Data Transfer Project, which aims to create an open-source, service-to-service data portability platform that enables users to move their data between online service providers is a good starting point for these efforts.²⁶

(Disclosure: New America receives funding from Apple, Facebook, Google, and Microsoft. View our full list of donors at www.newamerica.org/our-funding.)

Over the past several years, the harms and discriminatory effects that can result from internet platforms’ use of algorithmic content-shaping systems have become more pronounced. Research has indicated time and time again that these negative consequences are disproportionately shouldered by marginalized and vulnerable communities, including racial minorities, women, and LGBTQ+ individuals. In response, policymakers around the world are considering how to provide greater oversight of these algorithmic systems, and whether and how these systems should be regulated.

In the United States, the First Amendment limits the extent to which the government can direct internet platforms to moderate content on their platforms. However, the government can enact greater transparency and accountability requirements for these platforms.²⁷ Recently, members of Congress have introduced two bills, the Algorithmic Accountability Act and the PACT Act, to tackle these issues. Among other things, the Algorithmic Accountability Act would authorize the Federal Trade Commission (FTC) to issue regulations requiring internet platforms to conduct impact assessments of their algorithmic decision-making systems; evaluate these systems for bias, discrimination, privacy, fairness, and security; and refine their systems based on the results of these impact assessments.²⁸ As previously discussed, the PACT Act, includes numerous transparency reporting obligations for internet platforms, including for algorithmic curation processes²⁹ such as content moderation and content ranking.³⁰

The Honest Ads Act is another piece of legislation that has been introduced in an effort to tackle the need for greater transparency and accountability in the content shaping space. Introduced shortly after the 2016 U.S presidential elections, the Act would require online platforms to provide more transparency around the scope and scale of their online political advertising operations.³¹

Although the U.S. federal government has not yet enacted any legislation specifically addressing algorithmic transparency and accountability, anti-discrimination statutes that pre-date the digital age can and should apply online to ensure greater fairness and accountability around these systems. Going forward, policymakers should also clarify that all offline anti-discriminatory statutes, including the Civil Rights Act of 1964, the Fair Housing Act, and the Age Discrimination in Employment Act, apply in the digital environment. Where necessary, Congress and state legislatures should enact appropriate legislation to fill gaps or clarify the applicability of such laws.³² Stakeholders should also strive to ensure that these conversations center longstanding concerns related to civil rights, discrimination, and bias, and use these opportunities to bring in and lift up the voices of individuals and communities who are disproportionately impacted by the harms of these systems.³³

Further, in order to promote greater fairness, accountability, and transparency around these algorithmic systems, U.S. policymakers must pass comprehensive federal privacy legislation that reins in, and imposes guardrails on, internet platforms’ massive data collection practices.³⁴ This legislation should, at a minimum, limit the types of data that can be collected and the purposes for which it may be used, protect civil rights, prevent unlawful dicsrimination, advance equal opportunity, and provide redress for privacy violations.³⁵ As outlined above, extensive data collection practices are integral to fueling the creation, deployment, and refinement of algorithmic content-shaping systems, and they pose significant privacy risks to users. Today, we tend to view existing automated systems and their reliance on rampant data collection practices as a given. However, strong federal privacy legislation can change this and ensure greater rights for users.

In the European Union, policymakers are similarly considering how to encourage and require greater fairness, accountability, and transparency from platforms. For example, the EU Code of Practice on Disinformation, introduced in 2018, outlines self-regulatory standards that platforms can voluntarily sign on to in order to fight disinformation.³⁶ The code of practice includes a number of transparency and accountability commitments, including issuing disclosures around political advertising and demonetizing accounts that spread disinformation.³⁷ In addition, the previously discussed Digital Services Act (DSA) has emerged as a key component of the European Commission’s roadmap to rethink “Europe’s digital future”³⁸ and revise the existing legal framework for intermediaries and their responsibilities related to user content and conduct. Although the transparency and accountability obligations under the DSA are still being deliberated, conversations have thus far centered around the need for greater transparency and accountability around content moderation, digital advertising, and the use of algorithms. These conversations have also included recommendations to create an independent body that would oversee the implementation of these transparency requirements, as well as other things.³⁹ The European Parliament is also considering legislation that would require companies to conduct human rights due diligence around their operations.⁴⁰

Unlike the United States, the European Union has passed comprehensive privacy legislation in the form of the General Data Protection Regulation (GDPR). However, experts have outlined that although the GDPR takes necessary steps to safeguard user privacy, it also creates barriers that prevent companies from sharing data with researchers.⁴¹ Going forward, policymakers in the EU, United States, and beyond should work to improve data access mechanisms and policies for researchers and journalists, as this can fuel further analyses and move the ball forward with regard to fairness, accountability, and transparency efforts.

In addition, policymakers must ensure that any form of regulation or voluntary guidelines are rights-respecting, do not infringe on the freedom of expression or privacy rights of individuals, and do not undermine critical intermediary liability provisions.⁴² As policymakers make these considerations, civil society groups, civil rights organizations, and researchers should further collaborate to guide these conversations and ensure that any requirements or guidelines yield meaningful outcomes.

The previous four reports in this series outline a number of recommendations around how to promote greater fairness, accountability, and transparency around the use of algorithmic decision-making in specific categories of algorithmic content shaping systems and tools. These recommendations included: platforms should publicly share comprehensive information regarding the policies and practices that guide the creation, use, and recalibration of their algorithmic content shaping systems; platforms should provide users with digestible explanations of how these systems are implemented and access to a robust set of controls; and stakeholders in this space should further develop methods for evaluating and addressing bias, discrimination, and other concerning outcomes that can result from these systems. Based on the four reports we have published, as well as the event series we subsequently hosted, we have identified a number of additional cross-cutting recommendations that stakeholders should consider in order to further this work.

In particular, internet platforms should:

1. Establish corporate programs that enable pre-vetted researchers to access data related to algorithmic systems in order to further evaluation and research in the fairness, accountability, and transparency space.
2. Provide users with comprehensible and accessible explanations of how their data is being used to shape their online experiences and train algorithmic systems.
3. Enable users to determine if and how their data is collected and used by algorithmic systems to shape their online experiences, and featured in datasets that are used to train these systems. These tools should be accessible and comprehensible.
4. Solicit feedback from civil society organizations, researchers, and other stakeholders in order to continuously refine and develop user-focused controls and explanations.
5. Enable easy-to-use data portability, such as that demonstrated by the cross-industry Data Transfer Project, so that users have greater agency and control over which companies have access to and can use their personal data.

In addition, U.S. policymakers should:

1. Enact rights-respecting transparency and accountability requirements for internet platforms.
2. Pass comprehensive federal privacy legislation that provides users with rights over their information and limits how companies can use personal data. This legislation should protect civil rights, prevent unlawful discrimination, advance equal opportunity, and provide redress for privacy violations.
3. Clarify that all offline anti-discriminatory statutes, including the Civil Rights Act of 1964, the Fair Housing Act, and the Age Discrimination in Employment Act, apply in the digital environment. Where necessary, Congress and state legislatures should enact appropriate legislation to fill gaps or clarify the applicability of such laws.

Internet platforms, researchers, civil society organizations, and policymakers should:

1. Ensure that efforts to develop approaches for promoting fairness, accountability, and transparency around the use of these algorithmic systems account for variations in the services internet platforms offer and the subsequent goals of these algorithmic systems.
2. Encourage tiered models of transparency and accountability that consider the differing intentions, needs, and levels of comprehension that different stakeholders such as users, researchers, and policymakers have.
3. Make proactive efforts to include and lift up voices and experiences of civil rights organizations and impacted communities in conversations around fairness, accountability, and transparency.

Artificial intelligence and machine-learning tools are pervasive in our online ecosystem, and their use cases are likely to continue growing. Although these systems can provide users with tailored platform experiences, researchers have documented numerous instances in which they can also generate harmful results that are often discriminatory and biased. Given that these algorithmic systems hold a significant amount of influence over how users see and engage with the world, internet platforms should do more to provide fairness, accountability, and transparency around how they build, use, and refine these tools. They should also provide greater insight into what the effects of these systems are. Simultaneously, researchers, civil society, civil rights organizations, and policymakers should do more to hold companies accountable for creating and using algorithmic tools in a responsible manner. Automated tools are rapidly and constantly changing the way we interact with the digital sphere. Internet platforms should adopt robust transparency and accountability mechanisms so that algorithmic systems will be rights-respecting, and subject to review and redress for any harmful consequences.