Since the early 2000s, the amount of information on the internet has grown tremendously. Whether it be news outlets, social media, or e-commerce platforms, the online ecosystem has become a go-to destination for users seeking a variety of information and experiences. At the same time, users have faced a fundamental challenge in identifying credible sources and understanding which of them to use. In order to help users access high-quality, relevant, and accurate information and content, a number of internet platforms rely on proprietary algorithmic tools to curate and rank content for users.

This report focuses on search engines and platforms that offer news feeds, both of which deploy algorithms to identify and curate content for users. Many of these platforms use hundreds of signals to inform these ranking algorithms and deliver users personalized search and news feed experiences.

These algorithms control both the inputs and outputs of the information environment. They evaluate and process incoming information to identify what content is most relevant for users. They then determine which of these outputs a user should see and rank the outputs in a hierarchical manner. In this way, these platforms act as gatekeepers of online speech by exercising significant editorial judgment over information flows.¹

Most internet platforms have heralded the introduction of personalized search results and news feeds as a positive—and now integral—feature of their services. Many platforms assert that personalization enables users to access and engage with content that is more relevant and meaningful to them. Personalization features also enable platforms to achieve significant growth and boost revenue through avenues such as advertising.

However, there is a fundamental lack of transparency around how algorithmic decision-making around curation and ranking takes place. Because these practices can have a variety of negative consequences, this is concerning. In fact, many users are often unaware that such algorithms are being used by platforms to shape their online experiences.² Rather, these users believe that the subjective frame presented by curation and ranking algorithms is representative of reality. As a result, users have grown accustomed to outsourcing judgment, autonomy,³ and decision-making to internet platforms and their opaque algorithms, who decide—based on their perceptions of what user interests and values are—what users’ online experience should be.⁴ This disparity in algorithmic awareness and understanding is fostering a new digital divide between individuals who are aware of and understand the impacts of algorithmic decision-making, and those who do not.⁵

As algorithmic tools are increasingly used to curate and rank content on internet platforms, concerns around fairness, accountability, and transparency have grown. In particular, an increasing number of researchers have noted that users lack awareness of algorithmic decision-making practices. In addition, there is a significant lack of transparency from internet platforms regarding how these tools are developed and deployed, and how they shape the user experience. Further, researchers have outlined that users and content creators often lack meaningful controls over, and agency related to, algorithmic decision-making practices.

In addition, although these algorithmic curation and ranking tools remove the need for humans to make significant manual and individual decisions regarding millions of pieces of content, they do not remove the need for human editorial judgment in this process, nor do they reduce bias. The term “bias” in this context does not solely refer to inappropriate preferences based on protected categories like race or political affiliation. Rather, these tools compile insights into a broad range of weighted signals. Ranking algorithms then analyze the data to prioritize certain forms of content and certain voices over others. These algorithms also incorporate the judgments of the engineers who have developed them, particularly with regard to what information users are likely to find interesting and meaningful. In addition, algorithms can infer correlations in data that may reflect societal biases. Indeed, often times, these algorithms result from machine learning “black box” systems. This means that even though developers may know what the inputs and outputs of an algorithm are, they may not know exactly how the algorithm operates internally. Therefore, concerns regarding algorithmic bias and accountability have grown as these algorithmic decision-making practices have become more prevalent.

This report is the second in a series of four reports that will explore how automated tools are being used by major technology companies to shape the content we see and engage with online, and how internet platforms, policymakers, and researchers can promote greater fairness, accountability, and transparency around these algorithmic decision-making practices. This report focuses on the algorithmic curation and ranking of content in search engine results and in news feeds on internet platforms. It uses case studies on three search engines—Google, Bing, and DuckDuckGo—and on three internet platforms that feature news feeds—Facebook, Twitter, and Reddit—to highlight the different ways algorithmic tools can be deployed by technology companies to curate and rank content, and the challenges associated with these practices.

Editorial disclosure: This report discusses policies by Google, Microsoft, and Facebook, all of which are funders of work at New America but did not contribute funds directly to the research or writing of this report. New America is guided by the principles of full transparency, independence, and accessibility in all its activities and partnerships. New America does not engage in research or educational activities directed or influenced in any way by financial supporters. View our full list of donors at www.newamerica.org/our-funding.

Search engines have emerged as an essential tool as the internet has expanded, supporting key aspects of free expression online through information access and information dissemination. Search engines enable users to more effectively sift through and access endless amounts of information, and they have empowered individuals, businesses, and publishers to disseminate information.

Although the process of conducting a search on a search engine seems straightforward, the ways algorithms curate and rank search results raise a number of concerns. These algorithms underpin the immense power of search engines, which are able to determine what information a user sees, the type of results they can access, and which publishers and pieces of information a user engages with first. In this way, search engines play a significant role in shaping every one of their user’s perspectives and mindsets.⁶ Additionally, not all search engines operate in the same manner. Therefore, which search engine a person uses will also influence their viewpoints and opinions.⁷ As Wired’s Brian Barrett wrote, “The internet is a window on the world; a search engine warps and tints it.”⁸

The topic of algorithmic curation and ranking of search results has become especially prominent in the news recently, as conservative politicians in the United States have claimed that search engines, such as Google, and internet platforms, such as Facebook and Twitter, have instituted a liberal bias within their search results and news feed curation practices.⁹ However, there is little evidence that such bias is actually present.¹⁰

Technology companies that operate search engines can utilize artificial intelligence and machine learning in a number of ways.¹¹ These include powering speech to text searches and enabling visual searches. This section of this report, however, will focus on how these algorithmic tools are used to curate and rank search results. It will use three case studies—Google, Bing, and DuckDuckGo—to explore how different companies have structured and implemented the practices of curating and ranking their search results, and what challenges they have faced in the process.

Google is the world’s largest search engine.¹² It was founded in 1998 by Larry Page and Sergey Brin, and today operates under parent company Alphabet Inc. Over the past two decades, Google’s influence in the technology sector, and in society broadly, has grown significantly. As of July 2019, Google had 92.19 percent of the global search engine market share.¹³ It also currently ranks first for global internet engagement on Alexa rankings.¹⁴ Google’s search engine product, known as Search, is currently available in over 150 languages and over 190 countries.¹⁵

Brin and Page’s original vision of the Google search engine was primarily based on the tenets of citation analysis used by scholars, researchers, and scientists. In such research-oriented fields, the more a work is cited, the more it is generally considered legitimate and high-quality. Brin and Page saw value in this approach; by identifying web pages that other web pages frequently linked to, they would be able to identify similarly legitimate, or at least popular, online sources. From this sprang Google’s original search engine ranking algorithm, known as PageRank.¹⁶ However, as outlined by Safiya Noble, this approach comes with a range of problems. For example, when citing work in a research publication, all citations are given the same weight in the final bibliography, despite expected differences in how much the author relied on each work cited. In addition, in a research publication, all citations are weighted equally, regardless of whether the author mentioned a work and its contents to validate it, reject it, and so on. Brin and Page predicted some of these complications, and the ranking algorithm has evolved considerably since its original conception to account for further limitations.¹⁷

According to Google, its Search product aims to enhance a user’s search experience by providing them with personalized search results. These search results are customized based on personal user data collected by Google, such as browsing and purchase history, and are generated using a combination of manual and algorithmic tools. This personalized search feature also, however, serves as a major source of advertising revenue for the company.

Over the past few years, Google has come under significant criticism for its personalized Search feature. In particular, many researchers and internet activists have expressed concerns that by delivering search results that users are likely to click on and be interested in based on their prior searches and personal data, Google is creating a “filter bubble.”¹⁸

Recently, Google has stated that it is moving away from offering personalized search results.¹⁹ However, today users still receive personalized search results on the platform.

Underpinning Google’s search engine is a process called crawling. Crawling deploys software, known as web crawlers, to identify publicly available web pages. Web crawlers determine which websites to browse, how often to browse them, and how many pages should be browsed from each website. Typically, web crawlers select web pages to crawl based on previous crawls and sitemaps.²⁰ Once the web crawlers have identified a set of web pages, they visit them and utilize links on these web pages to identify other web pages. During this process, the web crawlers specifically look to identify new websites, changes that have occurred to existing websites, and dead links. These web crawlers then bring data about these web pages back to Google’s servers. When a crawler identifies a web page, Google’s system renders the content of the page, like a browser does, and works to identify signals such as keywords and website freshness (the recency of a website’s content). These signals are continuously monitored using the Google Search index. The Google Search index contains hundreds of billions of web pages and it includes an entry for every word seen on every web page that has ever been indexed. When a new web page is indexed, it is added to the entries for all the words it contains.²¹ When a user enters a query into Search, the Search algorithm identifies keywords in the query and matches it against web pages in the index. It then ranks the subsequent search results based on over 200 different signals.²² The search result ranking process is, for the most part, conducted using algorithms. According to Google, the company does not deploy human curation when ranking its search results.²³

Some of the factors and signals that influence the Search ranking algorithm are:

1. The meaning of a user’s query: In order to provide a user with a relevant result for their query, Google first needs to identify the intent behind this query. To do this, the company has developed language models, including natural language processing models, which are adept at understanding which particular keywords should be looked up in the index. Additionally, Google has developed a synonym system, which enables for search results to include results related to synonyms of keywords in the original query—for example, if the query included blanket, then the search results may also provide links to websites related to duvets as well. The introduction of this synonym system has enhanced Google search results in over 30 percent of searches across languages. Search algorithms also attempt to decipher what category of information a user is looking for, such as whether it is a specific or broad search, or a language-specific search. If a user searches for a trending keyword or topic, such as the results of the latest UFC fight card, Google’s freshness algorithm will interpret this as a signal that recent information may be more useful than older results.²⁴
2. Relevance of web pages: After assessing the meaning of a user’s query, algorithms begin to evaluate the content of different web pages in order to understand whether a page contains information that is relevant to the initial query. One of the clearest indicators that a web page may be relevant is if the headings or body of the page contain the same keywords as the ones in the search query. In addition to keyword matching, Google asserts that it uses aggregated and anonymized interaction data in order to determine whether the search results are relevant to the initial query. This data informs signals that enable Google’s machine-learning systems to better determine relevance. These relevance signals help the Search algorithm determine whether a web page contains information that answers the initial search query, or whether it simply repeats the same question posed in the query. For example, if a user’s query was for “books”, the algorithm would determine whether a web page contains relevant content aside from the keyword “books,” such as pictures, videos, lists, reviews, and so on. According to Google, although these search systems are constructed to seek out quantifiable signals in order to determine relevance, they are not structured to assess subjective notions such as the political ideology of a page’s content.²⁵
3. Quality of content: Google’s Search algorithms also aim to prioritize the most reliable and high-quality search results. In order to determine reliability and quality of content, Google’s systems identify signals that can help assess expertise, authoritativeness, and trustworthiness on a given topic. They also search for web pages that many users appear to value for similar search queries. For example, if other reliable and prominent websites link to a web page, that is considered a good indicator that the content of that web page is reliable. This is known as PageRank. PageRank is a mathematical formula that can be used to judge the “value of a page” on the web by assessing the quantity and quality of other pages that link to it. PageRank was one of the fundamental components of the original Google Search algorithm and it was inspired by the system scientists used to gauge the importance of scientific papers, which was to evaluate how many other scientific papers referenced or cited them.²⁶ However, after Google deployed a public PageRank score for each web page, bad actors began working to game this system, which led to a large volume of link spamming.²⁷ As a result, the public PageRank scoring was retired, although PageRank remains a component of the Search algorithm. Additionally, as other factors became increasingly more important for the Search algorithm and for ranking, other signals were incorporated and adopted into the algorithm.

   Google also uses aggregated feedback from Google’s Search quality evaluation process to further refine its ability to assess information quality. Further, Google uses spam algorithms in order to determine the quality of a page and ensure that low-quality, harmful or manipulative web pages are not ranked highly in search results.²⁸

4. Usability of web pages: When ranking search results, Google Search assesses whether web pages are easy to use and ranks those that are deemed more user-friendly higher than those that are not. Some of the signals that inform whether a web page is usable include whether the website is formatted properly for multiple browsers, whether it has been formatted for various devices and sizes (such as desktops, tablets, and smartphones), and whether the web page can be loaded by users with slower internet connections.²⁹ This can make the Google Search product more accessible to those using the service through different devices or in different regions. However, it may also render some otherwise high-quality sites inaccessible to these users if web page owners do not abide by Google’s user-friendly requirements, thus stifling information flows.
5. Context and settings: As previously mentioned, Google extracts insights from users’ personal data in order to inform and tailor search results. These data points include location, purchase history (as determined by crawling users’ Gmail inboxes for purchase receipts³⁰), past Search history, and Search settings. According to Google, a user’s Search settings on factors such as preferred language or SafeSearch—a tool that enables users to filter out explicit or sensitive content—also enable Google to understand which results are likely to be the most useful for them. Google has also stated that Search results may be personalized based on a user’s Google account activity. For example, if a user searches for “events near me,” the search engine may personalize recommendations based on events it thinks the user may be interested in. According to the company, these inferences are made so that search results can match a user’s interests. The company asserts that they are not, however, designed to infer a user’s race, religion, political affiliations, or other sensitive characteristics.³¹

The weight applied to each factor depends on the nature of the query. For example, the freshness algorithm may play a more prominent role for queries related to current events.³² In addition, Google has detailed specifications for how pages that may impact the “future happiness, health, financial stability, or safety of users” are weighted.³³ These web pages are known as “Your Money or Your Life” pages (YMYL) and they include websites that let users make purchases or pay bills, offer financial, medical or legal information, or produce news. Google has especially high page-quality rating standards for these pages, as low quality content on such pages could have a significant negative impact on users.³⁴ As a result, when Search algorithms detect that a query is related to a YMYL subject, it places more weight in the ranking system on factors such as authoritativeness, expertise, and trustworthiness.³⁵

As the largest search engine in the world, Google is responsible for curating and delivering a large amount of information to users. It therefore assumes some responsibility for the impact of its algorithms in shaping the worldviews of users. The platform, however, has come under significant criticism for not providing enough transparency and accountability around its search curation and ranking practices, particularly around how it personalizes search results.

According to many search engines, a personalized search experience can help users filter through the vast amount of information available on the internet and access results that are relevant and useful. Personalized search experiences also have helped platforms like Google achieve significant growth and boost revenue. Personalized search results were a key advantage of Google when it first released, and they became an integral feature of Google Search. In 2009, the company even made personalized search the default option for all users who were logged in to their Google accounts. Also in 2009, the company stated it would deploy an anonymous cookie in order to provide personalized search results to users that were logged out or didn’t have a Google account. This cookie operated separately from a user’s Google Account and web history, which were only available to users who signed in.³⁶ In a blog post, the company also stated they used contextual signals, such as those that aim to reduce the ambiguity of queries, in order to rank search engine results, even for users who were logged out. In this case, this means that Google would use information from a user’s recent search to clarify their current search. The platform said that this would not result in significantly different search results.³⁷ However, its deployment did raise a number of privacy concerns regarding how Google was collecting and using user information across its different Search modes.

Over the past few years, the platform has asserted that it has moved away from personalizing search results. In September 2018, for example, CNBC reported meeting with Google’s algorithm team and learning that because each search query requires a significant amount of context, the opportunities for personalization are quite limited.³⁸ Additionally, the company has suggested that personalization does not significantly improve the quality of the Search experience, and as a result they have moved away from it.³⁹ However, it is unclear whether these shifts are due to increased scrutiny on the practice of offering personalized search results or if the company is purposefully changing the way their Search algorithm works.

Many critics believe Google continues to curate and personalize search results to a large extent.⁴⁰ One of Search’s biggest critics is privacy-focused search engine DuckDuckGo. In June 2018, DuckDuckGo conducted a study in which 87 volunteers in the United States conducted searches on Google’s private browsing mode (known as “Incognito”) while logged out and on the regular Search platform. The volunteers searched for three politically charged topics: “gun control,” “immigration,” and “vaccinations.”⁴¹ The study revealed that most participants received unique results, some saw search results that others did not,⁴² and some received fewer domain results than others.⁴³ Additionally, the ranking of these search results also varied.⁴⁴ The researchers found more than double the variation in the search results when comparing Incognito searches to regular searches.⁴⁵ There was also significant variation between news and video results.⁴⁶ This suggested that even if users were conducting searches in Incognito mode (which aims to provide users with a private browsing option that does not save a user’s search history), and while logged out, they still received personalized search results. This is likely because websites can use IP addresses and browser fingerprinting in order to identify users even when they are searching in these modes.⁴⁷ According to the researchers, had the results been truly non-personalized, then all users would have received the same search results.⁴⁸

DuckDuckGo’s researchers worked to control factors that could have influenced these results, such as location, time, and being logged into Google, by having the volunteers conduct searches while logged out, and by having them conduct the searches at the same time and on the same day. In addition, they controlled for potential variances in location, which might have resulted in, for example, local news stories appearing in the search results, by reviewing all links by hand and comparing them to the city and state of the volunteer who viewed them.⁴⁹

Google responded to the research by calling it flawed,⁵⁰ stating that DuckDuckGo’s attempts to control for time and location differences had been ineffective. They also claimed that the researchers assumed that any difference in the search results automatically suggested personalization.⁵¹ Further, they highlighted that search results related to news and current events were likely to continuously change depending on daily occurrences. They also stated that personalization was performed on a small portion of overall queries, primarily related to location⁵² and utilizing previous searches in order to decipher context for a current search.⁵³ Google has also suggested that any variation in search results is likely due to factors, such as a user’s location, the language the search is performed in, and the distribution of Search index updates throughout Google’s data centers.⁵⁴ Despite these rebuttals, however, there are still significant concerns around Google’s efforts to personalize search results, and the lack of transparency and accountability around these practices.

In particular, internet activists, such as Eli Pariser, have asserted that personalized search results can create and maintain filter bubbles and promote certain biases or world views. In his book, The Filter Bubble: What the Internet is Hiding From You, Pariser argues that by providing users with content a system predicts they will like, internet platforms are filtering out and therefore preventing users from accessing information that may challenge their perspectives or broaden their horizons. This places these users in a “filter bubble” and amplifies their confirmation bias. It can also result in users becoming ill-informed, developing perceptions that are skewed towards one perspective, and even developing a distaste for ideas that are unfamiliar or contrary to their own. This is particularly concerning in the context of political discourse and rising political polarization around the globe.⁵⁵ This has become an increasingly prominent topic since the 2016 U.S. presidential election, in which numerous internet platforms, including Google, were accused of creating filter bubbles or echo chambers through their algorithmic content curation practices.

However, some researchers push back against the notion that an online filter bubble has enhanced polarization. For example, economists from Brown University and Stanford University studied the relationship between polarization and the use of online media in American adults between 1996 and 2012. They found that polarization has largely been driven by those Americans who spend the least amount of time online, such as those over the age of 75. According to the study, those belonging to younger demographics who use the internet more frequently, demonstrated little difference in their level of polarization in 2012 than they had in 1996, when online platforms were far less prevalent and influential.⁵⁶

Additionally, some conservative lawmakers in the United States allege that Google has demonstrated bias in how it curates and ranks its search engine results. According to this critique, this bias prioritizes and preferences liberal information sources, in particular ones that are critical of conservatives. In December 2018, Google CEO Sundar Pichai testified before the U.S. House Judiciary Committee on the topic of alleged bias against conservatives on the platform, denying that such bias exists.⁵⁷ This conversation has been particularly prominent in the political sphere as internet platforms have ramped up their efforts to remove misinformation and disinformation on their services. In the process, a number of politically charged web pages and pieces of content have also been impacted, contributing to claims of conservative bias.⁵⁸

However, as previously discussed, there is little evidence to support the notion that such political biases exists in Google’s search results. Google has stated that Search is designed to decipher the usefulness and relevance of a web page, not to promote the political and ideological viewpoints of the individuals who built or audited the system.⁵⁹ The company has also stated that it does not utilize human curation when ranking search results, and rather relies exclusively on algorithms in order to debunk some of these claims. According to Danny Sullivan, Google’s public liaison for Search, Google does not manually intervene on specific search results when addressing issues with ranking. This is because tweaking one search result or addressing one query when the search engine receives trillions per day does not have a strong impact on the overall Search experience.⁶⁰ Additionally, the platform has asserted that its systems are not designed to make subjective determinations about truthfulness on web pages. Rather, it uses a range of measurable signals—such as the PageRank signal, which is used to determine authoritativeness⁶¹—to assess how users and other web pages perceive the expertise, trustworthiness, and authority of a web page and its content.⁶² Google’s ranking algorithms then promotes these web pages, particularly during searches where the original query could surface misleading information.⁶³

However, just because Google does not deploy human curation during the search ranking process, it does not mean that no bias is present. Algorithms are not neutral and bias-free. The signals that an algorithm uses are designed to prioritize certain information or qualities over others in order to curate and rank content. This is a key part of how search engines and news feeds today work, and is often seen as integral to their operations. On the other hand, the term “bias” in this context does not solely refer to inappropriate preferences based on protected categories like race or political affiliation. Although an algorithm may contain biases based on how it sorts and ranks content, it is difficult to know exactly what these biases are. This is particularly true with black box machine learning systems. Therefore, it is difficult to draw reliable conclusions, such as whether algorithms are biased against a certain political party. Additionally, these algorithms incorporate the judgments, preferences, and priorities of the engineers who developed them, particularly around what information users are likely to find interesting and meaningful. Furthermore, as outlined by Dr. Safiya Noble, an associate professor at the University of California, Los Angeles, search results can also reinforce and perpetuate societal biases. In her book, Algorithms of Oppression, Noble outlines how in the early 2010’s Google’s search engine results related to women, in particular women of color, were overly sexualized and stereotyped. Noble also highlights a case in 2016, in which Google Images search results for “three black teenagers” delivered mugshots of African-American teenagers, whereas similar search results for “three white teenagers” delivered “wholesome and all-American” results. In this way, search engine algorithms can reinforce existing societal stereotypes, often disproportionately impacting already marginalized communities.⁶⁴

Although Google has shared some information about the signals that contribute to its search algorithm, it has not provided a comprehensive overview of all the signals, how they interact with one another, and what impact they have on online expression as a whole. One reason many platforms fail to provide comprehensive transparency around these signals is because these signals make up the platform’s algorithmic “secret sauce,'” and they therefore want to keep it confidential in order to maintain a competitive edge. However, even if it is valid to maintain confidentiality for certain operational details, it is important to promote transparency and accountability to the greatest extent feasible, and companies’ claims about trade secrets should not outweigh the public interest.

Additionally, many website owners have expressed frustration over the fact that Google does not always announce when it is updating or making changes to its search algorithm. As a result, after a change, some content creators find that their web pages are no longer ranking as well. The ranking of websites in search results is of vital importance to website publishers, as it influences the success of their websites. Typically, a user only clicks on the top few search results. The remaining search results on the page receive far lower clickthrough rates. As a result, being able to rank high in search results, and understand how various ranking signals impact one’s website and associated information flows, commerce outlets, and so on, is important.⁶⁵

Part of the reason Google may not make frequent announcements is because it regularly makes changes to its Search algorithms. In 2017 alone, Google performed over 200,000 experiments and subsequently instituted 2,400 changes to Search.⁶⁶ In July 2019, the company announced that over the past year they had made 3,200 changes to Search systems. These included updates for specific features or elements as well as broad core updates.⁶⁷ According to Google, when making changes to its Search systems, the platform identifies areas of improvement, develops a solution, and then tests that solution. After deciphering whether the solution is feasible and improves the Search experience, they then implement it. These algorithmic changes apply to a broad range of similar searches.⁶⁸ Thus far, Google has only provided public updates around some broad core changes they have instituted. The company has stated that they aim to provide site owners with prior notice of “significant, actionable changes to our Search algorithms,”⁶⁹ but in many instances this has proven to not be enough.

Greater transparency would enable publishers of web pages to better understand how their content is curated and ranked, and help them ensure they can adequately distribute their content. Currently, there are a number of search engine optimization (SEO) organizations and communities that have sprung up that speculate on algorithmic changes. However, without clear direction and information, the ability of these communities to understand how they can effectively exercise their free speech online is limited, and the ability of users to access content is therefore also limited. In addition, greater transparency would also help Google disprove or debunk growing claims of political bias.

Although Google’s search curation and ranking process is conducted primarily using algorithms, humans still play a role in this process. According to Google, the platform does not remove or delist search results. Rather, it seeks to promote higher quality content in the rankings over lower quality search results. The company has asserted that in a few rare exceptions they intervene manually in order to remove or delist content from search results. These cases include when the platform receives legal requests to remove or delist search results, when a web page violates Google’s webmaster guidelines, and when a webmaster of a page requests that their web page be removed or delisted.⁷⁰ Although Google states that they do not frequently engage in these practices, government and legal efforts to remove content have increased globally.⁷¹

Google provides transparency and accountability around its search result curation practices through its transparency report, which is published twice a year. In this report, Google reports on government requests to remove content across all its products, content delistings due to copyright, and requests to delist content under European privacy law (known as the “right to be forgotten”). Its data on government requests to remove content can be broken down by product, enabling for a greater understanding of how such requests impact the Search product in particular. The report includes metrics such as removal requests by the numbers, items specified by the requests, and a percentage breakdown of which products were affected. It also includes reasons for requests.⁷² This is a best practice that other platforms should adopt, as it provides transparency around how external parties are influencing search results, and how Google is responding. Google has also asserted that where possible it aims to inform website owners about requests for removal through its Webmaster Console. This is a vital portion of accountability as well.⁷³

Additionally, some research indicates that Google has intervened to manually alter search results when the results sparked controversy. For example, in December 2016 British Investigative Journalist Carole Cadwalladr wrote about how the top search result for “did the Holocaust actually happen?” in Google Search was a white nationalist web page which denied that the Holocaust had ever happened. Cadwalladr’s finding sparked outrage and eventually the results changed. Although Google has stated that it prefers “to take a scalable algorithmic approach to fix problems” rather than “fix the results of an individual query by hand,” many suspect that the company took corrective action nonetheless. However, others have suggested that the online outrage sparked by Cadwalladr’s article also contributed to this change in the results, as it drove traffic to certain web pages on this topic, thus impacting the search results and the ranking of these results.⁷⁴

Another way that humans play a role in Google’s search curation and ranking process is through the Search Quality Rating process. In order to ensure its search algorithms promote relevant and high-quality content, Google developed a rigorous testing process that involves live tests and review by thousands of trained external search quality raters around the world. This process is deployed every time Google considers implementing a change or update to its search algorithm. In order to roll out a change, Google must be able to determine that the change provides a net positive. This means that a significant amount of search results will be made more helpful without subsequently creating major losses in other areas. Making such changes to organic search results can take a large amount of time.⁷⁵

Search quality raters are external individuals based around the globe who help evaluate whether a website provides users with the content they were looking for. They also help evaluate the quality of search results based on the expertise, authoritativeness, and trustworthiness of the content. Each search quality rater represents a specific language and geographic expertise or perspective, in order to ensure Google’s Search product is useful around the world. Although these individual’s ratings do not directly impact the ranking of any web page, they enable Google to benchmark the quality of their results and identify areas for improvement.⁷⁶ This informs Google’s search algorithms, which aim to prioritize high-quality content and web pages.⁷⁷

In order to ensure that search quality raters are using a consistent approach, Google provides them with Search Quality Rater Guidelines, which outline Google’s goals for its ranking systems and includes examples for appropriate ratings. According to Google, in order to ensure consistency in the rating program globally, all search quality raters are required to pass a comprehensive exam and are continuously audited. Evaluators also assess each improvement to Search that is rolled out through side-by-side experiments in which evaluators see two different sets of search results, one with the change and one without. Evaluators must then identify which experience is of greater relevance and quality. This feedback is used to improve Search and launch efforts.⁷⁸ These search quality rater guidelines are publicly available, therefore providing a degree of transparency around these curation and rating efforts. However, not as much information is known about who these raters are and what perspectives, regions, and cultures they represent. Knowing this would enable researchers and users to get a better sense of how search results are being curated and rated.

Given that Google’s search engine is a vital way for website publishers to disseminate their information and gain traction, many publishers have invested significant time and resources into ensuring that they rank well in Google’s search results, a practice known as SEO. Google’s publicly available webmaster guidelines outline how publishers can ensure that Google finds, indexes, and ranks their website. It also provides guidelines on topics such as quality, as well as rules around prohibited or illicit activity such as spam, malware, and deceptive websites.⁷⁹

When a website violates Google’s webmaster guidelines, it can be penalized in one of three ways: Google can penalize the website, thus neutralizing the impact of the spam; demote the website; or remove the website from search results completely.⁸⁰ According to Danny Sullivan, Google’s algorithms can detect the majority of spam and automatically prevent the ranking system from promoting such content by demoting or removing it.⁸¹ The remainder of spam results are typically manually addressed by a spam removal team. They review the pages in question, typically based on user feedback, and flag them for penalty if they have been found to violate the webmaster guidelines.⁸² Manual actions can be used to penalize an entire website, subdomain, sections of a website, or specific pages. Manual action can also demote websites in search rankings and delist them.⁸³ If a web page owner feels that they have incorrectly or unfairly been penalized, they can submit a reconsideration request.⁸⁴ However, processing and responding to these requests often takes a significant amount of time, and this therefore can undermine the operations and success of a website for an extensive period of time. In order to provide greater transparency and accountability around this process, Google should enable appeals in a more timely manner.

Finally, in order to provide greater transparency and accountability around its search curation and ranking practices, Google needs to provide its users with greater controls around how their data is used and how their search experience is tailored.

In 2009, Google made Search the default search option for all users, including users who are not logged into a Google account.⁸⁵ Users who were signed in, however, could access a tab which outlined how Google had customized their search results, and how they could turn it off.⁸⁶ Today, a similar tab exists that explains to users how activity data, location data, and data from other Google products and services make Search work. The tab also lets users delete their search activity; choose whether their activity is saved on Google sites, apps, and services; and whether users would like personalized advertisements. Logged-in users therefore do have a range of controls over algorithmic curation available that enable them to disable the personalization of search results based on their account activity to a certain extent. However, as the DuckDuckGo study indicated, search results are often still personalized, even if a user is not logged in or if a user is using Incognito mode.⁸⁷ All users, regardless of whether or not they are logged in to a Google account or browsing in Incognito mode need to be able to access controls that enable them to opt out of algorithmic content curation and ranking during the search experience. They also need to have strong privacy controls over how their data is tracked, collected, and used. Additionally, similar controls need to be afforded to users who use Search but do not have a Google account, as they do not have access to the suite of settings that Search users with accounts do.

In terms of controls, Google also offers website owners a series of controls over how their website appears in Search results. They do this through the Webmaster Tools feature, which lets website owners provide granular instructions on how Google crawls and processes pages on their website. Website owners can also request a recrawl or opt out of crawling altogether.⁸⁸ This enables website owners to control how their content is processed to an extent as well. However, the algorithm eventually determines how well a website ranks and performs for each user.

Bing is an internet search engine that is owned and operated by Microsoft. It launched in June 2009⁸⁹ and enables a variety of search services including web, video, image, and map search.⁹⁰ When Bing launched, Microsoft sought to position it as more than a simple search service. Rather, the company claimed to provide a product that enabled consumers to rapidly acquire more relevant and informed insights from the web, and to use these insights effectively. Their marketing described this ideal search engine as a “Decision Engine.”⁹¹ Initially, Bing focused on four verticals: making a purchase decision, planning a trip, researching a health condition, and finding a local business.⁹² As with Google, by providing users with more relevant and informed search results, Microsoft also hoped to boost its growth and revenue.

Although Bing accounts for a small percentage of Microsoft’s overall revenue,⁹³ the search engine has grown to be the second largest in the world in terms of market share,⁹⁴ boasting 1.3 billion unique monthly global visitors⁹⁵ and ranking 31st for global internet engagement on Alexa rankings.⁹⁶ Although Google still dominates the search engine industry in terms of market share,⁹⁷ Bing is often considered the most comparable alternative available to users.⁹⁸

Bing has its own web crawler, known as Bingbot, which uses an algorithm to determine which websites to crawl, how often to crawl, and how many web pages to gather from each website. The algorithm chooses web pages to crawl by prioritizing relevant known URLs that are not indexed yet, and URLs that are already indexed but that need to be revalidated to check for changes or dead links. Bingbot also seeks to identify new web pages that have not been crawled or indexed yet.⁹⁹

Like Google, Bing uses a combination of algorithmic signals when ranking search engine results and uses human editors.¹⁰⁰ However, Microsoft has not recently made any disclosures about which signals it uses to rank search engine results. The latest major disclosure it made was in 2014, via a blog post on the role of content quality in Bing search results. The blog post outlined that the relevance of a result is a significant consideration for the Bing ranking algorithm. The relevance of a result is a function of three things: topical relevance to a user’s query (does the result sufficiently address the query?), content quality, and context (is the query related to a recent topic?, where the user is located?, etc.).¹⁰¹ Content quality is based on three primary pillars: authority (can the content be trusted?), utility (is the content useful and detailed?), and presentation (is the content well-formatted, accessible, and easy to find?). Authority is determined based on a range of factors including signals from social networking platforms, cited sources, name recognition, and information about the author. In order to assess the utility of a website, Bing’s models aim to predict whether the page’s content provides adequate supporting information, whether it is detailed enough for the intended user, and whether it includes supporting content such as videos, graphs, etc. The models also consider the level of expertise required to produce the content on the web page, with a preference for content that is unique and does not reproduce existing materials.¹⁰²

Microsoft has also stated that both the signals and human editors account for live search activity and real-time news events when ranking news results.¹⁰³ In addition, the search engine only uses metrics related to how many clicks a web page gets when it evaluates how a search ranking algorithm is working. Click metrics are not a primary signal that is considered when ranking search engine results.¹⁰⁴

According to Microsoft, the Bing search engine was constructed to identify which results most satisfied users. Based on these insights, Microsoft develops guidelines and training datasets. These training datasets are evaluated by search quality raters (known as “judges” for the Bing search engine) who operate in Bing’s Human Relevance System project.¹⁰⁵ These judges work to identify which search results are the most satisfying according to factors such as relevance and accuracy. They also use click metrics to evaluate whether users are satisfied with the search results they received.¹⁰⁶ In addition, Microsoft pushes out models to subsets of users in order to observe which results most satisfy real users. It follows a similar process when implementing updates to its ranking algorithm.¹⁰⁷

The Bing search engine also uses machine learning to scale the process of generalizing. Generalizing is when Bing judges¹⁰⁸ manually rank search results using a set of guidelines and is typically most accurate when it is performed by humans. However, because it is conducted manually, it cannot be done at scale. The use of machine learning in this instance therefore aims to provide users with search results comparable to those that judges would deliver, but at scale. This can only be achieved by generalizing the ranking algorithm as much as possible.¹⁰⁹ Today, approximately 90 percent of Bing search results are ranked based on machine learning.¹¹⁰

In addition, like other search engine’s ranking algorithms, Bing’s search ranking algorithms are continuously updated to attempt to provide users with a better experience.¹¹¹ These updates aim to refine and improve search results and remove spam from search indexes to protect users from negative or manipulative search results. The search engine uses algorithms to demote and penalize websites that violate Microsoft’s guidelines.¹¹² As a result, it observes how users react to search results, how the search engine assesses the search results, and what the actual ranking algorithm returns. This process cannot be perfect and the search engine algorithm therefore has to continuously be updated.¹¹³

Bing is the second largest search engine in terms of market share and it is therefore responsible for curating a significant amount of content for a large user base. Despite this, it demonstrates a lack of transparency around its search result curation and ranking process.

First, Microsoft does not publicly disclose substantial information around the signals that define how search engine results are curated and ranked. This makes it difficult for users and website owners to know how the search experience is developed and personalized, and which qualities, results, and voices are prioritized over others. In addition, Microsoft does not regularly share explanations of why its search algorithm is being updated. Given that these changes impact how and if publishers’ content is viewed by users, and given that these changes alter the scope of content users engage with, this is concerning.¹¹⁴ Microsoft does offer website owners some information on how to use the Bing search engine via its publicly available Webmaster Guidelines. These guidelines outline how providers can work to ensure their content is found and indexed within Bing.¹¹⁵ But, this resource does not offer live and up-to-date information on recent changes.¹¹⁶

Furthermore, although Microsoft deploys judges to help improve its search experience, it does not publicly share the guidelines these judges use.¹¹⁷ Some organizations, such as online blog Search Engine Land, appeared to be able to obtain copies of these guidelines and have written about them, but Microsoft itself has not publicly disclosed information around these efforts. Microsoft also does not share any information about who these judges are, and what perspectives, regions, and backgrounds they represent. This makes it difficult to understand how search results are benchmarked and curated, and which voices and perspectives are being considered when producing the search experience. In addition, according to a spokesperson, Microsoft deploys both algorithmic tools and human editors during the search curation and ranking process. Microsoft does not, however, provide any further information on the role of these human editors, how they differ or are similar to judges, and what role they play. The presence of human editing in this process creates the very real opportunity to instill bias in the search ranking process. Although, as previously noted, the term “bias” in this context does not solely refer to inappropriate preferences based on protected categories like race or political affiliation, this still raises concerns. Algorithmic biases, which can originate from their creators and from biased training data, are also a significant concern. However, given that Microsoft shares limited information about its algorithmic curation and ranking process, it is difficult to assess what these biases are and how they impact the search experience.

Like Google, Microsoft lets users control their search experience on Bing to an extent. When a Bing user is logged in to their Microsoft account, they can view and clear their browsing, search, location and other relevant activity history from the account. They can also manage and control some of the data that Microsoft collects. However, these controls are not available to users who are searching in Bing in a private browsing mode, or who are searching while not logged in.

In addition, Microsoft also demonstrates some concerning practices when it comes to accountability with the Bing search engine. According to Frédéric Dubut, the head of Bing’s spam team, Microsoft aims to assess intent when deciding whether to penalize a website for violating its guidelines (such as by spamming).¹¹⁸ However, how the company assesses intent is unclear. As previously highlighted, a web page can be penalized in a number of ways. These include neutralizing the impact of spam or negative intent, demoting a website in search rankings, or removing the website from search results.¹¹⁹ A publisher can submit a reconsideration request if they believe their website has been unfairly penalized. This is valuable, as it provides an appeals mechanism for users of Bing’s search engine. However, this process has been described as lengthy, thus raising the risk that an error on the part of Microsoft can seriously damage the success of a website.¹²⁰ In order to provide greater accountability around their search result ranking procedure, Microsoft should improve this procedure so that it generates resolutions in a timely manner.

Microsoft does, however, also demonstrate some positive practices. For example, it lets website owners request recrawls of their sites.¹²¹ Additionally, like Google, Microsoft receives various legal, copyright, and private party requests to remove and delist websites from Bing.¹²² The company issues an annual transparency report regarding content removal requests, which outlines the scope and scale of such requests. The report provides data on government requests for content removal, copyright removal requests, “Right to be forgotten” requests, and non-consensual pornography (“revenge porn”) removal requests. The report, however, does not break down these data points by Microsoft product, and as a result it is difficult to ascertain how often search results on Bing are impacted by these requests.¹²³

DuckDuckGo is an internet search engine that launched in 2008, largely based on Free and Open Source Software (FOSS).¹²⁴ Typically, search engines aim to distinguish themselves based on the comprehensiveness and accuracy of their search index, and the relevance of their search results.¹²⁵ DuckDuckGo seeks to further distinguish itself by providing users with strong privacy protections that also enable them to evade the so-called filter bubble created by the personalization of search results. According to DuckDuckGo, the platform does not profile users and delivers all users the same search results, regardless of their past search history. The platform also asserts that it prioritizes providing users with the highest-quality search results, rather than the largest number of results. Today, DuckDuckGo is the sixth largest search engine in the world by market share.¹²⁶ As of January 2019, DuckDuckGo achieved a new record in traffic with over 1 billion monthly searches.¹²⁷ It ranks 186th for global internet engagement on Alexa rankings.¹²⁸ DuckDuckGo uses its own web crawler, known as DuckDuckBot, and approximately 400 other sources in order to generate and curate its search results. These sources include other search engines, such as Bing, Yahoo!, and Yandex, and websites, such as Wikipedia.¹²⁹ As concerns around consumer privacy have grown—especially following the 2013 Snowden disclosures and more recent data-sharing controversies like the Cambridge Analytica scandal—DuckDuckGo has seen a significant increase in its user base and website traffic.¹³⁰

According to DuckDuckGo, the platform enforces encrypted HTTPS connections whenever websites provide them. When a user connects to an HTTPS-secured server, secure websites that offer encrypted HTTPS connections will evaluate a website’s security certificate and authenticate that it was issued by a legitimate authority. This helps secure sensitive information sent over an HTTPS connection from electronic eavesdropping.¹³¹ Additionally, it can prevent the information from being modified while in transit. DuckDuckGo also assigns each page a user visits a score that assesses to what extent that website is trying to mine the user’s data. In order to maintain user anonymity online, DuckDuckGo asserts that it blocks tracking cookies, which can be used to identify a user and their devices. It also scans and scores the privacy policies of different websites that a user visits. On the DuckDuckGo search engine, a user has the ability to clear their tabs and data automatically, at the end of a session or after a preset period of inactivity.¹³² In addition, although the company still provides advertisements, these are “contextual” ads that are based only on the content of a website (such as a current search query) rather than on a user’s behavioral profile, including their prior search history.¹³³ Further, DuckDuckGo has stated that it does not store personal user information; it does, however, maintain a log of all search terms that have been used on its service.¹³⁴

Despite the differences in DuckDuckGo’s practices, some studies have indicated that DuckDuckGo is able to return results with the same level of quality as Google.¹³⁵ However, this is generally accurate for broad topic search results, rather than niche topics.

Although DuckDuckGo seeks to provide users with a positive search experience that comes with strong privacy protections, the company does not provide a significant amount of transparency and accountability around its search curation and ranking practices. According to the DuckDuckGo research study conducted on Google’s ranking practices, a neutral search engine that is truly delivering non-personalized search results should be able to deliver the same results to all users, regardless of what browsing mode they are in. However, just because all users on the DuckDuckGo platform see the same results, it does not mean that these results are not curated and ranked using automated tools. The company does not share which signals it uses to perform this ranking. On the DuckDuckGo website, the company states “ranking is a bit opaque and difficult to discern/communicate on an individual query basis because of all the various factors involved (and which change frequently). Nevertheless, the best way to get good rankings (in nearly all search engines) is to get links from high-quality sites.”¹³⁶ It is therefore difficult to understand which factors DuckDuckGo’s search curation and ranking processes prioritize, and how these judgments impact users’ search experiences on the platform. By providing greater transparency, the platform could enhance its value proposition and demonstrate to users why they argue they are a better search engine choice.

One way the platform aims to deliver high-quality search results is by removing search results associated with content mill companies. Content mill companies are websites that publish numerous daily articles which are often produced by freelance writers (for example, eHow). These forms of content are considered low-quality, but they are written so they can rank highly in Google’s search index. DuckDuckGo, however, removes them.¹³⁷ The search engine has also begun experimenting with algorithms to remove spam links and other forms of low quality content. However, DuckDuckGo provides little transparency around the scope and scale of this process.

According to DuckDuckGo, the platform’s search engine supports user privacy, provides users greater protections around how their data is used, and aims to deliver neutral search results that do not exhibit bias and that prevent the creation of filter bubbles. However, the company does not provide adequate transparency and accountability around its ranking process, making it difficult for users and website owners to understand how expression is being controlled. The growth of the platform suggests there is a market for a service whose value proposition is built on protecting user privacy.¹³⁸ This value proposition should be extended to include transparency and accountability.

Search engines are not the only internet platforms that have adopted algorithmic curation and ranking practices. Many small and large online platforms, including social networking and review-based services, have also introduced these tools to curate and present content that is relevant to their interests and needs. The creation of these news feeds has opened up new avenues for users, businesses, brands, and content creators to create and disseminate information at great scale. They have therefore also helped to promote free expression online and boost the revenue of internet platforms. However, these methods of algorithmic curation and ranking also raise a number of concerns, particularly regarding algorithmic awareness and transparency, the creation of filter bubbles, and algorithmic bias. Like search engines, news feeds can shape user perspectives by prioritizing certain forms of content and deprioritizing others.

This section of the report focuses on how internet platforms deploy algorithmic curation and ranking practices in order to shape and operate their news feeds. It will use three platforms —Facebook, Twitter, and Reddit—as case studies of how such practices can vary and what concerns they surface.

Since its creation in 2004, social media company Facebook has expanded its services to include features such as messaging and “smart displays.” As of June 2019, Facebook has approximately 1.59 billion daily active users¹³⁹ and ranks third in global internet engagement on Alexa rankings.¹⁴⁰

Today, Facebook offers one of the clearest examples of algorithmic content curation and ranking in a news feed. The Facebook News Feed is composed of stories produced by a user’s friends, Pages a user follows, Groups a user is part of, and suggested content such as stories and advertisements. Facebook launched the first iteration of its News Feed in 2006.¹⁴¹ The launch of the News Feed drew significant backlash and controversy from users, even sparking calls to boycott the platform. In particular, users were concerned that the News Feed was eroding their privacy by making more information about them available to their friends and others on the platform, which Facebook asserts was not the case.¹⁴²

Despite controversy around this major change, the News Feed has emerged as one of the largest and most significant billboards and content hubs for users, brands, publishers, and influencers.¹⁴³ It has also been a major driver of advertising, generating a significant amount of revenue for the platform.¹⁴⁴ The News Feed was launched to drive further engagement—and thus revenue—on the platform; Facebook asserts it was also designed to present users with content that is relevant and meaningful to them. Prior to the launch of the News Feed, users on Facebook and comparable platforms like MySpace and Friendster had to seek out content posted by their friends or Pages by visiting individual pages. The News Feed brought this information together and curated posts based on predictions of what content users were interested in and would engage with. It ranked these posts so that users could view the content that was deemed the most relevant to them first. According to Facebook CEO Mark Zuckerberg, the average Facebook user has approximately 1,500 posts that could appear on their News Feed every day. However, users spend a limited amount of time on their News Feed, and as a result they are likely to only read and potentially engage with 100 of these posts. Facebook states that the News Feed curation process seeks to ensure that these 100 posts are the most relevant and meaningful to a user.¹⁴⁵

However, in its efforts to provide users with more meaningful interactions, Facebook is also aiming to increase the time users spend on the platform. This in turn drives increased revenue through avenues such as advertising, furthering the company’s bottom line. Additionally, as Facebook has come under increased scrutiny since the 2016 U.S. presidential elections and the Cambridge Analytica scandal, it has invested significant resources towards convincing regulators that it should be able to continue to self-regulate, rather than address the obvious threats its business model poses to user privacy and more fundamentally for democracy. Facebook’s assertions that the platform now prioritizes meaningful interactions is one such method of convincing regulators that Facebook is a safe and well-meaning platform.

An earlier iteration of the Facebook News Feed deployed an algorithm known as EdgeRank. EdgeRank was used to determine which stories should appear in a user’s News Feed based on three signals: affinity score, edge weight, and time decay.¹⁴⁶ Posts that had the highest EdgeRank score would appear at the top of a user’s News Feed. Because each user had a different affinity score, each user also had a different EdgeRank score. These scores were not public. Around 2011, as the News Feed algorithm evolved, EdgeRank was retired and the signals it relied on were incorporated into newer versions of the News Feed algorithmic system.¹⁴⁷

During this time, the Facebook News Feed garnered a reputation for looking and operating like a tabloid, since it heavily prioritized popular posts and advertisements. Additionally, the platform came under heavy criticism for promoting addictive practices that sought to maximize the amount of time users spent on the platform. This was because, at the time, how much time users spent on a platform was indicative of a platform’s popularity and success.¹⁴⁸

In January 2018, however, Zuckerberg announced that the News Feed was going to be altered so that it prioritized “more meaningful social interactions” with friends and family over content produced by businesses and brands.¹⁴⁹ Facebook stated that this shift would place less emphasis on posts that are popular and rather place more of an emphasis on “authentic” posts that encourage and receive significant engagement from a user and their network.¹⁵⁰ The new News Feed is reportedly based on three core principles. The first is that Facebook users value meaningful and informative stories, the second is that they value accurate and authentic content, and the third is that they value principles that guide safe and respectful behavior.¹⁵¹

With this new News Feed, Facebook claimed that it hoped users would spend quality time on the platform, rather than more time.¹⁵² However, it could be inferred that if a user is engaging with more meaningful and relevant content on the platform, they would also spend more time on the platform.¹⁵³ One year after the new News Feed algorithm was launched, a report by social media engagement tracking firm NewsWhip found that the platform had seen increased levels of engagement as well as greater amounts of content being posted and engaged with by friends and family.¹⁵⁴

The Facebook News Feed algorithm goes through four stages in order to identify stories, rank them, and produce a tailored News Feed experience for each of its users.¹⁵⁵

1. Inventory: The algorithm takes an inventory of what stories have been posted by a user’s friends and Pages a user follows. This is important to assess, as each News Feed is largely composed of content shared by a user’s connections.
2. Signals that inform ranking: The algorithm then evaluates each story using hundreds of thousands of signals. These signals include who posted a story and when it was posted, as well as more granular factors such as the time of day, and how fast a user’s internet connection is. This is particularly important for users with slower connections who can’t properly load certain forms of content.
3. Predictions: The News Feed algorithm utilizes machine learning in order to extract insights from a user’s past activity. These insights are used to predict how likely a user is to engage with a post, which is a metric for whether the user finds a post meaningful.¹⁵⁶ Some of the predictions the algorithm seeks to make include how likely a user is to comment on a story, how likely they are to spend time reading the story, and whether they would watch an entire video. It also makes some qualitative predictions such as how likely a user is to say that they found a story informative. While such predictions may be able to deliver relevant content to users in the short term, it presumes that a user’s behavior and interests will remain constant over time. It therefore can also result in the creation of a filter bubble and prevent users from engaging with new content that matches their potentially expanding interests.
4. Relevancy score: The News Feed algorithm then uses all of the signals and insights at its disposal to calculate a relevancy score. These signals are used to calculate a range of probabilities, including the likelihood a user clicks on a story, the likelihood a user is to spend time on a story, the likelihood a user engages with a story through likes, comments, and shares, the likelihood a user will find a story informative, the likelihood a story is click-bait (posts that aggressively seek out likes and engagement), and the likelihood that a story links to a low-quality website. These predictions are compiled into a relevancy score, which is an overall prediction on how meaningful a given story is for a user. Facebook calculates a relevancy score for every story and all of a user’s connections every time a user opens their News Feed.

Facebook’s News Feed algorithm is influenced by hundreds of thousands of signals that work to identify and rank content for each user. According to Facebook, these signals seek to prioritize content that reflects the three News Feed pillars, and they each represent a distinct data point that Facebook’s News Feed algorithm considers and processes when ranking content. These signals can be explicit, such as likes, or implicit, such as the time a user spends on a page before returning to the News Feed.¹⁵⁷ A study on Facebook’s News Feed Algorithm, based on information reflected in Facebook’s “News Feed FYI” blog, sorted some publicly disclosed signals into six categories:¹⁵⁸

1. Content signals: Content signals are factors that demonstrate how stories differ from one another. They include the format of a story (such as a link, video, or photo); the number of likes, comments, or reactions a story receives; and which friend or Page posted the story.
2. Source signals: Source signals are characteristics a user or Page demonstrates when they publish a post. These include the history of the Page and how often a Page has posted stories with click-bait headlines.
3. Audience signals: Audience signals are characteristics a user demonstrates when they consume a post, and they often reflect patterns in content consumption. These include how often a user uses the “hide” feature to remove content from their News Feed and how often a user watches videos, in part or in their entirety, rather than scrolling past them.
4. Action signals: Action signals represent a user’s behavior when it comes to a specific story. These signals include whether a user likes, clicks on, or engages with a specific story, and the amount of time a user spends reading a story or watching a video. The News Feed algorithm prioritizes active interactions, such as commenting and sharing a post, rather than passive interactions such as liking posts and click-throughs. This is based on the notion that active interactions require more effort and are therefore indicative of meaningful interactions.¹⁵⁹ The algorithm also tends to favor posts with comments and replies to comments, as they indicate meaningful interactions via conversations.¹⁶⁰
5. Relationship signals: Relationship signals are data points collected about the relationship between two users or Pages on the platform. These include how often two users engage with one another, and whether a user decides to unfollow a friend, Page, or Group.
6. Likelihood signals: Likelihood signals are the probabilities the Facebook News Feed algorithm calculates around how a user will interact with a post. They include the probability a user will like or comment on a story. These likelihood signals are compiled to determine how posts are ranked in the News Feed.

Facebook also provides temporary boosts to content known as “timely posts”. These are popular posts or news that are currently being discussed.¹⁶¹

According to Facebook, the platform also sought to prioritize “meaningful interactions” so that it could promote high-quality content in News Feeds and curb the spread of low-quality content such as spam, posts that are unverified, click-bait posts, and posts that seek to spread misinformation.¹⁶² However, as previously mentioned, such efforts also aim to increase the amount of time that users spend on the platform, and thus drive revenue through avenues such as advertising.

Most recently in July 2019, Facebook announced they would downrank and reduce the spread of posts that made sensationalized health claims. They would also do this for Pages aiming to sell products or services that were based on misrepresented or false health-related claims.¹⁶³ Additionally, in April 2019, Facebook deployed a new tactic, called Click-Gap, in order to reduce the amount of low-quality content that users see in their News Feed. A number of low-quality websites receive significant traffic from the Facebook platform. In order to tackle this, Facebook systems crawl and index the internet in order to identify such websites. They then downrank low-quality posts that link to these websites. This is based on the notion that such sites are relying on platforms like Facebook to drive views, and by doing this Facebook can stifle their efforts. This approach is similar to the approach Google’s PageRank algorithm used to rank results when it first launched. PageRank determined how high to rank a search result based on the number of websites and the quality of websites that linked to a given web page.¹⁶⁴ Because visibility drives impact, posts that are viewed less have less of an impact.

Facebook also uses a range of user-driven metrics in order to assess the quality of a post. These include whether users hide certain posts, and whether users report posts as spam in order to assess the quality of posts. According to Facebook, in this system, posts that are shared and engaged organically will rank higher on user’s News Feeds.¹⁶⁵ However, the same March 2019 NewsWhip report that found that the algorithmic changes to the News Feed had increased engagement also found that the changes had not succeeded in adequately tackling the spread of misinformation and low-quality content. ¹⁶⁶

Facebook has asserted that its current algorithmic curation and ranking model prioritizes meaningful interactions on the platform. However, this algorithmic system is also responsible for managing information flows and online speech, and presenting users with a certain experience based on the platform’s understanding of a user’s interests. This creates an opportunity to promote certain voices above others, and silence certain voices entirely. This may mean that the voices of dominant social and political groups will be amplified and the voices of disproportionately targeted and already marginalized groups will be silenced. These algorithmic practices are also directly responsible for influencing how users perceive their network and the world around them. As a result, greater transparency and accountability around Facebook’s News Feed algorithmic curation and ranking practices are needed.

In March 2019, Facebook introduced a “Why am I seeing this post?” feature in the News Feed. This feature, which is found in the right hand corner of each News Feed post, explains to users why the user is seeing a certain post (e.g. if it was posted by a friend or a Page the user follows, whether it is highly popular,¹⁶⁷ etc.), how their past activity (such as whether they regularly watch videos or click on shared links)¹⁶⁸ has informed the ranking of the posts in their News Feed, and what other factors typically influence the ranking of posts in the News Feed. This feature also provides users with access to controls that let them edit their News Feed and privacy preferences.¹⁶⁹ The News Feed preference controls enable users to select whose posts they see first, unfollow users or groups in order to hide their posts, reconnect with users or groups in order to see their posts again, manage snooze settings on certain users or groups, and hide apps from the News Feed. In addition, through another News Feed control tab on the left hand side of the News Feed page, users can opt to view posts in a reverse-chronological order, rather than through the lens of the algorithmic curation and ranking filters. The default setting that the News Feed will always revert to, however, is the algorithmically curated and ranking mode, known as “Top Stories.” In this way, Facebook enables users to control their News Feed experience to a degree. However, it does not give them the option to opt out of the algorithmically personalized experience entirely, and the algorithmically personalized News Feed is the default option.

Although Facebook has shared some information about the signals it uses to rank content in a user’s News Feed, it has not provided a comprehensive overview of the range of signals used, and how these signals collectively work together in the News Feed algorithm to determine the ranking of posts. It also does not explain how different signals are weighted.¹⁷⁰ Without greater transparency and accountability around how the platform is deploying these signals in their algorithmic curation and ranking practices, and around how these signals work together, users and publishers are unable to properly understand how their experience is being curated and how this can impact their worldview. They are also unable to understand exactly which characteristics of a post, user, or their network are prioritized during this curation and ranking process. This raises concerns that this algorithmic system can establish filter bubbles on the platform.

In addition, individual voices or communities that are suppressed by the algorithm are often left unable to understand why. This raises a number of concerns regarding algorithmic bias, and the extent to which algorithms reflect and exacerbate the judgments, priorities, and preferences of their creators and society at large. Given the limited set of user controls over the News Feed, impacted users are unable to effectively mitigate this situation. However, it is difficult to imagine a set of user controls that could effectively mitigate the issue of algorithmic bias. Given the black box nature of much algorithmic decision-making, developers and users may not be aware of any systematic biases in an algorithm.

Facebook does not currently offer an appeals process or channel for its News Feed curation and ranking efforts. Such a process could help remedy individual cases, even though it would not help to remedy systemic instances of bias.

Twitter is a microblogging and social network platform founded in 2006. It enables users to post messages known as “tweets” and has gained a reputation as a destination for live updates regarding news, politics, sports, and other current events. As of February 2019, the platform had 126 million daily users and¹⁷¹ it ranks 20th for global internet engagement on Alexa rankings.¹⁷²

Originally, Twitter’s news feed—known as a “timeline”—was not algorithmically curated. Rather, content on a user’s timeline was presented in reverse-chronological order. In 2015, Twitter introduced a feature known as “While you were away”¹⁷³ (later rebranded as “In case you missed it”),¹⁷⁴ which aimed to curate notable recent tweets that a user may have missed while they were not using the platform.¹⁷⁵ In 2016, Twitter introduced algorithmic curation into its timeline. This was an extension of the “While you were away” feature,¹⁷⁶ as it used the same algorithms. According to Twitter, it was designed to deliver users with the most relevant and useful tweets, rather than the most recent ones.¹⁷⁷ Twitter has asserted that both of these features were based on the notion that a tweet from a few hours ago may be more relevant and meaningful to a user than one that was posted five minutes ago. When using the reverse-chronological curation format, a user would miss out on such content.¹⁷⁸ As per the new timeline feature, algorithmically curated tweets appeared at the top of a user’s timeline. However, these algorithmically curated tweets are a small subset of the tweets that have been posted since a user last visited the platform.¹⁷⁹ As a result, if a user continued to scroll through the timeline, they would eventually begin seeing tweets in a reverse-chronological format.¹⁸⁰

Twitter’s decision to roll out an algorithmically curated timeline was met with some backlash. Many users were concerned that by introducing this feature, the platform was stifling the public square characteristics of the platform, as what content was relevant and meaningful would now be determined by an algorithm. Some critics of an algorithmically curated timeline have advocated for a reverse-chronological feed, as this is perceived as a neutral presentation of content. In a reverse-chronological timeline, hashtags would play a strong role in promoting and highlighting conversations and virality would be organic.¹⁸¹ In this sense, critics contend, Twitter could offer a democratic public square on its platform.¹⁸² Although the timeline algorithm could be used to surface content that is broadly considered relevant (such as headline news), it cannot reliably surface unexpected and diverse content that is also relevant, like an organically run public square platform could, as it makes judgments based on past user behavior.¹⁸³ The introduction of algorithmic curation in the timeline also faced backlash as it made users feel as if they had less control over their experience on the platform, and raised concerns over the creation of filter bubbles.¹⁸⁴

These concerns sparked the hashtag #RIPTwitter in early 2016.¹⁸⁵ When the new timeline feature rolled out, users had the option to opt out in a limited sense. They could choose to not see Top Tweets at the top of their timeline, but they would still receive curated tweets in other sections of their timeline.¹⁸⁶ In response to the outcry, in September 2018, Twitter enabled users to toggle between algorithmically-curated Top Tweets and non-curated, reverse-chronological ordered Latest Tweets.¹⁸⁷ Despite the public backlash, fewer than 2 percent of users opted out of algorithmic curation on the platform, which became the default option.¹⁸⁸ However, the fact that only 2 percent of users opted out of algorithmic curation does not necessarily mean that users were not opposed to algorithmic curation. Rather, because algorithmic curation became the default, opting out required an extra step, and many users may not have wanted to engage in a more time consuming process or known how to do so.

Despite the controversy, Twitter has insisted that its research indicates individuals have a more positive experience on the platform when engaging with Top Tweets first.¹⁸⁹ In a test on the beta version of the algorithmically curated timeline, which was performed on over 100 users and brands, the platform found that individuals tweeted and retweeted more often than they did when using a non-algorithmically curated timeline.¹⁹⁰ This, however, assumes that greater engagement is synonymous with a positive user experience. This is not necessarily true. However, greater engagement does drive greater revenue for the platform, which may be a reason it advocated strongly for the algorithmically curated version of the timeline.

The Twitter timeline can consist of numerous sections.¹⁹¹

Top Tweets: The Top Tweets on a user’s timeline are algorithmically curated and ranked using a range of signals. This often also includes tweets from accounts that a user does not follow but may be interested in.

Latest tweets: If a user opts out of algorithmic curation, they will view a reverse-chronological feed of the latest tweets.

In case you missed it: If a user is visiting the Twitter app less frequently, they will see this algorithmically-curated selection of Top Tweets. A user typically only sees this feature in their timeline feed if they have not visited the platform for a number of hours or days. The tweets in this section are less recent, and do not appear in reverse-chronological order. Rather, they are organized based on their ranking scores. As a result, the tweet at the top of this section is the tweet that has the highest ranking score out of all possible tweets from every account a user follows since the last time they logged in.¹⁹²

Happening now: This section occasionally appears at the top of a user’s timeline and it highlights specific events or subjects of interest. This was originally introduced to focus on sports events and was later expanded to include breaking and personalized news.¹⁹³

Trends for you: This algorithmically-curated section highlights popular trends and hashtags based on a user’s interests (as explained below). Users can also choose to have this content curated based on their location.

When a user opens Twitter, the platform collects and assesses every recent tweet from every account that a user follows and assigns each one a relevance score. This score aims to predict what content a user will find interesting.¹⁹⁴ It is based on a range of factors, including the number of favorites and retweets a tweet has received, and how often the user has engaged with a particular account recently. Simultaneously, Twitter’s algorithm considers a range of other signals, such as how long a user has been away from the platform, how many accounts a user follows, and how a user behaves and uses Twitter, in order to determine how the relevance scores will impact the content on the user’s timeline.¹⁹⁵ Content is then ranked based on a series of signals which assess how popular a tweet is and how accounts in a user’s network are engaging with it.¹⁹⁶ These signals include:¹⁹⁷

Recency: How recently a tweet was posted.

Overall engagement: How many retweets, clicks, favorites, and impressions a tweet has garnered. This signal also considers how much time users have spent reading the tweet.¹⁹⁸

Engagement relative to other tweets from the same user: How often users engage with the posting user through active engagements and impressions.

Rich media: The type of media that the tweet includes such as images, videos, GIFS, and polls.

The types of media users typically engage with: If a user typically engages with a specific type of content, such as photos or videos, then they are more likely to see tweets that contain these media formats.

Account engagement and interactions: How often a user engages with a particular author or account, the strength of the user’s connection to this account, the origin of this relationship,¹⁹⁹ and how much time a user spends reading tweets posted by this author, even if they do not engage with them.²⁰⁰

Signals such as account interactions, engagement, user interest, network activity,²⁰¹ how long a user has been away from the site, how many followers an account has, and the account’s location relative to users also play a role in how content is curated and ranked on the Twitter timeline.²⁰² Today, deep learning is the central modelling component in timeline ranking.²⁰³

Like Facebook, Twitter asserts that it has updated its ranking algorithm in order to improve the “health” of conversations on its platform. In 2018, this was done in order to combat instances of trolling, harassment, and abuse. The new algorithmic system uses behavioral signals in order to assess whether a Twitter account is adding to or detracting from conversations, based on how other accounts react to content. For example, if a user sends the same message to multiple users and they all block or mute the sender, this will suggest that the sender is detracting from conversations. If the recipients reply to or “heart” the messages, however, this suggests that the sender is positively contributing with interactions. The algorithm also considers signals such as whether an account has a confirmed email address and whether an account appears to be leading a coordinated attack. Those tweets that are identified as detracting from conversation will be deprioritized in the timeline and will therefore appear lower in search results or replies.²⁰⁴

As demonstrated by the backlash against Twitter’s decision to implement an algorithmically curated and ranked timeline, users and publishers have expressed concerns over how such a system manages online expression and creates and reinforces certain perspectives based on what it thinks a user is interested in. Like Facebook, Twitter also fails to provide significant transparency and accountability around its algorithmic curation and ranking practices.

Twitter provides its users with a range of limited controls over their timeline experience. These include the ability to unfollow, mute, and block certain accounts. Users can also select the “show less often” feature, which provides feedback on certain tweets to Twitter so that it can better tailor the timeline experience in the future. In addition, users can opt in to or out of permitting Twitter to personalize their experience based on their “inferred identity” and places that a user has been. As previously mentioned, users also have the option of toggling between an algorithmically curated timeline and a reverse-chronological timeline.²⁰⁵

There is some public information around which signals Twitter uses to curate and rank content on the timeline. However, the company has not released a comprehensive overview of these signals, how they work together to curate and rank posts, and how they are weighted. Without greater transparency and accountability around how this algorithmic curation is taking place, users and publishers are unable to fully understand and control how their worldview is being shaped and what specific characteristics the Twitter timeline algorithm is designed to prioritize. This once again raises concerns regarding the creation and reinforcement of filter bubbles.

Furthermore, given that Twitter is often viewed as a digital public square, the platform’s algorithmic curation and ranking practices raise concerns regarding which voices the algorithm determines are important and worth amplifying, and whether these determinations reflect the same values and judgments that humans would place when assessing public discourse. A lack of transparency around the signals the algorithm uses makes evaluating how the timeline algorithm impacts public discourse even more difficult. Furthermore, a lack of transparency around how the timeline algorithm operates and is constructed also raises concerns around hidden biases in the algorithm and its signals. These biases prioritize certain types of interactions and types of content over others, and can reflect the unintentional biases of their creators or the training data with which they were created. Given the limited set of user controls over the timeline, and the fact that the platform does not offer an appeals process or channel related to its timeline curation and ranking practices, users who feel as if they have been silenced have no method for recourse.

Reddit is a social news aggregation and discussion website that was founded in 2005. The platform has approximately 330 million monthly active users worldwide²⁰⁶ and is ranked 16th for global internet engagement.²⁰⁷ Reddit enables users who operate under pseudonyms, to create subpages, called subreddits, on specific interests or topics. In this way, the platform has become popular among particular interest or activity-focused communities, such as gamers and sports fans. This represents a significant difference between Reddit on one hand, and Facebook and Twitter on the other. Unlike its counterparts, which emphasize bilateral or multilateral relationships in which users interact on a broad range of topics and interactions between users, Reddit emphasizes users’ participation in thematic forums, or subreddits, with tangible implications for its use of content-shaping algorithms.

Reddit deploys a series of algorithms in order to rank posts and comments on its home page feeds as well as on each individual subreddit. The code for these algorithms is open-source and available publicly online.²⁰⁸ This ranking system is largely influenced by the platform’s user-driven voting system.²⁰⁹ All Reddit users who are logged in can vote on links and comments in order to indicate their meaningfulness and usefulness. In this system, an upvote indicates that a user finds content interesting and relevant, and a downvote suggests that the user finds the content uninteresting, off-topic, or otherwise not meaningful.²¹⁰ Links and comments that receive a significant number of upvotes will appear higher on the website’s front page or on the front page of a given subreddit. Each link and comment on the platform are assigned a number of points, known as a score, which loosely corresponds to the difference in the number of upvotes it has received and the number of downvotes it has received. The exact calculations of this figure are kept hidden, however, in order to prevent spammers and other actors with negative intentions from gaming the system.²¹¹ On the platform, comments are by default sorted using the “best” comments filter. As a result, comments with the highest number of upvotes are likely to be viewed more often.²¹² In addition, the posts with a significant number of comments are also typically ranked higher than others. This suggests an element of democracy on the platform, as Reddit seeks to rank the content that users engage with—and therefore value the most—the highest. ²¹³

The score that a post or comment receives translates into “karma” points for the posting user. Karma is an informal user ranking on Reddit measuring how much users value a particular account’s contributions to the Reddit community.²¹⁴ A user who frequently contributes high-ranking posts or comments will build a high karma score denoting their total net-positive impact on the site. This system is not foolproof, however. A user can easily gain karma points by reposting popular content across multiple subreddits and by posting content that aligns with the general mentality and values of a certain subreddit or the platform as a whole.²¹⁵ Additionally, as a user gains more karma points, or as a post or comment gains more upvotes or downvotes, it can spark a bandwagon response in which other users vote in line with the general trend. In order to prevent this, some subreddits hide karma totals for certain periods. However, this is not a complete solution.²¹⁶

Reddit deploys different algorithmic approaches when ranking posts and comments. When a user logs in, they can choose to view content on their homepage feed using a range of algorithmically-curated options. These curation options sort content into categories: best, hot, new, controversial, top, and rising.²¹⁷ According to a 2015 blog post by Amir Salihefendic, the CEO of Doist who has conducted significant research on Reddit’s algorithmic ranking practices, posts on Reddit under the “hot” category are ranked using an algorithm known as the “hot ranking algorithm.” This algorithm is impacted by signals including:²¹⁸

1. Submission time: The time at which a post was submitted is a major factor influencing how a post ranks on Reddit. The hot ranking algorithm ranks new stories higher than older stories.
2. The logarithm scale: The hot ranking algorithm uses the logarithm function to weigh the earlier votes higher than later ones. This means that generally, the first ten upvotes that a post receives will have the same weight as the next 100 upvotes. These 100 upvotes will in turn have the same weight as the next 1,000 upvotes, and so on. Therefore, a post that has 10 very recent upvotes and a post that has 50 older upvotes could rank similarly on the platform.
3. Downvotes: Reddit is one of the few platforms on the internet that deploys a downvote feature. Posts that get a large number of upvotes and downvotes, as well as that get a large number of downvotes, will therefore rank lower on news feeds. This particularly impacts content that is controversial.

Reddit’s comment ranking algorithm was theorized by Randall Munroe, an American cartoonist, engineer, and scientific theorist. He argued that the hot ranking algorithm would not be suitably applicable for ranking comments on the platform, as it would preference comments that were posted more recently, rather than the comments that were considered the most meaningful. The solution to this was to deploy Wilson’s Score Interval, which uses a confidence sort to treat a vote count on a comment as a statistical sample of a hypothetical full vote by opinion, similar to an opinion poll. This system provides each comment with a provisional ranking, that it is 85 percent sure the comment will reach. The more votes that a comment receives, the closer its score gets to this 85 percent confidence estimate. This system helps ensure that if a comment has only one upvote and zero downvotes, it will retain a 100 percent upvote rate. However, because the system does not have enough data on this comment, it will be ranked lower. If a comment received ten upvotes and only one downvote, on the other hand, the system could accrue enough confidence to place this comment above something with 40 upvotes and 20 downvotes, as it ascertains that by the time this first post gets 40 upvotes, it would have fewer than 20 downvotes. If the system is wrong, which it is 15 percent of the time, then it will work to get more data so that comments with less data are ranked lower. The confidence sort in this system is not impacted by submission time, but rather it is impacted by how many upvotes a comment receives compared to the total number of votes and the sample size. The more votes a comment gets, the more accurate its confidence score is.²¹⁹ However, when subreddits have a large number of posts, it is likely that most people simply read the comments in the “best” section and vote on them. This prevents other comments from gaining traction and²²⁰ can create a preference toward these pieces of content.²²¹ Users can also feel persuaded to vote for already popular content, due to a herd mentality.²²² Additionally, users can create multiple accounts in an attempt to rig the voting system.²²³

Reddit’s algorithms also come into play when curating content for /r/all, which is the home page that non-logged-in users see. When a user first creates an account on Reddit, they are subscribed to a list of default subreddits that aim to highlight the range of communities, interests, and genres of content available on the platform.²²⁴ Once a user has an account, they can curate their own home page feeds by subscribing to subreddits of interest to them, and unsubscribing from default subreddits if they prefer. However, users who prefer to not create an account are unable to pick and choose which specific subreddits they engage with. For these users, the Reddit homepage displays the /r/all page that contains algorithmically curated content from a range of subreddits on the platform in order to demonstrate the breadth of popular content available on the service.²²⁵ The algorithm used to sort this page tends to highlight material across subreddits that is new and has been upvoted a lot.

However, for its content to make it to the /r/all page, a subreddit often already has to have a large subscriber base capable of driving a large score. The default subreddits that users are automatically subscribed to are examples of these. However, by privileging these default subreddits, other, organically created and operated subreddits are less frequently highlighted. This can silence certain voices and render certain communities and interest groups invisible to non-logged in users of the service. However, subreddit moderators can voluntarily opt out of this curation as well. This is often done if moderators feel that their content is controversial or not fit for public consumption (also known as “not safe for work” or NSFW).²²⁶ In this way, the opt-out feature can act as a privacy mechanism.

Reddit’s algorithmic curation and ranking systems seek to prioritize and deliver relevant and meaningful content to its users. Like on Facebook and Twitter, this raises a host of concerns regarding which voices are prioritized and how these voices are identified. When it comes to providing transparency and accountability around its algorithmic curation and ranking practices, Reddit offers some novel approaches, but also fails to adopt some existing practices that are gradually becoming more common across the industry.

Reddit’s approach to transparency is novel, in that it publishes the code for its ranking algorithms publicly online in an open-source format. This enables users, researchers, and publishers to better understand how content is tailored and ranked on the platform, what characteristics Reddit’s algorithms preference, and how this may impact a user’s worldview. This is one way of potentially revealing the existence of a filter bubble, as well. However, in order to effectively use this resource and extract valuable insights from it, an individual would have to have a relatively extensive technical background. Therefore, although the platform provides some valuable information on its curation and ranking practices, there are barriers to accessing and understanding it. Reddit also does not have a company-issued page explaining to its users how the algorithmic curation and ranking system works. As a result, most public information about Reddit’s ranking system and the signals and processes it uses are based on research or speculation, rather than company-verified information. Therefore, Reddit should publish information in language that is accessible to individuals who lack a technical background, as well as the general public. This will provide greater transparency and accountability around how Reddit curates and ranks content across its home page and subreddits.

Because content on Reddit is curated and ranked primarily on the basis of user votes, users do not have as many additional controls over their news feed experiences on the platform. Aside from voting on content, Reddit users can hide posts on the front page news feed and in subreddits. They can also sort content using a range of filters, including the “new” category, which filters content in a reverse-chronological order. Aside from this, however, users do not have any significant further controls, as the assumption is that user votes are representative of what users find interesting and meaningful, and the Reddit algorithm curates and ranks based on this. Like Facebook and Twitter, Reddit does not offer an appeals process or channel for its news feed curation and ranking efforts.

As demonstrated in this report, the deployment of algorithmic curation and ranking practices by search engines and internet platforms have established their roles as gatekeepers of online information flows and online expression. Over the past few years, these practices have increasingly been dictated by the business models and services of these companies. Despite the growing prevalence of such algorithmic curation, internet platforms have demonstrated a fundamental lack of transparency and accountability around how such practices are implemented, and how these practices impact users, their worldviews, and publishers. Going forward, search engines and internet platforms, policymakers, and researchers should consider the following set of recommendations in order to promote greater fairness, accountability, and transparency around algorithmic decision-making in this space.

In particular, search engines and internet platforms that deploy algorithmic tools to curate and rank content in their search results or news feeds need to:

• Make a concerted and explicit effort to raise awareness around these practices, and provide adequate transparency around how they impact users’ experiences and free expression online. For the most part, there is little transparency around the algorithms that these platforms use, particularly when it comes to their operational mechanisms and how they translate inputs into outputs. Going forward, internet platforms should provide more effective and meaningful transparency around these algorithms’ decision-making practices.
  • Search engines and internet platforms should disclose and explain the various stages and procedures involved in collecting, curating, and ranking content in search engine results or in news feeds. This information should be updated whenever the platform decides to alter or refine its curation and ranking practices. This information should be available publicly and should be presented in a format that is easy to understand for members of the general public. This information should also be housed in a centralized and easy-to-access area, such as on a page dedicated to the topic on the platform’s website.
  • Search engines that employ a search quality rating process should publicly disclose what this procedure entails, including how any human raters are trained and evaluated, and how the search engine ensures that these individuals represent a diverse array of perspectives. In addition, if a search engine has developed guidelines for these raters, they should be publicly available, as this would enable users, researchers, and publishers to gain a better understanding of what values a search engine emphasizes in its curation and ranking process.
  • Where possible, platforms that engage in algorithmic content curation and ranking should provide additional resources that can help users, researchers, and publishers understand how a platform assesses and ranks content. A good example of this is the open-source code that Reddit provides for its news feed. However, because these resources often require a specific, technical expertise to understand and effectively use, they should not replace corporate public disclosures and explanations on algorithmic curation and ranking practices. Rather, they should supplement them.
  • Given that search engines are increasingly delisting search results due to legal requests, violations of their search engine guidelines, and requests under frameworks, such as the “right to be forgotten,” greater transparency and accountability needs to be provided around these procedures, and how they impact free expression. One method of doing this is by highlighting the scope and scale of these requests in a corporate transparency report. Search engines should also provide impacted website owners with notice of these removals and offer them the opportunity to appeal these decisions in a timely manner.
• Provide greater transparency to identify the different implicit and explicit preferences that are built into curation and ranking systems. Generally, this is difficult to do with algorithmic systems, as many of these biases are hidden and implicit. This is likely going to be the case for such curation and ranking systems as well. However, the fact that these systems are based on a hundreds of signals means that some of these preferences are also explicit.
  • Search engines and internet platforms should publicly disclose a comprehensive list of the signals their curation and ranking systems are based on. This information should also include how these signals interact and build off one another, and how they are weighted. If it is not possible to do this at a granular level, due to concerns over trade secrets and competition, for example, then these platforms should at least provide a comprehensive list of categories encompassing the different types of characteristics and signals that these systems consider, how these categories interact with one another, and how they are weighted. This will generate a greater understanding of which qualities are emphasized and valued more on a platform, and as a result which voices are amplified and silenced. This list of signals or categories of signals should be available in a single, public, central location that is easily accessible. This information should also be presented in a format that is easily comprehensible by a general audience.
  • When a search engine or internet platform updates the signals their ranking systems are based on, it can have a significant impact on the presentation and delivery of user and publisher content. As a result, platforms should strive to provide as many updates explaining these changes as possible. If the platform opts to publish categories of signals, rather than a comprehensive list of signals, these announcements should explain how these changes impact the overall category of signals. Currently, platforms make such announcements around changes such as those that aim to reduce the spread of misinformation or curb abuse. These announcements often outline the companies’ thinking and intention behind these algorithmic changes, and can be valuable for shining a light on the priorities of engineers who implemented these changes. This is because such announcements often underscore what values and assumptions the company made when approaching a content issue.
• Provide users with a robust set of controls which enable them to tailor their own search and news feed experiences. In particular, users should be able to:
  • Provide feedback on search results and posts in their news feeds. This should include the ability to hide, block, or filter out certain posts. These controls let users manage their own experiences and also provide the algorithms with valuable feedback on what content a user deems relevant and meaningful.
  • Determine whether and to what extent these systems are permitted to collect and use their personal data. Data such as a user’s location history, purchase history, or past browsing activity can be used to tailor a user’s online experience and inform targeted advertising. As a result, users should be able to control which of these data points are collected, particularly in instances where the data points are used for secondary purposes and are not integral for the company to provide the service. In addition, users should have strong controls related to data retention practices. This should include the ability for a user to clear their search history and delete the data a platform has collected on them.
  • Opt in to having their personal data used to refine and develop AI and machine-learning models. Currently, users are automatically opted in to having their data considered in datasets that train algorithmic models, with no method for opting out. This system raises significant privacy concerns, especially because these datasets can include highly personal information such as demographic characteristics, which can be used to target users in privacy-intrusive ways. In cases when highly personal data is being collected and used for potentially invasive purposes, such as facial recognition, users should always have to opt in to having their data used to train AI and machine-learning models. This is because as biometric information cannot be altered or changed like a credit card number can. Companies should also develop robust privacy policies around user data that is collected and used for training AI models.
  • Opt in to receiving algorithmically curated and ranked content. Ideally, users should only receive algorithmically curated and ranked content after affirmatively opting in—it should not be the default setting. However, if companies maintain that users should receive this content by default, users should be able to completely and easily opt out if they do not want to receive such content in their search results or news feeds. Additionally, this should be possible on search engines regardless of whether a user is browsing as a logged-in user, logged-out user, or private/anonymous browsing user.
• Enable publishers to understand and exercise some control over how their content is collected, curated, and ranked.
  • Both search engines and internet platforms should offer website publishers and content creators clear and detailed guidelines on how they can operate fairly and successfully on a given platform. For search engines, this means publicly sharing a detailed set of webmaster guidelines. For internet platforms with news feeds, this means publicly disclosing relevant rules and guidelines related to content creators.
  • Search engines should also let website publishers determine how their website is crawled. One way of doing this is using a sitemap. In addition, website publishers should be able to request recrawls if they feel their website was not adequately represented in a search engine’s index. Furthermore, website publishers should be able to opt out of crawling altogether if they so desire.
  • Website publishers who feel that their websites have been unfairly penalized in search engine results should be able to appeal these decisions and request a reconsideration. This appeals process should be timely and should provide adequate notice to the web page publisher of the outcome. Search engines that offer this appeals process should also educate website publishers so they know it exists and know how to use it.
  • Content creators who feel that their content has been unfairly penalized in news feeds should also be able to appeal these decisions. This appeals process should be timely. Internet platforms that offer this procedure should publicize information about this process and make it easy to understand.
• Internet platforms, policymakers, and researchers should collaborate on, promote, and fund further research on the impacts of algorithmic curation and content ranking. This is particularly important amid growing concerns around the impact of these algorithmic decision-making practices on democratic values, political polarization, and freedom of expression.
• Internet platforms, researchers, and civil society organizations should collaborate to develop a set of industry-wide best practices for transparency and accountability around algorithmic curation and ranking practices. These best practices should explicitly prioritize the public interest above corporate business models and concerns about trade secrets. This will help ensure that users are adequately educated and aware of these practices, have a range of meaningful controls at their disposal, and know how to use them. It will also promote greater accountability around these algorithmic decision-making practices and combat the need for knee-jerk legislation or regulation.