A New Program Has Helped Bring Better Media Literacy to Ukraine

Two years ago, in libraries, offices, and recreation centers across Ukraine, 15,000 citizens came together in small groups to talk about something that many of them had never really pondered before: their media consumption. They sat down at tables, introduced themselves, and spent a few hours mulling over what they watched on TV, what they viewed on the internet, what they read in the newspaper. Each person—ranging in age from 18 to well past 70—was given a handout with spaces to fill in to help them reflect on how much time they spent with various media forms of each day.

No one collected the handouts afterward, but the exercise triggered questions and chatter. What channels or newspapers did they subscribe to? Did they trust what they heard and read? From where, exactly, were they getting their information about the world?

“We were not telling them what to read. We were not questioning what they read,” said Mehri Druckman, who led the project as director of programs for IREX Ukraine. “The intent was to help them make decisions for themselves and how to be careful about what they were consuming, to help them not be manipulated.”

The program, called Learn to Discern, just might be working. Eighteen months later, IREX, a nonprofit education organization with international reach, conducted a follow-up study to determine whether Learn to Discern had any impact on its participants. (Disclosure: IREX sponsors a U.S.–Ukrainian journalist exchange that Slate takes part in.) The organization surveyed more than 400 people—200 who took part in the program and 200 of similar age and demographics who did not—asking them to read a few news items and answer questions about what was true and what was questionable. The results, to be released Thursday, showed that Learn to Discern participants performed better than their peers on assessments of whether they understood where their news was coming from and whether they could detect disinformation.

In addition, according to IREX’s report on the study, participants “were more likely to cross-check other sources, rate their skills higher in distinguishing true information from false, and were more confident in their ability to analyze the truthfulness of media content compared to the control group.”

On average, Learn to Discern participants got 63.8 percent of their answers correct on a test of disinformation analysis, while the control group scored 56.5 percent. The study asked people to read and answer questions about an article on a shooting at the Ukraine–Russian border (an objective story, according to IREX), as well as one about educational reforms in Ukrainian schools that would cut Russian out of language programs (a disinformation story). Participants also took an assessment to determine how much they knew about media outlet ownership and answered questions on whether they felt a sense of control over how they are influenced and informed by the news. This is a particularly salient point in Ukraine, where concerns over the spread of disinformation have been heightened for years, as reports from Russian-controlled media have featured demonstrably untrue statements about everything from the 2014 shooting-down of a Malaysian Airlines plane to Ukraine’s human rights record. (For specific details, see StopFake.org, an organization dedicated to debunking the falsehoods.)

The study is “groundbreaking,” said Renee Hobbs, a professor at the Harrington School of Communication and Media at the University of Rhode Island, and the founding co-editor of the Journal for Media Literacy Education, who was not involved in the study but reviewed an embargoed copy. “It suggests a model,” Hobbs said, “for how we might measure media literacy competencies acquired by adults through formal media education programs.”

The Learn to Discern program ran from 2016 to 2017 and was not offered in schools. Instead it aimed to reach working adults and retirees in places that they frequently visit. IREX trained 450 people who work in cities and towns across Ukraine to administer the Learn to Discern sessions, the majority of which lasted a half-day. Session leaders included librarians, teachers, police officers, and other community leaders—all of whom were in charge of recruiting trainees by spreading the word through their social networks.

Although Learn to Discern is based on a nearly 200-page curriculum (copies are available in English, Ukrainian, and Russian), the trainers were told that they didn’t have to follow the it step by step and could choose what activities to do with their fellow citizens. “We really empowered these trainers,” said Tara Susman-Pena, a senior technical adviser at IREX, who credits the program’s success to power of social networks and the high level of interaction that was encouraged in each session. Each trainer received a box that included the curriculum as well as T-shirts, tea, and cookies, and they were encouraged to turn the sessions into a discussion. “There was not a lot of lecturing,” Susman-Pena said. “The people in Ukraine have a history of having a lot of people lecturing at them,” she said.

In addition to the impact study released this week, IREX also created and published a videoabout checking the “ingredients” in the news that one consumes, which ran as a public service announcement on television stations across the country in February 2016. And it conducted interviews and surveys of Learn to Discern participants and trainers directly after the program ended in 2017. According to Druckman,

One of our citizen trainers described the process this way: “At the beginning of the training, participants were confident and even arrogant (especially older participants). By the middle, they were confused, asking ‘whom should we believe?’ After the training, they became cautious towards the information they consume.”

Here in the United States, where many of us feel desperate for a sign that our citizenry has the aptitude to detect and deflect fraudulent news and avoid the influence of Russian bots and disinformation campaigns, a program like Learn to Discern may represent hope. But Hobbs, who has analyzed and reviewed hundreds of studies on media literacy, issues a few notes of caution. The study of IREX’s program, while solid in its methodology and its openness about results, was conducted by IREX itself and not an independent research team. The control group, while close in demographics to the participants, was not a perfect match: 26 percent of people in the control group had a vocational education background, whereas only 17 percent of participants did.

Perhaps more importantly, the survey did not ask any questions about participants’ political leanings. People’s politics can greatly color their answers about what they trust. A recent report from the Knight Foundation, for example, showed that Republicans largely distrust the media, while Democrats trust it. It also found that 4 in 10 Republicans consider accurate news stories that cast a politician or political group in a negative light to always be “fake news.” In Ukraine, political alliances are typically linked to favorable or unfavorable views of Russian influence.

The Learn to Discern program was also lighter on its treatment of social media, such as Facebook, than broadcast television. The curriculum was designed in 2015 when social media was not used by as many people in Ukraine as it is today, and IREX’s previous studies had shown that TV was the medium used by most Ukrainians to get their news. Druckman of IREX said she that she and her colleagues are now updating the curriculum to focus on “responsible sharing” of news as part of new efforts on media literacy. She has developed a series of what she calls “Care Before You Share” activities to help people pause and think when a strong emotional response motivates them to share something they have heard or read. As her colleague Susman-Pena said, the emotional aspect of media consumption and sharing “is something we really need to examine.”

Despite these shortcomings and still-to-dos, Hobbs said Learn to Discern could offer many important lessons for media literacy education, which is having a moment in the spotlight.
Although the concept of media literacy has been around for decades, in the past few years several programs in public schools and universities are gaining momentum as educators look to help their students take a more critical eye to the news and information that is streaming at them. Examples include resources from the Center for News Literacy at Stony Brook University’s School of Journalism, the Checkology software for high-schoolers developed by the News Literacy Project, professional development and classroom materials offered by the Newseum, and even hands-on activities for teachers and parents of elementary and pre-K children, developed through library multimedia programs (like this one described in a Slate article last November) and by Faith Rogow and consultants at Insighters Ed.

Testing whether these kinds of programs are in fact leading to more discernment, and what elements make the most difference, is the next step. The IREX study measured the participants’ media literacy through the assessments based on actual news and disinformation articles—a research approach that Hobbs applauded—instead of simply asking participants whether they thought they were getting smarter about the media and relying on their self-reports

An analysis of the assessments exposed an interesting link: The people who showed knowledge of how the media works and who owns it were also able to discern disinformation. “It turns out that if you know something about how the sausage is made, it makes a real difference,” Hobbs said.

Susman-Pena said that the link between media knowledge and critical thinking has led her team to tell trainers: Don’t skip over the activities about news media control and who owns what. (An analysis by the organization Media Monitor Ownership, for example, shows that television in the Ukraine is controlled by a monopoly: the company Zeonbud.)

Susman-Pena is now working on how to adapt Learn to Discern for American audiences, based on demand from people in various American communities and using resources from IREX’s Center for Applied Learning and Impact, which receives funding from multiple donors. (The funding for the Ukrainian effort, by the way, came from the Canadian government as part of that country’s efforts to promote democracy.) One of the first spots for trying out the program is in Arizona, where a math teacher who is leading efforts to improve STEM and scientific inquiry heard about the program and is working to start pilot projects in some rural areas. Librarians around the U.S. are also contacting IREX, Susman-Pena said, to find out how to roll it out in their communities.

The IREX study in Ukraine also helped to surface areas that need more work across education systems of all kinds. Even though the study showed that Learn to Discern participants were more aware of the problem of disinformation than the control group, their performance on the assessments wasn’t exactly a shining example of deep understanding. They scored only an average of 63.8 percent correct on the test of disinformation (that translates to a lowly D, if we’re talking grades, with the control group getting an F), and both groups far did worse on the assessment of the objective news story. About one-third of their answers to questions about that story were correct.

That poor showing, according to the report’s authors, may mean “detecting markers of objective news may be more difficult than detecting manipulation and disinformation.” And learning how to seek out and evaluate relatively objective journalism may need to be on deck as another problem to solve. “The biggest challenge in my opinion it is to be very careful between raising healthy skepticism but not contributing to overall mistrust,” Susman-Pena said. “That is exactly the Kremlin strategy, to get people to think, Oh there is no such thing as real truth, there aren’t facts.” That is the last thing that IREX wants participants to learn. “Now on my mind,” Susman-Pena said, “is how to get people to seek out fact-based information.”

This article originally appeared in Future Tense, a collaboration among Arizona State University, New America, and Slate.