The Early Grades are Different: A Look at Student Surveys

Peer into a college classroom near
the end of a semester and you may encounter a perplexing sight: students
hunched over answer sheets, bubbling in answers and scribbling notes, seemingly
unsupervised and undirected. But these students aren’t taking a test. They are
completing student surveys, a tool used by colleges and universities to
evaluate professors. Now this practice, long a part of higher education, is
being tried in K–12 classrooms as part of efforts to reform the way teachers
are evaluated.

The reasoning is intuitive.
Throughout the year, students spend countless hours with their teachers, while
principals, who have long been the primary evaluators of classroom teaching
quality, may visit a classroom a few times a year. Students know whether their
teacher is able to manage the classroom and whether they feel supported as
learners. It shouldn’t come as a surprise, then, that research shows that student perceptions of
teacher quality may actually be more accurate than those of a principal.

In the changing teacher evaluation
landscape, student surveys have surfaced as a way to collect feedback from
those who know teachers best.

Using student surveys to
effectively evaluate teachers doesn’t come without its fair share of
challenges, however. Surveys rely on students being able to understand and
answer questions about their teachers. Older students seem to have no problem
doing this. The Bill & Melinda Gates Foundation’s Measures of Effective
Teaching (MET) project found that survey
results from students in grades four through eight can predict student
achievement gains, are more consistent than results from classroom
observations, and provide teachers specific feedback to improve their
practices.

But what about using student
surveys for children in pre-K through second grade? Asking young children to
evaluate their teachers can prove more challenging for a host of reasons. There
are concerns
about their ability to understand questions or give unbiased feedback. Current
research on the consistency and accuracy of student survey results is far from
robust, but a few studies have found that pre-kindergarteners
and kindergarteners can reliably rate their teachers. Still, many state and
local policymakers are still deciding whether surveys in the youngest grades
make sense.

Whether surveys are appropriate at
any age depends largely on how they are designed and implemented. An effective
survey must ask questions in a way that yields meaningful results.
For instance, rather than asking students to respond to the statement, “My
teacher is clear,” a high-quality survey might ask students to respond to, “My
teacher has several good ways to explain each topic that we cover in this
class.” Surveys should look different in pre-K through second grade than they
do later on because of literacy challenges and developmental differences. A
six-year-old probably wouldn’t be able to read the sentence above and might not
even comprehend it correctly if read aloud. And, if administered aloud, would a
child feel comfortable providing negative feedback? If administered in a small
group, would children feel pressured to agree with their peers?

Although most states do not use
surveys in the early grades to inform teacher evaluation, a handful of states
and districts have either experimented with or implemented K–2 student surveys.
Their experiences offer a glimpse into what this evaluation tool could look
like for others. Some of these states and districts have undertaken the design
process on their own, but many others use companies, like Tripod or Panorama
Education, which have expertise in student survey design. Most of the leading
companies designing student surveys, however, do not offer them for students in
kindergarten through second grade.

Tripod is one of the few that does. Tripod’s
surveys, which span from kindergarten through grade 12, were first developed in
2001 as a partnership between Harvard University Professor Ronald F. Ferguson
and Ohio educators. These surveys, now used in nearly 30,000 classrooms in 24
states, have been updated in response to feedback from research, field
experience, and input from key stakeholders to ensure that “survey items at the
Early Elementary level are specifically geared for students at this
[developmental] level.” Tripod says its K–2 surveys are “read aloud to
students, in small groups, by someone other than the classroom teacher,” and
students “only need to be able to recognize the words ‘No,’ ‘Maybe,’ and ‘Yes’”
to respond. Tripod’s analysis from “thousands of classrooms,” claims that
kindergarten students are able to distinguish between these words. It’s unclear
whether this analysis found whether students can fully comprehend the questions
asked. However, Tripod says that its surveys “have generated results that meet
rigorous tests of reliability,” and that early elementary surveys “have been
shown to predict achievement gains as well as other outcomes such as classroom
level behavioral engagement, emotional engagement, and motivational mindsets.”

Panorama Education also offers surveys in the
early grades. A representative from Panorama says the surveys the company has designed
for students in grades 3–12 have been validated, but “there is some research
that K–2 students may have a difficult time conceptualizing and comprehending
some items around climate, valuing of school, classroom engagement—making them
less reliable measures.” It still offers early elementary surveys when
requested, though. On Panorama’s survey, students choose between a smiling
face, neutral face, and frowning face, rather than “yes,” “maybe,” and “no” due
to the varied and often limited reading abilities of young children. When we
asked Tripod if it considered giving students the option of pictorial
responses, such as a smile for “yes” and a frown for “no” the company responded:
“pictorial responses can be more open to interpretation and do not always align
to the thoughts and feelings we are trying to capture through the survey
items.” Whether pictorial or word-based responses best capture students’ views
remains to be proven.

Tulsa Public Schools is one
district that recently started using Tripod surveys in K–2, but there has been backlash from dozens of early grade teachers
who expressed concerns about the appropriateness of the surveys for the age
group and their potential impact on the student-teacher relationship. In
response, the superintendent of Tulsa Public Schools created a task
force to study the use of the surveys, in the early grades. Tripod has been
working with the task force to answer questions about the surveys, create
resources to support survey administration, and even offered suggestions for reformatting
the surveys to address teacher concerns.

The state of Hawaii implemented
Tripod surveys in the early grades statewide but ultimately decided against
using them. Hawaii started using the Tripod student survey in the 2012–13
school year to constitute 10 percent of a teacher’s overall evaluation score
for all grade levels. Teachers and teacher unions lobbied to remove surveys
from early grades, and as of the 2014–15 school year, kindergarten through
second grade teachers did not use surveys at all. Despite the data provided by
Tripod, educators felt that students at this age did not have the capacity to
comprehend the purpose of the survey. They also raised concerns over the amount
of time and resources it took to implement the surveys, as they had to be read
aloud in as many as 14 different languages to accommodate the state’s diverse
student population.

Georgia had a related experience.
The state experimented with using student surveys to inform teacher evaluation
in 2012. According to Cindy Saxon, Associate Superintendent of Teacher and
Leader Effectiveness for the Georgia Department of Education, there was
feedback from teachers and parents after the pilot that led to the discontinuation
of the survey in K–2. Georgia experienced particular difficulties in the early
grades, where young students struggled to log into and navigate a complex
online survey platform. Saxon said that “what should have taken ten to fifteen
minutes might have taken an hour and a half” in younger grades. These
logistical barriers, and teachers’ perception that “students couldn’t give an
accurate view of their classroom” were enormous obstacles to survey use.

Additionally, the state evaluated the pilot survey’s results and found
that K–2 students tended to rate their teachers significantly higher across the
domains, and K–2 results were the least consistent with other evaluation
measures. In the most recent legislative session, lawmakers removed surveys
from Georgia’s evaluation system for all grade levels. Saxon felt that,
unfortunately, the surveys were often perceived as another assessment to be
used punitively against teachers, and there was concern about the time and
resources they took to administer.

 Massachusetts is another state
working to determine how best to incorporate student feedback into teacher
evaluations at all grade levels. Massachusetts’ overall evaluation system
provides significant flexibility to districts; the state doesn’t assign
specific weights or point values to different components of the evaluation.
Rather, it describes three broad categories of evaluation and recommends pieces
of evidence to collect in each one. One of these categories requires evidence
of “student and staff feedback.”

The state has also decided that
surveys for early grade students are not the answer. In 2013, Massachusetts
officials began collaborating with districts to develop and pilot standardized
student surveys in all grades. But according to Matthew Holloway, an Educator
Effectiveness Specialist at the Massachusetts Department of Elementary and
Secondary Education, teachers in the early grades gave feedback after the
initial pilot that “some of the barriers around literacy, reading, and writing,
and more general barriers” made a more traditional survey “an ineffective way
to get feedback.” With this in mind, the state worked with educators to create
a set of “Discussion Prompts” that capture “a broad
range of educator effectiveness standards” in what it feels is a more
developmentally-appropriate manner. The state recommends that teachers or a
third party use the discussion prompts with small groups of students to solicit
feedback. For example, the teacher might try to gauge whether the following
statement related to informal assessments is true for students: “When I am
stuck, my teacher wants me to try again before she or he helps me.”

Using the state-developed
discussion prompts is not mandatory; districts and school districts can choose
their own form of surveys or alternative evidence. Holloway stressed that
giving districts flexibility in how they collect student feedback “allows them
to emphasize their priorities.” He noted that he has seen the most success with
student feedback in districts that promote the process as a way for teachers to
“align student feedback to their own self-assessment and goal setting
processes,” where teachers work to improve their practice in a way that is
informed by students. Massachusetts is still in the early stages of
implementing student feedback as part of teacher evaluations and is examining what
is most beneficial to districts, teachers, and students.

It is possible that student survey
responses can shed light on teacher performance and encourage teachers to
tailor their instruction to better serve their students, but there are real
concerns states and districts have about their usefulness in the early grades.
States that have attempted to incorporate surveys in the early grades have
found them to be very time- and resource-intensive to implement. Further, early
grade teachers have expressed concerns over their students’ ability to
understand the purpose of the task.

Some states do appear to be eager
to find ways to integrate student feedback into the evaluation system. As one
state official put it, “kids deserve the opportunity to [take student surveys]…
and our kids can be empowered if they think they have a voice.” While the
research on whether young children can effectively evaluate teachers appears
inconclusive, there is little doubt that teachers can benefit from knowing what
their students think about their classroom experience. But using surveys to
inform evaluation is still new and inconsistent. If used solely to inform
teacher practice and professional development, reluctance on the side of the
teacher may be reduced. More research, however, is needed to determine if surveys
should be incorporated into high stakes decisions for teachers of young
children.