In Defense of Using Surveys in Assessment

This piece originally appeared in the April Newsletter for the New Leadership Alliance for Student Learning and Accountability. It’s a wonderful organization and resource for those of us committed to the assessment of student learning, and worth checking out.

I’ve just returned from several assessment and accreditation conferences, and at each and every one, speakers cast a questioning eye towards the use of survey data in assessment. I’ve always been of the mind that understanding student learning is a complex endeavor, and there was no simple “killer” assessment method which would establish what students know and can do. As a former Director of Assessment, I’ve used rubrics, set up and evaluated portfolios to look at general education skills, helped departments administer standardized tests to measure content knowledge, and yes, used surveys to document and understand student learning.
Every assessment method has its advantages and disadvantages, and each can be used and misused, but I’d like to make the case that survey results are a natural and necessary companion to other measures of student learning.
Why? Assessing student learning is complicated and can be resource intensive. It takes time to develop rubrics, and campuses with portfolios can attest that they can be a challenge to maintain and score. Before embarking on direct assessments like rubrics and portfolios—both methods that can yield powerful insights into student learning—it is useful to take a careful look at the survey results to see what is already known about the learning outcome to avoid spending time and resources addressing issues for which information is already available. For example, an institution might be interested in the development of writing skills. Looking at survey results about how often students are asked to write papers of varying length or which majors or divisions are assigning various types of writing, can help an institution make decisions about where to target their direct assessment and understand the types of work that are likely to be received. Survey results can also be used to focus the content of the direct assessment itself. If survey results suggest that most students are frequently being asked to support their opinion with a logical argument, then assessments can be designed to examine that in greater detail. By providing information about what students are already doing and where, results from surveys can help an institution keep its assessment efforts manageable, strategic, and focused on improvement.
Many schools examine learning outcomes, like critical thinking or mathematical ability, using a standardized measure or test. This tells you important information about student performance, but does not provide the whole picture. If 25% of students score below a threshold on a standardized math test for example, what should the course of action be? How can that score be improved? Linking the test score with survey results allows you to look at other environmental factors: how often do these students study; do they study in groups; whether or not they took any remedial or special programs; how often they interacted with faculty. The more we know about the students behaviors in college, the more we can design activities and programs that foster the desired outcome.
Surveys get at issues that matter-not only cognitive aspects of student performance, but also non-cognitive aspects, like student values, responsibilities and commitments to citizenship, and community service. Taken together, these allow an institution to get a sense of civic engagement of its students. By collecting this information, surveys give institutions a window into the gestalt of the college experience, and allow institutions to connect gains in student learning to academic experiences like participation in first-year seminars, frequent interaction with faculty members outside of class, and with other important aspects of the college experience such as service learning and leadership training. Because surveys often ask about a wide variety of experiences, behaviors and beliefs, institutions can make connections between seemingly disparate aspects of the college experience. That information can be used to better understand how students use and experience the college, and to target improvement efforts.
Survey results can be disaggregated easily to look at relevant subgroups. For many faculty and staff “Are we meeting our goals?” is a pretty simplistic question. What they really want to know is far more subtle—for whom is the campus successful; what is being learned; are students here thriving, and which ones? All of these questions speak to how well a school is fulfilling its educational mission. Surveys allow us to get an understanding of how different groups of students are experiencing the educational environment.
The ultimate goal in assessment is to inform institutional change, to actually make changes that impact what students get out of college. We do this by aligning the programs, practices and opportunities on campus to foster a better learning environment. There are many ways of gathering assessment evidence, and survey results are an essential piece in the puzzle that should not be downplayed or overlooked.