Results from the July Poll of the Month: Mode of Survey Administration
In HERI’s first-ever Poll of the Month in July, we asked folks about their preferred method of survey administration. We provided three options: paper only, web only, or a combination of both web and paper. In total, we had 59 respondents to our little poll, and the distribution of responses was as follows:
15% reported only using paper surveys
36% rely only on web administrations
Nearly half (49%) incorporate a combination of web and survey administrations into their cycle
For most of our surveys at the Cooperative Institutional Research Program (CIRP), we offer both web and paper options (this past year was an obvious exception for YFCY and CSS). Paper surveys, when administered to a captive audience (i.e., in a proctored setting), represent one of the best methods for ensuring a high response rate. We find that roughly 75% of our campuses that participate in the CIRP Freshman Survey utilize a paper administration, and the vast majority of these schools make the Freshman Norms every year (surpassing a 65% participation rate)! By contrast, just under one-third of “web-only” schools reach this same threshold for the CIRP Freshman Survey.
Administering surveys on paper to freshmen during orientation sessions or to seniors during graduation rehearsal or cap and gown distribution can certainly help to create a norming culture where participation is not only encouraged but also expected. Not all campuses, however, have the personnel resources or the time during orientation or the academic year to allow for an in-person paper administration. Web surveys provide a flexible alternative to a paper survey administration and also represent a “greener” administration option, which we’re finding many students appreciate. Two of CIRP’s surveys – the Diverse Learning Environments survey and the HERI Faculty Survey – are web only instruments, as those surveys provide opportunities for campuses to customize surveys with elective modules.
Campuses using web surveys, however, need to give strong consideration to a number of factors. How good are the email addresses students have on file with the institution? Is the site from which the emails are being sent whitelisted with the campus? Is there an incentive to encourage participation? What additional marketing tools are available to help get the word out about the survey beyond email?
With all of our web surveys, we allow campuses to upload two sets of email addresses to increase the likelihood that the email invitation for the survey reaches students. We also outline specific anti-spam instructions for reps to pass along to the information technology departments to ensure that survey invitations coming from Qualtrics (the vendor we now use for all web surveys except TFS) or DRC (our TFS vendor) do not get tied up in university spam filters. Incentives are certainly key in any survey administration but especially so in web surveys, as students (and faculty!) are bombarded with survey invitations from different units on campus as well as from marketing agencies. The poll of the month for August asks about specific kinds of incentives you use for your surveys, so take a few minutes to respond.
Additional marketing tools for web surveys are also incredibly important to consider. In the event that email invitations are getting tied up in potential respondents’ personal spam filters, having alternative strategies in place to let your sample know about the survey is key. We have seen folks rely on CIRP Infographics, post billboards on buses, and have scrolls in the dining hall as effective advertising methods. Don’t overlook the importance of marketing if choosing to do a web-only survey administration.
The combination of web and paper administration, particularly in the same survey, provides campuses with incredible flexibilities. If you start with web surveys, you can send a more targeted set of paper surveys to non-respondents. Obviously the reverse could also work effectively – following up with students who did not submit a paper survey with a targeted set of emails requesting they complete the online version of the instrument. We offer this hybrid option for the CIRP Freshman Survey, College Senior Survey, and Your First College Year survey.
Regardless of method of administration, it is important to ensure proper planning has been done and that personnel are available to assist in the administration process. Hopefully with the right combination of paper/web administration and early and often marketing, you will see those response rates inch higher! Just a reminder, you can go ahead and register for a paper, web, or hybrid administration of the 2014 CIRP Freshman Survey right now, and registration for the YFCY, CSS, and DLE will open in mid-September.
