HERI BLOG

We are spending this week preparing for the Annual Forum of the Association for Institutional Research, which begins May 18 in our backyard of Long Beach. I wanted to highlight several of presentations at the annual forum that draw from CIRP data. You can see the full schedule of presentations using CIRP data here.

I arrive on Saturday to present a pre-conference workshop with my colleague Dr. Jessica Sharkness from Tufts University. We use 2004 CIRP Freshman Survey and 2008 College Senior Survey data as a teaching tool in a half-day workshop on linear and logistic regression. The workshop is designed to provide participants with hands-on instruction for using SPSS for regression analyses. (Saturday, May 18, 12:30 – 4 p.m.)

On Sunday, CIRP Director John Pryor is joined by Erin Knepler from the University System of Maryland for a masters seminar on national surveys. John and Erin will discuss the need for national surveys and address recent criticisms. (Sunday, May 19, 12:15 – 2:30 p.m.)

On Monday, John is joined by HERI Director Dr. Sylvia Hurtado and UCLA doctoral candidate Adriana Ruiz Alvarado in a presentation that examines first-year student retention using validation theory. John, Sylvia, and Adriana use CIRP Freshman Survey and Your First College Year survey data to explore how students’ sense of validation from faculty and staff during their first year of college contribute to their intention to return to their native institution in the second year. (Monday, May 20, 10:30 – 11:15 a.m., 201A)

Tuesday is by far the busiest day for CIRP-related presentations. The day begins early with a presentation by Arnold Hook from the University of Louisville. Arnold leads a discussion group that examines possible links between TFS data on measures of self-concept with student retention. (Tuesday, May 21, 8:30 – 9:15 a.m., Grand Ballroom Table 4)

Sylvia and I are joined by Tanya Figueroa – a doctoral student at UCLA and member of our research team – to present findings from a study that uses data from the 2004 CIRP Freshman Survey and 2010 National Student Clearinghouse to examine the predictors of earning a bachelor’s degree in the biomedical sciences. We have been presenting findings from our project for the last eight annual forums, and we are now beginning to break down our STEM completion data into sub-disciplines. (Tuesday, May 21, 9:30 – 10:15 a.m., 103A)

CIRP Assistant Director Dr. Laura Palucki Blake joins C. Ellen Peters from the University of the Puget Sound late Tuesday morning for a presentation on how to use survey data for campus improvement. Laura and Ellen focus on strategies for presenting findings from analyses of survey data in a straightforward, easily digestible way for audiences that include administrators and faculty. (Tuesday, May 21, 11:30 a.m.–12:15 p.m., 202A)

Be sure to stop by Rachel Krug’s (Franklin Pierce College) poster during the lunch break as she presents findings from a cluster analysis on CRP’s Habits of Mind items. (Tuesday, May 21, 12:45–1:45 p.m., Hall B-Poster 42)

I wrap up the CIRP-related presentations for Tuesday with Adriana Ruiz Alvarado as we present findings from a three time-point structural equation model analyzing experiences that serve to accentuate or mitigate students’ academic disengagement throughout college. Analyzing data from the CIRP Freshman Survey, Your First College Year survey, and College Senior Survey, we can talk about particular college experiences that lead to greater academic disengagement (e.g., frequent drinking in the first year) or mitigate the extent to which students disengage in the classroom (e.g., getting enough sleep in the first year of college). (Tuesday, May 21, 3–3:45 p.m., 203B)

We have two presentations on Wednesday. Sylvia and I are joined by two of our graduate student researchers – Juan Garibay and Bryce Hughes – to present findings from our seven-year follow-up survey of the entering freshman cohort from 2004. We focus on students who earned a STEM bachelor’s degree and examine the undergraduate experiences that prompted these individuals to pursue graduate degrees in STEM, professional work in STEM, or non-STEM post-college pathways. (Wednesday, May 22, 9–9:45 a.m., 202A)

While we present our findings on STEM, Dr. Laura Palucki Blake will deliver a presentation focused on how to use survey data to improve student learning. Laura will touch on ways to partner with faculty to make meaning from survey data to connect with evidenced-based decision making on campus. (Wednesday, May 22, 9–9:45 a.m., 203A)

Again, you find the full schedule of CIRP-related presentation here. Be sure to stop by our booth in the exhibit hall during the day on Monday (10 a.m. – 5 p.m.) or Tuesday (10 a.m. – 2 p.m.). We look forward to conferencing with you!

The inaugural HERI Institute on Faculty Work/Life Issues is fast approaching (May 22-24), and I wanted to share some more information about the curriculum that Dr. Sylvia Hurtado and I have planned. We have designed the Institute to include a mix of research and practice perspectives for understanding faculty behavior and working conditions and improving teaching and learning on campus. The Institute features a number of opportunities for participants to network with one another and share best practices and effective strategies for recruiting and retaining faculty, supporting their work, and improving teaching and learning.

We have posted a preliminary agenda for the Institute here. The morning of the first day will feature presentations by me and Sylvia highlighting some of the key findings that have come from analyses of our national surveys of college faculty and insightful frameworks that can offer some perspective in understanding faculty behaviors and motivators.

Just before lunch, Sylvia and I will facilitate a conversation among participants to discuss key sources of stress that faculty encounter on campus and the ways in which campuses are addressing these issues.

After lunch Sylvia will present some of the latest research on how faculty and administrators can use data to assess and improve teaching and learning on their campuses. Afterwards, our afternoon breakout session will provide another opportunity for participants to network and learn from one another about campus-based strategies being implemented to improve teaching and learning through behavioral change. We will wrap up the day focusing on understanding the needs and experiences of faculty of color.

I will begin our Friday morning conversation by talking about part-time faculty, their connection to student outcomes, and how campuses can support this ever-growing segment of the academic workforce. Later in the morning, I will be joined by my UCLA colleague Susan Drange Lee to discuss issues related to faculty retention, retirement, and recruitment. I will give particular attention to issues of faculty retention and the coming wave of faculty retirements expected over the next decade. Susan will offer some insights on UCLA’s approach to faculty recruitment with a particular focus on recruiting for a more diverse faculty.

We plan to end the Institute with a sneak peek at the changes we have made for the 2013-2014 HERI Faculty Survey, which goes into the field later this summer.

For more information about the Institute, please visit our registration page. I hope you will consider joining us for what promises to be an intellectually engaging event.

The 2013 AERA annual conference is upon us, and HERI is excited to announce that 20 papers at the conference will present findings from analyses of CIRP data. Nine of these studies are being presented by HERI staff members or affiliated scholars; the remaining 11 studies have been conducted by researchers who were granted access to data through HERI’s data access process.

These presentations cover a diverse set of topics: four focus on faculty issues; three examine STEM student experiences; three investigate the influence of campus climate on diverse students’ experiences; and two consider student mobility patterns. Other topics include students’ social agency, graduate school plans, faculty interactions, college choice, affirmative action, college adjustment, academic achievement, and student participation in service learning.

A few of the highlights include:

HERI Assistant Director for Research Dr. Kevin Eagan presents with his co-author Dr. Audrey Jaeger on findings from the 2010-2011 Faculty Survey that address the tension between institutions’ increased use of contingent faculty and the ability of colleges and universities to respond to calls for increased community engagement. (Sunday, April 28, 12:25-1:55 p.m., Hilton Union Square, Ballroom Level – Continental 9).

Adriana Ruiz, a HERI graduate student researcher, presents with HERI Director Dr. Sylvia Hurtado on findings from CIRP’s Diverse Learning Environments survey. They identify differences across measures of institutional diversity in Latina/o students’ experiences with a hostile campus climate. (Tuesday, April 30, 5:05-6:35 p.m.).

Dr. Sylvia Hurtado presents with her research team in a symposium that highlights issues of STEM persistence and career trajectories for underrepresented students and faculty. Specific papers within this symposium examine engineering completion for all students and for Black students in particular; differences in biomedical science completion for pre-med and non-pre-med students; the experiences of female faculty of color in STEM; and the work to diversity STEM undergraduate education and the STEM workforce by underrepresented STEM pioneering faculty. (Wednesday, May 1, 12:25-1:55 p.m., Hilton Union Square, Ballroom Level – Continental 9).

Click here for a full list of papers presenting findings from analyses of CIRP data.

This morning on NPR I heard a story on the impact of being asked to categorize ones racial/ethnic background on questionnaires or other queries, such as medical forms.  This eight-minute piece reminded me again that a questionnaire is not a one-way street, but a two-way street. What we chose to ask, and how we choose to ask it, says a lot about what we value and who we are. It’s not just in the answers that we find meaning, but also in the questions.

How we ask says something about who we are and how we perceive the world, and when your perception is vastly different than those whom you seek answers from, discomfort arises. All too often that discomfort is on the side of the respondent, and not the researcher.  Unless we choose to get out from behind the forms, behind the numbers and the calculations, and actually interact with those we claim to know about, we only have a small piece of the puzzle.

This NPR piece reminded of me about that discomfort, and how very important it is to take every step possible to minimize that. Over a span of twenty years of my personally crafting questions I have certainly made a flew blunders. I credit some very wise educators I worked with early on in helping me see the importance of how we chose our words and what we ask, and like to think that I am a better researcher and human being because of that awareness.

Reflecting on this today, I again was reminded that pioneering work in asking questions is a legacy of the Cooperative Institutional Research Program (CIRP), which I have the privilege of currently directing. It was not until the 2000 that the US Census allowed people to represent themselves with more than one race or ethnicity. The CIRP Freshman Survey has done so since 1971.

We’ve updated the YFCY infographic to reflect results from the 2012 Your First College Year Survey.  There are new versions of both the standard version and the customizable one that you can edit to include your own college’s findings. 

While we were tinkering under the hood, we took the opportunity to clear up a few of the data points that were unclear in the original. Thank you to those users who scratched their heads and pointed how we could make it better.

We’d love to hear in the comments about how you are using the infographics on your campus.

I was happy to see so much attention in the twitterverse regarding the 2012 Freshman Survey–in particular the finding that more than ever before, students are going to college to get good jobs and make more money. One of the tweets from #AACU13 that stuck with me over the past week is a tweet from Ellen Schendel (@LNSchen):

“Most CAO’s report using assessment tools (NSSE, CLA, etc) but only 20% believe they are making good use of the data #aacu13”

Now that you have your CIRP survey findings-both at the national level and for your institution, how will you use them? Are there things that help your campus make better use of survey data and things that should be avoided? Here are five that come up often.  If you have more, we would love to hear them.  Please add them in the comments section below.

1) Too much demographic information. I’m in no way saying basic information about the survey is unimportant, I know it’s critical context for the results, but there is no faster way to kill interest in what you have to say than by starting off a presentation with how many students the survey was sent to. Put basic demographics, response rates, information about timing of the survey, incentives, etc. online or in an appendix, making certain to point out any important caveats that should influence discussion of the results. Let it be known where to find that information, and keep it up to date.

2) Overly complex statistical explanations. It is your responsibility to conduct statistically robust analysis, and your campus is counting on you for accurate and reliable data, but the details of how you limited the dataset, weighted the responses, dealt with missing cases, etc. are not the story here, the results are. Your campus wants to know what the survey reveals about where the institution is doing well and where it can improve. Your job is to provide results and to facilitate discussion that helps shape plans about how to improve. Lead with the results, and what they reveal about your students.

3) A dissection of every data point, in survey order. I work in surveys, so I think every question is important and worthy of discussion. It took me way too long to realize other people on campus did not share this view. Presenting survey results is not just about sharing numbers, percentages and significance levels. Part of our responsibility is to turn survey results into information the campus can use, and that starts with making informed choices about what to present and what not to. If a percentage has not moved in years, it might be time to stop reporting on it and move on.

4) Failing to connect data to issues on campus. What issues do you know will capture interest on campus? If the entire campus is talking about campus climate and diversity, don’t present findings on intended majors.  Lead with what the campus is interested in talking about.  Good presentations tell a story with data. The campus can use data to understand who their students are, what they do, and who they will become. Approach your survey results looking for the stories you can tell. If you think of reporting your results in terms of taking several items and connecting them together to make meaning, that not only will involve the campus in a deeper discussion, hopefully it will get them coming to you and asking what other stories can be told with the data you have.

5) Be a facilitator, not a gatekeeper. I visit a lot of campuses, and I frequently hear faculty and administrators say they did not know they had data on a topic, or talk about how difficult it is to get results from the office that “owns” the data. View the data you are providing as a foundation for talking about students—their experiences, their needs, their opportunities. It’s really just the start of the process of improvement. Share the results across campus with anyone who might be connected or interested in the issue or topic. Answer their follow-up questions, connect interested faculty, departments, or offices over email and keep talking about what the result might mean, what other information they might want or need to better understand the issue, and what actions they might want to take as a result.

With the release of The American Freshman: National Norms, Fall 2012, we know many of you will be thinking about how to put both the national findings and your own institutional results to good use on campus. Here’s how a few institutions are using their freshman survey results to inform and improve campus practices.

Ellen Peters, director of institutional research at the University of Puget Sound, on CIRP data and faculty development:

Knowing who’s in the classroom
“It’s easy to make assumptions as to who is sitting in our classroom. CIRP data shows the breadth of opinions in the classroom and how students think of themselves. Faculty are leveraging CIRP data to find out more about students and their feelings on certain issues, such as politics, social issues and diversity, that that can be hard to talk about in the classroom. We found that our students have very divided opinions on certain issues. CIRP data gives faculty a greater confidence to engage with students.”

Teaching to the students
“One CIRP construct we’re very interested in is academic self-concept. Knowing that our students rate themselves as above average on academic ability and writing ability but that they don’t view their mathematical abilities as strong is helpful for faculty in determining their approach to teaching. There’s a difference in teaching students who feel they know a lot as opposed to those who feel they have a lot to learn.”

Faculty using results
“We share CIRP data with faculty so they see it as applicable to their work. As a result, the visibility of the IR department and its value to the institution has increased. More and more faculty are coming to us and asking for CIRP data. Not only does this allow them to get data that is meaningful to them but it has allowed IR to play greater role on campus.”

Art Heinricher, dean of undergraduate studies at Worcester Polytechnic Institute, on using CIRP results to discover what motivates students:

Myth busting
“We use CIRP data to address some of the assumptions people have about WPI. There’s a misperception that most of our students are first generation students from blue collar backgrounds. That might have been true in the 70s but that hasn’t been true for a long time. Now about 50 percent of our students have at least one parent with an advanced degree. The data gives prospective students and parents a better and more honest picture of who our students are.”

What motivates students
“For a long time we have been very interested to find out what motivates students. We have been using CIRP data to see changes in this for our incoming student population over the last 10 years.”

Joanna Royce-Davis, dean of students at the University of the Pacific, on using CIRP data to improve first-year programming:

Improving the first-year program
“Results from CIRP surveys have helped inform first-year programming efforts and particularly in the development of the university’s leadership program. We discovered that when we compared our CIRP data to that of our peer institutions, our students are less likely to identify and recognize their leadership abilities. We found that students interpreted the idea of leadership as positional—as a title or a label—and not as a quality within themselves. We realized that students need to recognize their potential as leaders before we can begin to work on their capacity to be leaders. Leveraging CIRP data from the social self-concept construct, we addressed this need by incorporating an extended, off-campus orientation program focused on leadership development into our first-year program.”

“Our first year program also has a virtual component known as the Pacific One Word Project. As part of this project, students are invited to choose one word to describe themselves. They are then photographed holding their chosen word. It’s aimed at developing these students’ social and emotional competencies and gives them a powerful sense of self, and the results have been meaningful. We are already seeing greater activism, greater clarity of purpose and sense of self-efficacy. We are also seeing students who are more articulate about their leadership identity development.”

Sharing results with students
“Sharing data with students has proved very beneficial. We have found that students want to know about the conversations taking place or not taking place on campus. It also serves as a form of social norming, and provides students insight into the behaviors and values of their peers. This can help students determine if they need help, especially around issues of stress, which affect many of our students.”

Like the rest of the nation was, we at HERI were shocked and greatly saddened by the senseless violence in Newtown.

Since we are in the business of uncovering and promoting what those of us in higher education believe, I turned to the CIRP Freshman Survey to see what it could tell us in this case.

About two-thirds (67.5%) of students entering four-year colleges in 2010 agreed that “the federal government should do more to control the sale of handguns.”

In the 2011 HERI Faculty Survey we learned that three out of four college faculty believe that “colleges should be actively involved in social problems.”

As evidence of the collegiate  interest in social problems, to date two letters urging President Obama to take action have been circulated and signed by college presidents. One letter, written by Emerson College’s M. Lee Pelton, currently has the backing of 209 other college presidents, and a joint letter  from Lawrence M. Schall (Oglethorpe University) and Elizabeth Kiss (Agnes Scott College) has over 300 signatures.

All places of education, elementary through post-secondary, should be free from violence. Those of us with voices that can be heard owe it to all who have encountered such violence to do all in our power to make it stop.

A couple of weeks ago I received a phone call from an Assistant Vice President of Enrollment Management and Dean of Student Affairs at a local college. The school is a long-time user of the CIRP Freshman Survey, and she was calling with what she thought was an unusual request. She chairs the survey committee on campus, and without a dedicated IR office, they were looking at all the surveys they do, and trying to figure out what to do with the data. Since the school was not far, she asked, would I be willing to come out and meet with the survey committee and talk with them about CIRP, and about surveys more generally?

It took me about 5 seconds to say yes.

The survey committee wanted to talk a bit about the importance of survey research, and so we started by walking through the kinds of information that is found on each of our CIRP surveys, and how that allows a school to look longitudinally at the development of its students, both cognitively and affectively. At the institutional level, you can use this information to help provide a clear sense of the student experience. Knowing the frequency with which students engage in practices associated with student success like learning communities, talking with faculty, engaging with students from different backgrounds and experiences, etc. can help a school understand which issues need the most attention, and serve as a compass by which to monitor improvement over time.

When I asked, faculty had plenty of good ideas of how they might use the data to answer specific questions they had about student learning at the institutional level—can students apply skills and learning in new contexts, are the developing the skills and knowledge to be effective in the next courses in this sequence, in their career, in life in general. But faculty members also wanted to know about how survey results were valuable at the department or classroom level. Survey results at the department level are useful in discussions about what students should be able to know to do, and to look at the development of these skills as students progress through the major. It allows a department to come together and reflect on goals, methods, pedagogy and curriculum. At the classroom level, survey data can be informative in looking at specific initiatives, for example the use of problem-based learning, collaborative learning, or classroom technology.

We ended by brainstorming a bit about how to disseminate findings to the larger campus, and in doing so how to structure those discussions to provide time and space to reflect the findings, to deliberate about the meaning an implications and to use the information to design more effective programs or to change policies all with the same goal– to support student learning.

I hope more schools will contact us about using the survey results on campus. CIRP staff is more than ready to help schools work through the issues they face in using the results of our surveys face to face (and we really enjoy it).

We’ve been thinking a lot about the YFCY lately, and how we can help schools best communicate their results. Because the first year of college is such a big deal as it relates to retention, campus climate and overall students’ sense of belonging on campus, we decided to give the YFCY a little extra TLC this month by launching the YFCY customizable infographic, our first-ever customizable tool.

It allows institutions to enter their results next to the national results and compare the two. We hope to do this for future infographics as well, but we chose to start with the YFCY as we think it’s a perfect vehicle to demonstrate the many ways we are trying to make the findings easier to discuss on campus.

We’re also trying to get the word out about another great tool for YFCY users—a custom PowerPoint report that we create specific to an institution’s results. We know it can be challenging, even overwhelming, to receive so much data and then figure out how to make sense of it. That’s why we developed this tool–to help institutions clearly and easily communicate their YFCY results with others on campus and start a dialogue about how to use their results to improve first-year programming for students.

It’s available as a single-group report that displays data for your institution or comparison group or as a double-group report that compares data from two groups.

National data is certainly helpful in providing a comprehensive portrait of first-year students, and that can be accessed free of cost here (under PowerPoint Executive Summary Report). But for institutions that want to take a closer look at their individual results and how they compare on a peer or national level, these individualized tools can be very helpful and can be ordered here.

We are always looking for ways to customize the survey experience for users as we recognize that no two schools are the same. We hope tools like these help fulfill this goal. We’d love your feedback and suggestions.