Academic Assessment

May 2009 – Report on the Smith College Alumnae Survey

In 2008, an online Smith College Alumnae survey was offered to 1252 graduates of the college. Representing graduation years 1974 to 2007, 786 alumnae (62.7% of invitees) responded. The attached report by David Lopatto, Grinnell College and Carol Trosset, Hampshire College, outlines the results of the survey specifically from the perspective of evaluating the benefits of student summer research experience. Conclusions include:

  • Former summer researchers report greater involvement with teaching, research, and science in their current vocation.
  • Former summer researchers recall more involvement with post-graduate education and science jobs than the comparison group at the time of graduation and 5 years after graduation.
  • Former summer researchers report greater involvement with science jobs than the comparison group 10 years after graduation.
  • Former summer researchers report more Ph.D.s than the comparison group.

January 2009 – HHMI Summer Research Assessment Project

The final report of the qualitative assessment phase of our HHMI sponsored Summer Research Fellows (SRF) Program is complete. The report, Qualitative Evaluation of the Summer Research Program at Smith College, is intended to inform the development and implementation of a broader survey that will be distributed shortly to 1400 science majors distributed over 14 classes of Smith graduates. The project plan was developed in collaboration with Elaine Seymour, Center to Advance Research & Teaching in the Social Sciences, UC Boulder, and David Lapatto, Department of Psychology, Grinnell College. Along with Smith College, both are heavily involved with HHMI in undergraduate science education assessment. With Dr. Seymour’s retirement, the qualitative project was taken over by Liane Pederson-Gallegos.

With completion of the qualitative phase, the project’s center of gravity will move to Smith (Science Center and Institutional Reach) and David Lapatto’s program. Our results will be an anonymous element of Lapatto’s national initiative.

The executive summary of the report is reproduced below. You can find the full report here.

January 2009 Update: University of Colorado’s final report

Executive Summary

This is a formative evaluation report that summarizes participants’ feedback about the Smith College Summer Research program. It is based on 59 in-depth interviews with participants in Smith College’s summer research program from 1967 through 2006. The goals of the interviews were to identify salient aspects of the summer research experience from the perspective of the participants, and then to examine these issues through conversations with the interviewees. Findings from the interviews will facilitate the construction of more meaningful, valid survey questions, including questions about issues that the investigators would not otherwise have known to pursue. The in-depth interviews were necessarily limited in number so efforts were made to determine the range of issues and participants’ attitudes about these issues, rather than to establish how far the findings could be generalized across all of the program participants. The quantitative survey to follow has that task.

Findings

While the number of interviewees is too small to provide generalizations, it appears that the summer research program evolved considerably over time, becoming more organized and addressing problems that came up earlier. Problems mentioned by students who participated early on in the development of the Smith College Summer Research program were not mentioned by later participants.

Students gave overwhelmingly positive feedback about the summer research program. The gains reported were consistent with findings reported in other recent research and evaluation studies investigating the benefits to students of participating in undergraduate research programs in the sciences. There were, however, three additional findings specific to Smith College. These findings were: the benefit of students’ summer research projects evolving into their subsequent senior thesis projects; the benefits of conducting in-depth research in a supportive women’s college environment; and the alignment of the summer research program with college-wide efforts to promote hands-on research opportunities for students.

The summer research experiences were clearly highlights of their time at Smith College, and yet some of the experiences blended with benefits they gained from research promoted at Smith College through their classroom activities. The summer research program expanded on these opportunities, at times quite literally, as when research initiated in the classroom was developed into summer research projects, providing the opportunity for more in-depth work. In addition, students’ in-depth summer research often developed into honors thesis projects

There were five types of benefits students reported gaining as a result of their experiences with the summer research program. These were:

  • Research skills
    • Including lab techniques, writing skills, literature review skills, learning to work independently, presentation skills, and problem-solving skills;
  • Learning
    • Including enhancements in critical thinking skills, active learning skills, and an increased understanding of their particular scientific disciplines;
  • Self confidence
    • Resulted in raising students’ expectations of themselves, enhancing their perseverance, facilitating coping with particularly difficult challenges, and promoting learning difficult material;
  • Clarification of, and increase in, career ambitions
    • Promoted students’ academic and career-related achievements; and
  • Benefits of being mentored
    • Including greater identification with careers in science, and positively influenced the mentoring they subsequently gave to others.

Students expected (and alumnae respondents confirmed) that these gains would be transferable to their work, to graduate school and to life in general. The most significant effects alumnae reported persisting beyond graduation from Smith College were the elevation of their career ambitions and greater preparation for the demands of graduate school and subsequent careers.

Students believed that having ample time over the course of a summer to think about their science facilitated both deep conceptual learning and the development of critical thinking skills. These gains could not have been made in hands-on research in the classroom, as they would not have the extended time to work through their problems. In addition, on-site work that took place in various parts of the country, and occasionally beyond the U.S., were clearly extraordinary opportunities for students.

Most of those who had graduated from Smith College had quite a lot to say about both balance generally and two-body issues specifically. The overall finding was that work/life balance and two-body problems posed significant challenges for the alumnae. Overall, alumnae reported that their experiences at Smith College, and their summer research experiences in particular, contributed to feeling more empowered and better able to contend with gender discrimination than they would have been otherwise.

Students were asked to give advice to future mentors and students, as well as to the administrators of the program. Their advice to mentors most commonly concerned students’ need for clearer communication before, during, and at the conclusion of the project, including communication of faculty research advisors’ expectations of the student. Their most common advice to incoming students was to be fully engaged in their projects in order to gain as much as possible from the opportunity. Advice to administrators of the summer program and Smith College, in general, was to support the program and facilitate more opportunities for participants to interact with each other.

One way in which the range of positive to negative feedback was measured was by asking students to identify the highest and the lowest points of their experience. The high points were (in descending order of frequency):

  • Collegial work with mentor;
  • Learning gains;
  • Involvement in research;
  • Enjoyment of the experience; and
  • Personal growth.

Most of the students did not identify a low point. Of those who did, the low points were (in descending order of frequency):

  • Frustrations with research process;
  • Discomforts experienced during the summer;
  • Unpleasant aspects of research project; and
  • Need for more interactions with mentors and/or peers.

In sum, the interviewees were highly complimentary of the summer research program. The benefits alumnae took away from the experience were many and their suggestions for improvements were enhancements, as opposed to substantive changes, to the program. Virtually all of the interviewees said that, if they had the chance to do it all again, they would.