2003 Self Study: Student Research

2003 Self Study

Self Study: Student Research

Introduction

This essay covers our work on the educational effectiveness self-study topic undergraduate participation in research. For a student-centered organization of the investigation, we follow the chronology of opportunities available to a typical student and base that discussion on an existing presentation.

In the UC Davis institutional portfolio essay on Standard 2, Achieving Educational Objectives Through Core Functions, we describe a process to coordinate and improve the multiple advising efforts that take place on campus and the subsequent formation of the Undergraduate Advising Council (UAC). As a result of a UAC recommendation, the Student Affairs Advising Services office organizes an annual half-day workshop for departmental advisors, staff advisors, and outreach coordinators, virtually all of the campus employees with responsibility for student advising. For the spring 2002 workshop, the coordinator of the conference, Dennis Beardsley, asked the Vice Provost-Undergraduate Studies (VPUS) to give a presentation that would review and highlight various undergraduate opportunities. The campus already has a fairly extensive website on Undergraduate Research Opportunities, but the presentation was intended to contextualize the institution's commitment to undergraduate research so that advisors would have a comprehensive understanding of the range of offerings. Vice Provost Turner developed a PowerPoint presentation, and we will use that presentation to organize this essay. An evaluation followed the workshop and participants gave it a 3.7 on a 4.0 scale. However, the goals for the workshop differ from those for the educational effectiveness review. In the essay that follows, we will refer frequently to the kind of evidence of student learning required by the WASC process.

To establish a context, the presentation was introduced by the following quotation from the UC Davis Philosophy of Purpose that, as the institutional portfolio explains, functions as the campus mission statement. The relevant passage reads:

UC Davis has a history of focused attention on undergraduate education. The central elements of a liberal education-the arts and languages, history and philosophy, and the sciences-offer the opportunity for a broad general education combined with specialization in a scholarly discipline. Coupled with this are manifold opportunities for personal development through programs for academic enrichment, including undergraduate research, work-learn experiences, and extra-curricular student life.

Preparatory

In their initial encounters with graduating high school seniors and freshmen, the staff and faculty advisors counsel the students to take the classes required for their majors and to begin their general education courses. As we noted in our institutional portfolio, we have invested heavily in gateway courses such as the Physics 7 series and Chemistry 2 series that are required of so many of our students. The Standard 4 reflective essay in the UC Davis institutional portfolio reviews the status of the educational effectiveness of these curricula. The successful tackling of a solid first year curriculum is the rather obvious first step on the path to undergraduate research.

In addition to establishing a foundation for their majors and embarking on their GE requirements, UC Davis freshmen can pursue at least three academic programs that introduce them to the research arena; these are:

  1. Freshmen Seminars
  2. Integrated Studies
  3. Davis Honors Challenge

1) Freshmen Seminars

Freshmen seminar programs (FRS) certainly are not unique to the UC Davis campus. Our own FRS program began in the late 1980s under the auspices of the Teaching Resources Center (TRC). The list of courses offered reflects the breadth of UC Davis faculty research and interests. From courses on the history of chocolate to a primer on taxation policies, 15-20 students and a faculty member spend the quarter analyzing one significant topic. Most seminars provide students with opportunities for individual and collective oral presentations and written work. We do not claim that the work produced in these courses in any way "counts" as undergraduate research. We do make the case that students who have witnessed a faculty member's engagement with a focused topic and who have had a chance to interact with and prepare assignments in one of these courses is preparing himself or herself to undertake serious undergraduate research.

We were unable to significantly increase the number of FRS seminars offered following the financial downturns of the early 1990s. We did not modify faculty rewards for teaching them nor implement the staffing changes in the TRC that would have enabled us to handle more of them. At the Chancellor's Fall Conference on Undergraduate Education in 2001, the assembled participants uniformly encouraged the administration to both augment the faculty rewards and increase staff support. This recommendation stemmed from the positive experiences of the student and faculty participants in the fall conference, and it was also fueled by discussions of the Boyer Commission's Reinventing Undergraduate Education: A Blueprint for America's Research Universities. The research that informed that report and subsequent surveys on it reported by the director of the Reinvention Center at our fall conference provided sufficient evidence of educational effectiveness. Thus, to date, we have not conducted any Davis-specific surveys to determine if there is a connection between a student's enrollment in a FRS seminar and eventual participation in undergraduate research. If we can determine an empirically sound way to determine if there's a genuine connection, we will undertake such a study.

Student testimony on the value of FRS seminars is evident in an article in the on-line newsletter, Aggie Family Pack, April Trask, who's studying international relations, liked hearing from fellow students. "You can get the professor's opinion and the opinion of other students, whereas in lecture classes you're just taking notes as fast as possible. "Freshman seminars," Trask adds, "build skills for other classes. Like speaking. Research."

2 & 3) Integrated Studies (IS) and Davis Honors Challenge (DHC)

UC Davis has two unique freshmen programs that focus on high-achieving students. Both programs introduce students to modes of inquiry necessary for any sophisticated research project. Both programs connect small groups of students with faculty who have distinguished research profiles. DHC and IS have both been examined in recent campus documents referred to as the Harrison and Shackelford reports.

Dating back to the 1960s, Integrated Studies (IS) is the older program. Regents' Scholars and a select group of other students with high academic indices comprise this group of about 70 students who live in the same residential hall and take common GE courses and a seminar during their first year. Further, the Shackelford report indicates just how successful the IS students turn out to be. Note that they graduate in 4 years with much higher GPAs than their counterparts who graduate in 5 years.

The Davis Honors Challenge (DHC) is unique in that any student accepted to UC Davis can apply to the program and will be admitted to it if s/he demonstrates enthusiasm, leadership, and commitment. Unlike IS, the DHC has formal second, third, and fourth year offerings. DHC stresses active learning and collaborative student projects, thus preparing participants for research activities. In AY 2002-2003, the DHC will undergo its first administrative review.

Independent Studies

The UC Davis general catalog lists a variety of course numbers available to students to enroll in "independent studies." Most departments use "99" for lower division and "199" for upper division research courses. According to our annual report to the legislature on undergraduate instructional issues, 4147 students enrolled in these courses during the past year. These courses allow students to pursue a research topic in depth under the guidance of a faculty member. Currently UC Davis is participating in a University of California system-wide study of undergraduate courses that enable students to pursue research projects.

Sponsored research experiences

Sponsored undergraduate research programs proliferated on the UC Davis campus in the late 1980s and early 1990s as vehicles for moving women and underrepresented minority students into the graduate school pipeline. California's Proposition 209, eliminating race as a qualifier, caused us to modify state-funded programs in order to accommodate a broader range of students. Most members of the campus community applauded our ability to extend these programs to a larger student population. Unfortunately, the funding did not increase when the mandate was issued. Some of these programs are very specific to disciplines and are funded by federal or foundation sources. An excellent example here is CAMP (California Alliance for Minority Participants) that is funded by the National Science Foundation.

The Biology Undergraduate Scholars Program (BUSP) is worth examining because it is Davis-specific, and it was one of the campus's first sponsored projects for undergraduate research. Thus the faculty and staff who developed the subsequent campus programs were quite familiar with BUSP and sought to duplicate its strengths when possible. BUSP is also a particularly good choice for us because our academic plan calls for continued FTE investment in the Life Sciences, and almost 4,000 of our students major in one of the Biological Sciences, more than at our sister UC campuses. Finally, BUSP is a rather expensive program that has had a break in its external funding commitments. When the campus administration decided to continue the program with internal funding, the positive outcomes data described below played an important role. A 2002 report on the program states the problem as follows:

A campus study of the issue in the mid-1980s found that although 11% of freshmen biology majors were from minority groups, the proportion of minority groups decreased to 7% by graduation: a 36% greater attrition than that of the majority group students (Villarejo and Tafoya, 1995). Among this pool of minority biology graduates, relatively few were good candidates for graduate or professional school: the fraction of minority graduates with GPAs of B (3.0) or higher was about half that of non-Hispanic white graduates.1

BUSP was designed to close the gap between majority and underrepresented minority students. Its major planks include "supplemental academic instruction in basic chemistry, calculus, and biology; academic and personal advising; and practical experience and financial support through instruction in research laboratories." 2

For the purposes of this essay we are most interested in the undergraduate research component of the BUSP program. The authors compared the students accepted into the BUSP program who chose to pursue undergraduate research experience with BUSP students who opted not to pursue research. Thus the "researcher" and "non-researchers" below both refer to BUSP students. Note that the results of the statistical analysis in the study are expressed in terms of odds rather than probabilities. The initial results sound quite compelling:

Student researchers have 2.4 times the observed odds of graduating from UCD in any major than students who did not participate in research. Research experience is even stronger in the biological sciences: student researchers had 4.1-fold greater odds of graduating in the biological sciences, and 7.3-fold greater odds of graduation in biology with a 3.0 or greater cumulative GPA, than non-researchers. (13)

The authors note, however, the possible weakness of this conclusion. The actual causal relationship is unclear. It could be that the stronger students are those who are motivated to select a research path. Using a logistic regression analysis, the authors documented a reduced relationship between research and graduation but a meaningful connection nonetheless:

...A 1.9-increase in the odds of graduation from UCD in any major, a 3.9-fold increase for graduation in the biological sciences and a 6.6-fold increase in graduating in the biological sciences with a 3.0 or higher cumulative GPA (Table 7).

Even though the authors acknowledge that some of the evidence may be thin because of students self-selection and other factors, the high achievement rate of BUSP students and the solid outcomes research conducted by the faculty sponsors has led the campus to support the continuation of BUSP during a time when its external funding has become limited.

The second sponsored research program we will review is Mentorship in Undergraduate Research in Agriculture, Letters and Science (MURALS). At its inception, MURALS was intended to fill a gap in research opportunity programs for UC Davis minority students. Most of the "hard" sciences and Engineering departments had successfully sought external funds to develop programs for their students. However, no external agencies or foundations were clamoring to increase the number of underrepresented humanists and social scientists. In the pre-209 climate of the University of California, it was possible to dedicate funds to such purposes, and the campus successfully applied to the Office of the President to secure funds to underwrite a campus program to serve the needs of students in the humanities and social sciences. Within a few years of its implementation, the program expanded to include student majors in the College of Agricultural and Environmental Sciences. MURALS students express an interest in pursuing post-graduate work, are of junior standing, have a 3.0 minimum GPA and must make a two-quarter commitment to doing an undergraduate research project with a faculty sponsor. Faculty sponsors must make a two-quarter commitment to the students. The students are co-mingled for quarterly research presentations and are strongly encouraged to showcase their research in one of the venues that will be described in the next section of this report. Modest stipends of $500 per quarter per student and $250 per faculty member provide an incentive for participation. MURALS, like BUSP, faced financial uncertainty this year, but the evidence of its educational effectiveness was instrumental in the decision to continue support in spite of a precarious budget situation.

One of the most striking anecdotal proofs of the success of MURALS resides in the identity of the present faculty director of the program. Milmon Harrison, Ph.D, was himself a UC Davis undergraduate who participated in the MURALS program in the early 1990s. A community college transfer student, Harrison worked with faculty in African-American Studies and Sociology. He successfully competed for a multi-year graduate fellowship to UC Santa Barbara. He completed his Ph.D. there in 1999. The African and African-American Studies Department aggressively recruited him, and he joined the UC Davis faculty in 1999. He has a book forthcoming from Oxford University Press and now oversees the MURALS program on campus.

More comprehensive evidence of the success of MURALS is detailed in the 2000 Ph.D. dissertation, "Making a Difference: The Effects of an Undergraduate Research Mentorship Program on the Production of Minority Scholars" by Gail Ann Martinez. Martinez surveyed MURALS alumni as well as a non-MURALS cohort group identified with the assistance of staff from Student Affairs Research and Information (SARI). Like the authors of the BUSP study, Martinez notes that the self-selection of the MURALS students weakens some of her conclusions.

Nonetheless, her survey revealed that the majority of former MURALS students pursued graduate study. Seventy-two percent of the group enrolled in some form of graduate education. She notes that:

...Their enrollment in doctoral studies appeared to be where participation in the program "made a difference." The program sample resulted in 10.7 % enrolled in Ph.D. studies compared to the minority comparison sample at 1% and the non-minority comparison sample at 6.8%. Students' comments document their participation in the program being directly related to their positive academic outcomes. (ix)

In her final chapter, Martinez notes that MURALS, like so many other undergraduate research programs, assumes that motivated students will take advantage of attractive opportunities and will be better prepared to pursue academic careers. Consequently, it is difficult for a competent researcher to pinpoint with certainty that a particular program is responsible for the positive outcomes of the students. Martinez posed an open-ended question designed for the alumni to share their own conclusions about the role the program played in their choices. She also evaluated unsolicited comments that accompanied the returned surveys. She notes, "For the most part, the majority of these comments (85%) indicated that the experience in the program was positive and for many, was related to their postgraduate choices."

Both BUSP and MURALS are now available to a broader range of students than the minority students the programs were designed to serve. Subsequent outcomes research will shed light on their educational effectiveness with these expanded student participant pools.

Further evidence of UC Davis's commitment to undergraduate research can be seen in the fact that graduate student participants in the 1996-1997 UC Davis Program in College Teaching co-authored one of the first articles entitled, "Successful Mentoring of Undergraduate Researchers: Tips for Creating Positive Student Experiences." They collected data from many students on campus including MURALS and BUSP students as well as recipients of President's Undergraduate Fellowships (PUF). This is a competitive campus program, one open to all students, provides awards of up to $2,000 to support student research under the guidance of a faculty mentor. It too encountered financial difficulty in recent years. However, the campus identified an alternative revenue stream to continue to support the PUF awards. As the Shellito et all article attests, well-mentored undergraduates thrive in research situations.

Off-campus opportunities

By the time they are juniors or seniors, many UC Davis students are ready to temporarily leave the safety zone of the campus to pursue research in environments particularly appropriate to their majors. The programs include the Bodega Bay Spring Quarter and Summer Program, a long list of study abroad programs available are coordinated by the Education Abroad Center and the UC Davis in Washington Center. Both the UC Davis Washington Center and the UC Davis Internship & Career Center match students with research-oriented internships beyond the borders of the campus.

Each academic quarter 45 UC Davis students can spend 10 weeks in a Washington-based internship. They live together with students from our sister campuses in the newly constructed UC in Washington building. In addition to an internship, they must enroll in both an elective course and a research course. The Washington Center underwent an administrative unit review in 2001-2002. The campus committee examined the research component of the experience including a selective reading of student papers. In the past, faculty associated with the program have noted certain student ambivalence towards this requirement. During the quarter itself, many students complain about the time and energy the paper takes away from the internship and the opportunity to explore the DC area. However, alumni often express gratitude for having had to do the paper. The reviewers noted, "When asked whether the research paper requirement should be continued, 80% of the students agreed, 58% strongly agreed." We have not conducted any comprehensive follow up to the program to determine how large a factor the program is in students' career and professional school goals. We certainly have anecdotal evidence that they thrive after a quarter in Washington. At least seven of the 2002 Undergraduate Research Conference papers were from DC students. Many of these papers have been published in Explorations, the Undergraduate Research Journal. Alumni who now work or attend graduate school in the Washington area are frequent visitors to the new center and events held there. Those who work in Washington now recruit UC Davis interns for their offices.

Showcasing undergraduate research

The final stage in the research process is dissemination, and UC Davis encourages its students to showcase their work. Again, juniors and seniors are the students for whom we stress these opportunities. But any student can submit an abstract to the Undergraduate Research Conference (URC), now in its 14th year. They must also show faculty sponsorship for their proposals. In 2001, 81% of student presenters in the URC strongly agreed that they "learned and benefited" from participation in the conference. In their evaluation comments they stressed the confidence-building aspects of the experience. Organizers are particularly pleased with students respect for the scope of campus research activities. Many students reinforced the comments of one who noted, " I had a chance to learn about the research other students were doing on campus." The Campus Writing Center offers workshops in abstract writing for students, and those who opt to present a poster can also take a free workshop on computer-generated posters. Each year approximately 150 students participate in the conference. A selection of posters can be found here.

In reviewing the disciplines represented at the conference, we note the highest participation from Life Sciences students. For the past several years the coordinating committee has endeavored to increase the participation of students from the humanities, fine arts, and social sciences.

With support from the Office of Research and the Vice Provost--Undergraduate Studies, the Advising Services office has funds available to send some students to national conferences on undergraduate research or other conferences that will accept undergraduate papers. Every year, two students are selected to represent UC Davis at UC Day in Sacramento.

Publication is probably the most significant mode of research dissemination. Many faculty members in the sciences include their undergraduate mentees as co-authors on their papers. In the Humanities and Social Sciences, student contributions are often noted in acknowledgments. Students are encouraged to publish their own work in campus journals such as Explorations and Prized Writing. The latter publication is housed in the English Department, but students from throughout the campus can submit their best papers. The Department has an outreach program whereby copies of the journal are sent to regional high schools so that English teachers can have a better understanding of the qualities of high caliber undergraduate student writing.

The faculty editor of Explorations eloquently attested to the importance of undergraduate research by saying:

Beyond the confines of the classroom and the covers of textbooks, beyond the lectures and the laboratories and the assigned readings, the work of the research university seeks to impart something more---a sense of the special enthusiasm induced by the exploration and pursuit of knowledge.

Commentary

From the freshmen seminars in their first year to UC Day in their senior year, motivated UC Davis students have ample opportunities to progressively hone their research skills. In reviewing this list of opportunities we can see several areas for improvement. But perhaps we should begin this section by noting what areas the students and alumni have drawn our attention to through instruments developed and disseminated through Student Affairs Research and Information (SARI).

In the spring of 2001, SARI administered a census survey of the entire undergraduate population.3 Of the 18,796 undergraduates polled via email, 9,998 (or 53 percent) responded. The survey asked, among other things, about the direct working relationship between undergraduates and faculty members. The question posed was, "In your experience at this institution during this school year, about how often have you worked with faculty on research or creative projects?" The response "very often" was cited by 3 percent of the students; another 6 percent responded "often." Another 16 percent cited an occasional research relationship whereas 75 percent of the students responded "never." On a four-point scale, the mean score of the respondents was 1.37. The survey limited students to talking about a research relationship during only a single academic year of a student's four (or five) year academic career. And, although this survey suggests the progress that faculty need to make, the survey is encouraging in finding that seniors experienced many more research opportunities than did lower division students. On the four-point scale, the senior mean was 1.6 versus 1.2 for freshmen and sophomores and 1.3 for juniors.

These data should be interpreted in light of a second survey of students, that is, of alumni. Every three years, SARI conducts a survey of the previous year's June baccalaureate recipients, one year after graduation. Some questions about the degree of undergraduate research opportunities can be tracked over time while others are pertinent only to the most recent survey of the June 1999 baccalaureate degree recipients.4 As the report notes, "The proportion of undergraduates participating in research activities with faculty at least occasionally has increased appreciably over time, from a low of 40 percent in 1990 to a high of 61 percent in 1999." This figure - 61 percent as opposed to 25 percent reported in the census survey - presents a significantly rosier picture of research at UC Davis. We believe this second figure more accurately represents the status of research at UC Davis for three reasons. First, the 2001 survey asked about research opportunities during a single academic year rather than over a student's entire academic career. Second, the 2001 survey included lower division and upper division students as opposed to the alumni survey, which included those who had earned baccalaureate degrees.

Thirdly, we also believe that the alumni survey is a more accurate reflection on student research activities as alumni are better able to reflect on what constitutes a research experience and what experiences are valuable in the current employment, whether as graduate students or as a member of the workforce. To provide just one example, an unsolicited email from a 1999 alumna to a member of the Political Science faculty recorded: "I can honestly say that your [research] seminar was one of a kind. In fact, it mirrors [my employment on] the Council [of Foreign Relations] to a tee. Please tell your students (when they start complaining about the 30 pages) that if they are interested in working at the CRF/and/or Brookings they will be doing the same kind of discussions/writings as your class outlines. You are preparing them 2 years ahead of time." Students currently engaged in the academic process may not recognize research tasks as such but may, as this alumna, later acknowledge the usefulness of classroom activities.

The alumni report details more fully, for the first time, several dimensions of undergraduate research activity. Seventy-one percent of the respondents reported that they carried out an independent research or creative project for class as opposed to the 61 percent who reported working with faculty on research or creative projects. Moreover, that figure is higher for students who continue on to post-graduate education, with 77 percent of the post-graduate students carrying out an independent research project versus 67 percent without a post-graduate career trajectory. Sixty-seven percent of the post-graduate students reported working with faculty versus 57 percent without a post-graduate career trajectory. Although we believe that our mission is to introduce all students to research and the production of new knowledge, there is a self-selection process present, where students with a strong desire tend to select these research opportunities.

The alumni survey also asked about student satisfaction with the undergraduate research experience; "How satisfied are you now with the opportunities for independent study, research and created projects at UC Davis?" One half of the respondents indicated that they were very satisfied or satisfied with these opportunities, while 39 percent were neutral, and only 11 percent dissatisfied or very dissatisfied. A second aspect of satisfaction queried the degree of student preparation in research skills; "How well did your education at UC Davis prepare you for independent study, research and other forms of original, creative work?" A full 50 percent described the preparation as very good or more than adequate; 38 percent described the preparation as adequate, and only 12 percent felt that their preparation was less than adequate or poor.

Although the alumni survey has been an ongoing component of our evaluation tools for many years, the survey of 1999 baccalaureate recipients, in preparation for the WASC accreditation, was designed to survey more fully the research experience of undergraduate students. We plan to continue these questions to provide feedback on these issues on a continuing basis.

While continuing with student assessment via alumni surveys, we propose to foster the research component of undergraduate education by including criteria in the review process that focus on research in the classroom as well as the small group or one-to-one research relationship between students and faculty. Assessment of the role of research in undergraduate education must incorporate both dimensions of assessment, one that focuses on the student, the other, the faculty member. In particular, we recommend that the "student survey of the major" required in the review should incorporate a question on research opportunities.5 Second, department reviews should include a criterion in individual faculty evaluations that addresses how faculty members incorporate research into the curriculum. Implementation of these recommendations would help ensure that every faculty member addresses this curricular goal. The program review process provides an excellent opportunity to communicate the results of SARI's student surveys and to expand current evaluations of department plans for incorporating undergraduate research into the curriculum.

Conclusion

A part of the mission of a research university is to introduce students to research and to inspire in them a passion for discovery, and the University of California at Davis is well placed to pursue that mission. This has been an unspoken ethos on the campus in the past, and by selecting undergraduate research as one of the WASC self-study topics, we chose to become more self-aware of current practices as well as to develop new evaluation tools and to encourage faculty efforts by disseminating best practices. We recognize that we are only part-way down the path to ensuring that every undergraduate student engages in a meaningful research project. Nonetheless, we feel that there is widespread faculty support as well as student recognition of the importance of these undergraduate educational goals.

1 Amy E.L. Barlow and Merna Villarejo, "Making a Difference for Minorities: Evaluation of An Educational Enrichment Program. Unpublished paper April 19, 2002

2 Barlow and Villarejo, pp3-4.

3 Carol F. Wall and Steve P. Chatman, "UC Davis Undergraduate Educational Experiences," http://www.sariweb.ucdavis.edu/davisqe2q/OctoberReport/FallConfUndergradEducExp9_01.pdf

4 Gillian Butler, "Undergraduate Research Activities at UC Davis. Survey of June 1999 Baccalaureate Degree Recipients," http://www.sariweb.ucdavis.edu

5 See TPPRC Student Evaluation of Major Program