2003 Self Study: Integrative Essay

2003 Self Study

Self Study: Integrative Essay

Background

In July of 1999, Marge Callahan, Assistant to the Vice Provost, Sally Springer, Assistant Chancellor, Celeste Hunziker, Institutional Researcher, and Joe Kiskis, Professor of Physics and Vice Chair of the UC Davis Academic Senate were invited to represent UC Davis at a two-day WASC meeting in Berkeley, California. This meeting introduced a newly emerging accreditation process to campuses scheduled for WASC review. The new model of accreditation outlined at this initial meeting emphasized outcomes rather than inputs. It was intended to replace a highly structured process that forced institutions to adhere to a prescriptive set of 268 standards. In 1999 WASC envisioned sweeping changes, and the UC Davis team represented at the meeting recall hearing that virtually every dimension of the old process of accreditation was under scrutiny and likely to be overhauled.

Rather than pursuing a process designed to satisfy only an external set of demands, the new WASC approach was constructed so institutions could define their own needs and transform the accreditation process into one it could use to pursue its own goals. WASC also committed itself to like comparisons; research university team members would evaluate peers at other research campuses, faculty from faith-based institutions would scrutinize colleagues from campuses with a comparable profile. While WASC guidelines would have to satisfy the Federal Department of Education, the architects of the new process were confident that a viable system could meet both internal and external demands. Although the exact nature of the changes was still under discussion and yet to be approved by the Commission and the Department of Education, the WASC timetables indicated that UC Davis should prepare for its reaffirmation of accreditation. In the three and a half years since that have elapsed, UC Davis representatives have attended several other WASC-sponsored meetings and workshops. Dozens of steering committee meetings have been held on campus, and numerous campus constituencies have been engaged in the accreditation process. The campus faculty/staff newsletter has run many articles; the student paper has also documented the process. In this ostensibly paperless age, reams of paper have been expended upon multiple drafts of WASC-driven documents. "WASC update" is a common agenda item on numerous campus committees.

Many things have changed for the better since the 1999 meeting. UC Davis has reorganized the way its division of the UC Academic Senate oversees the undergraduate curriculum. Faculty have received and begun to implement a $150,000 Hewlett grant for general education improvement. The campus has approved a set of educational objectives and has begun the process of assigning metrics to them. These new educational objectives have stimulated an overhaul of the program review process. As our institutional proposal indicates, we have had the standard amount of leadership turnover in these years, but by a fortunate splash of serendipity, Callahan, Springer, Kiskis, and Hunziker are still in roles, albeit several of them with different titles, that connect them to the WASC accreditation process.

In December of 2002, UC Davis hosted the WASC capacity team. That sojourn allowed the visitors to follow up on issues embedded in our on-line institutional portfolio. Based on comments delivered at the December 5 exit interview, this team concluded that UC Davis has satisfied WASC's requirements for the capacity portion of the review process and is ready to move to the next phase, which is the educational effectiveness review. It includes the essays on educational technology and student research and this final essay that incorporates our perspectives on the whole of the WASC review process.

Impact

Stimulated by the questions posed for the WASC review, the steering committee began its deliberations by informally comparing undergraduate education practices on the UC Davis campus against a context of national trends. Like most card-carrying academics, we surveyed the literature. We quickly learned that Reinventing Undergraduate Education: A Blueprint for America's Research Universities, also known as the Boyer Report, is probably the most frequently cited report on the status of undergraduate education. Although it was published in 1998, it was familiar to only a handful of UC Davis campus administrators. Given its prominence in discussions about undergraduate education in research universities, it has proven to be a good tool for campus discussions. Our fall 2001 Chancellor's Conference extended it beyond the steering committee to a much broader campus audience. One of the first sessions of the conference evaluated UC Davis activities in terms of those identified in the Boyer report.

Thus the years of the current WASC re-accreditation process at UC Davis have coincided with other sources of increased attention to undergraduate education. This climate has led to many positive developments. The combination of the new WASC standards and the recommendations of the Boyer report has been most timely. While the WASC standards emphasize the development of a culture of evidence and a focus upon educational effectiveness, the Boyer report lays out specific suggestions for the particular strengths that research universities can draw upon to achieve that effectiveness.

Predating our current attention to the WASC process, there was a proposal to reorganize the Academic Senate committee structure. Senate leaders were eager to facilitate a more focused and organized approach to undergraduate education. The proposal to form a Undergraduate Council was first put forth in the fall of 1999. The Senate Representative Assembly ratified the final version in the spring of 2002. WASC provided a significant part of the motivation for positive action rather than further delay.

In September 2001, the Chancellor's Fall Conference was devoted to undergraduate education. The agenda for that meeting was, to some extent, influenced by the WASC priorities that we had set for ourselves. Some of the developments that followed the conference will be discussed.

A major issue that we identified at the beginning of the WASC process was our lack of formalized educational objectives. Thus a goal for the conference was to identify and develop a consensus on educational objectives. We were successful in that, and by April 2002, the Committee on Educational Policy endorsed the objectives. We were pleased by the December site team's positive comments on these objectives and the ways that we are beginning to use them.

Program review is one of the few existing campus-wide mechanisms by which we can hope to cultivate a culture of evidence and an attention to educational effectiveness and implement other positive changes. Thus the guidelines for program review are currently being revised by the Undergraduate Council to encourage those changes and to give the educational objectives operational significance.

Another item on the agenda of the fall conference was general education, an issue with which the campus has struggled off and on for many years. We detail GE developments in Standard 2 and the appendix of the institutional portfolio. With the formation of the General Education committee of the undergraduate council, we intend to align administrative initiatives such as the Hewlett grant with Senate priorities for undergraduate education. The December site team congratulated UC Davis on its progress in general education. In an Undergraduate Council meeting almost immediately following the exit interview, the chair of the GE committee articulated a cogent set of goals for upcoming GE committee meetings.

Participants at the fall conference recommended that the freshman seminar program be enhanced. As a consequence, there is now more flexibility on the number of units, and there is additional support that makes more seminars available. As our research essay indicates, these courses provide key curricular foundations for student research processes.

A topic that has been discussed extensively at UC Davis and at the fall conference is writing. Like GE, to which it is somewhat related, this subject has developed a mythic aura of intractability. Nevertheless, because it is such an important element of the Educational Objectives, the Undergraduate Council has decided to initiate an ambitious investigation of the current state of student writing as a foundation for possible recommendations to strengthen the requirements or modify instruction. Coincidentally, writing has also become a subject of discussion at the system-wide level of the Senate and administration.

Thus the impact of the WASC process has been both a generally increased attention to undergraduate education and also to some specific accomplishments in that area. In the exit interview, the WASC team recognized these positive developments and the foundation that they provide for the educational effectiveness review.

Common Themes or Issues that Emerged

The historical development of UC Davis led us to the present situation in which there is an unusually large number of students, faculty, facilities, and other resources outside of the traditional humanities and science core of a university. So in addition to the College of Letters and Science with its three divisions, we have the College of Engineering, the College of Agricultural and Environmental Sciences, and the Division of Biological Sciences. Then there are the graduate professional schools of veterinary medicine, medicine, law, education, and management. So that each of these parts can develop to its full potential, we have avoided excessive centralization.

Within our decentralized structure, individual initiative in creating and implementing innovative strategies is encouraged. The decentralized environment facilitates experimenting with new ways to achieve shared goals. While seeking and providing consultation is a community value that is always encouraged, within the decentralized organization, it is not a burdensome disincentive. Thus it is possible for an individual or a very small group of people with a vision to see it through to reality. Our capacity report offers many examples such as Science 1, the geology capstone course, and the Remote Collaboration Tool.

On the other hand, with this decentralized and sometimes ad hoc approach, there may not be as much attention given to dissemination or evaluation as might follow from a more rigid and centralized organization. Thus in our studies of both student research and educational technology, we happily discover a wealth of successful and impressive examples. However, the processes of systematic evaluation and dissemination that might lead to institutionalization of the benefits of the many success are somewhat uneven. Especially in the area of campus-wide policies on the evaluation of the educational effectiveness of initiatives, we have a ways to go. The presence of evaluative components that are built in from the beginning is quite variable and largely controlled by the circumstances of a given project and the interests of its leaders. While some improvement in evaluation and certainly in dissemination would be desirable, we must be mindful of the risk of creating a centralized evaluation bureaucracy that discourages innovation.

In a somewhat different category, there are well-established efforts in undergraduate student research and in educational technology. While they are generally associated with units that undergo periodic review, those reviews often do not require direct evidence of student learning in the context of stated educational objectives. In the case of academic units, the guidelines for program review are currently being revised to include increased attention to educational objectives and evidence of student learning.

In the case of educational technology and more generally information technology, many of the projects, especially some of the most expensive ones, are in administrative rather than academic units. In this case, the most natural body to deal with evaluation is AC4. This council already has an education subcommittee, which might take on this responsibility.

The Undergraduate Research Conference is an example of an established program supporting undergraduate research that operates outside of an academic unit. It seems manifest that it is a great benefit to the students involved and is confirmed by exit surveys of conference participants. Nevertheless, a more careful and focused evaluation of its effectiveness could be helpful. Although the program has grown, and many students participate, it is still a small fraction of all undergraduates and may even be a small fraction of all the undergraduates actively involved in research. While we believe that it would be good to increase participation, there are practical limits to how much the conference could be expanded in its present format. Thus it would be desirable to identify the aspects of student participation that are actually most beneficial and try to preserve those in a larger conference.

Thus in both of the areas of our self-study, we find a tension between the benefits of decentralization and centralization and the possibility that the careful addition of some additional evaluation of educational effectiveness would be desirable.

Findings

Many of our findings were quite encouraging while others were somewhat sobering. It was encouraging to learn that for over a decade we have been consistently improving our students' access to educational technology. As UC Davis faculty adopt new technologies, they foreground pedagogy and avoid flashy hardware and software that merely showcases technology for technology's sake. The directors of Mediaworks and the Teaching Resources Center exert enthusiastic leadership in this arena. Opportunities for undergraduate research continue to increase. On the other hand, there is a widely held view that more effort is needed to improve our students' communication skills. This is a problem with which many comparable institutions also struggle. The new Undergraduate Council has taken up the matter.

Because of the breadth of disciplines reflected within it, the WASC steering committee functions as a microcosm of the whole campus. We quickly learned that gathering evidence and implementing change are uneven across the campus. For engineering faculty, adherence to a serious standard of evidence is routine, the ABET requirements for their fields have fostered this approach. Faculty who have pursued external funding for undergraduate initiatives also embrace more structured approaches to evidence gathering. For many other departments the age-old benchmarks of academic success, e.g., graduate school acceptance and job placement, suffice.

In search of best practices, we discovered some very fine efforts on campus. Stellar efforts such as Physics 7 and the English Department's Computer-Aided Instruction Program stand out. Were it not for WASC, many of us would be unfamiliar with these and other well-developed projects. We plan to do a better job of circulating and sharing our own successes.

Prior to this review, many faculty members of the steering committee had only a limited familiarity with the Student Affairs Research and Information office (SARI), and the SARI Staff did not aggressively pursue connections with the academic realm. SARI staff supported the WASC steering committee by sharing existing data and incorporating new questions into instruments scheduled for deployment. Again, we were cheered by some of the results of SARI efforts and chagrined by other outcomes. Many UC Davis faculty members have resisted the efforts it would take to develop a tightly structured General Education curriculum on the grounds that our students were satisfying the goals of GE by taking a wide range of courses and selecting co-curricular activities that develop their skills. A SARI analysis conducted for WASC during the summer of 2002 demonstrated that our students are, in fact, selecting numerous courses from outside of their majors and colleges. See Criterion 2.2 of our capacity report for the details.

While this news was quite gratifying, the 2001 SARI survey asked undergraduates how their educational experience had been impacted by attending a major research university, and open-ended replies led us to the realization that we have more work to do in communicating the value of attending a research university to some of our students. The survey indicates that we need to be more proactive about promoting the benefits of attending a research university at an earlier point in students' academic careers.

Perhaps even more important than the particular results of these studies is the fact that faculty leadership is likely to look to SARI for consistent support in the future, and SARI has initiated efforts to keep their services in the view of the faculty. The WASC capacity team clearly recognized SARI's commitment to our mission and its value to our campus.

Recommendations

Several recommendations have emerged from the WASC institutional proposal:

Expand participation in undergraduate research
Given our commitment to undergraduate research, we clearly need to do a better job of sponsoring students from the humanities, arts, and social sciences. UC Davis is not alone in its efforts to improve in this arena. We may opt to emulate efforts at UCLA and SUNY Stony Brook. On these campuses unique venues for students in the non-sciences have been developed and promoted with some success.

Promote value of research university
The WASC process has reminded us that the value, benefit, and unique role of a research university is not self-evident to all. From our very first communications to our students through the graduation speeches, campus leaders and faculty need to explain and define the goals of a research institution.

Support the Undergraduate Council
The Undergraduate Council will enable us to be more responsive to the academic needs of students and faculty. The present chair of the council has been extensively engaged in the WASC process since the 1999 workshop in Berkeley. His leadership will provide a concrete connection between the two endeavors. Further, we have observed that the most functional of undergraduate councils on other campuses thrive when a strong partnership between the administration and the faculty council members can be maintained. Such a partnership was successfully forged to develop the materials for WASC, and we are confident that we can build on this foundation.

Continue to identify external funding resources
The innovations in General Education we have been able to make have been fueled in large part by grants from the Mellon and Hewlett Foundations. The Chancellor's Fall Conference for 2002 initiated campus discussions about pursuing a comprehensive fund-raising campaign. Recommendations from that conference reinforce suggestions from the previous fall conference on undergraduate education; our educational objectives warrant the pursuit of external dollars.

Establish metrics for educational objectives
Although we are very pleased by the ratification of our educational objectives, they will be much more meaningful after we have assigned metrics to them. We are addressing this through revision of the program review guidelines and development of a strategic plan.

Increase capstone courses
We have made much more progress on our freshman seminar program than on capstone courses. To stimulate more proposals, the vice Provost--Undergraduate Studies will be issuing a call for a Presidential Chair in Undergraduate Education. Priority will be given to proposals for capstone courses.

Develop incentives for educational effectiveness evaluation projects
As noted earlier, campus units have a very uneven track record on demonstrating the educational effectiveness of projects. UC Davis stimulated creative attention to educational technology by offering small grants to faculty willing to pilot course reforms. Through the auspices of the Teaching Resources Center, we will offer small grants to faculty or departments who propose innovative projects for the evaluation of educational effectiveness.

Evaluate all new educational technology projects
We recommend that the Academic Computing Coordinating Council, and in particular its education subcommittee, be charged with ensuring that all new educational technology projects involving a significant campus resource investment contain an evaluation of educational effectiveness. Of particular importance is the inclusion of an evaluation mechanism for educational technology projects with broad budgetary, technological, or pedagogical implications and/or applications. Mediaworks and TRC will collaborate in developing expertise, methods, and instruments to support the evaluation of projects in departments, campus organizations, and by individual faculty members.

Were the Goals and Outcomes of the Project Achieved?

The first general goal was to "develop educational objectives for the integration of teaching, learning, and research." As a result of efforts that began at the 2001 Chancellor's Fall Conference and continued with campus-wide consultation into the spring of 2002, we now have a set of seven educational objectives. There was an immediate convergence of these general goals and no enthusiasm for more specific statements. Thus while the objectives do not refer to research explicitly, we believe that undergraduate involvement in the research activities of the university can ideally contribute to obtaining most of the general objectives. In light of the new objectives, the Undergraduate Council is presently revising the guidelines for program review. The revised guidelines call for program-level objectives that are both specific to the major and also contribute to the general objectives. Since undergraduate participation in the department's research programs can be an important contributor to meeting educational objectives, it should be covered in the program self-study. Reviews will also describe how the program can help non-majors to meet the campus-wide objectives.

Our second general goal was to identify best practices for integrating research into the student learning experience. The research part of the self-study follows a chronological approach that shows, for each stage of an undergraduate's life on campus, the possible research activities or, at the earlier stages, those activities that prepare him or her for research. This is a catalog of successful approaches that can be referred to in expanding programs or in planning new ones. In addition to the many research opportunities that exist, we have found a broad range of activities that inform students about and prepare them for research. Over two-thirds of baccalaureate recipients reported having carried out independent research or creative projects for class, and three-fifths reported having worked with faculty on research and creative projects. Finally, with participation in the Undergraduate Research Conference or publication in Explorations, students can communicate their results. As mentioned, research experience is one of the most effective ways to reach the educational objectives, and the present process of revising the guidelines for program review will produce elements that encourage programs to involve students in research as a means of reaching the objectives of the major and the campus.

Our original intention of developing research goals for the ten academic initiatives has not turned out to be timely. Most of the initiatives are themselves still in early stages of development and are preoccupied with the first steps of getting started. At the exit interview, the WASC site team wisely reminded us of the campus's investment in these initiatives and urged us to scrutinize them in terms of their impact on undergraduate education. This is sound advice and the Vice Chancellor--Research, the campus administrator charged with oversight of the initiatives, immediately offered to convene a meeting so that we could follow up on that suggestion. We will recommend that each of the new directors develop a plan for involving undergraduate students in the research programs of the initiative.

The third general goal was to ensure that sound pedagogical principles inform our use of educational technology. The self-study reveals that we are well on the way to that goal. We are accumulating a solid base of evaluated experiments in the pedagogical aspects of educational technology. A number of examples are described in the self-study. To the extent that any general conclusion is possible, it might be to say that dramatic impacts should not be expected. An extensive application that is thoughtfully done can have either mildly positive or negative impacts on student learning. A middle ground that mixes traditional and new technology is often the most successful. The role of information technology in information transfer (email, class websites, etc.) related to instruction is completely routine. Its convenient aspects are uniformly appreciated. In addition some students are more likely to engage their professors in email discussions than they are in class or in office hours.

The deeper questions relate to pedagogy. Through Mediaworks, SITT, the Arbor, and other programs, support for the development of faculty members and their projects is excellent. With regard to the largest scale efforts, our best evaluations present a mixed picture. Based on those studies, it is difficult to make the case that a large-scale move to online classes would result in significantly improved student learning or acceptance. The most successful experiments mix traditional human interaction with the use of technology to illustrate concepts that are not otherwise easily presented. Furthermore, the cost analyses are not compelling in either direction. It seems clear that the use of technology for its own sake or for major cost savings is not justified. However, in the cases where technological means add something that cannot be done in another way, e.g., animate a concept or substitute for a new building, it can make a real contribution. Thus it appears that our relatively conservative and decentralized approach has been wise. In this way, individual initiative and creativity explores many paths, and successful approaches are discovered. A centralized gung-ho approach would probably have been a much more expensive endeavor with no proportional increase in student learning.

We find we have been quite successful in preparing our students for the use of information technology while on campus. We have a variety of means to accomplish that, and survey data indicate that our processes are meeting the needs. Through the computer expectation, summer orientation, Bovine Online, and fast Internet connections in the residence halls, new students easily get off to a good start. We have first hand knowledge of student needs while they are with us, so it is relatively easy to do the job well. For computing resources and skills that are related to majors, we find that departments are identifying the needs and taking the initiative to fill them. Although we have not carefully investigated the processes by which we identify the needs that students will have after they graduate, alumni surveys reveal that our present informal methods are working rather well. There is a high level of satisfaction with the preparation that they received at UC Davis. While it might be argued that a more organized approach is advisable, it would be complicated by the fact that future technology needs are notoriously difficult to predict. At the moment, we do not see a compelling reason to pursue this.

Thus we have accomplished the majority of the goals that we set for ourselves in the institutional proposal. In the process, we find many gratifying examples of important accomplishments in both areas. Many of those examples have associated studies of educational effectiveness. In our next steps, we will endeavor to widen awareness of educational objectives and the benefits of using direct evidence of student learning to evaluate educational effectiveness.

Next Steps

Since the WASC process began in 1999, we have not waited to remedy problem areas identified through the course of the WASC review. We will keep these items on the agendas of the campus. It may be that another fall conference on follow-up will be in order, and if it is, the Chancellor will devote one to this subject. We think the WASC website is quite strong and we plan to keep it current.

In their closing comments, the December WASC team acknowledged the pervasiveness of UC Davis good will and commitment to the pursuit of excellence. Even after a few short days they could see that all segments of the campus community care very deeply about our students and their well being. As we write this document, we face a very uncertain financial future and the possibility of war is everyday discussed in the media. Our educational objectives, our GE program, our students' ability to problem-solve and communicate are all essential if they are to excel in these troubled times. If the future is to be one that is more civilized and enlightened and in which each person has the opportunity to achieve his or her full potential, it will be built by young people who spend a few years with us to seek the benefits of higher learning.