This summary of evidence is an integrated discussion of Criteria 4.3-4.6. Many of the topics in these criteria, such as program review and planning, have already been covered in other criteria. The criteria in 4.3-4.6 are distinguished from those by an emphasis on the importance of having the capability to collect data, collecting appropriate data, and using that data to drive reviews, planning, and decision-making, including improvements to pedagogy and the curriculum. A key element is the collection and use of data on educational effectiveness and student learning.

Several units on campus are occupied with extensive data collection and analysis: Student Affairs Research and Information (SARI), Analytical Studies, Program Planning and Budget Operations, and the Data Warehouse.

Many aspects of campus planning are informed by data collection and analysis. The links contain a number of examples of studies tied to specific decision-making. The Thornton E-card, the use of "triples" in residence halls, the campus recreation survey, and the quick study on reasons for attending summer session are specific, unique examples.

There has been a concerted effort to make relevant data available to campus managers and staff for decision making via the web. Web-enabled decision support systems such as the Financial System and Payroll/Personnel System have been instrumental in providing managers and staff with up to date data which allows for better decision making at the local department level. Web-enabled access to campus data is also a major component in the campus' New Business Architecture initiative.

In the key area of educational effectiveness, a great deal of data is collected. Important processes are the SARI surveys, several of which have appeared in the links throughout our portfolio, and teaching program reviews, which have already been discussed. While these data are both broad and deep in their coverage, they are, for the most part, secondary. By that we mean that they are measures of student success after graduation or of student satisfaction. By these measures, we are very pleased with the quality of our data and programs. We are just beginning the discussion of how we might collect primary data on student learning that goes beyond the usual evidence from classwork and build an institution-wide commitment to the use of direct measures of cumulative student learning by the time of graduation. Significant progress in this area will follow if our faculty become convinced that this is an effective way to improve student learning.

There is a long-established and elaborate process of program review that is documented here and in the discussion of previous criteria. With regard to the central topic of educational effectiveness, the situation is mixed. In program review, data on the effectiveness of the program drives the recommendations in the report. Some of the recommendations can be directly implemented by the programs. In addition, these reports eventually find their way to the Deans who are in a position to implement other recommendations.

In some areas, such as courses offered by MediaWorks and the TRC, assessment is becoming established. However, as is the case with most of our peer institutions, the element that is largely missing in most of the campus planning is a broad institutional commitment to collect primary data on student learning and to use those in planning.

In moving toward the use of primary and cumulative data on student learning, the necessary step in improving our practices must be a description of the educational objectives that we would like to achieve and are therefore the elements of student learning that need to be assessed. We now have educational objectives in place. A first step in opening a discussion on assessment took place at the 2001 Chancellor's Fall Conference.

Another important step is the establishment of the Undergraduate Council. This council and one of its committees are specifically charged with addressing these issues. We find in the charges the following statements: "To define the goals and establish criteria for use in reviewing the quality and effectiveness of undergraduate teaching programs and/or majors. To develop and review campuswide educational objectives and criteria for evaluating educational effectiveness. To evaluate undergraduate program reviews to ascertain that the established educational objectives for programs have been addressed in a meaningful way. To work with the Office of the Provost to insure that undergraduate instructional programs and program reviews are considered in the planning and support of campus activities."

In addition to these positive initial steps, there are the real accomplishments of the College of Engineering. The programs (undergraduate curricula) in this College have separate accreditations from the Acceditation Board for Engineering and Technology (ABET). The most recent review was conducted in the fall of 2000, under recently revised criteria that are assessment-based, similar to those of WASC. The programs of the College of Engineering now have in place processes to measure educational objectives and outcomes and to use the results in a continuous improvement feedback loop. For example two instructive documents from the Chemical Engineering program can be found in the links below. It is likely that other colleges will be able to draw upon these impressive developments in Engineering.