| WASC Educational Technology Essay |
Preface
This report is designed to be read online or with a web
browser handy.
If it is printed and read and none of the links are followed, a
great deal will be lost. Most readers will want to follow at least a few
of the links related to the topics of most interest to them.
A reader may wish to consult Criterion 3.7 of our Capacity Report where
there is an overview of information and educational technology at UC
Davis and where additional references are available.
Introduction
Educational technology has long been a subject of substantial interest
to the UC Davis faculty. Consequently we welcomed the opportunity to
use the WASC reaccreditation process to review recent developments in
educational technology on our campus. In this study, we discuss
examples of both institutional and individual projects. In the cases
where there has been some form of evaluation, we document that as
well.
The goals are to study our programs that prepare students for the use of
information technology and to review the development of educational
technology and its impacts on pedagogy. In addition to giving an
overview of educational technology developments, we will give
particular attention to the projects that include an
evaluation of their educational effectiveness.
The first part of this discussion of educational technology will be organized
according to a chronology from the perspective of an undergraduate student.
The second part takes an institutional perspective.
There have been a large number of innovative and ambitious projects at
UC Davis. Some of them have been carefully evaluated. We find that in certain
circumstances, an appropriate use of technology is a sound pedagogical
option, and when it can be used to enhance the efficient use of
classroom space and thereby avoid the expense of a new building, it can be
cost effective. However, on balance, one cannot expect dramatic
impacts on student learning or costs.
We find that our evaluation of educational effectiveness is still
uneven, but that it is beginning, through the Teaching Resources
Center and Mediaworks, to
become more institutionalized. In the summary, we offer some suggestions for
further progress.
Educational Technology Organized by the Chronological Experiences of a Typical
Student
In this section, we describe the aspects of educational technology
that a typical undergraduate student is likely to encounter during his
or her career at UC Davis. It is in chronological order, so there are
initial experiences, technologies that are likely to be encountered in
lower division classes, a sample of upper division possibilities, and
integrative experiences. The results of surveys on student
satisfaction with their IT preparation and experiences at UC Davis are
included.
Initial student experiences
While many potential UC Davis undergraduate students are very familiar with
some aspects of computer use, their first experience with educational
technology in the context of the University of California is likely to be at
the beginning of the application process for undergraduate admission. The
University provides an array of websites to give potential students
information and help them decide if the University of California is right for
them. The application process itself can be carried out completely online.
Included in that material is the UC Davis
expectation for computing hardware and software for incoming
students. This is intended to be a baseline computer configuration and
student competence that instructors can count on and build upon if it
is appropriate for their classes. In this way, the expectations of
students and instructors are matched, and a foundation is laid for the
use of educational technology in classes.
A new UC Davis student is strongly encouraged to come to summer
orientation. At that time, s/he will receive a UC Davis computer
account and register online for fall classes. Davis was the first UC
campus to codify such an expectation. It was done as a
collaboration among faculty, campus Information and Educational
Technology (IET) professionals, and student affairs staff.
Because of the wording of this expectation, students eligible for
financial aid are better able to receive support for the purchase of a
computer.
Upon arriving on campus in the fall, a student in the residence halls
will find that the room is wired for a high-speed network connection
and that there are staff from
IET available to help with getting connected. For additional help
with configuring a computer for UC Davis, the bookstore sells a CD
called
Bovine Online for $4.50. It contains
setup, connectivity, anti-virus, and other utility software.
There are a number of other ways that students can get computing help:
Having achieved some familiarity with the UC Davis computing
environment,
a student can use the
Student Information System (SISWEB)
to enroll in classes, adjust class
schedules, view and print class schedules, print unofficial academic
records, make address changes, view accounts, view financial aid status,
etc.
As a part of orientation to the campus digital infrastructure, it
is important for students to become familiar with the
library and especially its electronic tools and resources.
Information available on the library link shows that it
provides a very wide range of instruction, orientations,
tours, and tutorials that new students can use to become familiar with
the library in general and with the electronic resources in
particular. As more collections move to a digital format, the library
is participating in the
Collection Management Initiative to evaluate
the impact of digital vs. print collections.
In first year classes, students will discover that email and class
websites are
commonly used for some of the communication between instructors and students.
The campus has developed the infrastructure to automatically generate an
Electronic Mailing List for each class. From the point of view of an
instructor, these features are very easy to setup and maintain.
It is also very simple for the students, who are added to the list
automatically.
An ambitious tool that is under continuous development and that is
also suited to
enhancing communication for classes is the web portal
MyUCDavis. It is a centerpiece of the
New Business Architecture (NBA).
MyUCDavis has many features that are designed to
facilitate student learning through parallel use by students and faculty.
At the portal, students will find their records,
access to class webpages, online quizzes,
grades, email, financial information, a planner, search engines,
library services, the Internship and Career Center,
and the Degree Navigator.
As students make choices for majors or approach graduation, student
advising by faculty and staff
can be augmented with
Degree Navigator.
It is an online advising tool to help students develop a study
plan with their adviser and evaluate their progress toward a degree.
Another feature of MyUCDavis is the QuickSurvey, which allows
Student Affairs Research and Information (SARI) to do
timely, informal surveys of student opinion.
The MyUCDavis project has benefited from extensive, careful planning.
See for example:
a
report to AC4>, a
report to a joint meeting of the
Academic Computing Coordinating Council and the Administrative Computing
Coordinating Council, and an
IT Times special issue (check the contents link in left sidebar and in
particular the article on
student involvement in the design).
Technology in the classroom
Lower division
Upon entering a classroom, a student will see that it has a
Smart Panel. The student's instructor can connect a
laptop computer to it and gain access to
a network connection and, in most classrooms and lecture halls, to an
installed digital projector.
Some areas of instruction have built up a more specialized and advanced
technology environment. For example,
an entering student is likely to take an English composition course.
Many of those are taught with
computer aided instruction (CAI). The CAI classrooms are used for
some of the lower division writing classes and for a
majority of the upper division ones. The class structure and the
classroom are set up so that students must use the technology and so that
close collaboration among students is facilitated.
For language instruction, a student might be in the
Language Learning Center.
The IT Times has an interesting
related article about Professor Robert Blake's work.
He was an "early adopter," and one of his innovative classes is described
below.
Many students enroll in the Chemistry 2ABC sequence, which is
one of the largest enrollment classes on campus.
It is a required course for many majors and
is a prerequisite for other required courses. The Chemistry 2ABC
courses have associated labs.
The Department of Chemistry is developing pre- &
post-laboratory activities that students access on-line for
Chemistry 2C with the expectation of extension
to the entire series. The project was undertaken to improve the
students' educational experiences in the laboratory by aiding the
students in preparing for the laboratory, by familiarizing the
students with the equipment and chemical reactions involved in the
experiments, by focusing the students' attention on the principles
underlying the chemistry, and by cementing the students' laboratory
experiences with a post-laboratory synopsis. The Dean's Office of the
College of Letters and Science, Mathematical and Physical Sciences
Division and Mediaworks have been instrumental in providing resources
to bring the project to fruition.
The pre-laboratory activities are two-fold in nature, a
presentation of the exercise to be undertaken in the laboratory of the
week and a short quiz to verify the student's comprehension of
conceptual material, laboratory procedures, and essential safety
practices. Use of a multi-media mode of instruction permits the
student to receive this material visually and audibly and is a
mechanism for student testing with immediate feedback. The
post-laboratory activities guide the student through detailed
calculations using the student's personal data, collected during the
laboratory class, to arrive at a verified result. This process
provides a valuable opportunity to reinforce instruction in the
concepts pervading the calculations.
The first full implementation was the 2C labs in spring 2002.
Guest access to the pre-labs is available
here.
Although only prelab quizzes and postlab exercises are
required, the students are using the online prelabs to prepare for the
lab as well as the quiz.
There will be more description of the broader context of this project below.
Entering freshmen have general education requirements to fulfill.
A
study funded by a Mellon grant focuses upon introductory general
education classes.
The purpose of the project is to implement and evaluate the use of information
technology in large undergraduate courses.
The project includes extensive effectiveness studies, which will be
described below.
Upper division
A student's discipline-related computing experiences are highly
dependent on his or her choice of major. For the most part, the
major-specific computing needs for upper division students are handled
at the department level. When financial support for department-based
student computing is needed, deans make large,
crucial contributions. But generally it is the department that
identifies the needs and decides how they should be met. Thus there
are a huge number of varied examples. To give some feel for the
different approaches, we will briefly discuss three: Environmental
Design, Physics, and social sciences.
The
Design program has been teaching the use of computer technologies
since 1989, when they purchased twelve Mac II computers for their first
departmental computer lab. Initially all computer instruction was
integrated within individual Visual Communication courses. As the
curriculum moved further from traditional graphics methods to more
emphasis on computer technologies and as the other emphasis areas in
the department began to integrate the use of technologies into their
instruction, the need for comprehensive and consistent computer
instruction became apparent. After experimenting with various methods
of integrating computer instruction into the curriculum, the
department developed a new required course for all design students,
DES 16: Introduction to Computers in Design. Currently, the
Design Program teaches eight courses entirely in
computer labs.
There are two main approaches to the teaching of technologies in a
classroom setting. In one the focus of instruction is on computer and
software proficiency. This is the model used by most community
colleges. It can also be useful in one-unit workshops, overseen by
graduate students, to bring the students within the "comfort zone"
for specific software. The Design Program chose a different model.
They teach design concepts on the computer while the
students are mastering the computer technologies. For all assignments,
the students write concept statements and do drawings before working
on the computer. Their belief is that if the students know what they
want to create, there is an incentive to figure out a "work-around"
rather than settling for the standard techniques determined by menu
options. In comparing slides of student projects over the last six
years, they have observed a remarkable increase
in the sophistication of student work.
Not only has the Design Program devoted FTE to the instruction of
computer technologies, but it devotes a good deal of departmental
funding for computer technologies. They support an 18-station
Macintosh G4 lab with high-end color printing. To serve 300 majors,
they keep the lab open 75 hours per week. While most of the computers
and software are paid for through Instructional Use of Computers
(IUC) funding, the department supports two full-time staff members,
and has purchased a $14,000 color plotter with department funds.
Physics majors are required to take both a lower division programming
course and an upper division computer lab associated with the junior
level mechanics course. This provides a basic skill foundation that
can be built upon in other courses. In addition there is a new upper
division course: Computational Methods of Mathematical Physics. The
department emphasizes programming. This gives students greater depth
of understanding and more flexibility in the application of computing to
physics.
The department has several faculty members whose research is heavily
computational. It recognizes that along with theory and experiment,
computational physics has become a third approach to discovery in
physics.
For students who would like to pursue it more
seriously, the department offers an
applied physics emphasis in the field.
The Social Sciences Division of the College of Letters and Science
has taken an approach that enhances access to computing for students
in several departments.
To meet the common needs of students in the quantitative areas of the
social sciences, Dean Sheffrin started a dedicated
computer lab for upper-division social science classes that use
specialized software. The lab was designed to offer
upper-division students more hands-on experience in computing.
This need was revealed in extensive discussions with faculty who are
were engaged in quantitative instruction on the campus, particularly
with the use of specialized software. To date, the lab has been used
by Sociology, Political Science, and Economics. Sociology is perhaps
the heaviest user of the lab to date.
In another project, the Institute for Governmental Affairs and the
Social Sciences Division collaborated on an
Instructional Use of Computing (IUC) project to provide
support for
faculty use of data in instruction. Activities included investigation
of appropriate software and data sources, a
new website with
information for faculty and students, and an upcoming
workshop on the use of spreadsheets in the undergraduate classroom. An
initial prototype of some of these activities was a program called
WEBSTAT, which allowed students in large classes to do data analysis on
the web. It was used by several instructors.
This makes it possible for large
introductory classes to do data analysis. Continuing development of
this project is planned.
The Social Science Data Service (SSDS), a part of IGA, provides
computing access for honors undergraduates for their senior projects.
Normally SSDS is reserved for graduate students and faculty.
The Division of Social Sciences invested in developing a 24 computer
facility in the Department of Psychology. It is used heavily in Psychology
upper-division statistics courses.
Undergraduates also use this lab in their senior projects.
Psychology also has totally revamped their lower-division methods
course, Psychology 41, to make it an interactive web-based course. They did
this with the assistance of Mediaworks, and it will prevent a bottleneck for
those majors. The course is being offered in its new format this quarter for
the first time.
Economics has developed an
undergraduate data course.
This was
developed as an intermediate step between introductory statistics
and advanced econometrics.
Finally, Dean Sheffrin encouraged faculty to continue innovating in
this area, and departments continue to examine this. As an example,
Political Science is now re-thinking its approach to undergraduate
opportunities for quantitative work. They are also looking into
sources of external funding to increase the scope of their activities.
Integrative work
The idea of encouraging students to produce electronic portfolios is
gaining support. UC Davis faculty attending the 2001 Chancellor's
Fall Conference had an opportunity to increase their understanding of
how other institutions have instituted portfolio projects. As a result
of consultation between IET professionals and faculty, a portfolio
component for MyUCDavis is being developed. This could be used by
advanced students as a capstone experience to demonstrate their
cumulative learning in their major and in relation to the Educational
Objectives. It could also serve as the basis for a resume.
There are many examples of integrative work by advanced students
which display both impressive research accomplishments and excellent use of
technology. Integrative work of this type is an indicator of
educational effectiveness.
Examples are available from the classes of
Professor Geoffrey Wandesforde-Smith such as
Environmental Science, Policy Making in Natural Resource
Agencies, and
International Law.
See
Awards for Excellence in Research and Writing
"Best of the Web" Sites by Students.
Professor Geoffrey Wandesforde-Smith was one of the presenters at the 2001
Summer Institute on Technology in Teaching (SITT).
Teaching with Various Technologies: Moving Beyond "Content Delivery"
Across Courses and Subject Matters
Another example of electronic presentation of research by students comes
from Professor Jeffrey Mount's
capstone course in geology.
Students in the
McNair Program produce full portfolios including research reports, CVs, and
other material.
A collection of posters (Powerpoint format) that they
have developed to present
their research results at the Undergraduate Research Conference and
at other conferences can be accessed
here.
These student presentations are evidence for the effectiveness of our
methods in that they
demonstrate cumulative student learning, involvement in research, and
appropriate use of communication technology.
General observations by instructors
Many instructors have found, through evaluation by student response, that a
lecture format with a mix of the traditional and the digital is the most
effective. A description of one experience can be read
here.
An excellent and very experienced instructor also reached the
conclusion that a mixed format is optimal.
Evaluation surveys
We have conducted surveys to gauge the effectiveness of our
efforts to prepare students for the use of information technology both
during their time at UC Davis and after graduation.
A 1996
survey of enrolled students showed that the vast majority of
students were using computers on a regular basis and that most were
satisfied with the internet access and related resources at UCD.
Only about a quarter of students felt that their instructors expected
them to know more about computing than they did.
The
survey of 1999 graduates
addressed the question: are UC Davis graduates equipped
with the computer skills needed to
excel in a technological era? In other words, has UC Davis
successfully met the educational needs and goals of
undergraduate students in this fundamental area?
Half of the June 1999 alumni respondents
are satisfied or very satisfied with the use
of information technology in instruction,
37% are neutral and 13% are dissatisfied
or very dissatisfied.
This cohort had been very heavy users of basic computing during their
years at UCD, with over half analysing data and using applications
specialized to their field of study.
80% felt that UCD had prepared them adequately or better in the area
of information technology.
1999 and 1996 graduates rate their preparation slightly better than
did 1993 and 1990 graduates.
The 2001 survey of undergraduates contained questions about the use of
information technology. The
full results allow either general or very specific reports to be
generated. There is also a
report with more description and commentary. A part of the summary
reads:
"There were five items that addressed the use of computers and
IT applications to provide or support the delivery of
instruction. The survey found students to be well satisfied with
both the amount of computer use in teaching and the amount
of time they spent using computers. Students were also open-minded
about exclusively on-line courses but the opinions of
those who had taken on-line courses were split, with about
half rating a traditional class as better and half stating that online
course were equivalent or better.
"On average, students agreed that classes using computers for
more than simple tasks did significantly improve instruction.
Engineering students and seniors led and HArCS students
were least in agreement. The second item in this section asked
about the amount of IT use in instruction. In response to this
question, the overwhelming majority (76%) said that it was
about right. Notably, only 3% said that instructors were using
computers too much. A much larger group (21%) wanted to
see more. Unlike those items that asked about specific IT
applications, this general item found little to no variation by
major, class level, sex, or origin (transfer or not). The third
item in the series asked students whether they would enroll in
a course that was on-line and had no regular class meetings.
The responses produced a very flat distribution. Some 28%
were negative, 39% were positive and 34% said maybe or did
not know what they would do. Only one-in-ten students were
definitely opposed, and there were 16% who were definitely
inclined to enroll in an exclusively on-line course. The last
item in this section asked students with experience in UC
Davis on-line courses whether they were better than,
equivalent to, or worse than traditional courses. Only 16%
claimed experience and their responses were tallied. Of these
16% with on-line course experience, 46% said that the
traditional class was better, 33% thought that the two types
were equivalent, and 21% reported that the on-line course was
better."
Section summary
We have described numerous examples of our programs to prepare
students for the use of information technology at multiple levels of
sophistication. Some impressive examples of student work are included.
In addition, general surveys indicate the success of our efforts.
A missing element in the picture is the evaluation of the
department-based upper division computing projects. Barriers to
serious evaluation are the
facts that most faculty probably think that they are self-evidently
beneficial and that
these efforts are either new or very rapidly changing and present
a difficult target for evaluation. It will be challenging to evaluate
these diverse and decentralized programs in a way that is meaningful
and cost effective.
Support for faculty
The campus offers many opportunities for faculty to prepare for the use of
educational technology. One of the most innovative, long-running, and
successful is the
Summer Institute on Technology in Teaching (SITT), which is run by the
Teaching Resources Center (TRC). In July 2002, the
9th annual session was presented.
The Summer Institute has been very successful in
providing faculty members an opportunity to efficiently learn how to use new
technology in their classes.
The demand from faculty for the week-long institute was so great
that the TRC doubled to 70 the number of people that it can accommodate.
There are now several hundred faculty who have participated.
This summer, programmers from the MyUCDavis team attended sessions so that
the IET efforts on the MyUCDavis portal will stay coordinated with the
needs of faculty.
Over the years, a real camaraderie has developed among the faculty who
participate in SITT. The organizers use past faculty participants to train
new classes of faculty. Faculty evaluations of the Institute show a
great appreciation
for the emphasis on teaching evident throughout the week-long
workshop.
A recent addition to MyUCDavis that is a response to faculty needs is the
quiz builder, which is explained on the MyUCDavis
tutorials page.
In fact, the flip side to the student view of MyUCDavis shows the many tools it
has to help faculty with class email, websites, and recording and submitting
grades.
Another major resource for faculty is
Mediaworks. Some background on the formation of this unit
can be found in the Summary associated with Criterion 3.7 of our
Institutional Portfolio. Mediaworks provides
a range of educational technology support for instructors. It includes
things as varied as the
partners program and full-blown multimedia development. In this way, a faculty
member can move to any technical level that makes sense for his or her
instructional purposes.
Educational technology projects at Mediaworks are primarily for regularly
scheduled undergraduate, graduate, and professional courses, with an emphasis
on large undergraduate courses that are experiencing enrollment pressure. The
Educational Technology group has expertise in multimedia and computer
programming. Members of this group serve as consultants and producers for
instructors who want to use media to present their course content in compelling
and engaging ways.
Since UC Davis aspires to be a community of learners whose members learn
from one another in a reciprocal fashion, it is fitting that the
ET Partners program harnesses the time and expertise of students to help
faculty members learn about and use educational technology
effectively. Students bring both a familiarity with technology and an
expectation that their education will be enriched by the appropriate
use of modern communication tools.
The ET Partners program, part of IET at Mediaworks, hires, trains, and
manages the student partners, and selects the faculty partners. The
primary aim of this program is to use the concept of the community of
learners so faculty members will use educational technology
effectively and easily.
Mediaworks is evaluating the effectiveness of the ET Partners program with
questionnaires on computer attitudes and skills
as well as focus groups on the impact of the program on both student
and faculty partners.
In an effort to make sure that faculty are aware of Mediaworks
services, an
openhouse was held recently.
As an example of the careful institutional planning discussed in Standard 4 of
our capacity report,
these projects are coordinated closely with the Teaching Resources Center, a
responsibility of the Vice-Provost-Undergraduate Studies, and the Classroom
Technology Services, which is part of Information and Educational Technology.
Classroom Technology Services works closely with the Registrar and the
Instructional Space Advisory Group. The Vice Provost for Undergraduate Studies,
the Vice Provost for Information and Educational Technology, and the directors
of Mediaworks and the TRC meet regularly.
Yet another important resource for faculty is
the Arbor, which
provides
support to faculty in the use of instructional technology to improve
the quality of learning at UC Davis.
Examples can be seen in their
showcase.
The TRC also offers a great deal of teaching and technology support:
A comprehensive reference is the
faculty technology guide.
Thus there is a strong institutional commitment to supporting faculty
in their use of educational technology.
Institutional perspective
For an institutional perspective, it is instructive to outline the
evolution of educational technology at UC Davis. In this part, we include
references to evaluations of the educational effectiveness of the
technology.
The impacts of educational technology on pedagogy are determined largely by
faculty adoption of the new methods. Faculty attitudes to new technology cover
the entire spectrum. Many faculty members were quick to embrace the
possibilities of the emerging technology and experimented with its use in their
classes. These were primarily dedicated and enthusiastic innovators. They
operated mostly individually, i.e. not as part of an institutional
initiative.
However, the campus has encouraged and supported this adoption with funding
opportunities through the TRC and Instructional Use of Computers grants. It
would be difficult to identify most of these projects. Among the most ambitious
early projects were those of Professors Matthews, Falk, Blake, and Walters.
Professor Harry Matthews taught a molecular biology course for first year
students in the medical school. Over a period of several years in the late
1990s, he moved an increasing amount of the lecture content into
electronic form. This included animations to illustrate key concepts. With much
of the course content available via the web or CD, there was more time for
interaction between the professor and small groups of students. This successful
experiment is described in several articles:
- Virtual Lectures Free Students to Think Critically
- Virtual Lecturers Free Students to Think Critically:
Vignettes of Best Practices
- Evaluation of Scalable Applications of Information Technology to On-Campus
Learning
- Evaluating On-Line Learning On-Campus
The one line summary of these effectiveness results is that
student performance improved, and their opinions of the course were positive.
Professor Richard Falk taught an introductory biology class for nonscience
majors. The discussion sections became email listserver-based virtual
discussion sections with topics drawn from current events. They included a
writing component. The lecture content was made available on the web. It was
restructured to include links to related sites, animations, and self-test
questions. This was a complete replacement for the lectures. However, the
lectures were still given, and a minority of the students chose to attend them.
Evaluation based on student surveys, student work, and grades is
also described in
links 2, 3, and 4 above.
Most of the students liked the web-based material. The virtual discussion
sections were successful in producing much more student involvement and
interest in biology and its relationship to issues of interest to them.
The quality and depth of the discussion were also enhanced.
In this electronic introductory biology course,
student performance was comparable to that in the traditionally
delivered course, and again student acceptance was very good.
Professor Richard Walters undertook an extremely ambitious project to develop
the Remote Collaboration Tool (RCT ne RTA). This is a large software
suite, which brings many
of the benefits of face-to-face interaction to interactions at a distance
through multimedia synchronous and asynchronous electronic communication.
This tool was
used successfully by Professor Robert Blake in introductory
Spanish language courses. It was also used by Professor Walters in an
online independent study
version of an introductory computing class. Students in the online version were
satisfied with the course and performed as well as those in the parallel
conventional version.
For more information, see the
Remote Collaboration Tool website and the list of
papers there, e.g.
- Implementing Chat Software in the Foreign-Language
Curriculum
- Evaluation Of The Spring 1998 RTA Pilot Program
- Outcome Analysis of Distance Learning:
A Comparison Between Conventional and Independent Study Instruction,
The students who used this tool for synchronous pairwise discussions
showed improvement in their language proficiency comparable to the
control group.
In parallel with these innovative projects, there were two institutional
efforts. SITT already has been described.
The Learning Environment Architecture Development (LEAD) project was
an
assessment to help
identify the tools, architecture, and infrastructure for the
learning environment at UC Davis. The
final report was published in early
2000. Among the projects that followed from this
is the web portal MyUCDavis described above.
The second phase of the development of educational technology at UC Davis
includes a more institutionalized approach. The central elements are
Mediaworks and the
Mellon Project.
The courses of Professors Falk and Matthews mentioned above became the pilots
for the ten general education courses for full study in the Mellon Project.
The aim of the project is to implement and evaluate the use of information
technology in large undergraduate general education courses. The evaluation
includes both
educational effectiveness and cost. The first course to receive a full
evaluation is Food Science 2. The papers are
The results from these studies were
mixed. After accounting for lecture hall space, the online version of the
course is less expensive per student than the traditional version. The
students who relied mostly on the web material rather than going to
lecture did slightly worse on the exams.
Since there are many uncontrolled variables in this study, it is
difficult to know whether a random assignment of students to groups rather than
the self-selection would produce the same results.
In a very recent and more comprehensive study, six of the Mellon classes
have now been analyzed. A
draft report and its
tables and figures are available.
This analysis for six courses
reveals that in terms of student performance and student preference,
a traditional course is favored over an online course. However, the
differences are very small. For a more complete and nuanced view, the
reader is strongly encouraged to examine this careful report at the
two links above.
A course that is not part of the Mellon Project was also recently evaluated.
It is a virtual lab section for a large introductory psychology course on
research methods. In this case, the students responded favorably to the format,
and their exam scores were no different from those in the traditional lab
sections. See
A virtual lab in research methods.
Another case is the
online laboratories that were developed for the large enrollment course
Application of Microcomputers in Agriculture. In this case, the
evaluation showed the same performance for the online group and
the traditional group.
The third phase in the deployment of educational technology at UC Davis
involves
projects in a strategic choice of classes. One example is the introductory
chemistry sequence, Chemistry 2ABC, which was mentioned and for which
links were given above.
The availability of rooms for the laboratory sections is a major
limiting factor on the number of students who can be accommodated in the
courses. The construction of more lab space is a very slow and expensive
process. However, a significant fraction of lab time is spent on pre- and
post-lab discussions that do not actually require the physical facilities of
the laboratories. So that more students can be accommodated in the present
physical facilities, the pre- and post-lab discussions are going online. This
means that the lab rooms will be used more efficiently.
Another motivation is
to provide a more uniform pre-lab student preparation and a more uniform lab
experience for the students that is less dependent upon individual lab TA
styles. In particular, the quality and uniformity of safety
instruction is enhanced greatly.
This project has been a collaboration of the Department of Chemistry,
Mediaworks, and the Division of Mathematical and Physical Sciences
(MPS) Dean's office. In the Chemistry Department,
William Fink
and Dianne Meador provide the chemistry content, the lab setups, and the
descriptions. Mediaworks does the programming and comes over to record the
video and audio. Moby (Coldfusion and Flash) are used to present the web pages.
The spring 2002 offering has been
evaluated. A Powerpoint
presentation is available. A new edition
of the
student evaluation form has just been released to this quarter's students.
Also the TAs report that the students are coming to lab better
prepared to do the experiments. To date, all indications are that the
project is working well.
A number of other projects indicate the increasing commitment of the campus to
educational technology and its evaluation.
Most of the general assignment classrooms have been updated to the
Smart Panels for digital display and internet connections.
Mediaworks, the new technical production, consulting, and support unit for
educational technology was established.
The
Teaching Resources Center, which provides support
for many aspects of teaching and learning at UC Davis, has increased its
commitment to educational technology.
Three years ago, they hired a director for
technological programs.
They plan to add further strength in instructional design and
project evaluation.
They have also increased their commitment to
technology by including more software and hardware for consideration in IUC
grants.
Among our standard review procedures are program reviews and student
evaluations of teaching. For both of these, we are considering ways in which
the evaluation of the educational effectiveness of technology could play a
bigger role. Course evaluations are administered at the department level with
the approval of individual faculty for their courses. Thus it may be sometime
before such questions are widely used. One possible approach is to encourage
SITT participants to include specific questions about the effectiveness their
new endeavors. The guidelines for program review are being revised. It may be
possible to include in them further encouragement for evaluating the
effectiveness of educational technology.
In 1998 the Academic Computing Coordinating Council (AC4) was formed. Its
mission statement reads: "The Academic Computing Coordinating Council
reviews proposals and plans that seek to promote the use of information
technology in instruction, research, and public service at UC Davis and makes
appropriate recommendations to the Provost and Executive Vice Chancellor. The
Council may also recommend its own proposals and plans. The Council's broad
representation assists the campus administration in ensuring that campus
resources are deployed to their most strategic advantage. The Council
recommends academic computing-related policy to the Information Technologies
Policy Board. Policy interpretation and financial decision impasses are also
referred to the Policy Board."
AC4 has subcommittees for education and course management systems.
The institutional context for AC4 and some of the developments that led
to its formation are described in Criterion
3.7 of our capacity report.
A very exciting new program is
Technocultural Studies.
It grew out of the
ArtsVision Initiative.
Students in the major will develop interdisciplinary skills that include
the use of software technology for digital media arts.
While a program that does not yet have any students cannot very well have a
large evaluation component, it should be noted that the plans for student
learning of technology are based on the past successful use of those
techniques.
We anticipate that this hybridization of the "two
cultures" will bear abundant fruit and will have a major impact on the
understanding and use of educational technology at UC Davis.
Summary
This description of the development and use of educational technology
at UC Davis reveals a careful approach. Most of the development for
classroom use has been at the initiative of individual faculty members or small
groups. From that perspective, the campus has taken a bottom up
approach. However, in its direct support for faculty efforts by TRC
and Mediaworks and in its indirect support via infrastructure, the
campus has an organized, strategic, and timely approach.
The evaluation of educational effectiveness has been variable.
Individual faculty members tend to be guided by student evaluations
and gut feelings. Serious studies of educational effectiveness are not
usually done by individual faculty members. However, if evidence
emerges that using such evaluations results in better student
outcomes, then UC Davis faculty members will be likely to embrace the
proven assessment tools. On the other hand, the larger projects,
e.g. the Mellon Grant and the Chemistry online pre- and
post-labs, are
already being seriously evaluated for their effectiveness.
In those cases where a careful evaluation has been done, the results
indicate that student learning, student satisfaction,
and costs are not dramatically affected by the use of technology.
In cases where new technology is mixed with traditional techniques,
student acceptance is good.
In cases where the use of technology can avoid the construction of a
new building, it can be justified on a cost basis.
A more uniform approach to the evaluation of
educational technology might include the following elements:
- Faculty who use significant educational technology should be encouraged
to include specific questions about its effectiveness on student class
evaluations.
- Departments should be encouraged to discuss the use and effectiveness of
educational effectiveness in program reviews.
- Mediaworks and the TRC should coordinate in
increasing their commitment to instructional design and project
evaluation. They should develop standard evaluation tools
that could be used with the projects they fund. In addition, they
should create
readily accessible and publicized descriptions of case histories and best
practices from UC Davis and other institutions.
|