RE: Assessment

Conference Updates

Braving the floods, mudslides, and sinkholes caused by El Nino and the exotic vagaries of California culture, Jim Belz attended the Second Annual Assessment Conference sponsored by the Graduate School of Business at the University of California at Fullerton. The Conference, held March 5 and 6, was on a much smaller scale than the IUPUI or AAHE conferences, which attract hundreds of attendees, but the featured speakers were excellent, the business slant was enlightening, and there were some very interesting presentations by students, some from the Fullerton campus and an entire delegation from Africa. The students focused on how the assessment process had been of benefit to them personally, a perspective that is less often heard.

Almost immediately following our graduation ceremony, four intrepid COD colleagues will traipse off to the terra ingcognita of Cincinnati, Ohio for the annual American Association for Higher Education (AAHE) Conference on Assessment. Lesli Beltran, Cynthia Cordes, Peter Klassen, and Russ Watson will be attending what is generally considered the premier national assessment gathering, and they are almost sure to return with some interesting new information and insights on assessment.

Take A New Look

Thanks to the efforts of Peter Klassen, the Committee's assessment web site continues to grow and improve. If you haven't visited the site lately, take a new look. We think you'll be impressed.

The URL is <www.cod.edu/outcomes>

New Initiative for General Education Assessment

One of the ways to assess general education outcomes for students is to administer tests at various points during their time spent at COD to demonstrate that they are making progress in such areas as computational skills, science reasoning, reading comprehension, writing skills, and critical thinking. The tests can be either locally developed or nationally normed standardized instruments, and they can either be administered in a blanket fashion or to a statistically significant representative sample.

For the better part of this academic year, the Student Outcomes Assessment Committee has been grappling with these testing issues and examining potential instruments. At the May 5 meeting, there was a unanimous decision to implement a project based on the CAAP battery of tests developed by ACT (American College Testing) of Cedar Rapids, Iowa. The College has a history of using certain ACT developed tests as placement instruments for entering students, and data from those can be integrated into the CAAP results.

The CAAP battery of tests consists of six different instruments covering various general education skills: reading; writing (recognizing errors); essay writing (actual composition of an essay); mathematics; science reasoning; and critical thinking. Each instrument takes approximately 40 minutes to administer.

In the project starting this coming Fall Quarter, the instruments will be administered in selected classes as mandatory class assignments to ensure student participation. Within a particular class, all six instruments will be administered during the same class period, but the students will be randomly assigned to take only one of the instruments. During the Fall cycle of testing, both on and off campus entry-level classes such as English 101, Sociology and Psychology 100, beginning technical courses, etc. will be targeted to create a stratified sample rich in entering students. The same battery of tests will be administered in the Spring Quarter of 1999, but this time targeting more advanced courses such as English 103, Biology 103, Psychology 240, etc., to create a sample rich in more advanced students. ACT assures us that by having a minimum of 100 students take each test during both Fall and Spring (a total of 600 students for each battery of six tests, a total of 1200 for the year), we will have a statistically significant sample to base analysis upon. The same cycle will be repeated in subsequent academic years, with analysis, reports to disciplines, and intervention planning interspersed to ensure that the findings are being acted upon.

The cost to the College is not insignificant ($10 per student per test equals $12,000 per year plus another $5000 per year in personnel time for administration and analysis), but the useful data generated should more than compensate for the money and effort involved.

FAQ: (Frequently Asked Questions)

Q. I've been hearing about things called Assessment Centers. What's that all about?

A. Assessment centers are a relatively new development, and so far the majority have been developed by large corporations, employment consultants, and business schools, but as their worth is proven, they may become more and more common in non-business settings.

Assessment centers may use traditional paper & pencil instruments to help evaluate students or potential employees, but the heart of an assessment center is a series of simulations in an actual (or very close to actual) working environment to assess not just what students know but how they are likely to behave and perform on the job.

Typically, the simulations may take one or more days, and the participants are carefully observed by specially trained evaluators. If the simulation takes place in the actual workplace setting, the participants are usually unaware of the who the observers are. In some cases, with the participants' knowledge, the simulations are videotaped, and the evaluators work from those tapes.

The various simulations attempt to replicate a typical day on the job. One popular simulation, for example, is called the "In Box." The participant is presented with a number of messages, requests, and meeting times and is expected to plan the day in a way that will prioritize the tasks and fulfill all the obligations. Another simulation may involve working with a task group to solve a problem. Another may involve writing a letter of response to an unhappy customer. And even the coffee breaks are used to observe how the participant interacts with others in a social setting.

In the corporate setting, assessment centers are used to help make hiring decisions, especially among a group of candidates who may be pretty equally credentialed. In the business school setting, assessment centers not only help the faculty and administrators assess the curriculum, but perhaps more importantly, they help students to understand where their strengths and weaknesses lie and what they need to do to remedy any performance shortcomings.

With creativity, aspects of the assessment center approach could be implemented for a number of programs at COD. For example, as part of a simulation a student might not only be asked to diagnose what is wrong with an air conditioner but also to interact with the customer to explain exactly what the problem is and to help the customer to make a decision whether to repair or replace the unit. Such an approach would provide useful assessment feedback both to the program and the individual student.

Assessment In Action : Distance Education & Assessment

Linda Slusar, Coordinator of theLibrary Technical Assistant program, is finding that Classroom Assessment Techniques are absolutely essential as she develops an entirely new distance education program at COD. Funded by a federal Library Services & Technology Act grant awarded by the State of Illinois, LTA Illinois is offering the first four courses of COD's LTA certificate program to 67 students from across the state. All the participants are highly-motivated adult learners who presently work in libraries but who have little or no formal training in librarianship. The grant provides a full scholarship for each student, and the participating library districts provide support such as internet access, FAX, e-mail addresses, etc.

Each week the students gather at five different sites (some hundreds of miles away from COD) to attend LTA 101: Today's Libraries, a four credit class that Linda conducts via an interactive television hookup. Linda is assisted by a librarian/mentor at each of the sites, an assistant teacher at COD who helps with much of the grading and individual feedback to students, and a student aid who helps with the clerical work.

As Linda notes, this is a whole new paradigm of teaching. For one thing, almost all of the non-verbal feedback and much of the immediate verbal feedback from students is missing. All of the students get to see Linda's presentation in its entirety, but she can only interact with one site at a time, there is always a four second delay in both directions, and even her view of the class is somewhat limited because it is a designated student who operates the camera at each site. To compensate for these differences, Linda has learned to talk more slowly, to provide carefully crafted rubrics for all assignments and make them all available on the internet, to include a wide variety of activities in each session, to accept that she does not have the same kind of power she would in a traditional classroom, and to rely on humor when there are glitches in the technology.

She also requires a variety of written input each class period, including prepared study questions over the readings and several spontaneous one-minute letters, some of which elicit responses for class discussion and others which are considered confidential and are only seen by the instructors. Written materials are either mailed or faxed to Linda at the end of each class, and she is able to provide written feedback to the assignments and the confidential one-minute letters by the next class period. Students are also encouraged to contact Linda by phone or e-mail if they run into difficulties.

Using these various forms of feedback, Linda is better able to understand the concerns and needs of these students, whom she may never meet in person, and make the necessary modifications in the course so that these distant learners have a high quality educational experience. Linda also believes that the methods and techniques she has learned as a result of this class will enhance the quality of her traditional classroom presentations.

Copyright © 1998 College of DuPage
Students' Outcomes Assessment Committee  · (630)-942-2081
klassen@cdnet.cod.edu
Updated December 16, 1999
Disclaimer