Memo

During Winter Quarter, 1999 the faculty teaching sociology at College of DuPage cooperated in assessing the Sociological Imagination. The form of this assessment was first developed and field tested in 1997. All faculty teaching sociology courses in the Winter Quarter were contacted and invited to participate in the assessment. With this 1999 assessment, the faculty teaching sociology marks an important series of accomplishments. First, the discipline assessment effort extends to all courses in our discipline. Second, we should be proud that our assessment extends into the sections of both full- and part-time faculty.

While not all faculty participated in the 1999 round of assessment, we received course materials from twenty-four sections representing both part- and full-time faculty in all of the courses offered. We eliminated materials from two sections due to concerns over the validity of the assessment procedures used in these sections. From the remaining sections, we recieved 472 student responses. A randomly sampling of 138 assessments were evaluated by three full-time faculty. These results are summarized below. At the end of this summary are four questions.

Eight-eight percent (88%) of the sampling came for Sociology 100 classes with the balance drawn from Sociology 120, 210, 220, and 230. Given the number of sections of Sociology 220 offered, this particular course was substantially under sampled in our voluntary effort. Fifty-eight percent (58%) of the sampling came from women. Twenty-four percent (24%) of the sample reported at least one pervious sociology course.

The first chart summarizes the percent of students earning each ranking on each of the four criteria. (For the specifics of these criteria see the Score Rubric attached to this memo.) The table attached to the chart lists the percent of students ranked at each of the four levels from 0 to 3. On the most general criteria – displaying a Sociological Imagination -- sixty-seven percent of the students responses evaluated to a 2 or 3 level. On the criteria of using theory only 35% of the assessments ranked at a 2 or 3 level.

.

Reviewing these data provides the bases for one of the issues we would like to consider: "What percent of students should be demonstrating level 2 or 3 skills in demonstrating a sociological imagination and in vocabulary use?" In other words, what should be set as our goal for these criteria? This is one of the questions on the response form.

The second chart summarizes the mean rankings of each of the readers on the four criteria. On three of the scales (Perspective, Vocabulary, and Accuracy) the readers reported a moderate level of variance. These variances are considerably larger than those observed in 1997. This indicates that there is a need to develop a "standardized" set of representative essays indicative of each level of the criteria. We will proceed with this task over the summer. This observation also demonstrates that the procedure of using multiple readers is robust to variance among individual readers.

On the criteria of theory, the readers had a higher level of agreement, ranking a substantial number of the students’ essays as lacking in theoretical ideas.

We evaluated the four criteria controlling for gender. This comparison of gender average rankings indicated no differences on three of the criteria. On the theory criteria males demonstrated a .3 lower average ranking compared to females. None of these differences between female and male performance achieved statistical significance.

The number of students reporting previous sociology courses were not sufficient to support testing for multiple course learning. Further, the number of students in course other than 100 were ot sufficient to support testing for course differences.

Future Actions:
Based on your feedback and our experiences we need to plan our next actions.

• We need to consider these findings and decide what, if anything, we wish to focus on in our future instructional activity. The attached questions provide a starting point for consideration and discussion. Please take a few minutes and send us back your reactions.
• We need to debrief the general operation of this assessment and make plans for the next cycle of assessment during Winter Quarter and reporting during the Spring of 2000.
• We will develop a sample package of essays demonstrating performance at the specified levels.

Please complete the following response form and return it to one of us within the next few weeks.

Thank you.

Peter T. Klassen, 942-2081 Dean R. Peterson, 942-3036 Deloris Wunder, 942-3072

______________________________________ __________________@ _____________________

Name: e-mail address if you want to participate in discussion:

1. What is your reaction to these data and patterns as summarized?

 

 

2. What percents of students should achieve at each of the following two highest standards for each of the criteria? (Each cell contains the current findings.)

Comments:

1. What comments do you have concerning the procedures and implementation of this assessment?

 

 

 

2. What ideas or suggestions do you have for instructional activities that support students learning these skills? (Describe your activity or instructional approach in as much detail as possible to help others understand and apply it.) [Continue your responses on additional pages.]