| A Tutorial Steering Committee was formed in 1996 to monitor tutorials in the curriculum. Comprised of a Narrative Editorial Board and a Policy Subgroup, the Committee began by developing mechanisms to assure that tutorial assessments meet the needs of the Committee on Student Promotion and Evaluation (CSPE I) and of the students themselves. Accordingly, the Policy Subgroup developed criteria for: |
|
|
• Writing useful narrative evaluations of students • Assessing observable behaviors in tutorials • Assigning grades in tutorials |
|
|
Grading criteria across the successive
units reflected the developmental processes of learning. It was expected that specific behaviors and skills
would be more consistently
demonstrated with experience as the students move through the tutorial units of Phase I. An assessment reference manual was developed and distributed to all tutors and students. In addition, criteria for grading in tutorials was created and distributed to each unit. The Committee was also responsible for the process involved in determining students’ semester tutorial grades on the OGSMU scale (outstanding, good, satisfactory, marginal, unsatisfactory) based on the average of students’ unit grades. (In 2001, tutorial assessment was changed from the OGSMU scale to pass-fail, a method felt to be more consistent with the small-group collaborative learning characteristic of problem-based learning.) |
|
|
|
To monitor the quality of narrative evaluations developed by tutors describing student's performance and areas for development, a narrative review process was developed. The process consisted of an anonymous and randomized review of blinded narratives based on agreed upon criteria. For each narrative, reviewers completed a four-item checklist, which included quality of the narrative and consistency of the narrative with the assigned grade. A compilation of review ratings and feedback was provided to each tutor being reviewed as a valuable tool for skill development. |
|
Implementation of the review process was incremental: • In its first year, Phase I-1 narrative evaluations of students for Units 1-4 were reviewed by the Narrative Editorial Board. • In the second year, the process was expanded to accommodate simultaneous faculty peer review of Phase I-1 and I-2 tutorial narrative evaluations in the fall semester. • The review process continued until 2001, reviewing the narratives of tutors who were new, who had <3 (on 4-point scale), and/or who had not been reviewed for a period of three years. Approximately 300 Phase I narrative evaluations were reviewed during the life of this project. The outcome was compelling. A marked improvement in the overall quality of Phase I Student Tutorial Evaluations was observed by the Committee on Student Promotion and Evaluation and attributed to the Tutorial Steering Committee’s innovative faculty peer review process Impact of the review process seemed to decline across time. In addition, the process was extremely time and labor intensive, prompting the decision to terminate the review process in the 2001-2002 academic year. The quality of tutorials is also assessed in an ongoing manner through student evaluation of tutors and through student evaluation of the curriculum block, which includes tutorials.
|
|
