McMaster University
Tutorial Evaluation in a Problem-Based Program Print E-mail

(A GUIDE FOR THE PERPLEXED)

Educational enterprises hope to produce changes in knowledge, skills or attitudes, preferably all three. Such changes are best realised if students are actively involved in their own learning. Problem-based Learning, particularly the small-group tutorial variety hopes to create an environment in which such change becomes the norm rather than the exception. The ultimate aim of such enterprises is to produce the truly autonomous, responsible student. Such hopes will, however, remain unfulfilled if we are unable to set in place procedures that help us determine whether such aims have been realised. Thus evaluation is essential.

Evaluation is essentially a political act. It "entails a view of society. People differ about evaluation because they differ about what society is, what it can be and what it ought to be. Much of the debate about evaluation is ideology disguised as technology" (Hamilton and others, Beyond the numbers game: A reader in educational evaluation, 1977 ISBN: 0821104160). We believe quite strongly that the primary purpose of evaluation is to benefit the student. A secondary and ancillary purpose is to convince administrators and faculty that the student has performed creditably.

It is instructive to reflect on how our perceptions of success and failure are affected by the context of evaluation. Given below is a table showing the cumulative failure percentages of 5 individuals.

Subject Failure percentage Years (cumulative)
T.C. 63.3% 24
B.R. 65.8 22
F.T. 69.3 17
R.A. 70.0 17
K.G. 70.8 17

A quick glance suggests that these people appear to have a longstanding record of outstanding failures. However, an alternative and a more conventional view would suggest otherwise (please see Table 2 below).

Evaluation is customarily divided into broad categories: FORMATIVE and SUMMATIVE.

Formative evaluation is periodic on-going evaluation that helps the student "form" or develop. For problem-based courses that attempt to produce autonomous self-directed learners this component is crucial. Summative Evaluation attempts to gauge the students’ performance at the end of a specific period, either at the middle of a term, end of a term or the end of a course. To add to the complexities, problem-based courses stress the inextricable linkage between process and content. This means that WHAT a student has learned is significantly related to HOW it was learned. It is thus necessary to evaluate both process and content.

In all the courses that comprise this Program, several different evaluation procedures are used. We believe that this is essentially fairer to students, giving them an opportunity to demonstrate their strengths in at least one, preferably more, procedures. Amongst the procedures used are tutorial evaluations using prescribed forms, problem summaries, essays on specific topics and a process-oriented problem solving exercise called variously a Group Triple Jump or a TRIPSE (Tri-Partite Problem-Solving Exercise). The relative weights allotted to each component vary with each course. As tutorials form the basis of all courses in this program, we will discuss some of the features of tutorial evaluation below.

Evaluation of tutorial performance is fraught with difficulties. We do not pretend that this is easy, nor will we hide behind a cloak of objectivity. We feel strongly that what is touted as objectivity is often the last refuge of the irresponsible. Responsible evaluation, particularly of tutorial performance, demands that individuals recognise their own fallibility, accept their own weaknesses and honestly do the best they can.

To ease this process, we have developed several categories under which tutorial performance can be evaluated. These were initially suggested by the first group of students that entered this program and we thus have instituted the notion of responsible students from the very outset. These categories are as follows:

  • Responsibility
  • Information gathering
  • Communication
  • Critical sense
  • Self-assessment

There are two parties to the tutorial process, the students and the tutors. Although the same categories can be applied to both, the expectations are different. These are described in the accompanying pages, where we contrast the expectations we have for outstanding (A+) and poor (D) performances. This should give you some flavour of the range of behaviours possible.

In our courses, formative evaluation will be held at the end of each tutorial. Using the descriptors provided, you will characterise your performance on each category on that day. This is your assessment. You will give yourself a tentative grade on each of the categories (except self-assessment) based on your performance and the descriptors provided. Following this the other students as well as the tutor in the tutorial group will give their comments. If the group agrees essentially with the comments you have made and the grade that you have allotted yourself, you can give yourself a high score for your ability to evaluate yourself accurately that day. The tutors take an active role in this process because they have had extensive experience with this format and are able to gauge individual performances in a broader perspective. This process requires practice and is not easy. However as the term progresses, you will be able to gauge your performance with confidence.

Midway during the term, a mid-unit evaluation will take place. This is an opportunity for each student to take stock of their performance up to that point. At this time, the performance of each student will be assessed and grades allotted not by that student but by the other students and the tutor. It must be emphasised that this is summative evaluation. The factors that will be taken into account are consistency of performance, improvement and growth. Thus a student may have performed at an outstanding level on several occasions, but the others may feel that the overall performance deserves a different mark. They may argue that the student, though capable of excellent performance, showed no steady improvement or was not consistent enough to warrant a summative grade in that category. The mark that is finally given is decided not by the student but by the tutor. The reasons for this are simple - with their experience the tutors are better able to temper the remarks made by the students, and it is the tutor’s responsibility to the University to allot the grade. If ongoing formative evaluation has been appropriate and rigorous, the comments made during this mid-unit phase and the grades allotted will come as no surprise. The tutor will allot a numerical score to make subsequent calculations easier.

A similar evaluation will be done at the end of the term. At that stage, the mid-course evaluation will be used as the basis and the issues discussed will be whether the student has improved or regressed in any of the categories. The mark allotted will be the final mark for tutorial performance for that term. Here again the tutors will assign the mark based on the comments made by the other students.


RANGE MEANING
90-100 Outstanding performance
85-89 Excellent performance
80-84 Very good performance
77-79 Good performance
73-76 Above average performance
70-72 Average performance
67-69 Satisfactory, but below average performance
63-66 Marginal performance
LOWER GRADES ARE FAILING

The Student in the Program:

From the outset we have made it clear that we are interested in teaching students who are willing to accept responsibility for their own learning to a substantial degree. Since this is a Coop Program, the behaviours you exhibit within the academic setting have considerable bearing on your performance in the work terms, whether in academia, industry or in other institutional settings. The categories that we have listed and describe emphasise our position.

The Tutor in the Program:

Tutors play a significant role in this program. In contrast to the TAs (Teaching Assistants) that you may have had in other courses, the Tutors are faculty members. They are thus older (perhaps wiser) and bring to the tutorial considerable experience not only in tutoring but also in pharmacology. They are teachers in the truest sense of the word. They treat students with respect and affection, and create an exciting environment in which students can develop and express their innate qualities. Although the tutors in this program have a basic interest in the discipline of pharmacology, obviously they are not experts in all aspects of the field, but they do have special interest and knowledge in the specific course they are involved in. The behaviours described in the list give you a flavour of the range possible. Remember that the tutor has a responsibility to you as well as the Program.

Subject Batting averages Years (cumulative)
Ty Cobb .367 24
Babe Ruth .342 22
Frank Thomas .307 16
Roberto Alomar .300 17
Ken Griffey Jr .292 17

Statistics obtained from www.infoplease.com and sportsillustrated.cnn.com