Self-Direction 2
The Question of Peer-assessment
by Tom Osborn
For a few months last year, I worked on the Counselling Courses again. I was with the full-time staff group, standing in for someone on leave. It was exciting to be back.
My last time had ended around the time I wrote that article about the starting weekend structure we designed. It had been possible to formulate and understand at least the beginning of the student-directed process that is followed in these courses. That was about five years ago. This time, it felt like it was possible to see the rest of the process, including its end, the assessment for the Diploma. And this article is going to be about the assessment by peers.
One of the courses I worked on had its assessment weekend while I was there. A general structure has evolved for the assessment procedure. It varies slightly from course to course, because the details are determined by the students for themselves each time. But it is sufficiently constant for the one that was used by this course to be representative.
The Diploma itself is firmly established, recognised by the Council for National Academic Awards. And established with this demanding enough external body, also, is the fact that the giving, or withholding, of the Diploma is decided by peer-assessment. This is perhaps remarkable in the present climate of the wide world outside.
We always have to remember that this very personal, intimate, private activity of counselling takes place in a context. It has an upstream, as Egan puts it. And the most upstream context beyond the family, beyond the organisation, is our society. And since I wrote that previous article, a lot has happened there too. The trendy right has got its second term. Any way-out educational venture is in constant danger. The job-market for "counsellor" has become more competitive, the demands laid on those who engage in this activity have become more severe.
I refer to these outside factors because the question of assessment concerns the outside world. Accreditation, externally safeguarded standards, a recognised label to show to prospective employer, these are outside-world matters. And assessment is done in their terms. So whoever does the assessing is bringing the outside world in across the boundary of the familiar inside world. And when the outside world is swept by economic recession, unemployment, shortage of resources, an ideology of competition, an ethic of productivity, then bringing it in is that much more painful.
When the people who do the assessing are also the people who are going to be assessed, the role-conflict involved acquires proportions which I have to describe as heroic. I did experience the assessment weekend as heroic - and traumatic. The course members had to straddle this boundary. It's a deeply traumatic position. The question is, does having been there add to their ability as counsellor? How much is gained, how much lost?
But first, let me describe roughly how the assessment structure worked.
During the three years of the course, and especially during the last one, each student prepares a portfolio. Into these portfolios go reports of experiences they have had as counsellor: particularised through description of style and atmosphere; analysis of interventions; discussion of contract and the implications of special circumstances; evaluation of response in the client; indication of strategy with any related method or conceptual basis; specification of criteria by which all these things are judged; and so on. Students make reports of their own work, and they are assessed on these portfolios.
The assessment itself, on this course, was carried out for each student's portfolio by a panel of three assessors. Two were chosen by the student, the third was selected by a random process. Of the first two, one assessor was chosen from the student's small group and one from the course as a whole outside that group. The third, the random one, came also from the members of the course who were not in that student's small group. All the portfolios were read by the assessors in the few days preceding one weekend, and there was a timetable of interviews at which students were "examined" by the assessors on the contents of their portfolios.
Such a system gives rise to a grotesque enough logistic problem. To timetable all the interviews with their combinations of people is a job for a computer. But the logistic difficulty is trivial compared with the Psychological. Only the Saturday was really available for interviewing, since Friday night and Sunday were needed for other activities essential to the assessment.
So imagine in one day spending four or five one-and-a-half hour sessions, in at least three of which you are analysing, discussing, judging a portfolio of work which has taken many months to compile, determining a significant aspect of someone's future, perhaps deciding conditions e.g. for more work on which to award a pass; and in one of which the same thing is being done to you.
And you are not a practised examiner evaluating candidates, but a beginner examining your companions!
A stimulating, challenging activity? Maybe. A tense situation which raises standards? O.K. for the brilliant few?
But is it collective? This is the nub of the paradox. Here we have a course which sets out to share resources, to programme jointly, to develop skills in co-operation, to exchange constructive feedback, and, finally, to equalise the responsibility for standards.
And what happens, finally, is it sounds like a competition again. I didn't go through it myself, but I don't believe I am projecting when I assert that it feels like a competition again. Some people make it with flying colours, some scrape through with the skin off their eyeballs, one or two fail.
It is true only a few people fail, just enough perhaps to keep that edge of passing or failing real. It's true each course determines that this shall not be just an all-or-nothing event, that for the one or two failures conditions can be imposed which can lead to getting the Diploma later by further development. It's true an appeals procedure is always built in. It's true there is a keen awareness of the danger of scapegoating, and the intention to avoid it.
But I am not concerned with the successful avoidance of problems in the process. What concerns me is the effect of this peer-assessment when it's working alright. The effect of judging your friend and companion and colleague. This is not judging for the purpose of giving feedback to be used by that companion in his or her on-going development. It is judging on one designated day for the purpose of attaching a mark, of granting or refusing an outside-world certification. And when the outside world is a place of such hostility.
This difference between these two sorts of judging is crucial. The first sort, on the course, has come to be called "self-monitoring". The intention, I believe, of the staff is that it should happen from an early stage. The most fruitful way to do it, I would say, is to be counselled, or see someone else counselled, and then report on the experience to the person who did the counselling. There's no question it is an invaluable learning tool, perhaps the best of all. But also, it has a clear connection, as a skill, with counselling.
To be able to say, honestly, and helpfully, how you experience someone, and make decisions about what part of that experience you report, on the basis of what they will be able to use in the process of their learning, is exactly one of the things you will need to do as a counsellor.
This self-monitoring tool needs itself to be learned. But the time and space and relationship structures within the course are there every time the course meets. No problem. There is a problem, but it lies elsewhere, and I shall keep you in suspense a moment longer before expanding on it.
The other kind of judging is the assessment kind. Now, if this is a learning tool it's a blunderbuss, where what you need is a bicycle. The purpose of a test is not to teach but to test and so is its effect. Nobody learns to drive by taking the driving test, though of course one learns a few things in the course of taking it.
The second point, as with the self-monitoring kind of judgement, is, has it a connection with counselling? If there is a connection, then it's not of the same clear and direct kind.
The best counselling is a resource-sharing, collective activity; the counsellor trusts the growth energy of the client, does not possess the answers, is open to bringing self in; there is a direction towards equality, within the counselling framework. Further: many of our problems, many of the reasons why people come to counsellors, are at root due to those very features of the outside world which denote competition.
And it is possible to argue therefore, that to have experienced this difficult situation, with its assertion of collectivity in a non-collective environment, does provide an added perspective to counselling, at a level of some sophistication.
I believe this to be true. But I have no clear idea to how many people it applies, though I think it may be rather few. And I am not convinced that this potential gain at present outweighs the losses.
Thirdly, assessing itself has to be learnt. Now, in this we have failed miserably. I just don't think that members of these courses, generally speaking, arrive at the assessment equipped to do it. Of course many of them muddle through pretty well. They're intelligent, mature people, ready to have a good stab at it. But they're amateurs. They have not had the flying hours, nor have they even had the basic training. And on this point I am not convinced that learning how to assess is worth the time and energy involved, in a cost-benefit sense.
Now, back to this problem about actually learning and doing the self-monitoring kind of judging. My perception was that in spite of the constant and ample opportunities, it was something that happened too rarely and too late on the courses. And my hypothesis is that the shadow of assessment actually interferes with the practice of self-monitoring.
The distress connected with the judging of the winner and the loser, the better and the worse, the passer and the failer, spreads over onto the judging which is loving, helpful and constructive. The invasion of the outside world across that boundary which is an inherent factor in the actual peer-assessment event occurs, if my hypothesis is right, in the imagery of the world, throughout the course, to contaminate the self-monitoring energy and enfeeble it through fear. It is precisely the dysjunction between the cultures on each side of that boundary which makes the issue. In an ideal world, we would be as sharing and collective outside as inside. But then in an ideal world, we would not have to fool around with diplomas.
Come to that, in an ideal world there would be no counsellors, or everybody would be one when needed. Actually, in the real world we all have to live with the existence of that boundary. It's just that most people manage to live one side of it or the other. Counsellors are in one of those professions that take the brunt of that dysjunction. It does seem to me an excess of masochism to then go and add the burden of peer-assessment!
And yet, as a matter of fact, I still. think it could probably work. I am in love with it, as I think are Brigid and Marcia and many of the rest. I want everybody to be a brilliant genius, and I firmly believe they can be. We don't, in the modern world, need a society based on slaves to produce a new Athens or Pericles.
What we do need is a new consciousness. And part of that consciousness has to do with an awareness of the boundary I've been talking about: between the place we share resources and the place we compete over them, and the cultural dysjunction between those two places. And this awareness can indeed come out of a peer-assessment system. But we mustn't kid ourselves. It is a heroic project. Above all, we mustn't kid our students.
Not kidding them involves, I believe, certain practical strategies which need to be brought with greater energy to the courses, by the staff. Here I am going to get prescriptive.
The self-monitoring kind of judging needs to be insisted on and structured fnom an early stage and made a regular, accepted, successful, demystified learning tool. The assessment kind of judging needs to be learnt and practised at some point. The distinction between the two needs to be conveyed. There needs to be a consciousness and clarity about the cultural dysjunction between the two. The staff need to learn more role-flexibility and reduce their reluctance to make decisions for students in this area of judging. The staff need to confront generally the issue of when to take responsibility for structuring, when to meet someone's need to be taught, when to provide a model or an authority.
These strategies relate also to problems of self-direction itself, and there is a strong thread that runs between self-direction and peer-assessment. Self-direction implies a knowledge of what you want: peer-assessment implies a knowledge of "standards".
To give effect to such strategies might enable peer-assessment to work. I don't think it does at present, and I don't think it will without them.
This article has been conceptual and impressionistic. There is no 'research' in it. Yet what kind of research can resolve these questions? Certainly only a new-paradigm approach could begin to work, one in which the subject of any research situation is involved in its purposes and design from the start. Ask these course members what they think of it all now, this peer-assessment, and my impression leads me to believe they would be critical over details but full of commitment to the value of having been through it.
I take my hat off to them, these heroines and heroes on the front line of transcending that boundary. But I have to say that I would consider their answers totally unreliable as a guide to resolving the issues I've raised!