assessment – Mechanical Sympathy

Cookie Cutter ‘Formal’ Exams

We were recently told that our board is moving to a formal exam for every course model. We’re told that this needs to happen because if we don’t use formal exam days for formal exams, we’ll lose the days. Perhaps we should lose the days. Formal exams are an echo from the past. Desperately trying to ‘keep’ them by forcing them on everyone isn’t the best approach to learning, it never was. Clinging to status quo thinking seldom produces outstanding results in anything.

This conundrum once again has me feeling the friction between academic and technology classrooms. To the majority of subjects in our school, an exam for every class simply means setting up more desks and running off more photocopies.

One of our auto-shop teachers tried running a ‘formal’ exam this semester. He had tinkered with a car and then had students diagnose it. Since he doesn’t have a 24 bay garage, he has to have students approach the car one at a time in order to diagnose it. Because he is expected to have all students in the room at the same time (exams are blocked into two hour scheduled time periods, one per day), he had students come up one at a time to diagnose and resolve the problems while the rest wrote written tests that did not reflect how students had learned in his class during the semester.

Cookie cutter exam schedules for cookie cutter learners.

The formal exam structure didn’t work at all in the shop. The first kid up shouted out, “do you want me to change out this fuse?” and suddenly everyone in the room knew an answer. It then kept happening. When you’ve been teaching students to collaborate on diagnostics all semester, why would you suddenly have a summative that demands they don’t? Even if that’s what a ‘formal’ exam is?

All that effort to create a genuine assessment within a standardized exam structure was wasted, but that doesn’t stop us from being expected to bring meaningful assessment to all our technology students in this cookie cutter final exam format. How meaningful can this two hour window be when our courses are tactile, stochastic and experiential? In a class where there is a linear progression from question to answer, and were the skills are assessed on paper this works a treat, but not in tech.

Coop avoids the exam problem by creating individual summatives (each student has an interview). Of course this means that each teacher is handling 25+ hours of assessment for each class they teach. I’m surprised that they can stuff all that meaningful assessment into a single exam week. While this resolves the problem of trying to fit individualized exams into cookie cutter academic schedules, it doesn’t address the complexity of creating an entire class set of experiential problems of equal complexity (you couldn’t have the same problem because the first student out would happily tell the rest what they are about to face). Creating individualized, immersive simulation for each student might be the ultimate in summatives, but a factory styled school system isn’t remotely designed to produce that kind of individualized learning opportunity.

Is this what an exam for every course looks like? Kinda like
the floor of a very serious factory, or a university…

Would I like to create a ‘formal’ exam that offers my computer students real-world, immersive, experiential computer technology problem solving? You bet, but expecting me to do that in a two hour window for dozens of students at a time suggests that the actual goal here isn’t meaningful and genuine so much as generic and formulaic, like most ‘formal’ exams.

‘Formal’ exam is code for a university-styled, written, academic assessment. It typically involves lots of photocopying and students sitting in rows writing answers to the same questions. The teacher then spends a lot of time trying to assign value to this dimensionless form of assessment. Like many other aspects of high school, formal exams are high school teachers imitating the university professors they wished they could be.

For hundreds of thousands of dollars with corporate sponsorship
and post-secondary support, Skills Ontario championships
create meaningful, experiential tech-assessment.

If you’re looking for an example of an immersive, complex, skills based assessment, we have a fantastic home-grown example. Skills Canada does a great job of creating experiential assessment of technology knowledge and tactile abilities, but with million dollar budgets and support from all levels of government, private business and post secondary education, they exist in a different world from my classroom. They’re also catering to the top 1% of 1% of technology students. I have to cater to the other 99.9% with nothing like that kind of budget.

I’ve been mulling over how I’m supposed to create meaningful assessment for my technology students in that two hour time slot and I’m stumped. No budget is forthcoming to purchase equipment and tools so that I can have every student doing the same thing at the same time – I don’t even have enough screwdrivers for all students to be building computers at the same time, let alone the computer parts needed to build them. Those would be computer parts that some students would not ground themselves properly when installing. Funding wouldn’t just need to be there for tools, it would also have to be there to replace breakage due to incompetence.

Technology teachers already struggle trying to explain technology costs to academics with only a vague understanding and little experience in apprenticeship and the trades. When students are heavy handed or absent minded it costs us money to replace what they break, yet we struggle to get funded on par with academic courses that do most of their work on paper.

Now we face the prospect of being forced to reduce our tactile, experiential, immersive learning into cookie cutter summatives that jive with the pre-existing academic scheduling. Just when you think we might be evolving beyond the 20th Century factory model of education it rears its ugly head and demands reductionist assessment for all. Wouldn’t it be nice if we were looking to diversify summatives instead of cramming them all into the same schedule that existed fifty years ago?

Assessment NOT for learning

Exams are in the bag and I’m wondering what the point was. Knowledgeable, capable students did well, incompetent students didn’t, but neither have the opportunity to learn from their exams. It begs the question: what is the point of an exam?

By high school most students think that education is something being done to them. The write-an-exam-get-a-mark approach only confirms this in their minds. If assessment isn’t for learning, what is it for? Beaurocracy? To maintain the teacher as the final arbiter in the classroom? Neither paperwork, nor maintaining hierarchical classroom structures hold much interest for me.

We’re currently being told that if we don’t make formal exams for all classes we’ll lose formal exam days. Good riddance I say! The end of a semester should include a debrief and a chance to review your summatives and assess the state of your own knowledge in terms of course expectations. This would provide a valuable pedagogical bridge between courses and empower students to take responsibility for their own learning.

From a teaching perspective, the debrief would mean that all the heavy, end of course summative assessment actually serves a purpose. It isn’t supposed to be punitive, and your grade in a class shouldn’t be a mystery to you. Assessment should be transparent and functional. Most importantly assessment should provide you with an opportunity to improve your learning; formal exams are none of those things, they are the black hole that learning falls into at the end of a course.

At the end of this course I’m going to get you to write a high stakes, stressful exam that is the same for all of you regardless of your learning styles. It’s going to assume you all have the same writing abilities. I’m then going to surprise you with the results!

I would love to ask the student who left half his exam blank, why did you do that? I’d like to understand where in his thought process he thought doing nothing was the way forward. I’d love to question the student who ignored obvious clues in a text and completely misunderstood its intent. I’m curious to see if, with a nudge, they are capable of seeing what was in front of them the whole time. I’d like to congratulate and confirm for the student who wrote a fantastic final that, yes, you really know this stuff.

There is a time and place in learning to ask the hard question: do you know what you’re doing? The end of course summative could be this reflexive learning opportunity, but not when it’s cloaked in formal exam tradition.

Instead of considering transparent, reflexive course summatives that provide assessment as learning, we’re clinging to formal exam models from the early 1900s designed to produce secretive, teacher dominated results that serve no learning purpose. If the organizational structure of a school schedule isn’t serving learning, what is it serving?

The Appearance of Credibility and Other Useless Pursuits

I’ve got two other posts on the back burner because I spent hours this weekend fabricating the appearance of credibility. It’s mid-term time, which means I’ve finally got to put together the dreaded markbook that I’ve been neglecting. I used to think I neglected it because I’m lazy, but that’s not really the case. I spend all sorts of time in and out of class getting materials, working on lesson plans and spending time individually with students. I spend most of my lunches with students offering them extra help or just space to tinker. I spend hours outside of school communicating with other teachers about education. These are not the actions of a lazy man.

So why am I so reticent to build up my markbook? Why does the idea of putting numbers into complex programs that divide and weigh marks make me roll my eyes and find something productive to do? Because it’s all about building a fiction.

Yeah, you are, but you’re a really difficult
to calculate number!

Like so much else of what we do in our nineteenth century education factory, the idea of reducing human beings to numbers so that we can define them smacks of reductive, Taylorist thinking, but reducing people to easily compared numbers is what the system demands. That grade has an aura of magic around it, we think it full of deep and profound meaning but it’s fabricated out of thin air.

Learning is a complex, rich process, but we don’t focus on that in education, we focus on gross simplifications in order to spin out self supporting statistics. We create numbers to justify the system, to give it the appearance of credibility and meaning. The system feeds the system with evidence of its own success. This goes well beyond k-12, post secondary is predicated on this fiction.

Each year we fabricate grades using complex alchemical processes. Last year I had staff say they couldn’t use Engrade because it didn’t offer enough fine control over category weighing. Our Ministry goes to great lengths to on this, and teachers agonize over it, yet no two do it the same, even in the same course, even on the same assignment.

The process of grading, from the teacher assessing a piece of work (and some of them also taking into account what the student’s sibling was like, or that they are in a bad mood that day, or that this is a nice kid who should be forgiven the odd error) to how it is entered in what mark program (it varies from teacher to teacher), makes this a very slippery slope. We’re asked to assess curriculum but in most cases the personality and circumstances of the student interfere with this to the point where getting a good read on the last, best example of their demonstrated skill is impossible. Even if it is possible, reducing their learning of complex subject areas down to a single percentage grade is absurd, yet that is what we do.

When someone says that grading is killing education I agree, but not because we should be living in a hippy commune doing whatever strikes us as fun. The fiction of grading supports other fictions, like passing. I wouldn’t trust anyone to do anything if they got it right 50% of the time, yet that is a pass in education. Grading is killing education because it is meaningless in terms of learning.

Now that I’ve built that set of grades up all is safe from questioning. You can’t question modern marking practices, they’re designed to prevent simple analysis. That markbook I built is really to make the grade I give appear credible. Look! There are mathematics at work here! This number must mean something important because it was calculated by a machine. Grade production is an arbitrary, fictitious structure based on the constantly moving sands of circumstance and personality. That it is used to discipline and direct students has more to do with enforcing the absurdity of the classroom situation than it ever did with learning. If you don’t sit in rows and capitulate you’ll fail!

If anyone says, ‘Hey! Why is that my mark?!?” I need only crack open the byzantine markbook and baffle them with categories and weights to quell any further questions. Assessment of learning has been made sufficiently obscure as to defy questioning.

What do I do? Nothing dear, you’re not qualified!
This may as well have happened in a classroom, it’s the
same approach.

We receive a great deal of PD around assessment and evaluation (you can’t serve the system unless you know what the system needs). You’d think, based on how assessment works, that learning was a professionally mandated enterprise that the layman couldn’t hope to comprehend, just the way the education complex wants you to think about learning, it’s something done to you not something you do yourself.

Unfortunately, until parents stop expecting us to reduce their children to numbers this isn’t going to change.

Until post-secondary institutions stop empowering the mythology of marks by basing entrance requirements mainly on high school grades this isn’t going to change.