grading – Mechanical Sympathy

Proliferation of Fifties

Our school is the only local high school in the area. If students want Catholic or special education, they get on a school bus for over an hour a day of commuting down to Guelph. I’m a big fan of choice so, while I think it mad, I don’t have much to say about a student who wants to spend over 194 hours a year (that’s over 8 full days of riding 24 hours a day) on a bus to Guelph and back for specialized education, as long as it’s a choice they’ve made.

Ontario’s high school streams seem pretty straightforward,
they are anything but in practice.

Our public board think it wise to ship our essential level students down to Guelph for special education. This isn’t a choice, it’s a system driven process. The Guelph school for this doesn’t fill up with locals so the surrounding community schools are expected to ship their most at-need students out of their home communities every day. This is an ongoing pressure in our community.

At our recent heads’ meeting there seemed to be support for the idea of our school being a comprehensive, community school that serves everyone, but we struggle to run essential sections because parents resist putting their children into it, the board doesn’t section us to run smaller essential classes and many teachers in our school would rather be teaching academic students. It’s an uphill struggle to create a comprehensive local school that supports everyone in our community.

Because we aren’t sectioned for essential classes (those smaller sections are given to the specialist school in Guelph), we end up populating applied level classes with essential students. It is so difficult to align parent perception, board support and student ability that we place all non-academic students into the same room. This is where the proliferation of fifties comes in.

A teacher in our school recently said, and in retrospect I agree, that we place essential students into applied classes and lower course expectations to accommodate them. This not only does the essential students no favours, it also dilutes applied curriculum goals.

The people running the education system tend to be successful professional educationalists; very experienced with the system having spent little time outside it. These educators see kindred spirits in academically streamed students who are successful in school and make effective use of the system. These teachers want to teach students like themselves. Asking them to work with students who find school a challenging environment or aren’t on the same academic trajectory they experienced is difficult for them.

The predisposition of teachers makes academic curriculum somewhat sacred, but applied classes aren’t. Applied students should be on apprenticeship and college skilled labour tracks that demand hands on (applied?) skills. While less theoretical in approach, applied classes are supposed to be rigorously skills focused. When you put students who lack basic literacy and numeracy into a grade 10 applied class you make grade appropriate learning nearly impossible.

How do teachers manage this? If you fail a student, you get called into promotion meetings at the end of the semester where the grade you’ve given becomes the starting point for an inflationary process that floats fails up to passes. The best way to avoid this is to simply award a 50%. What is a fifty when it’s really a 42? At its best, a fifty means a student has not reached minimal expectations for a class. Would you want the mechanic working on your brakes to have gotten there with fifties?

The teacher I was talking to suggested that the number of fifties being handed out has mushroomed in the past few years. Those statistics aren’t made available to us because they would make a travesty of curriculum expectations, but I suspect he is right. A fifty means the government gets to say graduation rates are up. A fifty means the ride ends at graduation because no secondary program would accept a student with a D average. A fifty means you’re not sitting in promotion meetings watching your semester of careful assessment being swept away to support policy.

The range of student skill in my classes is astonishing. My current grade 9 classes range from students who could comfortable complete grade 11 computer engineering curriculum next to students who appear unable to read, yet I’m supposed to address that range of skills in a 50-100% range in a single course.

Perhaps we will find a way to reintegrate Ontario’s carefully designed secondary school streaming system, but considering the various pressures on it in our area, it’s going to be an uphill struggle.

NOTE

Re: school busing children…

Time isn’t the only resource being spent. School buses get 6-8mpg, Guelph is about 15 miles away. A (very conservative) 30 mile round trip (it’s much higher if you want to consider all the pickups and drop-offs) is a (very conservative) 15 litres per day of diesel (probably double that for your typical start/stop run), per bus, and we have a number of buses making that trip 194 days per year.

Someone better than I can calculate the overall environmental impact (how many other vehicles are also held up burning fuel while these buses grind down to and back from Guelph every day?). Making an economic (let alone moral) argument for shipping our essential students out of their home communities seems impossible.

Cookie Cutter ‘Formal’ Exams

We were recently told that our board is moving to a formal exam for every course model. We’re told that this needs to happen because if we don’t use formal exam days for formal exams, we’ll lose the days. Perhaps we should lose the days. Formal exams are an echo from the past. Desperately trying to ‘keep’ them by forcing them on everyone isn’t the best approach to learning, it never was. Clinging to status quo thinking seldom produces outstanding results in anything.

This conundrum once again has me feeling the friction between academic and technology classrooms. To the majority of subjects in our school, an exam for every class simply means setting up more desks and running off more photocopies.

One of our auto-shop teachers tried running a ‘formal’ exam this semester. He had tinkered with a car and then had students diagnose it. Since he doesn’t have a 24 bay garage, he has to have students approach the car one at a time in order to diagnose it. Because he is expected to have all students in the room at the same time (exams are blocked into two hour scheduled time periods, one per day), he had students come up one at a time to diagnose and resolve the problems while the rest wrote written tests that did not reflect how students had learned in his class during the semester.

Cookie cutter exam schedules for cookie cutter learners.

The formal exam structure didn’t work at all in the shop. The first kid up shouted out, “do you want me to change out this fuse?” and suddenly everyone in the room knew an answer. It then kept happening. When you’ve been teaching students to collaborate on diagnostics all semester, why would you suddenly have a summative that demands they don’t? Even if that’s what a ‘formal’ exam is?

All that effort to create a genuine assessment within a standardized exam structure was wasted, but that doesn’t stop us from being expected to bring meaningful assessment to all our technology students in this cookie cutter final exam format. How meaningful can this two hour window be when our courses are tactile, stochastic and experiential? In a class where there is a linear progression from question to answer, and were the skills are assessed on paper this works a treat, but not in tech.

Coop avoids the exam problem by creating individual summatives (each student has an interview). Of course this means that each teacher is handling 25+ hours of assessment for each class they teach. I’m surprised that they can stuff all that meaningful assessment into a single exam week. While this resolves the problem of trying to fit individualized exams into cookie cutter academic schedules, it doesn’t address the complexity of creating an entire class set of experiential problems of equal complexity (you couldn’t have the same problem because the first student out would happily tell the rest what they are about to face). Creating individualized, immersive simulation for each student might be the ultimate in summatives, but a factory styled school system isn’t remotely designed to produce that kind of individualized learning opportunity.

Is this what an exam for every course looks like? Kinda like
the floor of a very serious factory, or a university…

Would I like to create a ‘formal’ exam that offers my computer students real-world, immersive, experiential computer technology problem solving? You bet, but expecting me to do that in a two hour window for dozens of students at a time suggests that the actual goal here isn’t meaningful and genuine so much as generic and formulaic, like most ‘formal’ exams.

‘Formal’ exam is code for a university-styled, written, academic assessment. It typically involves lots of photocopying and students sitting in rows writing answers to the same questions. The teacher then spends a lot of time trying to assign value to this dimensionless form of assessment. Like many other aspects of high school, formal exams are high school teachers imitating the university professors they wished they could be.

For hundreds of thousands of dollars with corporate sponsorship
and post-secondary support, Skills Ontario championships
create meaningful, experiential tech-assessment.

If you’re looking for an example of an immersive, complex, skills based assessment, we have a fantastic home-grown example. Skills Canada does a great job of creating experiential assessment of technology knowledge and tactile abilities, but with million dollar budgets and support from all levels of government, private business and post secondary education, they exist in a different world from my classroom. They’re also catering to the top 1% of 1% of technology students. I have to cater to the other 99.9% with nothing like that kind of budget.

I’ve been mulling over how I’m supposed to create meaningful assessment for my technology students in that two hour time slot and I’m stumped. No budget is forthcoming to purchase equipment and tools so that I can have every student doing the same thing at the same time – I don’t even have enough screwdrivers for all students to be building computers at the same time, let alone the computer parts needed to build them. Those would be computer parts that some students would not ground themselves properly when installing. Funding wouldn’t just need to be there for tools, it would also have to be there to replace breakage due to incompetence.

Technology teachers already struggle trying to explain technology costs to academics with only a vague understanding and little experience in apprenticeship and the trades. When students are heavy handed or absent minded it costs us money to replace what they break, yet we struggle to get funded on par with academic courses that do most of their work on paper.

Now we face the prospect of being forced to reduce our tactile, experiential, immersive learning into cookie cutter summatives that jive with the pre-existing academic scheduling. Just when you think we might be evolving beyond the 20th Century factory model of education it rears its ugly head and demands reductionist assessment for all. Wouldn’t it be nice if we were looking to diversify summatives instead of cramming them all into the same schedule that existed fifty years ago?

Assessment NOT for learning

Exams are in the bag and I’m wondering what the point was. Knowledgeable, capable students did well, incompetent students didn’t, but neither have the opportunity to learn from their exams. It begs the question: what is the point of an exam?

By high school most students think that education is something being done to them. The write-an-exam-get-a-mark approach only confirms this in their minds. If assessment isn’t for learning, what is it for? Beaurocracy? To maintain the teacher as the final arbiter in the classroom? Neither paperwork, nor maintaining hierarchical classroom structures hold much interest for me.

We’re currently being told that if we don’t make formal exams for all classes we’ll lose formal exam days. Good riddance I say! The end of a semester should include a debrief and a chance to review your summatives and assess the state of your own knowledge in terms of course expectations. This would provide a valuable pedagogical bridge between courses and empower students to take responsibility for their own learning.

From a teaching perspective, the debrief would mean that all the heavy, end of course summative assessment actually serves a purpose. It isn’t supposed to be punitive, and your grade in a class shouldn’t be a mystery to you. Assessment should be transparent and functional. Most importantly assessment should provide you with an opportunity to improve your learning; formal exams are none of those things, they are the black hole that learning falls into at the end of a course.

At the end of this course I’m going to get you to write a high stakes, stressful exam that is the same for all of you regardless of your learning styles. It’s going to assume you all have the same writing abilities. I’m then going to surprise you with the results!

I would love to ask the student who left half his exam blank, why did you do that? I’d like to understand where in his thought process he thought doing nothing was the way forward. I’d love to question the student who ignored obvious clues in a text and completely misunderstood its intent. I’m curious to see if, with a nudge, they are capable of seeing what was in front of them the whole time. I’d like to congratulate and confirm for the student who wrote a fantastic final that, yes, you really know this stuff.

There is a time and place in learning to ask the hard question: do you know what you’re doing? The end of course summative could be this reflexive learning opportunity, but not when it’s cloaked in formal exam tradition.

Instead of considering transparent, reflexive course summatives that provide assessment as learning, we’re clinging to formal exam models from the early 1900s designed to produce secretive, teacher dominated results that serve no learning purpose. If the organizational structure of a school schedule isn’t serving learning, what is it serving?

The Appearance of Credibility and Other Useless Pursuits

I’ve got two other posts on the back burner because I spent hours this weekend fabricating the appearance of credibility. It’s mid-term time, which means I’ve finally got to put together the dreaded markbook that I’ve been neglecting. I used to think I neglected it because I’m lazy, but that’s not really the case. I spend all sorts of time in and out of class getting materials, working on lesson plans and spending time individually with students. I spend most of my lunches with students offering them extra help or just space to tinker. I spend hours outside of school communicating with other teachers about education. These are not the actions of a lazy man.

So why am I so reticent to build up my markbook? Why does the idea of putting numbers into complex programs that divide and weigh marks make me roll my eyes and find something productive to do? Because it’s all about building a fiction.

Yeah, you are, but you’re a really difficult
to calculate number!

Like so much else of what we do in our nineteenth century education factory, the idea of reducing human beings to numbers so that we can define them smacks of reductive, Taylorist thinking, but reducing people to easily compared numbers is what the system demands. That grade has an aura of magic around it, we think it full of deep and profound meaning but it’s fabricated out of thin air.

Learning is a complex, rich process, but we don’t focus on that in education, we focus on gross simplifications in order to spin out self supporting statistics. We create numbers to justify the system, to give it the appearance of credibility and meaning. The system feeds the system with evidence of its own success. This goes well beyond k-12, post secondary is predicated on this fiction.

Each year we fabricate grades using complex alchemical processes. Last year I had staff say they couldn’t use Engrade because it didn’t offer enough fine control over category weighing. Our Ministry goes to great lengths to on this, and teachers agonize over it, yet no two do it the same, even in the same course, even on the same assignment.

The process of grading, from the teacher assessing a piece of work (and some of them also taking into account what the student’s sibling was like, or that they are in a bad mood that day, or that this is a nice kid who should be forgiven the odd error) to how it is entered in what mark program (it varies from teacher to teacher), makes this a very slippery slope. We’re asked to assess curriculum but in most cases the personality and circumstances of the student interfere with this to the point where getting a good read on the last, best example of their demonstrated skill is impossible. Even if it is possible, reducing their learning of complex subject areas down to a single percentage grade is absurd, yet that is what we do.

When someone says that grading is killing education I agree, but not because we should be living in a hippy commune doing whatever strikes us as fun. The fiction of grading supports other fictions, like passing. I wouldn’t trust anyone to do anything if they got it right 50% of the time, yet that is a pass in education. Grading is killing education because it is meaningless in terms of learning.

Now that I’ve built that set of grades up all is safe from questioning. You can’t question modern marking practices, they’re designed to prevent simple analysis. That markbook I built is really to make the grade I give appear credible. Look! There are mathematics at work here! This number must mean something important because it was calculated by a machine. Grade production is an arbitrary, fictitious structure based on the constantly moving sands of circumstance and personality. That it is used to discipline and direct students has more to do with enforcing the absurdity of the classroom situation than it ever did with learning. If you don’t sit in rows and capitulate you’ll fail!

If anyone says, ‘Hey! Why is that my mark?!?” I need only crack open the byzantine markbook and baffle them with categories and weights to quell any further questions. Assessment of learning has been made sufficiently obscure as to defy questioning.

What do I do? Nothing dear, you’re not qualified!
This may as well have happened in a classroom, it’s the
same approach.

We receive a great deal of PD around assessment and evaluation (you can’t serve the system unless you know what the system needs). You’d think, based on how assessment works, that learning was a professionally mandated enterprise that the layman couldn’t hope to comprehend, just the way the education complex wants you to think about learning, it’s something done to you not something you do yourself.

Unfortunately, until parents stop expecting us to reduce their children to numbers this isn’t going to change.

Until post-secondary institutions stop empowering the mythology of marks by basing entrance requirements mainly on high school grades this isn’t going to change.

The New Efficiency

This African proverb passed me on the
internet last week, and left me thinking.

Originally published on Dusty World in June of 2015:

https://temkblog.blogspot.com/2015/06/the-new-efficiency.html

Last semester I had an energetic grade 9 suddenly stop his interaction with the internet and wonder out loud (and it was asked in all seriousness): “why is it in video games and movies old people are so cool, with hidden knowledge and special powers, but in real life they just suck?”

He received an avalanche of ‘how could you say that?!’, but then everyone went on to say how wonderful their grandparents were. Everyone loves their grandparents, but no one was willing to defend age and by extension experience in and of itself.

When this African proverb popped up I immediately felt the pinch of that class discussion (yes, I know, we were talking about the value of age and experience in a class where I was supposed to be teaching computer engineering, I guess my kids won’t be ready for whatever standardized test they invent for it).

What role does age and experience have in the information age? This proverb also refers to libraries, which have been facing their own test of relevance thanks to the Googliable nature of information.

Information technology has made personal knowledge irrelevant. The life experiences of human beings have become meaningless, replaced by internet searches. Why would you bother to ask your grandfather how to change the brakes on your car when you can just Google it? Once a useful source of information, the elders around you are now objects of affection and little more, they serve no function. You can get the information you need without any of the static (anecdotal stories that accompany the information). This sanitized, machine driven version of knowledge has many benefits.

You can reduce complex human knowledge (for example, the development of literacy) into simplistic, easy to quantify standards and then make sweeping suppositions about the results. Banal opinion based on internet ‘fact’ is the new intelligence. Like any opinion you hear online, carefully crafted grading schemes end up becoming the truth, which fits nicely into the antiseptic version of knowledge the information age peddles.

Another benefit is the downward social pressure on human communities. When you plug people into a centralized source of information you wean them from the social necessities of family, community and even nation. When no one needs anyone else (but they do need an ISP), you have removed all the social static and laid the groundwork for a kind of hypercapitalism that will make past look like the middle ages!

When we try and argue for meaningful learning (in anything other than a poster), we are met with educational administration making sad faces and saying it’s not viable. The reasonable provision of caps on class sizes is just such an attempt, which is why the meme on the right goes straight to the heart of this issue.

Tangible data that grossly oversimplify human endeavour are how we roll nowadays. As the poster states, class caps mean nothing, but fail to hand out a piece of paper with grades so abstract that they are meaningless along with computer generated comments, and ‘everyone loses their minds!’

There is some push back against the dimensionless facts that drive the information age. You find it in the physical world in grass roots movements like slow food or maker spaces where you see individuals trying to wrest control of production from the hands of remote systems. In these places the idea of human interaction is key to the process of learning. They are trying to build communities in an arid digital landscape that is bereft complex human interaction… unless they are under a corporate banner; communities designed for marketing purposes. What would be the economic sense in creating a community solely for the benefit its members?

Ironically, human interaction is less and less a factor in human education. The push to integrate technology into pedagogy without considering its implications has infected education systems with the same efficiency that we now enjoy everywhere else. We can hardly expect the personally demeaned yet highly efficient funployees in the private sector to demand anything other than consistent menial labour, it’s what they do. Developing complex personal relationships in order to effectively mentor and teach aren’t very efficient/economically viable. They are certainly discouraged in the brave new world of 21st Century education where teachers are now facilitators, reduced to getting out of the way of learning and making sure the #edtech is working.

One of my students from many years ago is now out in the world. She was sitting in a restaurant a few weeks ago watching two employees, a teenage girl and an older woman on their break. The older woman kept trying to start a conversation. The teen ignored her, buried in her phone until she finally snapped, ‘What? What do you want?” She was incensed that this woman had interrupted her texting time. She was probably in withdrawal because they don’t let her have the phone while working. I can bet which one of those two employees gets better performance reviews, though she sounds like an ass.

Maybe human experience is meaningless nowadays. Maybe old people are useless and libraries are a waste of space (great idea: replace every one in school with franchise coffee shops to balance the books!). Maybe we don’t need each other to learn any more, it’s certainly not as efficient.

LINKS

Watch the new efficiency infect the UK’s Labour Party
“In 2015 we are living in a cold, cruel, and desolate country in which benefit sanctions, foodbanks, poverty wages, and ignorance reign, governed by a clutch of rich, privately educated sociopaths whose conception of society has been ripped straight from the pages of a dystopian novel.”

A Thin & Fragile Pretense

I’m still mulling my way through The World Beyond Your Head, by Matt Crawford. It’s a slow go because I’m re-reading and thinking over what I’m looking at, often paragraph by paragraph.

On page 153-4 Crawford is talking about the way in which we depend on established values when transacting with each other. He is talking about how he bills his motorcycle repairs, but I found a surprising correlation between this and my current views on grading:

P.153-54 The World Beyond Your Head by Matt Crawford

This could easily be re-written to describe my own battle with grading:

Consider the case of a teacher. In handing a final grade to a student, I make a claim for the value of what they know about what I have taught them, and put it to them in the most direct way possible (a grade). I have to steel myself for this moment; it feels like a confrontation. (I hate grading, I feel it actively discourages learning by implying there is a definitive end)

The point of having posted criteria, rubrics, due dates, class rules, and the use of complex grading systems with byzantine weights and balances, is to create the impression of calculation, and to appeal to the authority of an institution with established rules. But this is a thin and fragile pretense observed by me and my student – in fact the grade I present is never a straightforward account of the skill of a student. It always involves a reflection in which I try to put myself in the shoes of the other and imagine what he might find reasonable. (Freeing myself from the tyranny of grading programs is both professionally satisfying and existentially terrifying – what are we all doing here if not making numbers?!?)

This lack of straightforwardness in valuing learning is due to the fact that learning is subject to chance and mishap, as well as many diagnostic obscurities. Like medicine, teaching and learning are what Aristotle calls “stochastic” arts. Especially when working on complex skills at the high school level, in trying to teach one discipline (learning how to code), I may unearth problems in another (the student has little grasp of basic logic). How should I grade for work done to solve a problem beyond the realm of what I’m supposed to be teaching? Should I hand off this new problem to spec-ed, or simply blame previous grades and move on? (I do neither, I consider a student who is able to overcome previous failings to catch up to his peers to be superior to a student who is simply going through the motions because this is easy repetition for them) This question has to be answered when I formulate a grade, and in doing so I find that I compose little justificatory narratives.

When a student receives a grade, I usually go over the reasons with them in detail, and I often find myself delaying the presentation of the grade, because I fear that my valuation isn’t justified (I can never have all the facts needed to be completely accurate). But all my fretting about the grade has to get condensed into a simplistic number for the sake of systemic learning on an established schedule (our education system is predicated on the receiving of numbers that are so abstract as to be virtually meaningless). Whatever conversation may ensue, in the end the grade achieves a valuation that is determinate: a certain amount of educational value exchanges hands. As the student leaves the class for the last time, I want to feel that they feel they have gotten a square deal in terms of me not using grades as either a gift or a punishment; I want to come away feeling justified in the claim I made for what I think they know and can do. (but many teachers don’t – empathy and grading can be safely made mutually exclusive thanks to the absolute truth of mathematics, the more complex the calculation, the truer the grade it produces must be)