The sentence was: this source may be biased, so it is important to consider other perspectives. It appeared, with minor variations, in nine of the thirty-one papers I marked that week, and under the rubric we were given it earned full credit for evaluation of sources every time.
The rubric was not badly written. I helped write it.
I want to lay out, in order, why I no longer believe critical thinking can be taught as a general skill, because the argument only works as a sequence and every step of it is one I resisted.
The first step is the observation above, and by itself it proves nothing. Nine students produced a formula. Students have always produced formulas, and a rubric will always be gameable by somebody who has worked out what it rewards. That is an argument about assessment design, not about the skill.
The second step is what happens when you close that hole. We rewrote the rubric to require the student to name the specific interest that would produce the specific distortion. This is harder to fake and it worked, in the sense that the formula sentence disappeared. What replaced it, in most papers, was a slightly longer formula. The student named an interest — the author works for the industry — and asserted a distortion without being able to say what a distorted claim would look like as opposed to a true one, because that requires knowing something about the industry.
The third step is the one that changed my mind, and it came from a comparison I did not set up deliberately. The same students, in a unit on a topic several of them knew well — a local water dispute that had been in the news for two years and involved their families — produced source evaluation that was genuinely good. Not formulaic. Specific, sceptical in the right places, credulous in defensible ones, and different from student to student in ways that tracked what each of them actually knew.
They did not acquire a skill between March and April. They acquired subject matter, and the thing we call critical thinking turned out to be what having subject matter looks like from outside.
The fourth step is the generalisation, and it is where I have to be careful, because this is the point at which the argument gets overstated by people I broadly agree with. I am not saying transfer never happens. There is a real literature showing modest transfer from explicit instruction in specific reasoning patterns — base rates, controls, sample size — and I take it seriously and I teach those things. What that literature does not show, and what our rubric assumed, is that a general disposition to think critically can be trained and then deployed across domains at anything like the strength it has inside a domain the student knows.
What follows is uncomfortable for a subject like mine, which sells itself on producing exactly that disposition.
If the skill is domain-bound, then a course promising critical thinking in the abstract is promising something it cannot deliver, and the best it can honestly offer is critical thinking about the four or five things it teaches properly. That is a much smaller claim and a much less fundable one. It also implies that the way to make students better at evaluating sources is to teach them more content, which is the exact position that everyone in my faculty spent the 2010s arguing against.
I am not certain of this. What I am certain of is that nine students wrote the same sentence, that it was true, that it was worth no marks whatsoever, and that we gave it full ones for two years before anyone noticed — including me, who wrote the rubric, and who read that sentence perhaps a hundred times before I read it.
