System 1 error detection

As highlighted in Part 1 of this blog series, for many decades there has been a call inside and outside education proclaiming that young people ought to learn critical thinking skills at school. This is a worthy endeavour, but not a recent belief. Socrates was executed for ‘corrupting the youth’ of his time with similar thoughts. However, as previously explained, effective critical thinking is a symptom of a developing domain expertise. It is not generic. It does not stand alone separate from relevant content knowledge.

Critical thinking (as we saw with chess masters), is actually ‘System 1 thinking’ at its best. With a large amount of relevant knowledge and skills stored in long-term memory, experts can quickly and intuitively know when something is incorrect within their domain of expertise and have the ability to explain why it is erroneous.

Critical thinking is a symptom of relevant domain knowledge.

However, System 1 thinking requires relevant knowledge in long-term memory for an individual to quickly, but effectively detect errors. Without the relevant knowledge and skills stored in long-term memory, when faced with misinformation or inaccuracy, at best, a novice will ignorantly let the error pass undetected. At worst, the individual will integrate the error into their existing schema.

Ignorance is a symptom of the lack of relevant domain knowledge.

Critical thinking should obviously be a goal of educating young people. But this is a slow burning process. It only genuinely develops as individuals build their knowledge and skills across a specific domain. Unfortunately, there are no quick solutions to cultivating this domain-specific cognitive trait.

What can be done for those outside their expertise? Or for young people who are only beginning the journey of building domain expertise?

Before answering these questions directly, let me offer a somewhat strange analogy: the ability or inability to detect innuendo.

Innuendo identification

We’ve all had the experience of rewatching a childhood movie decades later and realising how much lighthearted innuendo was included that kept our parents chuckling. When we were young, we didn’t have the ability to detect the suggestive undertones of the movie content because we didn’t have the relevant and required background knowledge.

Critical thinking works exactly the same way. Without relevant background knowledge, we just don’t have a mechanism to detect misinformation. We are just like the naive children with blank faces while the adults in the room are laughing at the appropriately hidden innuendo.

Priming System 2 thinking

While reading Shakespeare with my students, if I suspect a waning of enthusiasm or motivation, a sure way to reengage cognitive engagement in teenagers is to suggest that the next scene has some cheeky innuendo for anyone paying attention. The adolescent students in the room suddenly become unwavering detectives for all things possibly a bit suggestive. Without this pre-warning, the vast majority of my Modern English-speaking high school students ignorantly read over much of Shakespeare’s smut. From the standpoint of what was illustrated above, this makes sense. When today’s students approach the Elizabethan language tied to the historical context of 16th century Britian, they lack much of the knowledge required to appreciate the nuance of the language. However, when hormone driven high school students are deliberately looking for the rude parts of the play, they usually discover what they are hoping to find, often finally using the commentary notes that I have suggested from the beginning.

What I have done here is prime System 2 thinking. When students find out there is potential insinuation, they focus their attention. They become deliberate and effortful in finding something they know is there. They are no longer in autopilot (System 1), they have cognitively slowed down to focus on finding the solution (System 2).

My central argument is that we need to prime System 2 thinking in our students when they approach the content produced by generative AI.

Sceptical Thinking vs Critical Thinking

Sceptical thinking, the more pessimistic cousin of critical thinking, is required when novices approach ChatGPT.

Scepticism is different to critical thinking. Because of the latter being tied to relevant content knowledge, those using critical thinking can successfully explain why something is incorrect. Scepticism can’t do that. By itself, sceptical thinking can’t explain why something is misleading, bias or just wrong. However, the utility of scepticism is in its assumption that inaccuracy will likely exist. For the sceptic, they won’t inherently trust what is being presented to them. Their default is not to believe, but to question.

Please don’t interpret the form of scepticism I am proposing as some sort of post-modernist distrust of oppressive power grabs or truth claims. That is something else entirely. Rather, I am arguing that sceptical thinking is a necessary lens which we ought to consciously turn on when approaching something we know may not be trustworthy. It’s a BS detector in our cognitive and emotional tool belt.

Scepticism primes system 2 thinking

This is important: even when a novice does not have the required content knowledge stored in long-term memory, scepticism primes their consciousness to trigger System 2 thinking. Just like my students trying to find the rude parts in Shakespeare’s old language, scepticism causes an individual to attempt to detect the flaws in something they assume may be present. The novice will deliberately slow down and effortfully focus their thinking to find what could need verification. By slowing down their thinking, they can access other trusted sources to fact check what is dubious.

Because of the current limitations of generative AI, it would be significantly beneficial for our young people, and novices in general, to make scepticism their default position when interacting with this technology.

Large caveat: scepticism is exhausting

Admittedly though, a sceptic’s life is cognitively exhausting. As explained in Part 1, System 2 thinking takes significantly more cognitive energy to maintain. Some of the assumptions of cognitive load theory are very relevant to this point. Our working memory is a significant bottleneck to human cognition. I explain the significance of this further here.

Using relevant content knowledge from long-term memory to critically evaluate the accuracy of information is dramatically more efficient than the System 2 process I have detailed above. Cognitively, it is just so much easier!

The cure for ignorance is not scepticism, but knowledge.

However, an individual can only experience this tremendous efficiency if they have done the hard yards of building this schema into long-term memory first. Without this schema, I argue that scepticism is a novice’s last line of defence.

Discerning when to opt out

Thus, it is important for students to be guided in developing the relevant metacognition skills to discern when it is in fact best to just stay clear of ChatGPT all together. For many tasks, the mental effort required to undertake the System 2 fact-checking of this synthetic content may actually exceed the expected time-saving efficiency. The proficiency that was hoped to be gained from including the artificial intelligence in the workflow may actually become counterproductive.

At some stage I will discuss how I’ve had students successfully use their sceptical thinking to find and fact check the errors generated by ChatGPT. I think that this has become a new obligation for teachers across most subject areas.

However, this cannot impede our ‘traditional’ duty as educators to assist students in building a broad and deep knowledge base across the disciplines taught at school. I hope you can see that in the world of developing AI, both sceptical thinking and content-enabled critical thinking are required.