The Illusion of Knowing

Advanced Learning has commissioned me to write a piece about the uses and abuses of data in schools. My thesis, if that's not too grand a term, is that while data can be extraordinarily useful in helping us make good decisions, too much data leads, inexorably, to overload. When we have too much data we start doing silly things with it, just because we have it. The cost of bad data is the conviction that we have figured out all the possible permutations and know exactly what we're doing and why we're doing it. This is an illusion.

Here's the introduction:

The more data schools collect, the better they will understand their students, right? Well maybe not. Much of the data schools collect distorts how we think and warps decisions we make about what to teach. The illusion of knowing is comforting, but maybe we’d be better off confronting the uncomfortable truth that we know far less than we suppose. As we find stuff out, we reveal new boundaries to what we know. As the island of knowledge grows, so does its shoreline and beyond that shore is a vast ocean of ignorance. Nate Silver said “Even if the amount of knowledge in the world is increasing, the gap between what we know and what we think we know may be widening.”

And, if you're interested in reading more, here's the paper. See what you think.

Acknowledgements and thanks must go to Jack Marwood, Dylan Wiliam, David Thomas, Chris Wheadon, Rob Coe and Colin Hegarty who have all helped expand the coastline of my ignorance in different ways.