Book Notes - The Checklist Manifesto

Books have always been important to me. My grandmother was a librarian and as she babysat me, I was able to spend hundreds, if not thousands, of hours searching, consuming, and falling in love with books. That passion continues today, but I find that I have less time to devote to extended, deep periods of reading.

I’m going to follow CGP Grey’s lead and start publicly sharing notes on books I’ve read. Here’s his explanation of why he started:

I’m going to attempt, for a little while anyway, to make public some of my notes from some of the books that I’ve read. This is partly because people are forever asking what I’m reading, but it’s mostly as a way to try and encourage myself to read both more deeply and more frequently – a target I have been trying, and failing, to hit for all my adult life.

All projects and hobbies of mine eventually die from lack of attention if they cannot serve multiple purposes. So it is my hope that these notes will add even more reason to engage more frequently with long-form writing.

There isn’t anything close to a public clamor for my reading list, but I do hope that this practice helps me with his second goal - increasing my incentives to read books with more depth and more frequency. Additionally, I hope that formalizing this process will help me draw better connections between the books I read and what it could mean for education and/or individual productivity.

With that said, I’m going to apply Grey’s template to the last book I finished, The Checklist Manifesto:

The Big Idea

Professionals of all varieties - programmers, doctors, teachers, pilots, etc. - perform cognitively demanding tasks on regular basis in high-stakes settings. They apply the skills they’ve honed through training, their understanding of their field, and their general experience to execute complex tasks. But no matter how competent or prepared these professionals are, they still make mistakes.

“In a complex environment, experts are up against two main difficulties. The first is the fallibility of human memory and attention, especially when it comes to mundane, routine matters that are easily overlooked under the strain of more pressing events.”

…

“A further difficulty, just as insidious, is that people can lull themselves into skipping steps even when they remember them. In complex processes, after all, certain steps don’t always matter. Perhaps the elevator controls on airplanes are usually unlocked and a check is pointless most of the time. Perhaps measuring all four vital signs uncovers a worrisome issue in only one out of fifty patients. “This has never been a problem before,” people say. Until one day it is.”

There’s nothing wrong with confidence in your ability to practice your profession, but it does breed a degree hubris that even when seemingly benign, can lead to significant errors. Gwande provides several examples of this in the medical field, where skipping seemingly obvious steps can lead to someone having their right knee opened up even if they’re in for an ACL repair on their left knee.

It’s not hard to see how this could translate to the classroom. A teacher might typically use exit tickets to gauge how much students understood the concepts taught in each lesson. It’s easy to imagine a scenario where the teacher assumes a lesson went well and forgoes this step. This may only happen once in a while, but it could be the case that while the lesson seemed to go well, students actually misunderstood the concept that was taught. It’s a much lower-stakes situation than a botched surgery, but even this small disruption could alter the success and pacing of future lessons.

The concept of minimizing mistakes in routine processes is also applicable to data analysis. Each data analysis project involves follows the broad format of importing, cleaning, transforming, and reporting data. If small steps along the way aren’t double-checked, a tiny error can lead to significant problems with the results.

Gwande stresses that using checklists to minimize error isn’t meant to dumb down the work of professionals. Instead, he argues that:

“It is common to misconceive how checklists function in complex lines of work. They are not comprehensive how-to guides, whether for building a skyscraper or getting a plane out of trouble. They are quick and simple tools aimed to buttress the skills of expert professionals. And by remaining swift and usable and resolutely modest, they are saving thousands upon thousands of lives.”

Key Passages

When the B-17 bomber was developed in the mid-1930’s, it was the most complex aircraft ever built. It had four engines when nearly every plane had one or maybe two engines. This, along with the plane’s other features, made the task of piloting it incredibly complex. The first time the plane was tested, it crashed shortly after takeoff not due to any mechanical issue, but to pilot error. The new design was simply too complex for one person to manage on their own.

Development on the plane continued and another test flight was scheduled. This time, the pilots developed what is now a staple of aviation: a checklist. It was concise, but it ultimately helped flight crews man the B-17 without incident from that point forward.

I found this to be a terrific example of intellectual humility by the test pilots. They were extremely well-trained, but they were willing to admit to themselves that while they were experts, their brains weren’t reliable enough to ensure perfect execution every single time.

“Substantial parts of what software designers, financial managers, firefighters, police officers, lawyers, and most certainly clinicians do are now too complex for them to carry out reliably from memory alone. Multiple fields, in other words, have become too much airplane for one person to fly.”

Educators and data analysts alike find themselves with “too much airplane” right now. We need to have the humility to:

1. Admit that these are jobs we can’t do alone.
2. Develop tools to help overcome the problem of “too much airplane.”

Actionable Items

“You want people to make sure to get the stupid stuff right. Yet you also want to leave room for craft and judgment and the ability to respond to unexpected difficulties that arise along the way”

Checklists must strike an important balance. They need to cover enough critical points in a process to minimize the “stupid stuff” that could happen, but they also need the flexibility and brevity to be useful in a real-world setting. Gwande explains the difference between a good checklist and a bad one:

“Bad checklists are vague and imprecise. They are too long; they are hard to use; they are impractical. They are made by desk jockeys with no awareness of the situations in which they are to be deployed. They treat the people using the tools as dumb and try to spell out every single step. They turn people’s brains off rather than turn them on.

Good checklists, on the other hand, are precise. They are efficient, to the point, and easy to use even in the most difficult situations. They do not try to spell out everything—a checklist cannot fly a plane. Instead, they provide reminders of only the most critical and important steps—the ones that even the highly skilled professionals using them could miss. Good checklists are, above all, practical.”

After developing a good, concise checklist comes the most critical step: implementation. As Gwande first developed his surgical checklist, he practiced using it with his surgical team to figure out what worked, what didn’t, and how they could improve it before using it in an actual surgical setting.

Once the checklist was tested, it needed to gain adoption with other surgeons and their teams. Gwande started introducing it to chief surgeons - the idea was that if a hospital’s leadership was able to embrace the change, others would follow. This is exactly what happened, and the incidence of surgical errors began to drop. But then something else began to happen:

“Spot surveys of random staff members coming out of surgery after the checklist was in effect did indeed report a significant increase in the level of communication. There was also a notable correlation between teamwork scores and results for patients—the greater the improvement in teamwork, the greater the drop in complications.”

The checklist process actually helped keep surgical teams more aware of what each person was doing by establishing a template for regular communication between everyone in the operating room. It’s an example of the hidden benefit of checklists:

“Just ticking boxes is not the ultimate goal here. Embracing a culture of teamwork and discipline is.”

Setting up checklists can not only improve processes, it can help build stronger teams. For educators and analysts alike, that’s a goal worthy of pursuit.