The medical community uses checklists. They've been using checklists for a while. A number of studies found that they only help for a few months, while they're new.
That said, "the medical community" is not a homogeneous monolith, and you can absolutely find regional variation in what checklists are used for, how detailed they are, how closely they're followed, how people are accountable for keeping to them, etc.
"The Checklist Manifesto" chose to overlook the studies about how transient the benefit of checklists is.
> A number of studies found that they only help for a few months, while they're new.
This 2014 review looked at 34 studies that looked at the effectiveness of safety checklists:
> The main findings were improved communication, reduced adverse events, better adherence to standard operating procedures, and reduced morbidity and mortality. None of the included studies reported decreased patient safety or quality after introducing safety checklists.
> Safety checklists appear to be effective tools for improving patient safety in various clinical settings by strengthening compliance with guidelines, improving human factors, reducing the incidence of adverse events, and decreasing mortality and morbidity. None of the included studies reported negative effects on safety.
The studies concerning long-term impact of surgical checklists I found all claim an improvement.
> Sustained use of the checklist was observed with continued improvements in process measures and reductions in 30-day surgical complications almost 2 years after a structured implementation effort that demonstrated marked, short-term reductions in harm. The sustained effect occurred despite the absence of continued oversight by the research team, indicating the important role that local leadership and local champions play in the success of quality improvement initiatives, especially in resource-limited settings.
>Only 3 of 22 operating stations (13.6%) had a functioning pulse oximeter prior to the intervention; accordingly, a pulse oximeter (model 7500; Nonin Medical Inc) was installed at each operating station as part of the implementation effort
The addition of essential medical equipment surely won't bias our results!
One could argue the essential medical equipment was missing due to lack of checklists ensuring its existence. That said, yeah, it would have been better to do 2 separate studies then: one on adding the equipment and then one on checklists.
> A number of studies found that they only help for a few months, while they're new.
Interesting, that kind of surprises me. Without going to research it right now, I wonder how/if the efficacy is affected by things like going through them in tandem with a partner, or doing them out loud or with exaggerated physical acknowledgements (like the Japanese train drivers who point out signs/notices along their route as a way to maintain focus on those notices).
A speculative hypothesis related to this (based on my own experience as someone who has read the Checklist Manifesto and is a big advocate for checklists):
Over time people internalize the checklists because they unwittingly conclude the reason for the checklist is to learn a new procedure rather than foolproof their procedures and avoid errors. As a result, they start to skip the checklist or get sloppy with it.
There are a couple other points related to this that I believe Gawande does address in his book:
1. Doctors (more generally experts or egoists) will be resistant to checklists because they find them unnecessary (given their experience or expertise).
2. Checklists need to managed with regular review and updates.
You can boil Gawande's book down to: "Start using checklists because they're really effective." But his book goes deeper than that and addresses some of the underlying human factors involved in getting an organization to use checklists effectively.
> Doctors (more generally experts or egoists) will be resistant to checklists because they find them unnecessary (given their experience or expertise).
At the end of Chapter 7 of Gawande's book (The Checklist Manifesto) is this revealing tale:
<quote>
Nonetheless, some skepticism persisted. After all, 20 percent did not find it easy to use, thought it took too long, and felt it had not improved the safety of care.
Then we asked the staff one more question. “If you were having an operation,” we asked, “would you want the checklist to be used?”
So basically people stop doing checklists and then claim they don’t work. Checklists are great, but they have to be used. Instead of treating them as a learning process, I treat them as something I explicitly do not have to learn. It frees my mind for other things.
I don't know that this is the case. I was just speculating in response to the claim upstream that Gawande "chose to overlook the studies about how transient the benefit of checklists is." I too would like to see the citations. If there was a study showing this regression, I would be interested to see if it controlled for this sort of factor.
But I agree with you wholeheartedly. I don't want my mind cluttered with stuff that can be handled by simple script.
"Dropping protections because the failure rate dropped" is a common failure mode, as seen in: Checklists, vaccines, financial regulations, and I'm sure many more arenas.
That was the explicit justification for the Supreme Court finding that the Voting Rights Act wasn't necessary, from memory - states weren't doing the things it banned, so it wasn't necessary to ban them any more.
Makes me feel like there's a similarity between implementation of checklist methodology and implementation of agile methodology. Depends so much on culture and processes and people.
While we're guessing, I would figure it just becomes drilled into people as automatic & a given, the same way you buckle your seatbelt without thinking about it.
> "The Checklist Manifesto" chose to overlook the studies about how transient the benefit of checklists is.
It's been almost a year now since I read this but I'm fairly certain Atul touched on this, either in this book or in his book "Better", that after those implementing the changes left, the departments often fell back into old habits.
This isn't just with checklists, but a typical behavior of organizations (at any scale). Organizations are systems, systems tend to fall into a steady state of performance, quality, etc. after some period of time. If you draw the people's attention to something (checklists, quotas, threats, praise, etc.) you can see a temporary boost in performance, but unless the culture changes the performance tends to return to that same steady state (maybe slightly enhanced or worse).
In order for the benefit to stick, you need to actually change the system. Otherwise, complacency or other things will return when the pressure and other early benefits wear off.
In other words it isn't having a checklist, it is actually checking that people follow the checklist. Note that follow the checklist is different from checking all items on the checklist which is an obvious work around to the requirement that you follow a checklist.
GitHub and GitLab support task checklists in Markdown and also project boards which add and remove labels like 'ready' and 'in progress' when cards are moved between board columns; like kanban:
- [ ] not complete
- [x] completed
Other tools support additional per-task workflow states:
- [o] open
- [x (2019-04-17)] completed on date
I worked on a large hospital internal software project where the task was to build a system for reusable checklists editable through the web that prints them out in duplicate or triplicate at nearby printers. People really liked having the tangible paper copy.
"The Checklist Manifesto" by Atul Gawande was published while I worked there. TIL pilots have been using checklists for process control in order to reduce error for many years.
Evernote, RememberTheMilk, Google Tasks, and Google Keep all support checklists. Asana and Gitea and TaskWarrior support task dependencies.
A person could carry around a Hipster PDA with Bullet Journal style tasks lists with checkboxes; printed from a GTD service with an API and a @media print CSS stylesheet:
https://en.wikipedia.org/wiki/Hipster_PDA
I'm not aware of very many tools that support authoring reusable checklists with structured data elements and data validation.
...
There are a number of configuration management systems like Puppet, Chef, Salt, and Ansible that build a graph of completable and verifiable tasks and then depth-first traverse said graph (either with hash randomization resulting in sometimes different traversals or with source order as an implicit ordering)
Resource scheduling systems like operating systems and conference room schedulers can take ~task priority into account when optimally ordering tasks given available resources; like triage.
I'll often be on call with customer and create a checklist on MacOS Notes on the fly. Then will copy paste that in slack or github for simple tracking.
Mattermost supports threaded replies and Markdown with checklist checkboxes
You can post GitHub/GitLab project updates to a Slack/Mattermost channel with webhooks (and search for and display GH/GL issues with /slash commands); though issue edits and checkbox state changes aren't (yet?) included in the events that channels receive.
“A simple, free and powerful way to manage your team's recurring checklists and procedures“ https://www.process.st/ Paid features include 3rd party integrations. (Disclaimer: neither a customer nor employee of the company.)
Do any of these let you re-use your checklists, though? Planner at least is pretty useless for this, as I don't think there is a 'reset all items to un-checked' feature, or even basic stuff like re-arranging your list.
Everything in my comment was describing Planner. You cannot re-arrange checklists on a task in Planner on mobile. And I'm not clear if you were describing Planner in your comment, but I just tested the workflow described and it results in a second copy of my task with the same checklist items checked.
That said, "the medical community" is not a homogeneous monolith, and you can absolutely find regional variation in what checklists are used for, how detailed they are, how closely they're followed, how people are accountable for keeping to them, etc.
"The Checklist Manifesto" chose to overlook the studies about how transient the benefit of checklists is.