I don’t generally use a to-do list unless something is really important.
If I’m taking off on a long trip and need to be sure I don’t forget something, I’ll make a list the night before. If I forget something, adjusting on the plane or the road can be difficult or impossible. But I don’t start the workday with a to-do list. I know the day is going to throw me some surprises, and what’s important by the end of the day won’t be the same as what was important in the morning. So I don’t bother with a list. I just start the day, do what’s important and figure I’ll get a lot of important work done. Most days I do.
When I heard Craig Silverman talk about how effective checklists are in preventing errors, I decided I needed a checklist. After all, what’s more important than accuracy?
I had seen mentions of Craig’s checklist on his website, Regret the Error. But I guess I kind of dismissed it as too simple a tool for a veteran journalist like me. And, you know, I’m not a to-do-list sort.
But when Craig led his accuracy workshop for TBD in October, he called the checklist the most effective error-prevention system ever developed (read more about that in his Checklist Charlie post). He noted that veteran pilots use checklists. Famed US Airways pilot Chesley Sullenberger (Sully!) used a checklist for his emergency landing in the Hudson River. Surgeons using checklists reduce their error rates significantly, Craig noted.
When Craig’s discussion shifted to journalism, it started getting uncomfortably familiar. One of the most common sources of error, he said, is the veteran journalist relying on his or her (OK, maybe I should avoid gender-inclusive pronouns here and just say “my”) experience.
About the time Craig had me won over to his checklist point of view, he said something surprising: He has frequently encouraged journalists in workshops and in his blogs to develop their own checklists, but no one has ever told him that they’ve done it. Well, now they have. Here’s Craig’s checklist, which is excellent. Mine follows (starting with his and adding/editing slightly):
Here’s my checklist (with points that I’ve added or changed from Craig’s in boldface, with elaboration below):
Ask sources to spell name & title; then verify what you wrote
Record or transcribe interviews
When someone cites numbers, ask for (and check) source
Ask “how do you know that?”
Verify claims with reliable sources
Save links and other research
Ask sources what other reports got wrong
Note facts that need further verification
Cut and paste (with attribution) quotes from digital documents.
Final Checks Before Submission
Numbers & Math (have someone check your math)
Names (check vs. notes & 1 other source)
Titles (people, books etc.)
Compare quotes to notes/recording/transcript
Check attribution (insert link if from the web)
Verify URLs (check them and check whether cited content is still there)
Phone numbers (call them)
Spelling & Grammar
Have you assumed anything? (If so, verify, hedge or remove.)
If you have any doubts, recheck with the original source.
Where your understanding is weak, read the final copy to someone who does understand.
Correct any errors you found in your archives, databases or other resources you control (but be certain you have verified the new information).
Get names right. I wish I could tell you how many times my name has been misspelled by professional journalists (some of them asking me for a job). I guarandamntee you that every time you spell someone’s name wrong, that person will remember for a long time. And will wonder what else you got wrong. So get the name right to begin with. Here’s what I had to say about name verification in my blog post that accompanied Craig’s accuracy workshop:
Screw up a name and readers who know how that person spells the name will not trust anything else you write. And the source will certainly question your ability or commitment to getting anything else right.
Ask every character to spell her name. Any time you have access to a character whose name you will be using, ask the character to spell his name for you, however common the spelling (if it seems like a stupid question, make a self-deprecating joke or a reassuring comment about your dedication to accuracy). Print the name clearly in your notebook as spelled, then read it back to the character as you’ve written it. If you assume you know how to spell the name and just read your presumed spelling back to the character, you risk error by at least two means: the character is not fully engaged and isn’t paying attention when you spell the name wrong or the character does not hear you well, whether because of an impairment, accent or background noise, and confirms an incorrect spelling. Asking “usual spelling?” presumes that you and the character have the same understanding of what’s “usual” and that you heard the character’s pronunciation correctly. Maybe the character answers you “usual spelling.” Still seek confirmation: “John with an H?” or “Steven with a V?” Better yet, spell out the name and use the question: “John with an H, J-o-h-n?” In addition to ensuring accuracy, this redundant exercise underscores to the character that he’s talking for the record and that you’re taking notes of what he says and planning to print his name. If you are recording, still write the name down in a notebook, computer or cell phone. You should not assume that the recording will be audible.
Get it in writing. Ask the character to write her name in the notebook for you. Then read it back to her to make sure you can read her handwriting. Ask for a business card.
Check it out even when you get it in writing. Don’t presume that a business card or a document or a name plate on the source’s desk is spelled correctly. I had a colleague at API whose name plate on his door was spelled wrong. He didn’t notice the error until I pointed it out to him. I let people copy and reproduce my handouts for newsroom training sessions. Sometimes the person who’s reproducing does some introductory material at the top and my name gets misspelled Buttrey. You could have that document and presume you have my name in writing, but all that means is that you have it spelled wrong in writing. A reporter at The Oregonian received a business card from a source who deals regularly with Asian clients. The reporter didn’t realize and the source didn’t say that the person spelled his name phonetically on the card, to help clients pronounce it correctly.
Check resources. Check the Internet (Google, LinkedIn, Facebook, a source’s organization’s website) and other resources (your clips, if you work at a newspaper) about a person, but don’t presume they are always correct. I had a source tell me his name was spelled “Lawler” but our paper always spelled it “Lawlor” (and a check of our clips showed that we frequently did misspell it).
Nail it down. If you’re covering an event rather than an interview, get a program or agenda beforehand. Try to locate the main people and ask them if their names are spelled correctly in the program. If someone you don’t know speaks or does something during the event, try to get to him as quickly as possible and get his name. If that is not possible, ask someone who would know. Then try to run the person down by phone to verify. Your notebook should have each name spelled right, verified by the character. If possible you should have at least one printed or digital source with the name also on it.
Check source of numbers. People citing numbers in interviews are operating on memory, unless the report is right in front of them. And memory is unreliable (more on that shortly). Especially when numbers are concerned. I’ve done it myself. I can remember interviews and panel discussions and Q&A sessions after workshops where I threw out numbers, wondering as I said them, how accurate my memory was. I usually hedge them, but people seldom pin me down, even if they are writing about the numbers. When I bothered to check my numbers, I’m usually off, at least a little. Asking the source of numbers both helps you verify that number and often gives you access to a report with more numbers (sometimes more relevant than the ones the source was citing) and other helpful facts.
Ask “how do you know that?” This is the most important question a reporter can ask. When people tell you things, you usually don’t know whether they are sharing facts they know firsthand, things they think they remember, rumors they heard or opinions they hold or heard. “How do you know that?” helps you learn and judge the accuracy of the things people tell you. I think the most common source of reporter error is repeating misinformation from sources. Judith Miller’s infamous errors about weapons of mass destruction came from trusting sources who were lying or misinformed. She said, “If your sources are wrong, you are wrong.” I say, when you ask, “How do you know that?” you reach more firsthand sources and you sniff out the innocent and dishonest errors of your sources. Remember, your name goes at the top of the story, not your sources’. “How do you know that?” is also a great question for editors to ask reporters. I also like a second question my friend and sometime-training colleague Rosalie Stemer encourages reporters and editors to ask: “How else do you know that?”
Seek documentation. Here’s what I wrote on this in my blog post on accuracy tips:
Find official data, records and reports that can confirm, refute or expand upon what you have been told. Photographs or videos might help you verify some details. If you are writing about a court hearing you didn’t attend, get the official transcript.
And here’s what I wrote on my Training Tracks blog at the American Press Institute about how an old video helped me correct the mistaken memories of several sources:
Even the most reliable sources can be wrong in the least controversial of stories. In 1996, I was writing a series about a team that had won the Iowa girls high school basketball championship 25 years earlier. I interviewed every member of the championship team, the Farragut Admiralettes. I interviewed the star and coach of the other team (both of Farragut’s coaches had died). I interviewed journalists who covered the game and fans who watched.
Again and again, I heard the same story about how Farragut won: Mediapolis ran up an early lead as Barb Wischmeier, who was 6-foot-1, scored a lot of points. Then Farragut’s coach sent 5-foot-2 Tanya Bopp in to guard Wischmeier. Bopp drew a bunch of charging fouls, flustered the bigger girl and Farragut came from behind to win the championship. It was one of the most important and memorable events of these women’s lives. People remembered the game in vivid detail. Some remembered specifically that Bopp drew three fouls or four.
I had no reason to doubt anyone’s story. I got a videotape and watched it, looking for some details to add to my story, to help me help the reader see the championship game, the moment of celebration. I was confused after watching the tape, so I watched again, counting the fouls. Bopp drew one foul on Wischmeier. It did turn the game around but it happened only once. I couldn’t trust anyone’s memory about the game, as vividly as they recalled it. In the retelling and reliving of that game, the key moment grew to legendary proportions.
How much more can the fog of war or the selectiveness of partisanship distort memory or twist facts? Sometimes verification is tougher than watching a videotape, but verification is one of a reporter’s most important duties.
Note facts that need further verification. I think writing is an essential part of the reporting process. As you write, you will remember things that might not be in your notebook. Note in the text in some way by highlighting, adding a parenthetical note (but watch out that you don’t publish the note). For instance, I wrote this draft in Word and highlighted in yellow the Judith Miller quote above, “When your sources are wrong, you are wrong.” I was writing on my laptop on a bus, without Internet access except on my phone. The highlight reminded me to check the quote later (and change “when” to “if”). Same with “Sully” (couldn’t recall his first name when I was writing).
Cut and paste quotes. I pass this tip along with extreme caution. A common excuse (and a flimsy one that I have never bought) cited by plagiarists is that they were sloppy in cutting and pasting. So be sure to insert attribution (and a link) before you paste. But cutting and pasting of quotations from the Internet or source documents remains the best way to ensure accurate quotations. If you’re typing the quote, it’s easy to leave out a “not” (or type “now” instead) or make a typo that changes “trial” to “trail” or some sort of other error that spellcheck won’t catch and that even a sharp eye might miss.
Numbers and math. Don’t just check your own math. Ask someone else to check it for you. This way, you won’t be repeating a mistake you made initially. And collaboration in accuracy is a good practice anyway.
Check names. Don’t just check them against your notes. Check against another source (a reliable web source, a business card, etc.). I used to work with a guy whose last name was Sullinger and people called him “Sully,” so I wrote that as Sully’s last name above. In searching for his first name online, I typed “pilot, sully,” and as I started typing “sullinger”, Google suggested “Sullenberger” after I typed the first few letters. Checking those links, I quickly spotted my mistake and fixed it. I had the last name wrong (see “assumptions” below).
Check attribution. Have you attributed (and linked) appropriately and correctly the facts you don’t know firsthand? This both ensures accuracy and prevents accidental plagiarism.
Verify URLs and phone numbers. Go to the URL (cut and paste it from your story) and make sure it’s right and that the cited material is still there. Call the phone number.
Assumptions. Scan your story for assumptions. When you find them, challenge them and verify, remove or hedge appropriately.
Check your doubts. If anything is in conflict or in doubt, re-report and nail it down. Go to the original (or best) source, if possible. If the spelling of the name in your notes doesn’t match the business card or the web resources you check, check with the person herself. Hardly anyone minds a call to ensure the accuracy of your content.
Read the actual passage to an expert. If you are writing about something you don’t understand well (a legal, medical or technical process, for instance), read the final copy to someone who does understand. Steve Weinberg and George Kennedy favor running the whole story by sources in all cases. I don’t go that far, but I respect both men and pass along that advice for consideration for your checklist. And on occasion, I have run an entire story past a source, with my editors’ approval.
Fix errors in resources. If, in the process of your reporting and verification, you find errors in your own archives, databases or other resources, be sure to correct those errors, so they don’t lead to future errors. And be sure to note the conflict in an unpublished note with your story, so editors relying on the archives don’t repeat the mistake.
Don’t be daunted if this looks time-consuming. Some of these checks will take seconds. For most stories, running through a checklist takes only 10 to 15 minutes, if that. If you’re under deadline pressure, you can check the first few paragraphs and post that, saying the story will be updated. Then check the rest and update. Accuracy is worth a few minutes, because the damage an error causes lasts way longer than the delay a checklist causes.
Two newspapers published errors in stories about me in 1991. I remember them two decades later. I don’t want people remembering my errors that long.
So what’s your checklist? What will you add to Craig’s and my lists? Craig recommends printing out your checklist and laminating it, so you can use it again and again. If it’s good enough for Sully, it’s good enough for me.
(Thanks to Mallary Tenore for noting my accuracy and verification tips on Poynter’s website yesterday. She gave me the nudge to finish this checklist and blog post, which has been on my to-do list for a couple months. Or would have been, if I kept such a list.)