Journalists should treat information we gather on social media the same way we treat information gathered any other way, or an assurance from Mom that she loves you: Check it out.
My #twutorial series hasn’t been updated since late October, but I always planned to do a post on verifying information gathered in social media. Given the errors some journalists made in reporting on the Sandy Hook massacre and in the original reporting on Manti Te’o's fake girlfriend, this feels like a good time to stress accuracy and verification.
The most simple and important advice I can give is that Twitter is like any other information source — documents, anonymous tips, news releases, press conferences, interviews, databases — it can provide valuable information or deliberate lies or innocent errors. Your job is to verify the information that looks useful. As with all the other information you gather, you can verify lots of different ways, and no single technique works for everything.
Some of the tips I provide here will be specific to Twitter or to social media generally. Some will be general verification tips applied to Twitter. And I’m sure I won’t cover all the ways you could verify information from tweets. As with all reporting, resourcefulness is essential. Develop some verification techniques of your own (and please remember to share them in the comments here).
Evaluating Twitter sources
Use Twitter regularly in reporting. The more you use Twitter, the better you will understand it and the more you will use it responsibly and effectively. If you use it rarely or sparingly and suddenly have to use it on a big, breaking story, you will be more prone to errors.
Develop Twitter sources. As you use Twitter in reporting, you will develop regular sources, just as you have people that you regularly check in with by phone or in person. You will be better able to judge the reliability of a source you have been following regularly than one whose tweets you are seeing for the first time. If it’s a politician or business executive, you’ll be more likely to know whether he or she tweets for himself or herself, or whether a public-relations aide handles the tweets. You’ll know better whether the person is inclined to sarcasm. You’ll know whether the account is that person’s actual account or a spoof account mocking the person. If you regularly follow people who tweet about the community or topic you cover, you will know those sources better and use their tweets more accurately and easily in your reporting, just as you’ll have better personal rapport — and get more accurate and useful information — with sources you interact with regularly.
Examine the profile. If you’re assessing the accuracy of a tweet by someone you don’t know, go to that person’s profile (click the name or username) and see what else you can learn about the person. If the person has just been on Twitter briefly (you can see when someone created an account at When Did You Join Twitter?), that should be a red flag (not meaning for sure that the tweet is inaccurate or bogus, but meaning you need to learn more about the person). If the person has hundreds or thousands of tweets, you can scroll back through them and get a sense of personality, affiliations, biases and interests. If the bio names affiliations, you might be able to find an email address or phone number to reach the person directly (or to verify that he or she is real). The bio may include a link to a blog or website that tell you more about the person or give a way to contact the person.
Seek connections. Check the people the person follows and those who follow her. Check the people she is conversing with in tweets, those she retweets. Do you know someone in common who can vouch for the person or give you a way to reach her? Do the followers or the people she follows say something about her interests, ideology or profession?
Check time of tweet. If someone is tweeting immediately after a breaking news event, he or she is more likely to be an eyewitness. Someone tweeting hours later might be tweeting something he heard from the news media or on social media. My first blog post about Twitter and breaking news, more than four years ago, involved a tweet minutes after a plane crash, before the first news report. The time was helpful in helping establish validity.
Check for photos. If someone is tweeting about a breaking news event or an entertainment event, check to see if she has tweeted photos. Lack of photos doesn’t necessarily mean she wasn’t there, but photos can help validate. The plane crash survivor helped by tweeting a photo (not a great photo, but click on it and you’ll see it larger, with a horizontal row of lights, clearly the windows on an airplane and an elevated light for the tail):
Can’t see much, but that’s the crash site.http://twitpic.com/ut2c
— Mike Wilson (@2drinksbehind) December 21, 2008
Check location. This isn’t as helpful as it used to be. I have location enabled in my tweets, but my location doesn’t show up in the tweets (if anyone knows why, please fill me in). And most Twitter users don’t have location enabled. But if a tweet shows location, the location is hyperlinked and you can click and a Google map will open, showing where the tweet came from. If it’s from the location of the event, that lends credibility to the tweet. A location nowhere near the tweet would raise questions. The person may still be tweeting accurate information, but may be getting it second-hand. (Lack of location doesn’t tell you anything.)
Tom Bridge was the only person with a location I noticed in recent tweets when I was writing this. Though the location isn’t visible in the embed below, it was visible in my timeline and when you open the tweet in its own URL (as in the screen grab below the tweet). The location hyperlink is circled in red.
Losing Musial and Weaver on the same day… rest in peace, men, you played the game so, so, so very well.
— Tom Bridge (@tbridge) January 20, 2013
Look for confirming tweets. Use Twitter’s advanced search feature to find other tweets about the same incident. A cluster of first-hand accounts can provide confirmation as well as details. If the first tweet didn’t show a location, other tweets might. Or other tweets might provide photos or even videos. In the 2008 plane crash example I used above, only one person was tweeting. Try searching with a location filter, but also search some keywords without the location filter because most Twitter users don’t enable location. Omit retweets from your search, to minimize your chances of confusing an echo chamber with a cluster.
Check previous tweets. Some of the most famous breaking news tweets were these from @ReallyVirtual the night of the raid on Osama bin Laden’s compound in Abbottabad, Pakistan:
Helicopter hovering above Abbottabad at 1AM (is a rare event). — Sohaib Athar (@ReallyVirtual) May 1, 2011
Go away helicopter – before I take out my giant swatter :-/
— Sohaib Athar (@ReallyVirtual) May 1, 2011
A huge window shaking bang here in Abbottabad Cantt. I hope its not the start of something nasty :-S
— Sohaib Athar (@ReallyVirtual) May 1, 2011
The tweets don’t show location. But tweets in the week or two before the raid discuss Pakistani culture, including one three days before the raid that mentions a power outage in Abbottabad and one earlier tweeting about the weather there.
Abbottabad had power for 6/24 hours or so today, still waiting for the power brokers to have pity on us :-\
— Sohaib Athar (@ReallyVirtual) April 28, 2011
Hailing in Abbottabad! Crazy weather returns.
— Sohaib Athar (@ReallyVirtual) April 12, 2011
Question to the Lahoris: Where in Lahore does one find imported shisha tobacco from brands OTHER THAN Al Fakher? Please share.
— Sohaib Athar (@ReallyVirtual) April 25, 2011
LOL @ “Its a hard time on Pak comedians. Firstly Liaqat Soljer, Mastana, Babbu Baral & now Moen Akhter. Worried about Altaf Bhai”
— Sohaib Athar (@ReallyVirtual) April 25, 2011
@ReallyVirtual’s profile also includes a link to his website (I think I recall that it did at the time, too. Though the blog was inactive at the time, the site included considerable information about him, including blog archives that went back to 2007 and personal information that was consistent with the content of his tweets. From the profile and earlier tweets, you can get pretty strong confirmation that this person was a software consultant living in Abbottabad.
Connect outside Twitter
My Digital First colleague Mandy Jenkins calls Twitter a tip service. If someone called in with a good tip, whether you knew the source or not, that’s just a starting point. It’s the same when you get a tip through Twitter. You need to connect directly to ask some questions so you can assess the information (especially if you can’t confirm information using some of the techniques discussed above).
Send a direct message. If the person follows you, you can send a direct message, asking him to call you. If he doesn’t follow you (and you don’t follow him), you should follow him. He may get a notification that you’re following, and if he follows you back, you can DM.
Send a tweet asking to talk. If you tweet starting with @ and the person’s username, asking her to call and including your phone number, she will probably see your tweet in her mentions and might give you a call. Don’t worry about competition seeing the tweet. If you start the tweet with the person’s username, only people who follow you both will see the tweet in their timelines. That’s a common occurrence if you’re tweeting at another journalist or a friend with whom you share mutual friends. But for a random stranger tweeting about a news event, the benefit of possibly talking to the source is worth the small risk that competitors will see your tweet. (A competitor could also see your tweet by going to your own Twitter profile, but that’s also a small risk.) If you want a further layer of protection in a competitive situation, don’t hit “reply,” but just start your tweet with the person’s username. If you click “reply,” a competitor looking at your tweet could click “view conversation” and see the newsy tweet that caught your attention. However, keep in mind that clicking “reply” helps the person you want to talk to know what you want to talk about.
Seek an email address or phone number. Some people (but not many) include an email address in their Twitter bio. Be sure to check for one. More likely, the person might mention a workplace. You could seek an email address or phone number on the workplace’s website and try to reach the person directly. Or you could leave a message at a general number for the workplace. If the person’s profile includes a link to a blog, website or social profile such as Facebook or LinkedIn, see if you can find an email address (mine is on my about page), leave a comment or send a message asking the person to call you.
Seek connections. Check followers (and people followed) of the person, as well as recent tweets, for people you may know in common, who can make an introduction (or tell you how to reach the person). The person’s tweets might also give an indication where the person works or another way to reach him or her.
Ask good questions. If you can reach the person by email, phone, social media or in person, you need to have a good interview. Ask detailed questions. Ask what the person saw and did. Ask how they know what they have said. Be sure to ask what they saw in person and what they heard from someone else. If something doesn’t add up, ask tough questions and try to nail it down.
Ask for photos. If the person has posted photos to social media, ask for permission to publish them. Ask for other photos as well. Ask for photos even if he hasn’t posted photos. Ask for photos from where the person is at the moment (if he or she is still in a relevant location). For instance, in the plane crash mentioned above, the survivor later tweeted about being in the Continental lounge while the airport tried to sort things out for the survivors. If you can get him to call you, ask him to send you a photo of the passengers in the lounge. It would have some news value (and be a better photo than the one he shot of the plane) and would provide further confirmation that he was actually in the plane crash (and unwillingness to send such a photo might raise a red flag).
Continental keeping us locked up at the presidents club until they can sort everything out.Won’t even serve us drinks.
— Mike Wilson (@2drinksbehind) December 21, 2008
Seek other sources. When you reach someone directly, ask them who else saw or experienced what you’re covering. The more sources you can reach, the more angles to the story you can get and the more opportunities you have to contradict an intentional lie or an inadvertent error from a source.
Evaluate the situation
Verification can be time-consuming. You want to be sure of your facts and you want to report efficiently. You can’t necessarily take every step I’ve outlined here every time you use a tweet in a story. Consider the import and impact of the content and the consequences if it’s wrong. If you’re rounding up fan reaction during the Super Bowl, you might be able to assess credibility just by looking at a person’s tweets and profile. Besides, you know the actual event took place and you know from the context that the tweet comes from a fan. Does it really matter if the tweet doesn’t use the person’s real name?
But if someone is the first source about a crime or disaster, you may need to take several steps outlined here to be sure that you get your facts right.
You can seek verification of facts from people on Twitter. This is another place where using Twitter regularly and developing a strong following will help. If lots of people in your community or your topic area follow you, a request for help might get useful responses (but keep in mind that you need to vet and verify those responses, too). If a topic or event has a hashtag, using the hashtag can extend your reach beyond your own followers.
Andy Carvin of NPR is the master at crowdsourcing verification. He uses a technique that journalists should use sparingly and carefully: Retweeting information he hasn’t yet verified and asking questions to help him verify (or refute). For instance, two years ago during the Libyan uprising, he asked his followers for help analyzing a munitions shell with a six-pointed emblem on it. An Arab media Facebook page speculated that it came from Israel:
— Andy Carvin (@acarvin) March 12, 2011
Andy’s Twitter followers quickly began doing research on munitions and their markings and debunked the identification of it as an Israeli shell.
For journalists without experience at this kind of crowdsourcing, I recommend consulting with an editor before repeating an unconfirmed fact in a crowdsourcing request this way. The potential for people retweeting the rumor without your questions or your caution that it’s unconfirmed is significant, so you should proceed carefully with this approach. (If you’re a solo entrepreneurial journalist, I’d still encourage seeking out a friend to act as a sounding board if you’re trying a crowdsourcing move that repeats an unconfirmed report.)
You can crowdsource without repeating an unconfirmed tweet, though. Say you saw a tweet from someone saying they heard an explosion at a shopping mall. From the tweet, you don’t know whether that’s a crime, a disaster, a car backfiring, legal (or illegal fireworks) or an explosion related to routine construction or demolition. But if you retweet, you could start a panic among people with loved ones working or shopping at the mall. You can crowdsource with a general inquiry, though: “Anyone know what’s going on at Southside Mall? Call me if you’ve heard/seen anything unusual: 123-456-7890.”
The girlfriend hoax
Because the accounts tied to Te’o's fake girlfriend, Lennay Kekua, have been deleted, it’s difficult to say exactly how the Twitter accounts might have helped reporters track down information about her if anyone had tried harder when the story about her death broke last fall. (I will invite the Deadspin writers to explain what they did in a guest post or in comments. Or if they address it elsewhere, I will link to it. Timothy Burke, one of the co-authors, and Deadspin Editor-in-Chief Tommy Craggs discussed the reporting and editing of the story with Poynter’s Mallary Tenore, but didn’t address the social media issues.)
I’ve already blogged twice about why reporters telling the story originally should have sought an obituary or death notice when they first were told Lennay Kekua had died. And, of course, if you had contacted Stanford, seeking information on when she graduated and what she majored in, looking for some professors or classmates to interview, you’d have found she didn’t actually go there.
Any thorough reporting on Kekua would have led to her social media accounts, because the Deadspin story about the hoax says that the only digital indications of her (before the web became littered with stories based on the hoax) were Twitter and Instagram accounts.
A Google cache provided the profile page pictured below for one of three usernames she used at different times.
Since her Twitter account is private, you can’t check her followers or the people the account was following to try them for sources and you can’t read her tweets. The “Sleep beautifully our sweet angel” line does seem to indicate that she has died. I would try to follow her, but it’s doubtful that the hoaxter would give a journalist access.
Using Topsy, you can read all 92 tweets posted by @lovalovaloveyou, an earlier username for Kekua. Or you can look up tweets mentioning @LennayKay, another username, before Sept. 12, the date of her purported death. Someone started using that username in the past week, so you just see those tweets searching @LennayKay’s tweets. Blogger Tyler Burns describes how he used All My Tweets to find all of Te’o's tweets to the three Twitter accounts his fake girlfriend used.
You might try tweeting to the people she tweeted at or those who tweeted at her. I see two #ff (follow Friday) tweets @lovalovaloveyou, and I might try contacting those people to see if any were her friends. Of course, some of those people were probably in on the hoax, so they might have given you bogus information that you might have regarded as confirmation. It’s hard to say for sure that digging through the social media clues would have broken the story. But maybe you’d connect with someone who was in on the hoax and would boast about it (after all, it doesn’t appear they did anything illegal, whether Te’o was a victim or a participant). Or maybe talking to people who hadn’t actually seen her would feed the suspicions based on checking for an obituary or checking with Stanford or looking for a record of her supposed car crash.
How do you verify?
I’m sure I haven’t covered all the techniques for verifying information from tweets (or other social media) here. Please share some of your tips in the comments.
Final note: This is twice in a week that I’ve referenced the “If you mother says she loves you …” cliché. I think it fit both places, but I will put that one into mothballs for a while.
Attribution and more resources
I don’t think I was the first person to think of many, if any, of the verification methods mentioned above. Most I read about somewhere or learned in a workshop or from a colleague, but I can’t recall where I learned them all. But many of them come from the list below. This is an updated list of resources to help journalists ensure accuracy, originally published here Sept. 17, 2011:
- Craig Silverman’s Regret the Error blog.
- Craig’s accuracy checklist.
- Craig’s Eight Simple Rules for Doing Accurate Journalism.
- Craig’s Best Practices for Social Media Verification
- Craig’s post about AP’s verification of user-generated content.
- Craig’s blog post on errors journalists made in Sandy Hook reporting.
- The liveblog, podcast and slides from Craig’s 2010 workshop for TBD and American University.
- My accuracy and verification tips.
- My accuracy checklist (inspired by Craig’s).
- Mathew Ingram blog post on accuracy in Sandy Hook reporting.
- Reuters Handbook of Journalism accuracy chapter.
- Jonathan Stray’s Measuring and improving accuracy in journalism.
- Mandy Jenkins’ Accuracy and accountability checklist for social media.
- Chip Scanlan’s Getting it Right: A Passion for Accuracy.
- Andy Carvin’s Storify curations (includes some good examples of using social media to vet and verify information).
- Poynter chat on handling inaccurate tweets.
- Eduardo Loara on the Error Prevention Project.
- Megan Garber’s Andrew Lih wants to wikify fact-checking.
- The Report an Error Alliance.
- Jay Rosen’s We Have No Idea Who’s Right: Criticizing “he said, she said” journalism at NPR.
- My Confessions of a teenage fact checker.
- My Sharing stories with sources before publication is risky, but can improve accuracy.
- Craig compiles “must-reads” to help with social media verification.
- B.S. Detection for Digital Journalists (video of Mandy and Craig in a workshop for the Online News Association).
- Mandy’s B.S. Detection for Journalists.
My slides for a 2011 workshop on accuracy:
Craig’s slides for that workshop:
Craig’s slides for a B.S. Detection for Digital Content workshop:
Earlier #twutorial posts
This post continues the #twutorial series I started last July. Here are the previous posts in the series: