I followed this up with a subsequent post on Saturday, Jan. 16.
The reaction to How News Happens may tell us more about the news industry than the study itself does.
The study of the news ecosystem in Baltimore was published today by the Pew Research Center’s Project for Excellence in Journalism, and news of the report was first published Sunday. The New York Times, Los Angeles Times, editorsweblog and more tweets than I could count trumpeted the finding that most news originates with newspapers and those upstart blogs contribute barely a trickle of original news. The favorite fact cited was that 95 percent of stories reporting fresh information came from the endangered old media, newspapers primarily.
This reinforces something that newspaper journalists and executives have been reassuring themselves of ever since we learned about those pesky bloggers: that bloggers and talk radio and really everyone would have nothing to talk and write about if not for newspapers.
Though the report was limited and flawed, it produced some valuable understanding of how journalism works today. But its most important findings were not that most news comes from newspapers. I’m glad PEJ is conducting this kind of research. I hope it does more.
But this research has too many flaws and limitations to be taken very seriously, and few of those flaws were noted in some of the coverage. Poynter’s Bill Mitchell and David Carr of the New York Times raised important issues and Jeff Jarvis bluntly pointed out dangers in reading much into the study (though he noted it has value).
I will just run through issues that the study (or those embracing the 95-percent number) should have taken into account (or that future studies should address):
- The study focused on types of news that old media emphasizes: crime, government, the justice system, health care. That skewed the findings inherently in the direction of old media. In fact, one of the six stories selected, on juvenile justice, began with an enterprise story in the Baltimore Sun. So of course the other reporting in that case was going to be derivative of the Sun’s original reporting. If old media are losing audience because they are not covering stories that are relevant to their communities, this study would have no value in telling you whether anyone was covering the relevant information.
- Areas the study steered clear of included neighborhood news, sports, arts, schools (the study was conducted in July, when school is out). One story (dealing with the sale of a historic movie theater) dealt with entertainment and business, though it didn’t fit tidily into either genre. These are areas where specialized blogs and web sites are contributing or could contribute, depending on the community, meaningfully to the news ecosystem.
- The study reported on only six story narratives from one week (a summer week, as noted, when the news flow, not to mention staffing of any or all news outlets, might have been far from typical). While those six stories were studied in great detail, can six stories during a summer week tell you very much?
- The study’s coverage of Twitter reflected an incomplete understanding of the value of Twitter in providing news to a community. The report took extensive note of the use of Twitter feeds by news outlets and by the Baltimore police department. But it paid no attention to whether and how stories break from Twitter reports by the general public. I suspect that Twitter produced significant discussion about at least three of the stories studied, if not all six. I would be surprised if that discussion didn’t break some news that old media either reported later or missed entirely.
- While the study encompassed all of the so-called mainstream news outlets, the new-media outlets studied were limited to those that “produce or disseminate local public affairs news.” While that selection of sources fit the narrow selection of issues studied, Mark Potts’ blog post on news sources in Baltimore (posted in June, the month before the study) includes several news outlets not studied by PEJ. If Baltimore Real Estate Investing Blog did a better job of reporting what might happen with the sale of that theater (the report noted that the advance reporting by the media mostly failed to raise the possibility that actually happened), the study wouldn’t know, because its selection of news outlets was too narrow. Update: I don’t recommend speculation, but indeed I was pretty close on this one, though I had the wrong blog. I just learned that astrogirl’s galaxy guide did in fact blog extensively about the sale of The Senator theater. I count six posts about the Senator the week that Pew studied the Baltimore media. Without studying such an active blogger, the study’s analysis of this story line simply is not valid. Please read her comment below (from Laura) and you can read her post about this post and the Pew study. Also, please note the comment from me, adding a response from Tom Rosenstiel, director of the Project for Excellence in Journalism. (I address Rosenstiel’s response and further concerns it raises, in the subsequent post.)
- The study would have been more valid in a city served by a strong online-only news operation. The study noted that with a reference to such a city, San Diego, though the reference to such operations as aggregators revealed the bias of the study. San Diego and many other cities have local online news operations with stronger reputations (nationally at least) than those mentioned in Baltimore. This study would have had more validity if it addressed how much original reporting comes from operations such as the Voice of San Diego, Oakland Local, MinnPost, West Seattle Blog (Seattle would have been an interesting city for this study) or St. Louis Beacon. I would love to see such a study of our local news ecosystem, including Eastern Iowa News.
- The finding that one of the stories broke in a blog was heavily minimized. A plan to put listening devices on Maryland Transit Administration buses was reported first in the Maryland Politics Watch blog. As reported in the study, a Baltimore Sun blogger three days later linked to the original post and called the state for comment, stirring up enough fuss that the idea was quickly scrapped. So the report tempers the grudging recognition that new media broke the story by saying that nothing happened until real media weighed in. But is that the full story? Since the study only checked official Twitter accounts, we don’t know whether the original post stirred up some tweeps or generated some calls from citizens to the MTA. But we do know that at least one of these six stories (17 percent) originally broke in new media. And what if that’s not typical? If the typical rate is one-third, I guarantee you would find some six-story samples with only one (or no) story breaking in new media. Or what if that’s not accurate for this sample? I can almost assure you the police-shooting story (and possibly another in the sample) broke on Twitter before it broke in any of the media outlets PEJ studied. How would this study be viewed differently if its conclusion said one-third of the stories studied broke in new media before they did in old media (in a story sample skewed in favor of old media)?
But if you argue that the study was completely accurate and valid and important, it had several disturbing findings that were at least as important as, if not more important than, the finding that most news comes from old media:
- 83 percent of stories in the study were essentially repetitive, meaning it’s going to be difficult for newspapers to claim enough value to make paywalls work. And don’t think that the paywalls will keep people from repeating newspaper stories. Broadcast outlets were doing that long before they could link to or rip off online stories.
- In the stories examined, the media were mostly (63 percent) reacting to the government. The report repeatedly mentions the lack of enterprise reporting from even traditional news outlets.
- The report found extensive lifting of material from press releases and other reports, some of it apparently amounting to plagiarism (though the report did not provide details): “We found official press releases often appear word for word in first accounts of events, though often not noted as such. In the growing echo chamber online, formal procedures for citing and crediting can get lost. We found numerous examples of websites carrying sections of other people’s work without attribution and often suggesting original reporting was added when none was. We found elements of this in several major stories we traced.”
I am glad to see someone studying the emerging ecosystem. I hope to see deeper studies that tell us more. And I hope the media who report about those studies make a better effort to understand and report what the studies say. And what they don’t.
Thank you for taking an in-depth look at what this study did – and didn’t reveal. This is precisely the kind of valuable analysis that the best blogs offer, using relevant links to arguments that support your thesis or present other arguments.
Your point regarding first reports of news events coming from Twitter are relevant – if not “reporting,” per se – and I’d love to see more from Pew on the new news ecosystem in local news that’s evolving, especially regarding weather and other natural disasters.
One area where it’s clear that there is plenty of original reporting that wasn’t in the study is tech news, although it’s by nature a distributed phenomenon that isn’t localized in way that a newspaper’s coverage ever was. Papers may have personal tech columnists, like Rob Pegoraro at the Washington Post, but the thousands of folks who descended upon Vegas last weekend are by and large not associated with traditional media. The original reporting out of the WSJ or NYT is certainly matched by online-only outlets.
The results of the study make sense, in terms of sourcing. Your concern about how much came from press releases or was repetitive, however, might also provide some evidence for why readers are looking to alternative sources online for more information or perspective.
LikeLike
Thanks for the mention, Steve. Baltimore is actually a real beehive of activity–there are some local sites and blog committing real journalism there, and my company, GrowthSpur.com, is already supporting some of them with ad sales training and our about-to-launch local ad sales network.
As Jeff Jarvis said, so much of this winds up being self-serving and protective of the traditional media. Of course they account for 95 percent of the original reporting (well, that may be high, but let’s use that). But you know what? Five years ago, it was 99.99 percent (or more). Five years from now–it might be 50 percent. Less if the Sun goes down. And I wouldn’t rule out that scenario for a second.
This is a very fluid situation, with many, many more new players than people realize, in Baltimore and elsewhere. It’s possible to miss the forest by concentrating on a few dead trees (sorry, couldn’t resist!)
Mark Potts, GrowthSpur.com
LikeLike
I think, in this kind of research snapshots are not really relevant.
Ecosystem is dinamic system so we need at list two points on the timeline of Baltimore newsflow. Comparison in time is really needed
Otherwise we try to show trends using one cell of data.
LikeLike
Your post got me thinking not such about the limitations of the research as about what would be good to get more of — regardless of how much we got now.
Just looking at my own checkered past as a sometimes successful, sometimes less-so beat reporter, I wonder what really made the difference in pushing me to strong enterprise in some circumstances and routine, half-assed coverage in others.
And in the current environment, I wonder what encourages useful collaboration in advancing the story, regardless of who got there first. Michelle McClellan had a good Tweet earlier this evening –” Conflict frame: Old media vs. new media narrative overplayed? Sure there are folks on extreme ends, but many more seem to be doing both.”
To the extent we get more and more of the old and new groups doing both — digging up new stories and advancing stories already out there — we got an ecosystem with some juice.
LikeLike
[…] This writer in Iowa, meanwhile, has a worthwhile critique of the study, which is being seized on by some as evidence that only newspapers can report real […]
LikeLike
I hope this post gets as much attention as the Pew article. Because it’s better
twitter.com/fab_science
LikeLike
[…] began to protest, among them Steve Buttry, an innovation coach at Gazette Communications. Buttry produced a very deep and critical analysis of the research, and Guardian columnist Jeff Jarvis also weighed […]
LikeLike
Bam! (that’s the sound of this blog post hitting the nail on the head)
Great analysis Steve. I had my doubts about this study that you and Jeff Jarvis did an admirable job of justifying.
Keep it up.
LikeLike
[…] began to protest, among them Steve Buttry, an innovation coach at Gazette Communications. Buttry produced a very deep and critical analysis of the research, and Guardian columnist Jeff Jarvis also weighed […]
LikeLike
[…] recently noted some flaws in a study that generated some encouraging results for “old media” (if you ignored all […]
LikeLike
I’ve always felt there was an ecology to news, with print reporters doing most (not all) of the investigative work and broadcasters ( I include bloggers in broadcasters) taking that work and redistributing it. While I agree with your observations about the limitations of the study I can’t help but note that the study findings conform to my own experience of news consumption. I read blogs and on those blogs I get commentary and opinion and links to new stories that have been found by “old” media (for the most part print. The only topics where I see news generated by bloggers is in technology and entertainment. If I want to know what’s happening to my taxes or kid’s school I have to go to the newspaper. In fact I wish my newspaper would do a lot more of this. However the papers seem to be trying to cut costs and are going for less and less local content. The blogosphere so far is not stepping in to fill the void.
LikeLike
[…] began to protest, among them Steve Buttry, an innovation coach at Gazette Communications. Buttry produced a very deep and critical analysis of the research, and Guardian columnist Jeff Jarvis also weighed […]
LikeLike
Steve, thank you for the update noting my blog.
I can tell you from experience that it is very frustrating to work for months to break a story in a direct, hands on investigative fashion, only to have one’s original reporting that is telling a whole other side to a story be almost totally ignored.
The Pew study is correct that on The Senator Theatre story, most of the mainstream press was parroting official government sources most of the time, and most of the blogs were simply commenting on those official government pronouncements in the form of “news” stories.
As a result, a big lie was perpetrated on the public, and a wonderful man, who ought to be seen as a local hero, was badly harmed, along with his family.
Unfortunately, I don’t think the Pew researchers even knew my blog was there.
Not just for the week of the auction, but for months before and after it, I’ve been following this story. People who carefully read through my blog will find that I have documented many facts that were never reported in the rest of the local media on this story.
The reason I was able to report things that nobody else did was simple — I sensed that something did not add up and went into the theatre to begin investigating what was actually going on. What I found out so outraged my sense of justice that I have to admit, my blog gets emotional at times and is definitely biased. This is because when I started investigating the situation, I realized that some really good people were getting thrown under the bus by our city government and the news media, and that essentially nobody was helping them. I then felt a need to help them myself if I could, including becoming an unpaid volunteer staff member at the theatre. Keep in mind I did not know anyone at the theatre before all this started to happen, although I had been to the theatre for movies and events many times.
Did I get too close to my story? From a traditional media standpoint of pretending to be unbiased while actually having a bias (in this case their bias was that of the city government), I suppose I did and still am.
Did I report original, truthful information that nobody else had bothered to find out? Absolutely.
My reporting has dropped off recently. There’s still a huge part of this story to be told, but for the time being, in the interest of preserving the theatre for future generations, it’s time to put aside the muckraking hat, because there’s actually a proposal on the table for the future of that theatre that makes a lot of sense. We’ve been saying the theatre needs to be owned by a non-profit. WTMD and Towson University might be able to provide that future.
So now it’s time to be gracious and cooperative and work hard to make that very logical case that is supported by huge amounts of theatre industry data and experience nationwide. The likely top contending proposals for the theatres future come down to one that makes a whole lot of sense based on models across the country, and one that absolutely will not work, also based on experiences of other theatres all over the country, as well as past experience at The Senator. The challenge now is for the Friends of The Senator organization, of which I am a member, to convincingly make the case that the WTMD/Towson proposal is far superior to the other RFP proposals received.
There is more to the story that needs to be told eventually. For now, the theatre’s future is very much at stake.
LikeLike
Thanks for that thoughtful reply, Laura. I should also note that Laura (astrogirl) has written more about this on her blog: http://astrogirlguides.blogspot.com/2010/01/analysis-on-pew-study-of-baltimore.html
LikeLike
Great post. I agree – although I take issue more with the interpretations of the study findings than the validity of the study itself.
You are absolutely right in saying that the study is limited – although almost all are, unless you have unlimited time and resources to produce one.
I’m not sure it’s 100% fair in your analysis above though to characterize crime, health care, the justice and government as “types of news that old media emphasizes.” At least not crime and health care, two issues that touch everyone’s lives and that my little neighborhood newspaper covers regularly. Might have been interesting to add sports or something else “lighter” in the mix I’d agree, but what exactly would count as “neighborhood news?” The trick with a study like this is you have to carefully define your terms so that you can be sure you are getting accurate counts. And even if they are “old media” stories, I’d say they are still important enough to community life to at least enable them to make a good argument for their inclusion.
Second, Rosenstiel did say that the study would turn out differently if it was San Diego, and I’d agree we need a study there and it would be fascinating to compare two cities at different stages. But if they only did San Diego instead of Baltimore, then the more traditionalist types would say – wait! San Diego is much more advanced than some (most?) other cities in its online local news ecosystem, so how is that generalizable? Just saying I don’t think the study “less valid” because it wasn’t done in a city like San Diego. I think you can make an argument for doing it in a more “average” city, keeping in mind that in any case study its hard to generalize the results.
And I could be wrong, but I didn’t think they tried to mask the role of blogs and Twitter in breaking news – although the traditional news orgs sure did in writing about this study. I would love to be able to design a study that cast a wider net than this one did – e.g. looking at more than just official Twitter accounts – but I can imagine that the reason they did that was simply because they weren’t sure how to operationalize it – how are you sure that you’ve captured everybody in the city’s Tweets on a topic? I mean obviously you can use search to help you, but I can see how they probably limited it for methodological reasons as much as anything else.
It’s unfortunate that they appeared to have overlooked the blog mentioned in the above comment. However, it’s simply not in the nature of science itself to ever be able to “prove” anything with a single study, and I don’t really see evidence that the methodology is quite as flawed as you make it out here, although I don’t know every detail about how they did it. The important thing though is that each study provides us a way to quantitatively test our assumptions. I think there is just as much ammo in here for “new” media as for “old” media – and I’m firmly in the camp that sees the new media potential here.
Carrie Brown-Smith
University of Memphis, assistant professor
@brizzyc
LikeLike
Thanks, Carrie. You make an excellent point that a study of Baltimore is as valid as a study of San Diego would be. Every city is unique and the study did note differences between the Baltimore media and other cities.
We disagree, though, on whether crime and health care are issues that favor old media. I haven’t worked at a metro paper that didn’t have at least one full-time cops reporter and someone full-time on the health beat. I’ve worked at papers that had multiple reporters on both topics (though I suspect that has changed).
As for Twitter, tweets from the public absolutely are part of the news ecosystem. You couldn’t guarantee that you would get everything, but you could search a few keywords and hashtags and be sure that you are getting a piece of the Twitter picture.
Also, I would say that it’s more than “unfortunate” that they overlooked astrogirl’s blog. It’s inevitable (and I predicted it in the original version of this post). The study took a narrow view of blogs. To do that and conclude incorrectly the extent to which blogs derive their content from old media raises serious questions about the validity of the study. Here’s the section of the report dealing with the sale of the Senator:
http://www.journalism.org/analysis_report/auctioning_senator_theater
Read that, knowing that the study omitted a blog posting six times that week (that’s a fact that I counted) and whose author contends she was doing original reporting that old media missed, and see how much of this section you think is valid. And if one-sixth of the study has credibility issues, doesn’t the whole study?
LikeLike
Hi Steve,
Okay, I’ll grant that it does appear to have been a big oversight and a flaw in the study – although I’d like to see more detail on how they selected the 10 “new media/blogs” sources they did look at to better assess the criteria they used for inclusion. Would be nice to see that in the methods section, actually.
Not sure whole study therefore is entirely lacking in credibility, though agree with you may have understated the contribution of blogs to the local media ecosystem. To me, the reliance on official sources finding was really interesting. It’s an interesting paradox. On the one hand, we have all of these new media tools to better and more easily incorporate a broader range of viewpoints and sources in stories, and from my research, even the “old media” people do actually get that. Some of the teachings of various orgs like APME and CCJ etc. over the years emphasizing things like not covering crime in the old fashioned, police blotter type of way and instead thinking more along the lines of “public safety” have sunk in, too. But on the other hand, immediacy (and short staffing in traditional orgs) also means that you go to that old official well again and again, for all the reasons it’s always been popular in a deadline environment – it’s a predictable source of constant info that, at least theoretically, requires less time to verify than a news tip from somebody you don’t know at all on Twitter.
Cheers,
CB
LikeLike
We’re in full agreement on the finding about old media’s use of official sources. With staff cuts, the “round up the usual suspects” form of reporting is thriving more than most aspects of journalism.
LikeLike
Tom Rosenstiel, director of the Project for Excellence in Journalism, responded to my criticism of the study. His email:
The staff that did the research on this particular thread have responded to your note. It is attached. (The attached statement follows the email.)
As you will see, we did capture the Astrogirl posts and were fully aware of them. Based on her own self descriptions, she does not represent herself, nor does she function, as a news organization. That put her outside the parameters of the quantitative element of the study. She was an interesting voice on the subject of the theater, but she was not a news gathering outlet, and it was the ecosystem of news we were studying here. A different study, looking at the robust debate that occurs in response to news gathering, would include many such voices.
The response from the research staff:
Steve,
Thanks for the careful reading and response to, our study “How News Happens.” As we say in the introduction: it is “only one attempt at trying to understand who is producing news and the character of what is produced. Additional reports could tell more. But this snapshot was in many ways a typical week—marked by stories about police shootings, state budget cuts, swine flu, a big international soccer game in town and a mix of fires, accidents, traffic and weather.”
Our audit of Baltimore media did identify the Astrogirl blog you cite. As Ms. Serena acknowledges, it is an organ of advocacy that favors the point of view of her friend, the former owner of the theater. She volunteered at the theater, is active in the “Friends of the Senator Theater,” opposed the auction and participated in the protests. The advocacy element takes the blog out of the category of general interest, public affairs news outlets included in this study. A number of the identified blogs and online-only media outlets did fit these parameters. Even as she wished we had included advocacy sites in the study of news outlets, she said, “The fact that the Pew study was flawed by leaving out some new media sources does not detract from this conclusion being essentially accurate. What might change if a larger sample of new media was used is the percentage, but probably just a bit.”
To address your other point, the study wasn’t limited to the six story lines; we did first-level analysis of all content captured for three days during the week. It formed the basis for our analysis of who produced the most content, which were the leading news topics, an analysis of package news reports on television and radio and a breakdown of local, national and international coverage in newspapers. For deeper study, we followed six storylines related to public affairs that resonated in the media that week and that attracted coverage from multiple sectors. Note that in our analysis of each story line we sought out and included stories produced by blogs and websites that were not part of the original 60-outlet sample.
Again thank you for your attention to this report. We look forward to conducting further research on the subject
PEJ
LikeLike
[…] Steve Buttry of Gazette Communications had a busy week of blogging. Here’s a look at Pew Study of who is originating news content and Steve’s notes with th… […]
LikeLike
[…] blogged Monday about weaknesses in the How News Happens study by the Pew Research Center’s Project for […]
LikeLike
[…] Old media find comfort in study of Baltimore media (they didn’t look very close) (stevebuttry.wordpress.com) […]
LikeLike
[…] “Old media find comfort in study of Baltimore media (they didn’t look very close), by St… […]
LikeLike
[…] Baltimore Brew—are providing a new layer of local coverage in that city (in spite of what the Pew Report would have you believe). That pattern is being repeated elsewhere, with some […]
LikeLike
[…] Baltimore Brew—are providing a new layer of local coverage in that city (in spite of what the Pew Report would have you believe). That pattern is being repeated elsewhere, with some […]
LikeLike
[…] of all the local news media in Baltimore. It has limitations, outlined in a couple detailed blog posts a commenter referenced on NAC, but it’s still a useful piece of work. Put simply, for all the […]
LikeLike
[…] I have documented significant and deliberate holes in one of the most prominent (and often-cited) such reports, How […]
LikeLike
[…] a study of Baltimore media released earlier this year by PEJ, this study tried to measure new media by old media’s yardstick. Guess how that one’s going to come out every […]
LikeLike
[…] So even if there were some methodological difficulties in that study, which a few people I know have pointed to, 84 percent is still remarkably […]
LikeLike
[…] Old media find comfort in Baltimore study (they didn’t look very close) […]
LikeLike
[…] on research I regarded as flawed: Academics measure new media (again) by old-media yardstick and Old media find comfort in study of Baltimore media (they didn’t look very close). I wrote a second time about the Baltimore study: Pew doesn’t understand news ecosystem well […]
LikeLike
[…] might be nearly half as much as from Facebook, which would make it an important referral source. I noted more than a year ago that ignorance of Twitter or bias against it was one of the reasons an earlier (and often-cited) […]
LikeLike
[…] of all the local news media in Baltimore. It has limitations, outlined in a couple detailed blog posts a commenter referenced on NAC, but it’s still a useful piece of work. Put simply, for all the […]
LikeLike
[…] began to protest, among them Steve Buttry, an innovation coach at Gazette Communications. Buttry produced a very deep and critical analysis of the research, and Guardian columnist Jeff Jarvis also weighed in. So is the study – […]
LikeLike
[…] is when you use the curated content as the starting point for a commentary, as I did in my reaction to the faulty (but widely quoted) 2010 study of Baltimore’s news ecosystem by the Project…. Or you can aggregate various related pieces of content and provide your own analysis and […]
LikeLike
[…] noted earlier, I was initially disappointed in the 2010 study of the Baltimore news “ecosystem,” and on closer examination, I said the “old-media biases by the researchers were so […]
LikeLike