This page is part of a graduate thesis. Learn more.
This thesis has attempted to justify and test the application of argumentation and informal logic theories to journalism. Based on a conception of “argument” as an attempt to demonstrate something that is, was, or will be true about the world, this thesis sees much of the work of journalists as arguments capable of evaluation based on the strength of their reasons and evidence in support of a conclusion.
Concepts from argumentation and informal logic were applied to a critical case sample of Pulitzer Prize-winning journalism. As Pulitzer Prizes are generally considered to reward the best American journalism, they should be the source of journalism containing the best of arguments from American journalists. According to the logic of the critical case model, success or blunder in Pulitzer Prize-winning argument provides a basis for inferring the state of argument quality in journalism generally.
Journalism’s theory of democracy assigns journalists an important role: providing information to citizens to use in thinking about issues facing local communities and the wider world. But for people who accept information based on the reasons and evidence accompanying it, this thesis raises serious questions about whether journalism can be relied upon for the information it strives to produce. That is, journalism’s democratic mission cannot be relied on as a guarantor that journalism will be very useful.
That the most prominent feature of the stories studied in this thesis was a lack of evidence was unexpected. Combined with the logic of a critical case sample, it is also alarming.
When a critical-thinking scholar such as Johnson (1988) gently discourages newspaper readers from accepting most reportage about polling at face value, because it omits so much methodological detail, the reaction is a head-nodding “amen,” but also a measure of sympathy. The journalist is probably on deadline, given little space for the story, and working from only a press release. When reading these stories, tools for assessing argument quality are not exactly ignored, but the purpose of using them probably changes. The story could not hope to provide enough evidence to confirm that the poll’s results were “true,” nor is it probably trying to. But it could, at least, alert readers to the existence of new research that might be worth some independent study.
In other words, the most harm that Johnson can cause to our relationship with such day-to-day journalism is probably just to encourage more skimming. If there are weak arguments in these daily stories, they are presumably offset by the value of the stories with more time and effort involved.
By contrast, the reporting studied here is not deadline-driven and is relatively free of space limitations. It is the product of tireless digging, prodding, and questioning that leads to a huge supply of source material to use.
The results were not just any investigative reporting on behalf of the public, but the recipients of the highest honor that American journalism can bestow. The reporters in these stories have had a chance at their best shot, and the profession has confirmed their success. So it is difficult to see why readers’ tools for argument assessment should not be on full-blast when approaching these stories.
The result of the present attempt at a full-blast analysis creates the alarm. While it’s up to readers to decide what kind of reasons and evidence they want to see before accepting a conclusion, they would have pretty good justification to not accept much of what these Pulitzer Prize-winning stories said. Could readers walk away from the Los Angeles Times knowing that the Bell City Council sought a city charter to boost its salaries? Could they walk away from the Sarasota Herald-Tribune knowing that one-third of privately insured homeowners relies on an at-risk insurer? Could they walk away from the Milwaukee Journal Sentinel with excitement about a new era in medicine? In each case and more, readers would have a good case for saying “no.”
The logic of the critical case study leads to the conclusion that argumentative blunder is not, then, a characteristic of only hurried, deadline-driven reporting; if it is found in top-quality journalism, it is likely to be found pretty much everywhere. The bind that this conclusion creates is that it nearly defeats the rationale for skimming the day-to-day material. Although, as this thesis found, there are bright spots to be had, in general there is little left to look forward to. And to repeat Hitchcock’s (2005) advice: “There is no point in taking bad information into account, still less in devoting time and effort to acquiring it.”
So where does all of this leave journalism? Is it worth devoting time and effort to acquire?
It is, admittedly, a little tempting to rebel — to delete one’s New York Times iPad app, curl up with Thoreau, and never come back. But that would not be justified, at least not by the findings of this thesis. There are too many questions worth pursuing first.
Is it really this way?
There is much work to be done to see whether the conclusions drawn in this thesis withstand scrutiny.
To begin, two additional routes of objecting to the validity of this thesis, each with ties to journalism sociology, arose while conducting the analysis.
The first objection aims at the theoretical backing sketched in chapter 2, which argued that journalists intend for their work, especially their investigative work, to provide information with which citizens in democracies can make decisions. In the case of the stories reviewed here, those decisions might be about what to think of public officials in Bell or about how the insurance in Florida should be regulated.
The objection would take the reaction to Johnson (1988) sketched above and advance it further to say that the journalism offered in the Times and Herald-Tribune, despite its depth, was not meant to contribute to deciding about Bell or insurance. Instead, it too was meant to act as a burglar alarm of sorts, albeit a much more powerful alarm than that provided by the average poll story. The journalism was meant to alert readers to the issues discussed in the stories, but under the assumption that any conclusions readers drew about the issues came from their own research.
The impact of this objection would be to defeat the rationale given in this thesis for applying argumentation and informal logic to the stories. Argumentation and informal logic are designed to help their users decide what to think and do. It would be unfair to the stories to critique them with these tools when the tools would use standards the stories never meant to meet; some other yardstick of “alarm” would have been more appropriate. Sociological research, then, would provide some insight into the ends of the reporters here or in similar situations.
The second objection would challenge the assumption that journalism’s “theory of democracy,” which would have journalists attempting to aid decision-making by the general public, is actually put into practice. This objection would draw on previous research suggesting that journalists shape their writing for an “audience” of political and economic elites, not the general public.
The effect of this objection would be that many the critical comments made in this thesis would need reexamination, in light of needing to consider what sorts of assumptions and prior knowledge about the “audience” the reporters might have made. If the reporters were not trying to explain something foreign to “readers” but instead summarizing, organization, or otherwise synthesizing knowledge already possessed by the target audience of savvy politicos, then the standards for what would have been a “good” argument almost certainly would have changed accordingly. The acceptable warrants, being field-dependent according to Toulmin, might have changed, for example.
Turning to argumentation and informal logic, this thesis chose to not draw extensively from research into fair and charitable reconstructions of arguments (for example, Hitchcock, 1996), but studies into reconstruction of journalistic argument are needed. This need is evidenced primarily by the difficulty faced here in determining what precisely Paige St. John argued in her reporting about the insurance industry. All attempts were made in this thesis to approach her stories fairly, as well as similarly jittery stories in the Los Angeles Times. But a dedicated investigation into the matter, and into journalistic argument generally, would help determine if the negative conclusions drawn about the stories in this thesis attacked legitimate targets. Indeed, part of the justification for argumentation and informal logic at all is the need for tools to assist in understanding and assessing messy, disorganized argumentation.
One potential route of testing the fairness of this analysis would be through the interpretations of ambiguous or suggestive language. As stated, many authorities in the stories reviewed were left unnamed or listed with questionable credentials. Conclusions were hinted at but never explicitly said. These ambiguities might have served simply as “very effective [forms] of propaganda,” as Marlin (1984) put it, or deserved dismissal as an improper “shifting of commitment based on expert opinion” (Walton & Macagno, 2011, p. 31). But following May’s (1988) concept of “invited inferences,” the potential meanings suggested by these ambiguous statements could be examined for other, reasonable interpretations than the ones reached in this thesis.
Kovach and Rosenstiel (2010) attempted to solve the same problem by turning to semiotics, and their attempt was useful in the context of their book. At the same time, they lacked theoretical rigor. Additional research on how journalists attempt to draw conclusions, and how denotation and connotation might mask those conclusions, might also be fruitful.
Research into the principle of charity — interpreting an argument in its best possible light before critiquing it (for example, Scriven, 1976) — could present fascinating insights into how much effort an analyst (or a reader) should put into trying to figure out what a reporter is concluding. For instance, such research could examine what is known about journalistic routines to allow for more accurate assumptions about what journalists would or would not want to say in their stories. It could also investigate the assumptions about readers’ knowledge or beliefs that reporters incorporate into their stories but leave unstated (assumptions that are changing with the development of online metrics — see Anderson, 2011a).
It is also possible that this thesis has not attended enough to how statements made by reporters in stories interrelate to support conclusions. The need for greater insight into reconstructing journalists’ arguments also implicates the need for greater understanding of how the arguments work through argumentation theory, such as through various approaches to the structure of argumentation (see Henkemans, 2000).
Indeed, the field of journalism seems ripe for the full weight of argumentation theory to be dropped on it, but without attempting to use argumentation and informal logic simultaneously, as this thesis has. Two reasons suggest a potentially fruitful future for research into argumentation and journalism.
First, although Toulmin’s models were useful in this thesis for providing definition to the arguments under study, the depth of Toulmanian studies and their potential application to journalism was hardly explored. A good starting place might be with Toulmin’s notion that the force of arguments is field-dependent, particularly in when certain warrants are more acceptable. Future research could examine what sorts of warrants are uniquely acceptable in journalism. Another direction might be to challenge whether Toulmin’s models really are all that useful to fairly complex journalistic argument. Is it possible to fit the arguments into the relatively simple model while remaining fair to journalists?
Second, many of the topics in argumentation included in the literature review helped clarify the development of ideas in informal logic, but were not themselves major parts of this thesis’s analysis. One option would be for arguments by journalists to receive the quantification given to arguments from others delivered through journalism, such as candidates in political debates (Brossmann and Canary, 1990).
There is also still room to challenge the relevance of argumentation to journalism in the first place. When Ripley (2008) sought to demonstrate that advertising could be considered an argument for the purposes of the field, she slotted her sample advertisement into theories of not just Toulmin but Aristotle, Johnson and Blair, van Eemeren and Grootendorst, and others, a feat that this thesis has not accomplished.
Yet another method for testing the validity of the conclusions reached here is to note that this thesis, despite frequently painting with the broad brush of “journalism,” has focused on only the text articles that were part of the Pulitzer Prize-winning series. But journalism, including that of the series’ under study here, encompasses photos, videos, illustrations, and mixtures of everything online. Even faults found consistently in text journalism should not lead to a rejection of the profession’s whole output. For that matter, faults found in one kind of text journalism, that of newspapers, should not impugn other text journalism, such as magazines.
Moreover, the techniques from argumentation and informal logic used to evaluate text journalism might not apply cleanly to journalism in other media. Television remains a dominant news source for Americans; evaluations of it (for example, Buss and Hofstetter, 1997) are perhaps a higher priority for researchers interested in journalism’s contribution to democracy than would be further testing on text journalism. Journalists are also continually experimenting online with new formats that present challenges for the evaluative tools in use here. It is unclear, for example, how to use Toulmin models with journalism produced as a music video (Garber, 2011), or whether to even try understanding such journalism as argument or as something else.
Finally, a comment about fallacies. Although fallacies did not play as large a role in this thesis as was anticipated, they are a core part of analyses using informal logic and critical thinking. For consistency, this thesis leaned on Damer’s set of fallacies, but, as said, many such sets exist, each with their own standards for when reasoning is fallacious. The exercise of this thesis should be repeated with these different definitions to see how contentious the labeling (or non-labeling) in this thesis of any “fallacies” might have been.
Additionally, definitions of fallaciousness change over time. One fallacy particularly vulnerable to shifts in meaning is what Damer called Appeals to Irrelevant Authority, which this thesis cited frequently against the journalism under study. But who and what qualifies as expertise experiences shifts in the long-term. What does someone have to do these days to become an expert? What qualifies someone as an expert in the eyes of readers of journalism?
These questions are important because they affect whether the reporters whose stories were studied here, particularly those from the Journal Sentinel, could or would be considered “experts.” But the questions are important also because some journalists are gaining reputations online as experts in topics through their use of new technology.
Perhaps the most prominent recent example is that of NPR’s Andy Carvin. By interacting with his followers on Twitter and “curating” Tweets from around the world about the Arab Spring, Carvin became a “one-man Twitter news bureau” with “essentially the readership of a small newspaper,” including people with political clout, such as America’s ambassador to the United Nations (Farhi, 2011). Carvin’s specialty can be called journalism, but unlike the research of the reporters at the Journal Sentinel, the source of his working knowledge is practically impossible to quantify, and certainly not the product of only his mind. So using informal logic to measure Carvin’s qualifications as an expert probably would require a different set of questions than the ones asked here. And Carvin himself is part of a push, starting in the 2000s, to use the internet to draw on knowledge and expertise from readers (see Gillmor, 2004, p. xxv, for one of the early statements), which has extended to Twitter (Ingram, 2011b).
Why is it this way? What can be done?
Assuming for the moment that the findings of this thesis hold some water, perhaps the preferable route is not to run from journalism but to recognize and try to tap its potential.
This search can begin by attempting to figure out why journalistic arguments turn out the way they do. To be sure, journalists are not typically trained in universities or in newsrooms to think in reasons, evidence, and conclusions, at least not in the way this thesis envisions them. Nor, importantly, are the copy editors, web producers, and desk editors whose decisions play an important role in creating news stories. Sociological research can ask: In what way, if at all, do journalists perceive themselves as giving reasons and evidence towards conclusions, and how does their answer affect their journalistic output? Do traditional newspaper reporters and online-savvy journalists differ in their perception? Would either group be willing to change their minds?
The views of journalists toward seeing their work as argument could be studied in the manner of Ettema and Glasser (1998). They examined how journalists structured their stories to draw moral conclusions while maintaining the self-perception that they do not draw moral conclusions. In the same way, the differences between journalists’ self-perception about their own descriptive conclusions and their actual stories might be worth another in-depth, ethnographic study. Newsrooms themselves, too, are in need of additional study to see whether their routines or the training of other staff besides reporters contributes to the kind of arguments eventually consumed by audiences.
Sociological research can also provide further insights into epistemological and “argument (0)”-type questions (Hample, 1985). Specifically, given that many of the stories found wanting in this thesis were that way because of half-baked offerings of evidence, Hample’s concern for the cognitive processes of storing and retrieving information used in arguments is relevant to the study of journalistic argument in at least two ways.
First, how do journalists keep and retrieve information, not just mentally but otherwise, and how do they choose to use it in articles? Why do they post it (or not post it) online? Second, how do journalists answer the “argument (0)” question of when an argument is “ready” for distribution? It is possible that the writers and editors who produced the stories under study here felt perfectly comfortable with the amount of evidence the stories included. The journalists might have felt that they presented sufficient evidence for a persuasive argument. Obviously, this thesis would challenge that position. The next step might be for both sides to examine more deeply what they consider “evidence” and, reaching back to Toulmin, whether the warrants in play are field-dependent, with the “fields” being something like “journalism” and “informal logic.”
If much of the evidence missing from the stories was sitting in reporters’ notes or desk drawers and, at least from the informal logic perspective, could have made their arguments more persuasive, then further research could investigate why the reporters left the evidence behind and whether they might be persuaded to include it. This concern is similar to that of Mindich (1998), who examined journalists’ “naive empiricism,” but differs in that it is as concerned with the evidence omitted from stories as with the evidence included. Did the evidence not included in stories still help persuade journalists to reach the conclusions they published, and if so, why did they feel that evidence need not be part of their stories?
These questions investigate why the problems exist, but can they be fixed?
One would like to think that there is reason for optimism given journalists’ usual preference for “facts.” They might be more persuaded if they were shown the research by Rupar (2006) suggesting that, often, only minor edits are required to patch up holes in stories. There is also some positive influence beginning to permeate into journalism from the computer science community, as evidenced by a recent budding “show your work” movement among online journalists (Thompson, 2011).
That said, it is fanciful to hope the Toulmin model might one day replace the inverted pyramid as the diagram of choice for journalists. After all, trend stories survive despite Jack Shafer’s best “bogus trend story” efforts. But there are small steps that can be promoted to encourage adoption of ideas from argumentation and informal logic into everyday journalism.
First, joking aside, a strength of the Toulmin model is its way of forcing the mind to pick one claim and trace its backing, warrant, data, and potential rebuttals. Such mental exercises might come in handy for journalists who, as was seen in this study, struggled to keep their storylines straight. Introducing the model into the field might bring some needed focus to their stories.
Second, the critical questions from Browne and Keeley (2004) might provide a good set of questions for editors to ask of their reporters and for reporters to ask of themselves. This is not to suggest that skill in asking and answering the questions comes naturally, but as with the Toulmin model, the questions provide a useful way to focus the mind. A reporter asking “Are there rival causes?” before releasing an article might have avoided some of the problems found in the Los Angeles Times story about the motivation behind Bell’s city charter. Similarly, reporters determining what the issue and conclusion of a story are before writing might add clarity to their work.
Third, every opportunity the internet provides for publishing supplementary materials should be explored. The default setting should be to post the documents reporters relied on for their stories to their websites; ideally, a service such as Document Cloud could be used to annotate stories to direct readers to exactly where the information for a particular claim came from. Additionally, uncertainty about whether reporters omitted information unfavorable to their conclusions from their stories could be relieved by posting transcripts or audio from their interviews online. The problems faced by the Milwaukee Journal Sentinel in establishing its reporters’ authority could have been reduced had they posted full lists of the experts they spoke with and publications they read.
Lives well lived
This study began by noting an interest in determining how to best spend the time we have (and even briefer time with the good fortune of having full use of our faculties). For those seeking to strengthen and question their thinking through the study of arguments, the question remains: Is a life that includes journalism well lived?
The question, of course, requires the work of more than journalists, argumentation theorists, or philosophers. But, despite what was written in the introduction, this thesis will have accomplished a small goal if it inspired some thinking on the matter.
However, the high rates of disappointment among scientists regarding the accuracy of day-to-day, “study X concludes Y” reporting about new research (see Pellechia, 1997, p. 50) could justifiably lead to skepticism of journalism’s ability to provide even this “alert system.” ↩
Rebellion would also entail an assumption that has not been justified here: That the purpose of reading the news is to learn something beyond the content of “what’s in the news” itself. ↩
See some possible support for this “burglar alarm” notion at the end of the analysis of St. John’s October 24 story, included in the appendix. ↩