Matt Hill

Poppers Classroom

Computers & Writing 2013

In a recent Mother Jones article, Kevin Drum writes about the speed with

which computers are advancing and that they will ultimately lead to my statement above. Using Moores Law, which claims that computing power doubles every eighteen months, Drum writes that computers should be able to imitate the human brain by the year 2040. If this model is accurate, and some physicists claim that Moores Law will eventually burn itself out (1), all machine grading of writing could be feasible by 2040. Drum isnt the only mainstream press writer who is certain that computers will eventually be able to take over human cognition tasks. [NEXT FRAME] John Markoffs recent New York Times story on machine-scored essays asks us to Imagine taking a college exam, and, instead of handing in a blue book and getting a grade from a professor a few weeks later, clicking the send button when you are done and receiving a grade back instantly, your essay scored by a software program (1). Further, the machine would allow you to re-write the essay for a better grade. The cost of such a miracle might be professors having little to no say in how the essays are scored, administered, or taught. While such a practice might be desirable in large courses (lets say introduction to Sociology), [NEXT FRAME] composition professors depend on their autonomy to theorize, design, implement, and assess their own course material. In essence then, the computer grader removes a large component of the work that we do. [NEXT FRAME] Today, I think with you about the machine scoring of writing. To do so, I examine Karl Poppers four claims about knowledge and apply his recommended epistemology to machine scoring. My goal is to afford Popper a deep understanding and to apply his notions of epistemology to a serious concept within composition.

article still in draft: please contact mhill30@du.edu to cite

Matt Hill

Poppers Classroom

Computers & Writing 2013

Machine scoring is not a new concern, but it has become a more pressing one recently as industry and various governments push for computer scoring in testing settings (where it is already in use) and in teaching settings as well (see Ericsson & Haswell, 2006; Markoff, 2013; and Strauss, 2013). Poppers intellectual, political, and philosophical lifespan is odd to trace. I will not trace it today, but such oddity illuminates what Popper seemed to value as the most important of intellectual habits: curiosity and complexity. Such habits show themselves in the article which forms the bulk of my curiosity with Popper today, Optimist, Pessimist and Pragmatist Views of Scientific Knowledge. Before even reading this piece, one can see how Popper views epistemological concerns: they are not an opposition between two sides. Poppers title allows for the notion that these are more complex issues that allow for a range of possibilities. He is not immune to splitting our epistemologies in two (open vs. closed society or tory vs. whig history, for example), but I will assert that these binary distinctions become enmeshed in the three values of optimist, pessimist, and pragmatist. I ask, is it possible to stand in each of those three major camps simultaneously or to hop back and forth between the camps, as is evidenced by the and in Poppers title? For example, can one be optimistic and pragmatic about the role of machine scoring as it relates to writing instruction? Are Poppers categories mutually exclusive, or do they allow some flexible thought? The answer is yes they are mutually exclusive, but Popper provides a fourth way of viewing knowledge. Before specifically addressing the machine scoring, please indulge me for a few minutes as I summarize my understanding of Poppers epistemological views. article still in draft: please contact mhill30@du.edu to cite 2

Matt Hill

Poppers Classroom

Computers & Writing 2013

His claims about knowledge are fluid and would shift throughout his career, so I do not claim to present his final views on knowledge. Rather, I rely on his statements from roughly the middle of his career in the aforementioned essay. First, Popper anchors the piece by claiming that epistemology is THE glue of all philosophical and intellectual pursuits, not just those of science, but of ethics, politics, or aesthetics, for example. [NEXT FRAME] Popper defines optimist knowledge claims in the way that seems to make sense: [NEXT FRAME] we are capable of grasping knowledge, how we attain it, and how we share it. In other words, humans can justify knowledge claims (3). [NEXT FRAME] Pessimists hold the opposite view. [NEXT FRAME] We cannot justify our knowledge claims, and any attempts to do so indicate an ethical breach. Popper does not believe the pessimist view to be tenable, but he is not entirely unsympathetic to their claims. Neither is he entirely sympathetic nor unsympathetic to the optimists. Popper reminds readers of a third epistemological [NEXT FRAME] possibility: the 20th century intervention of [NEXT FRAME] pragmatist epistemology, which essentially views theories of knowledge as tools. In such an instrumental view, concerns over justification of knowledge do not matter. A tool either works or it does not. All that we can do is improve upon the tool (or, I suppose, weaken it). Popper does not support this intervention as it essentially disavows the notion of scientific knowledge as inherently intellectual. Viewing knowledge as simply instrumental ignores any ethical, moral, or aesthetic concerns over the search for and implementation of knowledge. [NEXT FRAME] The instrumentality of contemporary science can arise in at least two flavors. First, in her introduction to the 2011 edition of The Best American Science article still in draft: please contact mhill30@du.edu to cite 3

Matt Hill

Poppers Classroom

Computers & Writing 2013

and Nature Writing, Mary Roach provides her rationale for the salve that is good science writing. Such writing cures us of ignorance and provides us with access to the world (xvi), but it is also a gauge with which we measure our political and social selves. We often see this type of practicality as desirable in that a need is being met in the most efficient way possible. The second flavor of practicality is related to the first and is evidenced by the Canadian governments recent decision to fund only research that has a clearly practical function in mind before the research has been conducted. The president of the Canadian National Research Council (NRC) is on record saying, [s]cientific discovery is not valuable unless it has commercial value (Plait). Both these practical views of science fit closely with what Popper means by the pragmatist view of science and what scientific knowledge may do for us. For the pragmatists, Popper claims that scientific rigor allows us to have access to how the world works in a nearly indisputable manner. In essence, then, we are able to justify (3) what it is we know and how we know it and implement knowledge in a practical tool. Under such a model, there are two states of being: knowing and not knowing. Science moves us away from the latter and into the former. [NEXT FRAME] As an alternative to the optimist, pessimist, and pragmatist views, [NEXT FRAME] Popper offers a fourth epistemological possibility, one that he does not name and that he prefers as it values knowledge itself as temporary and unstable but with the goal of trying to move, molecule by molecule if necessary, closer to truth and not to merely technological superiority. His notion of truth depends on his understanding of rational; that is the willingness to be scrutinized. If we follow article still in draft: please contact mhill30@du.edu to cite 4

Matt Hill

Poppers Classroom

Computers & Writing 2013

Popper through the question of machine scoring, we would find that as a tool, it might eventually be effective, but it would not equate with any sense of knowledge in our field. The instrumental application of knowledge, especially when done acontextually as machine scoring likely will be, does not equate with a productive, progressive view of knowledge in general. For the desire to deeply interrogate how we know what we know helps us understand how we compose what it is we know. With this in mind, it is possible that we should at least consider adopting machine scoring, though I think Patty Ericsson and Rich Haswell provide a more than compelling argument against machine scoring in their edited collection, Machine Scoring of Student Essays. [NEXT FRAME] What Popper posits is an epistemology without guarantees, and that seems an appropriate notion to apply to machine scoring. He asserts that true knowledge as both practical and theoretical exists but that there is no certainty that we will find it nor will we know what to do with it if and when we do find it. As such, our best choice when making decisions is not to concern ourselves with utopic conceptions of our options. Rather, we should consider what options we have in front of us and make a decision towards the future by being open to learning from mistakes and, in essence, by being willing to be stupid (231 248). Poppers ultimate political aims for knowledge (and these aims are indeed utopian in their own right) is to structure a society, a way of living together, based upon the idea of not merely tolerating a man and his convictions but of respecting him and his convictions (236) and to reach such a society (what Popper calls an open society) through non-violence (237). We thus should alter our choicesor throw them out, but Popper carefully article still in draft: please contact mhill30@du.edu to cite 5

Matt Hill

Poppers Classroom

Computers & Writing 2013

avoids advocating full-scale, revolutionary change along the lines of mid-20th century fascist or communist methods (234)by being willing to re-examine such choices, even if we seem certain of their veracity, efficacy, or appropriateness. Through such a lens, it seems appropriate [NEXT FRAME] to consider that machine scoring may eventually reach the abilities of human cognition that allows us to discern context, strengths of argument, audience awareness, or appropriate design. If the prevailing view among scientists (that human brain power is reducible to the number of operations per second) is accurate, then it seems reasonable to presume that computers will eventually be able to pay attention to more important rhetorical strategies, or genre conventions, or design principles, for examples. What we need in thinking about machine scoring is more along the lines of the scientists questioning Google Glass. Researchers at the Virginia Tech Transportation Institute are currently investigating the difference between looking and seeing with regards to new technologies and their impact on our driving (Simons & Chabris). The notion is that while technologies such as Google Glass or Apples Siri voice assistant seem to free our hands and eyes, our overall senses are still affected when we use these technologies because our senses might be more integrated than we previously thought. Further, research such as the gorilla in a basketball game demonstrates that just because our field of view is fixed in a particular direction or even at a particular object, we do not necessarily see what is there. Such critical research as championed here by psychology researchers Daniel Simons and Christopher Chabris is perhaps what we should be striving for in relation to machine scoring. These scholars point to the emergence and intersection article still in draft: please contact mhill30@du.edu to cite 6

Matt Hill

Poppers Classroom

Computers & Writing 2013

of new technologies and essentially ask, How is this new technology helping us solve old and new problems? while not dismissing the future appropriateness of the technological intervention. As it appears right now, [NEXT FRAME] our fields general stance against implementing machine scoring makes sense to me as the technology is driven mostly by efficient and economic decisions, and not by any understanding of sound pedagogy, theory, and scholarship. [NEXT FRAME]

A Short Postscript According to paperraater.com, [NEXT FRAME] my talk today is worthy of an A. I, for one, embrace our future robot overlords.

article still in draft: please contact mhill30@du.edu to cite

Matt Hill

Poppers Classroom Works Cited

Computers & Writing 2013

Drum, Kevin. Welcome, Robot Overlords: Please Dont Fire Us? Mother Jones May/June 2013. Web. 23 May 2013. <http://www.motherjones.com/media/ 2013/05/robots -artificial- intelligence-jobs-automation?page=1> Ericsson, Patricia Freitag and Richard Haswell. Ed. Machine Scoring of Student Essays: Truth and Consequences. Logan, UT: Utah State UP, 2006. Print.

Markoff, John. Essay-Grading Software Offers Professors a Break. The New York Times 4 April 2013. Web. 25 April 2013. <http://www.nytimes.com/2013/04/05/ science/new-test-for-computers-grading-essays-at-college-level.html?_r=0> Plait, Phil. Canada Sells Out Science. Bad Astronomy. Slate Mag., 13 May 2013. Web. 26 May 13. <http://www.slate.com/blogs/bad_astronomy/2013/05/13/ canada_and_science_nrc_will_now_only_do_science_that_promotes_economic_gai n.htm> Popper, Karl. After the Open Society: Selected Social and Political Writings. Ed. Jeremy Shearmur and Piers Norris Turner. London: Routledge, 2008. Print. Roach, Mary, ed. The Best American Science and Nature 2011. Boston: Houghton Mifflin Harcourt, 2011. Print.

Simons, Daniel J. and Christopher F. Chabris. Is Google Glass Dangerous? The New York Times 24 May 2013. Web. 24 May 2013. <http://www.nytimes.com/2013/ 05/26/opinion/ sunday/google-glass-may-be-hands-free-but-not-brain- free.html?smid=fb-nytimes&WT.z_sma=opinion_GGM_20130524> Strauss, Valerie. Can Computers Really Grade Essay Tests? The Washington Post 25 April 2013. Web. 25 April 2013.

article still in draft: please contact mhill30@du.edu to cite