Making the Grade
I was at dinner with a colleague this week—midterm week. Predictably, talk turned to the scourge of all professors: grading essays. There are few tasks in the life of a college professor less fulfilling than grading student essays. Every once in a while a really good essay jolts me to consciousness. I am elated by such encounters. To be honest, however, reading essays is for the most part stultifying. This is not the fault of the students, many of whom are brilliant and exuberant writers. I find it trying to wade through 25 essays discussing the same book, offering varying opinions and theories, while keeping my attention and interest. How many different ways can one ask for a thesis, talk about the importance of transition sentences, and correct grammar? For some time it is fun, in a way. One learns new things and is captivated by comparing how bright young minds see things. But after years, grading the essay becomes just part of the worst part of a great job.
Flickr, MacVicar
So how might my colleagues and I react to news that EdX—the influential Harvard-MIT led consortium offering online courses—has developed software that will grade college student essays? I imagine it is sort of like how people felt when the dishwasher was invented. You mean we can cook and feast and don’t have to scrub pots and wash dishes? It promises to allow us to focus on teaching well without having to do that part of our job that we truly dread.
The appeal of computer grading is obvious and broad. Not only will many professors and teachers be freed from unwanted tedium, but also it may help our students. One advantage of computer grading is that it is nearly instantaneous. Students can hand in their work and get a grade and feedback seconds later. Too often essays are handed back days or even weeks after they are submitted. By then the students have lost interest in their paper and forgotten the inspiration that breathed life into their writing. To receive immediate feedback will allow students to see what they did wrong and how they could improve while the generative impulse underlying the paper is still fresh. Computer grading might encourage students to turn in numerous drafts of a paper; it may very well help teach students to write better, something that professorial comments delivered after a week rarely accomplish.
Another putative advantage of computer grading is its objectivity and consistency. Every professor knows that it matters when we read essays and in what order. Some essays find us awake and attentive. Others meet my eyes as they struggle to remain open. As much as I try to ignore the names on the top of the page, I can’t deny that my reading and grading is personalized to the students. I teach at a small liberal arts college where I know the students. If I read a particularly difficult sentence by a student I have come to trust, I often make a second effort. My personal attention has advantages but it is of course discriminatory. The computer will not do that, which may be seen by some as more fair. What is more, the computer doesn’t get tired or need caffeine.
Perhaps the most important advantage for administrators considering these programs is the cost savings. If computers relieve professors from the burden of grading, that means professors can teach more. It may also mean that fewer TA’s are necessary in large lecture courses, thus saving money for strapped universities. There may even be a further side benefit to these programs. If universities need fewer TA’s to grade papers, they may admit fewer graduate students to their programs, thus going some way towards alleviating the extraordinary and irresponsible over-production of young professors that is swelling the ranks of unemployable Ph.D.s.
There are, of course, real worries about computer grading of essays. My concern is not that the computers will make mistakes (so do I); or that we lack studies that show that computers can grade as well as human professors—for I doubt professors are on the whole excellent graders. The real issue is elsewhere.
According to the group “Professionals Against Machine Scoring of Student Essays in High-Stakes Assessment,” the problem with computer grading of essays is simple: Machines cannot read. Here is what the group says in a statement:
Let’s face the realities of automatic essay scoring. Computers cannot ‘read.’ They cannot measure the essentials of effective written communication: accuracy, reasoning, adequacy of evidence, good sense, ethical stance, convincing argument, meaningful organization, clarity, and veracity, among others.
What needs to be taken seriously is not that computers can’t grade as well as humans. In many ways they grade better. More consistently. More honestly. With less grade inflation. And more quickly. But computer grading will be different than human grading. It will be less nuanced and aspire to clearly defined criteria. Are sentences grammatical? Is there a clear statement of the thesis? Are there examples given? Is there a transition between sentences? All of these are important parts of good writing and the computer can be trained to look for these characteristics in an essay. What this means, however, is that computers will demand the kind of clear, precise, and logical writing that computers can understand and that many professors and administrators demand from students. What this also means, however, is that writing will become more mechanical.
There is much to be learned here from an analogy with the rise of computer chess. The great grandmaster Gary Kasparov—who famously lost to Deep Blue— has perceptively argued that machines have changed the ways Chess is played and redefined what a good chess move and a well-played chess game looks like. As I have written before:
The heavy use of computer analysis has pushed the game itself in new directions. The machine doesn’t care about style or patterns or hundreds of years of established theory. It counts up the values of the chess pieces, analyzes a few billion moves, and counts them up again. (A computer translates each piece and each positional factor into a value in order to reduce the game to numbers it can crunch.) It is entirely free of prejudice and doctrine and this has contributed to the development of players who are almost as free of dogma as the machines with which they train. Increasingly, a move isn’t good or bad because it looks that way or because it hasn’t been done that way before. It’s simply good if it works and bad if it doesn’t. Although we still require a strong measure of intuition and logic to play well, humans today are starting to play more like computers. One way to put this is that as we rely on computers and begin to value what computers value and think like computers think, our world becomes more rational, more efficient, and more powerful, but also less beautiful, less unique, and less exotic.
Much the same might be expected from the increasing use of computers to grade (and eventually to write) essays. Students will learn to write in ways expected from computers, just as they today try to learn to write in ways desired by their professors. The difference is that different professors demand and respond to varying styles. Computers will consistently and logically drive writing towards a more mechanical and logical style. Writing, like Chess playing, will likely become more rational, more efficient, and more effective, but also less beautiful, less unique, and less eccentric. In other words, writing will become less human.
It turns out that many secondary school districts already use computers to grade essays. But according to John Markoff in The New York Times, the EdX software promises to bring the technology into college classrooms as well as online courses.
It is quite possible that in the near future, my colleagues and I will no longer have to complain about grading essays. But that is unlikely at Bard. More likely is that such software will be used in large university lecture courses. In such courses with hundreds of students, professors already shorten questions or replace essays with multiple-choice tests. Or they use armies of underpaid graduate students to grade these essays. It is quite likely that software will actually augment the educational value of writing assignments at college in these large lecture halls.
In seminars, however, and in classes at small liberal arts colleges like Bard where I teach, such software will not likely free my colleagues and me from reading essays. The essays I assign are not simple responses to questions in which there are clear criteria for grading. I look for elegance, brevity, insight, and the human spark (please no comments on my writing). Whether or not I am good at evaluating writing or at teaching writing, that is my aspiration. I seek to encourage writing that is thoughtful rather than writing that is simply accurate. When I have time to make meaningful comments on papers, they concern structure, elegance, and depth. It is not only a way to grade an essay, but also a way to connect with my students and help them to see what it means to write and think well.
And yet, I can easily imagine making use of such a computer-grading program. I rarely have time to grade essays as well or as quickly as I would like. I would love to have my students submit drafts of their essays to the EdX computer program.
If they could repeatedly submit their essays and receive such feedback and use the computer to catch not only grammatical errors but also poor sentences, redundancies, repetitions, and whatever other mistakes the computer can be trained to recognize, that would allow them to respond and rework their essays many times before I see them. Used well, I hope, such grading programs might really augment my capacities as a professor and their experiences as students.
I have real fears that grading technology will rarely be used well. Rather, it will too-often replace human grading altogether and in large lectures, high schools and standardized tests will impose a new and inhuman standard on the way we write and thus the way we think. We should greet such new technologies enthusiastically and skeptically. But first, we should try to understand them. Towards that end, it is well worth reading John Markoff’s excellent account of the new EdX computer grading software in The New York Times. It is your weekend read.
-RB
What is a Fact?
What is a fact? Few more thorny questions exist. Consider this, from Hannah Arendt’s essay, “Truth and Politics:”
But do facts, independent of opinion and interpretation, exist at all? Have not generations of historians and philosophers of history demonstrated the impossibility of ascertaining facts without interpretation, since they must first be picked out of a chaos of sheer happenings (and the principles of choice are surely not factual data) and then be fitted into a story that can be told only in certain perspective, which has nothing to do with the original occurrence?
Facts are constructed. They are not objective. And there is no clear test for what is a fact. Thus, when Albert Einstein was asked, how science can separate fact from fiction, brilliant hypotheses from nutty quackery, he answered: ‘There is no objective test.” Unlike rational truths that are true outside of experience and absolute, all factual truths are contingent. They might have been otherwise. That is one reason it is so hard to pin them down.
Steve Shapin reminds us of these puzzles in an excellent essay in this weeks London Review of Books. Shapin is reviewing a new book on Immanuel Velikovsky by Michael Gordin. Velikovsky, for those born since the 1960s, caused an uproar in the 1960s and 70s with his scientific claims that Venus was the result of a dislodged piece of Jupiter, that comets led to the parting of the Red Sea, that it dislodged the orbit of Mars threatening Earth, and caused the relocation of the North Pole, not to mention the showering of plagues of vermin onto the earth that nourished the Israelites in the desert.
Gordin’s book is about how American scientists went ballistic over Velikovsky. They sought to censor his work and schemed to prevent the publication of his book, Worlds in Collision, at the prestigious Macmillan press. At the center of the controversy was Harvard, where establishment scientists worked assiduously to discredit Velikovsky and stop the circulation of his ideas. [I am sensitive to such issues because I was also the target of such a suppression campaign. When my book The Gift of Science was about to be published by Harvard University Press, I received a call from the editor. It turns out an established scholar had demanded that HUP not publish my book, threatening to no longer review books for the press let alone publish with them. Thankfully, HUP resisted that pressure, for which I will always be grateful.]
For these Harvard scientists, Velikovsky was a charlatan peddling a dangerous pseudo science. The danger in Velikovsky’s claims was more than simple misinformation. It led, above all, to an attack on the very essence of scientific authority. What Velikovsky claimed as science flew in the face of what the scientific community knew to be true. He set himself up as an outsider, a dissident. Which he was. In the wake of totalitarianism, he argued that democratic society must allow for alternative and heretical views. The establishment, Velikovsky insisted, had no monopoly on truth. Let all views out, and let the best one win.
Shapin beautifully sums up the real seduction and danger lurking in Velikovsky’s work.
The Velikovsky affair made clear that there were radically differing conceptions of the political and intellectual constitution of a legitimate scientific community, of what it was to make and evaluate scientific knowledge. One appealing notion was that science is and ought to be a democracy, willing to consider all factual and theoretical claims, regardless of who makes them and of how they stand with respect to canons of existing belief. Challenges to orthodoxy ought to be welcomed: after all, hadn’t science been born historically through such challenges and hadn’t it progressed by means of the continual creative destruction of dogma? This, of course, was Velikovsky’s view, and it was not an easy matter for scientists in the liberal West to deny the legitimacy of that picture of scientific life. (Wasn’t this the lesson that ought to be learned from the experience of science in Nazi Germany and Stalinist Russia?) Yet living according to such ideals was impossible – nothing could be accomplished if every apparently crazy idea were to be given careful consideration – and in 1962 Thomas Kuhn’s immensely influential Structure of Scientific Revolutions commended a general picture of science in which ‘dogma’ (daringly given that name) had an essential role in science and in which ‘normal science’ rightly proceeded not through its permeability to all sorts of ideas but through a socially enforced ‘narrowing of perception’. Scientists judged new ideas to be beyond the pale not because they didn’t conform to abstract ideas about scientific values or formal notions of scientific method, but because such claims, given what scientists securely knew about the world, were implausible. Planets just didn’t behave the way Velikovsky said they did; his celestial mechanics required electromagnetic forces which just didn’t exist; the tails of comets were just not the sorts of body that could dump oil and manna on Middle Eastern deserts. A Harvard astronomer blandly noted that ‘if Dr Velikovsky is right, the rest of us are crazy.'
Immanuel Velikovsky
It is hard not to read this account and not think about contemporary debates over global warming, Darwinism, and the fall of the World Trade Center. In all three cases, outsiders and even some dissident scientists have made arguments that have been loudly disavowed by mainstream scientists.
No one has done more to explore the claims of modern pseudo science than Naomi Oreskes. In her book Merchants of Doubt written with Erik Conway, Oreskes shows how “a small handful of men” could, for purely ideological reasons, sow doubt about the ‘facts’ regarding global warming and the health effects of cigarettes. In a similar vein, Jonathan Kay has chronicled the efforts of pseudo scientists to argue that there was no possible way that the World Trade Towers could have been brought down by jet fuel fires, thus suggesting and seeking to “prove” that the U.S. government was behind the destruction of 9/11.
Oreskes wants to show, at once, that it is too easy for politically motivated scientists to sow doubt about scientific fact, and also that there is a workable and effective way for the scientific community to patrol the border between science and pseudo science. What governs that boundary is, in Oreskes words, “the scientific consensus.” The argument that global warming is a fact rests on claims about the scientific method: value free studies, evaluated by a system of peer review, moving towards consensus. Peer review is, for Oreskes, “is a crucial part of science.” And yet, for those who engage in it know full well, peer review is also deeply political, subject to petty and also not so petty disputes, jealousies, and vendettas. For this and other reasons, consensus is, as Oreskes herself admits, not always accurate: “The scientific consensus might, of course, be wrong. If the history of science teaches anything, it is humility, and no one can be faulted for failing to act on what is not known.”
Just as Einstein said 50 years ago, in the matters of establishing scientific fact, there is no objective test. This is frustrating. Indeed, it can be dangerous, not only when pseudo scientists sow doubt about global warming thus preventing meaningful and necessary action. But also, the pervasive and persuasive claims of pseudo science sow cynicism that undermines the factual and truthful foundations of human life.
Arendt reminds us, with a clarity rarely equaled, that factual truth is always contingent. “Facts are beyond agreement and consent, and all talk about them—all exchanges of opinion based on correct information—will contribute nothing to their establishment.” Against the pseudo scientific claims of many, science is always a contingent and hypothetical endeavor, one that deals in hypotheses, agreement, and factual proof. Scientific truth is always empirical truth and the truths of science are, in the end, grounded in consensus.
The trouble here is that scientific truths must—as scientific—claim to be true and not simply an opinion. Science makes a claim to authority that is predicated not upon proof but on the value and meaningfulness of impartial inquiry. It is a value that is increasingly in question.
What the challenge of pseudo science shows is how tenuous scientific authority and the value placed on disinterested research really is. Such inquiry has not always been valued and there is no reason to expect it to be valued about partial inquiry in the future. Arendt suggests that the origin of the value in disinterested inquiry was Homer’s decision to praise the Trojans equally as he lauded the Achaeans. Never before, she writes, had one people been able to look “with equal eyes upon friend and foe.” It was this revolutionary Greek objectivity that became the source for modern science. For those who do value science and understand the incredible advantages it has bestowed upon modern civilization, it is important to recall that the Homeric disinterestedness is neither natural nor necessary. In the effort to fight pseudo science, we must be willing and able to defend just such a position and thus what Nietzsche calls the “pathos of distance” must be central to any defense of the modern scientific world.
When science loses its authority, pseudo science thrives. That is the situation we are increasingly in today. There are no objective tests and no clear lines demarcating good and bad science. And that leaves us with the challenge of the modern age: to pursue truth and establish facts without secure or stable foundations. For that, we need reliable guides whom we can trust. And for that reason, you should read Steven Shapin’s latest essay. It is your weekend read.
-RB
Making an Appearance
“In contrast to the inorganic thereness of lifeless matter, living beings are not mere appearances. To be alive means to be possessed by an urge toward self-display which answers the fact of one’s own appearingness. Living things make their appearance like actors on a stage set for them.”
-Hannah Arendt, The Life of the Mind, vol. 1: Thinking
Political theorists are likely to associate the phrase the “urge to self-display” with a characteristically “Arendtian” politics. But here, Arendt takes self-display to characterize something much more basic and fundamental—the sheer life of human beings. Despite Arendt’s imagery of the actor appearing on a stage, self-display does not seem at all to invoke the greatness of individuality or of heroic deeds. It is merely the “fact of one’s own appearingness.” What could Arendt mean by characterizing human life by the fact of appearing, and what does it mean to say that human beings, as opposed to “lifeless matter” makes their appearance?
In The Life of the Mind, Arendt describes the phenomenon of appearing as human beings’ appearing to others in a way that is subject to the particular perspective of the spectator.
“To appear,” she writes, “always means to seem to others, and this seeming varies according to the standpoint and perspective of the spectator”. In this interpretation, the fact of appearingness is a fact of the world in which we live; it is the fact of plurality and the irreducibility of perspectives that signals that men, not Man, populate the world.
But the fact of appearance also has a moral and political significance that goes beyond this almost formal description of the dual position of subjectivity and objectivity that human beings occupy with respect to one another. If we turn to Origins of Totalitarianism, a text that is not often read in connection with The Life of the Mind, we are confronted with a striking and terrifying picture of the loss of appearingness, which confronts us fully with the implications of Arendt’s characterization of human beings as beings who must make their appearance.
In Origins of Totalitarianism, Arendt uses the term “rightlessness” to describe the condition of European Jews under the Third Reich. In that regime, Jews were not merely “deprived of life, liberty, and the pursuit of happiness, or of equality before the law and freedom of opinion,” but made so irrelevant that “nobody wants even to oppress them”. The ultimate expression of invisibility was the genocide in the death camps of the Final Solution. However, the effectiveness of these camps in rendering people invisible did not lie simply in the physical destruction of millions. The camps sought to destroy what Arendt called the “moral man,” or that aspect of human beings that is subject to moral judgment and valuation. This term attaches not to moral behavior, but to the presence of individual human beings in the world that makes it possible to see them as individuals in the first place.
In the camps, the boundary between life and death and between individuals was so attenuated that it was nearly impossible to distinguish any one person from another, living or dead. The invisibility of individuals this lack of boundaries engendered was so thoroughgoing that it obscured even the most heroic of deaths: “[i]t belonged among the refinements of totalitarian governments in our century that they don’t permit their opponents to die a great, dramatic martyr’s death for their convictions….The totalitarian state lets its opponents disappear in silent anonymity”. Even the most heroic of acts was disposed of simply and without regard or comment, just as those deaths that occurred daily, and both were made invisible along with the individuals in and through whom these deaths occurred.
The crucial point is not that death was made routine, but that the camps ensured that with these deaths any marker of the victim’s having ever been alive also disappeared along with him. The individual prisoner was barely distinguished from the others and seen only as one in a series in which his exact position was irrelevant. As a group, the prisoners were invisible to the world, and as individuals, they were invisible to the world and to one another as distinct people.
The result was an attenuation of the line that separates the lives of individuals as they have lived it from mere physical life and death and the elimination of the world as a stage on which individuals could make their appearance. And in the absence of this stage, death could be nothing more than a “seal on the fact that he had never really existed”.
Making one’s appearance in the world, as an actor does on a stage, is not about being extraordinary. Nor is it a merely formal description of how human beings perceive the world around them and are perceived by other human beings. Rather, appearingness is the essential condition of being recognized as a member of the community of human beings and the world and of being treated accordingly. As the events of the past century have made strikingly clear, appearingness is a condition that we could lose or of which we could be stripped. Our condition of humanity is something that we must create—create by making our appearance in the world. Arendt’s words about our basic condition of appearance alerts us to the dangers of invisibility and should make us suspicious of any situation in which people exist in a condition of invisibility.
In our own time, the Occupy Wall Street movement has helped to bring to light some of those who have been made invisible in poverty. This act of opening up a space in which an individual might make their appearance in the world is, I think, one of the movement’s greatest accomplishments. And a politics of visibility is not just about our own visibility or our own great accomplishments, but about creating stages upon which people can make their appearance and exposing and tearing down those scaffoldings that bar some from entering these stages.
If we see the OWS movement as a politics of appearance, then the albeit valid criticisms about the lack of a definite agenda and the like do seem to lose some of their force. But this does not mean that the movement is a success in Arendt’s terms. The movement has certainly brought us to the stage, but what we all—the invisible and the visible—do with this opening and how we make our appearance onto it remains the political question that only the individual actors, and not any movement, can and must answer.
-Jennie Han
Sign up for HAC News
Recent Comments
- Louis N. Proyect on Hannah Arendt – a film by Margarethe von Trotta.
- Hannah Arendt: The Movie « Hannah Arendt Center for Politics and Humanities | FentCiutat on Hannah Arendt: The Movie
- The Future of Higher Education | The Full Montbel on MOOCs: The Debate Continues
- Amor Mundi 5/12/13 « Hannah Arendt Center for Politics and Humanities on Arnold Gehlen on Arendt’s The Human Condition
- Shaun on The Courage to Do What is Right
Archives
- May 2013
- April 2013
- March 2013
- February 2013
- January 2013
- December 2012
- November 2012
- October 2012
- September 2012
- August 2012
- July 2012
- June 2012
- May 2012
- April 2012
- March 2012
- February 2012
- January 2012
- December 2011
- November 2011
- October 2011
- September 2011
- August 2011
- July 2011
- May 2011
- April 2011
- January 2011
- December 2010
- November 2010
- October 2010
- September 2010
- August 2010
- July 2010
- June 2010
- April 2010
Tags
Blogroll
- Black Octavo
- Daily Dish
- Fortnightly Review
- Hannah Arendt Center
- Hannah Arendt Center Multimedia at Vimeo
- Harpers
- NY Review of Books
- Praxis at Big Think
- The Contemporary Condition
- The Point
- The Relative Absolute
- Via Meadia
- Who's Afraid of Social Democracy?
- WordPress Blog
- WordPress Planet