Newshot

The Part About Machines (On AI Literary Scandals)

Original text(via HN AI)

This essay originally appeared in Spanish on the author’s Substack. It appears here in the author’s own translation.

The news has already traveled the world, but for those who haven’t yet heard: three weeks ago, El País published an article declaring that the award-winning novel Donde el agua trabaja, by Peruvian writer David Silva Rojas, was, according to Pangram, 99 percent AI-generated. The program also attributed 97 percent of the author’s answers in a subsequent interview to AI. A personal essay about his writing process: 80 percent AI. And the author’s Instagram post denying his use of artificial intelligence: 84 percent AI.

On November 6th, the Mexican publishing house Almadía reaffirmed its decision to give Silva Rojas the award and publish the book, following what it said was an examination of the “materials provided by the author.” Its statement did not specify which materials, nor did it detail what findings led the publisher to uphold the award.

Silva Rojas’s novel is about a young Peruvian woman who leaves Ayacucho, a southern Andean town, to work the land of a winery in the coastal region of Ica. Since most of the discussion surrounding the novel has been monomaniacally focused on identity, it’s perhaps appropriate to mention that I am a young Peruvian woman, that my entire family is from Ayacucho and Ica, and that many of my relatives—my parents in their youth, my uncles for many years, and my grandparents and great-grandparents for nearly their entire lives—have been agricultural workers. And also that, thanks to the great escalator that is social mobility, I have had one foot in San Francisco for over four years, and have spent much of that time reporting on and following the artificial-intelligence gold rush.1Two additional facts reveal how close this story is to my own life. First: I did my first journalistic internship at El País in mid-2024, although I no longer have any professional ties to the newspaper, nor am I trying to defend it here. Second: like Silva Rojas, I lived in Japan for several years, although I didn’t return writing like Bashō.

As a Peruvian, I can say that Donde el agua trabaja relies on mawkish clichés about the working-class experience in Ica. As a reader, I can say it’s a pretty bad book. And as someone who has spent years in and around the tech world, I can say that, despite Almadía’s statements, it’s hard to believe the novel wasn’t written with the help of artificial intelligence. But as a Latin American writer I’d like to point out something much more concerning. The reaction to these events has been a display of ignorance and obscurantism on the part of many intellectuals from our continent, who have claimed that Pangram “doesn’t work,” who have insisted that the “attack” on Silva is nothing more than a display of racism, and who have, in short, made it clear that what they truly want is to bury their heads in the sand and refuse to confront—beyond the scandal of the new—the arrival of this new technology.

As soon as El País published its report, certain writers proclaimed that AI detectors were also flagging as AI-generated the first page of Frankenstein or the U.S. Constitution. Yet their screenshots came not from Pangram but from TextGuard, a tool that (as an AFP investigation revealed earlier this year) misclassifies human text as AI-generated to then charge its user to “humanize” it. Said differently: those writers spread misinformation based on a fraudulent program. It didn’t matter. Their images went viral, and fueled the popular and erroneous notion that it is impossible to detect AI-written text.2I am not adding the link to these tweets, or to any of the other texts I am criticizing, because my interest here is not to accuse any author, but to show a tragic general trend.

Other writers conducted the reverse test: they published AI-generated texts that Pangram classified as human and argued, based on this, that Pangram is unreliable. And it’s true that Pangram is not error-proof. In fact, in one of its latest internal tests, the company reported a 0.37 percent false negative rate and a 0.0090 percent false positive rate in English, ranges validated by external observers. But this second group of writers was publishing examples of false negatives—that is, cases in which the detector fails to identify artificial text. These are different cases from false positives, in which human text is attributed to AI. Unlike false positives, false negatives on Pangram are much easier to create. Showing that this program overlooks certain artificial texts is not the same as demonstrating that it frequently accuses real writers of using AI.

A third group of writers said it’s wrong to “trust the judgment of a program still under development over the dignity of a human.” Obviously, it is possible to detect AI-generated text using our intuition. Once you become familiar with chatbots—whether you use them with suspicion and curiosity, or simply because your job forces you to—you can’t forget the sound of their voice. You start to recognize it whenever you receive a disposable email, or open LinkedIn, or read a bureaucratic document written to be ignored. But it’s also obvious that none of those who defended Donde el agua trabaja because we should “promote human judgment before trusting the judgment of a machine” have made much use of their own judgment. If they had, it would have been obvious to them that the novel has the voice of a large language model.

Does Pangram discriminate against those who don’t write in English? No. Does it discriminate against writers of color or with disabilities? Also no. But perhaps its reliability has not been examined in literary fiction? No, it has. Again, this program isn’t infallible: Pangram performs much worse when examining short texts, and it’s easy to outsmart with a little effort. But it remains the most reliable detection tool available to us. OpenAI and Anthropic might, in a few months, improve their language models to the point of making them undetectable—but for now we do live in a world where it is possible to detect most AI-generated text. Denying this is like denying that we have technologies like airplanes, computers or vaccines. Airplanes sometimes crash, computers sometimes short-circuit, and vaccines sometimes fail to immunize a tiny sliver of the population. But these are generally reliable and necessary technologies: the same is true for Pangram.3 November 10th update: As I said in the essay, Pangram is not a perfect tool, and I think skepticism and concerns about those who use it bluntly are valid. But we cannot have a serious discussion about the consequences of using tools like these in literature unless we’re working from the same basic facts.(1) It is false to say that no comparative studies have evaluated these detectors’ effectiveness on fiction; here is one conducted in English by outside researchers. (2) It is false to say that Pangram has not been tested on Spanish prose. Here is the model card for version 4.1, which reports a false positive rate of 0.0026 percent on a sample of Spanish-language texts. (3) It is true that early, rudimentary detection systems had biases, but those cannot simply be extrapolated to this tool, which uses a different methodology. (4) Pangram says in its privacy policy that it excludes texts you run through it (such as Silva Rojas’s novel) from training. In Bartz v. Anthropic, Anthropic agreed to pay $1.5 billion to settle a class-action lawsuit over downloading and retaining pirated copies of books. OpenAI is facing a similar copyright lawsuit brought by the New York Times. If you have compelling evidence that Pangram is using your writing to train its models, I invite you to assemble a legal team as soon as possible: you could soon be millionaires. (5) And if you have compelling evidence that the company is lying about the rates reported in its model card and other internal studies, I invite you to file a complaint with the SEC and/or the FTC: that would be fraud on the company’s part. If your doubts extend even to studies conducted by third parties, I hope you understand my comparison to conspiracy theorists. (6) Of course, the point about false negatives applies to any detector. But I’m glad we’ve moved from discussing whether detectors work at all to discussing which specific texts they work on.

We ask our engineers, our scientists and our tech bros to have a minimum level of general knowledge. We expect them to understand the ideological outlines of the last election and to know their country’s history, or at least to be aware that Spain once colonized us. Not knowing this is considered ignorance. The bar is even higher for our writers and critics, whom we expect to take an interest in all kinds of major developments in the world, especially if they are going to publicly comment on them. And yet—how many of our intellectuals could broadly describe the basic workings of artificial intelligence without resorting to clichés like “the machine” and “the algorithm”? How many understand the difference between a detection tool like Pangram and the first sponsored detector that pops up on Google? How many have even a minimum level of general knowledge when it comes to this technology? And how many prefer to acknowledge their ignorance rather than take refuge in bizarre, denialist conspiracy theories? Unfortunately, over the past few weeks we have discovered that the answer is very few. Almost none.

Then, our writers spoke about race. They said that it was troubling that a Spanish newspaper would use an imperialist, white, heteropatriarchal American machine to criticize a Peruvian author. They said that we are always the ones attacked, and I include myself here: us brown people, third-world minorities. And it’s true that we always receive more scrutiny, often for racist reasons—but that doesn’t mean that, sometimes, the accusations against us aren’t true.

If we want to play the identity game, we could also say that Donde el agua trabaja is the novel of a lawyer educated in the Global North who exploited the experiences of the women of Ica. We could cancel Silva Rojas. We could play this game forever. But I think it’s beneath us. Perhaps Silva Rojas has the misfortune of writing exactly like a large language model in each and every single one of his published texts. That would be fabulous, a real moment of magical realism—let’s give him the benefit of the doubt. Let’s read Donde el agua trabaja as if we were pure formalists, that is to say: as literature.

When I read the book earlier this month I didn’t find it atrocious. There are even a couple of endearing lines: “The day becomes a single crust of sand,” “The sky is white, washed by the heat.” (If you asked me, I’d say the text doesn’t seem to have been wholly invented by AI, but rather assembled, varnished and polished by it, constructed by it from the drafts and interviews Silva Rojas conducted in Ica.) Symbols of Peruvianness float through: chicha morada, chocotejas, caporales, Quebranta grapes, paracas winds. (As Borges once wrote, “what is truly native can and often does dispense with local color.”)

Grammatically, the book is defined by parataxis and corrective negation. Variants of the formula “It’s not X. It’s Y” appear over thirty times (“It doesn’t cure. It sustains.” “It doesn’t relieve. It orders.” “It’s not panic. It’s urgency.” “It’s not doubt. It’s calculation.”) The book is obsessed with the body, a word that reappears nearly ninety times throughout the text, almost in every other page. The body “weighs,” “hurts,” “tenses up,” “hesitates for a second,” “loosens,” “remains loose,” “wakes up,” “remembers,” “gets up anyway.” The body “takes time to settle,” “resists settling in,” “has already gotten used to it,” “moves forward,” “continues,” “keeps going” and “just keeps going.” The symbolic structure of the book weaves together this body—always a female one—with nature and water. For example, the mother’s eyes “are two dark, sunken seeds,” and to redeem herself and heal her trauma, the protagonist’s friend secretly bathes in a ditch in the middle of the night.

I would say the book’s central characteristic is its monotony. Because of its language and syntax, yes, but also because the story’s singular tone is catatonic misery. The protagonist suffers. In silence. She is humiliated. Betrayed. Hurt. The wind blows. She looks at the horizon. She stays quiet. She suffers a little more. This is how the book is written, often five words or less per sentence; the prose has the effect of a piano with a single key played at a piston-like rhythm. And, given the number of times phrases about the body, silence or water appear, it is curious that the book doesn’t have a single funny line or a moment of comic irony—though then we should also remember that large language models still can’t write good jokes.

The lives of workers, especially those who labor under forced conditions, are hard, even painful. But when I think of my family in Palpa—my father going out dancing all night and then walking to the orchard to pick oranges at four in the morning; my grandmother, far from her children, harvesting loquats with her heart in her mouth—I also think that they are some of the most vibrant people I know. They found humor and lightness in certain moments of the day, in the privacy of their homes, in their own minds. And not only humor: anger, disgust, jealousy, guilt, desire, terror, boredom, frustration, happiness, devotion, well, contradictions. The best Peruvian literature—I think here of a recent book like Algo nuestro sobre la tierra, by Joseph Zárate, or a classic one like Broad and Alien Is the World, by Ciro Alegría—understands all this: that our lives, the lives of the workers, contain much more than our most miserable moments.

Donde el agua trabaja only makes you feel sorry for its protagonist, a faceless, empty doll whom the author tortures again and again. Why? For whom?

It is useful to compare the Almadía scandal with more recent ones in the Anglo- and Francophone literary worlds. In English, the Trinidadian writer Jamir Nazir, and in French, the Haitian-born writer Thélyson Orélien, both won prestigious awards and were later accused of having used AI in the writing process. Both vehemently denied the allegations. (In the week between the publication of this essay in Spanish and its translation into English, a similar scandal involving the Caine Prize for African Writing began to draw renewed attention in the English-language world.)

In Orélien’s case, subsequent investigations confirmed that he had indeed used AI while writing, and his publisher, the Canadian Éditions du Boréal, severed ties with him. In Nazir’s case, the committee that awarded him the prize decided to stand by their choice, even after it came out that his writing style was completely different before the invention of ChatGPT, and even after an interview in the Atlantic revealed that the author was unable to recall the names of the poems that, according to him, had inspired his writing. (As in Almadía’s case, the jury that dismissed the accusations did not publish the bulk of its research; the Caine Prize partially did, in a statement that Pangram flagged as 100 percent written with AI.)

But let’s leave Pangram aside. The truly interesting question is why these three works, so alike, won these literary prizes. Orélien’s novel (the only one of these three titles I haven’t read) is about a Haitian immigrant who crosses South America, the Darién Gap and the U.S.-Mexico border to reach Canada. Nazir’s short story is about a farmer in rural Trinidad who tries to kill his wife by drowning her in a well. Silva Rojas’s novel is about a Peruvian woman who migrates to a rural hacienda where she is forced to work and, later, where she is sexually abused. All three deal with survival, identity, oppression, migration and/or rural life, the resistance of small acts, and violence against women. Nazir’s short story and Silva Rojas’s novel even share a central motif: a mysterious noise that seems to come from water (the grove of trees around a well for Nazir, an irrigation ditch for Silva Rojas) and that is symbolically linked to femicidal violence. If we could sum it up in one sentence: “Women suffer, nature remembers.”

I don’t think the similarities are a coincidence. Perhaps the publishing industry, in its desire to make amends for the decades in which it only published white writers from the national elite, has decided to correct in the opposite direction and ended up producing a new set of platitudes, one easy to reproduce mechanically. Perhaps it’s that, if you’re from the developing world, your only way to win a literary prize is to write in a way that makes academics with hyphenated surnames shed tears in their apartments in Miraflores, La Condesa or Barrio Norte—to write so they can feel good about themselves while a Silvio Rodríguez record wafts in from the kitchen. Perhaps, as an Argentinian friend said, AI came to literally confirm what we already knew: that this kind of identity-based worldview is banal, formulaic and produces bad literature. Or as another friend, also a Peruvian writer, said: “Immigration, water, land, women. They love that shit. They swallow it happily.”

When I came to the United States I wanted to write about snotty meritocrats, deliriously ambitious strivers, the strange new things being bred in San Francisco. But my professors and editors expected me to write about the border between Mexico and the United States, about immigration. When I said no, they didn’t know what to do with me. I wonder if something similar is happening here. These topics—immigration, water, land, women—are important, and great novels have been written about them, but they aren’t the only things us third-worlders can write about, and we shouldn’t receive awards for repeating bad clichés about them. It’s what gringos like to call the soft bigotry of low expectations. Benevolent racism. How could a Peruvian writer care about anything other than his misery? How could a Peruvian writer be anything other than a victim—a victim in his books, and then a victim of those who accuse him of having written a book with AI?

When the news first broke, I spent several days talking with many friends, also Latin American writers, about this scandal. We all think the book is bad and suspect it was assembled with AI, but we don’t know if we can say it publicly. We’re young—23, 24, 25 years old—our careers are barely getting started. The judges of this prize have power in the publishing world, dozens of international awards and the respect of the Global North. An open letter in support of Almadía collected signatures from some of the most established writers in the region. The literary universe is small and shrinking all the time. Everyone knows each other. Why say anything?

Because we must acknowledge the magnitude of what is happening. I feel no sympathy for the commentators who keep urging us to ignore AI, who repeat that in a few years we’ll look back on all this as an irrelevant fever dream puffed up by a few billionaires. Don’t get me wrong: I, too, would love to wake up and see everything go back to the way it was before November 2022. Why not? I hate the texture of the images generated by artificial intelligence, creepy and plastic, now inescapable in shop windows. I hate the ubiquity of AI text, the fact that suspicion is now impossible to shake off. I hate that it’s easier than ever to make pornography with my and my friends’ faces. I wish none of this were happening.

But it is happening. The world is changing. I’ve been following this technology for years, and for a long time I was cynical about it: I told myself that large language models would soon stop improving. Instead, their capabilities have only grown more prodigious. The only thing I believe now is that the years ahead of us are going to be strange and difficult. Our societies will change in response to this technology as much as they changed in response to the printing press, the locomotive, the internet and smartphones. Look at the controversy surrounding this novel; listen to yourselves, listen to us, arguing about the role of the author, of publishers, of the work of art in the age of its mechanical generation. If this past month is any indication, we will only continue to have more debates like this one, debates that will carve out ever-larger sections of our society.

The role of intellectuals is to shed light on these crossroads, to, as George Scialabba once wrote, understand their contemporary moral and political significance for the present with singular originality and force. And yet, confronted with a genuinely new phenomenon, our intellectuals have chosen to close their eyes. They have happily chosen ignorance.

Over the next few years we’ll have to create new rules surrounding the use of AI in literature. Perhaps we want to do as Gallimard will start to do next year and run the manuscript of every novel through Pangram, thus ensuring that literature remains the exclusive domain of the human. Perhaps we want to allow for experimentation with this technology as long as the author acknowledges its use. Perhaps we want to push the Oulipian dream to its limits and train our LLMs until they can write a masterpiece. I don’t know. What I do know is that I don’t want the world to be flooded with blatantly mediocre novels written by chatbots which we pass off as human simply to preserve the pride of certain writers. That seems to me the most certain way to destroy the future of literature.

Over the coming decades, we will need new ways of thinking, feeling and understanding our humanity. We will need new norms and new institutions. We will need to breathe new life into many old debates. And we will not be able to do so if people prefer, rather than acknowledging these radical transformations of our world, to hide behind stubborn habits and self-serving fantasies.

If a real novel is ever written about the workers from the south of my country, about people like my family, I’d like it not to be an obedient and dull book like this one. I’d like it to be frightening, full of rage at both the traditional racism of the Hispanists and the benevolent racism of the Latin American cultural elite. I’d like it to be impossible to mistake for the work of a machine. I’d like everyone to hate it when it comes out. And I’d love for it to win no prizes at all.

Image credit: lamanyana (Flickr / CC BY-NC-SA 2.0)

If you liked this essay, you’ll love reading The Point in print.

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by HN AI
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗