What Kind of Writer Is ChatGPT?

Annals of Inquiry

Last spring, a graduate student in social anthropology—let’s call him Chris—sat down at his laptop and asked ChatGPT for help with a writing assignment. He pasted a few thousand words, a mix of rough summaries and jotted-down bullet points, into the text box that serves as ChatGPT’s interface. “Here’s my entire exam,” he wrote. “Don’t edit it, I will tell you what to do after you’ve read it.”

Chris was tackling a difficult paper about perspectivism, which is the anthropological principle that one’s perspective inevitably shapes the observations one makes and the knowledge one acquires. ChatGPT asked him, “What specific tasks or assistance do you need with this content?” Chris pasted one paragraph from his draft into the text box. “Please edit it,” he typed.ChatGPT returned a condensed and slightly reworded version of the paragraph. The tweaked version removed the French term grande idée and Americanized the British spelling of “marginalization,” but otherwise wasn’t obviously better. Soon, Chris gave up on getting ChatGPT to edit his work. Instead, he outlined a passage that he wanted to add to the paper. “Please write this paragraph as you deem fit,” he instructed.

The response was stilted. “I want the language to be alive yet direct and to the point,” he typed. But the result still wasn’t quite right. He told ChatGPT: “I like this, but write it slightly more as a story. Don’t overdo it.”

In the course of an hours-long exchange, Chris came to realize that ChatGPT’s writing wasn’t up to his standards. He tried other approaches. At one point, he shared the text of a relevant book chapter and, in a sort of Socratic dialogue with the model, asked a string of questions: “Can you give an example that illustrates this?” “What do you think it refers to?” “Could you give an illustrative example with myth instead?” Later, Chris asked the A.I.’s opinion about the strength of a paragraph he had written. “Can you put this argument to the test? Is it true, in your analysis?”

When ChatGPT came out, many people deemed it a perfect plagiarism tool. “AI seems almost built for cheating,” Ethan Mollick, an A.I. commentator, wrote in his book “Co-Intelligence.” (He later predicted a “homework apocalypse.”) An agricultural-science professor at Texas A&M failed all his students when he became convinced that the whole class had used A.I. to write their assignments. (It turned out that his method of detection—asking ChatGPT whether it had generated the papers in question—was unreliable, so he changed the grades.) A columnist in The Spectator asked, “How can you send students home with essay assignments when, between puffs of quasi-legal weed, they can tell their laptop: ‘Hey, ChatGPT, write a good 1,000-word paper comparing the themes of Fleabag and Macbeth’—and two seconds later, voila?”

But “voilà” was not the word for whatever ChatGPT was doing for Chris. He was not outsourcing his exam to ChatGPT; he rarely made use of the new text or revisions that the chatbot provided. He also didn’t seem to be streamlining or speeding up his writing process. If I had been Chris’s professor, I would have wanted him to disclose his use of the tool, but I don’t think I would have considered it cheating. So what was it?

I asked Chris to explain what he thought he was doing. “You could say that while my own role is a sort of orchestrator and guide, ChatGPT’s role is mainly the execution and supporting of the structure of my work,” he told me. This made sense in theory, but it didn’t seem to capture the chaotic transcript I had read. Chris then rattled off several additional contributions that ChatGPT had made to his process. He was struggling to explain how the tool had actually helped him; he seemed to lack the language to describe this new style of collaboration.

Chris is not the only one. Last year, Stacey Pigg, a communications professor at North Carolina State University, systematically studied thirty-five YouTube and TikTok videos in which people used generative A.I. to help craft research papers. Pigg partitioned the videos into short segments, each of which captured a single digital interaction that she could label and categorize. Pigg ultimately identified forty-four things that writers had asked ChatGPT to do, such as rewording findings for a lay audience or evaluating the quality of citations. Some of these activities—asking the model to craft a conclusion section from scratch, say—could be considered cheating, but most were harder to categorize. Writing with A.I. was not a “coherent or stable phenomenon,” she wrote. “Instead, the study suggests disparate, fragmented uses of generative AI technologies.” More recently, the nonprofit that organizes National Novel Writing Month, NaNoWriMo, announced that it would allow participants to use A.I., and then said in a statement: “to categorically condemn AI would be to ignore classist and ableist issues surrounding the use of the technology.” A firestorm soon followed; some critics accused NaNoWriMo itself of classism and ableism, and the organization updated its statement. One thing that seemed clear was that no one really had a good grasp on how writers were using A.I. tools, and which uses might be more acceptable than others.

For the writers Pigg studied and the students I interviewed for this article, ChatGPT was not so much a perfect plagiarism tool as a sounding board. The chatbot couldn’t produce large sections of usable text, but it could explore ideas, sharpen existing prose, or provide rough text for the student to polish. It allowed writers to play with their own words and ideas. In some cases, these interactions with ChatGPT seem almost parasocial. Chris told me that Chat—his nickname for ChatGPT—was a “good conversation partner.” Another student, who was profiled in a recent paper on the subject, nicknamed the chatbot Lisa and described “her” as “a partner and even a friend.” ChatGPT raises difficult practical issues about originality and plagiarism. But the binary question “Is it cheating?” hides the possibility that something new and inventive might be going on here.

Writing is hard. It requires us to use multiple parts of the brain in an improbable symphony of high-strain effort. Our hippocampus summons relevant facts; the prefrontal cortex tries to organize them. A brain region known as Broca’s area helps us to narrate in a familiar inner voice; our verbal working memory stores and manipulates the narration as we transfer it to the page. Meanwhile, our brain recruits our spatial working memory, which evolved to track our location in physical space, to orient our words within a whole. (In the lab, if you ask a person to trace a pattern with her free hand—a standard method to exhaust spatial working memory—her ability to structure an essay diminishes.)These mental demands may help to explain the eccentric habits of writers. To write his first book, “The End of Nature,” Bill McKibben left the din of his Manhattan apartment, moving to a remote millhouse in the Adirondacks; Maya Angelou would write in hotel rooms, removing the art work from the walls to simplify the space. There’s a romance to isolation, but in these cases a more practical motivation lurks: eliminating distractions makes it easier for people to coerce their brain circuitry to write.

Many authors reduce the cognitive demands of writing through the use of familiar habits and rituals, designed to ease the brain into a mode conducive to literary production. Haruki Murakami pairs his writing sessions with ten-kilometre runs, believing that the physical and the mental are closely related. Agatha Christie found it easier to come up with plot ideas while eating apples during long soaks in a bath. Anne Lamott advised writing a “shitty first draft” as fast as possible, warming up the brain for the harder task of producing professional-quality prose. Ernest Hemingway achieved a similar effect by stopping each session in the middle of the page so that he’d have an easier time starting the next day. “When you’re still going good,” he supposedly remarked, “that’s the time to stop.”

Writers also obsess over the tools that might make their craft more tolerable. Quentin Tarantino writes the first drafts of his screenplays longhand; Neal Stephenson wrote The Baroque Cycle, a series of eight books set in the seventeenth and eighteenth centuries, with a fountain pen. (For a while, the Science Fiction and Fantasy Hall of Fame in Seattle exhibited his handwritten manuscript, empty ink bottles, and discarded blotting paper.) Many writers, myself included, swear by specialized writing programs such as Scrivener, which will organize research materials and partition projects into numerous pieces. Others remain loyal to whatever software they first mastered. George R. R. Martin still uses WordStar 4.0, which was originally released in 1987. He runs it on Microsoft DOS and saves his manuscripts to floppy disks. When a person is struggling to write, all of these details matter. If using a fountain pen or retreating to a mountain house can make the cognitive load of producing text even somewhat more bearable, writers will consider it.After observing Chris, I started to wonder whether ChatGPT could be understood as his version of WordStar or eating apples in the bath: a brain hack used to make the act of writing feel less difficult. I tested my theory by asking ChatGPT for help with the first paragraph in this section, about the neural processes involved in writing. I started by seeking research assistance: I described the passage I hoped to write and asked ChatGPT for “facts about the neuroscience of writing that emphasize that writing is in fact hard.” It organized its response into seven topics, including “High Cognitive Load” and “Executive Function,” providing a few sentences of notes for each. I asked the chatbot to dive deeper into the role of temporal lobes, as that seemed promising. ChatGPT replied with another list, this time summarizing six subtopics.

These responses gave me some ideas for what to research later, but they weren’t directly useful. On technical topics, ChatGPT seemed tuned to a sort of book-report blandness, as when it told me that “writing often requires converting subtle emotional tones and nuances, whether in dialogue, narrative descriptions, or persuasive arguments.” I had even less success when I asked the chatbot to provide a quote from a scientist which emphasized the cognitive difficulty of writing. I was given a perfect quip—“Writing is the most complex and demanding task our brains can undertake”—attributed to the Harvard psychologist Steven Pinker. In reality, it was too perfect: I could find no evidence that Pinker had made this claim. ChatGPT appeared to have hallucinated it.

Following the approach I learned from Chris, I changed tactics. I gave ChatGPT a rough outline to expand:

Can you rewrite the following paragraph in more interesting language like you might encounter in a New Yorker story: ‘Writing is hard. It involves many parts of the brain to work together. The hippocampus is necessary to retrieve memories. The temporal lobes are also involved.’

The result wasn’t terrible. ChatGPT came up with a symphony metaphor that was similar to what appears in my final version of the paragraph. But the writing wasn’t good, either. The chatbot mixed metaphors—just a sentence after describing the brain’s activities as a symphony, for example, it described the work of the temporal lobes as a dance—and its language was heavy with adjectives and was often over the top. “Every word is a testament to our brain’s remarkable capabilities,” ChatGPT declared, as though it were speaking in the voice of David Attenborough rather than that of The New Yorker.

At first, I struggled to understand why anyone would want to write this way. My dialogue with ChatGPT was frustratingly meandering, as though I were excavating an essay instead of crafting one. But, when I thought about the psychological experience of writing, I began to see the value of the tool. ChatGPT was not generating professional prose all at once, but it was providing starting points: interesting research ideas to explore; mediocre paragraphs that might, with sufficient editing, become usable. For all its inefficiencies, this indirect approach did feel easier than staring at a blank page; “talking” to the chatbot about the article was more fun than toiling in quiet isolation. In the long run, I wasn’t saving time: I still needed to look up facts and write sentences in my own voice. But my exchanges seemed to reduce the maximum mental effort demanded of me. Old-fashioned writing requires bursts of focus that remind me of the sharp spikes of an electrocardiogram. Working with ChatGPT mellowed the experience, rounding those spikes into the smooth curves of a sine wave.

It’s possible to explain my experience with ChatGPT in cognitive terms. If writing requires a person to store information using multiple types of working memory at the same time, then back-and-forth conversations with ChatGPT may provide moments of respite, by temporarily offloading some of this information. (A more complete break—for example, scrolling through one’s phone or checking one’s e-mail—would be much more disruptive.) So even seemingly unproductive interactions might provide the subtle benefit of increasing your over-all writing stamina. Collaborating with A.I. can also offer you a high-tech “shitty first draft,” allowing you to spend more time editing bad text and less time trying to craft good text from scratch. ChatGPT is not so much writing for you as generating a mental state that helps you produce better writing.

When I first spoke to Chris about writing with ChatGPT, I was too preoccupied with the question of who was doing the writing. I should have also been thinking about how chatbots change the experience of filling a blank page. “Not all text is either human-authored or synthetic,” Alan M. Knowles, a researcher who studies human-A.I. interactions, recently wrote in the journal Computers and Composition. “These are both meaningful categories that should not be discarded, but they are insufficient for discussing how writers use GenAI in practice.” Knowles describes the collaboration between writers and A.I. as “rhetorical load sharing.” A.I. isn’t writing on our behalf, but neither is it merely supporting us while we write from scratch; it sits somewhere in between. In this way, it is both on the spectrum of writing hacks and rituals and also, in some sense, beyond it. This helps to explain our discomfort with the technology. We’re used to writers moving to a quiet location or using a special pen to help get their creative juices flowing. We’re not yet used to the idea that they might chat with a computer program to release cognitive strain, or ask the program for a rough draft to help generate mental momentum.

When I asked a few professors about A.I.-assisted writing, I was met with mixed feelings. One instructor told me that easy writing assignments, such as short response paragraphs that nudge students to complete their reading, are easily “GPT’d,” and would likely need to be eliminated. But, for longer and more complex assignments, such as Chris’s graduate-school paper on perspectivism, teachers seem to be discovering, to their relief, that “load sharing” with ChatGPT—though it is a new and unfamiliar technique—still requires students to think carefully and write clearly. It might make the process of completing an assignment feel less daunting, but it’s not a shortcut to receiving a higher grade.

I tried ending this essay in a few ways, but my editor wasn’t happy with any of them. When I ran low on ideas, I decided to ask ChatGPT to share the load. Could it write a pithy concluding line for me? “In the end, the true value of tools like ChatGPT lies not in making academic work easier, but in empowering students to engage more deeply with their ideas and express them with greater confidence,” the chatbot suggested. That’s not terrible, I thought. But it still needed some work.