\u201cIt\u2019s a huge drain on our small staff,\u201d said executive editor Matt Lord. \u201cPractically speaking, there\u2019s no world in which we can continue reading every single submission we get very closely and not make faster judgments in some cases.\u201d<\/p>\n
The people who run Boston\u2019s prominent literary magazines<\/a>, university-affiliated academic journals<\/a>, and scientific publications, such as the New England Journal of Medicine<\/a>, say they\u2019re increasingly struggling to tell the difference between human and AI writing. While these publications serve relatively narrow audiences, their contributions are key for new thinking about the world. And their struggles are part of a broader battle over how to preserve human understanding in the age of AI.<\/p>\n \u201cEmbodied human experience will always remain a human strength,\u201d said Bill Pierce, editor of the Boston-based literary magazine AGNI. He said AI-generated writing simply cannot convey the depths of what a person can.<\/p>\n \u201cSomeday maybe it will be able to tell us what the hell it is,\u201d he said of AI. \u201cBut it would still not be telling us what we are.\u201d<\/p>\n But some say AI can be beneficial for writing when the value of an article is the original, scientific research.<\/p>\n \u201cIf someone gets help with AI to do the writing, particularly if English isn\u2019t their first language and it communicates authentic data, then fine. I\u2019d rather have a well-written article,\u201d said Dr. Eric Rubin, editor-in-chief of the New England Journal of Medicine. \u201cThere\u2019s no question that articles from foreign authors have improved tremendously.\u201d<\/p>\n That journal, however, also publishes opinion pieces and letters to the editor<\/a>. That\u2019s where AI writing can be harmful.<\/p>\n \u201cWe don\u2019t really care about what ChatGPT\u2019s opinion is,\u201d Rubin said.<\/p>\n Up until very recently, the prospect of using AI to generate competent prose seemed far off. But new scandals have emerged, including one at the New York Times, which cut ties<\/a> with a freelance book reviewer who used AI to help write a book review, and at a British Commonwealth short story competition, where an author was accused<\/a> of using AI to write a prize-winning short story. AI-generated books are also flooding Amazon and even seeping into bookstores<\/a>.<\/p>\n Even publications that haven\u2019t seen a massive uptick are aware of the threat of machine-made submissions. Many are wrestling with what guidelines to implement for writers and researchers. <\/p>\n Like other publishers, MIT Press adopted general rules<\/a> when large language models first began to take off. It prohibits applicants from listing AI models as an author, and requires authors to be completely transparent about how they used AI in the research and writing process (some individual MIT journals also have implemented more specific guidelines). <\/p>\n But editors across the organization\u2019s 43 journals kept receiving papers they suspected were produced with the help of AI, minus the disclosures. Some of the signs are well-documented, such as the frequent use of certain phrases, a heavy reliance on em-dashes<\/a>, and syntax such as \u201cit\u2019s not X, it\u2019s Y.\u201d<\/a><\/p>\n Not only has there been what Nick Lindsay, the MIT Press director of journals, calls \u2018a significant uptick\u2019 in papers they suspect are written by AI, but also a few instances where an author has submitted up to five papers to a journal in a single day, which is a massive number. <\/p>\n \u201cThat can be a real problem,\u201d Lindsay said. \u201cEach submission actually costs the publisher something,\u201d adding that the platforms MIT Press use for submission, which help them manage a high volume of papers, typically charge the organization anywhere from $10 to $32 each time someone submits an article.<\/p>\n While AI is hampering some publications, others aren\u2019t as concerned. And in some cases the tech can be beneficial, such as in fact-checking large datasets, and translations that Rubin, of the New England Journal of Medicine, referenced.<\/p>\n Shyam Madhani, editor-in-chief of the Yale Law Journal, said the publication took an open approach to crafting its first AI policy<\/a>, which it created this year and requires writers to disclose how they used the technology, adding that its usage would not deter the journal from considering their work.<\/p>\n \u201cThere\u2019s no consensus either inside the journal or outside in legal academia on how to evaluate AI use in scholarship,\u201d he said. <\/p>\n He argued that some of the concerns typically raised, such as prose being too formulaic or arguments that have already been published by other authors, are not AI problems, but rather issues with the quality of writing or novelty of the paper itself. Those are factors for which the journal is already reviewing, so the use of AI is secondary. <\/p>\n Of course, that puts a greater emphasis on reviewing submissions to ensure that they are sound to publish.<\/p>\n \u201cWe\u2019re trying to make sure we adapt as things go on,\u201d Madhani said. The editorial staff has established an approach \u201cmostly not on how to evaluate pieces that use AI writing, but how to enforce a policy where our main job as editors is to ensure the accuracy of the pieces.\u201d<\/p>\n At the New England Journal of Medicine, Rubin said that editors are implementing AI into its own review process<\/a>. That includes statistical editors, who use the tech to help fact-check models.<\/p>\n But there are other challenges. Rubin said there is little the journal can do if the science itself is AI-generated. Images are another problem.<\/p>\n In May, the NEJM had to retract<\/a> a case study regarding black \u201cbronchial casts\u201d that were removed from a man\u2019s lungs following a forest fire, after its authors admitted<\/a> to using artificial intelligence to manipulate the image. At issue was a crooked ruler placed at the top of the samples that authors used AI to straighten. <\/p>\n \u201cThat was a kind of dumb one,\u201d Rubin said. \u201cIt seems like we\u2019re always adjusting to the last crisis.\u201d<\/p>\n To help with the opinion pieces, Rubin said the journal is experimenting with AI detectors, but is unsure of their efficacy. He declined to disclose which tools his team is using, out of fear that researchers and experts would use them ahead of submitting their papers. <\/p>\n Despite the challenges AI is causing and the potential impact it could have going forward, Rubin believes people will always be at the heart of the journal\u2019s work \u2014 a sentiment that was shared across publications of various disciplines.<\/p>\n \u201cWe\u2019re primarily a clinical research journal. You can\u2019t do clinical research without sitting down with patients,\u201d he said. \u201cNot everything is built into the literature.\u201d<\/p>\n Aidan Ryan can be reached at aidan.ryan@globe.com<\/a>. Follow him @aidanfitzryan<\/a>.<\/p>\n