Are Book Reviewers Using ChatGPT?

A recent revelation I had.

I want to start by saying that talking about your book review, even a good one, is not a good look. I will be complaining about book reviews, and that feels even worse.

I know that getting any review is good. First, it means that you rose to the level of review. Not all books make that mark. And, I’m talking here about reviews in trade journals specifically, which people in the industry do rely on to purchase books for their store or library (they can’t read all the advance copies they get). One editor told me that schools and libraries won’t buy books that don’t have a certain number of trade reviews from places like School and Library Journal, Library Journal, Publisher’s Weekly, and Horn Book Review. So getting a review here helps you.

So, if any review is a good review, why am I about to complain?

Well, I had two really strange experiences that feel of-the-moment that I want to put out there.

The first was a review of A Most Perilous World: The Young Abolitionists and Their Crusade Against Slavery, which came out in June. First, it went to print with the factual error that my most recent book was—I kid you not—Well of Spoons. I’m not sure what that book is about, but my book was Well of Souls (that still hasn’t been fixed in the review). But what was actually more upsetting (and has been fixed) was that the reviewer said Lewis Douglass was in the first Black military regiment, the 1st South Carolina Volunteers. At first, this just felt like a lazy mistake— Douglass was in the 54th Massachusetts, which was the first official Black military regiment, while the 1st was an unofficial regiment. At the time, I thought, “Well, if they’d read basically the second half of the book, they’d know it was the 54th because I talk about that over and over again.” Being in the 54th nearly killed Douglass. Less egregious is the line that George “travel[ed] West to help bolster anti-slavery efforts,” but still wrong. He went West to make money and get out from his dad’s shadow, and I felt like I made that really clear in the book. This, I mostly let go of. I got other really great reviewers from folks who got it, so I personally didn’t feel like I needed it.

Then, I got a review from Library Journal for Go Back and Fetch It: Recovering Early Black Music in the Americas for Banjo and Fiddle. To be honest, this is such a niche book that I didn’t expect to get any reviews from trade publications for it. I’m much more interested what the music community thinks of it and in the coverage we are going to be getting from music and culture magazines. They spelled my name wrong (with a C), which is actually fine because people do it all the time and whatever.

Here’s where I’m actually upset. It says we examine “19 samples of Black music from West Africa, the Caribbean, and the United States spanning the period from 1687 to the 1860s.”

West Africa? Huh? The title literally includes “in the Americas”… there are no tunes from West Africa because that’s not what we are doing.

The next line also confused me: “The book begins with the first account of this kind of music, 1867’s Slave Songs of the United States, edited by abolitionists.” It is absolutely true that we include this book and that it is from 1867, but the author has just written that the music spans 1687 to 1860s, so why would the first example be from 1867?

Somehow, at that point, it hit me. I’ve been reading a lot about generative AI, and I’ll be honest that it terrifies me for many reasons, the most of which being water usage and environmental destruction and the least of which the deintellectualization of society, which is still pretty f*ing bad.

This review didn’t feel lazy, it felt like a poorly-informed chatbot had written it.

It felt beyond lazy. I read it again and it felt vague and slightly off in the way that my student’s writing does when I strongly suspect them of using ChatGPT. Throw the West Africa in there because “the banjo is from West Africa” (wrong, but the internet still believes this old research) or enslaved people were taken from West Africa (except they were taken from ports across the continent, but that is something often quoted on the internet). Start at 1867 because that is what we reference in the introduction, but it is not actually where the content starts and that’s all the chatbot got.

I thought back to the Publisher’s Weekly review. Maybe that wasn’t lazy either. The mistake about Douglass’s regiment felt like a mistake the internet would make, not being able to parse out the nuance between first official and first unofficial regiment.

I realized maybe the reviewers are using ChatGPT to make these reviews “easier” to write.

I told this to my dad, who knows I’ve been really interested in how ChatGPT is changing creative writing, and my personal theory that nonfiction writing might be our salvation against the bots.

“Oh, you’ve heard about the peer review thing, right?” he asked.

“No…”

He proceeded to tell me about how academics who have to peer-review articles for journals have been turning to AI. It takes a lot of time to do these reviews, but they are both necessary for the author of the article (who will need that publication to get tenure) and beneficial to the reviewer (who can say they review articles for the journal as part of their promotion packages). So instead of reading the article and providing feedback, they feed it into a genAI bot and asked for a peer review. And the academic jury is still out on whether this is OK.

Am I the asshole? Am I too suspicious?

Here’s the problem: now that we are in a world where people are using ChatGPT for everything, what’s to say that reviewers at trade publications aren’t using it to get through their work? They read 250 pages to write a paragraph summary, and are probably not getting paid much for it. I can always imagine an amount of skimming that a reader would do for that, but today, I can much more easily imagine someone feeding in the publisher’s description for a summary, and then adding some of their brief takes from reading the first couple of chapters, if that, and bam, what used to take a week of work now takes an hour.

There’s so much to say and write about this, but I’m left with: if someone used ChatGPT to review the book that I worked really hard on, do I want their review at all? No, not really. A bad human review is better than any good robot review.