More on the disturbing plausibility of ChatGPT

April 12, 2023

Prompted by a post on Mastodon (which, like all Mastodon posts, I can no longer find), I asked ChatGPT to tell me about my own papers. The response started out well but quickly got much worse. I will indent my comments on its response.

Q. What are some articles written by Michael P. Taylor?

A. Michael P. Taylor is a vertebrate paleontologist and open science advocate. He has written numerous articles and papers on topics such as the taxonomy and systematics of sauropod dinosaurs, open access publishing, and scholarly communication.

So far, so good. In general, LLMs seem to be good at synthesizing overviews. It’s specific facts that trip them up.

Here are a few examples of his articles:

Taylor, M. P. (2009). A re-evaluation of Brachiosaurus altithorax Riggs 1903 (Dinosauria, Sauropoda) and its generic separation from Giraffatitan brancai (Janensch 1914) (Dinosauria, Sauropoda). Journal of Vertebrate Paleontology, 29(3), 787-806.

The first one gives is pretty good. The only mistake is that it’s oddly appended “(Dinosauria, Sauropoda)” to the end of the article title. The journal, volume, issue and page-numbers are all correct.

Taylor, M. P. (2011). Sauropod dinosaur research: a historical review. In A. W. A. Kellner, & D. A. S. Silva (Eds.), Vertebrate paleontology and sedimentology of the Jurassic-Cretaceous boundary in the northern and central parts of the Lusitanian Basin, Portugal (pp. 11-43). Sociedade de Geologia de Portugal.

I did write a book-chapter titled “Sauropod dinosaur research: a historical review”. But it came out in 2010, not 2011. And it ran from pages 361–386, not 11-43. And it was in Dinosaurs and Other Extinct Saurians: a Historical Perspective, not Vertebrate paleontology and sedimentology of the Jurassic-Cretaceous boundary in the northern and central parts of the Lusitanian Basin, Portugal (which doesn’t exist). Which was edited by Richard T. J. Moody, Eric Buffetaut, Darren Naish and David M. Martill, not A. W. A. Kellner & D. A. S. Silva. And published by the Geological Society of London, not Sociedade de Geologia de Portugal.

Taylor, M. P., & Wedel, M. J. (2018). Why sauropods had long necks; and why giraffes have short necks. PeerJ Preprints, 6, e27537v1.

I and Matt did co-write a paper with a similar title in PeerJ. But it’s Why sauropods had long necks; and why giraffes have short necks, not the other way around. And it came out in 2013, not 2018. And it was in PeerJ proper, not PeerJ Preprints.

Taylor, M. P. (2020). The open-access wars: How to free science from academic paywalls. Wired.

I never wrote any such paper. There is an article out there whose HTML title is the one given here, though the visible title is completely different: “The war to free science”. It’s in Vox, not Wired; it came out in 2019, not 2020. And it’s by Brian Resnick and Julia Belluz.

Taylor, M. P., & Knope, M. L. (2021). The future of scholarly publishing is in our hands. The Conversation.

As far as I recall, I never wrote anything for The Conversation. There seems to be no article named “The future of scholarly publishing is in our hands”. And I have never met or corresponded with, far less co-authored with, anyone with the surname Knope.

All in all, a pretty horrible performance — but, and this is the scary part, one that even someone pretty familiar with my work could very easily have swallowed.

I made a very similar point a few months ago in These new “artificial intelligence” programs don’t know what they’re talking about. If the responses were obviously wrong, there wouldn’t be too much of a problem here. But this is close enough to fool most people most of the time.

As more and more of what we read is likely to be written by LLMs in the next years, I am concerned that we’re going to be dealing with an awful lot of this kind of thing. The ability to pan gold out of the stream of lies is going to become really important.

5 Responses to “More on the disturbing plausibility of ChatGPT”

  1. To say it simple: You can’t trust in the results from ChatGPT. I made a similar experience: I asked ChatGPT for a short vita about a old friend who worked 10 years ago about extinct mammals. ChatGPT mixed the vita of my old friend with the vita of an older classical musician with the same name. Funny to read, but wrong.

  2. Antonio Dias Says:

    This sounds to me like a feature of AI, and not just some unfortunate bug.

    Since power is held by those with no belief in quality, only quantity, an “AI’ that cranks out plausible content is perfect.

    Since those in power have no other interest than retaining and increasing their power, having our cultural conversations buried in an avalanche of plausible disinformation most likely fulfills this prime interest.

    What worries me the most is that as we continue to put more and more of our attention and efforts online, within this electrified sphere, we become its, AI’s, unwitting allies.

    All it would take to defeat it would be to turn off the power.

    This will happen at some point, one way or the other.

    Our question is, will it only happen over our dead bodies?

  3. It is no bug, but it is although no dark intention. ChatGPT is based on a neuronal network and this again is based on our own brains. ChatGPT calculates for every word in a sentence which word is the most likely. Our brain works very similar when we are speaking. But our brain has other parts for correction in order to minimize spoken mistakes – but, occasionally, we make such mistakes as well, e.g. freudian slips. The AI beyond ChatGPT makes more such mistakes because it lacks the remaining abilities of our brain to evaluate. It is not yet technically feasible.

  4. Mike Taylor Says:

    “Our brain works very similar when we are speaking.”

    I don’t think that’s true at all — at least it’s not my lived experience. When I speak, I am actively trying to communicate a specific truth about external reality — not just juggling words.

  5. llewelly Says:

    the term “neural network” as used in computing was coined in the late 1950s, when almost nothing was known about how neurons or brains work, and it was applied to mathematical constructs that had older origins. It’s ahistorical to say they’re “based on our own brains” because the requisite knowledge of how brains work did not exist at that time. Even today, anything “based on our own brains” would necessarily rely on a combination of theories (some of which will sooner or later be disproven or altered), uncertain hypotheses, approximations, and educated guesses, because there’s still many unknowns in the requisite areas of science.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: