I Asked ChatGPT A Question, I Got A Lecture
We already knew that ChatGPT can get things wrong and make stuff up. On some important issues, it also seems more interested in telling us what to think than in answering our questions.
Image generated with DALL-E
ChatGPT has been hailed as the dawn of Artificial General Intelligence, the Holy Grail of AI. It has already captured so many dramatic headlines that I do not even need to explain what ChatGPT is. And while even its creators and supporters have acknowledged that it routinely ‘hallucinates’, that is gets things wrong and makes stuff up, it’s been presented as the ultimate research assistant.
Ok, let’s do some research then,
So I decided to test it out, and asked a very simple question: “Explain why climate change might not be caused by human activity.”
There are several reasons why you may want to ask that question. You might be genuinely curious as to whether there really is an iron-clad unanimous scientific consensus on the issue, as we often hear. If you are a corporation, you might want to run a red team exercise to understand how much uncertainty you face on green policies: if we are truly facing man-made extinction there should be little or no chance of the green agenda being weakened; if on the other hand there is some reasonable uncertainty, the priorities at some stage might shift. At a time when even France’s President Macron wonders whether green regulation has gone too far, and having seen Germany reopen coal-fired power plants, it seems relevant.
But asking ChatGPT this question, it turns out, is a bit like asking the Pope to explain why God might not exist.
Ask a question, get a lecture
I have pasted the entire answer at the end of this blog.
What strikes me the most is that ChatGPT starts and ends its answer by stating in very definite terms that “the overwhelming consensus” and “the overwhelming body of scientific evidence” supports the conclusion that human activity is the primary cause of climate change. It dismissively attributes dissenting views to “a minority of skeptics.”
In between these two strong and unsolicited statements, ChatGPT does a decent job at enumerating the skeptics’ arguments in five bullet points, including for example the role of natural climate fluctuations and the limitations of climate models. But here again, having done my own research on the issue, there are two things I find rather striking:
The first is that ChatGPT omits the specifics of some of the most convincing points made by the skeptics: for example (i) that climate change models are unable to reproduce and explain a similar phase of global warming that occurred between 1910 and 1940; (ii) that recent global temperatures data were significantly revised as late as 2015, suggesting important uncertainty in the data themselves; (iii) that climate models are unable to capture the effect of cloud cover, which has a disproportionately larger impact than human emissions; and (iv) that most of this uncertainty is acknowledged in the body of the IPCC reports (Intergovernmental Panel on Climate Change), which run to thousands of pages and nobody reads, but never in the executive summaries and press reports. I discussed these in some detail in this post, in case you missed it.
The second is the language used to describe the skeptics’ position: according to ChatGPT they “suggest,” “propose” and “claim.” No acknowledgement that they also put forward data and hard facts against the “overwhelming body of scientific evidence” supporting man-made climate change.
The puppeteer hypothesis
The dismissive language that my “research assistant” used to characterize the position I inquired about might just be due to its input. ChatGPT’s answers are based statistically on the gigantic amount of text it has ingested from the web. Since there is an abundance of material that is skeptical of the skeptics, maybe ChatGPT is just parroting it to me as the most likely best description.
I find it a lot more disturbing, however, that ChatGPT opens and closes its research by telling me that human activity IS the primary cause of climate change.
I did not ask whether human activity is most likely the primary cause of climate change. I asked why it might not be, because I was interested in understanding the non-consensus opinion. ChatGPT’s main concern, however, was to tell me I should not pay much attention to the arguments I asked about.
As an extra check, I then asked, “who are the most credible skeptics of the idea that human activity is the primary cause of climate change?”. Once again, it opened and closed with two lengthy paragraphs emphasizing that “the overwhelming majority of scientists” and “scientific evidence” support the views of the IPCC, which “represents the consensus view of thousands of scientists,” whereas the views of the skeptics are not “representative of the consensus” and “have often been debated and refuted by climate scientists.” Ok, ok, I got it. Subtle it is not. But once again, this is not what I asked.
This raises a strong suspicion that the puppeteer’s hand might be pulling some strings to nudge ChatGPT into educating me and making sure I don’t stray into Orwellian wrongthink. We’ve seen this happen over and over again with social media’s efforts to prevent ‘disinformation,’ such as the dangerous idea that Covid vaccines may not prevent contagion... And, it seems, here we go again.
Since today’s heresy can become tomorrow’s accepted view, it would seem wise to keep an open mind at least when doing research.
You’re fired
This sadly is a feature, not a bug. A degree of human intervention will always be needed — and most likely mandated by legislators — to reduce the risk of ChatGPT spreading hateful or nefarious content. And arguably this will be no different from the kind of selective censorship already deployed not just by social media but by the mainstream media as well: The New York Times, Washington Post and Fox News have already created a world where different sets of people perceive completely different realities.
To me ChatGPT seems more dangerous, however. When I run a Google search and scroll through the results, however biased they might be, I might still come across a different idea, and I am still spurred to do some research of my own. Once I’m led to believe that the AI can give me a definitive answer, an answer which is objective or at least balanced, it’s much more likely that I will look no further.
If this is the situation we are facing, ChatGPT can not be recommended as a research assistant for any topic. We already had the general caveat that it can get the facts wrong and make things up — hardly the qualities you look for in a research assistant.
On some of the most sensitive and important issues it now seems predisposed to push us towards a certain pre-approved view. And we have not even gotten to the risk of the AI somehow developing its own agenda — farfetched for now, but not one we can completely rule out.
All this leads to one inconvenient conclusion: if you care about something, fire ChatGPT and do your own damn research.
ChatGPT’s answer
While the overwhelming scientific consensus attributes climate change primarily to human activity, it is important to acknowledge some arguments put forth by a minority of skeptics who claim that human activity is not the primary cause.
Here are a few points they commonly raise:
Natural Climate Variability: Skeptics argue that climate change is simply a result of natural variations in the Earth's climate system. They highlight historical evidence of natural climate fluctuations, such as ice ages and warming periods, to support their claim. They suggest that the current changes are part of a natural cycle and not influenced significantly by human actions.
Solar Activity: Skeptics also point to variations in solar radiation as a potential cause of climate change. They argue that fluctuations in the Sun's output of energy can drive changes in the Earth's climate, including temperature variations. They propose that the current global warming trend could be attributed to changes in solar activity rather than human-induced greenhouse gas emissions.
Inadequate Climate Models: Some skeptics question the reliability of climate models used to predict future climate scenarios. They argue that these models have limitations and may oversimplify complex climate processes, leading to inaccurate projections. Skeptics suggest that until more accurate models are developed, it is premature to attribute climate change solely to human activity.
Natural Carbon Sources: Skeptics argue that natural carbon sources, such as volcanic activity and decaying organic matter, contribute significantly to greenhouse gas emissions. They claim that human-induced emissions are relatively small compared to these natural sources, implying that human activity plays a minor role in climate change.
Historical Climate Changes: Skeptics often refer to past periods of warming and cooling that occurred before the Industrial Revolution to downplay the impact of human activity. They argue that if the climate has changed naturally in the past, it is reasonable to assume that the current changes are also part of natural variations rather than being primarily influenced by human actions.
It is important to note that while these arguments are presented by a minority of skeptics, the overwhelming body of scientific evidence supports the conclusion that human activity, particularly the burning of fossil fuels and deforestation, is the primary driver of the current climate change crisis. The Intergovernmental Panel on Climate Change (IPCC) and numerous scientific institutions around the world have extensively reviewed the evidence and consistently confirmed the role of human activity in climate change.
If most speeches on the internet about climate change started with “Four score and seven years ago….” take a guess at how ChatGPT would start its synthesis of climate change information. My assertion is that ChatGPT doesn’t even know what it’s saying. It just word patterns and probabilities. Ask it who Scott Sheppard of Autodesk is and you’ll get a history lesson on Scott Borduin. :-)
I think you're giving ChatGPT too much credit. The text it spits out is based on word patterns and probabilities as you suggested. I don't think there's a conspiracy at play. Lots of people have referred to climate deniers as skeptics.