ChatGPT- The Smooth Talking Stochastic Parrot

6 min readJun 13, 2023

“Is ChatGPT sentient ?”, “Can ChatGPT reason ?”, “Why do we believe the answers from ChatGPT?” and finally …

“Are the good intentions of the AI Baptists enough to save us all or Will the AI Bootleggers teach the AI to misbehave & inherit the earth ?(Or what is left of it, after the ’26 OpenAI wars) ”

The answers for the 1st two questions are relatively easier viz., “Not yet, but eventually we will have ‘Robot People’ among us …” , the 3rd one is a little more nuanced (we will dig deeper below) and for the 4th one, we will turn to Marc Andreessen.

My take : Humans search words to express concepts and ideas; ChatGPT just expresses words without any underlying reasoning or concepts. It just gives you “the optimal rhetorical answer to your prompt”
“What the large language models are good at is saying what an answer should sound like, which is different from what an answer should be” — Rodney Brooks [Here]
There is a difference between finding words to say something and saying anything by stringing statistically appropriate words — that is the difference between current ChatGPT and the humans
We should neither anthropomorphize the ChatGPT LLM, nor be seduced by it’s linguistic sophistication and manners.

And, … Hallucination is a feature, not a bug ! (my blog here)

First, ChatGPT vs a (normal) Parrot ! While the comparison is in jest, there is some truth to it !

Are we a Stochastic Parrot ?

Sam Altman (CEO of OpanAI) definitely thinks so ! (And Prof.Domingos had a humorous retort)

But (usually!) we don’t string words together without a reason — we reason and then look for the right words to express that concept !

What exactly is a stochastic parrot you might ask. Matei Zaharia has an interesting example in his edX class LLM101x :

If we ask ChatGPT to fill-in the word “Avocados are ___” it will say “green” rather than “blue”.
It is not because it has any clue what an avocado is; it’s answer is based on the training data tokens, where “green” has the highest statistical probability than any other word (because it knows that words with similar meaning often tend to occur in similar contexts); the set “avocado” and “green” has a probability distribution context, but “avocado” and “blue” does not.
For that matter ChatGPT doesn’t even know that “green” is a color — “green” is nothing but a token (in a context) for ChatGPT.
In short, it repeats the optimum from a stochastic probability distribution rather than any worldly semantic understanding or reasoning.
In fact, ChatGPT sees only numbers (as it’s internal representation) — but that is OK, as we don’t know what internal representation our brain has ! Even though we perceive words, images and colors, that need not be how they actually exist in the real world ! Scary, isn’t it !!
“Ray Kurzweil says we’ll solve AI by reverse engineering the brain, but more likely we’ll reverse engineer the brain by solving AI “— Domingos

Back to stochastic behavior …

Unfortunately, as Sam Altman says, we also do a similar thing — we learn grammar the same way — of course, we don’t need millions of samples and a billion dollar in training ! Once we comprehend which word goes with which other words, we use them that way — may be there is a stochastic engine in our brain or probably there are millions of tiny statistical engines !!

In case you are wondering, parrots are not stochastic - neither are we (except when a parrot was substituting for Sam Altman in Twitter ;o)). A good read [Here]

Language comprehension and consciousness — Is ChatGPT a smooth talking stochastic Parrot ?

As Prof. Pedro Domingos says “ChatGPT buyers beware — Technically, a language model is a device for making noise look like signal (e.g., making the output of a speech recognizer look coherent, regardless of what was actually said)”

Geoffrey Moore has an interesting blog (Understanding ChatGPT: A Triumph of Rhetoric) on this topic. Very insightful and succinct.

Moore’s description of ChatGPT as a Stochastic Parrot : “ChatGPT is operating entirely on form and not a whit on content. By processing the entirety of its corpus, it can generate the most probable sequence of words that correlates with the input prompt it had been fed”

We need to stop thinking of ChatGPT as artificial intelligence. It creates the illusion of intelligence, but it has no semantic component
ChatGPT is a vehicle for transmitting the wisdom of crowds, but it has no wisdom itself
It constructs its replies based solely on formal properties, it is selecting for rhetoric, not logic !!
It is delivering the optimal rhetorical answer to your prompt, not the most expert one. It is the one that is the most popular, not the one that is the most profound. In short, it has a great bedside manner, and that is why we feel so comfortable engaging with it.

Note the emphasis on good bedside manners — it is the by product of it’s goal to be rhetorically seductive !

“Large language models are smooth talkers (like the ones who got all the dates in high school). They have become incredibly good at producing text that looks right” — Bertrand Meyer [Here]

Where is ChatGPT relevant ?

Geoffrey Moore has an answer where to use ChatGPT and when not to touch it —

“It is the most patient of teachers, and it is incredibly well-informed. .. it can revolutionize technical support, patient care, claims processing, social services, language learning, and a host of other disciplines”
But .. “wherever ambiguity is paramount, wherever judgment is required, or wherever moral values are at stake, one must not expect ChatGPT to be the final arbiter”
That is simply not what it is designed to do. It can be an input, but it cannot be trusted to be the final output
Well put !!

In short, what is ChatGPT ?

Let me end the blog quoting Bertrand Meyer “… the equivalent of a cocky graduate student, smart and widely read, also polite and quick to apologize, but thoroughly, invariably, sloppy and unreliable” !!

“It gives an answer with complete confidence, and I sort of believe it. And half the time, it’s completely wrong.” — Rodney Brooks [Here]

What about the Emergent Behavior of ChatGPT ?

Marc Andreessen has a detailed blog — he has points on both sides, but concludes that “AI would be a way to make everything we care about better”.

The AI image generator Midjourney, known for it’s emerging behavior, does an excellent job of generating pictures — the pictures are interesting precisely because they are very non-human !

The creative side of Generative AI is promising, but we have a few ways to get there. OTOH, the emergent behavior is what could make them (possibly) dangerous …

Gizmodo continues “As evidenced by Brooker’s experiment, relying solely on AI to generate engaging material is a fool’s errand. Artificial intelligence is actually really stupid, considering it can only repackage ideas that already exist, and using it solely to create content is a hilariously bad idea”