Hallucination is a feature not a bug !

You may have heard this sentiment expressed about Generative AI and ChatGPT in numerous contexts. While it may sound like a catchphrase, this statement holds significant relevance and carries various implications…

Hallucination & Generative AI

• In the realm of artificial intelligence, the term ‘hallucination’ often conjures images of distorted perceptions or false sensory experiences. However, within the context of Generative AI, hallucination takes on a unique and fascinating dimension.
• It refers to the phenomenon where AI models generate outputs that diverge from the underlying reality captured in the training data.
• While this may initially seem like a flaw in the model’s performance, a closer examination reveals that hallucination is not merely a bug to be fixed but rather a feature inherent to the generative process.

In this blog, we delve into the intriguing world of generative AI hallucination, exploring ̶i̶t̶s̶ ̶c̶a̶u̶s̶e̶s̶,̶ its implications, and creative potential. F̶r̶o̶m̶ ̶d̶i̶s̶t̶o̶r̶t̶e̶d̶ ̶i̶m̶a̶g̶e̶s̶ ̶t̶o̶ ̶n̶o̶n̶s̶e̶n̶s̶i̶c̶a̶l̶ ̶t̶e̶x̶t̶,̶ ̶w̶e̶ ̶u̶n̶c̶o̶v̶e̶r̶ ̶t̶h̶e̶ ̶c̶o̶m̶p̶l̶e̶x̶i̶t̶i̶e̶s̶ ̶o̶f̶ ̶h̶a̶l̶l̶u̶c̶i̶n̶a̶t̶i̶o̶n̶ ̶a̶n̶d̶ ̶i̶t̶s̶ ̶r̶o̶l̶e̶ ̶i̶n̶ ̶s̶h̶a̶p̶i̶n̶g̶ ̶t̶h̶e̶ ̶f̶u̶t̶u̶r̶e̶ ̶o̶f̶ ̶A̶I̶-̶d̶r̶i̶v̶e̶n̶ ̶c̶r̶e̶a̶t̶i̶v̶i̶t̶y̶ ̶a̶n̶d̶ ̶i̶n̶n̶o̶v̶a̶t̶i̶o̶n̶.̶

Note : I have a little bit more to say about the above section, at the end !

Currently Generative AI is used for opportunistic applications, not the ultimate applications

• An interesting blog aptly named “On holding back the strange AI tide” and LinkedIn post by Prof.Ethan Mollick is very insightful.
• Ethan says and I quote “Companies keep picking the same few safe* projects to focus on for AI: customer service, support copilots, sales, and “talk to my documents.” All of these are existing IT solutions that can be given an AI gloss”

And, that is challenging primarily because that is not an optimal sweet spot for Generative AI

• It works, works better than the tools we have now — but to apply Generative AI for these applications, we have to do good amount of scaffolding ! “.. trying to stop AI from being weird” !
• I have written about the Guardrails and other intricacies of Generative AI [here] and [here]. Initial initial slides [here] for my talk at Nvidia GTC [here]
• “.. these approaches often highlight the weak points of AI more than technologists expect. No matter how much RAG or fine-tuning you do, your LLM system will hallucinate details. Thus areas where accuracy is expected, like talking to internal documents or interacting with customers, can be more challenging than people think. And external-facing systems are vulnerable to prompt injection” — Prof. Ethan Mollick

Generative AI is best used natively (including it’s hallucination) than simple automation

• “.. even worse, the substantial benefits of AI are going to be greatly reduced by trying to pretend it is just like previous waves of technology..”
• “.. But LLMs are inherently weird and powerful in ways that don’t fit into typical IT solutions, or even our conventional approaches to organizational capabilities and boundaries. Make sure you are devoting at least some effort to doing much stranger and more powerful experiments, because AIs are only getting better from here” — Prof. Ethan Mollick

These are the stupidest the models will ever be !

• Said Sam Altman during an interview with Bill Gates [here]
• We are on a very steep improvement curve and will stay there for next 5–10 years, driving the cost of intelligence to nearly zero !

Which means they will get more creative and will hallucinate more — but the good kind of hallucinations !

In short, definitely use generative AI wherever you get performance improvements (with appropriate guardrails), also look for futuristic, non-linear, disruptive opportunities

Our work systems are not built for AI, so we will need to rebuild them — as Agents : License to Operate !

• This is an interesting and frightening reality. We shouldn’t apply Generative AI just to automate isolated components within existing work flow — such as customer service (internal or external) or the automation of operations. We should reengineer our processes using Generative AI ! That is Operations Optimization in this new world !!
• Agents and License to Operate ! The other day I saw an interesting company sema4.ai — LinkedIn announcement [here] and Mayfield [here].
• Their goal “.. build category-defining intelligent agents that connect context to action, changing how work gets done in the enterprise”

I have much more to discuss about agents and the essential skills required for achieving Artificial General Intelligence (AGI) — perhaps a topic for another day. You can find my preliminary thoughts on AGI in the rough-cut blog [here]

Cognitive Plumbing !

Mayfield has an interesting diagram (below). Slightly self serving, highlighting their investments, still useful — food for thought !

[Update] A recent blog by Navin talks about Cognition As A Service (CaaS) [here].

• That makes sense. Of course, it is different from rule based automation. In fact it need not be automation at all — meaning humans can be in the loop as needed by the workflow; even escalation would be part of it.
• I do have couple of disagreements with Navin Chaddha’s CaaS blog — the travel example is an agent and doesn’t have cognition per se; same with the fraud alert — it is a rule driven agent; we can send forms and block transactions now. The Digital Co-worker, Digital Twin et al are CaaS. The key is to let humans do what they can do best and machines do their best.

Epilogue

You might be wondering why I had some sentences in s̶t̶r̶i̶k̶e̶ ̶t̶h̶r̶o̶u̶g̶h̶ it is because that initial section was written by ChatGPT v3.5 (rest is mine) ! The s̶t̶r̶i̶k̶e̶ ̶t̶h̶r̶o̶u̶g̶h̶ is my edit - it doesn’t know what I plan to write under the title; the formatting is mine too, of course.

What a better way to start a blog on Hallucinating Generative AI than by enlisting a Hallucinating Generative AI to compose the introduction !

And, it did an excellent job — without knowing anything about Hallucination or that it is Hallucinating, it effectively argues for the significance of Hallucination !! Take another look at the opening section “Hallucination & Generative AI”