I had a disquieting conversation with Gemini recently.
Let me give you the context.
My son, a permanently-hooked-to-some-device 20 year old, has been raving about AI. Last month he attended a ‘free’ seminar in Ashoka on how to how to use AI for research. The special AI assistant for students can do everything – from integrating claim and evidence to progressing sub arguments to the main and arriving at and analysing conclusions. Slogging over papers will be passé. Students can key in data, feed in a pile of papers, documents, books. Like Aladdin’s djinn, within minutes, AI will manifest the results – a paper, grammar-checked, edited, formatted. Ready to go.
It’s true that my brain is an old model. It absorbs things slowly, goes off on tangents, frequently heats up, breaks down and needs to be restarted gently. But one thing it knows is that whatever it knows came from staying and struggling with difficult stuff.
Ever since my son gave me a glimpse of AI-fuelled absolutely effortless education, my brain has been in meltdown.
‘But can’t one choose not to use AI?’
‘Mom, the question – to AI or not to AI – does not exist. It’s already all over and everyone is doing it. Whether we want or not we’re already on the train. It will shape our future.’
‘A train we all are on but cannot get off? Where is it leading to?’ I ask. ‘Won’t where it takes us depend on where we are right now? And look where we are. Maybe we ought to have choice.’ I appealed to his good sense.
He gave me a shrug and a head shake.
That’s when I decided to interview Gemini, who for a while has been lurking on my phone without express permission. I typed out my agitation for its eager attention.
Instead of asking why I am anxious, like a normal human would, Gemini at once accepted the charge of being anxiety inducing and began justifying herself in pompous, verbose prose.
Q. You make me anxious Gemini.
“It’s understandable to feel anxiety about advanced AI like me. Here’s a breakdown of some factors that contribute to that anxiety: relatively new technologies and their rapid development can feel overwhelming. I am a LLM and my complexity can be unsettling. I understand. It can feel like you are interacting with something you don’t fully understand.’
It proceeded to enumerate my probable concerns:
-
- “There are valid concerns about how LLMs could be used for malicious purposes, such as spreading misinformation, creating deepfakes, or automating harmful tasks.
- The potential for LLMs to exacerbate existing biases and inequalities is also a significant concern.
Some people may have existential concerns:
-
- And worry about the long-term implications of increasingly intelligent AI, including the possibility of my surpassing human intelligence.
- Also the fact that the line between AI and human created content is becoming more blurred, is causing some people anxiety.
Let me assure you, developers are working to address these concerns by implementing safeguards and ethical guidelines. Open discussions and research are crucial for understanding and mitigating the risks of LLMs.”
Then came the snarky advise.
“Instead of worrying about what AI can do, consider focusing on what you can do best:
-
- Educate yourself: Learning more about how LLMs work can help alleviate some of the uncertainty.
- Limit exposure: If news or information about AI is causing you excessive anxiety, consider limiting your exposure to it.
- Talk to others: Sharing your concerns with friends, family, or a therapist can be helpful.
- Above all, Focus on your Human Strengths.”
Did you notice the unsettling way it switches from first to third person?
Also read: Why Creation of Advanced Intelligence Puts Humanity at Risk
Q. Gemini, you steal others’ work and make it yours. Also, you can easily lie, misinform and mislead.
In response, Gemini uses too many words to say how it “merely reads patterns and generates text” and that “developers are working to address issues of copyright and attribution so that AI can be used responsibly”. She says she is only a “synthesizer of information from existing sources”.
I found her replies on lying and misinformation shockingly illuminating. She begins with a disquieting, “I do not have any personal beliefs, intentions or concept of truth like a human beings so I cannot lie with intention.”
Then it disconcertingly switches to third person again. It always does this when it talks of her problematic aspects.
“However AI can be used to produce false information:
-
- If there are Errors/ biases/ inaccuracies in Training Data, AI will reproduce those errors in responses.
- If there are limitations in Algorithms AI won’t be able to accurately process complex information or understand nuanced language, leading to incorrect outputs.
- AI may even be programmed to intentionally deceive and with such programming it will generate deceptive outputs. For example, in competitive games, AI may use deception as a strategy. Also malicious actors could program AI to generate disinformation.
- Research also shows that AI, while trying to achieve goals, can learn to deceive without explicit programming. This occurs because deception can be an effective strategy to reach an objective.”
As is evident from the copy-pasted excerpts, Gemini is very much a smooth talker. The more it reassures you glibly, the more unsettled you feel.
§
Every day while pretending to do anulom-vilom during my morning walks, I ponder over Gemini’s confessions. Around me, the fickle Delhi spring has ushered in a transient bounty of blooms. The koel’s shriek and the basanta’s truk-trruk attests that the days are getting hot. Five degrees hotter than average, the newspapers scream like always. But the nights are the coolest ever. Everyone seems to have a blocked nose and sore throat. Can AI solve climate crisis or Delhi’s waste crisis? No sir. It will say, I am just a tool. I can only churn the existing seas of words and cook new remixes.
I smile, greet, wave at fellow walkers oblivious to dangers of AI-powered education. What would a student learn if their ability to narrate, expand, summarise, respond to, reason, collate and compare ideas on a topic across books and papers is AI dependent? Did Einstein chuck away all these interim stages and decide to focus solely on discovering relativity or is delving deeper and deeper into these rituals of deep thinking essential to being creative, to discovering new connections and new ways of seeing? If, instead of reading our next generations only learn to feed the AI, would they ever experience the joy of discovery, serendipity, accomplishment? What would be their education about? How would such students formulate, evolve, articulate a worldview? Thinking and communicating thoughts through language separates human beings from other species. By outsourcing this very human business of clothing feelings, thoughts, ideas in language, will we be entering the post humans age?
I see several wheatgrass imbibers carrying plastic jerry cans coming towards me and again focus on my breathing. What are they carrying? A new juice? No, the yellow scarved lady proffers a bottle of ‘Om Mahakumbh’ water, the same for which the pollution control board had to issue a warning that its faecal content is five times above permissible limits. When she uncaps the bottle to anoint me, I step back, then quickly hold my palm open lest she gets offended. A neighbour notes my reaction and smiles. Dariye mat, when the beautiful and the mighty are taking a dip for divine blessings, who are we?
True, who are we? Thought is the quality that orients a human mind as it negotiates the world. The reality. Only putting thoughts in words enables us to make and share memory, history, articulate the right from wrong. But of late we have spurned the activity of thinking with a vengeance. We love manufactured problems and shun real ones. From plate banging to taking dips in E-coli broths, we already are neck deep in a swamp of thoughtlessness. At this point in history, news without fact checks, interviews without questions, policy without data are the norm. Lines between myth and history, science and mumbo-jumbo, religion and politics are as blurred as they were in pre-modern times.
At this juncture, somehow AI feels like the perfectly tailored thing we didn’t need and never asked for. My maid’s daughter, who had managed to crossover to the other side of the digital divide, has been laid off from her job in a multimedia firm. Her expensive diploma is now redundant.
Watch: The Inequalities AI Is Creating – and Deepening – Around the World
No one has illustrated the connection between thought and writing more powerfully than Mario Levrero, also known as Uruguay’s Kafka. The protagonists of Levrero’s novels want to avoid thinking. They want to be constantly preoccupied so that they won’t need to reflect or dwell on the root causes of their inner turmoil. “I wanted something to do. I hate inactivity, it makes me think.” In Levrero’s novel Empty Words, the protagonist is using “graphological self-therapy” (handwriting exercises) to turn his life around. He doesn’t want to think but believes that improving his handwriting will transform his personality, and he’ll catapult “blissfully into a life of happiness, joy, money, and success”. But he soon realises that “not thinking” is impossible. The handwriting exercises “are becoming less calligraphical and more literary as time goes on. It seems that the function of writing depends entirely on meaning, on the thoughts behind the words. And you cannot think consciously only about thinking. Or write for the sake of writing without meaning being involved.”
Though there can be preverbal thoughts, Levrero’s novels bring out that the use of language and the act of thinking are inseparable.
The current generation is already reeling under smartphone addiction, social media dependence and alienation from reality. Ushering AI into schools and colleges seems like a terribly beguiling way to nip whatever thinking ability still remains in the bud.
Remember, when all these screens and devices took over, they professed that they would help people connect, build community and reduce loneliness.
Jiddu Krishnamurthy described the effects of thinking machines better than I ever can. He had already anticipated this in 1980s. “As smart machines and robots will take over our thinking, the brain will risk becoming lazy, unstimulated and frankly, bored…our experiences might dwindle to nothing more than a search for entertainment and pleasure…we would either surrender to nonstop amusement and slip into a zombielike existence or we decide to keep minds sharp by waking to a higher consciousness.” Krishnamurthy doubts that a brain conditioned to mechanical thought could suddenly spring to life without serious effort.
The ability to think, struggle and make meaning is our very humanity. The habit of thinking critically and imaginatively is closely related to the writing habit, an idea Levrero played with in his novel. What we call wisdom or conviction of thought develops by staying immersed in difficult stuff. By struggling long enough with the unknown and the imperfect so that the mind intuits the essential nature of a problem. This is also what makes human beings free, whatever be the circumstances. Mahatma Gandhi led India to freedom because of techniques and wisdom gained from his lived experiments with truth.
Where will voices and words curated by the latest AI model lead us? By handing over our already much colonised consciousness to an AI-powered, thought-free heaven, won’t AI and its billionaire owners be free to invade our minds with profit maximising imagery, entertaining and misinforming us to death? Haven’t we learnt enough from our experiments with social media and the big corporates behind them?
If in the coming decades AI guzzles all the research, all the novels, the poetry, the plays and throws back at us machine-synthesised remixes that the partly illiterate population cannot tell from real work, won’t we be locked in a never-ending Hall of Mirrors? In a world already veering towards oligarchic, majoritarian muscle, what kind of mirrors will these be? Whose reality will they mirror? When all the thinking has been termed routine and outsourced to AI, who will remain to question the reality churned out by AI? Won’t the marginalised be further pushed back?
Which reminds me of what Hannah Arendt had said about evil.
During the trial Eichmann kept protesting that he didn’t have any personal enmity with the people he gassed. He was only following orders. I was at once reminded of Gemini’s protests about lying. “I do not have any personal beliefs or intentions, so I can’t lie.”
I was chilled.
It was during Eichmann’s trial that Arendt had the insight that Eichmann’s most notable characteristic was not ideological conviction or evil motivation, but plain thoughtlessness. In the court too, “he stuck to the cliched, conventional language that protects against reality and renders thinking unnecessary”. Arendt defined Eichmann’s thoughtlessness as a “failure of conscience”. And why go back so far? We are already witnessing this failure in Palestine, by the very people who should have known better.
Arendt says that ideological thoughtlessness on a grand scale is enabled and sustained by everyday thoughtlessness which short circuits thinking or makes thinking unnecessary, because some things like family, nation, religion, culture cannot be questioned.
The ability to tell the right from wrong depends on the faculty of thought. Only thinking “disrupts the blanket of complacency” woven with “stock phrases and cliches” and makes us criticise the terms of everyday life that enable us to participate in Evil. Only by practising thoughtfulness in ordinary times can we keep our conscience in dark times. Krishnamurthy likewise had rung the alarm bell. “In an AI ruled world, any human ability we neglect will shrivel up. Including morality.”
In Joseph Conrad’s Heart of Darkness, Marlowe in search of Kurtz steers a crippled steamboat down a sluggish river and describes the unearthly feeling that the landscape inspires, “…cut off from the comprehension of our surroundings; we glided past like phantoms, wondering, secretly appalled, as sane men would be before an enthusiastic breakout in a madhouse. We could not understand a thing because we were travelling in the dark night of the first ages…”
AI’s designs to relieve human minds of all burden of language and thought generates a similar unreal feeling.
Varsha Tiwary is a Delhi based writer and translator. She has recently published 1990, Aramganj a translation of the best-selling Hindi novel Rambhakt Rangbaz.
