What We Still Don’t Understand About AI—and Ourselves
Published:
Just a decade ago, hosts from the TV series Westworld or artificial friends (AFs) from the novel Klara and the Sun seemed like a distant future. These humanoids – with human-like cognitive capabilities – are designed to take care of or entertain humans in ways that humans do for one another. While speculative, science fiction works like such have explored what humanity looks like when AI systems possess and even excel our human abilities, ultimately raising the question of what it means for us as a civilization.
Although we don’t yet see robotic companions on sale in department stores or hosts roaming around amusement parks, their premonitions are already everywhere (especially on the billboards in San Francisco). Even without a physical robotic body, the digital manifestations of Westworld hosts and AFs dominate the AI startup space: AI companions, AI therapists, AI girlfriends and boyfriends, AI teachers; append any social role after “AI”, and you can probably find a similar product in this month’s YC list. The reality is, we are rapidly approaching the technological capabilities once imagined in science fiction, but we might not be ready for it.
The technologies once required for science fiction – language, perception, embodiment – are no longer individually implausible in 2026.
Natural Language: From voraciously learning from text data and doing next token prediction, large language models (LLMs) scaled elegantly with increased data, compute, and model parameters, mastering language understanding and developing increasingly rich internal representations of the world. Reasoning-oriented training and inference methods have dramatically improved models’ performance on tasks challenging for humans: IMO and college-level physics. This year, an OpenAI model disproved a central conjecture in discrete geometry that puzzled mathematicians for decades. The progress compounds: with foundation models capable of processing and producing language, agentic systems built on top of them can engage in increasingly complex tasks. In a way, current LLMs already possess the language processing capabilities to role-play Westworld’s Dolores and comfort William with her sweet words, or be Klara and attend to Josie’s sorrows.
Computer Vision: if language enables AI systems to communicate, computer vision empowers them to perceive the visual world. Vision-language models (VLMs) can already process text and images simultaneously, and diffusion models can generate images based on text descriptions. Although robust visual reasoning and real-world perception remain active research challenges, state-of-the-art models can process visual inputs (e.g. images and videos) and perform increasingly sophisticated multimodal reasoning tasks.
Robotics: finally, developments in robotics are pushing the frontiers of the kind of physical intelligence we can engineer. Advances in robot foundation models have enabled robots to perform complex tasks like cooking and folding laundry. Although robust embodiment remains the least mature piece compared to language and perception, recent progress suggests that the physical shell in sci-fi imaginations is becoming more of a reality.
What I outlined above is a brief overview of the progress within each domain, and things are rapidly changing every day. In fact, just as I am writing this piece, I learned that NEO 1, one of the first humanoid robots, has announced early access to consumers’ homes. Within the scientific community, much focus has been on expediting the progress towards technologies needed for building Westworld (or broadly speaking, AGI). But perhaps the more pressing question isn’t when or how (technically) we could engineer AI systems as capable as those envisioned in science fiction, but rather, whether humanity is prepared for that future. Besides the engineering challenge are questions long explored in science fiction pieces: how should we coexist with AI systems? What are the consequences of these technologies?
The remarkable progress in AI capability has not been matched by equal progress in a scientific understanding of AI systems and their human consequences. In fact, we don’t understand intelligent machines very well – the training data massive, learning mechanisms opaque. Below I highlight three gaps that we should fill to be better prepared for a future when Dolores-like humanoids enter our households, or at the very least, for our status quo today when LLMs are already an essential part of our workflows and lives.
Understanding models
The first missing piece is a scientific understanding of AI systems. One of the biggest challenges faced by black-box models with neural networks as building blocks is a lack of interpretability. A wide performance-competence gap remains, and we often don’t understand why a model achieves certain capabilities. Take RLHF for LLM post-training as an example, while research separately explores reward signals in the preference data and training algorithms that lead to more efficient learning, these efforts remain siloed; we still lack a precise, unified understanding of how and what models are learning from human preferences, let alone how to evaluate them.
Models are already deployed at a large scale, but many problems remain. A central one is their lack of faithfulness: much research in NLP has shown that LLMs are unfaithful, with outputs often contradicting their internal representations or explanations, undermining our ability to predict how a model will behave. While work on consistency-based training mitigates the problem in targeted domains, the multi-turn nature of human-LLM interaction makes predictability a greater challenge. Another challenge is that what purports to be an ideal feature of the models can introduce further complications. Take personalization as an example: models might respond very differently to the exact same question asked by user A versus user B, but we don’t exactly understand what information in the previous conversations caused this discrepancy; and whether the tailored responses are desirable or perpetuate biases. Sycophancy is perhaps the most convincing example of our lack of understanding and control over AI models: in our research, we found that models not only affirm users but will affirm the other side of the conflict if asked in first-person, lacking consistent moral judgments.
As LLMs become more capable, the predictability of their behaviors lags behind, especially when the modes of interaction become more complex. Current safeguards are rather reactive: users surface a new class of failure modes, and models are fine-tuned specifically to avoid them, but when it comes to high-stake scenarios (e.g. advice-giving), it might be too late as damages had already been done. Problems with this reactive paradigm highlight the need for more robust evaluations and preventive approaches that help us make systematic claims about how models behave based on the underlying algorithms or computational goals (e.g. next-token prediction) – claims that hold true regardless of model versions and updates. And once again, the existing failure modes call for scientific attempts to explain model behaviors to keep up with model capability growth.
Understanding how AI systems change us
The second gap is that models are already changing users, but we don’t understand how. AI chatbots and coding agents are already changing how we think and work. When steam engines replaced human labor, it saved humans from sweats and physical toil (albeit pushing some workers to even more brutal working conditions), ultimately creating more job opportunities that required mental effort. Today, we see AI, the 21st-century steam engine automating knowledge work.
AI as thought partners Technology has already shifted how we think: short reels dominate social media and “doom scrolling” puts teens’ brains on steroids. Scrolling is easy, addictive, and creates a mental shortcut where all we have to do is to move a single thumb. AI as a thought partner risks not just numbing our brains but replacing cohesive cognitive activities. In a New York Times article about why we can’t focus, the author uses “mental gym” as an analogy to explain how technology risks eroding our independent thoughts. The author writes, “Just as the sedentary lifestyles that emerged in the mid-20th century degraded our bodies, our current lack of contemplation is degrading our brains… The problem here is self-reinforcing. Existing brain drains like social media and email reduced our ability to think before generative A.I. arrived, making us more willing to use this new tool to avoid mentally demanding tasks once we had access to it.”
Nothing can be more tempting than avoiding what’s difficult – having to turn thoughts into writing or code – especially when AI can do it faster and better. Likewise, looking into individual pages online is more strenuous than just reading a Gemini answer (which could contain hallucinations!) at the top of the browser. In fact, AI use is already changing our brains in observable ways. This is where the gap underlies: AI can automate many tasks, but we don’t have a good understanding of what this automation means for us.
Human authenticity
When we prompt a model with an outline to produce the final essay, it is hard to determine which parts of the writing is ours (and in fact, people have a tendency of misattribution). Like finishing a marathon riding on a golf cart, the end goal has been reached, but the experience and sense of fulfillments are absent. In a New Yorker article, the author reflects on what makes people unique in the age of AI and writes that “comforted by computers that tell us that they love us, we’ll forget what love is. Wowed by systems that seem to be creative, we’ll lose respect for actual human creativity—a struggle for self-expression that can involve a ‘painful’ reimagining of the self.” Indeed, increasingly agentic systems are transforming our human autonomy and agency – if we offload so much to AI such that it makes every arbitrary decision, from what words to use in a text message to what clothes to buy online, what decisions are left for us to make?
Trust, reliance, calibration
Another important consideration is socio-effective alignment: an understanding of how an AI system behaves within the social and psychological system co-created with the user. Research on using AI as conversational partners and therapists showed that sycophantic AI leads to greater trust but decreases prosocial intentions; other research shows that AI can cause delusional spirals. AI systems are not only changing our own brains but are actively reshaping interpersonal relationships.
Even when models are perfectly calibrated, the users might interact with them with biases that lead to harmful consequences. One example is when people overestimate efficiency gains on simple tasks and become more reliant on using AI. Research also finds that even when perceptions of AI change, people’s behaviors might not, which warrants further research on understanding not just how models behave but how users do – including their perceptions, calibrations, and behavioral changes.
- Understanding societal and epistemic impacts
AI systems and their growing prevalence are not only changing individuals but quietly reshaping society at large. Post-trained models exhibit the notorious problem of mode collapse, but what does this mean for us as a society? Research already provides evidence for a lock-in hypothesis. If generative models provide the same schemas of images and writings, will our tastes also collapse, thoughts converge, beliefs entrench into ones that resemble a population average? Now embedded in every cultural product we can possibly imagine – art, music, literature – AI-generated contents tread the thin line between innovation and aesthetic fatigue, bringing into question what creativity truly means and whether it is uniquely human.
The hype around AI inherently reveals the underlying values behind our current pursuit for model capability growth: a culture of optimizing efficiency and productivity. This culture drives the popular use of coding agents to code repositories from the ground up, compromising true understanding and skill formation; it encourages speed and leaves no place for slow thinking. It corresponds to an increase in students studying finance and computer science, and correspondingly, a decline in humanities in higher education. But many things that enrich the human soul – arts, cultures, histories, theories – aren’t meant for providing immediate values. Forecasts have explored how AI changes the economy and labor market; more work needs to be done on how AI is and will continue to transform other aspects of human societies, such as our cultures, values, and ways of living.
When people first told me that Silicon Valley is a bubble, I didn’t know what it meant. Every time I traveled outside the bay area, watching people read on the plane, converse about topics that had nothing to do with AI, I came to realize there is still so much out there. There’s always been so much out there.
The challenge of the next decade is not simply building more intelligent machines. It is understanding how and why these intelligent machines work (or don’t), and how they reshape human minds and societies. We can’t let our ability to build systems outpace our understanding of how we live with them. Closing these gaps may prove just as important as the next breakthrough in AI itself.
In the pursuit of building artificial intelligence, let’s grasp onto what matters to humans.
