Despite their advanced capabilities, AI tools cannot fully comprehend sensory concepts like humans do, according to a new study led by The Ohio State University. Explore the study’s findings and their implications for AI development.
In a new study, researchers led by The Ohio State University revealed that artificial intelligence, no matter how advanced, struggles to understand sensory concepts the way humans do.
The study, published in the journal Nature Human Behaviour, underscores the intrinsic differences in how humans and AI perceive the world, indicating that AI’s current limitations stem from its reliance on language-based models.
Why AI Falls Short
The core of the issue, according to lead author Qihui Xu, a postdoctoral researcher in psychology at Ohio State, is that large language models (LLMs) focus primarily on language and occasional image data, which does not equip them with the rich sensory and motor experiences that humans possess.
“A large language model can’t smell a rose, touch the petals of a daisy or walk through a field of wildflowers,” Xu said in a news release. “Without those sensory and motor experiences, it can’t truly represent what a flower is in all its richness. The same is true of some other human concepts.”
The Research
Xu and her team compared the knowledge representation of 4,442 words between humans and advanced LLMs from OpenAI (GPT-3.5 and GPT-4) and Google (PaLM and Gemini). The researchers used two measures:
1. Glasgow Norms: This measure asked for ratings on nine dimensions such as arousal, concreteness and imageability.
2. Lancaster Norms: This evaluated how words relate to sensory information (such as touch, smell, vision) and motor activities.
The goal was to see how closely AI models aligned with human perceptions. In areas where emotions and sensory inputs were crucial, LLMs significantly trailed behind human understanding.
Findings and Implications
The study found that AI tools excelled in representing abstract concepts that don’t require sensory input. However, they fell short in comprehending words tied to sensory and motor experiences.
“From the intense aroma of a flower, the vivid silky touch when we caress petals, to the profound joy evoked, human representation of ‘flower’ binds these diverse experiences and interactions into a coherent category,” the researchers noted in their paper.
The findings suggest that the lack of sensory grounding in AI leads to a fundamental disconnect in how AI and humans understand the world.
Future Prospects
Despite these limitations, Xu remains optimistic about the evolution of AI. LLMs that incorporate image data performed better in understanding vision-related concepts, hinting at potential improvements.
According to Xu, future models augmented with sensor data and robotics may better grasp the physical world and perform more human-like inferences and actions.
“They obtain what they know by consuming vast amounts of text – orders of magnitude larger than what a human is exposed to in their entire lifetimes – and still can’t quite capture some concepts the way humans do,” Xu said. “The human experience is far richer than words alone can hold.”
Yet, with continued advancements, AI may close this gap over time.
The study sheds light on the nuanced challenges in AI development and emphasizes the unique and irreplaceable richness of human sensory experiences.
Contributors
The study co-authors are Yingying Peng, Ping Li and Minghua Wu of the Hong Kong Polytechnic University; Samuel Nastase of Princeton University; and Martin Chodorow of the City University of New York.
Source: The Ohio State University