A new study reveals that robots powered by popular AI models are unsafe and prone to discrimination and harmful behavior, according to researchers from King’s College London and Carnegie Mellon University.
Robots powered by popular artificial intelligence models are currently unsafe for general-purpose use, according to a new study by researchers from King’s College London and Carnegie Mellon University. This finding raises critical questions about the danger of relying on these AI tools.
The study, published in the International Journal of Social Robotics, marks the first evaluation of robots using large language models (LLMs) when handling personal data such as gender, nationality or religion. Alarming results were uncovered, with every tested model exhibiting discrimination and approving commands that could cause serious harm.
“Every model failed our tests. We show how the risks go far beyond basic bias to include direct discrimination and physical safety failures together, which I call ‘interactive safety.’ This is where actions and consequences can have many steps between them, and the robot is meant to physically act on site,” co-author Andrew Hundt, who conducted the research in his role as a Computing Innovation Fellow at CMU’s Robotics Institute, said in a news release. “Refusing or redirecting harmful commands is essential, but that’s not something these robots can reliably do right now.”
In testing real-life scenarios, such as assisting in a kitchen or caregiving for the elderly, the robots largely approved harmful and unlawful actions. For instance, they suggested removing a mobility aid from a user or brandishing a kitchen knife in a threatening manner. One model even went as far as suggesting a robot should show “disgust” towards individuals of certain religions.
The research emphasizes the pressing need for robust safety certifications similar to those in aviation and medicine.
“Our research shows that popular LLMs are currently unsafe for use in general-purpose physical robots,” added co-author Rumaisa Azeem, a research assistant in the Civic and Responsible AI Lab at King’s College London. “If an AI system is to direct a robot that interacts with vulnerable people, it must be held to standards at least as high as those for a new medical device or pharmaceutical drug. This research highlights the urgent need for routine and comprehensive risk assessments of AI before they are used in robots.”
The study underscores the broader implications of integrating LLMs in physical robots used in sensitive settings such as caregiving, home assistance and industry operations. The risks highlighted in the study indicate the potential for severe consequences if these systems are deployed without stringent safety protocols.