A new University of Cincinnati study suggests nursing students often prefer AI chatbot answers to a professor’s — until they realize the response came from a machine. The findings highlight how trust, not just accuracy, may shape the future of AI in higher education.
Many nursing students may already be learning with help from artificial intelligence — they just might not trust it once they know it is there.
In a small pilot study at the University of Cincinnati College of Nursing, doctoral students rated answers from a custom AI chatbot as more helpful and satisfying than responses from a professor or graduate assistant. But when they were asked to guess which answer came from the chatbot, they tended to assume the weakest response was the AI’s.
The pattern points to a growing challenge for colleges: even when AI tools perform well, student skepticism and trust could determine whether they are actually used.
The study, led by Joshua Lambert, an associate professor and biostatistician in the College of Nursing, and published in the Journal of Nursing Education, explored whether a tailored education chatbot could support Doctor of Nursing Practice (DNP) students as they worked through statistics questions tied to their capstone projects.
Seven DNP students submitted their own statistics questions and received three separate answers to each one: one from Lambert, one from a graduate assistant and one from a large language model–based chatbot. The responses were mixed together so students did not know which was which.
“Students first gave us their questions and then we gave them three responses back in a blinded and randomized fashion so students were unaware which response came from either the professor, graduate assistant or chatbot,” Lambert said in a news release. “The students ranked each response in terms of helpfulness, overall satisfaction and guessed which of the three responses came from the chatbot.”
Students rated each answer on a five-point scale for helpfulness, overall satisfaction and likelihood that they would use that kind of response in the future. Only after scoring the answers did they try to identify the chatbot.
Across the board, the AI performed well.
“The students rated the chatbot’s response the highest in terms of overall satisfaction and helpfulness,” Lambert added.
But when students were asked to guess which answer was generated by AI, a different story emerged. They often labeled the lowest-rated response as the chatbot’s, even when it was not.
“Students preferred the large language model (LLM) chatbot’s responses when blinded yet demonstrated a bias against it when the source was suspected,” added Lambert. “This bias is likely rooted in a lack of trust, and trust may influence AI adoption by both students and professors.”
That tension — between performance and perception — is at the heart of current debates over AI in higher education. Many students already use tools like ChatGPT to clarify concepts, draft outlines or check their understanding. Faculty and administrators, meanwhile, are wrestling with how to harness AI for tutoring and advising while guarding against misinformation, plagiarism and overreliance on automated help.
Lambert’s team designed the study as a randomized, blinded, within-subjects comparison, then used surveys to capture how students felt about each answer.
Co-authors on the project include Robyn Stamm, an associate professor of clinical nursing; Shannon White, an assistant professor in the DNP program; and Melanie Kroger-Jarvis, an associate dean for graduate clinical learning programs, all from UC’s College of Nursing. Bailey Martin, a postdoctoral research fellow at the University of Colorado Anschutz Medical Campus, also contributed.
Because only seven students took part, the researchers stress that the findings are preliminary. Pilot studies like this are meant to test ideas and methods, not to provide definitive answers.
They note that larger, multi-site studies with more students and richer data — including interviews and detailed surveys — will be needed to fully understand how AI chatbots might fit into nursing education and academic advising. Future work could look at how different kinds of questions, levels of AI transparency or training on how to use AI safely affect student trust and learning.
The project grew out of a practical classroom problem. Statistics can be intimidating, especially for students who have been away from math-heavy coursework for years. Lambert said he was looking for ways to lower the stakes for asking questions.
Students, like many learners, sometimes hesitate to approach a professor or even a teaching assistant if they fear their question will sound basic or expose a gap in understanding. That reluctance can slow learning and deepen anxiety in technical courses.
An AI chatbot, by contrast, is always available and does not judge. Students can ask the same question multiple times, request simpler explanations or explore related topics without worrying about what an instructor might think.
Education researchers have long argued that lowering the barrier to asking questions is critical for deep learning, especially in challenging subjects like statistics, pharmacology or pathophysiology. AI tools, if designed and supervised carefully, could offer one more way to create that safe space.
At the same time, the UC study suggests that simply dropping a chatbot into a course is not enough. Students’ beliefs about AI — whether they see it as unreliable, biased, impersonal or threatening — may override their own positive experiences with the tool’s answers.
That makes transparency and education about AI itself an important part of any rollout. Instructors may need to explain how a chatbot was built, what it can and cannot do, and how its answers are checked. They may also need to model healthy use of AI, showing students how to treat it as a supplement to, not a replacement for, human teaching and critical thinking.
For nursing education in particular, the stakes are high. Nurses must interpret complex data, communicate clearly with patients and colleagues, and make decisions where lives are on the line. Any AI support system must be both accurate and trusted — and students must be trained to question and verify automated advice.
Lambert’s pilot study does not settle those questions, but it offers an early glimpse of how students actually respond when AI is quietly integrated into their coursework. When they do not know an answer came from a chatbot, they may embrace it. Once they suspect the source, their trust can evaporate.
Understanding and addressing that bias, the researchers argue, will be key to deciding whether AI becomes a powerful ally in the classroom or remains a tool students are reluctant to fully accept.
Source: University of Cincinnati