The Human Side of AI Evaluation

Organizations developing and deploying generative AI (genAI) expect that users do not all interact with AI systems in the same way. Differences in trust and engagement can influence how people adopt AI, evaluate its outputs, and rely on it to support decisions.

In a new study published in Communications in Computer and Information Science, "Putting the User First: A User-Centered Approach to AI System Evaluation," Exponent's Nichole Breeland, Jake Whritner, Christine Yu, Aarit Ahuja, and Rachel Kelly examine how user attitudes influence interactions with genAI systems. Building on prior research, the authors proposed and evaluated a four-persona framework to characterize AI users. Their results, however, supported that AI users are better characterized by two independent dimensions: AI engagement (i.e., extent to which someone is enthusiastic about genAI and/or actively uses it) and AI scrutiny (i.e., extent to which someone demonstrates critical evaluation and caution of genAI).

Based on survey responses from 500 participants, the authors used these dimensions to further develop the four-persona framework to include the: 

1. Principled Skeptic: Characterized by low engagement and high scrutiny
2. Indifferent Non-User: Characterized by low engagement and low scrutiny
3. Critical Adopter: Characterized by high engagement and high scrutiny
4. Uncritical Adopter: Characterized by high engagement and low scrutiny

The study found clear differences in how the four AI user groups interact with and perceive genAI. Critical and Uncritical Adopters reported the highest levels of trust and reliance, perceptions of transparency and explainability, and competence and self-efficacy, while Non-Users and Skeptics scored lower on these measures. However, users characterized by higher scrutiny — including Critical Adopters and Principled Skeptics — were significantly more likely to fact-check AI-generated information before relying on it.

The study highlights important differences in how people engage with and evaluate AI systems. Recognizing these differences can help organizations develop AI systems, improve user experiences, tailor onboarding and training, and consider transparency mechanisms for different types of users. Our human factors and data science experts can translate these differences into actionable strategies for companies designing AI systems, tailoring onboarding and training, and implementing transparency and explainability mechanisms aimed to support the needs of different user groups.

From the publication: "Engagement and scrutiny operate independently and jointly shape user behavior and perceptions."