Comparing Psychometric Properties of Expert-Developed and AI-Generated Personality Scales
Lydia Oeljeklaus, Stefan Höft, Daniel Danner
Source record
Source: Crossref
Published: Aug 1, 2025
DOI: 10.1027/2698-1866/a000095
Open original source ↗Source abstract
Abstract: Artificial intelligence (AI) techniques such as natural language processing can be used to construct personality scales. The present study compares the psychometric properties of scales in the domains of Emotional Stability and Conscientiousness developed by experts with those of scales in these domains that were generated by the AI chatbot ChatGPT. We conducted two online studies ( N = 155 and N = 459) in which respondents completed inter alia the expert-developed and the AI-generated scales. Results show that all four scales demonstrated similar reliability and similar content, construct, and criterion validity, but differences regarding fit with a latent measurement model and measurement invariance across genders. We discuss the implications of these findings for the integration of ChatGPT into the process of scale construction and give general recommendations for prompting ChatGPT for automatic item generation.
Evidence graph
No public relationships recorded yet.
Integrity note: This page is a factual metadata record created by deterministic ingestion. It is not a claim that the work moves a mathematical frontier or has been independently verified.