Leveraging prompt-driven generative AI for systematic reviews in digital psychiatry: A stage-matched comparative proof-of-concept for healthcare researchers and clinicians

Source abstract

The scale and pace of evidence generation in digital psychiatry increasingly exceed the capacity of traditional systematic literature review (SLR) methods. Large language models (LLMs) are gaining traction in evidence synthesis, yet limited guidance exists for integrating generative AI into transparent, reproducible SLR workflows. To develop and evaluate a prompt-driven, modular decision-making framework for adaptable SLR workflows in digital psychiatry, and to compare stage-matched performance relative to a consensus-based human reference process. We conducted a mixed-methods, stage-matched comparative evaluation of three GPT-5 mode variants (Auto, Agent, and Deep Research) against a consensus-based human-led SLR workflow using a registered digital psychiatry review (PROSPERO CRD42025648122) as a case example. Nine SLR tasks were implemented within the ChatGPT interface using structured RISEN prompts: preliminary searches; research question, eligibility criteria, and search strategy development; article screening; data extraction; article summarization; critical appraisals; and descriptive results synthesis. Performance was evaluated using task-specific assessments of accuracy, sensitivity, specificity, inter-rater agreement, reporting compliance, hallucination monitoring, and workflow feasibility. All GPT-5 mode variants were feasible across SLR stages, although performance varied by task and mode. No fabricated study-level content was identified during structured hallucination auditing. Auto mode performed best for structured rule-based tasks requiring efficiency and implementation-ready outputs (e.g., eligibility criteria). Agent mode excelled in conceptual integration and interpretive tasks (e.g., research question generation, critical appraisals, descriptive synthesis). Deep Research mode most closely approximated human reasoning for higher-order synthesis tasks, performing best in full-text article screening, article summarization, and narrative synthesis. Prompt-driven LLM workflows are feasible and semi-efficacious for selected SLR tasks in digital psychiatry when deployed within a modular, human-in-the-loop decision-support framework. Findings support task-specific deployment of an adaptable modular framework, enabling healthcare researchers and clinicians to modify LLM workflows according to methodological appropriateness, confidence in performance, and institutional review practices.