Psoriasis is a common chronic inflammatory skin disease affecting millions worldwide, characterized by erythema, scaling, and itching. Despite advancements in treatments—including topical medications, phototherapy, and biologics—patients still face reduced quality of life and low treatment satisfaction. Studies show that over 50% of moderate-to-severe psoriasis patients report poor disease control [1], impacting both physical health and psychosocial functioning. Effective psoriasis management requires collaborative doctor-patient efforts, involving complex knowledge about etiology, recurrence mechanisms, treatment options, and lifestyle adjustments [2].

Patient education is crucial in psoriasis management. Adequate understanding of the disease, triggers, and treatment options can significantly improve adherence and self-management, enhancing quality of life. However, traditional education methods, such as in-person consultations or printed materials, are time-consuming and burdensome for clinicians [3]. With the rise of digital health, patients increasingly seek medical information online, further straining healthcare providers [4].

Recent advances in artificial intelligence (AI) offer new possibilities for medical communication [5]. Large language models, as advanced natural language processing tools, can generate human-like responses and are widely used in education, customer service, and medical consultations [6]. In healthcare, LLMs hold promise for delivering quick, accessible information, particularly in non-English contexts like Chinese. However, their application requires rigorous evaluation to ensure accuracy, reliability, and safety [7].

This study evaluates four mainstream Chinese closed-source LLMs in addressing common psoriasis patient queries. By systematically analyzing response quality, we aim to guide clinicians and patients in using AI tools and inform future research and development.

Four mainstream Chinese closed-source LLMs were selected: DeepSeek-R1 [8], DeepSeek-V3 [9], Qwen-3 [10], and GLM-4 [11]. These models were chosen for their strong performance in Chinese natural language processing tasks and widespread use.

Nine dermatologists from multiple hospitals contributed 40 high-frequency psoriasis-related questions. Each expert provided up to 10 questions, ensuring coverage of five key domains:

All questions were formulated in Chinese, reflecting real-world patient inquiries, and were reviewed for relevance and representativeness.

The 40 questions were input into the four LLMs, and responses were recorded for evaluation.

Nine dermatologists independently assessed response quality using a 10-point Likert scale (1 = lowest, 10 = highest). Criteria included:

Experts also noted deviations from evidence-based guidelines and whether responses included recommendations to consult a physician.

This multicenter expert consensus study represents the first comprehensive evaluation of Chinese large language models in the context of psoriasis patient education. The findings reveal a dual narrative of notable promise alongside critical limitations, offering valuable insights for both clinical practice and AI development. The observed performance variation across models—with mean scores ranging from 5.95 to 9.88—clearly demonstrates that not all commercially available LLMs are equally suited for specialized medical consultation. Among the evaluated models, Qwen-3 emerged as the top performer, achieving a mean score of 9.12. This success underscores the technical feasibility of developing LLMs capable of delivering high-quality, patient-centered responses. In contrast, GLM-4 exhibited significant inconsistency in its responses, with a standard deviation of up to 3.05, highlighting the ongoing challenges in ensuring clinical reliability across diverse medical queries.

A key strength identified in this study was the universal adherence of all models to fundamental safety protocols. Notably, every evaluated LLM successfully avoided generating dangerous or misleading content, and an overwhelming majority (87.5%) proactively emphasized the necessity of consulting a physician. This represents a substantial improvement over traditional online health information sources, which often lack rigorous oversight and may propagate misinformation. Furthermore, the models excelled in providing clear, structured explanations for foundational topics such as the side effects of topical steroids. By distilling complex medical concepts into patient-friendly language, these LLMs demonstrated their potential to serve as valuable educational tools. Such capabilities could significantly reduce the burden on clinicians by addressing routine patient inquiries, allowing healthcare providers to allocate more time to complex cases and personalized care.

Despite these strengths, the study also uncovered persistent gaps in the models' performance, particularly in specialized areas of psoriasis management. Approximately 12.5% of responses deviated from evidence-based guidelines, with notable inaccuracies arising in discussions of biologics' mechanisms of action, combination therapy sequencing, and complex management scenarios. These inconsistencies likely stem from inadequate domain-specific training data and insufficient clinical validation processes. While the models performed well in identifying common psoriasis triggers—such as stress and infections—their management advice often lacked depth and personalization. The tendency to provide generalized responses represents a critical limitation, as it fails to account for the heterogeneous nature of psoriasis, which manifests in diverse phenotypes and requires tailored treatment approaches.

These findings resonate with broader observations in the literature regarding the limitations of LLMs in medical applications. Prior studies have noted that current models often struggle with context-dependent reasoning, a capability essential for managing chronic conditions like psoriasis, where individualized care is paramount. Our study extends these observations to the Chinese-language dermatology context, where linguistic nuances and regional treatment preferences introduce additional layers of complexity. For instance, while the models occasionally referenced traditional Chinese medicine, these mentions were not systematically integrated into evidence-based recommendations, reflecting a gap in culturally relevant training data.

In summary, this study highlights both the potential and the current limitations of Chinese LLMs in psoriasis patient education. While they excel in safety and foundational knowledge dissemination, their performance in specialized and personalized care remains inconsistent. Addressing these gaps will require enhanced collaboration between AI developers and clinical experts, as well as the incorporation of more robust, domain-specific training datasets. Until these improvements are realized, LLMs should be viewed as supplementary tools—valuable for basic education but insufficient for guiding complex treatment decisions without clinician oversight.

Chinese LLMs demonstrate substantial potential as supplementary tools for psoriasis patient education, particularly in improving health literacy regarding disease fundamentals, medication safety, and basic self-management. Qwen-3 currently delivers the most clinically reliable performance among evaluated models. However, significant limitations persist in handling specialized therapeutic areas (notably biologics and complex management scenarios) and providing individualized recommendations.

Key implications for practice:

Future development should focus on enhancing evidence-based reasoning through curated dermatology datasets, implementing specialist review protocols, and developing hybrid human-AI education systems. Until these advancements are realized, LLMs should be positioned strictly as informational supplements for clinician-guided psoriasis education.