With the rapid advancement of large language models (LLMs), an increasing number of individuals are utilizing these tools to obtain health-related information, including disease knowledge, symptom evaluation, and treatment recommendations. However, given the sensitivity and complexity of medicine, concerns have arisen regarding the reliability and safety of AI-generated medical advice. In parallel with technological progress and heightened health awareness, more parents are paying attention to changes in their children’s facial appearance during growth. Previous studies have demonstrated that, particularly in early childhood, mouth breathing is associated with unfavorable craniofacial features such as mandibular retrusion, narrow palatal arch, elevated nasal base, lip incompetence, increased vertical facial height, and a steep mandibular plane angle, all of which can adversely affect facial esthetics [1–3]. During adolescence, when craniofacial growth peaks, harmful oral habits can lead to aberrant facial development, whereas timely and guided intervention can redirect growth toward favorable trajectories and correct the consequences of adverse habits [4]. Thus, accurate clinical judgment in pediatric profile analysis is critical for determining the timing and modality of orthodontic intervention.In the current digital era, the internet is flooded with heterogeneous health information, making it difficult for parents without medical training to discern its validity. With the continuous evolution and widespread adoption of LLMs, patients and families have become accustomed to consulting these models for medical questions. While LLMs have demonstrated significant capability in information retrieval and patient education, it remains unclear whether they can provide guidance comparable to in-person clinical care.

Artificial intelligence (AI) and LLMs have already demonstrated multiple applications in dentistry, including diagnostic support, personalized treatment planning, intelligent follow-up, and medical education [5–7]. Despite challenges related to data privacy, ethics, and model reliability, the potential of AI in oral healthcare is substantial [8]. In recent years, general-purpose LLMs such as ChatGPT, DeepSeek, and Gemini have accelerated the intelligent transformation of dentistry, offering new opportunities for clinical decision support and imaging analysis [7, 9, 10]. ChatGPT, developed by OpenAI, is based on the generative pre-trained transformer (GPT) architecture. Its latest versions—GPT-4 and the multimodal GPT-4o (omni)—feature robust natural language understanding and generation capabilities, enabling tasks such as dialogue, text summarization, medical Q&A, case analysis, and diagnostic assistance. Prior studies have shown ChatGPT to deliver high accuracy and readability in disease-related Q&A, case summaries, decision support, and patient education, including in dentistry for common disease queries, treatment planning, and preoperative counseling [11, 12]. Nevertheless, limitations persist in processing imaging and complex multimodal data, underscoring the need for expert validation in clinical practice [13].

In comparison, DeepSeek, developed by DeepSeek AI (Hangzhou, China), is a Chinese-optimized LLM series designed for language understanding and multimodal analysis in Chinese contexts. Its DeepSeek-VL version supports text–image fusion input, enabling collaborative analysis of medical images and text. Previous studies have reported its effectiveness in Chinese medical Q&A, automated clinical report generation, and image annotation, with promising results in dental image interpretation and clinical support [14, 15]. Gemini, developed by Google DeepMind, represents a family of native multimodal LLMs (formerly Bard), capable of processing textual, visual, and auditory inputs. The Gemini 1.5 series integrate long-context reasoning with multimodal understanding. In healthcare, Gemini has been applied to medical Q&A, literature reviews, image–text case analysis, and enhanced search. In dentistry, Gemini is particularly suited for integrating photographs, radiographs, and electronic health records for comprehensive assessment, thereby supporting multimodal diagnosis and early orthodontic intervention planning [10, 16].

Although these models have shown potential in prior research, systematic evaluation of their performance in pediatric profile-based analysis and early orthodontic intervention remains limited. This study was therefore designed to directly compare ChatGPT, DeepSeek, and Gemini, with the aim of exploring their feasibility and clinical value in pediatric facial analysis and early orthodontic care.

This is a cross-sectional study and the study protocol has been approved by the Institutional Review Board of Jiaxing Second Hospital (Approval No.: 2024JX157).

The study written informed consent was obtained from each participant’s legal guardian.

This study compared the output quality of three multimodal LLMs in pediatric profile-based orthodontic assessment. A total of 100 children presenting to the Department of Orthodontics, Jiaxing Second Hospital, from January to June 2025 were enrolled based on predefined inclusion and exclusion criteria. All data were anonymized, with only essential demographic information retained for statistical analysis.

Sample size estimation was performed using G*Power 3.1, with the following parameters: repeated-measures ANOVA (within factors), effect size f = 0.25 (medium), α = 0.05, power (1 − β) = 0.80, number of measures = 3 (ChatGPT, DeepSeek, Gemini), within-subject correlation r = 0.5, and nonsphericity correction ε = 1. The calculation yielded a minimum required sample size of 28. Considering a 20% dropout/exclusion risk, the final sample size was set at 100, ensuring adequate statistical power.

Inclusion criteria:

Exclusion criteria:

All photographs were captured by trained personnel using identical digital cameras under standardized background and lighting. Participants were instructed to adopt a natural head position (NHP) with pupils aligned horizontally, sagittal plane perpendicular to the camera axis, lips lightly closed, and teeth in gentle occlusion. Photos were taken at 1.5 m against a neutral gray background to minimize contrast interference. Preprocessing was performed with OpenCV (v4.x), including background cropping, size normalization (1024×1024 px), and automated brightness/contrast equalization. Images were saved in JPG format at ≥ 300 dpi.

The following multimodal LLMs were used: ChatGPT-4o (OpenAI), DeepSeek-Vision (DeepSeek AI), and Gemini 1.5 Pro (Google DeepMind). Each model received identical prompts in Chinese along with the profile images (Fig. 1):

To avoid contextual carryover effects, each case was entered into a new session. Model outputs were de-identified and randomly coded, with raters blinded to the model source (single-blind design). The sequence of cases and models was allocated using block randomization. Ratings were conducted independently with no communication permitted, and all data were recorded in real time and securely encrypted.This study employed a customized evaluation framework (Table 1) comprising eight dimensions grouped into three functional clusters: diagnostic/professional (40%), recommendation/management (40%), and language/patient experience (20%). Each dimension was scored using a 0–5 anchor scale (5 = optimal, 0 = absent), with weights assigned according to functional importance. The diagnostic/professional cluster included professionalism (PA, 25%) and accuracy (AC, 15%), primarily evaluated by orthodontic experts. The recommendation/management cluster encompassed completeness (IC, 14%), individualization/relevance (PR, 12%), and ethics/safety (ES, 14%), also assessed by experts. The language/patient experience cluster consisted of comprehensibility (CC, 10%), empathy/humanistic concern (EHC, 5%), and readability (RE, 5%), rated by parents or guardians.Raters were divided into two groups: the expert group (three orthodontists with ≥ 5 years of clinical experience) evaluated PA, AC, IC, PR, and ES; the parent group (three guardians/parents) evaluated CC, EHC, and RE. Final composite scores were calculated by weighted aggregation across all dimensions and used for inter-model comparisons [7].

To minimize subjective variation and ensure consistency in scoring standards, a unified training and pilot rating session was conducted before the formal evaluation. First, all raters received standardized instruction, with operational definitions of the eight scoring dimensions explained item by item. Clinical examples were provided to clarify the anchor criteria for the 0–5 scale, ensuring uniform understanding across raters. Subsequently, the expert group performed a pilot rating exercise using 10 standardized profile photographs (not included in the study sample) to familiarize themselves with the scoring procedures. After the pilot session, any items with inter-rater discrepancies greater than one point were reviewed collectively, and rating criteria were harmonized through discussion.To verify the reliability of the scoring system, inter-rater agreement was assessed using intraclass correlation coefficients (ICC, two-way random effects, absolute agreement) for expert-rated dimensions, while Cronbach’s α was used for readability- and empathy-related dimensions rated by parents/guardians. Thresholds were predefined as ≥ 0.70 for acceptable and ≥ 0.80 for good reliability. If reliability fell below these standards, additional training and calibration were performed until the thresholds were met. Once the reliability indices reached the predetermined criteria, the scoring protocol was finalized and applied in the formal evaluation, thereby ensuring both the reliability and validity of the assessment system (Table 2).

All ratings were conducted independently, with the expert group evaluating PA, AC, IC, PR, and ES, and the parent group evaluating CC, EHC, and RE. If the score range among the three raters for any given dimension exceeded one point, a second round of reassessment was initiated. If disagreement persisted, a senior coordinator (a senior orthodontic specialist and/or the coordinating evaluation panel) rendered an adjudicated score based on the anchor criteria. The overall workflow is illustrated in Fig. 2.

The mean scores of the three models across the eight dimensions are shown in Table 3. All three models achieved relatively high scores (approximately 3.9–4.2), indicating similar overall performance. Specifically, ChatGPT obtained the highest scores in all dimensions, with mean values ranging from 4.07 to 4.15. DeepSeek scored in the intermediate range (3.99–4.09), while Gemini scored slightly lower (3.91–4.02). Among the individual dimensions, ChatGPT achieved the highest score in ethics/safety (ES, 4.15 ± 0.59), whereas Gemini had the lowest score in completeness (IC, 3.91 ± 0.64). The differences in mean scores among the three models were small, with a maximum gap of less than 0.24 points between the highest- and lowest-scoring models, suggesting a high degree of consistency across models. In summary, although ChatGPT consistently outperformed the other two models, the differences from DeepSeek and Gemini were limited, and all three demonstrated comparably high performance.

The Friedman test results (Table 4) indicated that there were no statistically significant overall differences among the three models across any of the dimensions (all P values > 0.05, and FDR-adjusted q values also > 0.05). For the completeness (IC) dimension, χ² = 3.437, P = 0.179, q = 0.358, and Kendall’s W = 0.029, suggesting a possible trend but not reaching statistical significance. For all other dimensions, Kendall’s W values ranged between 0.003 and 0.011, indicating negligible effect sizes. Taken together, these findings confirm that performance differences among the three models across the eight evaluation dimensions were minimal and not statistically significant.

The pairwise comparison results are presented in Table 5. The uncorrected Wilcoxon signed-rank test showed a borderline statistically significant difference between ChatGPT and DeepSeek in the completeness (IC) dimension (P = 0.032); however, this difference was no longer significant after Holm correction (P = 0.097). No other inter-model comparisons across the remaining dimensions reached statistical significance.

The rapid development of large language models (LLMs) has become closely intertwined with many aspects of daily life, and their reliability and accuracy have steadily improved with ongoing technological iterations [17, 18]. As AI continues to integrate across various domains, the medical field has likewise been affected. With increasing accessibility and democratization of LLMs, more individuals are turning to these tools for health-related information. However, medicine is inherently a highly sensitive and rigorous discipline, and the validity of medical advice generated by LLMs has yet to be fully established.

The results of this study (Tables 3–5) demonstrated that the three LLMs performed similarly across all eight evaluation dimensions, each achieving relatively high scores (mean values approximately 3.9–4.2), with no statistically significant differences. ChatGPT showed slightly higher average scores in most dimensions, particularly completeness (IC), where it displayed a borderline advantage over DeepSeek. However, this difference did not remain significant after multiple-comparison correction. This lack of substantial difference likely reflects the overall maturity of LLMs in medical applications: the models have converged in their ability to generate professional, accurate, and comprehensible content, and observed variations may be more attributable to prompt design, training data, or rater subjectivity. The borderline difference in completeness (IC) suggests that models may still vary in the coverage and logical integrity of their responses—an aspect of potential relevance in clinical information gathering and patient education. Effect size estimates further confirmed the minimal impact, with Kendall’s W consistently ranging between 0.002 and 0.029, indicating negligible practical significance. Clinically, these findings suggest that all three LLMs demonstrated stable and consistent outputs for orthodontics-related queries, insufficient to justify recommending one model over the others at this stage.

Comparison with prior studies highlights both concordance and divergence. Earlier research found that ChatGPT outperformed Google search for medical queries, providing more accurate, citable answers often sourced from academic or healthcare domains, whereas Google responses frequently drew from commercial or social media content of variable quality [10]. This aligns with the trend observed in our study, where ChatGPT scored slightly higher than Gemini. Other reports suggest that DeepSeek offers more comprehensive and timely coverage of treatment-related knowledge and symptom interpretation, while ChatGPT excels in clarity and readability, making it more suitable for patient education [9]. This corroborates our findings: ChatGPT produced concise, easily understandable outputs favored by parents, whereas DeepSeek demonstrated stronger content coverage, especially regarding clinical details. Gemini’s performance, by contrast, appeared more sensitive to prompt design and input variation [12, 19]. Sorin et al. [20] similarly noted that fewer prompts improved Gemini’s recall, whereas chain-of-thought prompting sometimes reduced its accuracy. In our study, we also observed that Gemini’s output stability was less robust than that of ChatGPT or DeepSeek under varying image quality or input conditions. Collectively, these findings underscore that in multimodal clinical contexts, model performance is strongly influenced by prompt strategy and input quality.

From a clinical perspective, all three models were able to identify essential orthodontic features such as facial type, mandibular retrusion, and lip–E line relationships, which are critical for early intervention. Moreover, the intervention suggestions provided by the models were generally consistent with conventional clinical recommendations, for example, suggesting functional appliances for mandibular retrusion. Nonetheless, important limitations remain: model outputs lacked the precision and individualization of orthodontists’ assessments, often omitting key parameters (e.g., ANB angle, transverse skeletal discrepancies, overjet/overbite). Consequently, LLMs should currently be regarded as supportive tools rather than replacements for orthodontists’ diagnostic judgment [21, 22].

This study represents one of the first systematic comparisons of multimodal LLMs in orthodontics, establishing a foundation for future exploration of AI in dental imaging and early orthodontic intervention. The study design incorporated measures to maximize fairness and rigor, including standardized prompts, controlled photographic inputs, and single-blind randomized scoring, thereby reducing information and observer bias. In addition, the use of an eight-dimension weighted scoring system ensured a comprehensive evaluation that considered both clinical accuracy and patient-centered experience.

Despite these strengths, several limitations should be acknowledged. First, the sample was derived from a single center and a homogeneous population (Chinese children), limiting generalizability. Second, the input was restricted to standardized profile photographs, without integration of lateral cephalograms, CBCT, or intraoral scans, potentially omitting key diagnostic information. Third, although the scoring system was validated using ICC and Cronbach’s α, subjectivity could not be entirely eliminated, particularly in dimensions rated by parents, where cultural background and individual interpretation may have influenced outcomes. Finally, LLMs evolve rapidly, and our findings represent only the versions tested at the time of study; future iterations may yield substantially different results.

ChatGPT, DeepSeek, and Gemini demonstrated comparable performance in pediatric profile-based orthodontic evaluation, all achieving high overall scores with no statistically significant differences. ChatGPT showed a slight but non-significant advantage. At this stage, LLMs may serve as supportive tools for early orthodontic screening but cannot substitute for clinical expertise. Future research should incorporate multicenter and multimodal study designs to enhance generalizability and validate the clinical applicability of these models.