Depression is among the most prevalent psychiatric disorders worldwide and a leading cause of disability.¹ Its global prevalence has risen steadily over the past three decades, with the United States alone seeing depressive symptoms increase approximately threefold from 2017 to 2020.² Despite this high burden, depression remains substantially underdiagnosed and undertreated, with many individuals never seeking professional help or doing so only after symptoms have become severe and impairing.³ Untreated depression is associated with worse prognosis, highlighting the importance of early identification for timely intervention and improved outcomes.^4,5 Emerging digital health tools offer opportunities to expand access and may enable earlier depression detection.⁶

One promising avenue for automated depression detection lies in the analysis of language.⁷ Advances in natural language processing (NLP) and machine learning (ML) now enable scalable, automated analysis of linguistic data to detect depression.⁸ The use of NLP and ML to detect depression has grown exponentially in recent years. However, despite this progress, it remains uncertain how accurate these approaches are overall and whether their performance generalizes reliably across different contexts. Importantly, much of this work has been published in computer science journals, making it less accessible to psychiatric researchers and clinicians who might benefit from these insights. Clinical researchers must be engaged and informed to critically evaluate, guide, and translate these tools into meaningful clinical use.

Several prior reviews have mapped the landscape of NLP and ML approaches for detecting depression, summarizing common text sources, feature extraction strategies, and model architectures. ^8–15 However, despite the rapid expansion of this research area, none of these reviews quantitatively evaluated the accuracy of these models across studies. In this systematic review and meta-analysis, we evaluated the performance of NLP- and ML-based models in detecting depression using datasets where depression status was determined independently (e.g., through validated questionnaires or clinical diagnoses). We further examined study-level moderators, including language, text source, feature type, and model class.

This systematic review and meta-analysis evaluated the performance of NLP and ML models for detecting depression from text data. The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guidelines (Table S1).¹⁶ The study protocol was preregistered with the International Prospective Register of Systematic Reviews (PROSPERO; ID CRD42024513390).

Screening and review management were conducted using Covidence software (Veritas Health Innovation, Melbourne, Australia). In the first and second phases (title and abstract review, full text review), two independent raters screened each record. Disagreements were resolved by consensus between reviewers. To ensure consistency and reliability, all reviewers participated in six structured training sessions prior to data collection: two sessions each for title screening, abstract screening, and data extraction. Meetings continued periodically throughout the screening and extraction period.

Inclusion criteria required that studies: (1) used adult participants’ (≥ 18 years) own spoken or written text data to detect (concurrent) depression; (2) employed formal measures of depression (e.g., structured clinical interview, self-report questionnaire, clinical diagnoses); (3) applied machine learning (ML) models to predict the person’s depression status (i.e., binary classification) or depression severity; (4) relied solely on textual input for model development and prediction (i.e., studies incorporating additional modalities such as audio, video, or demographic variables were excluded if they lacked a text-specific model); (5) were peer-reviewed and fully available in scientific databases; and (6) provided sufficient information to extract classification performance (e.g., accuracy, precision, recall, F1, or equivalent).

For the purpose of this review, we included only studies that evaluated model performance on an external sample (e.g., through cross-validation or independent test data), ensuring that reported estimates reflected true predictive capacity rather than overfitting to training data. Large Language Models (LLMs) that were used directly for text classification were not required to include additional validation, as these models are pretrained on large, diverse corpora and apply learned representations without task-specific parameter fitting, reducing the risk of overfitting. However, when LLMs were used solely for feature extraction (e.g., deriving sentiment or emotion scores) and the extracted features were subsequently entered into a separate classification or prediction model to detect depression, such studies were required to include a validation process to ensure the robustness and generalizability of the results.

Studies relying exclusively on social media text were excluded. We made the decision to exclude social media text in order to ensure the inclusion of high-quality studies with clear clinical relevance. Social media data pose several limitations¹⁵: first, depressed individuals may change their online activity, leading to underrepresentation and biased samples. Second, in many social media datasets, depression status is inferred directly from the text, for example, by external annotators rating whether the text contains depression-related content, so the same material is used both to define and to predict depression. This circular annotation strategy undermines label validity and limits generalizability to clinically assessed depression.

HF and two reviewers per study independently extracted data from eligible studies using a standardized template and assessed risk of bias using the ML-specific quality assessment tool developed by Ciharova et al.,¹⁷ adapted from established instruments (PROBAST,¹⁸ Cochrane RoB 2.0,¹⁹ ROBINS-I²⁰). When studies reported results from multiple ML models (e.g., based on different feature sets or classifiers), we extracted the performance metrics of the model that achieved the highest performance. Any discrepancies were resolved in group discussion until consensus was reached. In addition to reading the full text of each article, extracted responses were validated using NotebookLM, which supported but never replaced reviewer judgment. This process was implemented to maximize accuracy and agreement while ensuring that decisions were grounded in direct review of the original studies.

Extracted variables included: study sample size; text source and language; depression measure; feature extraction method (e.g., statistical, linguistic, embeddings, transformer, hybrid); classifier type (traditional machine learning, neural, transformer); type of validation (e.g., k-fold cross validation, train-test split); and reported outcomes (e.g., accuracy, precision, recall, and other model performance metrics).

Given the absence of a standardized framework for evaluating the quality of machine learning studies, we adopted the quality assessment approach used in prior research on ML-based prediction models.¹⁷ In line with that framework, study quality was assessed across four domains: (1) adequate sample size (≥ 100 participants), (2) balanced class distribution in cases where participants are divided into multiple groups, such as depressed vs. non-depressed (no group more than ten times smaller than others), (3) appropriate model validation (model parameters tuned on a training set and evaluated on an independent test set), (4) use of a validated outcome measure assessing depression presence or severity (a.k.a., ground truth). These criteria were designed to minimize risk of overfitting and enhance generalizability to unseen data. Since using validation was an inclusion criterion, item 3 was not relevant and is excluded from the quality assessment table. See Table S2 for the risk of bias table.

The primary outcome was pooled classification accuracy, which quantifies the overall proportion of classifications made by the model that were correct, or in this case, the overall rate at which the model correctly identifies both depressed and non-depressed individuals. Secondary outcomes included precision (positive predictive value), sensitivity (recall), and the area under the receiver operating characteristic curve (AUC). Precision is the proportion of correctly predicted positive cases among all predicted positive cases, whereas sensitivity is the proportion of actual positive cases correctly identified. High precision reflects fewer false alarms, while high sensitivity reflects fewer missed cases. AUC considers both sensitivity and specificity in its calculation, showing the balance between predicting a positive outcome when the outcome is indeed positive and predicting a negative outcome when the outcome is indeed negative. Mathematically, it represents the area under the curve plotting sensitivity (true positive rate) against 1 – specificity (false positive rate). When possible, we also calculated balanced accuracy, defined as the average of sensitivity and specificity. By giving equal weight to correctly identifying positive and negative cases regardless of their prevalence, it provides a less sample-biased estimate of model performance, particularly in datasets where one class (e.g., depressed or non-depressed) is more frequent than the other. The F1-score was used for interpretation in the systematic review but was not included as an outcome in the meta-analysis. This is because it is a harmonic mean of two ratios that already contain partial information about sample size, making its sampling variance difficult to estimate for meta-analytic weighting.

The search yielded 35,000 records. After removing duplicates and screening titles and abstracts, 892 full-text articles were reviewed, and 123 publications contributing 129 effects were included in the qualitative synthesis (Fig. 1). When articles reported results based on multiple datasets, each dataset-specific effect was treated as a separate study.

Over half of the included studies (73/129, 56.6%) used the Distress Analysis Interview Corpus (DAIC) or related datasets derived from it, including three studies that used DAIC alongside another dataset. The DAIC is a well-known benchmark dataset comprising semi-structured clinical interviews. Its Wizard of Oz (DAIC-WOZ) subset,²¹ released as part of the AVEC 2016 challenge, includes 189 interviews conducted by a virtual agent, with audio, video, and transcript recordings available. An extended version (E-DAIC)²² was later released for AVEC 2019, expanding the sample to 275 interviewed participants. In both datasets, depression severity was labeled using the Patient Health Questionnaire-8 (PHQ-8)²³. Table 1 summarizes studies using unique datasets, and Table 2 summarizes those using DAIC data.

Across studies, a total of 35,171 unique participants (removing duplicates from overlapping datasets) contributed 58,413 text samples (e.g., utterances, transcripts, or messages). Publication years ranged from 2013 to 2025, with 95 articles (77.2%) published since 2020, reflecting growing interest in automated text-based depression detection.

Among the 56 studies using unique datasets, 20 (35.7%) analyzed English text, 14 (25%) analyzed Chinese text, and 22 (37.5%) examined other languages (e.g., Italian, Spanish, Turkish, Korean, Russian, German, Malay, Thai). One study (1.7%) combined Chinese and English datasets by translating the Chinese texts into English. The DAIC datasets included English-language clinical interviews.

The number of text observations ranged from 53 to 15,950 (mean = 1,216; median = 210). Key categories of data sources included: Structured clinical interview (19/56, 33.9%), responses to open-ended questions (e.g., “ Describe your weekend activities”; 27/56, 48.2%), text messaging and chat logs (6/56, 10.7%) or interaction with therapists (i.e., text-based therapy; 4/56, 7.1%).

The great majority of studies framed automated depression detection as a binary classification task (100/129, 77.5%). In these cases, models were trained to identify whether an individual (or a given text sample) was produced by a depressed vs. non-depressed person. Typically, “depressed” status was defined using a clinical cutoff on a depression severity questionnaire (for example, a PHQ-9 score ≥ 10) or based on a formal diagnostic evaluation.

This study revealed a substantial increase in studies using NLP- and ML-based approaches for automated depression detection. Among studies included in the quantitative synthesis, models achieved a pooled accuracy of 80%, correctly distinguishing depressed from non-depressed individuals in roughly four out of five cases. For comparison, meta-analyses using resting-state fMRI found similar accuracy (~ 80%)²⁵, whereas those using wearable AI reported higher accuracy (~ 89%)²⁶. Yet given the substantial heterogeneity, results should be interpreted cautiously.

Subgroup analyses showed significant differences in accuracy across several factors. Studies conducted in English yielded lower accuracy (77%) than those in Chinese or other languages, possibly reflecting linguistic or cultural differences²⁷ or dataset-specific effects. This finding warrants further investigation into whether cultural and language factors affect automated detection. Models trained on structured clinical interviews achieved the highest accuracy (84%), whereas those analyzing free-form patient–therapist conversations performed worse (70%), suggesting that direct questioning about mood elicits clearer linguistic signals of depression.

Interestingly, lexicon-based features showed the highest pooled accuracy (86%), outperforming more complex or hybrid models (74%). One explanation is that targeted linguistic markers of depression, (e.g., frequent use of negative emotion words or first-person pronouns) are robust across contexts, allowing simpler models to perform well. However, this finding is based on few studies and requires further validation. Traditional machine learning models performed comparably to transformer-based models (81%), potentially indicating a performance ceiling, possibly due to limited input text that provides insufficient linguistic signal. Other factors may include between-person variability in how depression is expressed, small and imbalanced datasets, insufficient transformer fine-tuning, shallow linguistic cues in the data collection task, and differences in model optimization or evaluation procedures. These factors may constrain even advanced models, underscoring the need for richer longitudinal data and more personalized and context-aware modeling approaches. Consistent with this interpretation, only the type of text source significantly explained some between-study variance in meta-regression (accounting for 13.5% of heterogeneity), suggesting that depression-focused linguistic content may be more critical than model complexity for detection accuracy.

Secondary analysis reveals a pooled AUC of 0.79, suggesting that NLP models can detect depression from text at a level that could be clinically useful, especially for early screening or augmenting clinicians’ assessments. The pooled precision (0.78) and recall (0.76) suggest that, on average, models show a reasonable balance between correctly identifying depressed individuals and avoiding false positives. However, there was substantial variability across studies, likely because these metrics are complementary, and researchers can optimize one at the expense of the other. This trade-off means that while some models favor higher sensitivity to reduce missed cases, others prioritize precision to avoid false alarms.

This study is one of the few systematic reviews and the first meta-analyses to quantitatively evaluate NLP and ML techniques for detecting depression. Engaging clinical researchers in this rapidly evolving field is essential to ensure these tools are properly understood, evaluated, and directed toward real clinical needs. To maintain a high standard, we included only studies that validated their models against independent measures of depression. Nonetheless, several limitations should be acknowledged. The high heterogeneity means the pooled accuracy should be interpreted cautiously, as individual study results varied widely. Although moderator analyses explained some variability, much remains unexplained, likely reflecting differences in validation methods, preprocessing, or sample characteristics.

Furthermore, many papers did not report all the standard classification metrics. The field would benefit from more standardized reporting. Relatedly, accuracy alone can be misleading in imbalanced datasets where high accuracy may simply reflect the majority (non-depressed) class rather than true model performance.

An additional limitation is that over half of studies used variants of the DAIC corpus, meaning a large portion of the literature is built on the same or very similar data, which limits generalizability. While the DAIC is a valuable benchmark, there is a clear need for new datasets that cover different populations, languages, and modes of communication.

Gender differences were not examined, as few studies reported this information. Given that depression is more prevalent among women, they were likely overrepresented in training data, potentially biasing model performance and increasing misclassification risk for men.²⁸ Finally, automated detection differs conceptually from tracking within-person change. Identifying who is depressed does not necessarily translate to monitoring fluctuations or improvement over time, which may rely on different linguistic dynamics.

In summary, NLP and ML can detect depression with good accuracy across a variety of settings. The growing interest since 2020 has yielded many promising approaches. Advancing the field will require greater standardization in reporting, the use of more diverse datasets, and explicit attention to gender and cultural fairness to address the substantial heterogeneity observed. With continued refinement and validation, text-based depression detection systems have the potential to complement traditional assessment methods and broaden access to mental health screening.