This retrospective study was conducted in three steps. First, we developed the simplified scale (sAIS) using the 100 most frequent injuries collected in the Northern French Alps Trauma Registry (TRENAU). Second, for the internal validation, we examined the sAIS classification performance by randomly selecting 100 injured patients in the TRENAU Registry, which were rated by 10 French physicians. Third, we externally validated the sAIS by randomly selecting 100 injured patients included from a different dataset, the Trauma Registry of Acute Care (TRAC) of Lausanne University Hospital (Lausanne, Switzerland), which were rated by eight Swiss physicians and two research nurses. The study was conducted and reported in accordance with the Standards for Reporting Diagnostic Accuracy Studies (STARD) 2015 guidelines (6, 7).

The primary outcome was the accuracy of the sISS calculated using the sAIS compared with the rISS calculated using the rAIS ©2008 and ©2015.

We present continuous data as means and standard deviation (SD) when normally distributed or medians and interquartile range (IQRs) when not normally distributed. We report categorical data as numbers and percentages. We used Student's t-test to compare continuous and normally distributed data and the Mann-Whitney test for continuous and non-normally distributed data. We defined a two-tailed p-value of < 0.05 as statistically significant.

First, we assessed the difference between the two methods at the whole trauma population level. We estimated the mean difference between the rISS and the sISS, which represents a measure of the accuracy. We considered a clinically relevant limit of equivalence of ±4 ISS points for the bias. For a SD of the ISS of 9.5 and a limit of equivalence of 4 ISS points, 97 patients were required to ensure a power of 80% with a significance level of 5%. We estimated the Pearson correlation coefficient (r) as another measure of accuracy. The relationship between the sISS and the rISS was described by using scatterplots and local polynomial regression in a calibration plot.

Second, as a measure of the precision at an individual patient level and to examine the agreement between the sISS and the rISS, we used the Bland-Altman method to plot the bias and the limits of agreement (LoA). Assuming a normal distribution, the LoA represent the mean of the difference ± 2 SD of the difference. We considered a relevant LoA range of ±9 ISS points. We used two different limits of ISS variation. For the precision at an individual level, we used a LoA range of ±9 ISS points, corresponding to an increase in the severity of an injury from an AIS 4 to 5, as described by Ringdal et al (11). For the accuracy at the population level, we chose a narrower limit of equivalence of ±4 ISS points as clinically relevant. In addition, as the ISS is used to classify major trauma (ISS ≥ 16), we assessed the agreement of major trauma classification by using the Cohen’s kappa statistic between the two methods (12). We performed a complete case analysis as no missing values were reported.

As we suspected an imperfect gold standard bias, an independent trained coder reviewed the rISS of the patient cases of the external validation dataset when the difference between rISS and sISS was outside the calculated LoA limit. We performed a sensitivity analysis of the sISS compared with the corrected rISS. Analyses were performed with Stata version 16 (Stata Corporation, College Station, TX, USA).

The mean of the difference between the rISS and the sISS was 0.97 (95% CI, -0.03 to 1.97) in the internal validation dataset and − 1.77 (95% CI, -3.04 to -0.49) in the external validation dataset, which is included in the equivalence limit of ± 4 ISS points (Table 3). For 11 cases of the external validation dataset, the difference between the sISS and rISS was outside the calculated LoA. After recoding by an independent coder, the mean difference was − 0.86 (95% CI, -1.87 to 0.15).

The Pearson correlation coefficient was 0.82 (95% CI, 0.75 to 0.88) in the external validation dataset and 0.89 (95% CI, 0.84 to 0.93) after correction. The sISS slightly underestimated a lower ISS (< 9) and overestimated a higher ISS (> 25) (Fig. 1).

The Bland-Altman plot showed a low estimated bias, but a LoA range slightly outside the predefined relevant range of ±9 ISS points in the uncorrected external validation dataset (Fig. 2; Table 3). After correction for an imperfect gold standard bias, the LoA range was narrower (-11.1 to 9.4), but remained outside the predefined relevant limits of agreement of ±9 ISS points (Fig. 2). The proportion of patient cases outside the LoA was 11% in the external validation dataset and 4% in the corrected external validation dataset. Most outliers presented a higher ISS (V-shape in the Bland-Altman plot). The calibration plot and the Bland-Altmann plot for the internal validation dataset is presented in the Additional File 3.

The sAIS allows to calculate the sISS in institutions without a capacity of trained and certified staff to code the rISS. This issue affects not only low- and middle-income countries (LMICs), which carry the highest burden of injuries and yet struggle to initiate care improvement programmes, but also high-income countries (HICs). In HICs, only the main trauma centres can finance certified coders. Thus, reliable data to estimate the burden of injuries and calculate the ISS for benchmarking are essential, even in smaller hospitals with limited resources that still care for injured patients. A less precise, but more accessible method for ISS coding would facilitate the implementation of quality improvement programmes in settings with a high incidence of traumatic injuries, including audit and benchmarking. However, staff scoring the sAIS still require training to understand medical terminology and accurately extract information from medical records. Notably, a simple and available ISS coding tool could facilitate the inclusion of LMICs in international trauma research collaborations. One of the criticisms of clinical trials of LMICs is the lack of data on patient severity, particularly ISS data (26). While most severe trauma cases occur in LMICs, the majority of trauma trials were conducted in HICs (27). Nevertheless, many trials conducted in HICs were underpowered and experienced difficulties in patient recruitment (28, 29). Inclusion of patients from LMICs could help to conduct large trials, such as the CRASH trials (30).

A key strength of the study is that we used data collected by robust and established trauma registries. In addition, AIS and ISS coding were performed by AAAM-trained nurses. Despite this, our study has some limitations. As Swiss regulations require written consent for non-interventional studies, we included exclusively patients with written informed consent. This pre-selection of cases may have led to a selection bias in the external validation dataset. However, randomization and stratification on the ISS ensured sufficient representativeness for the purpose of this diagnostic accuracy and validation study, including the use of appropriate statistical methods at the population and individual levels. Of note, an imperfect gold standard bias may have reduced the performance of the sAIS method or simply reproduced the weakness of the reference method. We performed a robust external validation using different study participants and trauma cases and not just a temporal validation like many validation studies (31). Nevertheless, external validation was based on data collected from a similar population in terms of case-mix as the data used for the sAIS development and internal validation. The time periods for the collection of the two datasets were also different. A study exploring inter-rater reliability will be necessary, as well as an external validation study in settings with a different socio-demographic index and including larger populations.