The effect of public health system performance on child well-being: an analysis through the construction and selection of composite indicators
AngélicaC.G.Santos1
MatheusLibório1
AndréCoimbra1
MarcosFlávio1
S.V.D’Angelo1
PetrEkel2
HasheemMannan3
HeveraldoR.Oliveira1
IaraSibeleSilva2
1
A
A
A
A
A
State University of Montes Claros – Montes ClarosMG2
A
Pontifical Catholic University of Minas GeraisBelo HorizonteMG3University College Dublin – Dublin – Irlanda
Authors: Angélica C. G. Santos1, Matheus Libório1, André Coimbra1, Marcos Flávio S. V. D’Angelo1, Petr Ekel2, Hasheem Mannan3, Heveraldo R. Oliveira1, Iara Sibele Silva2
1 State University of Montes Claros – Montes Claros – MG
2 Pontifical Catholic University of Minas Gerais – Belo Horizonte – MG
3 University College Dublin – Dublin – Irlanda
Abstract.
Interest in child health and well-being is growing. Given the multidimensional nature of well-being and the health system, composite indicators (CIs) are a useful tool. This study proposes an analysis of the correlation between the CI of child well-being and the CI of the public health system's performance. To this end, a proposal for selecting the weighting scheme for constructing the CIs is presented. Among the schemes used, the CIs constructed with entropy weighting showed the highest correlation. The uncertainty analysis indicates that the ranking of countries obtained with entropy weighting is the most stable among the methods adopted, demonstrating that the selected CIs are robust. The explanatory power indicates a strong correlation between the CIs obtained and the GDP, validating the compatibility with the reference variable of the multidimensional phenomenon, and the discriminant power, demonstrates that the CIs have high informational diversity, facilitating the differentiation of countries by decision makers. The analysis of the correlation of CIs generated to represent child well-being and the health system indicates that countries with better health system performance tend to have higher levels of child well-being, reinforcing the role of the public health system for child well-being in the search for sustainable development.
Keywords:
Well-being
Health System
Multidimensional health phenomenon
Composite Indicators
1. Introduction
The growing interest in the search for improvements in child well-being has stimulated several studies in recent decades, especially after the declaration of the Sustainable Development Goals (SDGs) by the United Nations [1]. Goal 3 of the SDGs, which seeks to ensure healthy lives and promote well-being for individuals of all ages [2], underscores the need to evaluate well-being alongside health system performance.
Given this, although it is possible to relate well-being to the public health system through effects on key aspects of quality-of-life measurement [3] and economic and social factors [4], such relationships do not capture the multidimensional nature of these phenomena. To overcome this barrier, given the multidimensionality of well-being and the public health system, synthesis measures—known as composite indicators (CIs)—are required.
It should be noted that the construction of CIs presents some points of attention. Among them is the choice of the weighting scheme and its repercussions on scores and on the quality and reliability of the CI [5]. The possibility of weighting the sub-indicators by different methods yields CIs with different scores [6], raising doubts about which CIs best represent the relationship between well-being and the health system and whether they have satisfactory quality and reliability.
In this sense, the method used to construct composite indicators can influence the ability to assign importance to sub-indicators [7]. Sampling errors can affect reliability by generating estimation errors of the composite indicator [8]. The review and development of indicators can be affected by a lack of transparency and subjectivity in decisions about aggregation methods and sub-indicator selection [9].
Seeking to overcome these difficulties, this study aims to construct CIs for child well-being and the public health system's performance using different weighting schemes. To assess the connection between child well-being and the public health system in low- and middle-income countries, CI selection is based on the correlation between the CI child well-being and the CI performance of the public health system. The analysis of robustness and quality of the results is done through measures of uncertainty, explanatory power, and discriminant power.
This article contributes to improving the construction, selection, and validation of CIs. Choosing the best weighting scheme and verifying the robustness of the results help provide a better understanding of the relationship between child well-being and the performance of the public health system, providing valuable data for managers, politicians, and society.
The article is organized as follows: after this introduction, Section 2 presents the methodology for constructing CIs and their validation. Section 3 presents the analysis of the robustness and quality of the results. Section 4 displays dimensions and sub-indicators. Section 5 presents the discussion of the results, and Section 6 presents the conclusions.
2. Construction of Composite Indicators: Methods Adopted
The construction of CIs has been developed by a wide variety of methodological approaches [10, 11]. The first step in building a CI is data scaling, to standardize data at different scales to a dimensionless scale. This allows for consistent comparison and analysis of data, regardless of its original scale. For example, the scale transformation by the max./min method. Convert the data to a range [0,1] by the following function:
1
or
2
where
is the value of the sub-indicator
of the unit under review k;
is the highest value of the sub-indicator
among the units under review
;
is the lowest value of the sub-indicator
among the
units under review.
If the sub-indicator has positive polarity, the scale transformation shown in the Eq. (1), in which the higher value of
provides
equal to 1. In the case of negative polarity, The transformation presented in the Eq. (2), in which the higher value of
provides
equal to 0.
The choice of the weighting scheme is the second essential decision. Methods can be objective, data-driven, or subjective, and may consider expert or stakeholder opinions [12].
2.1. Equal weighting scheme
The use of equal weights in constructing CIs is a common practice in studies of quality of life and well-being. Equal weighting is often seen as a straightforward and neutral approach, but it does not always reflect the true importance or contribution of each component indicator [12]. After the transformation of the scale of the sub-indicators by Eq. (1) or Eq. (2), considering all weights equal, the CI can be obtained by the arithmetic mean through the function:
3
where
is the corresponding weight of the sub-indicator j of the unit under review k and
is the normalized sub-indicator.
The weights
can also be defined by other objective approaches that can offer more robust and informative results than the equal-weights method [13]. However, these weighting schemes are more difficult to communicate to a broader audience [11] and can introduce additional complexities [13].
2.2. Weighting scheme by factor analysis
The weights in Eq. (3) can be determined by factor analysis, which synthesizes data to describe correlated variables in a smaller, independent set of derived variables, with minimal loss of information [14].
The objective of factor analysis is to represent a set of variables with a smaller number of factors, emphasizing their relationships. To do this, it considers the relationship between the underlying factors and the observed responses, rather than the individual responses themselves [15]. The variance of the data can be decomposed into values estimated by common and single factors [11] through:
4
where
(j = 1,…,n) are the original values of the standardized variables with zero mean and unit variance; the factor weights related to the variable
are represented by
. The common uncorrelated factors, each with a zero mean and unit variance, are represented by
. Finally,
represents the specific factors Q, which are assumed to be independent and identically distributed with mean zero.
There are several approaches to treat the method given in the equations presented in Eq. (4) [11]. In this study, the analysis model will be used, in which the estimated CI values are equal to the component score multiplied by its proportional variation [16]
2.3. Entropy weighting scheme
Shannon entropy is a measure of uncertainty or complexity in a system [17]. The Shannon Entropy weighting mechanism ensures that each variable is assigned an objective weight that reflects its data variability [18]. The lower the variable's uncertainty, the greater its weight in the weighting.
In the entropy weighting scheme,
Indicators and
samples are defined in the evaluation, and the measured value of the
-th indicator at
-th sample is recorded as
. After the normalization of the sub-indicators, the entropy calculation is done through the following equation:
5
and,
6
The degree of importance and weight of each variable are defined by Eq. (7) and Eq. (8), respectively:
7
and,
8
where
are the sub-indicators j of the unit under review k, after the scale transformation (see Eq. 1 and Eq. 2), n is the total number of sub-indicators, m is the total number of units under analysis, E is the entropy, I is the degree of importance, and w is the weights.
2.4. Benefit of the doubt weighting scheme
Benefit of the Doubt (BoD) is a method of constructing CIs based on Data Envelopment Analysis [19]. The method was proposed by [20] to evaluate macroeconomic performance and is now applied to measure other multidimensional phenomena, such as the Human Development Index [21] and social exclusion and vulnerability [22].
The method uses sub-indicators as outputs and simulates an input of 1 for all units under analysis. The focus is exclusively on aggregating the various sub-performance indicators, without explicit reference to the inputs used [23]. It measures the relationship between a unit's performance and its benchmark performance.
In the absence of true weights, the BoD assigns each sub-indicator a weight determined by the data to construct the composite score [24]. The weights are obtained based on the reference performance of each unit [10]. The individualized weighting process assigns low weights to the worst-performing sub-indicators and higher weights to the best-performing sub-indicators [25]. After transforming the scale of the sub-indicators (see Eq. 1 and Eq. 2), it is possible to construct the CI by BoD by applying the following equation:
9
with the following restrictions:
10
11
where
is the corresponding weight of the sub-indicator j of the unit under review k
The use of the BoD requires attention, as it is sensitive to extreme values, outliers, and measurement errors [10]. In addition, as it is an endogenous measure, generated internally to the model, it may not be compatible with the theoretical framework of the multidimensional phenomenon. In turn, the exogenous weightings of the sub-indicators, generated outside the model, disregard their relative importance, thereby compromising results when the dimensions have different numbers of sub-indicators [26].
3. Method selection, robustness, and quality analysis
The weighting scheme for the CIs of child well-being and health system performance was chosen based on Spearman's correlation coefficients among the indicators. The objective is to identify the method that presents the greatest association in the classifications of each unit for both indices. Next, the robustness and quality measures of the results are shown.
To test the robustness and quality of the CIs generated by the selected method, the analyses of uncertainty, explanatory power, and discriminant power of the CIs constructed by the selected weighting scheme are performed.
3.1. Uncertainty Analysis
Uncertainty analysis offers a measure of CI stability [10]. It allows checking how much a country's ranking varies with changes in the weighting scheme, in other words, the degree of uncertainty of the results. The robustness measure of the positions of the units under analysis in the CI ranking is operationalized by Eq. (12):
12
where
is the relative change in the position of all the countries of the CI
, m is the maximum possible variation for the system, Rank (CIj) is the ranking of the j-th country not CI, e Rank (
) is the ranking of the j-th country of the f indicators constructed by the weighting obtained by different schemes.
3.2. Explanatory power
A measure often adopted for indicator validation is the correlation between the composite indicators and a conceptually significant external reference variable [27]. The connection with the external variable indicates how well the CI aligns with the reference variable of the multidimensional phenomenon [28]. External validation is obtained by correlation with the external reference variable in Eq. (13):
13
where R is the correlation coefficient between the composite indicator CI and the external variable Ext, CIj is the score of the country's composite indicator j,
is the average of the composite indicator scores, Extj is the country's external variable j e
is the average value of the external variable.
Based on the rule of thumb of [29], it is possible to indicate the explanatory power of the CI based on its correlation with the most conceptually significant variable of the multidimensional phenomenon: R > 0,90 very strong, 0,70 < R < 0,90 strong, 0,50 < R < 0,70 moderate, 0,30 < R < 0,50 weak, and R < 0,30 insignificant.
3.3. Discriminating power
The dispersion of the CI scores indicates the indicator's discriminant power across countries. The greater the degree of dispersion, the greater the degree of differentiation, and the more information can be derived [30]. It offers a measure of information diversity and signals the ease (or difficulty) of differentiating countries. The measure of the discriminant power of the CI used is the entropy index of [17]:
14
where H' corresponds to the diversity of Shannon's information, CIk is the value of the composite indicator under analysis of country k.
4. Data: dimensions and sub-indicators
Data on the sub-indicators of child well-being and the performance of the public health system were extracted from the World Health Organization (WHO) database. Data from World Health Statistics 2022, available at https://www.who.int/data/gho/publications/world-health-statistics, were collected. The sample for analysis comprises data from 62 middle- and low-income countries with available data.
In this study, eight sub-indicators representing five dimensions are adopted: Demographic: life expectancy at birth (years) and child dependency rate; Child Health: mortality rate of children under five years of age and prevalence of overweight; Education: one year before the age of entry into primary school for men and women; Maternal Health: sub-index of coverage of services in reproductive, maternal, neonatal and child health and stillbirth rate External variable: GDP per capita (current US$).
The health system's performance is measured using five sub-indicators that represent three dimensions. First, service: density of physicians (per 10,000 inhabitants), density of nursing and midwifery personnel (per 10,000 inhabitants), and density of dentists (per 10,000 inhabitants). Second, Capacities to respond to public health risks and emergencies: average of 13 international health regulations basic capacity scores. Third, expenditure: internal expenditure of general government on health as a percentage of general government expenditure; External variable: GDP per capita (current US$)2010–2019.
5. Results and discussion
Tables 1 and 2 present the descriptive statistics of the CIs of child well-being and performance of the public health system constructed by the methods of equal weights, factor analysis, entropy weights, and BoD.
Child well-being | Md | Mín. | Máx. | ||
|---|---|---|---|---|---|
Equal weights | 0.554 | 0.600 | 0.226 | 0.054 | 0.918 |
Factorial | 0.789 | 0.863 | 0.216 | 0.178 | 1.000 |
Entropy | 0.552 | 0.596 | 0.225 | 0.055 | 0.915 |
BoD | 0.536 | 0.571 | 0.221 | 0.066 | 0.898 |
Health System | Md | Mín. | Máx. | ||
|---|---|---|---|---|---|
Equal weights | 0.255 | 0.218 | 0.170 | 0.029 | 0.863 |
Factorial | 0.270 | 0.223 | 0.205 | 0.000 | 1.000 |
Entropy | 0.245 | 0.204 | 0.171 | 0.027 | 0.871 |
BoD | 0.494 | 0.451 | 0.292 | 0.000 | 1.000 |
The results shown in Fig. 1 indicate that the CIs of child well-being and health system performance exhibit a strong positive correlation, and that the entropy weighting scheme shows the strongest correlation.
Fig. 1
Spearman's correlation.
The results of the robustness analysis are presented in Table 3 and indicate that the entropy method shows the least variation among the methods analyzed. The lower uncertainty obtained from entropy weighting corroborates the validity of the results generated by the selected weighting scheme.
Well-being | Health System | ||
|---|---|---|---|
Equal weights | 0.0012 | Equal weights | 0.0011 |
Factorial | 0.0026 | Factorial | 0.0012 |
Entropy | 0.0011 | Entropy | 0.0011 |
BoD | 0.0013 | BoD | 0.0024 |
The quality analysis of results from CIs constructed using the entropy-weighted method demonstrates that indicators of child well-being and the performance of the public health system have strong explanatory power (Fig. 2).
Fig. 2
Correlation graph between the explanatory variable and the indicators.
The analysis of the discriminant power, presented in Fig. 3, shows that the CIs constructed with the entropy-weighted method exhibit a class distribution that allows clear differentiation of results by country; that is, they enable identification of differences between the evaluated units. The results presented in Fig. 3 indicate that countries have more concentrated performances in child well-being around 0.75. Regarding the CIs of the health system, the countries under analysis obtained more differentiated CIs; that is, the CIs indicate that the countries have different levels of performance in the health system.
Fig. 3
Histograms and results of the analysis of the discriminant power (H')
In summary, the data indicate that the entropy weighting scheme is more appropriate for constructing CIs when evaluating the influence of the health system's performance on child well-being. The results show a positive relationship between the performance of the countries' public health systems and child well-being indicators.
6. Conclusions
The growing interest in improving child well-being is accompanied by the challenge of synthesizing the multiple dimensions involved. In this sense, Goal 3 of the SDGs highlights the importance of jointly analyzing the health system's well-being and performance.
Given the multidimensional nature of well-being and the public health system, CIs are a valuable tool for assessing the relationship between the two. Seeking robust and high-quality CIs, this study presents robustness analyses of four methods for selecting the best method for constructing the proposed indicators and a quality analysis of the results of the indicator built with the chosen method.
The results of the correlation between the CIs' well-being and the health system's performance indicate, according to the literature, that countries with better public health system performance achieve better results in child well-being indicators. These results corroborate the public health system's role in achieving the SDGs.
In selecting the weighting scheme, the entropy scheme showed the strongest correlation with the CIs and the best robustness. In the analysis of the quality of the results, it was found that they have strong explanatory and discriminant power.
The results demonstrate the importance of developing studies that delve into the techniques for constructing and evaluating CIs. The construction of CIs, whose results are verified, is a valuable tool for developing actions that help countries promote improvements and overcome their difficulties more effectively.
A
Funding
This study was funded by FAPEMIG [APQ-00691-23, APQ-00528-24, APD-00860-25, PPE-00087-24] and CAPES. Author Marcos D'Angelo also acknowledges the personal research fellowship from CNPq [308265/2022-0].
A
A
Author Contribution
Angélica Santos: wrote the main manuscript text; Matheus Libório: conceptualization and methodology; André Coimbra: analysed data; Marcos Flávio D'Angelo: supervision; Petr Ekel: supervision; Hasheem Mannan: validation; Heveraldo Oliveira: analysed data; Iara Silva: concepptualization. All authors reviewed the manuscript.
A
Data Availability
The data used were extracted from the World Health Organization (WHO) database. Data were collected from the 2022 World Health Statistics, available at https://www.who.int/data/gho/publications/world-health-statistics.
References
1.
Clark H, Coll-Seck A, Banerjee A, Peterson S, Dalglish S, Ameratunga S, et al. A future for the world's children? A WHO–UNICEF–Lancet Commission. Lancet. 2020;395(10224):605–58. https://doi.org/10.1016/S0140-6736(19)32540-1.
2.
United Nations. (2015). Transforming our World: The 2030 Agenda for Sustainable Development. Available at: https://sdgs.un.org/2030agenda
3.
Barzylovych A, Ursakiі Y, Nadezhdenko A, Mamatova T, Chykarenko I, Kravchenko S. (2021). The Influence of Medical Services Public Management on the Population' Life Quality. WSEAS TRANSACTIONS ON ENVIRONMENT AND DEVELOPMENT. https://doi.org/10.37394/232015.2021.17
4.
Dhungana B. Government Health Expenditure and Policy for Public Health Outcomes: A Systematic Literature Review. MedS Alliance J Med Med Sci. 2023;6(3). https://doi.org/10.3126/mjmms.v3i6.66615.
5.
Zhou P, Ang B, Zhou D. (2010). Weighting and Aggregation in Composite Indicator Construction: a Multiplicative Optimization Approach. Social Indicators Research, pp. 169–181. https://doi.org/10.1007/S11205-009-9472-3
6.
Burgass MJ, Halpern BS, Nicholson E, Milner-Gulland EJ. Navigating uncertainty in environmental composite indicators. Ecol Ind. 2017;75:268–78. https://doi.org/10.1016/j.ecolind.2016.12.034.
7.
Libório MP, Diniz AM, Rabiei-Dastjerd H, Martinuci OD, Martins CA, Ekel PI. (2023). A decision framework for identifying methods to construct stable composite indicators that capture the concept of multidimensional social phenomena: the case of social exclusion. Sustainability, 7. https://doi.org/10.3390/su15076171
8.
Mauro V, Giusti C, Marchetti S, Pratesi M. (2021). Does uncertainty in single indicators affect the reliability of composite indexes? An application to the measurement of environmental performances of Italian regions. Ecological Indicators, 127. https://doi.org/10.1016/j.ecolind.2021.107740
9.
Cavicchia C, Vichi M. Statistical model-based composite indicators for tracking coherent policy conclusions. Soc Indic Res. 2021;156:449–79. https://doi.org/10.1007/s11205-020-02318-7.
10.
Nardo M, Saisana M, Saltelli A, Tarantola S. Tools for composite indicators building. Eur Comission Ispra. 2005;15(1):19–20.
11.
OECD. Handbook on constructing composite indicators: methodology and user guide. OECD publishing; 2008.
12.
Greco S, Ishizaka A, Tasiou M, Torrisi G. On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting Aggregation, and Robustness. Soc Indic Res. 2019;141. https://doi.org/10.1007/s11205-017.
13.
Maričić M, Egea J, Jeremic V. A Hybrid Enhanced Scatter Search—Composite I-Distance Indicator (eSS-CIDI) Optimization Approach for Determining Weights Within Composite Indicators. Soc Indic Res. 2019;144:497–537. https://doi.org/10.1007/s11205-018-02056-x.
14.
Härdle W, Simar L. Factor Analysis. Applied Multivariate Statistical Analysis. Springer; 2019.
15.
Jackman S. (2020). Introduction to Factor Analysis. Em W. H. Finch, Exploratory Factor Analysis. Sage Publications. https://doi.org/10.4135/9781544339900
16.
Vidoli F, Fusco E. (2018). Compind: Composite indicators functions based on frontiers in R (Compind package version 2.0). Available at: https://pbil.univ-lyon1.fr/CRAN/web/packages/Compind/vignettes/Compind_vignette.pdf
17.
Shannon CE. A mathematical theory of communication. Bell Syst Tech J. 1948;27(3):379–423.
18.
Gupta S, Tanushree T, Rai A, Applications. (2020). Combined Shannon's Entropy and Data Envelopment Analysis Model: Methods and. 2020 International Conference on Contemporary Computing and Applications (IC3A), pp. 260–264. IEEE.
19.
El Gibari S, Gómez T, Ruiz F. Building composite indicators using multicriteria methods: a review. Z Für Betriebswirtschaft. 2019;89:1–24. br/10.1007/s11573-018-0902-z. https://doi-org.ez93.periodicos.capes.gov..
20.
Melyn W, Moesen W. 1991. Towards a synthetic indicator of macroeconomic performance: unequal weighting when limited information is available. Public economics research papers, pp. 1–24.
21.
Ravanos P, Karagiannis G. 2021. A VEA Benefit-of-the-Doubt Model for the HDI. Social Indicators Research, pp. 27–46. https://doi.org/10.1007/s11205-020-02589-0
22.
Libório M, Diniz A, dos Santos A, Nobre CN, Vieira DA, Mannan H, et al. Benefit-of-the-Doubt in the Spatial Analysis of Child Well-Being in European Countries. Child Indicator Res. 2024. https://doi.org/10.1007/s12187-024-10145-2.
23.
Cooper W, Seiford L, Zhu J, editors. Handbook on data envelopment analysis. Dordrecht: Kluwer Academic; 2004.
24.
Gulati R, Kattumuri R, Kumar S. A non-parametric index of corporate governance in the banking industry: An application to Indian data. Socio-Economic Plann Sci. 2020;70. https://doi.org/10.1016/j.seps.2019.03.008.
25.
Carrino L, Farnia L, Giove S. Measuring Social Inclusion in Europe: a non-additive approach with the expert-preferences of public policy planners. J Royal Stat Soc Ser A: Stat Soc. 2023. https://doi.org/10.1093/jrsssa/qnad106.
26.
Libório MP, Karagiannis R, Diniz AM, Ekel PI, Vieira DA, Ribeiro LC. The Use of Information Entropy and Expert Opinion in Maximizing the Discriminating Power of Composite Indicators. Entropy. 2024;26(2). https://doi.org/10.3390/e26020143.
27.
Dialga I, Giang LT. Highlighting methodological limitations in the steps of composite indicators construction. Soc Indic Res. 2017;131(2):441–65. https://doi.org/10.1007/s11205-016-1263-z.
28.
Correa Machado AM, Ekel PI, Libório MP. Goal-based participatory weighting scheme: Balancing objectivity and subjectivity in the construction of composite indicators. Qual Quant. 2023;57(5):4387–407. https://doi.org/10.1007/s11135-022-01546-y.
29.
Hinkle DE, Wiersma W, Jurs SG. Applied statistics for the behavioral sciences. Volume 663. Boston: Houghton Mifflin; 2003.
30.
Zhu Y, Tian D, Yan F. Effectiveness of entropy weight method in decision-making. Math Probl Eng. 2020;2020(1). https://doi.org/10.1155/2020/3564835.