Mental Health Inventory: Sensitivity and Specificity of the Portuguese Version of the MHI-38 and MHI-5

Abstract

Detection of mental distress cases is essential in clinical practice, especially in primary care. Screening instruments could be useful and effective tools to help identify them. This study sought to evaluate the utility and capability of the Mental Health Inventory (MHI)-38 and MHI-5 in identifying cases suffering from mental distress from those who do not. The validity and accuracy of these MHI versions were tested using as gold standards: two samples of adults, one with clinical complaints (n = 33) and another without clinical complaints (n = 31); and the scores in Scales of Psychological Well-Being and in a psychopathology inventory (Minnesota Multiphasic Personality Inventory 2). Receiver operating characteristic curves were used to define cutpoints, which is the Youden’s index for the optimization criteria. The data analysis indicated an optimal cutpoint of 7.19 for MHI-38_TOTAL and of 53 (recoded to a 0–100 scale) for MHI-5_TOTAL to differentiate clinical cases from those who are not. These results indicate cutpoint values similar to those of previous studies in the case of MHI-5 and provide useful reference values for MHI-38. The need to replicate this study with larger samples and with controlled clinical conditions and type of pathology is also discussed.

Keywords

Mental health screening well-being distress validity

Introduction

The prevalence of mental disorders has been increasing over the years, reaching high levels in many developed countries (World Health Organization [WHO], 2001). Although mental disorder is not usually fatal by itself, it is a major cause of disability worldwide (Brundtland, 2000). It is troubling to consider that there are data that indicate that mental health problems symptomatology is associated with impairments in functioning and therefore in the quality of life of those suffering from it, comparable to or greater than those demonstrated by people with chronic medical conditions (Means-Christensen, Arnau, Tonidandel, Bramson, & Meagher, 2005).

However, the stigma associated with the term “mental illness” alienates many people from specialized mental health services, which are oriented for serious conditions (e.g., schizophrenia, paranoia, major depression, or organic brain syndromes). Thence, numerous clinical cases are not specifically targeted. There are many people who suffer from emotional distress—associated to or resultant of chronic diseases, unemployment, poverty, violations of human rights, or natural disasters—that are not identified or treated. When these cases are left untreated, there are not only an increase of the individual morbidity but also an overuse of the general medical services (Von Korff, Ormel, Katon, & Lin, 1992) that causes great overload for these settings and consequently for physicians in primary care (Means-Christensen et al., 2005; Thorsen, Rugulies, Hjarsbech, & Bjorner, 2013).

Consequently, there is a need to have a better scrutiny of the mental problems and disorders and to improve the recognition and detection by primary care providers of cases of debilitating distress that truly must benefit of treatment to orient and treat them at the best of times and in an accurate and cost-effective way. Nonetheless, due to time constraints and policies, mental evaluation is occasionally overlooked (Borus, Howes, Devins, Rosenberg, & Livingston, 1988). Although a clinical interview with a mental status examination is a good technique to identify psychiatric morbidity or stress-related disorders and psychosocial problems (Rosenthal & Akiskal, 1985), it is also too expensive and time consuming for professionals.

Screening instruments are a way of improving the early detection of a diversified kind of problems related with mental health in primary care, contributing to more immediate and effective interventions. They are like diagnostic tests in the way that are used to determine if a person has the attribute in reference, but they are instead given to a large group of asymptomatic people.

Screening tests should be ideally brief, easy to complete, and easily scored to facilitate epidemiological studies and the identification of cases at risk that deserve specific clinical attention. They should also have good indicators of reliability and validity and specific norms of reference for different population groups of age, sex, socioeconomic status, language, and cultural background (Means-Christensen et al., 2005).

The Mental Health Inventory (MHI) is a screening tool that reflects a broad concept of mental health, that is, not only the absence of psychopathology but also the presence of a “state of well-being in which every individual realizes his or her own potential, can cope with the normal stresses of life, can work productively and fruitfully, and is able to make a contribution to her or his community” (WHO, 2005). It is a widely used and recognized instrument developed by Veit and Ware (1983), under the Rand Corporation’s Health Insurance Study. The factorial analysis of the original version, with 38 items (MHI-38), confirmed a general factor of mental health, two related but distinct higher order factors, Psychological Well-Being and Psychological Distress, and five correlated factors of lower order, which correspond to the subscales of the instrument: Positive Affect and Emotional Ties; and Anxiety, Depression, and Loss of Emotional or Behavioral Control, respectively (Ware & Gandek, 1994). This factorial structure was replicated with samples of diverse regions of the United States (e.g., Veit & Ware, 1983). In different countries, where the MHI has been subjected to translations and validity studies, the data have revealed high levels of internal consistency and the same structure (e.g., Florian & Drory, 1990; Heubeck & Neill, 2000; Ostroff, Woolverton, Berry, & Lesko, 1996).

Over the years, other MHI’s shorter versions were constructed. One has five items (MHI-5) and has revealed a correlation between .95 and .93 with the original MHI-38 (Ware & Gandek, 1994). This brief version has been extensively used in research in the last decades, or per se or included in some large instruments, namely, the Medical Outcome Study (MOS) Short Form 20 and 36 (SF-20; Stewart, Hays, & Ware, 1988; SF-36; Ware & Sherbourne, 1992).

For various reasons, mental health indicators can be unique or have different relevancy to specific cultures (Vaingankar et al., 2014). So, translation, validation, and setup standards for these instruments in other countries will allow to increase the attainment of multidimensional value in an international level, and of clinical trials and research in general, since it will use methods to make health comparisons between countries and between groups within countries (Ware & Gandek, 1994).

In our research in the literature, we did not find either national or international studies that indicate cutpoint values for the MHI-38 version. There are international studies which indicate several specific cutpoints for research or for screening mental health for the MHI-5 version (see “Discussion” section); however, there are no studies with Portuguese samples indicating and studying specific cutpoints for it.

Thus, the main goal of this research was to find evidence of criterion validity and propose cutpoints for the different measures of the MHI-38 and MHI-5 for the Portuguese population. With that in mind, accuracy measures were calculated to determine the effectiveness and accuracy of the MHI as a screening measure.

Method

Participants and procedures

Two groups were studied: a clinical group (CG; participants with clinical psychiatric history) and a nonclinical group (NCG; participants without any psychopathology diagnose or relevant clinical history).

After a rigorous scrutiny of the pre-established conditions and after an exclusion of six subjects who showed critical results on Minnesota Multiphasic Personality Inventory’s (MMPI) validity scales (high levels of inconsistency, bizarre, or insincerity according to the test standards, that is, T scores greater or smaller by the threshold indicated by the test standards for each validity scale) or physical diseases that induce a high level of psychological distress (e.g., cancer), 64 individuals participated (CG: 31 and NCG: 33). They were all Portuguese and had no suspected mental disability, cognitive impairment, or critical clinical situation with implications of cognitive functions (see Table 1 for a more detailed sample’s characterization).

Table 1.

Sample’s demographic characteristics.

	CG (n = 31)		NCG (n = 33)
Age (years)	n	M (SD)	n	M (SD)
18–74		45.00 (16.5)		33.79 (14.9)
18–23	5		13
24–36	4		10
37–55	14		6
56–74	8		4
Sex
Male	6		18
Female	25		15
Educational level
Basic	8		1
High school	12		5
Higher education	11		27
Professional status
Employed	15		13
Retired	8		2
Student	6		14
Unemployed	2		4

CG: clinical group; NCG: nonclinical group.

The two groups had a similar distribution for age and employment status (χ²_(2, _N _= 64) = 45.32, p = .09 and χ²_(3, _N _= 64) = 7.56, p = .06, respectively). However, there were significant differences regarding sex (χ²_(5, _N _= 64) = 8.45, p = .004) and education (χ²_(5, _N _= 64) = 16.34, p = .006), which are taken into consideration in the analysis of the results.

CG was recruited from a public hospital in Azores (Hospital da Horta) and from a private clinic in Lisbon (Clínica Psiquiátrica de São José). All participants were considered by their psychologist and/or psychiatrist able to participate in the study and were inpatient in short-term hospitalization (17) or outpatients (14), with different kinds of mental pathologies; mostly were under some psychopharmacological treatment. NCG recruitment was conducted through the diffusion of the study in the social network of students and professors of the university.

In both groups, informed consent was obtained, and a brief individual interview was conducted with the purpose of collecting demographic data and other information about clinical history. The order of application of the instruments was it follows: (1) MHI, (2) Scales of Psychological Well-Being (SPWB), and (3) Minnesota Multiphasic Personality Inventory-2. In CG, the application of the instruments was conducted in one to two individual sessions, in the institutions involved in the study in accordance with their ethics committees. For the NCG, individual or small group sessions (5 to 6 participants) were scheduled and the applications were performed in just one session that lasted 2 hours. Faculdade de Psicologia da Universidade de Lisboa Ethics Committee approved this procedure.

Instruments

Mental Health Inventory

MHI (Veit & Ware, 1983) is a psychological well-being and distress self-report measure. We consider two versions: the original MHI-38 and the short MHI-5.

MHI-38 provides eight measures: a global scale—Total Mental Health (MHI-38_TOTAL); two partial scales—Psychological Well-Being (MHI-38_PWB) and Psychological Distress (MHI-38_PD). These encompass two and three subscales, respectively: Positive Affect and Emotional Ties and Anxiety, Depression, and Loss of Behavioral/Emotional Control. Each item requires a response on a six-point scale, excluding two items that have a five-point scale; each point is associated with the frequency or the intensity level of the behaviors, feelings, or thoughts the person experiences. Higher scores indicate a higher level of a global mental health and on the specific dimensions of it.

The scoring was conducted in accordance with the original version. Moreover, all measures were calculated and recoded so that we could have comparability between the final measures of all scales. We used the following procedure: first, the missing data (protocols with more than one unanswered item per subscale or more than three on MHI_TOTAL were not accepted) were replaced by the theoretical average of the respective item (i.e., “3” for items with a response scale with six points or “2.5” for items with a response scale with five points). Second, all the items with a five-point scale (item 9 and item 28) were recoded to a six-point scale (the values obtained in these items were divided by 6). Third, after obtaining the sums of the scores for the eight measures, to allow the comparison of the scores of scales with different number of items, all were divided by the number of their constituent items (e.g., the total score of the Depression scale was divided by five and, in turn, the sum of Positive Affect scale was divided by 11). This procedure was done for all subscales. Fourth, the values in the subscales that form the Psychological Well-Being scale (MHI-38_PWB) and the Psychological Distress scale (MHI-38_PD), respectively, were summed. Since each scale has a different number of associated subscales, the values of MHI-38_PWB scale were divided by two and the MHI-38_PD scale by three. Finally, the values obtained from these two scales were added and the values of the total scale (MHI-38_TOTAL) were attained. This means that the values of the MHI-38_TOTAL range between 2 and 12.

The MHI-5 comprises five items, each one coming from one of the subscales of the original version and having only one global measure (MHI-5_TOTAL), with higher scores indicating a better mental health. The response is given in a six-point scale; thus, the total score ranges between 5 and 30. Usually, the scores obtained in this version are transformed into a 0 to 100 scale and we followed the same procedure.

Veit and Ware (1983) reported internal consistency coefficients from .81 to .96 for the MHI-38 subscales. The Portuguese version (Silva & Novo, 2002) provides Cronbach’s alpha between .73 and .95 for the MHI-38 subscales and .96_, and .88 for MHI-5_TOTAL (Novo, 2004). In this study, the Cronbach’s alpha for the total sample was between .74 and .98 (see Table 2) and for the two groups separately was between .57 and .95 (for CG) and .66 and .95 (for NCG).

Table 2.

Descriptive statistic, Cronbach’s alphas, and MANCOVA for MHI-38 and ANCOVA for MHI-5 for CG and NCG.

MHI	Total sample			NCG	F(1, 60)	η²
	N = 64		n = 31	n = 33
	M (SD)	α	M (SD)	M (SD)
MHI-38
Psychological distress	3.93 (1.06)	.92	3.14 (0.84)	4.68 (0.62)	36.324*	.377
Anxiety	3.69 (1.05)	.92	2.94 (0.81)	4.38 (0.72)	30.687*	.338
Depression	3.98 (1.13)	.85	3.17 (0.91)	4.73 (0.71)	31.262*	.343
Loss of control	4.14 (1.14)	.94	3.31 (1.00)	4.92 (0.56)	30.900*	.340
Psychological Well-Being	3.45 (0.96)	.94	2.96 (0.97)	4.07 (0.75)	26.458*	.306
Positive affect	3.25 (0.97)	.94	2.60 (0.70)	3.86 (0.79)	24.019*	.286
Emotional ties	3.65 (1.18)	.74	2.96 (0.90)	4.29 (0.99)	14.234*	.192
MHI-38 total	7.38 (1.91)	.98	5.92 (1.35)	8.75 (1.21)	40.349*	.402
MHI-5^a
MHI-5 total	59.74 (19.94)	.97	44.62 (14.52)	73.94 (12.49)	36.653*	.379

CG: clinical group; NCG: nonclinical group; ANCOVA: analysis of covariance; MANCOVA: multivariate analysis of covariance; MHI: Mental Health Inventory; α: Cronbach’s alpha.

^aRecoded values in a scale of 0 to 100.

*p < .001.

Scales of Psychological Well-Being

The SPWB scale (Ryff, 1989) is an 84-item self-report scale. Each item is a sentence that requires a response on a six-point scale; each point is associated with the agreement/disagreement level of the behaviors, feelings, or thoughts referred on the sentence in question. The instrument provides the following six psychological well-being dimensions that are theoretically based: Autonomy, Environmental Mastery, Personal Growth, Positive Relations with Others, Purpose in Life, and Self-Acceptance. The Portuguese version also provides a global measure (Novo, 2003). Higher scores mean a higher level of well-being in general and in each measure indicate a higher level in the specific dimension. The internal consistency coefficients of the original version range from .81 to .88 (Ryff, 1989). The Portuguese version (Novo, 2003) showed .75 to .86 Cronbach’s alpha values for the partial measures and .93 for the global measure.

Minnesota Multiphasic Personality Inventory 2

The MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) is a 567-item self-report inventory that aims for the evaluation and characterization of personality and psychopathology. We used the experimental Portuguese version (Silva, Novo, Prazeres, & Pires, 2006). The test provides several final measures. We considered the five validity scales that are used to determine the subject’s test-taking attitude and to identify an invalid profile (VRIN: Variable Response Inconsistency; TRIN: True Response Inconsistency; L: Lie; F: Infrequency; and K: Correction), and eight clinical scales which reflect different psychological characteristics (Hs: Hypochondriasis; D: Depression; Hy: Hysteria; Pd: Psychopathic deviate; Pa: Paranoia; Pt: Psychasthenia; Sc: Schizophrenia; and Ma: Hypomania) (Graham, 2012).

Statistical analysis

MHI-38 and MHI-5’s validity was examined through two criteria: an external clinic criterion—provided by a previous identification of persons with and without mental disorder—and an empirical criterion—provided by the scores of the instruments used as reference standard, namely, the MMPI-2 for assessment of pathological distress and the SPWB for psychological well-being. These methods allowed to identify accuracy measures of the MHI-38 and MHI-5: (a) Sensibility (Se)—identify the probability of people who have the attribute who are detected by the index test, that is, a true positive; (b) Specificity (Sp)—identify the probability of people without the attribute who are correctly labeled by the test, that is, a true negative; (c) AUC—the area under the receiver operating characteristic (ROC) curve (i.e., index of the amount of diagnostic information supplied by the screening measure); and (d) the Youden’s Index (i.e., a single statistic that can capture the performance of a dichotomous diagnostic test, not affected by the attribute prevalence, providing their maximum potential effectiveness and summarizing sensitivity and specificity).

These methods also permitted to find optimal cutpoints because since the MMPI and the SPWB are robust instruments, as well as the clinical cases identified previously by mental health professionals, and they constitute “gold standards” (i.e., a benchmark or a scale which can classify accurately people as a case or a non-case, making, ideally, no misclassifications). This is essential because the indexes mentioned beforehand are linked to the cutpoint chosen, that is, for each one of the possible cutpoints elected for the measure under investigation, there is an associated pair of diagnostic sensitivity and specificity, meaning that as the cutpoint decreases, the sensitivity decreases, inasmuch the specificity increases (Kelly, Dunstan, Lloyd, & Fone, 2008) and a gold standard will allow to find the most suitable cutpoint.

In the case of the MMPI-2, the elevation of the profile was used as criterion. Elevations (T ≥ 65) on three or more scales were considered profiles of clinical relevance, that is, indicative of a pathological condition (Graham, 2012), while all other profiles were considered as indicators of a nonpathological condition.

In the case of SPWB, the value considered as indicative of well-being was a SPWB_TOTAL’s raw score ≥359, mean reported in a study with a Portuguese adult large sample (Novo, 2003); lower scores were considered as not indicative of well-being.

These reference standards—(a) CG and NCG and (b) MMPI-2 profiles (pathological/non-pathological) and the SPWB_TOTAL (high/low levels of well-being)—made possible the identification of the cutpoints for MHI-38_TOTAL, MHI-38_PD, and MHI-38_PWB and for MHI-5 _TOTAL. Different ROC curves were constructed for them and accuracy indexes were calculated (Se and Sp) using the cutpoints found previously. For the MHI-38_TOTAL, using two reference standards, we considered: (a) all participants in the CG and with pathological MMPI-2 profiles scoring below the MHI-38_TOTAL’s optimal cutpoint (i.e., with low levels of mental health) are considered True Negatives and (b) all participants from the NCG or with normal MMPI-2 profiles scoring higher than the MHI-38_TOTAL’s optimal cutpoint (i.e., high levels of mental health) are considered True Positives. In the case of the MHI-38_PWB: (a) all cases which are below the cutpoint in SPWB_TOTAL and below the MHI-38_PWB’s optimal cutpoint are considered True Negatives (i.e., low levels of well-being in the reference standard and low levels in the index test) and (b) all cases that are above the SPWB_TOTAL cutpoint and above MHI-38_PWB’s optimal cutpoint are considered True Positives (i.e., high levels of well-being in the reference standard and high levels in the index test). For the MHI-38_PD, the MMPI-2 profiles were used as the reference standard: the same rational was used as it was explained for the MHI-38_TOTAL, but the optimal cutpoint in this case was of the MHI-38_PD. Finally, for the MHI-5_TOTAL, the Clinical Condition was used as the reference standard and the rational was the same as for the MHI-38_TOTAL, but the optimal cutpoint was of the MHI-5_TOTAL.

Results

Multivariate analysis of covariance (MANCOVA) (MHI-38) and analysis of covariance (ANCOVA) (MHI-5) analysis showed, after controlling for sex and education variables, that there were significant effects between the CG and NCG in the different MHI-38 and MHI-5 measures (MHI-38: F(5, 56) = 8.110, p < .000, Wilks' Λ = .580, partial η² = .420; MHI-5: F(1, 60) = 36.653, p < .000, partial η² = .379). When compared to NCG, the CG obtained significantly lower scores on the different MHI’s subscales, which constitute evidence of empirical or criterion validity (see Table 2). Given the high range of ages, the MANCOVA and ANCOVA were conducted again, with the difference that this variable (age) was also controlled: the effects between the CG and NCG in the different MHI-38 and MHI-5 measures were likewise significant (MHI-38: F(5, 55) = 7.002, p < .000, Wilks’ Λ = .611, partial η² = .389; MHI-5: F(1, 59) = 31.821, p < .000, partial η² = .350).

With the aforementioned reference standards, for the three measures, we found areas under the ROC curve (AUC’s) higher than .86, which means the test, in both versions (MHI-38 and MHI-5), has an excellent ability to accurately distinguish persons with clinical complaints and/or MMPI-2’s pathological profiles from those who do not have clinical complaints and/or have MMPI-2 “normal” profiles, because an AUC ≥ 0.80 is indicative of a useful screening instrument (Holmes, 1998). Moreover, all the cutpoints pointed out by the Youden’s Index showed high values (higher than 73%) of sensitivity and specificity (see Table 3 and Appendix 1).

Table 3.

Summary of AUC, sensitivity and specificity given by the ROC Curves analysis.

	Clinical condition				MMPI-2 profiles				SPWB_Total
MHI	Cutpoint^a	AUC	Se	Sp	Cutpoint^a	AUC	Se	Sp	Cutpoint^a	AUC	Se	Sp
38_TOTAL	7.19	.945	93.9	83.9	7.19	.904	96.7	79.4	6.40	.946	81.6	100
38_PWB	3.64	.900	72.7	96.8	3.64	.856	73.3	91.2	3.08	.926	81.6	100
38_PD	3.89	.930	93.9	77.4	3.89	.900	96.7	73.5	3.57	.903	79.6	100
5_TOTAL^b	53	.931	90.9	80.7	53	.891	93.3	76.5	53	.904	73.5	100

MHI: Mental Health Inventory; MMPI: Minnesota Multiphasic Personality Inventory; ROC: receiver operating characteristic; AUC: Area under the curve; SPWB: Scales of Psychological Well-Being; Se: Sensibility; Sp: Specificity.

^aOptimal threshold given by the Youden’s Index.

^bRecoded values in a scale of 0 to 100.

With all the criteria used for the different scales and subscales, MHI-38_TOTAL, the MHI-38_PWB, MHI-38_PD, and MHI-5_TOTAL all showed values of Sensibility and Specificity higher than 70% (see Table 4).

Table 4.

MHI-38 and MHI-5’s accuracy indexes using clinical condition, MMPI-2’s clinical profiles and SPWB_Total mean criteria.

Criteria	Values		Accuracy measures^a
Criteria	Values		Se	Sp
	MHI-38_TOTALⁱ
Clinical condition	≥7.19	<7.19
NCG (n = 33)	31^b	2^c	.94	.80
CG (n = 31)	5^d	26^e	.94	.80
MMPI-2—clinical profiles
Normal (n = 30)	29^b	1^c	.97	.80
Pathological (n = 34)	7^d	27^e	.97	.80
	MHI-38_PWBⁱⁱ
SPWBTotal^f	≥3.64	<3.64
High (n = 34)	30^b	4^c	.88	.70
Low (n = 30)	9^d	21^e	.88	.70
	MHI-38_PDⁱⁱⁱ
MMPI-2—clinical profiles	≥3.89	<3.89
Normal (n = 30)	29^b	1^c	.97	.70
Pathological (n = 34)	9^d	25^e	.97	.70
	MHI-5_TOTAL^iv
Clinical condition	≥53	<53
NCG (n = 33)	32^b	1^c	.97	.70
CG (n = 31)	10^d	21^e	.97	.70

MHI: Mental Health Inventory; MMPI: Minnesota Multiphasic Personality Inventory; SPWB: Scales of Psychological Well-Being; Se: Sensibility; Sp: Specificity; CG: clinical group; NCG: nonclinical group.

^aSe (Sensibility) = a/(a + c); Sp (Specificity) = d/(b + d).

^bTrue Positives.

^cFalse Negatives.

^dFalse Positives.

^eTrue Negatives.

^fScales of Psychological Well-Being (PWB_TOTAL); i – Mental Health Inventory-38: Total; ii – Mental Health Inventory-38: Psychological Well-Being Scale; iii - Mental Health Inventory-38: Psychological Distress Scale; iv – Mental Health Inventory-5: Total.The values that are in bold are the optimal cutpoints indicated in table 3.

Discussion

This study suggests that the MHI’s Portuguese version, in MHI-38 and MHI-5’s formats, constitutes a valid tool in mental health screening, demonstrating good psychometric properties in the sample studied. Its use appears to be especially useful to differentiate people with low levels of mental health (i.e., with probable psychopathology) from those with high levels of mental health (i.e., with no probable psychopathology): with all the criteria used (external clinical criterion as a reference standard, i.e., people previously evaluated by a psychiatry and/or psychologist as having a mental condition or not; and empirical reference standard, i.e., MMPI-2 and SPWB scores), we found high values in the different measures of accuracy of the MHI’s global measures.

From the ROC curve analysis, a 7.19 cutpoint is suggested for the MHI38_TOTAL. For the MHI-38_PWB, a cutpoint of ≥3.64 is proposed; in turn, for the MHI-38_PD, a cutpoint of ≥3.89 is recommended. These cutpoints are the ones that best combine values of sensitivity and specificity. However, since a perfect test, with no misclassifications, that can completely discriminate subjects with and without the attribute that is evaluating, does not exist, depending on the situation, sometimes it is better to increase or decrease the cutpoint chosen. If higher levels of sensitivity are desired (i.e., if you want to identify all subjects with mental health), the cutpoint should be smaller. But, if higher levels of specificity are desired, to more effectively reduce the false-negative cases, that is, pathological or clinical cases, a higher cutpoint is recommended. Considering that in primary care, where it is essential to detect most people who are suffering from psychological distress and mental disturbance and not to miss a possible clinical case, it will be more useful to use a higher cutpoint. In this way, we can identify more cases that must benefit of posterior evaluation with diagnostic purpose.

With respect to the MHI-5, it is proposed as indicator of general mental health a cutpoint of >53 (in a scale 0 to 100). Nevertheless, for clinical purposes, it might be better to use a higher cutpoint, since most of the participants from the clinical sample were inpatients, which possible led to worst results. In fact, comparing with other studies that proposed cutpoints for specific clinical conditions with other populations, the optimal cutpoint found in this study is lower. In the study of Means-Christensen et al. (2005), they demonstrated that a cutpoint of 23 or less (corresponding to 72 or less on 0–100 scale) can identify patients that may benefit from a more thorough evaluation for panic disorder or major depression specifically. Rumpf, Meyer, Hapke, and John (2001) suggested similar cutpoints, depending on the diagnostic groups: 60 for mood disorders, 70 for anxiety disorders, and 65 as the most appropriate as a single cutpoint for all groups. Beukel et al. (2012) suggested an optimal cutpoint of 70 for the purposes of screening applications.

As said earlier, since the participants were psychiatric patients, some in an acute state and taking medication can cause the cutpoint to be lower. This can affect the optimal cutpoint chosen because we used the Youden Index, which summarizes the sensitivity and specificity of the test, this meaning that it considers the proportion of true positives in relation to the proportion of true negatives or vice versa. Since the participants in our CG scored quite low values in the MHI, this also means that our true negatives have lower levels than perhaps the samples gathered in the studies mentioned beforehand that did not recruited them in places specialized in the treatment of psychological diseases. This means that, in our case, a person to be considered a true positive simply needs to score values higher than those shown by our clinical sample. Since these values were quite low, because of the reasons mentioned, means that a low cutpoint will certainly identify true negatives and a higher one may no longer. On the other hand, in the other studies, the participants who were considered true negatives did not show such low values in MHI, meaning that their cutpoints will be higher (because the bigger score in MHI, the better mental health the person have), which will allow them to consider a true positive only if a person scores high. It should be noted that this difference in values scored in the MHI could also be due to the living and health conditions. On the countries where those samples were gathered, namely Germany and Holland, that are better quality of life and support to the health than in Portugal. The general health investment in Portugal is about 30% lower than the European Union average and the prevalence of psychiatric disorders is 43%, the second highest in European Union (Xavier, M., et al., 2013).

To our knowledge, this is the first study to indicate cutpoints for the MHI-38, which is important in clinical contexts, since a questionnaire with more items allows a bigger qualitative information about the person who takes it, which in turn will allow for more appropriate options at the referral level as it provides more specific indicators of depression, anxiety, and emotional/behavioral control, or of the reduction of positive affect and of emotional ties. Besides that, it gives more access points to explore problematics in the beginning of treatment if necessary. MHI-5, in turn, provides a useful tool for research. This short version could be valuable for very specific cases of primary health care, so physicians can refer patients for further evaluation.

Therefore, the use of MHI can have an impact both in the early detection of psychopathology and in the monitoring of the course of treatment. This is particularly relevant as mental problems or disorders have become the central cause of disability and one of the major causes of morbidity and premature death in the world. In addition, most people with mental health problems first seek help in primary care; however, due to time constraints, 50% of cases of mental disorder are not identified by general practitioners (Xavier, Baptista, Mendes, Magalhães, & Caldas-de-Almeida, 2013).

There are some limitations inherent to this study, so it is advisable to continue the research in this area. Since measures of diagnostic accuracy are very sensitive to the characteristic of the population in which the test is evaluated (Šimundić, 2009), we need more studies with larger samples and with different levels and types of psychological distress.

The small sample size is a limitation, although it should be noted that our sample has both sexes represented, is diversified in terms of geography, age, schooling, and employment status. Also, because we used the Youden J statistic it may eventually outweigh the limitation of the reduced sample size, since this index is not affected by the attribute prevalence of the condition in the population studied.

Nevertheless, the sample size might not be representative of the different levels and types of psychological distress and well-being in the general population. This limitation leads to a more specific one: the clinical sample is mainly composed of people with high levels of pathology with a disparity of diagnosis. To better clarify the role of each type of pathology, future investigations with populations with evenly distributed severity levels and pathology types should clarify the suitability of the MHI to different contexts and increase confidence in the generalization of these results. Would also be interesting to evaluate the prevalence of mental health levels in different populations, to intervene in a more specific way in a clinical context. It would also be important to conduct predictive validity studies of the measures examined here.

Replication of this study with people who show moderate levels of psychological distress and are not undergoing treatment, to perceive the sensitivity of the instrument to people not yet identified as possessing a clinical chart, as well as to investigate different cutpoints for different psychopathologies would also be interesting. Although it is arguable that mental health should only have one cutpoint to address a general psychopathology factor, since it is a dimensional concept, evidence suggests that many disorders are comorbid, recurrent/chronic, and exist on a continuum with normal-range functioning. Also, not a single mental disorder has ever been established as a distinct, episodic, discrete categorical condition and that most common mental disorders are unified by a single psychopathology dimension represents lesser-to-greater severity of psychopathology (Caspi et al., 2014; Kotov et al., 2017). Imposing a categorical terminology on naturally dimensional phenomena leads to a considerable loss of information. So, replicating this study with a larger sample, with different types of psychopathology, comorbidities, and severity levels could be more advisable and useful.

The imbalance between female and male frequency is another limitation. Evidence show that the prevalence of disturbances and levels of well-being is differentiated for each of the sexes. Therefore, the greater number of women observed in this study may bias the results of it. Thus, future studies may increase validity evidence by using more gender-balanced samples. Likewise, future research may represent the diversity of socioeconomic status, education, among other demographic variables and consider their influence on the results.

Appendix 1

Optimal cut-off points for the MHI-38 and MHI-5.

MHI-38_TOTAL			MHI-38_PD			MHI-38_PWB			MHI-5_TOTAL
	Se (%)	Sp (%)		Se (%)	Sp (%)		Se (%)	Sp (%)		Se (%)	Sp (%)
Clinical condition
≥2.74	100	0	≥1.2	100	0	≥1.21	100	0	≥17	100	0
>4.31	100	12.9	>2.1	100	12.9	>1.55	100	9.68	>23	100	10
>5.67	100	38.71	>2.93	100	38.71	>2.56	96.97	25.81	>33	100	29
>5.96	96.97	48.39	>3.08	100	51.61	>2.86	93.94	54.84	>40	100	48
>6.4	96.97	74.19	>3.48	93.94	70.97	>3.15	90.91	70.97	>47	97	58
>7.19	93.94	83.87	>3.89	93.94	77.42	>3.35	84.85	77.42	>53	91	81
>7.52	84.85	87.1	>4.05	84.85	83.87	>3.42	78.79	80.65	>60	79	87
>8.04	69.7	96.77	>4.2	78.79	90.32	>3.64	72.73	96.77	>67	67	94
>8.61	60.61	100	>4.51	60.61	96.77	>3.88	51.52	100	>73	58	100
>9.29	36.36	100	>4.73	51.52	100	>4.59	27.27	100	>80	30	100
>10.01	12.12	100	>5.41	12.12	100	>5.09	6.06	100	>87	9	100
>10.94	0	100	>5.69	0	100	>5.45	0	100	>90	0	100
MMPI-2 Profiles
≥2.74	100	0	≥1.2	100	0	≥1.21	100	0	≥17	100	0
>4.31	96.67	8.82	>2.1	96.67	8.82	>1.55	96.67	5.88	>23	96.67	5.88
>5.67	96.67	32.35	>2.93	96.67	32.35	>2.56	96.67	23.53	>33	96.67	23.53
>5.96	96.67	44.12	>3.08	96.67	44.12	>2.86	93.33	50	>40	96.67	41.18
>6.4	96.67	67.65	>3.48	96.67	67.65	>3.15	90	64.71	>47	96.67	52.94
>7.19	96.67	79.41	>3.89	96.67	73.53	>3.35	83.33	70.59	>53	93.33	76.47
>7.52	86.67	82.35	>4.05	86.67	79.41	>3.42	80	76.47	>57	83.33	76.47
>8.04	70	91.18	>4.2	80	85.29	>3.64	73.33	91.18	>63	76.67	82.35
>8.61	63.33	97.06	>4.51	63.33	94.12	>3.88	53.33	97.06	>70	63.33	91.18
>9.29	36.67	97.06	>4.73	53.33	97.06	>4.59	26.67	97.06	>77	50	97.06
>10.01	10	97.06	>5.41	13.33	100	>5.09	3.33	97.06	>83	23.33	97.06
>10.94	0	100	>5.69	0	100	>5.45	0	100	>90	0	100
SPWB_TOTAL
≥2.74	100	0	≥1.2	100	0	≥1.21	100	0	≥17	100	0
>4.31	97.96	20	>2.1	95.92	13.33	>1.55	97.96	13.33	>23	95.92	6.67
>5.67	93.88	60	>2.93	91.84	53.33	>2.56	97.96	53.33	>33	93.88	40
>5.96	91.84	80	>3.08	89.8	73.33	>2.86	85.71	80	>40	89.8	66.67
>6.4	81.63	100	>3.57	79.59	100	>3.08	81.63	100	>47	85.71	80
>7.19	73.47	100	>3.89	77.55	100	>3.35	71.43	100	>53	73.47	100
>7.52	65.31	100	>4.05	67.35	100	>3.42	65.31	100	>57	67.35	100
>8.04	48.98	100	>4.2	59.18	100	>3.64	51.02	100	>63	59.18	100
>8.61	40.82	100	>4.51	42.86	100	>3.88	34.69	100	>70	44.9	100
>9.29	24.49	100	>4.73	34.69	100	>4.59	18.37	100	>77	32.65	100
>10.01	8.16	100	>5.41	8.16	100	>5.09	4.08	100	>83	16.33	100
>10.94	0	100	>5.69	0	100	>5.45	0	100	>90	0	100

Se: Sensitivity; Sp: Specificity; MMPI-2: Minnesota Multiphasic Personality Inventory 2; SPWB_TOTAL: Scales of Psychological Well-Being; MHI-38_TOTAL: Mental Health Inventory-38: Total; MHI-38_PWB: Mental Health Inventory-38: Psychological Well-Being Scale; MHI-38_PD: Mental Health Inventory-38: Psychological Distress Scale; MHI-5_TOTAL: Mental Health Inventory-5: Total.

Footnotes

Article Notes

References

Beukel

T. O.

Siegert

C. E.

Dijk

Ter Wee

P. M.

Dekker

F. W.

Honig

(2012). Comparison of the SF-36 Five-Item Mental Health Inventory and Beck Depression Inventory for the screening of depressive symptoms in chronic dialysis patients. Nephrology Dialysis Transplantation, 27, 4453–4457. doi:10.1093/ndt/gfs341

Borus

J. F.

Howes

M. J.

Devins

N. P.

Rosenberg

Livingston

W. W.

(1988). Primary health care providers' recognition and diagnosis of mental disorders in their patients. General Hospital Psychiatry, 10, 317–321. doi:10.1016/0163-8343(88)90002-3

Brundtland

G. H.

(2000). Mental health in the 21st century. Bulletin of the World Health Organization, 78, 411–411. Retrieved from http://www.who.int/iris/handle/10665/57388

Butcher

Dahlstrom

W. G.

Graham

Tellegen

Kaemmer

(1989). MMPI-2: Manual for Administration and Scoring. Minneapolis, MN: University of Minnesota Press.

Caspi

Houts

R. M.

Belsky

D. W.

Goldman-Mellor

S. J.

Harrington

Israel, Moffitt

T. E.

(2014). The p factor: One general psychopathology factor in the structure of psychiatric disorders? Clinical Psychological Science, 2, 119–137. doi:10.1177/2167702613497473

Florian

Drory

(1990). Mental Health Inventory (MHI) – Psychometric properties and normative data in the Israeli population. Psychologia—Israel Journal of Psychology, 2, 26–36.

Graham

J. R.

(2012). MMPI-2: Assessing personality and psychopathology (5th ed.). New York, NY: Oxford University Press.

Heubeck

B. G.

Neill

J. T.

(2000). Confirmatory factor analysis and reliability of the Mental Health Inventory for Australian adolescents. Psychological Reports, 87, 431–440. doi:10.2466/PR0.87.6.431-440

Holmes

W. C.

(1998). A short, psychiatric, case-finding measure for HIV seropositive outpatients: Performance characteristics of the 5-item mental health subscale of the SF-20 in a male, seropositive sample. Medical Care, 36, 237–243. doi:10.1097/00005650-199802000-00012

10.

Kelly

M. J.

Dunstan

F. D.

Lloyd

Fone

D. L.

(2008). Evaluating cutpoints for the MHI-5 and MCS using the GHQ-12: A comparison of five different methods. BMC Psychiatry, 8, 10. doi:10.1186/1471-244X-8-10

11.

Kotov

Krueger

R. F.

Watson

Achenbach

T. M.

Althoff

R. R.

Bagby

R. M.

, … Zimmerman

(2017). The Hierarchical Taxonomy of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126, 454. doi:10.1037/abn0000258

12.

Means-Christensen

A. J.

Arnau

R. C.

Tonidandel

A. M.

Bramson

Meagher

M. W.

(2005). An efficient method of identifying major depression and panic disorder in primary care. Journal of Behavioral Medicine, 28, 565–572. doi:10.1007/s10865-005-9023-6

13.

Novo, R. F. (2003). Para além da eudaimonia: O Bem-Estar Psicológico em mulheres na idade adulta avançada [Behond Eudaimonia: The Psychological Well-Being in women in advanced adulthood]. Lisboa, Portugal: Centro de Investigação em Psicologia, Universidade de Lisboa.

14.

Novo, R. (2004, September). O Bem-Estar subjetivo no contexto da saũde mental em adultos e idosos. [The subjective well-being in the context of mental health in adults and the elderly]. II Congresso Hispano-Português de Psicologia, Faculdade de Psicologia e de Ciências da Educação de Lisboa, Lisboa, Portugal.

15.

Ostroff

J. S.

Woolverton

K. S.

Berry

Lesko

L. M.

(1996). Use of the Mental Health Inventory with adolescents: A secondary analysis of the Rand Health Insurance Study. Psychological Assessment, 8, 105–107. doi:10.1037/1040-3590.8.1.105

16.

Rosenthal

R. H.

Akiskal

H. S.

(1985). Mental status examination. In Hersen

Turner

(Eds.), Diagnostic interviewing (pp. 25–52). Boston, MA: Springer. doi:10.1007/978-1-4615-6826-1_2

17.

Rumpf

H. J.

Meyer

Hapke

John

(2001). Screening for mental health: Validity of the MHI-5 using DSM-IV Axis I psychiatric disorders as gold standard. Psychiatry Research, 105, 243–253. doi:10.1016/S0165-1781(01)00329-8

18.

Ryff

C. D.

(1989). Happiness is everything, or is it? Explorations on the meaning of psychological well-being. Journal of Personality and Social Psychology, 57, 1069. doi:10.1037/0022-3514.57.6.1069

19.

Silva, M. & Novo, R. (2002). Inventário de Saúde Mental: Adaptação portuguesa para adultos de idade avançada [Mental Health Inventory: Portuguese adaptation for elderly]. Lisboa, Portugal: Centro de Investigação em Psicologia, Universidade de Lisboa.

20.

Silva, D., Novo, R., Prazeres, N., & Pires, R. (2006). Inventário Multifásico da Personalidade-2: Versão Experimental Portuguesa [Minnesota Multiphasic Personality Inventory-2: Portuguese adaptation]. Lisboa, Portugal: Centro de Investigação em Psicologia, Universidade de Lisboa.

21.

Šimundić

A. M.

(2009). Measures of diagnostic accuracy: Basic definitions. EJIFCC, 19, 203.

22.

Stewart

A. L.

Hays

R. D.

Ware

J. E.

(1988). The MOS short-form general health survey: Reliability and validity in a patient population. Medical Care, 26, 724–735. doi:10.1097/00005650-198807000-00007

23.

Thorsen

S. V.

Rugulies

Hjarsbech

P. U.

Bjorner

J. B.

(2013). The predictive value of mental health for long-term sickness absence: The Major Depression Inventory (MDI) and the Mental Health Inventory (MHI-5) compared. BMC Medical Research Methodology, 13, 115. doi:10.1186/1471-2288-13-115

24.

Vaingankar

J. A.

Subramaniam

Abdin

Picco

Chua

B. Y.

Eng

G. K.

Chong

S. A.

(2014). Development, validity and reliability of the short multidimensional positive mental health instrument. Quality of Life Research, 23, 1459–1477. doi:10.1007/s11136-013-0589-0

25.

Veit

C. T.

Ware

J. E.

(1983). The structure of psychological distress and well-being in general populations. Journal of Consulting and Clinical Psychology, 51, 730. doi:10.1037/0022-006X.51.5.730

26.

Von Korff

Ormel

Katon

Lin

E. H.

(1992). Disability and depression among high utilizers of health care: A longitudinal analysis. Archives of General Psychiatry, 49, 91–100. doi:10.1001/archpsyc.1992.01820020011002

27.

Ware

J. E.

Jr. Gandek

(1994). The SF-36 Health Survey: Development and use in mental health research and the IQOLA Project. International Journal of Mental Health, 23, 49–73. doi:10.1080/00207411.1994.11449283

28.

Ware

J. E.

Jr. Sherbourne

C. D.

(1992). The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection. Medical Care, 473–483. Retrieved from https://www-jstor-org.web.bisu.edu.cn/stable/3765916

29.

World Health Organization. (2001). The World Health Report 2001: Mental health: New understanding, new hope. Geneva, Switzerland: Author.

30.

World Health Organization. (2005). Promoting mental health: Concepts, emerging evidence, practice. Geneva, Switzerland: Author.

31.

Xavier

Baptista

Mendes

J. M.

Magalhães

Caldas-de-Almeida

J. M.

(2013). Implementing the World Mental Health Survey Initiative in Portugal–rationale, design and fieldwork procedures. International Journal of Mental Health Systems, 7(1), 19. doi:10.1186/1752-4458-7-19