|Year : 2017 | Volume
| Issue : 1 | Page : 45-50
Heterogeneous group discussion to improve reliability and validity of data tool: A global mental health assessment tool – Primary care version study
Vaishali D Tendolkar1, Prakasha Behere2, Zahir Quazi3, Abhai Gaidhane3
1 Department of Mental Health Nursing, SRMM College of Nursing, Wardha, Maharashtra, India
2 Department of Psychiatry, Dr D. Y. Patil Education Society (Deemed University), Kolhapur, Maharashtra, India
3 Department of Community Medicine, Jawaharlal Nehru Medical College, Wardha, Maharashtra, India
|Date of Web Publication||25-Jul-2017|
Vaishali D Tendolkar
Mental Health Nursing, SRMM College of Nursing, Sawangi (Meghe) Wardha, Maharashtra
Source of Support: None, Conflict of Interest: None
Background: Mental illness constructs are not universal and are likely to be burdened with ethnocentric conceptualization. Different terminology is used to refer to the same aspects of the translation process, making it harder to achieve clarity. Heterogenous focus group discussion is particularly suited for obtaining several perspectives about the same topic. The benefits of heterogenous focus group discussion include gaining insights into people's shared understandings of a concept and the ways in which individuals are influenced by others in a group situation. Objectives: 1) To find out reliability of GMHAT/PC Marathi. 2. To assess the validity of the GMHAT/PC Marathi for the diagnosis of psychiatric illness. Research Design: cross sectional study Sample: Persons aged 15 yrs and above, reporting to psychiatry units. Sample size: 500. Material: Global Mental Health Assessment Tool – Primary care Marathi. Result: The Cronbach's alpha coefficient estimated to 0.854 with the reliability coefficient of r=0.921. The 'k' value is 0.744 which indicates that the tool has good diagnostic accuracy. The overall sensitivity of the tool was found to be 96.51% (CI = 94.21% to 98.08%) (Assumed sensitivity = 90%). The estimated specificity of the tool is 74.75% (CI =65.02% to 82.94%). Conclusion: GMHAT/PC -Marathi is a valid, quick and comprehensive assessment and diagnoses of mental disorders for Marathi speaking people. GMHAT/PC- Marathi in psychiatric settings appears to have very good sensitivity and specificity. The mean duration of the interview was around 13.4 minutes which makes it feasible for routine use in the outpatient departments.
Keywords: Global Mental Health Assessment Tool – Primary Care Version, heterogeneous group discussion, reliability, validity
|How to cite this article:|
Tendolkar VD, Behere P, Quazi Z, Gaidhane A. Heterogeneous group discussion to improve reliability and validity of data tool: A global mental health assessment tool – Primary care version study. J Datta Meghe Inst Med Sci Univ 2017;12:45-50
|How to cite this URL:|
Tendolkar VD, Behere P, Quazi Z, Gaidhane A. Heterogeneous group discussion to improve reliability and validity of data tool: A global mental health assessment tool – Primary care version study. J Datta Meghe Inst Med Sci Univ [serial online] 2017 [cited 2019 Jun 16];12:45-50. Available from: http://www.journaldmims.com/text.asp?2017/12/1/45/211582
| Introduction|| |
Mental health is a national wealth and must be preserved, protected, and promoted for the growth and development of societies and nations.
Mental health problems are one of the leading causes of disability in the world. Early and accurate detection of mental health problems, followed by an appropriate treatment and management plan, may help to reduce the global burden on health and social care systems caused by mental disorders.,
A large proportion of people with mental disorders worldwide fail to receive appropriate help in spite of developments in new treatments for mental illnesses (psychological and social as well as medicinal) and as a consequence suffer in silence. In developed countries, this may be due to the stigma attached to mental illness, leading to reluctance to ask for help for any kind of mental health problem. Another important reason could be primary care health services providing inadequate training and poor skills for detecting and treating people with mental health problems.,
Mental illness constructs are not universal and are likely to be burdened with ethnocentric conceptualization. Different terminologies are used to refer to the same aspects of the translation process, making it harder to achieve clarity. Poorly translated instruments threaten the validity of research data and the safe aggregation of global data sets. There is no practical means to assess the validity and conceptual equivalence of new or existing translations except by post hoc psychometric validation. While most practitioners agree that the overall aim of translation is to produce a new language version, which is both conceptually equivalent with the original and relevant to the new target culture, the actual methods employed differ.,
Focus group interviewing is particularly suited for obtaining several perspectives about the same topic. The benefits of focus group research include gaining insights into people's shared understandings of everyday life and the ways in which individuals are influenced by others in a group situation.,
The benefits to participants of focus group research should not be underestimated. The opportunity to be involved in decision-making processes (Race et al. 1994), to be valued as experts, and to be given the chance to work collaboratively with researchers (Goss and Leinbach 1996) can be empowering for many participants. If a group works well, it may explore solutions to a particular problem as a unit (Kitzinger 1995), rather than as individuals.,
There is a growing recognition both in developed and developing countries that comprehensive mental health services cannot be provided without the active involvement of the primary care health teams. The role of primary care health professionals is crucial in (a) early detection of mental disorders, including psychotic illness, (b) management of common mental disorders such as depression, (c) getting advice on diagnosis and management of patients with mental illness from specialists, and d) providing care (specially for physical health) to people with severe and enduring mental illness in close liaison with specialist mental health professionals/teams. A proper assessment and identification of mental health problems at primary care level is, therefore, essential in providing appropriate care to people suffering from mental disorders in any community.,
The Global Mental Health Assessment Tool – Primary Care Version (GMHAT/PC) has been developed to assist health professionals to make a quick and comprehensive standardized mental health assessment. It is a computerized clinical assessment tool developed to assess and identify a wide range of mental health problems in the primary care. It generates a computer diagnosis, a symptom rating, a self-harm risk assessment, and a referral letter.,
Psychiatrists and other mental health workers usually find it difficult to use an appropriate word substitute while conversing with the clients. At times there is thin line of difference in the meaning attached to the words by people from different geographical, cultural, and ethnic backgrounds. While translating the tools for data collection, it is therefore mandatory that the tool must be adapted culturally for the target population. This not only improves the quality of data collected but also increases the credibility of the study findings.,
Researchers adopt various methods for translating the tool and adapting them best to the local population keeping in mind the data needs. The standard method of translation and cultural adaptation includes forward translations by two or more bilingual translators independently and reverse or back translation of the translated matter into source language by two or more bilingual experts other than forward translators, independent of the one another. Some also include opinion of panel of experts, and in rare conditions, focus group discussions and focused interviews are conducted. Focus group discussion, when used, is mostly homogenous focus group discussion. Heterogeneous focus group discussion is never given thought while translating a tool for data collection. However, a heterogeneous group can yield good word substitutes, more acceptable to the target population. This method can also ensure the acceptability of the words used. It also provides for multiple word usage in translation so that the words substitutes are readily available for the therapist/assessor and thereby improves the reliability and validity of the tool.
The study aims at assessing the role of heterogeneous focus group discussion while translating and culturally adapting GMHAT/PC into Marathi through assessing the validity of GMHAT/PC Marathi for diagnosis of psychiatric illness among Marathi speaking population.
Objectives of the study
- To find out reliability of GMHAT/PC Marathi
- To assess the validity of the GMHAT/PC Marathi for the diagnosis of psychiatric illness.
| Methodology|| |
The study was conducted in the outpatient department (OPD) and inpatient department (IPD) of hospitals with a psychiatric unit in Wardha, Nanded and Barshi. The three selected hospitals are representative of three geographical and culturally as well as linguistically diverse regions of Maharashtra. They cater to the health-care needs of population belonging to varied cultural and socioeconomic backgrounds from rural and urban communities.
The study design was cross-sectional study.
Persos in the age group of 15 years and above, reporting to psychiatric OPD and admitted to psychiatric wards, and of both genders who gave consent to participate in the study are selected purposively. Uncooperative psychotic clients were excluded from the study.
The sample size was five hundred.
The material used for data collection is GMHAT/PC Marathi Version (Computer Assisted GMHAT-PC-Marathi). This tool covers diagnosis of wide range of mental disorders, provides guidelines for management, and can provide a referral letter to the psychiatrist. The tool is user friendly. It provides for detailed yet focused health interview of the patient. The assessment program starts with basic instructions giving details of how to use the tool and rate the symptoms. The following screens consist of a series of questions leading to a comprehensive yet quick mental state assessment, focusing sequentially on following the symptoms or problems. Primarily the tool is developed and translated into different languages for its use by other health professionals including nurses working in primary care setting. It is a very handy tool and is free flowing in nature. The diagnostic program takes account of severity of symptoms (moderate to severe). It also generates alternative diagnoses and comorbid states based on the presence of symptoms related to other disorders. In addition, it includes an assessment of risk for self-harm. The program also contains management guidelines for these disorders.
The tool is translated and validated for use in seven languages so far which include Hindi and Tamil languages of India. The purpose of developing GMHAT/PC tool was to empower the health personnel other than psychiatrists to diagnose and refers the mentally ill clients for treatment.
Translation and cultural adaptation of the tool
The translation process included forward translation by investigator and two other translators independently, correction by the language expert, and heterogeneous focus group discussion followed by the reverse translation by two independent linguists and finally four focused interviews.
Process of forward translation
The tool was translated into Marathi by the investigator from original English version. It was then independently translated into Marathi from English by two bilingual persons. One of them is official translator for Sakal Group of Papers, holding the postof sub-editor cum translator and is currently working in Konkan region. His mother tongue is Marathi and holds master's degree in English. He provided valuable inputs in his translation with words of local relevance. Another translator also has Marathi as her mother tongue and holds master's degree in Marathi Language. She is working as Marathi Language teacher in one of the English medium CBSE pattern schools and has proficiency in both languages. All the three translated versions were then compared and contrasted and multiple words for same meaning were used to improve the local utility in all regions of Maharashtra. This combined version in Marathi was then corrected by third renowned bilingual expert from the Department of Translation, Rashtrabhasha Pracharini, Wardha. The corrections were then incorporated in the tool, and the tool was then presented for discussion to a focus group of heterogeneous nature.
Heterogeneous focus group discussion for social and cultural acceptability
As a part of translation process, a heterogeneous focus group discussion was arranged in the psychiatry OPD of one of the settings. Among the 16 participants, there were psychiatry residents, psychiatric nurses, patient, patients' relatives, undergraduate nursing students, psychologists, attendants (grade four employees), postgraduate students of community health nursing, psychiatric social worker, and a clerk.
Inputs for appropriate word substitutes for (1) semantic equivalence (which aims to assess whether the words have the same meaning, whether the item has more than one meaning, and whether there are grammatical errors in the translation); (2) idiomatic equivalence, which refers to assessing whether the items from the original instrument that are difficult to translate were changed into an equivalent expression that has not changed the cultural meaning of the item; (3) experiential equivalence, which refers to noting whether a particular item is applicable in the new culture and if not, replacing it with an equivalent item (no items were replaced with new ones); and (4) conceptual equivalence, which seeks to assess whether a given term or expression, even if properly translated, assesses the same aspect in different cultures were sought during the discussion. Suggestions were incorporated in the tool.
Process of backward (reverse) translation
The Marathi translated tool was then retranslated into English by two bilingual language experts. One of them is a renowned sociology professor from Nagpur and Ph.D. Guide in Department of Sociology at Rashtrasant Tukdoji Maharaj Nagpur University. He has taught sociology in English medium to the students for about 35 years before he retired. He has authored six books out of which two are in English. Another translator is working as assistant professor in Department of Psychology at a reputed higher education institute and teaches psychology in both English and Marathi medium. He is also a good orator and is actively involved in mental health awareness campaigns. Both the translators independently retranslated the tool in English from Marathi version provided to them.
The linguistic and the phonetic aspects of the translation were taken care of through reverse translation and focus group discussion wherein the group was extremely heterogeneous. In addition, four clients were interviewed in depth (focus interview) with computerized GMHAT-PC Marathi to check the correctness of the words, their meaning and their pronunciation.
Computerization of the tool
The finalized tool was then sent for computerization to the same organization in Jaipur who had earlier done the computerization of the English and Hindi version of the tool. The computerized tool in Marathi was then tested on four clients from psychiatry OPD through focused interview. Few new words were then included in the tool. The latest version of the computerized tool was used for data collection in the study. The tool is thus an essential blend of subject expertise, technological expertise, and cultural relevance.
Method of data collection
The subjects for the study were selected purposively from among those who reported to the psychiatric OPD and IPD. The subjects were explained about the study, and written informed consent was taken from them. The investigator collected data by conducting face-to-face computer-assisted structured interview using GMHAT-PC Marathi version. The psychiatrist, unaware of the GMHAT-PC's reports; then immediately after the data collection; performed the assessment of each subject. Psychiatrist's (Clinical) diagnosis was considered as “Gold standard.” The clinical diagnosis by the psychiatrist was entered in the computer software. Printed report of each subject was taken. The two diagnoses were matched in the analysis.
For the purposes of finding inter-rater reliability and feasibility for the use of the tool, 30 subjects were simultaneously rated by the investigator and another psychiatric nurse who was trained for the use of GMHAT-PC software.
Administration of the tool
All the instructions for administration and rating of the tool items were followed strictly.
The study was conducted only after the sanction by the Institutional Ethical Committee of the University. Written consent was taken from the participants before recruiting them. They were explained about the study in the language they understand, i.e. Marathi. No harm of any nature to the participants is foreseen in the study.
| Observations and Results|| |
The study aims at assessing the reliability and validity of Marathi version of GMHA/PC for diagnosis of psychiatric illness in Marathi speaking population.
The distribution of subjects according to their demographic characteristic reveals that 29.4% of the subjects were in the age group of 15–30 years, 42.4% were in the age group of 31–45 years, 21% were in the age group of 46–60 years, 6.4% were in the age group of 61–75 years, and 0.8% were above 75 years of age (Graph 1). The mean age was 45 ± 3.92 years. Among the subjects, 56.8% were males and 43.2% were females.
Reliability statistics between computer diagnosis and the clinical diagnosis (Gold standard – the psychiatrist's diagnosis) reveals that the computer diagnosis and the clinical diagnosis matched for 387 as cases (psychiatric illness) and for 74 as no cases (no mental illness). The Cronbach's alpha coefficient estimated for the given crosstabs was 0.854 with the reliability coefficient of r = 0.921 which indicates that the tool is reliable [Table 1].
The Kaiser-Meyer-Olkin value was 0.762 indicating sample adequacy and construct validity of the tool. Bartlett's test of sphericity was highly significant at 1128° of freedom and the estimated P value is 0.000. This shows that the GMHAT-PC Marathi is has very good construct validity [Table 2].
|Table 2: Kaiser-Meyer-Olkin and Bartlett's test for all items in the tool|
Click here to view
On the basis of Varimax Rotation with Kaiser Normalization, 15 factors have been extracted out of 48 variables. These 15 extracted factors explain 66.939%, i.e., up to two-third of the variability for the performance of GMHAT-PC Marathi.
The K value is 0.744 which indicates that the tool has good diagnostic accuracy. The overall sensitivity of the tool was found to be 96.51% (confidence interval [CI] = 94.21%–98.08%) (assumed sensitivity = 90%). The estimated specificity of the tool is 74.75% (CI = 65.02%–82.94%) (assumed specificity = 80%). The positive likelihood ratio = 3.82 (CI = 2.72–5.37), negative likelihood ratio = 0.05 (CI = 0.03–0.08), positive predictive value = 93.93% (CI = 91.17%–96.04%), and negative predictive value = 84.09% (CI = 74.75%–91.02%). The sensitivity and specificity values are compatible with assumed levels thus indicating diagnostic accuracy of the tool. This shows that GMHAT/PC had good face and construct validity.
The inter-rater case agreement reveals that 14 cases of rater 1 (investigator) matched with that of rater 2 (another psychiatric nurses). However, there is a mismatch of 1 case assessed by rater 1 and 2 cases assessed by the rater 2. The Cohen's kappa k = 0.800 with very good strength of association at 95% CI ranging from 0.586 to 1.000. The reliability coefficient r = 0.889 which is much higher than the acceptable level of r = 0.70 [Table 3].
|Table 3: Inter-rater case agreement statistics (both psychiatric nurses)|
Click here to view
The mean time taken for interview is 13.4 which is comparatively less for mental health assessment. This was probably possible due to multiple word synonyms that provided the interviewer with word alternatives readily available.
| Discussion|| |
The process of translation incorporates most of the mandatory elements of translation and cultural adaptation recommended by the researchers and authorities for the mental health assessment tools. The investigator in this study has gone a step further by conducting heterogeneous focus group discussion to broaden the applicability of GMHAT-PC Marathi version to most population in Maharashtra [Figure 1].
The findings of this study are supported by the findings in other studies with GMHAT-PC.
Sharma and Copeland et al. in a feasibility study among nurses reported the mean age (standard deviation) in the male group was 69.5 (10.5) years and for the female group it was 61.0 (12.3) years. The age range was 28–87 years. The mean time (standard deviation) taken for the administering GMHAT/PC by nurses in the male group was 13.3 (6.4) min, and in the female group, it was 14.9 (10.1) min. There was an excellent level of agreement between the nurses (GMHAT/PC) diagnosis and the psychiatrist diagnosis of mental illness with the correlation coefficient, i.e. Kappa 0.76 (95% CI [0.61, 0.91]). There is good sensitivity of 0.73 95% CI (0.56, 0.90) and excellent specificity of 0.98 (95% CI [0.95, 1.00]).
Krishna et al., in their study, reported the levels of agreement between the consultants and the GMHAT/PC for a diagnosis of mental illness showed a kappa value of 0.72 (95% CI 0.62–0.83). There is good sensitivity (0.77) and excellent specificity (0.96), with GMHAT/PC correctly identifying 67 out of 86 subjects with mental illness and 80 out of 83 as without. The GMHAT/PC correctly identified 20 out of 33 patients with an organic illness, and 134 out of 136 as without. The feedback of interviewers and participants was satisfactory. The average time taken was 14 min.
Sharma et al. in their study reported that patients from primary care and community psychiatric outpatient clinics and a small sample of inpatients were interviewed for 2 months using the GMHAT/PC. A proportion of patients were simultaneously rated by a psychiatrist and a general practitioner for inter-rater reliability. All patients also completed the Hospital Anxiety and Depression Scale (HAD). To conduct the interview was easy in all settings and took 10–15 min for patients who had psychiatric symptoms. The age range was 19–64 years, and the mean age was 38. Sixty-one patients (51.3%) were women and 58 (48.7%) men. Inter-rater agreement on mental state symptom groups ranged from 0.49 to 1 (kappa). The computer diagnosis correlated highly with the clinical diagnosis, and there was a good level of agreement between HAD ratings and GMHAT/PC ratings. These data suggest that the GMHAT/PC is an easy to administer computerized tool which can be used in the primary care for the standardized assessment of mental health problems.
A study with GMHAT-PC by Krishna et al. reported that 49 (60%) males and 33 (40%) females in the age range of 13–68 with a mean age of 36.5 years participated in their study. The overall mean time taken to administer GMHAT/PC was 16.3 min (median 16.5, range 5–35 min). None of the patients declined their consent to participate in the study. There is a good level of agreement between the psychologists' (GMHAT/PC) diagnoses and the psychiatrists' (clinical) diagnoses of any mental illness, Kappa 0.96, 95% CI (0.89, 1.00). There is good sensitivity (0.94) and specificity (1.0), with psychologists correctly identifying 64 out of the 65 participants diagnosed with mental illness and 16 out of 16 of those without it. The level of agreement for the diagnoses of neurotic illnesses was good; Kappa 0.90, 95% CI (0.78, 1.00). Sensitivity was 0.85 with psychologist correctly identifying 17 of the 20 participants diagnosed with neurosis. The specificity was 1.0, with the psychologists correctly identifying 62 of the 62 participants not suffering from neurosis. The level of agreement for depression shows Kappa 0.85, 95% CI (0.73, 0.97), sensitivity (0.91), and specificity (0.94) with the psychologists correctly identifying 31 of the 34 participants diagnosed by the psychiatrists as suffering from depression and 45 out of 48 of those without it. All gave positive feedback. The psychologists who interviewed patients found GMHAT/PC a very useful training tool to detect mental disorders.
However, heterogeneous focus group discussion was not conducted while translating the tool into target language in these studies.
| Conclusion|| |
The findings of the study indicate that GMHAT/PC-Marathi is a valid, quick, and comprehensive assessment and diagnoses of mental disorders for Marathi speaking people. GMHAT/PC-Marathi in psychiatric settings appears to have very good sensitivity and specificity. The mean duration of the interview was around 13.4 min which makes it feasible for routine use in the OPDs. Both patients and psychiatric nurses found the GMHAT/PC Marathi not only acceptable but also useful for quick and comprehensive standardized mental health assessment.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
World Health Organization. Mental Health Atlas 2014. France: WHO; 2014.
Sharma VK, Krishna M, Lepping P, Palanisamy V, Kallumpuram SV, Mottram P, et al.
Validation and feasibility of the Global Mental Health Assessment Tool – Primary Care Version (GMHAT/PC) in older adults. Age Ageing 2010;39:496-9.
Sharma VK, Copeland JR. Detecting mental disorders in primary care. Ment Health Fam Med 2009;6:11-3.
Bodke P, Bhosle R. Prevalence of psychiatric illness in rural Maharashtra. Medpulse Int Med J 2014;1:119-22.
World Health Organization. International comparability of health interview surveys. EUROHIS: Developing Common Instruments for Health Surveys. Nasikiv A, Gudex C, editors. Amsterdam, The Netherlands: IOS Press; 2003. p. 6.
deVijver FV, Tanzer NK. Bias and equivalence in cross-cultural assessment: An overview. Rev Eur Psychol Appl 2004;54:119-35.
Chávez LM, Canino G. Toolkit on Translating and Adapting Instruments. Cambridge, MA: Human Services Research Institute; 2005. p. 9-14.
Corfman KP. The importance of member homogeneity to focus group quality. In: Kardes FR, Sujan M, editors. NA Advances in Consumer Research. Vol. 22. Provo, UT: Association for Consumer Research; 1995. p. 354-9.
Gogtay NJ, Thatte UM, Dasgupta B, Deshpande S. Use of the WOMAC questionnaire in Mumbai and the challenges of translation and cross cultural adaptation. Indian J Med Ethics 2013;10:33-5.
Sharma VK, Lepping P, Cummins AG, Copeland JR, Parhee R, Mottram P. The Global Mental Health Assessment Tool – Primary Care Version (GMHAT/PC). Development, reliability and validity. World Psychiatry 2004;3:115-9.
Krishna M, Lepping P, Sharma VK, Copeland JR, Lockwood L, Williams M. Epidemiological and clinical use of GMHAT-PC (Global Mental Health assessment tool-Primary care) in cardiac patients. Clin Pract Epidemiol Ment Health 2009;5:7.
Sharma VK, Wilkinson G, Dowrick C, Church E, White S. Developing mental health services in a primary care setting: Liverpool Primary Care Mental Health Project. Int J Soc Psychiatry 2001;47:16-29.
Sharma VK, Jagawat S, Midha A, Jain A, Tambi A, Mangwani LK, et al.
The Global Mental Health Assessment Tool-validation in Hindi: A validity and feasibility study. Indian J Psychiatry 2010;52:316-9.
] [Full text]
[Table 1], [Table 2], [Table 3]