Measuring client-perceived quality of maternity services in rural Vietnam

Abstract

Objective. To examine the feasibility, reliability and validity of a 20-item scale for measuring perceived quality of maternity services provided at commune health centres in rural Vietnam.

Design. A survey of 200 women who gave birth in July–August 2000 and 196 pregnant women in 34 communes in Quang Xuong District, Thanh Hoa Province, Vietnam.

Main outcome measures. Inter-rater reliability, internal consistency and factor structure of the scale were examined. The associations between perceived quality and client characteristics were also investigated.

Results. The instrument had relatively good inter-rater reliability and internal consistency. Except for two items: ‘good clinical examination’ and ‘adequacy of health workers for women’s health’, the scale exhibited good agreement between the two raters, with kappa values ranging from 0.54 to 0.84. The Cronbach’s alpha coefficients for the dimensions ‘health care delivery’, ‘health facility’, ‘interpersonal aspects of care’ and ‘access to services’ were 0.72, 0.64, 0.72 and 0.33, respectively. Respondents were positive on items related to the dimensions ‘interpersonal aspects of care’ and ‘access to services’, but negative on the dimensions ‘health care delivery’ and ‘health facility’. The maternity status of clients was found to influence the perceived quality of maternity services.

Conclusions. The feasibility, reliability and validity of the instrument were established in the context of rural Vietnam. Its application in evaluating other health care programmes should be an important follow-up action for the Vietnamese government.

maternity services, multidimensional scale, quality of care, Vietnam

The assessment of quality of service has posed a challenge for improving the efficiency and effectiveness of primary health care in Vietnam, where basic health services including maternal and child health services are provided at commune health centres. Most evaluation studies deal with quality either according to the ‘technocratic’ perspective of health care professionals or from the lay perspective of clients or communities [1]. In the former perspective, services are judged to be good quality if they reach standards defined by health professionals [2–4]. In the latter, clients play a central role in defining and assessing quality of health care [5–8].

There has been some debate about using the client’s perspective in evaluation of the quality of services. While many stakeholders have viewed the client’s perspective as a meaningful indicator of health services quality, others have dismissed the views of clients as too subjective. For the latter point of view, how a client feels is important, even though the provider’s assessment of reality may be different [9], because at a minimum, the subjective assessment of quality by clients can still provide useful input to help the provider understand and establish acceptable standards of services [10]. As it is very difficult to assess a full range of evaluations, particularly negative ones [11,12], the client’s perspective has been seen as an undetachable part of health care evaluations.

Client satisfaction has been widely used in the lay measurement of quality of health services. Despite its benefits, there has been growing criticism of its measurement. Satisfaction ratings reflect the personal preferences of the client, the client’s expectations and the realities of the received care; the latter can be affected by different components of that care [13]. Satisfaction ratings, being both a measure of care and a reflection of the respondent, therefore do not reflect objective reality. To overcome this problem, some organizations emphasize the measurement of client perception instead. For example, the Joint Commission on Accreditation of Healthcare Organizations has replaced the term ‘satisfaction’ with ‘perception of service’ [14].

Client-perceived quality is a subjective, dynamic perception of the extent to which expected health care is received [15]. The advantages of perceived quality measurement have been pointed out by several authors [16]. However, most studies on client perspective of quality of services have been conducted in developed countries [17–19], and only a few reports are available for developing countries [10,20,21]. In general, the methods adopted in these studies were inadequately described, while the reliability and validity of the instruments used were seldom addressed, making them difficult to apply beyond their limited contexts [20,21].

There have been few attempts at developing a multidimensional scale to measure perceived quality of care in developing countries. In a study of Bangladesh hospitals, Andaleeb [10] investigated five dimensions of perceived quality of care: responsiveness, assurance, communication, discipline and ‘bribe money’ paid to health staff. Haddad et al. [20] developed and validated a 20-item instrument for use in Guinea, with the dimensions of health care delivery, personnel and health facility. Later, Baltussen et al. [21] adapted this scale for use in Burkina Faso, and identified via factor analysis four dimensions, namely, health personnel and conduct, adequacy of resources and services, health care delivery, and financial and physical accessibility of care. Although reliability and validity of the Haddad instrument appeared to be satisfactory for Guinea and Burkina Faso, it is still necessary to justify its application in other primary health care contexts because ‘the presentations of quality are based, in part, on constructs that belong to a specific context or culture’ [20].

The aim of this paper is to assess the feasibility, reliability and validity of the 20-item scale of Haddad et al. [20] for measuring the client-perceived quality of maternity services provided at commune health centres in rural Vietnam. By adapting the original scale of Haddad et al. [20] to the Vietnamese language and culture, we intended to obtain a version that is conceptually equivalent to the original instrument, whilst being comprehensible to the rural community of Vietnam.

Methods

Study design

Similarly to the study conducted in Burkina Faso, we initially conducted a qualitative study with 12 in-depth interviews and six focus group discussions, to assess whether the 20-item scale of Haddad et al. [20] was relevant to rural Vietnam. Participants in the qualitative study were women who had given birth within the previous 3 months and thus were representative of women experiencing maternity care in rural Vietnam. The findings of the qualitative study revealed conceptual equivalence in the way rural Vietnamese and Guineans perceived the quality of services. Despite considerable overlap between the original items and the determinants defined by the Vietnamese, some of the original questions were slightly amended to reflect the context of the maternity service delivery network in rural Vietnam. For instance, the item ‘time spent to explain patient’s illness’ was modified to: ‘time spent to explain health status of the woman’, because childbirth is generally perceived as a normal process rather than an illness in Vietnam. Opinions on the quality of service provided by the commune health centres were solicited. For each item in the questionnaire, respondents could choose one of three options: favourable (+1), neutral (0) or unfavourable (−1). The instrument was prepared in Vietnamese and pre-tested on a separate group of 16 women who had given birth within the 3 months before data collection.

Subjects

The study was conducted during July–August 2000 in Quang Xuong District, Thanh Hoa Province, located 150 kilometres south of Hanoi. The 41 communes of the district were stratified into five areas according to socio-economic and geographical conditions. A list of women who were either pregnant or had given birth within the previous 3 months was generated from routine reports for the National Expanded Programme of Immunization, and also antenatal care provided by commune health centres and Quang Xuong District health services. The list was considered by the local health workers to be complete. It contained 1 218 pregnant women and 1 059 women who had recently given birth. A sample of 210 pregnant women (prenatal group) and another sample of 210 women who had given birth within the previous 3 months (postpartum group), were randomly selected from the list (18.4%). A total of 200 women (102 who delivered at a health care setting and 98 who had delivered at home) in the postpartum group, and 204 pregnant women in the prenatal group, gave their written informed consent to participate, the overall response rate being 96%. Eight subjects in the prenatal group were excluded due to incomplete information recorded in their questionnaires. The final sample thus consisted of 396 women from 34 communes across Quang Xuong District. Eighty per cent of the respondents identified themselves as farmers, and >90% of them had received 6 years or more education.

To minimize possible bias in the information collected, research assistants from institutions in Hanoi visited the subjects either at home or in the rice field. Before interviewing the subjects, the interviewers confirmed their pregnancy status (for the prenatal group) or birth location (for the postpartum group). If a subject had been misclassified, a replacement was then randomly chosen from the list. To assess the agreement between two interviewers (raters), i.e. whether the two research assistants can obtain similar ratings for a particular variable on the same subject (inter-rater reliability), two research assistants interviewed subjects in the postpartum group twice within 1 week. The subjects were informed clearly about the objectives and the procedure of the study. Their participation was voluntary and they were free to withdraw at any time without any negative consequences. The protocol followed the ethical principles of the Helsinki Declaration [22] and the National Health and Medical Research Council of Australia [23], and was approved by the local health authorities and the Human Research Ethics Committee of Curtin University.

Data analysis

Factor analysis based on principal component extraction followed by oblique rotation was used to examine the structure within the adapted 20-item scale. Internal consistency of the measurement scale was investigated through Cronbach’s alpha coefficient, while inter-rater reliability was assessed by the kappa statistic. Finally, regression analysis was used to explore the association between the characteristics of clients and their perceived quality score.

Results

Data from 396 subjects (196 pregnant and 200 postnatal) were analysed. The validity and reliability of the instrument were analysed amongst the women in the postnatal group. The results are presented below.

Factor analysis

Results of the factor analysis are presented in Table 1. Four factors were obtained, with eigenvalues 4.41, 1.88, 1.48 and 1.14. The total variance explained was 44.5% and the communalities ranged from 0.37 to 0.63. The four factors identified, in order of percentage variance explained, may be labelled ‘health care delivery’ (22%), ‘health facility’ (9.4%), ‘interpersonal aspects of care’ (7.4%) and ‘access to services’ (5.7%).

Table 1

Factor analysis and inter-rater reliability of the instrument

Items	Factors							Communality	Kappa
	1	2	3	4
Health care delivery
Good clinical examination	0.68	0.02	−0.29	0.03	0.56	0.39
Good diagnostic skills	0.67	0.14	−0.19	−0.22	0.57	0.76
Quality of dispensed drugs	0.63	−0.01	−0.16	0.19	0.62	0.79
Recovery of patients	0.62	0.30	−0.14	0.23	0.48	0.60
Prescription of drugs	0.57	0.19	−0.11	−0.05	0.37	0.72
Monitoring of patient’s recovery	0.57	0.16	−0.35	−0.09	0.51	0.54
Fee for provided services	0.38	0.05	−0.18	−0.15	0.39	0.70
Health facility
Adequacy of medical equipment	0.15	0.75	−0.20	−0.05	0.59	0.70
Adequacy of rooms	0.21	0.73	−0.08	0.08	0.55	0.69
Adequacy of staffing	0.05	0.62	−0.11	0.16	0.46	0.54
Adequacy of health workers for women’s health	0.15	0.60	−0.38	0.28	0.47	0.35
Interpersonal aspects of care
Compassion for patients	0.25	0.16	20.79	−0.04	0.63	0.84
Respect for patients	0.19	0.14	20.73	−0.03	0.56	0.80
Openness to patients	0.31	0.02	20.65	0.14	0.48	0.77
Honesty	0.09	0.37	20.56	0.19	0.44	0.54
Time spent to explain health status of the woman	0.52	0.27	20.55	0.17	0.51	0.60
Time devoted to patients	0.49	0.26	20.50	−0.01	0.42	0.64
Access to services
Distance to commune health centre	0.08	0.23	−0.08	0.69	0.53	0.74
Access to credit	0.17	0.25	−0.31	0.54	0.50	0.65
Ease of obtaining drugs	0.28	0.21	−0.17	20.45	0.39	0.76
Percentage variance explained after rotation	22.0	9.4	7.4	5.7

Internal consistency

Descriptive statistics for the total perceived quality of maternity services score and its subscales are presented in Table 2. The internal consistency, as reflected by Cronbach’s alpha coefficients, was found to be satisfactory with the exception of ‘access to services’.

Table 2

Descriptive statistics and internal consistency of subscales and total perceived quality score

	Health care delivery	Health facility	Interpersonal aspects of care	Access to services	Total score
Number of items	7	4	6	3	20
Possible range	−7 to +7	−4 to +4	−6 to +6	−3 to +3	−20 to +20
Mean	4.52	1.57	4.95	2.26	14.19
Median	4	1	5	2	12
Standard deviation	2.06	1.74	1.49	0.92	3.98
Cronbach’s alpha	0.72	0.64	0.72	0.33	0.77

The item scores by postpartum and prenatal groups are presented in Table 3. The subjects responded negatively to the physical conditions of commune health centres, especially medical equipment and staffing. The mean scores for items making up ‘health facility’ were much lower than the overall mean score of 0.72. In particular, the mean item score for ‘adequacy of medical equipment’ was 0.29 for postpartum and −0.18 for prenatal groups. They also responded negatively to the manner in which maternity services were delivered, especially with regard to the ‘recovery of patients’. Within the dimension ‘access to services’, although ‘ease of obtaining drugs’ appeared to be highly appreciated, the respondents were rather negative with respect to ‘distance to commune health centre’. In contrast, they responded positively to ‘interpersonal aspects of care’. The mean scores for its underlying items (except ‘time spent to explain health status’) were high. This result was rather different from previous studies in which communication and conduct of health staff were negatively perceived [20,21].

Table 3

Item scores by postpartum and prenatal groups

Items	Postpartum							Prenatal
	Home birth			Clinic birth			Mean	95% CI
	Mean	95% CI	Mean	95% CI
Health care delivery	3.52	3.01, 4.03	4.49	4.09, 4.88	3.09	2.81, 3.38
Good clinical examination	0.46	0.35, 0.57	0.59	0.49, 0.68	0.66	0.59, 0.73
Good diagnostic skills	0.41	0.29, 0.54	0.61	0.49, 0.72	0.73	0.64, 0.82
Quality of dispensed drugs	0.79	0.70, 0.88	0.83	0.75, 0.91	0.64	0.56, 0.71
Recovery of patients	0.39	0.28, 0.50	0.44	0.33, 0.54	0.21	0.14, 0.29
Prescription of drugs	0.65	0.54, 0.76	0.74	0.64, 0.53	0.53	0.44, 0.61
Monitoring of patient’s recovery	0.50	0.37, 0.63	0.71	0.62, 0.81	0.52	0.44, 0.61
Fee for provided services	0.71	0.61, 0.80	0.86	0.79, 0.93	0.77	0.70, 0.84
Health facility	2.05	1.75, 2.36	2.06	1.80, 2.32	0.43	0.23, 0.63
Adequacy of medical equipment	0.29	0.15, 0.43	0.29	0.16, 0.42	−0.18	−0.31, −0.05
Adequacy of rooms	0.54	0.43, 0.66	0.58	0.49, 0.68	0.21	0.15, 0.27
Adequacy of staffing	0.58	0.47, 0.70	0.56	0.43, 0.68	0.33	0.17, 0.47
Adequacy of health workers for women’s health	0.74	0.65, 0.84	0.69	0.59, 0.79	0.16	0.07, 0.24
Interpersonal aspects of care	4.93	4.60, 5.25	5.37	5.13, 5.61	4.13	3.91, 4.36
Compassion for patients	0.90	0.84, 0.96	0.96	0.92, 1	0.83	0.77, 0.88
Respect for patients	0.92	0.86, 0.97	0.97	0.94, 1	0.89	0.85, 0.94
Openness to patients	0.83	0.75, 0.91	0.91	0.85, 0.97	0.79	0.73, 0.85
Honesty	0.91	0.86, 0.97	0.99	0.97, 1	0.72	0.65, 0.80
Time spent in explaining health status of the woman	0.63	0.51, 0.76	0.73	0.63, 0.83	0.53	0.45, 0.61
Time devoted to patients	0.80	0.72, 0.88	0.88	0.82, 0.94	0.69	0.62, 0.75
Access to services	2.31	2.10, 2.51	2.44	2.27, 2.61	1.72	1.59, 1.85
Distance to commune health centre	0.71	0.60, 0.83	0.72	0.62, 0.83	0.45	0.36, 0.54
Access to credit	0.87	0.77, 0.96	0.91	0.84, 0.99	0.47	0.37, 0.57
Ease of obtaining drugs	0.93	0.86, 0.99	0.96	0.92, 1	0.92	0.88, 0.96
Total scale	12.59	11.52, 13.66	14.31	13.56, 15.07	9.36	8.77, 9.96

Inter-rater reliability

Other than ‘good clinical examination’ (0.39) and ‘adequacy of health workers for women’s health’ (0.35), the remaining items exhibited good agreement between the two raters, with kappa values ranging from 0.54 to 0.84; see Table 1.

Association with client characteristics

Among the various characteristics of clients (age, income, education, occupation and maternity status), maternity status was found to influence the overall score on perceived quality of services. In particular, the prenatal group tended to have lower scores (mean = 9.36, SD = 4.20) than the postpartum group (mean = 13.48, SD = 4.67); P <0.001. Also, women who delivered at home (mean = 12.59, SD = 5.28) perceived significantly lower quality of delivery services than women delivered at a health setting (mean = 14.31, SD = 3.85); P < 0.01.

Discussion

It is known that potential confounders can be introduced in the transfer of instruments designed in one health care system to another [24]. Based on a Westernized conceptual framework [5], Haddad et al. [20] developed a 20-item scale to measure perceived quality of general primary health care in Guinea.

In the context of rural Vietnam, a commune health centre typically provides a range of basic services including delivery. An officer-in-charge, such as a secondary midwife or assistant doctor specialized in maternal and child health, is responsible for running the maternity service. Childbirth can be life threatening for both mother and baby, so that the perception of delivery is expected to be different from that of a normal health issue. Therefore, adaptation of the original scale to reflect the specific and cultural context of delivery services is necessary. Qualitative results showed that the modified instrument was conceptually equivalent to the original version, yet it was feasible and comprehensible to rural Vietnamese clients.

With regard to psychometric properties, the Vietnamese version had relatively good internal consistency and construct validity. Four factors were identified from the factor analysis: ‘health care delivery’, ‘health facility’, ‘interpersonal aspects of care’ and ‘access to services’. Improvement in access to primary health care is an important objective of health sector reform in Vietnam and other developing countries. Therefore, this subscale on accessibility to services is useful for health professionals to evaluate the effectiveness of an existing health care delivery network.

Unlike those of previous studies [10,21], our subjects responded positively to the dimension ‘interpersonal aspects of care’. In the rural lowlands of Vietnam such as Quang Xuong district, health personnel at commune health centres are recruited locally and are familiar to the residents. This may explain the appreciation of health staff by respondents. Nevertheless, the perception of ‘time spent to explain the health status of the woman’ was rather poor, reflecting to a certain extent the current provider-centred practices in the Vietnamese health care system, where counselling has not been formally introduced and was rarely provided to clients [25].

Respondents were quite negative in their responses to the ‘health facility’ subscale. It has been established that the physical environment of health settings can impact on client-perceived quality of care [26–28]. Although clients may not be able to evaluate whether a specific technical procedure is appropriate, they can, however, assess quality according to the availability of medical equipment and the behaviour of health staff dealing with it.

Although the communication skills and the conduct of health personnel were highly appreciated, adequacy of staffing and their competence were perceived as poor in this survey. Such a finding is consistent with the recent shortage of skilled health personnel and degraded health facilities at primary health care level in Vietnam [25,29]. After the pre-service training at medical schools, many health professionals do not have any opportunity for continuing education and consequently are not attaining prescribed national standards [25].

In the literature, inter-rater reliability of client-perceived quality of services is seldom addressed and the measurement procedure of the scale used is often not clearly described. The relatively high inter-rater reliability obtained in this study therefore complements previous findings concerning the validity of applying this 20-item scale in developing countries [20,21]. Similar to the Burkina Faso study, Cronbach’s alpha coefficient for the ‘access to services’ construct was relatively low. This could be attributed to the small number of items and heterogeneous characteristics of this subscale [21]. Cronbach’s alpha is only a measure of reliability to the extent that the scale measures a single latent variable. The subscale ‘access to services’ consisted of heterogeneous items grouped a posteriori by convenience, namely, distance to commune health centre, access to credit and ease of obtaining drugs. Subscales derived from factor analysis are merely hypothetical concepts rather than actual measurement entities [30]. The low internal consistency might also be due to the conceptual inappropriateness of the item ‘access to credit’. Unlike other health care systems, ‘access to credit’ for primary health care in rural Vietnam is very limited, and it is still a new concept for residents of Quang Xuong district. Further study is thus needed to improve the reliability of this instrument.

Several issues and limitations should be considered in conjunction with the findings. Firstly, the observed scores could be confounded by maternity status, parity and delivery location of the subjects. Those women who had not given birth at a commune health centre recently might judge the quality of delivery services based on their past experiences with a commune health centre for other (non-delivery) health services. For parous women, their perceptions of quality could be based on their previous delivery experiences as well. Similarly, a pregnant woman might judge the quality based on her experiences of antenatal care services rather than actual delivery. The influence of experience on the perception scores is consistent with the view that ‘for some health problems, expectations develop during the process of health care delivery and are revised in the light of experience’ [7]. Secondly, the study sample was selected from a list compiled from the official reporting system. Although the routine reports of the National Expanded Programme of Immunization and antenatal care programmes were generally considered complete, selection bias could not be completely ruled out because there might be some women who delivered at home but were not captured in our list. Thirdly, the variable ‘health care setting’ was not included in the data analysis as the number of respondents in each institution was small.

Applying the amended 20-item scale of Haddad et al., we have demonstrated the feasibility, reliability and validity of the instrument for measuring client-perceived quality of delivery services in the context of rural Vietnam. It is imperative that follow-up action should be taken by the government to utilize the instrument for evaluating primary health care programmes in Vietnam.

Acknowledgements

This paper represents part of Mr Duong’s doctoral study at Curtin University of Technology. The authors are most grateful to the Direct Aid Program of the Australian Embassy in Vietnam for its support through the Program for Appropriate Technology in Health (PATH). Special thanks go to the women and health workers of Thanh Hoa Province, who readily assisted in the study. The views expressed in this article are those of the authors, and do not necessarily reflect the policies of any organizations. Finally, thanks are due to the Editor-in-Chief Dr Perneger and two anonymous reviewers for their helpful comments and suggestions.

Author notes

¹Program for Appropriate Technology in Health, Hanoi, Vietnam, ²Curtin University of Technology, Public Health School, Perth, Western Australia, ³University of Melbourne, Royal Hospital, Melbourne, ⁴The Macfarlane Burnet Institute for Medical Research and Public Health Ltd, International Health Unit, Melbourne, Victoria, Australia