Abstract

A process evaluation was conducted to evaluate four channels (drama, video, community educators and leaflets) used in a community-based IEC (Information, Education and Communication) HIV/AIDS intervention in rural Uganda. Semi-structured interviews (n = 37) and focus groups (n = 3) were held with community members working as field staff. In addition, two questionnaire surveys (n = 105 and n = 69) and eight focus groups were conducted with the target community. Over 85% of the community had seen at least one drama or video show. They rated them as relevant and realistic. However, the messages `taken home' were not always those intended by the plays. Access to community educators (CEs) was not equal. The CEs had a tendency to avoid the educated, rich and their older relatives. Those who had met with a CE rated them as knowledgeable and trustworthy, but felt they would rather be taught by a trained health worker. Around 80% of the community said they had seen the leaflets, but had not necessarily read them. Although appreciated by some as reference material, low literacy levels and a lack of reading culture meant that some leaflets may have gone astray. These findings suggest that a multi-channel approach may be required to overcome weaknesses inherent in individual channels.

Introduction

Behavioural change interventions continue to offer the best chance of preventing further spread of HIV/AIDS in developing countries (Cohen, 1993; Mann et al., 1992). Yet little is known about which interventions are effective and why (Oakley et al., 1995; Clift, 1998; Van Dam, 1989), partly because the `active ingredients' of behavioural interventions are often inadequately defined (Stephenson, 1999) and the barriers to implementation rarely explored (Janz et al., 1996). Process data are widely recognized as critical to understanding why interventions succeed or fail (Galbraith et al., 1996; Speller et al., 1997; Stewart, 2000).

We report on a process evaluation of an IEC (Information, Education and Communication) intervention trial in a rural adult community, implemented by the Medical Research Council (MRC) Programme on AIDS in Uganda. The IEC intervention was conducted in 12 parishes (administrative units of around 10 villages) in Masaka and Ssembabule districts, situated approximately 150 km south-west of the capital, Kampala. Subsistence farming is the primary economic activity, and while the Buganda tribe and the Catholics predominate, both districts are ethnically and religiously heterogeneous. The outcome of the trial (described elsewhere) is measured using biological indicators (including HIV incidence) obtained through sero-surveys and behavioural indicators (such as reported condom use) obtained through a KABP survey.

The IEC intervention was guided by the Behavioural Change for Interventions model (King and Wright, unpublished review). This model emphasizes the integration of four key components, which are viewed as requisite to the adoption of safe sexual behaviours: knowledge acquisition, skills development, attitudes development and motivational support. Developed specifically to guide the design of interventions, the model synthesizes previous models (such as the Theory of Reasoned Action) and is based on the premise that knowledge alone is inadequate. These four key components fed into the content of messages and design of the IEC intervention, as outlined in Table I. The aim of this evaluation was to inform the trial implementers and future IEC interventions by examining the effectiveness of the intervention components as perceived by the community.

Methods

Following Patton (Paton, 1987, 1996), four process measures were selected a priori, which were considered relevant and useful to the trial implementers and future community-based IEC interventions:

  • Reach and accessibility. To what extent does the channel reach the target population?

  • Acceptability. How does the community rate the channel?

  • Understanding the message. Is the intended message received clearly and understood by all members of the community?

  • Message retention. Does the community remember the message?

Although poorly theorized, it is generally accepted that community participation in the research process can be empowering (Beeker et al., 1998; Campbell and Williams, 1999), can increase community support for an intervention, and can contribute to interventions that are credible and culturally appropriate (Galbraith et al., 1996; Janz et al., 1996). In this evaluation, opinions of the IEC channels were gathered from two participant perspectives—those of community members recruited as volunteers to implement the trial in their communities (field staff) and those of community members who were recipients of the intervention (henceforth referred to as the community).

Field staff views of the four IEC channels, as well as their perceptions of community response to the channels, were explored through 37 semi-structured interviews. A systematic, non-probabilistic sampling strategy was used to focus on those most closely involved in the intervention [community educators (CEs) and parish coordinators (PCs)], and to be representative with respect to age, gender and parish. The interview schedules were piloted beforehand (n = 7) and interviewing stopped when it was felt that saturation point had been reached (i.e. no new information was being given). Field staff fitting the sampling criteria were identified and interviewed whilst moving from village to village during day visits to parishes. The direction taken through the parish was random and selection of interviewees was based on availability (convenience sampling). Of the 37 interviews, 23 were conducted in the local language (Luganda) by the second author and the rest (including all the PC interviews) were conducted in English by the first author.

The interview data were augmented by three field staff focus groups. Six randomly selected PCs attended a discussion, facilitated by the author and conducted in English (the PCs spoke both English and Luganda). CE focus groups were held in two of the parishes with 15 (eight men) randomly chosen CEs. These were facilitated by the second author and conducted in Luganda. All the groups were held before or after scheduled monthly meetings.

Community opinions of the IEC channels were collected via a small-scale survey, using two questionnaire schedules. Questionnaire items were designed following discussion with the implementers and the Luganda translations were cross-checked by local MRC staff. The pilot phase (n = 31 for the first questionnaire on general awareness and n = 10 for the second one evaluating the drama/video shows) led to substantial revisions. The first questionnaire asked about awareness and uptake of all channels [n = 105, of whom 46% were males and 47% were youth (under 25)]. Filter questions identified respondents who had met with a CE (n = 54) or read at least one leaflet (n = 68) and they were asked to complete sections about these channels. Respondents were approached at random by the second author while gathering for community meetings and during designated parish visits. The second questionnaire ascertained community views of the drama and video shows (n = 69, of whom 49% were male and 51% were under 25). The second author surveyed randomly chosen audience members immediately after community meetings. Compliance was high; there was one refusal and only three questionnaires were unusable.

Focus groups (n = 8) were held with 53 drama/video audience members (27 men and 30 aged under 25). The topic guide was pre-tested during a pilot session. Groups, quota sampled for age and sex, met prior to the start of a show and explored community experiences and opinions of the IEC channels. After the show, they reconvened to discuss issues raised in the plays. Participants fitting the selection criteria were recruited just prior to the discussion by the second author.

A study of official documentation and discussion with IEC staff provided a background to the evaluation. The qualitative and quantitative data were gathered simultaneously, and triangulation between data sources enabled verification of findings and exploration of divergent opinions (Denzin, 1989; Milburn et al., 1995; Mason, 1996).

The interviews and focus groups were audio-taped, translated and transcribed by the second author. Themes were identified and codes established by reading through several transcripts. Subsequent transcripts were coded according to these themes and organized using the qualitative software package NUD*IST. The questionnaires were coded by the first author and analysed using Epi-Info. Open-ended questions were post-coded using a set of codes established through discussion between the authors and based on the findings of the pilot phase. Responses which did not fit easily into codes were discussed and coded by group decision.

Results

Drama and video shows

Although video and drama are different media, they have been examined together because they have similar characteristics as far as the process measures are concerned (see Table I).

To what extent does the channel reach the target population?

Drama and video shows attracted large numbers, and appeared to reach all sections of the community. Of survey respondents, 88% said they had seen at least one drama show and 86% said they had seen at least one video. Field staff believed that they came `expecting to get new information' and also because they enjoyed the plays. As with other IEC channels, many people took an interest because they had lost relatives to AIDS:

Everyone wants to listen to those meetings because most of them have lost a relative so they are very interested... [Male CE]

Not everyone attended. Other community events, work commitments and inadequate mobilization were all suggested as reasons preventing people from attending. Few individuals actively refused to attend. These were mainly either older people, who felt they had nothing to learn, or those believing themselves infected, who felt it was too late.

Field staff perceived that women turned out in greater numbers. They believed that men were more often away during the day and women were more interested in AIDS issues. Occasionally, certain individuals were prevented from attending; women by their husbands and children by their parents. Overall, however, the plays appear to be accessible to almost all community members.

How do people rate the plays?

Questionnaire respondents were asked to rate the plays using Likert scales (e.g. not at all informative, quite informative and very informative). They were as likely to find the plays `very informative' (25%, n = 69) as `very entertaining' (17%, n = 69), reinforcing field staff views that people come to learn as well as to enjoy. Only 4% felt the plays had `no particular message', and the stories were rated as relevant and realistic (`This play was aimed more at other people, not me', 90% disagreed or disagreed strongly, and `The outcome of the story is just like real life', 90% agreed or agreed strongly). Comments overheard by field staff supported these findings:

PC ...you hear people saying, `Ah...that is the real thing. What [the drama group] have been playing is so like what we experience in our community [...] These people who write these plays, they really know what is going on. [Male PC]

The inevitable presence of young children at shows did attract complaints from a small number of parents. Since the plays were held outside and open to everyone, it was not practical to prevent young children from attending, except where parents took this initiative themselves. There was some evidence that parents did occasionally bar their children, even those within the target age range. During a focus group discussion, three 13-year-old girls claimed their parents prevented them from attending on the premise that they would be `spoiled'.

The questionnaire data suggest that the majority of respondents (n = 68, 34 men) were comfortable with the presence of children at plays. When asked, `Were you happy for children under 11 to see this play?', 97% of men and 71% of women said `yes'. As the chief carers, women were significantly less happy (χ2 = 5.74, P = 0.016, d.f. = 1). In fact, it was not the plays per se that adults, and particularly parents, were unhappy with, but the discussion of condoms. It was the plays promoting condoms which received complaints:

IVR Why were they negative [about the play]?

D They said it included things about the condom and should not have been staged in the presence of children. But they liked the other plays very much... [Female Drama Member]

Both field staff and community members felt that, on the whole, complaints about condoms had decreased as the trial progressed. Many parents had come to accept the necessity of teaching young people about condoms.

Did they understand the main message?

Over 90% of survey respondents agreed that `This play taught me many new things about HIV/AIDS'. They were asked to name the main message of plays both straight after performances and also after some time had elapsed (Table II). It is clear that not only were respondents able to state specific and positive messages from plays, but they were also able to remember these messages some time later.

The focus groups yielded rich data on participant attitudes by providing an opportunity for participants to discuss the issues raised by the plays. This can be illustrated by focusing on one play, Omusika (The Heir), summarized in Box I.

The issues raised varied across groups. A discussion between older women focused on the dilemma facing the first wife; when the HIV-positive husband returns home, should she refuse to have sex with him? This led to heated discussion about the relative lack of power that HIV-negative women have in negotiating sex with positive partners. The group clearly blamed the man for the ensuing problems brought to his family:

F1 We have seen that men sometimes cause problems in the home because if a man leaves his wife and goes off to look for an heir just because the wife delivers only girls, instead of praying to God so that his wife also delivers a boy [...] if that man had stuck to one partner, all those problems wouldn't have come to their family. [Older Woman, FGD5]

A discussion between young men focused more on the role of witch doctors in treating HIV positive people who claim to have been bewitched. In contrast to the previous group, they felt that the woman was equally to blame:

M1 ...we have now learnt that the woman is the cause of all this suffering because the man is toiling with work to bring money for his wife but the wife instead looks for other men to support her...

M1 You could delay in coming back because you want to bring many things for your wife, you are struggling to please her but she instead says you have delayed in coming back and looks for other men...

M6 As you know, women have little thinking, they could think that men delay on work because of other women and so they do the same. [Male Youth, FGD 4]

The probable take-home messages differ substantially between these groups. Furthermore, rather than adopt new perspectives, both groups have used the story to reinforce their own, existing views on gender roles in the transmission of HIV. Other focus group data also suggested that at least some of the audience chose to ignore the messages confronting traditional practices (refusing to accept female heirs) and interpreted this as a play about marital faithfulness.

Do people ask questions afterwards?

Providing opportunities to ask questions can increase the likelihood that messages are understood. Few (4%) of the survey respondents asked questions after the shows. It was hypothesized that the large and mixed (age and sex) audience served as a deterrent, especially for a youth asking in the presence of his/her parents. Furthermore, attending video and drama shows, and especially asking questions afterwards, are not normal everyday experiences for these villagers and they may have felt somewhat overwhelmed. Indeed, field staff cited `shyness' as a possible explanation, but of the survey respondents who did not ask a question (n = 64), only 2% said they feared speaking in front of a large group. Furthermore, field staff suggested that often, those who were reluctant to ask in front of an audience would approach the PC or CE individually after the meeting instead.

In fact, 73% of survey respondents simply said they could not think of anything to ask. A further 25% said they were not given an opportunity or had `moved on' due to lack of time. `Moving on' was the main reason cited by focus group participants. Because crowds took time to gather, the plays often started late and the imminent sunset prohibited prolonged discussions after meetings.

Questions do not always represent genuine information needs. Some of the field staff noted that difficult questions were occasionally posed simply to cause arguments, particularly by youth. Most field staff agreed that those asking questions were predominantly older men and male youths. It would appear then, that although this channel is as accessible to women as it is to men, it is less beneficial to women as an opportunity to resolve their queries.

Do people prefer drama or video shows?

When asked to state a preference, 59% (n = 90) of survey respondents said they preferred to watch drama (t-test NS, P = 0.09). This finding was reiterated by field staff, most of whom felt that the community preferred the drama shows because `In the drama they see real people acting but in the video they just see a picture' [Female PC]. The fact that plays felt `more real' was most commonly cited by survey participants as a reason for preferring drama (36%, n = 53). They also felt it was more instructive, partly because the actors were known to them and it was possible to ask questions (34%). Others felt that it was easier to see and hear the drama compared with video (13%).

Of the 41% who preferred videos, most said that the videos offered better variety, with many different scenes (51%). Some enjoyed the novelty of a video (11%), while others preferred to watch video because they were not impressed by the behaviour of the drama group members (11%).

CEs

How accessible are CEs?

Of the survey respondents who had met with a CE and completed the CE section of the questionnaire (n = 54), the majority rated them as visible (`I hardly ever see the CE around', 80% `disagreed' or `disagreed strongly') and usually available (`the CE in my village is usually available if I have a question I want to ask about HIV/AIDS', 89% `agreed' or `strongly agreed').

Around 80% of questionnaire respondents said they had met with a CE. In contrast, monitoring data suggest a mean of 3.5 attendances per person over a 4-year period. It therefore seems likely that certain individuals meet many times with a CE, while others never meet at all. Our qualitative data suggest that access to CEs was not equal. In several audience groups, members argued about whether CEs did their work; those who had never been visited by the CEs were critical, while the others defended them.

In one of the CE focus groups, the CEs were unexpectedly defensive. A previous group with audience members from the same parish had led to a debate about whether or not the CEs worked and it seemed probable that certain members of this group had reported back to the CEs. They responded by saying:

...the problem we have had with some people is that when we go to teach them they send us away, telling us that they do not want to be taught. But when you [the MRC] ask them, they just say that they have never been taught. [CE, CE-FGD2]

Such disputes suggest that existing relationships within villages may affect accessibility. As one focus group member put it:

....some people have hatreds which started long ago, so when such a person has a problem she may fear to approach the CE. [Older woman, FGD5]

Field staff believed the highly educated and rich were often avoided by CEs because they tended to react badly to someone of `lower status' telling them what to do. Younger CEs also found it difficult to talk to people who were older than them, particularly relatives. The CEs claimed that if they found it difficult to approach a certain household, they would ask a fellow CE to visit instead. It is not known to what extent this collaboration took place in reality.

As far as the community was concerned, gender was not a particular barrier to talking with CEs. However, on a practical level, the CEs generally found it easier to access men:

I can meet men in very many places [...] but these ladies, it is sometimes difficult. Because if you go to a woman's home several times, the husband might oppose. [Male CE]

Monitoring figures suggest that CEs meet as often with men as they do with women and there was no strong evidence from the interviews to dispute this.

How do people rate the CEs?

The majority of respondents who had met with a CE thought the CEs knew `a lot about HIV/AIDS' (94%, n = 54) and all respondents said that `if a friend or relative had a problem or question regarding HIV/AIDS, I would advise them to go and talk to the CE'. Despite this, almost 80% said they would rather be taught by a trained/qualified health worker than a CE. In fact, at the beginning of the trial, lack of formal qualifications meant that some CEs found it difficult to be accepted:

PC At first [the community] rejected them. They said, `you have always been in our village. How have you suddenly become a health worker?' [Male CE]

Since most respondents appear to be satisfied with the knowledge levels of their CE, it seems probable that the preference for trained personnel is really about undervaluing the opinion of someone who is already familiar. This was vocalized by a community member:

...there are some people who despise CEs that they know very well, saying that `how can a person of such character teach them?'. [Female youth, FGD8]

This view was reiterated by some of the field workers.

In small communities, confidentiality is clearly an important aspect of CE work. The majority of survey respondents (82%) believed their CE `would probably keep a secret if I told him something confidential about my sexual behaviour'. A focus group participant explained:

We have not yet heard any rumour about [the CE] disclosing people's secrets because if it were so, people would not let him visit them. [Female youth, FGD8]

The acceptability of this channel depends, to an extent, on each individual CE and their reputation in the community. Ideally, they should serve as positive role models for other community members (McAlister, 1991; Freudenberg, 1995). Not surprisingly, CEs came under a certain amount of pressure to maintain exemplary behaviour. For instance, the community complained to the PC and voted to dismiss a CE who was refusing to care for a daughter sick with AIDS.

Do people ask questions?

Compared with drama and video shows, community education provided greater scope for asking questions and clearing up misunderstandings, particularly for those who met with a CE individually. Nearly 65% of those who met with a CE alone said they asked a question (n = 17) compared with 46% of those who met the CE in a small group (n = 41). Without exception, respondents in one-to-one meetings who did not ask a question said they `could not think of anything to ask'. Although this was also the main reason cited by respondents in small group meetings (46%, n = 22), `not being given an opportunity' was also a common reason (36%). A minority (14%) were either scared or embarrassed in front of the CE. Those who did ask were generally satisfied with the answer. Over 78% of small group participants and 90% of one-to-one participants felt they had their question answered adequately. Questions notwithstanding, it appears that the teaching style adopted was fairly didactic. For instance, according to survey data, the topic of conversation was decided by the CE in 88% of one-to-one meetings and 76% of small group meetings. The CEs themselves said that their meetings generally took the form of unsolicited information-giving sessions.

Do they understand the messages?

Of those who had met with a CE in the last month, four out of five of those who had one-to-one meetings and four out of five of those in small group meetings could state a positive message. After 3 months, four of the six who were in one-to-one meetings and 84% (n = 25) of those who were in small group meetings could cite a positive message. Thus it seems that, at least for those who had met a CE in a small group, ability to state a positive message did not decrease with time since exposure.

Leaflets

How many people are aware of the leaflets?

As a result of intensive distribution, the leaflets enjoyed high saliency, with around 80% of survey respondents claiming to have seen them (prompted awareness). In fact, a few of the field staff felt that, towards the end of the trial, the community had become saturated:

People no longer take [the leaflets], saying they have many of them at home. [Female PE]

What do people do with the leaflets?

Of the survey respondents who recognized the leaflets when shown, 67% claimed to have read the general information leaflet, 65% to have read the condom leaflet, 71% to have read the STD leaflet and 74% to have read the mosquito leaflet. However, when asked if they would be happy to answer a questionnaire about these leaflets, 13% of those who claimed to have read the leaflets declined, saying that they had not read them properly. Of those who answered the questionnaire on leaflets, around 60% claimed that they kept the leaflet afterwards (range 53% for STD leaflet to 68% for condom leaflet) and around 20% claimed to have given them to a friend/relative (range 16% for condom to 22% for general information). However, several focus group participants were sceptical about whether leaflets were read and kept:

F4 I only read a bit because I did not think they were very important. But I kept them in my house anyway.

F3 One can easily say that she will read the leaflet but those leaflets were distributed and very few people took the responsibility of reading them. [Older women, FGD5]

Low literacy rates [62% (Statistics Department [Uganda], 1995, 1996)], and comments made both in community focus groups and field staff interviews suggest that some of those who took leaflets were unable to read them. Field workers and focus group participants did not consider this a problem since CEs or family members, particularly children, could read aloud to the illiterate individual. It was acknowledged, however, that this probably happened rarely. It was not just high illiteracy rates that limited the effectiveness of the leaflets, but also a general lack of reading culture. This was mentioned by several field staff:

PC The [leaflets] would be effective but our community does not take reading as a priority. [Male PC]

How do people rate the leaflets?

The majority of survey respondents (over 75%) rated the leaflets as `quite informative', `quite easy to understand' and as having the `right amount of detail'. They were appreciated by the community because they served as a reference guide: `[the community] say they contain information which cannot rot' [Male CE]. An evaluation of HIV/AIDS prevention campaigns in Kenya (Witte et al., 1998) also found that take-away materials were popular. Leaflets were considered useful in combating intermittent rumours spread by individuals or organizations antagonistic to AIDS prevention messages. Furthermore, parents, reluctant to discuss sexual matters with their children, could hand them these leaflets instead.

At first, the message of the mosquito leaflet, that mosquitos could not transmit HIV, was met with disbelief. The leaflet had to be reinforced by talks from CEs and other field staff. Complaints were voiced about the condom leaflet, because it was viewed as promoting condom use among young people. These complaints came mainly from elderly people and tended to decrease as the trial progressed.

Do people understand the message?

Table III shows the percentage of respondents who were able to state a positive message from each of the leaflets, against time since exposure.

At the end of the questionnaire participants were asked if there was anything in the leaflets that they did not understand or agree with. Some of the questions raised were fairly sophisticated suggesting quite a high level of general understanding. Examples include:

If you have an STD, it can stop you from having children but if you treat STDs you might end up with twins. Is this true?

How can a condom get damaged if you carry it in your pocket?

The proboscis of a mosquito is like an injection needle. How can a needle spread HIV but not the mosquito?

Exposure to different channels

Thus far, the channels have been treated as separate entities, but since similar topics were covered by each channel, messages from one channel may have reinforced messages from another. Many survey respondents had been exposed to more than one media; 60% (n = 104) had seen both a drama and video, and of these 63 people, 53 had also met with a PE. Over a third (36%) had been exposed to all the channels (PE, drama/video and at least one leaflet). Thus it seems likely that some overlap in the impact of each channel occurred.

Discussion

Although the effectiveness of channels such as drama (Elliott et al., 1996; Harvey et al.2000) and peer education (Lamptey, 1991) have been evaluated elsewhere, few have focused on process measures or been conducted in community-based, developing country settings. By drawing on community experiences, we identified several strengths and weaknesses of the channels.

Reach and accessibility

The accessibility of drama and video shows is both a strength and weakness; on the one hand, a wide and diverse audience is reached, but because the message must be acceptable and relevant to a cross-section of the community, there is little opportunity to target messages at specific groups, if so desired. For instance, some messages (e.g. about condom use) must be indirect enough to reach over the heads of young children while maintaining usefulness to adults. While in theory CEs are constantly accessible to the community, existing community dynamics may mean that they are available to some more than others. The highly educated, rich and the elderly are particularly likely to be missed out by a CE. However, accessibility may be enhanced by having more than one CE per village and by careful selection procedures. In this intervention, there were usually two CEs per village and the selection criteria gave high priority to respectability in order to maximize the acceptance of CEs within the community. This approach represents a divergence from more traditional peer education that tries to match educators closely with the target population (Janz et al., 1996). As a result of intensive distribution strategies, the leaflets enjoyed high visibility within the community but there is little control over what happens later. It appears that some may go astray.

Acceptability

`Cultural competency', or the extent to which HIV prevention messages have been tailored to target audiences, has been defined as a characteristic of successful intervention programmes (Clarke, 1995; Holtgrave et al., 1995; Janz et al., 1996). All the channels were rated as highly acceptable. Community members seemed particularly impressed that the drama and video plays bore such close resemblance to their own lives. Recognizing oneself in the characters is said to be important if HIV risks are to feel real (Woodcock et al., 1992). Of interest to program implementers choosing between drama and video, is the fact that the community expressed a preference for drama. Since video interventions tend to require expensive equipment and technical expertise, it may be more feasible, and no less acceptable, to concentrate on drama.

Those who had met with CEs rated them as approachable, knowledgeable and trustworthy yet it appears that most people would actually prefer to be taught by a qualified health worker or at least by a stranger. However, introducing outside experts would be less sustainable and CEs would no longer be constantly available. The didactic teaching style of many CEs is not necessarily a weakness, since it reflects the traditional teaching approach to which many of the target population are accustomed. Attempts to introduce participatory teaching approaches in rural African populations have met with difficulties (Laver et al., 1996; Kinsman et al., 1999). Lack of reading culture and low literacy levels meant that leaflets were probably deemed less acceptable than the other channels. They may play a supportive role by providing durable reference information.

Understanding the message

Although leaflets offer the most direct control over messages to the community, they provide no opportunity to clarify points, except when used in conjunction with another channel. Both drama/video and CEs provide opportunities to ask questions although individuals are far more likely to put questions to a CE, especially in one-to-one meetings. The extent to which CEs provide clear and accurate messages depends on the quality of their training and individual ability. In this intervention, training and supervision of CEs was given priority and our experience suggests that this is requisite to the effectiveness of this channel. Drama and video offer the least control over take home messages because the plays are open to individual interpretation. Post-show talks can be used to highlight the main themes but the plays alone cannot dictate messages. Audience members may adopt more punitive or `black and white' messages than intended by the stories. Although not necessarily harmful, fear arousing messages have had little success in promoting behavioural change (Sherr, 1990). Furthermore, just as the plays intend to bring about positive attitude change, it is equally possible to use the stories to justify or reinforce existing views and prejudices.

Message retention

Across all channels, many people were able to remember the main message after 3 months, although it should be borne in mind that asking individuals to name a main message is much less discerning than probing for specific facts. Furthermore, being able to remember a main message does not always mean that the message has been understood nor that any behavioural change has taken place as a result. Three months or more after exposure, individuals were most likely to remember the message from a CE, followed by drama/video and then leaflets. However, the CE figure may be artificially high because, in contrast to drama/video and leaflets, it is not possible to verify answers to this question. In other words, we had to assume that the message given really did come from a CE, whereas messages given after plays and reading leaflets could be checked against drama scripts and the leaflet content. Even with leaflets, however, it may be that respondents were citing a message which they guessed, or assumed, was contained in the leaflet because of the leaflet title. They may actually have heard the message through another channel. This might explain why respondents found it more difficult to remember a message from the general information leaflet—because it was more general, it was more difficult for respondents to hazard a guess. Given that over a third of respondents had been exposed to all the channels, care should be taken in assuming that the main message cited from a specific channel did indeed come from that channel.

Although the evaluation suggests a positive community response overall, several limitations need to be considered. The evaluators sought to be as objective as possible, but since this was an in-house evaluation, it was impossible to avoid association with the MRC. However, the neutral stance of the researchers was emphasized to interviewees and care was taken to reassure field staff that they were not being evaluated as individuals. Despite these efforts, the results are likely to have been subjected to a desirability bias.

The setting in which behavioural interventions are evaluated can also impact on results (Stephenson, 1999) and certain factors here may have contributed to a more positive outcome. Firstly, the intervention is operating in an area where HIV incidence is high (7.6 per 1000 person years in, 1998). Many people have already lost relatives and friends to AIDS, and therefore motivation to learn about AIDS is likely to be high. Secondly, access to information about AIDS in these parishes is not very good. Although there have been National AIDS campaigns, 65% of women and 36% of men in rural Uganda have no exposure to mass-media (Statistics Department [Uganda] and Macro Inc., 1995). It is likely that the intervention had relatively little competition with other media. Furthermore, what has been evaluated here is not the channels per se, but the channels as implemented by this particular organization. Thus it cannot be assumed that the same mix of channels, transposed to a different context, will give the same results.

Conclusion

Process is an important, but frequently neglected aspect of intervention evaluations. Not only do end-point indicators often fail to capture the essence of the intervention, but exploring `what happened' can provide vital clues to understanding the outcome results. The story behind this IEC intervention supports the idea that messages presented in several different communication formats can enhance learning (Windsor et al., 1984; Green et al., 1980; Clarke, 1987). No one channel, used in isolation, scores highly on all four process measures. On the other hand, in a multi-method approach, the channels may work synergistically to reinforce messages and overcome weaknesses inherent in individual channels.

Table I.
Channel Description 
Drama Held at village community meetings. 
 CEs and PCs mobilize the community to attend. MRC staff representatives attend each meeting. 
 Drama group performs one of a set of seven plays, written by a consultant dramatist in collaboration with MRC staff. The drama is usually followed by a short talk and opportunity for questions. 
 One drama group per parish. Members are volunteers recruited by the programme. 
 The drama group rehearses once a week and is visited once a month by the consultant dramatist, who offers advice and support. 
 The group receives US$9 per show which is put towards props or used as initial capital for income generating projects 
 Two plays in two different villages staged in each parish per month (one on a weekday afternoon and one on Sunday afternoon). 
 Issues addressed by plays include: cultural practices and AIDS, sugar daddies, abstinence, condom use and misconceptions, peer pressure on adolescents, and treatment seeking for STDs (six parishes only). 
Video Shown at village community meetings. 
 CEs and the PC mobilize the community to attend. 
 Videos brought by MRC staff. They show recordings of the same set of seven plays (see above), plus four further plays, being performed by different parish drama groups. 
 One video shown in each parish every month on weekday afternoons. Shows taken to a different village each time. 
Community education 24 CEs in each parish; 57% male, mean age 35.5. 
 Volunteers, initially trained by the MRC. Supervised regularly by the PC and MRC staff through monthly meetings and spot checks. CEs are paid a monthly incentive of US$4.50. 
 Each CE conducts around 10 one-to-one or small group meetings with members of their village each month. For each meeting they complete a monitoring form detailing the people met and topics covered. CEs also distribute condoms and leaflets. 
 Teaching is based on 16 standard lessons. The six most common lessons addressed by CEs were condom use, HIV testing, how HIV is spread, STD treatment seeking behaviour, marital faithfulness and preventing the spread of HIV. 
Leaflets Designed by MRC staff and pre-tested among the community. 
 Distributed at all intervention activities by the PC, CEs and MRC staff. 
 Four separate leaflets covering the following topics: general information regarding HIV/AIDS, treatment of STDs (six parishes only), the mosquito and HIV, and myths about condoms. 
Channel Description 
Drama Held at village community meetings. 
 CEs and PCs mobilize the community to attend. MRC staff representatives attend each meeting. 
 Drama group performs one of a set of seven plays, written by a consultant dramatist in collaboration with MRC staff. The drama is usually followed by a short talk and opportunity for questions. 
 One drama group per parish. Members are volunteers recruited by the programme. 
 The drama group rehearses once a week and is visited once a month by the consultant dramatist, who offers advice and support. 
 The group receives US$9 per show which is put towards props or used as initial capital for income generating projects 
 Two plays in two different villages staged in each parish per month (one on a weekday afternoon and one on Sunday afternoon). 
 Issues addressed by plays include: cultural practices and AIDS, sugar daddies, abstinence, condom use and misconceptions, peer pressure on adolescents, and treatment seeking for STDs (six parishes only). 
Video Shown at village community meetings. 
 CEs and the PC mobilize the community to attend. 
 Videos brought by MRC staff. They show recordings of the same set of seven plays (see above), plus four further plays, being performed by different parish drama groups. 
 One video shown in each parish every month on weekday afternoons. Shows taken to a different village each time. 
Community education 24 CEs in each parish; 57% male, mean age 35.5. 
 Volunteers, initially trained by the MRC. Supervised regularly by the PC and MRC staff through monthly meetings and spot checks. CEs are paid a monthly incentive of US$4.50. 
 Each CE conducts around 10 one-to-one or small group meetings with members of their village each month. For each meeting they complete a monitoring form detailing the people met and topics covered. CEs also distribute condoms and leaflets. 
 Teaching is based on 16 standard lessons. The six most common lessons addressed by CEs were condom use, HIV testing, how HIV is spread, STD treatment seeking behaviour, marital faithfulness and preventing the spread of HIV. 
Leaflets Designed by MRC staff and pre-tested among the community. 
 Distributed at all intervention activities by the PC, CEs and MRC staff. 
 Four separate leaflets covering the following topics: general information regarding HIV/AIDS, treatment of STDs (six parishes only), the mosquito and HIV, and myths about condoms. 
Table II.

Respondents able to remember a message after watching an IEC play

Response to `What was the main message of the play?' Straight after play (n = 68) At least 3 months later (n = 10)a 
aIn the general awareness questionnaire, respondents were asked to cite the main message of the last play they had seen. The sample of 10 are those who had not seen a play for at least 3 months. 
Specific and positive message from play 43 (63%) 5 (50%) 
General IEC message unrelated to the play 21 (31%) 3 (30%) 
Negative/false message from play – – 
Neutral or negative message, unrelated to play 1 (2%) – 
Don't know/vague response 3 (4%) 2 (20%) 
Response to `What was the main message of the play?' Straight after play (n = 68) At least 3 months later (n = 10)a 
aIn the general awareness questionnaire, respondents were asked to cite the main message of the last play they had seen. The sample of 10 are those who had not seen a play for at least 3 months. 
Specific and positive message from play 43 (63%) 5 (50%) 
General IEC message unrelated to the play 21 (31%) 3 (30%) 
Negative/false message from play – – 
Neutral or negative message, unrelated to play 1 (2%) – 
Don't know/vague response 3 (4%) 2 (20%) 
Table III.

Respondents able to remember a message from the leaflets over time

When read? General information (n = 48) Mosquito (n = 51) STD (n = 33)a Condom (n = 44) 
aSix parishes only. 
In the last month 40% (10) 89% (9) 100% (3) 83% (6) 
Between 1 and 3 months ago 44% (9) 58% (12) 70% (10) 62% (13) 
Over 3 months ago 42% (26) 43% (28) 61% (18) 61% (23) 
When read? General information (n = 48) Mosquito (n = 51) STD (n = 33)a Condom (n = 44) 
aSix parishes only. 
In the last month 40% (10) 89% (9) 100% (3) 83% (6) 
Between 1 and 3 months ago 44% (9) 58% (12) 70% (10) 62% (13) 
Over 3 months ago 42% (26) 43% (28) 61% (18) 61% (23) 
1
Present address: 6 Windsor Place, Stirling FK8 2HY, UK

The authors would like to thank all the community members who gave of their time to participate in this research and the MRC staff who so willingly accommodated the evaluation into their working schedules. We would also thank Mr John Kinsman and Mr Jeremy Armon for their helpful comments on an earlier draft.

References

Beeker, C., Guenther-Grey, C. and Raj, A. (
1998
) Community empowerment paradigm drift and the primary prevention of HIV/AIDS.
Social Science and Medicine
 ,
46
,
831
–842.
Clarke, N. M. (
1987
) Social learning theory in current health education practice.
Advances in health Education and Promotion
 ,
2
,
251
–275.
Campbell, C. and Williams, B. (
1999
) Understanding the impact of a community-led HIV prevention programme in South Africa: context, conceptual framework and methodology.
Australian Journal of Primary Health
 ,
15
(4),
9
–23.
Clarke, I. (
1995
) Are our information, education and communication-based interventions too superficial?
Tropical Doctor
 ,
25
,
167
–170.
Clift, E. (
1998
) IEC interventions for health: a 20 year retrospective on dichotomies and directions.
Journal of Health Communication
 ,
3
,
367
–375.
Cohen, J. (
1993
) Sombre news from the AIDS front.
Science
 ,
260
,
1712
–1713.
Denzin, N. K. (1989) The Research Act: A Theoretical Introduction to Sociological Methods, 3rd edn. Prentice-Hall, Englewood Cliffs, NJ.
Elliott, L., Gruer, L., Farrow, K., Henderson, A. and Cowan, L. (
1996
) Theatre in AIDS education—a controlled study.
AIDS Care
 ,
8
,
321
–340.
Freudenberg, N. (1995) What do we know about AIDS prevention strategies: a selective review of the literature. In Freudenberg, N. and Zimmerman, M. A. (eds), AIDS Prevention in the Community: Lessons from the First Decade. American Public Health Association, Washington, DC.
Galbraith, J., Ricardo, I., Stanton, B., Black, M., Feigelman, S. and Kaljee, L. (
1996
) Challenges and rewards of involving community in research: an overview of the `Focus on Kids' risk reduction program.
Health Education Quarterly
 ,
23
,
383
–394.
Green, L. W., Kreuter, M. W., Deeds, S. G. and Partridge, K. B. (1980) Health Education Planning. Mayfield, Palo Alto, CA.
Harvey, B., Stuart, J. and Swan, T. (
2000
) Evaluation of a drama-in-education programme to increase AIDS awareness in South African schools: a randomized community intervention trial.
International Journal of STDs and AIDS
 ,
11
,
105
–111.
Holtgrave, D. R., Qualls, N. L., Curran, J. W., Valdiserri, R. O., Guinan, M. E. and Parra, W. C. (
1995
) An overview of the effectiveness and efficiency of HIV prevention programmes.
Public Health Reports
 ,
110
,
134
–146.
Janz, N., Zimmerman, M., Wren, P., Israel, B., Freudenberg, N. and Carter, R. (
1996
) Evaluation of 37 AIDS prevention projects: successful approaches and barriers to program effectiveness.
Health Education Quarterly
 ,
23
,
80
–97.
Kinsman, J., Harrison, S., Kengeya-Kayondo, J., Kanyesugye, E., Musoke, S. and Whitworth J. (
1999
) Implementation of a comprehensive AIDS education programme for schools in Masaka District, Uganda.
AIDS Care
 ,
11
,
561
–601.
Lamptey, P. (1991) An overview of AIDS interventions in high risk groups: commercial sex workers and their clients. In Chen, L. C., Sepulveda, A. J. and Segal, S. J. (eds), AIDS and Women's Reproductive Health. Plenum Press, New York, pp. 151–163.
Laver, S. M., Van den Borne, B., Kok, G. and Woelk, G. (1996/7) Was the intervention implemented as intended? A process evaluation of an AIDS prevention intervention in rural Zimbabwe.
International Quarterly of Community Health Education
 ,
16
,
25
.
Mann, J., Tarantolla, D. and Netter, T. (eds) (1992) AIDS in the World. Harvard University Press, Boston, MA.
Mason, J (1996) Qualitative Researching. Sage, London.
McAlister, A. L. (
1991
) Population behaviour change: a theory-based approach.
Journal of Public Health Policy
 ,
12
,
345
–361.
Milburn, K., Fraser, E., Secker, J. and Pavis, S. (
1995
) Combining methods in health promotion research: some considerations about appropriate use.
Health Education Journal
 ,
54
,
347
–356.
Oakley, A. Fullerton, D. and Holland, J. (
1995
) Behavioural interventions for HIV/AIDS prevention.
AIDS
 ,
9
,
479
–486.
Patton, M. (1987) How to use Qualitative Methods in Evaluation. Sage, Newbury Park, CA.
Patton, M. (1996) Utilization Focused Evaluation: The New Century Text. Sage, London.
Sherr, L. (
1990
) Fear arousal and AIDS: do shock tactics work?
AIDS
 ,
4
,
361
–364.
Speller, V., Learmonth, A. and Harrison, D. (
1997
) The search for evidence of effective health promotion.
British Medical Journal
 ,
315
,
3661
–3363.
Statistics Department [Uganda] (1995) The 1991 Population and Housing Census. Ministry of Finance and Economic Planning, Entebbe, Uganda.
Statistics Department [Uganda] and Macro Inc. (1996) Uganda Demographic and Health Survey 1995. Statistics Department [Uganda] and Macro Inc., Calverton, MD.
Stephenson, J. M. (
1999
) Evaluation of behavioural interventions in HIV/STI prevention.
Sexually Transmitted Infections
 ,
75
,
69
–71.
Stewart, W. (2000) Use of process evaluation during project implementation: experience from the CHAPS project for gay men. In Thorogood, M. and Coombes, Y. (eds), Evaluating Health Promotion: Practice and Methods. Oxford University Press, Oxford, pp. 84–98.
Van Dam C. J. (
1989
) AIDS: is health education the answer?
Health Policy and Planning
 ,
4
,
141
–147.
Windsor, R. A., Baranowski, T., Clark, N. and Cutter, G (1984) Evaluation of Health Promotion and Education Programmes. Mayfield, Palo Alto, CA.
Witte, K., Cameron, K. A., Lapinski, M. K. and Nzyuko, S. (
1998
) A theoretically based evaluation of HIV/AIDS prevention campaigns along the trans-Africa highway in Kenya.
Journal of Health Communication
 ,
3
,
345
–363.
Woodcock, A. J., Stenner, K. and Ingham, R. (
1992
) Young people talking about HIV and AIDS: interpretations of personal risk of infection.
Health Education Research
 ,
7
,
229
–247.

Comments

0 Comments