Posted on: 30th November 2016
Faiths Act Sierra Leone (FASL) was set up in 2011 by the Tony Blair Faith Foundation (which is now part of the Tony Blair Institute for Global Change) that closed in October last year after five successful years of operation.
FASL was designed to complement the Government of Sierra Leone’s (GoSL) distribution campaign that launched in 2010, and aimed to contribute towards a reduction in malaria morbidity and mortality through increased knowledge, awareness and practices of prevention methods amongst vulnerable groups, promoted through religious communities.
This document gives an overview of the headline achievements and key findings of the endline evaluation. It explores the lessons learned from delivering the programme in Sierra Leone and presents a series of recommendations to inform the design and implementation of similar programmes.
Sierra Leone suffers from an exceptionally high level of mortality due to malaria. It is a preventable disease, yet it remains the primary cause of death for 25% of the population (and for 38% of children under five), it also causes four in ten hospitalisations.1 Sierra Leone is one of seven countries in Sub-Saharan Africa where, despite progress in reduction of mortality and transmission, malaria infection still affects more than quarter of the population.2
In 2010, only 36% of households owned at least one Insecticide-Treated Net (ITN), and many families did not understand why they should use them, or how to use them effectively.3 In addition, government resources were limited and a lack of infrastructure meant that only half the population could access healthcare within a 5km radius of their home. In 2010, the Government initiated the first mass ITN distribution campaign, which by 2013 had increased ITN availability per household to 62%, but with only 39% sleeping under one.4
In 2011, to complement the government’s distribution campaign and respond to the problem of effective usage, TBFF launched FASL, a programme centred on engaging faith communities to communicate simple key malaria messages aimed at improving prevention and treatment behaviour, especially for pregnant women and under-fives.
In a country where 99% of the population affiliate with a religious community,5 and in the absence of sufficient government infrastructure or a Community Health Worker (CHW) policy, 6 the engagement of faith leaders in disseminating public health messages presented an opportunity.
Their extensive networks, positions of trust with their communities, and influential voice on key values such as health and social issues, meant they were well placed to play a role in bringing communities together over the shared issue of preventing malaria. A community-led approach also has potential to extend the reach to pregnant women, young children, and other hard to reach populations.
Engaging religious leaders and volunteers from within the faith communities, and in close partnership with the Sierra Leone government’s National Malaria Control Programme (NMCP) and the Inter-Religious Council of Sierra Leone (IRC-SL), TBFF identified an innovative, sustainable and potentially cost-effective approach to malaria control that leverages and strengthens existing local resources, with capacity for nationwide reach. Analysis of FASL budget expenditure suggests the model is cost-effective: the cost per person directly reached was 0.32 GBP (based on direct in-country projects costs) and the cost of training and resourcing each MFA and MFC was £1,919 and £55 respectively. This indicates the potential for the model to be a cost-efficient way of reaching high numbers of hard-to-reach people with simple messages that encourage changes in practices and behaviour related to social issues.
The WHO Malaria Report 2016 reveals significant progress in malaria control in Sierra Leone over the period in which FASL was operational, with the country achieving the highest decline in malaria deaths in West Africa between 2010 and 2015, and an almost 30% reduction in new cases.
Between 2011 and 2016, FASL reached over 3 million people across Sierra Leone, mobilising a network of more than 700 religious leaders and 20,000 volunteers to communicate life-saving messages of malaria prevention.
FASL used a simple Training of Trainers (ToT) cascade model for disseminating its messaging to households and communities, empowering participants to take ownership of the issue in their own communities.
Religious leaders were trained in Muslim and Christian pairs to be Malaria Faith Ambassadors (MFAs) to support efforts to prevent malaria in their communities. These leaders were equipped with the knowledge and skills to facilitate malaria discussions with their communities, and to recruit, train and mobilise community volunteers (given the title Malaria Faith Champions, MFCs) who then passed on their knowledge of malaria prevention and effective treatment through household visits, based on government (WHO-endorsed) key messages. A robust combination of tools and strategies were used, not only to achieve effective Behaviour Change Communication (BCC) in different communities, but also to ensure the sustainability of the programme by making it community-led and owned.
FASL’s volunteers were given comprehensive training including: education on malaria and how best to prevent and treat it; religious perspectives on volunteerism and its value in effective community advocacy; skills to communicate and change behaviour using a simple picture book; how to plan and monitor activity and collect data. Monthly feedback sessions enabled regular opportunities for interaction and sharing of knowledge. Social mobilisation accompanied local campaigns with targeted use of media and community events, such as theatre, parades, sporting activities and group discussions. Activities at community level were reinforced by wider mass media campaigns in main population centres.
Over a nine-month period, the model enables one religious leader to train 60 community volunteers (in sets of 20 at a time) with each volunteer visiting 40 households – meaning just one religious leader indirectly reaches up to 2,400 families (up to 13,400 people based on an average Sierra Leonean household size of 5.9).7
During the pilot phase (2011-12) the programme worked with the Ministry of Health and Sanitation (MoHS) and the IRC-SL to train 97 religious leaders and 3,000 community volunteers. By April 2012, they had reached 100,000 households (600,000 individuals) across all 14 districts.
Seeing potential in the pilot model, in 2013 GlaxoSmithKline partnered with TBFF to scale up the programme with an aim of reaching 80% of the population. Building on existing partnerships with the NMCP and IRC-SL, and working closely with a local delivery partner and an expert advisory committee of Muslim and Christian leaders, by the end of the second phase in 2014, the programme had mobilised a total of 634 faith leaders (MFAs) and over 14,000 volunteers (MFCs), reaching over 30% of the population by April 2014, focused on eastern and southern districts. At this point, the materials and messages were simplified in response to lessons learned in districts with lower literacy rates.
In 2014, the protracted Ebola epidemic meant the programme had to pause its efforts on malaria and leverage its networks of volunteers to spread Ebola prevention messages through local media programmes and preaching from the pulpit, in support of the Government’s efforts to eradicate the disease.
From 2015 to 2016, working in partnership with Caritas Freetown, the third and final phase simplified the model for a post-Ebola context where a broader range of information was being communicated to communities. The model was adapted to cover a greater number of households with fewer religious leaders in five districts prioritised for their high prevalence of malaria and low coverage of malaria outreach (Port Loko, Kambia, Koindugu, Kono and Moyamba). Materials and training was adapted to incorporate key messages on the differences between Ebola and malaria. The number of messages was reduced from five to three key messages, focused on the recommendations by the World Health Organisation (WHO) and aligned with the Government’s Malaria Control Strategic Plan (see Box 2). This phase trained 109 MFAs and 5,690 MFCs who reached over 1.4m people (79% of the target population of the five districts) through household visits and community outreach activities.
In 2015-16, the FASL team conducted an internal assessment (pre- and post-training) with trained religious leaders from four districts, which demonstrated their knowledge of correct malaria symptoms and treatment had increased by 45%. At the end of the assessment period, the team verified that 99% of the households had been re-visited a month after the initial visit to assess take-up of the messages and any change in practices at household level. Three out of four of the re-visited households demonstrated positive behavioural change as certified by the MFCs, with 75% of visited households with at least one ITN reporting they had used it the previous night.
FASL THREE KEY MESSAGES
- Sleep under a treated net every night: Give priority to pregnant women and children under-5
- Protect our pregnant women: Encourage them to go to antenatal check-ups to receive a free Long Lasting Insecticide-Treated Net (LLITN) and Intermittent Preventative Treatment (IPTp) of malaria in pregnancy.
- Act quickly when you or your child has a fever: Go to the health center and take the FULL dose of malaria medication (ACT) given.
Two external evaluations occurred over the life of the programme.
The first was conducted by Ipsos MORI and Dalan Development Consultants during phase one (2011-12). It comprised a baseline survey of 504 respondents and repeated the survey after one-year of 1,008 respondents in four of FASL’s operational districts (Bo, Makeni, Western Rural Area and Western Area Urban). This evaluation found that one in five people had received a visit from someone about malaria who they thought was an MFC or from TBFF, with 45% recognising the picture book and 29% recognising the household certification sticker. The evaluation found an increase over the period in those correctly using bed nets and in children completing the full courses of medication. The report concluded that awareness of malaria prevention and treatment was high but that more should be done to address the high numbers of reported cases in children and in helping people translate their knowledge into practice (e.g. correct bed net use and not using nets with holes in).
Between January and November 2016, a second evaluation was undertaken of different locations by local organisations, Focus 1000 and Dalan Development Consultants, in order to explore the evidence of the effectiveness and impact of FASL in changing attitudes, behaviours and practices to prevent malaria through increased cohesion between faith groups. Mixed method research was conducted in two stages (baseline and endline) in two sample districts (Kambia in Northern Province and Port Loko in Western Area Province). A quantitative household survey of 1,583 people from 583 households measured shifts in knowledge and practices and the community level at the endline, complemented by Focus Group Discussions (FGDs) and Key Informant Interviews (KIIs) with 340 volunteers and community members to assess the levels of interaction and cohesion, as well as the extent of penetration and attribution to FASL activities. The research also conducted in-depth interviews with key informants (stakeholders and partners) to gain insight into lessons learned and perspectives for the sustainability of the programme model.
IMPACT ON COMMUNITY COHESION
The FASL programme capitalised on Sierra Leone’s positive inter-faith environment to promote behaviour change for the prevention and proper healthcare treatment of malaria amongst the population. Overall, the evaluation showed that FASL was successful in building on the tolerant religious environment to foster improved interactions and collaboration between the dominant faith communities – Muslim and Christian. Working together and training together, often in each other's places of worship, faith leaders and their communities had the opportunity to overcome misconceptions of the other and promote greater understanding and trust. Overall, evidence from interviews with faith leaders and community volunteers showed that:
- Inter-faith interactions have improved as a result of the joint training and community sensitisation activities between faith leaders. As a result of FASL, 87% of participating faith leaders and volunteers reported seeing an increase in positive interactions between different faith groups in their communities (an increase of 14 percentage points from the baseline).
- As a result of attending joint training and community sensitisation activities, there is evidence that some faith leaders and their communities have overcome misconceptions of the other, and have changed negative perceptions they had about the values and beliefs of the other faith.
- MFAs and MFCs reported that the FASL model has contributed to improving trust by providing an opportunity to learn more about the other faith.
The relationship with the other Faith Ambassadors in my community is good. He is a Muslim and we are now best of friends. We are living in the same community but we have never had any form of interaction before, but because of this Faiths Act programme we can now interact. Whenever I invite him to Church he honours my invitation.
Key Informant Interview Malaria Faith Ambassador in Kambia District
IMPACT ON KNOWLEDGE AND PRACTICES
The findings from this evaluation show a generally high level of awareness about malaria across both districts both before and after the intervention, but notable improvements were seen in practices relating to children under five and pregnant women. Knowledge of the causes and symptoms of malaria was fairly high amongst the surveyed communities at the baseline, and it remained high, measured for example by the number of respondents who correctly identified mosquito bites as the only cause of malaria transmission (96.4%) and fever as the main symptom of malaria (87.8%). The endline survey demonstrates an increased awareness of the correct (Government and WHO-recommended) malaria prevention and treatments, and, importantly, a decline in incorrect prevention and treatments, indicating a positive effect on knowledge about the most effective malaria prevention practices.
The key findings are as follows:
- The proportion of people who think malaria can be prevented by sleeping under either a bed net or an ITN every night increased by 3.5 percentage points on average to 92.7%, reflecting the relatively high levels of knowledge at the start.
- The proportion of people who had misconceptions, for example that drinking polluted water or eating contaminated food causes malaria halved.
- Respondents mentioning the WHO-recommended Artemisinin-based Combination Therapy (ACT) as the most effective treatment from malaria increased on average by 6.5 percentage points across both districts.
- The majority of households in both districts reporting having at least one ITN that was used the night before the endline survey and this increased by 7.2 percentage points on average to 97.5%.
- The proportion of household with one or more child under-five that slept under the bednet the night before the survey increased significantly by an average of 29 percentage points in both districts.
- At the endline, 75% of pregnant women reported that they slept under an ITN the night before the survey.
- There was a slight increase of 4.9 percentage points in households in both districts reporting that children under-five with a fever were taken to a medical facility. The proportion of those who sought medical care within 24-48 hours increased significantly in Port Loko by 14.4 percentage points.
- The proportion of pregnant woman who took WHO-recommended anti-malarial drugs during their pregnancy significantly increased by 17.1 percentage points to 54.6% on average across both districts.
PENETRATION AND ATTRIBUTABLE CONTRIBUTION OF FASL
The KAP survey examined respondents’ exposure to malaria campaigns generally, and to FASL’s malaria treatment and prevention campaign. In order to verify the reach and assess FASL’s contribution, the survey tested participants’ recall of FASL-branded materials and the activities of MFCs. In total 70% (1,099) of those surveyed in intervention households responded to questions about the extent of penetration by MFAs and MFCs. The survey found that:
- Over 90% of respondents had a household visit from someone to discuss malaria in the previous six months, of these over 80% think their visitor was a FASL volunteer (MFC).
- High proportions of people recognised pictures of materials of the FASL programme, with the household sticker being the most widely recognised items (by 97% of respondents).
- Those who identified their visitor as a FASL volunteer demonstrated a higher level of knowledge (by an average of 7 percentage points) of malaria symptoms and correct malaria prevention and treatments such as knowledge of important malaria symptoms such as fever, sweating, loss of appetite, dizziness and headache.
- Respondents who were able to recall MFCs by their title were more likely (by 9 percentage points) to believe malaria to be preventable by sleeping under a bed net and almost twice as likely to seek treatment at a medical facility for symptoms.
At programme closure, discussions were underway on the benefits of mobilising networks like FASL’s for other health and social issues, such as water, sanitation and hygiene (WASH). While the GoSL has recently adopted a Community Health Worker (CHW) strategy, it will take some time to become effective; networks of FASL volunteers could play a crucial role in managing the transition or facilitating the strategy using their outreach networks. At present, there is an opportunity at district level to ensure religious leader networks and CHWs are coordinated and complementing each other’s efforts.
Collaboration and effective local partnerships are critical to ensuring interventions complement existing systems and to ensuring sustainability. Building and sustaining effective local partnerships with people on the ground was core to FASL’s operational model. From the start, we worked closely with local partners to design activities to complement the existing bed net distribution programme, and to learn from local organisations’ experience of healthcare in Sierra Leone, supplementing our experience of training and engaging faith communities. Embedding these partnerships early gave helped ensure local ownership.
Initiatives that rely on behavioural change need to engage whole communities. Changing behaviour often means not just changing an individual’s attitude and practices, but changing the sociocultural norms underpinning an entire community. Collaboration based on community needs steers action away from competition towards positive outcomes for the community as a whole. FASL tackled one of the greatest obstacles to long-term human development by building understanding and knowledge together as a household and then as a community, recognising that change occurs at individual, household and community level. Early linkages between government, local civil society and community/religious leaders meant FASL volunteers were better equipped to engage their communities. These partnerships helped cross boundaries, enabling each group to tap into each other’s networks – and this was leveraged particularly effectively during the Ebola crisis. Religious leaders can strengthen existing support structures within these complex networks because they already have local buy-in and trust. In turn, health teams benefit from access to their faith networks and can build greater trust in government institutions through direct engagement with community leaders. This can help them to get a better understanding of the nuanced local issues and reality faced by the communities they serve.
Plan evaluation from the beginning. Evaluation should explore not just whether a change has occurred, but what change ocurred, and how. Failing to think about an evaluation at the start means you don’t know if you are measuring the right things, collecting the right data, or asking the right questions. For behavioural change programmes this is particularly challenging because attribution is less directly traceable and change is a lengthy, social process. It is important, therefore, to measure outcomes periodically over a long period to track changes over time. Failing to have a consistent evaluation strategy from the start compromises the ability to compare mid-point and end-point evaluations, or to make adjustments based on emerging findings. This is particularly challenging in an environment like Sierra Leone where there is a high concentration of activity (complicating attribution) and weak infrastructure (e.g. bad quality phone and internet reception, particularly in rural areas). A control trial would provide a useful measure of comparison with a non-operational location.
Projects must be designed with transition in mind. A robust exit strategy should be developed early on involving a capable local partner. This means that if there are areas for capacity development to ensure it can be successfully handed over, these can be addressed and developed along the way. Responding to lessons learned during its initial handover in its final phase, the FASL team had to develop further procedures to monitor processes for finance and funding, programme management, resource allocation, data collection and management, training of staff and assessment of the capacity within the partnership to deliver and sustain the programme’s objectives in the longer-term.
Community cohesion requires both dialogue and clear understanding of the groups interacting. Creating opportunities for religious communities to collaborate with one another helped to break down misconceptions of the other and promote a better understanding of different beliefs, but we found that encouraging this interaction is not easy where there is inter-group tension. Significant efforts have to be put into building effective dialogue. Initiatives seeking to bridge the gap between religious groups – whether to build cohesion or tackle a specific issue faced within the community such as malaria – first need to understand the context (e.g. sources of tension) in order to build stronger foundations for an effective interaction. Our evaluation found that not everyone changed their view about differences in religious belief, but they did learn how to put differences to one side for a common cause.
Simplicity. The reliance on volunteers, the proliferation of various campaign messaging (especially in the wake of Ebola), and the very low literacy rates in Sierra Leone (27% for women and 45% for men) meant the programme benefitted from simplifying its approach. Fewer messages presented pictorially were easier for the volunteers to learn and easier for communities to absorb. Keeping things simple also helps with quality control; the cascade model presents some challenges for ensuring messages are conveyed accurately during transmission and do not get diluted (regular monitoring is vital to check this).
Value for Money. Assessing FASL’s cost-effectiveness was challenging due to the different delivery phases, changes in the model and consistency in the data collection. But future similar programmes should explore the cost-effeciency of different models (e.g. size of cascade groups, messaging on single subjects versus integrated health messaging) as well as the cost of the volunteering model (e.g. levels of subsistence needed to sustain volunteers long-term and calculating the value of time-spent). This would help to provide a robust justification for the relative cost-effectiveness and sustainability compared to other schemes such as the CHW programme.
TBI aims to incubate ideas, demonstrate their impact and scalability, and advocate evidence-based approaches to others. The success of FASL provides evidence to suggest this innovative and potentially cost efficient methodology could be developed, modified and replicated in other countries with similar public health burdens. The following recommendations are intended to help any organisation designing and implementing similar programmes, as well as partners in Sierra Leone intending on continuing to use the FASL model and networks.
Adapt and replicate the FASL model in other countries with similar public health issues and limited social infrastructure - particularly related to easily preventable death from diarrhoea, measles, malaria, cholera, pneumonia, poor nutrition, lack of immunisation or basic maternal healthcare. The FASL model is flexible and could be integrated into wider social development policy in national preventative health programmes in developing countries with hard to reach communities and strong networks of religious communities. Use of the model for integrated prevention or behavioural change campaigns – providing the messaging is kept simple – could prove even more cost-effective.
Leverage existing faith communities to deliver life-saving messages to an entire population. They are to be found in all areas, including hard-to-reach ones. Working with faith communities has potential to be cost-efficient where extensive faith networks already exist because they do not need to be artificially created and they can provide continuous, reliable access, enabling longitudinal monitoring and assessment of changes in attitudes and behaviours. They can also provide much-needed dissemination infrastructure for government where social welfare resources are limited. But more research is needed into the cost-benefit for volunteers in terms of their time spent, into how cultures of volunteerism differ between contexts, and how volunteerism can be incentivised in the long-term.
Invest in evaluation early. Build evaluation into the design of the programme to ensure data is measured consistently over implementation, and to ensure the right data (e.g. to determine cost-effectiveness) is being collected. Consider a control group design to help make a stronger case for attribution to impact.
Consider how to expand reach into remote communities. Using faith leaders’ helps to access hard to reach populations, but also consider the gender balance of volunteers (i.e. ensuring women are included as they are likely to have greater access to female community members in some contexts) and to logisitical support such as motorbikes to extend reach even further.
Build local ownership into transition planning from the start. To ensure a responsible exit, donors should engage early with credible local partners to ensure capacity is sustained beyond the life of the project intervention. Integrating it into local government systems where possible will ensure its sustainability.
A holistic, community-based approach is needed to promote behaviour change. Changing attitudes towards certain issues and influencing different behaviour requires investment in a supportive environment, stimulation of learning, and regular dialogue with the target community. Changing behaviour often means changing the sociocultural norms underpinning an entire community. An individual’s relationship with their family, their peers, their community and wider society can substantially affect how that person behaves.
- Sierra Leone Malaria Control Strategic Plan, 2011-15
- World Malaria Report, 2016 http://apps.who.int/iris/bitstream/10665/252038/1/9789241511711-eng.pdf?ua=1
- Sierra Leone: Multiple Indicator Cluster Survey (MICS), 2010
- Sierra Leone: Malaria Indicator Survey (MIS), 2013
- Sierra Leone Census, 2004 (78% Muslim (primarily Sunni) and 21% Christian)
- The first policy was developed in June 2012 and it was revised in February 2017 http://www.chgn.org/wp-content/uploads/2014/11/Sierra-Leone-CHW-policy.pdf