With the adoption of the CMAM model at sub-national to national levels in countries such as Ethiopia, the need for a coverage method that can provide information about coverage over large areas has become important. In the Conference on Government experiences of Community-based Management of Acute Malnutrition and Scaling Up Nutrition held in Addis Ababa, Ethiopia in 2011, the Ethiopian delegation reported that a key learning point of their CMAM scale-up experience was the “need for systems to monitor and improve quality of services early on”1 2 and highlighted the importance of strengthening monitoring and evaluation to maintain quality of programming. Figure 1 gives an idea of the geographic scale of the acute malnutrition needs and the current programming scope.

Figure 1: Ethiopia CMAM scale up

Audit cycle as a framework for monitoring and evaluation

An audit is a quality improvement and monitoring method that seeks to improve service delivery through systematic review against specific criteria and standards and the implementation of change. The most commonly used framework for an audit is the audit cycle shown in Figure 2.

Figure 2: Audit cycle framework

Over the years of development of the CMAM model, the use of coverage survey methods such as CSAS, SQUEAC and SLEAC within the audit cycle framework as a tool for establishing current practice has been established3. These tools can be used and implemented in a timely manner and at reasonable cost. However, in the current context of wide adoption of CMAM and scale-up, the issue of scale and space is an additional factor to take into account when establishing current practice. CMAM programs across a much larger geographic scale will often be expected to have much higher spatial variation of coverage. Hence, from an audit cycle framework, documenting that spatial variation is necessary not only to assess which areas are meeting coverage standards and the corresponding program reforms needed, but also on how resources can and should be allocated.

CSAS, SQUEAC and SLEAC are methods that allow for the assessment of spatial variation of coverage4 5 6. However, they are not optimal for assessing coverage in large areas (e.g. national, sub-national or regional) as they tend to take longer time (thereby increasing cost) for a limited increase in area size surveyed.

S3M as a wide-area coverage survey method

The Simple Spatial Survey Method or S3M was developed from the CSAS coverage survey method as a response to the widespread adoption of CMAM by ministries of health. Large-scale programmes need a large-scale survey method and S3M was developed to meet that need.

S3M uses improved spatial sampling and more effective use of data, has been developed to address these issues of space, time and cost and could be valuable as a coverage monitoring and evaluation tool over large areas. S3M has been created / invented by Mark Myatt based on work commissioned by Concern Worlwide7 to address this issue of scale in the context of CMAM programming in Ethiopia.

The design philosophy behind the creation and development of S3M is that of “building on the shoulder of giants”. Three key principles are at the foundation of its design and development of S3M.

Construct from existing, familiar, tested, and easy to use components

There are numerous methods or techniques available for variety of survey purposes that have all stood various field tests and applications. Recognising what each method is good for and then harnessing this appropriately in combination with other methods lead to the creation of robust approaches to the assessment of coverage.

Keep it simple

The component methods utilised in the development of new approaches provide results and information considered essential to coverage assessment. Simplicity of the techniques developed is ensured which allows for easy, quick and cheap implementation without compromise on richness of information on coverage.

The best is the enemy of the good

For a method that utilises various different approaches / component methods, there will always be tensions between component methods. There would be instances when some methods are not implemented in the “classical” or “ideal” way. They are adapted in a manner that allows them to work well together with all other component methods within the overall S3M framework.

S3M was designed to:

  • Be simple enough for MoH, NGO and UNO personnel to perform.

  • Be able to survey areas up to ten times larger than the CSAS method for approximately twice the cost whilst maintaining the spatial resolution of CSAS surveys.

  • Provide a general survey method. S3M can be used to survey and map:

    • Coverage of selective entry programmes such as CMAM, universal programmes such as the expanded programme on immunisation (EPI), growth monitoring programme (GMP) and supplementary feeding programme (SFP) over wide areas; and,

    • Levels of indicators such as (e.g.) infant and young child feeding (IYCF) practices, water, sanitation and hygiene (WASH), and prevalence over wide areas.

S3M survey design

S3M primarily uses a two-stage sampling design.

The first stage sample is a spatially-stratified sample of clusters. This is done through the following steps:

  • Step 1: Find a map and decide the area to be represented by each sampling point (Figure 3)

  • Step 2: Draw a grid over the map (Figure 4

  • Step 3: Create an even spatial spread of sampling points (Figure 5)

  • Step 4: Select communities to sample and label each sampling point (Figure 6)

  • Step 5: Test triangulation (Figure 7)

Steps in S3M stage 1 sampling

Find a map Grid the map Layout sampling points Select sampling villages Triangulate
First stage sample

The second stage sample is active and adaptive case finding or snowball sampling. In active and adaptive case finding, the surveyors identified potential SAM cases by talking to key informants and using local terms for malnutrition and associated illnesses. Key informants most commonly used during the survey were: village leaders, traditional birth attendant/midwife, elderly and women/mothers in the community. The search for SAM children was adapted according to information provided by the key informants and the community.

The identified children were then measured using a MUAC tape and with oedema test to assess whether they were SAM. If SAM, the carers were asked whether they were being enrolled / treated in a CMAM programme.

In addition, case-finding included children who were being treated in the programme who have already recovered or are recovering (i.e. does not meet SAM case definition anymore) but are still in the programme until they meet discharge criteria. The aim of the case finding is to find all or nearly all SAM cases in each of the villages sampled.

Results

The coverage of CMAM in Wolayita and South Wollo zone had a wide spatial variation (i.e. high spatial heterogeneity) as seen in Figure 8 and Figure 9.

Treatment Coverage Treatment Coverage
CMAM coverage in Wolayita and South Wollo Zones

Wolayita Zone IYCF

Figure 10 (below) shows the IYCF indicator set for Wolayita Zone. From this, it is clear that the main issue with regard to IYCF practices in Wolayita zone is that of age-appropriate dietary diversity (bottom middle) and exclusive breastfeeding (upper middle) particularly in the south-west area of Wolayita zone (Kindo Didaye woreda and parts of Kindo Koyisha and OFA woreda).

In the north-east area of the zone, on the other hand, show relatively good IYCF practices particularly in Deguna Fanigo woreda and parts of Damot Gale and Damot Woyide woreda.

Figure 10: IYCF indicator set for Wolayita Zone

South Wollo Zone IYCF

Figure 11 (below) shows the IYCF indicator set for South Wollo Zone. From this, the impact and extent of age-appropriate dietary diversity (bottom middle) and exclusive breastfeeding (upper middle) issues seem more pronounced and widespread throughout the zone with most areas having poor dietary diversity and only a patch of good exclusive breastfeeding practice in the middle to north-east of the zone.

Figure 11: IYCF indicator set for South Wollo Zone

Comparing to CSAS in Wolayita zone

The S3M for Wolayita zone took 11 days to complete (excluding training of surveyors) by 9 survey teams of 3 to 4 surveyors each.

CSAS in Deguna Fanigo woreda only in Wolayita zone took about the same time with the same number of surveyors. S3M in Wolayita zone (4511.71 sq. km) with 12 woredas cost 30,000 USD compared to about 10,000 USD for CSAS in only Deguna Fanigo woreda (401.5 sq. km).

This was a three-fold increase in cost for over an 11 times increase in area surveyed.

S3M provided spatial distribution applicable to a large area and a detailed map of coverage showed where more effort on program intervention was needed.

It also provided information / results on IYCF practices in the whole zone.

Conclusion

S3M can provide timely and more cost effective at scale coverage assessment compared to CSAS which is the closest comparable spatial method available. As such, S3M has all the requisite qualities of a tool that could operationalize the audit cycle framework for monitoring and evaluating CMAM coverage in Ethiopia at a large-scale.

Endnotes



  1. Emergency Nutrition Network. “Conference on Government Experiences of Community-Based Management of Acute Malnutrition and Scaling Up Nutrition: Conference Report,” 1–52, Addis Ababa, 2012. http://files.ennonline.net/attachments/1444/cmamconferencereport.pdf. 

  2. Emergency Nutrition Network. “Government Experiences of Scale-Up of Community-Based Management of Acute Malnutrition (CMAM): a Synthesis of Lessons,” 1–76, Addis Ababa, 2012. http://files.ennonline.net/attachments/1374/cmam-conference-2012-synthesis.pdf. 

  3. Myatt, Mark, Ernest Guevarra, Lionella Fieschi, Allison Norris, Saul Guerrero, Lily Schofield, Daniel Jones, Ephrem Emru, and Kate Sadler. Semi-Quantitative Evaluation of Access and Coverage (SQUEAC)/ Simplified Lot Quality Assurance Sampling Evaluation of Access and Coverage (SLEAC) Technical Reference, Washington, DC: FHI 360/FANTA, 2012. 

  4. Myatt, Mark, Teshome Feleke, Kate Sadler, and Steve Collins. “A Field Trial of a Survey Method for Estimating the Coverage of Selective Feeding Programmes..” Bulletin of the World Health Organization 83, no. 1 (January 2005): 20–26. 

  5. Guevarra, Ernest, S Guerrero, and M Myatt. “Using SLEAC as a Wide-Area Survey Method.” Field Exchange, no. 42 no. 42 (2012): 40. 

  6. Guevarra, Ernest, A Norris, S Guerrero, M Myatt. Assessment of Coverage of Community-based Management of Acute Malnutrition Technical Brief, CMAM Forum, 2012. http://www.cmamforum.org/Pool/Resources/Assessment-of-Coverage-of-CMAM-CMAM-Forum-TB-One-Oct-2012.pdf 

  7. The survey design for this project was based on the original survey design drafted by Mark Myatt as commissioned by Concern Worldwide. These original documentation is still the basis and guiding document for all S3Ms that have been implemented to date.