3 Indicators
3.1 CMAM coverage
CMAM coverage usually pertains to coverage of SAM treatment. Historically, there have been two coverage estimators in common use: point and period coverage.
Point coverage is the number of current SAM cases in a treatment programme divided by the total number of current SAM cases.
Point coverage uses data for current cases only. It is calculated using the following formula:
$nbsp;
\[\begin{aligned} \text{Point coverage} & ~ = ~ \frac{C_{in}}{C_{in} ~ + ~ C_{out}} \\ \\ where: & \\ \\ C_{in} & ~ = ~ \text{current SAM cases in the programme} \\ C_{out} & ~ = ~ \text{current SAM cases out of the programme} \end{aligned}\]
Point coverage provides a snapshot of programme performance, putting a strong emphasis on the effectiveness and timeliness of case-finding and recruitment (Myatt et al. 2012).
Period coverage, on the other hand, uses data for both current and recovering cases. It is calculated using the following formula:
\[\begin{aligned} \text{Period coverage} & ~ = ~ \frac{C_{in} ~ + ~ R_{in}}{C_{in} ~ + ~ C_{out} ~ + ~ R_{in}} \\ \\ where: & \\ \\ R_{in} & ~ = ~ \text{recovering SAM cases in the programme} \end{aligned}\]
Period coverage is the number of current and recovering cases in a treatment programme divided by all current SAM cases and recovering cases. It approximates treatment coverage much better (albeit with limitations) as it accounts for children who are no longer cases but are in the programme.
Point and period coverage both have their limitations.
Point coverage of a programme with good case-finding and recruitment and short lengths of stay can be misleadingly low because there are too few current cases. For example, a coverage survey found:
| Number of current cases | 2 |
| Number of current cases in the programme | 0 |
| Number of current cases not in the programme | 2 |
| Number of recovering cases in the programme | 34 |
In this scenario, the point coverage estimator returns:
\[ \text{Point coverage} ~ = ~ \frac{C_{in}}{C_{out}} ~ = ~ \frac{0}{2} ~ = ~ 0 ~ = ~ 0\% \]
but the period estimator returns:
\[ \text{Period coverage} ~ = ~ \frac{0 ~ + ~ 34}{0 ~ + ~ 34 ~ + ~ 2} ~ = ~ 0.944 ~ = ~ 94.4\% \]
In this regard, the point coverage estimate penalises good performance and the period coverage most likely better depicts the coverage of the programme.
On the other hand, a programme with poor case-finding and recruitment and long lengths of stay due to late presentation and/or late admission may have a period coverage that is misleadingly high because of high number of recovering cases. In such a scenario, the two estimators will yield very different results. For example:
| Number of current cases | 12 |
| Number of current cases in the programme | 3 |
| Number of current cases not in the programme | 9 |
| Number of recovering cases in the programme | 22 |
\[ \text{Point coverage} ~ = ~ \frac{C_{in}}{C_{out}} ~ = ~ \frac{3}{12} ~ = ~ 0.250 ~ = ~ 25.0\% \]
but the period estimator returns:
\[ \text{Period coverage} ~ = ~ \frac{3 ~ + ~ 22}{3 ~ + ~ 22 ~ + ~ 9} ~ = ~ 0.735 ~ = ~ 73.5\% \]
In this example, point coverage is the more reflective coverage of the programme.
It should be noted also that period coverage has a tendency to an overestimation bias. This is because the current period coverage estimator does not take into account cases of acute malnutrition who have recovered spontaneously but were never enrolled in any treatment programme.
An estimator of coverage that does include both recovering cases that are in the programme and recovering cases that are not in the programme and, thus, provides an unbiased estimator of overall programme performance is:
\[\begin{aligned} \text{Single coverage} & ~ = ~ \frac{C_{in} ~ + ~ R_{in}}{C_{in} ~ + ~ R_{in} ~ + ~ C_{out} ~ + ~ R_{out}} \\ \\ where: & \\ \\ R_{out} & ~ = ~ \text{Recovering SAM cases not in the programme} \end{aligned}\]
It is for these reasons that the coverage assessment technical guide (Myatt et al. 2012) recommends that only one of these estimators be reported and the choice of estimator to report should be guided by specific programme features and characteristics (such as lengths of stay in the programme) that would justify the choice of reported estimator. In a recent Epicentre review (Epicentre 2015) this has been highlighted as a source of confusion and issues given the possibility of period coverage being chosen more as a coverage estimator rather than point coverage because of it being a higher estimate even if the programme characteristics do not merit its use. The review suggests that both coverage indicators could be reported, with sufficient context (e.g. on length of stay, timeliness of admissions, etc.) to allow for their interpretation.
In response to the review and to address the limitations of the coverage estimators, development work has been conducted on further developing and improving the coverage estimators to address this confusion and the issues around them (Balegamire et al. 2015). This work focused primarily on improving the period coverage estimator to address the overestimation bias described earlier and to make it more closely approximate the treatment coverage estimator formula shown above.
The problem with this estimator is that \(R_{out}\) (i.e. the number of recovering cases that are not in the programme) is unknown and may be difficult to collect accurately. This problem of estimating the number of recovering cases not in the programme (\(R_{out}\)) may be addressed using a simple mathematical model2 proposed by Balegamire et al. (2015).
\[\begin{aligned} \frac{C_{in} ~ + ~ C_{out}}{C_{in}} & ~ \approx ~ \frac{R_{in} ~ + ~ k ~ \times ~ R_{out}}{R_{in}} \\ \\ where: & \\ \\ k & ~ = ~ \text{a correction factor} \end{aligned}\]
\(R_{out}\) can then be expressed in terms of the known variables:
\[ R_{out} ~ \approx ~ \left \lfloor ~ \frac{1}{k} ~ \times ~ \left ( ~ R_{in} ~ \times ~ \frac{C_{in} ~ + ~ C_{out}}{C_{in}} ~ - ~ R_{in} ~ \right ) ~ \right \rfloor \]
Given the possibility that no cases in the programme (\(C_{in}\)) are found, the calculation is adjusted by adding 1 to \(C_{in}\). To arrive at a whole number value for \(R_{out}\), the calculation is rounded off towards zero.
\[ R_{out} ~ \approx ~ \left \lfloor ~ \frac{1}{k} ~ \times ~ \left ( ~ R_{in} ~ \times ~ \frac{C_{in} ~ + ~ 1 ~ + ~ C_{out}}{C_{in} ~ + ~ 1} ~ - ~ R_{in} ~ \right ) ~ \right \rfloor \]
This leaves the problem of deciding a suitable value for the correction factor (\(k\)). A reasonable candidate for \(k\) is the ratio of the mean length of an untreated episode to the mean length of a CMAM treatment episode.
\[ k ~ = ~ \frac{\text{Mean length of untreated episode}}{\text{Mean length of a treatment episode}} \]
Possible value for mean length of untreated episode is 7.5 months (Garenne et al. 2009) which is the common value used when estimating programme case-loads from prevalence estimates (Myatt 2012). Mean length of a treatment episode can be estimated by calculating the mean length of stay in the CMAM programme using routine monitoring data. In general, A value of 2.5 months could be used in the absence of better information or when the validity of routine programme monitoring data is suspect. Using these values, k is:
\[ k ~ = ~ \frac{\text{Mean length of untreated episode}}{\text{Mean length of a treatment episode}} ~ = ~ \frac{7.5}{2.5} ~ = ~ 3 \]
The inclusion of recovering cases means that the single coverage estimate is mathematically constrained to return a coverage estimate that is greater than or equal to the point coverage estimate. The underestimation present in the point coverage estimate has, to some extent, been corrected. The inclusion of recovering cases that are not in the programme means that the single coverage estimator is mathematically constrained to return a coverage estimate that is less than or equal to the period coverage estimate. The overestimation present in the period coverage estimate has, to some extent, been corrected.
Given this single coverage estimator, a shift in terminology is proposed that is more descriptive and specific with regard to what the estimator is actually measuring, allowing both measures to be reported together without confusion. Point coverage is now named case-finding effectiveness to more precisely reflect it as a measure of the programme’s ability to find and recruit current cases. This indicator assesses how good the treatment programme is in finding cases of SAM and then getting them to treatment. Period coverage that has been improved into the single coverage metric is now named treatment coverage as this is the estimator that approximates this coverage indicator the closest.
3.1.1 Requirements
This indicator will require additional programmatic information to be operationalised specific for Liberia.
It would be ideal that a Liberia-specific value for mean length of a treatment episode be estimated using routine programme data from a selection of cured cases from a selection of health centres providing CMAM services in the two counties in which the survey is to be implemented. This data should be collected from the beneficiary cards of cured cases. For each county, a total of 30 health centres from the full list of health centres in the county can be selected systematically. Of these 30 health centres, a sample of 30 records of discharged cured cases should be selected. From these records, data on date of admission and date of discharge should be extracted collated into a data format (ideally a comma-separated value or CSV file) with each of these values in a separate column.
This data will allow the calculation of the mean length of stay in the CMAM programme of discharge cured cases.
\[ \text{Mean length of stay} ~ = ~ \frac{\text{Length of stay in programme (days)}}{\text{Number of records}} \]
For data collection, enumerators will require:
A height board;
A digital weighing scale; and,
A MUAC tape.
Traditionally, if data collection is done using pen and paper, enumerators are provided with z-score tables for weight-for-height so that they can determine if child is SAM. Since this survey will utilise digital data collection (see Section 5), the digital form will determine whether the child is SAM by MUAC and/or weight-for-height and/or oedema and will instruct the enumerators on what to do next if the child is a SAM case.
Referral slips will also be needed so that children who are identified as SAM but not in the programme can be referred.
3.2 Vitamin A supplementation
The standard estimator for vitamin A supplementation is the proportion of children aged 6-59 months who received two age-appropriate doses of vitamin A in the past 12 months.
In standard surveys such as the DHS and MICS, this indicator is adjusted to a recall of 6 months for a single age-appropriate dose of vitamin A.
In order to determine whether supplementation with vitamin A is age appropriate, vitamin A supplementation should first be assessed on the child’s health card. Provision of vitamin A card is usually recorded on the child’s health card with the corresponding does given. If health card not available or if health card is lost or if the child doesn’t have a health card at all, then the mother/caregiver will have to be asked whether their child has received vitamin A in the past 6 months and respond by recall. If the mother/caregiver says yes, then the next question to ask will be which type of gel capsule was provided. The blue vitamin A gel capsule containing 100,000 IU of vitamin A is given to children 6-11 months. The red vitamin A gel capsule containing 200,000 IU of vitamin A is given to children 12 - 59 months. A sample of the blue and the red gel capsule (or a photo of the capsules) can be used to aid the mother/caregiver in answering this question.
Given this, two indicators can be reported on vitamin A supplementation.
Any vitamin A supplementation in the past 6 months.
Age-appropriate vitamin A supplementation in the past 6 months.
3.2.1 Requirements
Either samples and/or photos of both the blue and red gel vitamin A capsules will be needed per enumerator to use in aiding the mother to answer which type of vitamin A gel capsule has been provided to the child.
3.3 Iron-folic acid (IFA) supplementation for pregnant women
Population-based surveys typically report the percentage of women with a live birth in the two to five years before the survey who received and took IFA supplementation during their most recent pregnancy. Because antenatal care (ANC) is typically the main platform for IFA supplement distribution for pregnant women, survey questions on antenatal care attendance and timing of the first antenatal care visit can provide information on the use of this platform to deliver IFA supplementation. Sununtnasuk, D’Agostino, and Fiedler (2015) propose a falter point framework3 that utilises four indicators that proxy the four critical points at which the ANC approach to IFA distribution might falter in IFA supplementation coverage to pregnant women. These indicators are:
At least one ANC visit during most recent pregnancy
Receipt or purchase of IFA tablet/s
IFA consumption
Adherence to 180 days of supplementation
3.3.1 Requirements
It would be ideal for each enumerator to have a sample/s and/or photos of an IFA tablet to show the respondent as an recall aid or prompt. Given that some mothers may have to recall as far back as 5 years and that they might not be able to remember that the tablet is called, showing them a sample of the tablet might help them remember if they have taken it.
3.4 Micronutrient powder supplementation
The indicator for coverage of micronutrient powder supplementation is the proportion of children aged 6-23 months who consume micronutrient powder supplements. Depending on the programme protocol on mechanism of distribution and effective intake of MNP, a full indicator set on MNP supplementation can be devised that will be similar to the IFA supplementation falter point or bottleneck framework. For example, if MNPs were being provided through the health centres or health posts, then the following indicators can be assessed hierarchically:
Health centre / health post attendance in the past month
Awareness of MNP
Consumption of MNP
3.4.1 Requirement
It would be ideal to provide enumerators with sample/s and/or photo/s of MNP sachets used in Liberia. If there are various brands of MNP distributed, it would be good to have sample/s and/or photo/s of each one. These sample/s and/or photo/s can be used to help mothers or caregivers recall or know what an MNP is and be able to ascertain whether they do know it or have heard of it and whether their child has consumed it.
3.5 IYCF counselling
There are no standard indicators for IYCF counselling. Any indicator developed for this programme will depend on the mechanics of how the IYCF counselling is delivered and who the target beneficiaries are. In terms of mechanism, what is known so far is that these sessions are delivered via the health clinic/health post and that the target beneficiaries are pregnant women. Given this, similar approach to the IFA supplementation coverage of falter points/bottle necks can be used with the following indicators:
At least one ANC visit during most recent pregnancy
Awareness of IYCF counselling (have they been advised IYCF counselling when they attended ANC)
Attendance to IYCF counselling
3.5.1 Requirement
It would be ideal to have the training materials used for IYCF counselling and the materials used when conducting IYCF counselling. These materials can be used as a recall aid for mothers/caregivers who are being asked about whether they know or have heard about IYCF counselling and whether they have attended.
References
Myatt, Mark, Ernest Guevarra, Lionella Fieschi, Allison Norris, Saul Guerrero, Lily Schofield, Daniel Jones, Ephrem Emru, and Kate Sadler. 2012. Semi-Quantitative Evaluation of Access and Coverage (SQUEAC)/ Simplified Lot Quality Assurance Sampling Evaluation of Access and Coverage (SLEAC) Technical Reference. Washington, DC: FHI 360/FANTA.
Epicentre. 2015. “Open review of coverage methodologies: Questions, comments and ways forward.” Epicentre.
Balegamire, Safari, Katja Siling, Jose Luis Alvarez, Ernest Guevarra, Sophie Woodhead, Alison Norris, Lionella Fieschi, Paul Binns, and Mark Myatt. 2015. “A single coverage estimator for use in SQUEAC, SLEAC, and other CMAM coverage assessments.” Field Exchange, March.
Garenne, Michel, Douladel Willie, Bernard Maire, Olivier Fontaine, Roger Eeckels, André Briend, and Jan Van den Broeck. 2009. “Incidence and duration of severe wasting in two African populations.” Public Health Nutrition 12 (11): 1974–82.
Myatt, Mark. 2012. “How do we estimate case load for SAM and / or MAM in children 6 59 months in a given time period ?” CMAM Forum.
Sununtnasuk, Celeste, Alexis D’Agostino, and John L Fiedler. 2015. “Iron+folic acid distribution and consumption through antenatal care: identifying barriers across countries.” Public Health Nutrition 19 (04): 732–42.