Tostan is looking for a Multivariate Analysis Consultant to investigate the specific effects of the CEP. To learn more and apply read on or download the consultancy terms of reference here.
Context
Tostan is a non-governmental organization based in Africa since 1991. For 25 years, Tostan has been involved in many communities. It implements a holistic education program to enable communities to take charge of their own development, with a view to driving profound change in many areas (governance, education, health, economy and environment). This Community Empowerment Program (CEP) provides community members with skills that enable them to take direct control of their own community’s development. The program uses the ‘Organized Dissemination’ model, which encourages participants and communities to share new information and ideas with their friends, families, neighbors and social network.
Tostan has been the subject of several mentions in studies and evaluations reports which show, among other things, that: it achieves overall good results in community participation thanks to its efforts to work on health goals identified by the communities themselves; her practices in community development and the abandonment of excision are cited as examples; its community-led, organized dissemination model is cited as an example of best practice in the field and is recommended to other organizations working for the abandonment of FGC; Tostan’s role in the FGC abandonment movement in Senegal is cited as one of eight innovations in history highlighting women’s empowerment and gender equality; the government of Senegal has largely based its National Action Plan for the abandonment of FGC on the Tostan program that promotes a rights-based approach; the CEP is cited for its best practices in promoting gender equality and capacity building for girls; the CEP is cited as one of the best strategies when it comes to delaying or preventing early marriage; the Parenting Practice Reinforcement program (Tostan’s post-CEP module) leads to significant improvements in the interactions between primary caregivers and infants, as well as the changes in attitudes that surround beliefs about parenting roles.
In 2014, Tostan received a grant from the Bill & Melinda Gates Foundation to strengthen its monitoring and evaluation system for projects and programs. As part of this grant, Gates contracted with a consulting firm called ITAD to work with Tostan. Thus, between October 2014 and December 2016, Tostan benefited from ITAD technical assistance to improve decision-making in terms of planning, implementation and strategic management of CEP within Tostan. At the end of this partnership, a series of operational recommendations were made to promote a better use of the information generated by the monitoring and evaluation system within Tostan, in particular to influence strategic, programmatic and ‘commercial’ decisions.
The partnership with ITAD has also sharpened Tostan’s appetite for more evidence about the impacts of the projects and programs it is implementing, namely the CEP.
What comes out, as evidence, of quantitative studies of the impact of the CEP?
Demonstrating convincing evidence that the CEP has the expected impacts will be increasingly critical in Tostan’s efforts to gain support and buy-in from donors. Several initiatives have been implemented in this direction by external partners, initiatives whose strengths and limitations can be appreciated.
Two important data collection initiatives were rolled out in 2006 and 2009 to assess the impact of the CEP. These evaluative studies resulted in three reports[1]. From the review of these three reports, Tostan drew the following lessons: the studies were unable to demonstrate a clear impact of the CEP beyond knowledge, attitudes and intentions, not because they do not exist not, but because of the limitations of the methodology used. The approach used (experimental villages / villages-controls, baseline / final evaluation) dilutes the impact of the program if it does not properly control the inherent basic differences between the villages that benefited from Tostan interventions and those that did not benefit. The raw data (averages and percentages) for the two types of villages that are the basis for all inferences on the impact of the CEP depend both on the intrinsic differences between the two village types and their exposure to the Tostan program. A valid assessment of the impact of the CEP requires effective control of these differences and this has not been done. The entire analysis is based on comparisons of the raw data without even testing the statistical significance of the observed differences. In fact, finding villages-controls to meet this methodological approach is not easy. The ‘organized dissemination’ strategy promoted by the CEP makes that comparable villages, which can serve as ‘controls’, are generally ‘contaminated’ by Tostan’s interventions.
The NGO Path has studied the impact of a Tostan intervention consisting of a PRCC +[2] in 45 communities of the department of Goudiry (Eastern Senegal). It conducted a before / after analysis using two baseline surveys (with an interval of 8 months) and two final surveys to measure changes in knowledge, attitudes, and practices induced by Tostan’s intervention. The fact that the measurements on the two basic surveys were statistically similar suggested that any changes observed between these basic surveys (combined) and the final surveys could be ‘attributed’ to Tostan’s intervention.
What alternative to use?
In Tostan, it is recognized that it is difficult, if not impossible, to demonstrate, without any doubt, the impact of CEP on certain social practices without a longitudinal methodological approach that considers a reference situation that will be carefully compared to a final situation after the implementation of the CEP.
In an intervention context where it is difficult to find control villages free of any influence of the CEP, the approach used by PATH and consisting of carrying out two baseline studies within a certain time interval can be an interesting alternative. But, in general, no other evaluative study of the Tostan program has proceeded in this way. The existing databases at Tostan level are all based on methodologies that are based on a comparative analysis between a baseline study and a final assessment, and on a comparative analysis of experimental communities and control communities with all the limitations related to constitution of this last type of communities.
An alternative for Tostan is to consider the direct beneficiary communities of Tostan’s interventions, the results of the baseline study and those of the evaluation after the end of the implementation of the CEP and to proceed carefully with multivariate analyses.
Multivariate analysis can be used to evaluate the effect of the CEP when controls do not work. This kind of analysis can be done at the individual level. This will involve comparing the odds of having a given outcome in target populations with the same characteristics that were exposed differently to Tostan interventions. Individual participation will be used to estimate the intensity of exposure without which it is impossible to make reliable inferences about the role of Tostan in the observed differences.
Exploration of the effects of the CEP through a multivariate analysis
In October 2013, Tostan launched a project called ‘Campaign for Generational Change in 3 years’ (GC3Y). This project was to build on the growing momentum of the CEP in West Africa to instill a historic, large-scale movement for the abandonment of harmful practices, the empowerment of girls and women, and the promotion of human rights.
The GC3Y project has been implemented in 150 villages: 40 in Guinea, Guinea-Bissau and Mali respectively, and 30 in Mauritania. In accordance with the monitoring and evaluation plan that accompanied the implementation of the project, a baseline study was conducted prior to the interventions and a final evaluation of the project after three (3) years of implementation.
At the level of each country of implementation, a sample equal to 20% of the intervention villages was considered for the two studies, ie 8 study villages respectively in Guinea, Guinea-Bissau and Mali, and 6 study villages in Mauritania.
A ‘before and after’ analysis of experimental villages and control villages was conducted. But the assessment of the project’s own effects was compromised by the fact that the control villages were ‘contaminated’ by the project interventions through the strategy of ‘organized dissemination’ of knowledge promoted by the CEP. However, the determination of the own effects of the CEP has become a real need of Tostan in that this knowledge will facilitate the determination of the most plausible impacts that the CEP may have in the communities of implementation.
Objective of the Consultancy
With this consultancy, Tostan reiterates the need to document the specific effects of CEP in the implementing communities. Based on the limitations experienced in setting up control villages, the Consultant will explore the own effects of the CEP through another alternative that is multivariate analysis. He/She will use GC3Y project data (baseline study and final evaluation) and/or other data for experimentation of multivariate analysis in the search for the effects of CEP in the communities of implementation.
The MERL department will attach great importance to learning opportunities for its staff (multivariate analysis, statistical significance test, interpretation of results, STATA as an alternative to SPSS) that will be induced by this consultancy.
Scope of work
The Consultant will be expected to carry out and deliver the following responsibilities:
- Work directly with Tostan staff to develop analytic questions.
- Drive the exploration of existing databases and applicate a variety of statistical methods/approaches to develop analytical models.
- Collaborate with different functional teams to validate assumptions and technical approach.
- Simplify and communicate key findings and insights to non-technical audiences.
- Enhance capacity of MERL staff on multivariate analysis.
- Help drive innovative analytical solutions across Tostan.
Duration
We expect this scope of work to be fully completed in 60 working days.
Preferred Qualifications
Education
An advanced post-graduate university degree in data science, statistics, demography or related discipline.
Experience
- Proven experience as Data Analyst in statistical analysis, quantitative analytics, forecasting, predictive analytics, multivariate testing, and/or optimization algorithms.
- Strong background in multivariate statistics and significant experience using SPSS and/or STATA.
- Ability to analyse and interpret results, as well as articulate the implications of findings.
- Experience presenting research findings to diverse audiences.
- Excellent written, verbal, and interpersonal communication skills.
- Detail oriented.
Language
- Able to effectively communicate in French and English, in both written and oral forms.
To apply
Send your CV indicating all past experience from similar work, your daily rate pretention and at least three (3) professional references at: madycisse@tostan.org.
Deadline: July 6, 2018.
[1] Diop Nafissatou, Moreau A, Benga H – Evaluation of the long-term impact of the Tostan programme on the abandonment of FGM/C and early marriage: Results from a qualitative study – January 2008.
UNICEF – Évaluation à long terme du programme de Tostan au Sénégal : régions de Kolda, Thiès et Fatick – Septembre 2008, Document de travail, UNICEF.
CRDH – Rapport de l’évaluation d’impact de l’intervention de Tostan en milieu rural sénégalais : Période 2004-2007, régions de Tambacounda et de Kolda – Mai 2010.
[2] The CEP with an additional Gender module
