As we at Tostan continue to improve upon our monitoring and evaluation (M&E) system, we have had the opportunity to partner with Dimagi—a software social enterprise that develops technologies to improve service delivery in underserved communities—to create an electronic data collection system. This system comes in the form of a new platform—CommCare—which Tostan will be using to automate data collection from the field, minimize errors that occur when collecting and processing by hand, and expedite evaluation. “[Before,] we were faced with a problem of completeness, consistency and reliability. With the implementation of CommCare, all this is resolved,” explains Bakary Guèye of Tostan Senegal.
The CommCare platform is customizable to the specific M&E needs of the Community Empowerment Program (CEP). For example, the application includes tools for monthly and quarterly program M&E questionnaires.
In addition to the benefits in the field, working with CommCare was significant for staff development. Fatoumata Diallo, a Malian supervisor, is proud of her work with CommCare: “This new technology is very innovative. It has improved the quality of our work and that is very motivating for us. A lot of us were not familiar with tablets before using CommCare; now we can do many things with the tablet.”
Dimagi trained the Monitoring, Evaluation, Research & Learning (MERL) team in the nuances of designing electronic surveys. This gave MERL staff the opportunity to share experiences with and learn from other CommCare users through participation in trainings and online discussion forums. After completing their training with Dimagi, the MERL staff then transferred their newly acquired knowledge to program supervisors in the field.
In January of this year, the MERL team piloted the updated M&E system using CommCare in Senegal. 22 Tostan supervisors used the electronic data collection system during three CEP baseline studies in Senegal: six in the Fouta region, eight near Goudiry, and eight in the regions of Kolda and Sédhiou. These Senegalese supervisors used and will continue to use CommCare for program monitoring.
In February, seven Tostan supervisors in Mali also began using the application to meet their monitoring needs. Korotoumou Diarra, a supervisor in Mali explains, “Before CommCare…sometimes data got lost (if a questionnaire was lost). Now, as soon as you save data on the tablet, we are reassured that it is safe before sending it.” Many users also cited the advantages of being able to erase or modify participant responses electronically, rather than across multiple sheets of paper.
MERL learned many lessons during the two-country pilot phase. Testing with future users in the field was an important step so that changes could be made before large-scale trainings. For example, in the development stage, MERL staff realized the importance of finalizing the questionnaires before building the application, since even small changes to the questionnaire could require large adjustments to the application. Additionally, the CommCare system does not accommodate some non-roman characters, so the programming team needed to adapt the local language translations—Pulaar, Mandinka, Soniké, and Bambara—to respond to this constraint.
Even though there are many challenges when designing a customized M&E tool, the advantages far outweigh any potential roadblocks: the CommCare system dramatically reduces the time devoted to data collection, transfer and M&E report production. The system facilitates data usage and analysis through the automatic transmission of data from the field to the MERL team at Tostan International. Additionally, the application produces an indicator report within two days of receiving data from the field, a task which previously took several weeks. Finally, the application allows MERL to remotely monitor supervisor performance, making it possible for the department to intervene if data collection activities are irregular and provide support when they encounter problems in the field.
In the coming months, MERL will complete this first pilot phase of the new M&E system, then enter into the second. The transition process will include an in-depth analysis of data from the baseline study in Senegal. MERL will update and improve the CommCare application for the second pilot based on feedback from the first. This is critical because it enables the team to create the strongest possible system for data collection and analysis, and ensures that the data collected properly informs decision-making at all levels throughout Tostan.
Bakary summarizes his feelings toward CommCare: “We are on the right track. With Commcare, Tostan has more reliable and complete data. For now, this new system with CommCare works with the CEP [Community Empowerment Program], but I have hope that all other projects to come—like the Youth Project, RPP, Peace & Security—will benefit from CommCare.” Tostan aims to eventually have a completely mobile system in all six countries where we work to meet the needs of each component of our grassroots development program.
Contributions from Seynabou Diouf and Vivien Manel, MERL Officers
