As schools closed and social distancing measures went into place in March 2020, I realized that my job as a Data Capture and Utilization Officer was about to get challenging. The education programs that I support as a member of the Research and Evaluation team at FHI 360 often integrate monitoring with program implementation and therefore involve in-person visits to our beneficiaries. Uncertain as to when these face-to-face interactions could safely resume, our team quickly realized we would need to adapt. My colleague Yvonne Cao began to formulate a strategy for adapting what information we would collect while I explored new options for how to collect data.

The options

Since I did not want to overwhelm our project staff with a laundry list of data collection products, I first organized the options into three categories and six types:

Decision tree

I then created a decision tree to guide my colleagues in selecting the most appropriate monitoring tool for their context. I hope that this tool can also help you select the right monitoring tool for your COVID-19 context. By answering the following three questions about the audience you are trying to reach, you can determine whether in-person, remote-by-text or remote-by-voice data collection is possible:

1. Can you physically reach your target audience?
2. Does your target audience have access to a phone or computer?
3. Is your audience literate?

Click to expand full decision tree; Source: Brian Dooley, 2020

In-person data collection

• Can you physically reach your target audience? Yes

If you are working in a context where social-distancing measures are limited or lifted, then traditional in-person data collection may still be an option. In-person data collection is my preferred approach in low-resource contexts because I do not presume everyone has network access, or even a phone or computer. To avoid missing out on hearing from key voices and biasing my sample, I first consider interviews with offline mobile forms, or self-reporting through paper data collection.

1. Offline mobile forms: Offline mobile forms are a type of survey software that you can install on any smartphone or tablet, with options for both Android and iOS. Data can be entered anywhere offline, stored on the device and uploaded to a central server once your interviewer has network access.

   Technology
   If your organization can set up and deploy open-source software, you can use the standard free software called Open Data Kit (ODK). I have been using ODK since 2012 on most of my projects and have helped a few governments set up servers for ongoing data collection. But, if you are new to mobile data collection, I would recommend using KoboToolbox. By signing up for a free account, the Harvard Humanitarian Initiative will set up space on their servers for you and provide unlimited storage for any humanitarian or COVID-19 response project. If your data collection activity is not humanitarian, you can still upload up to 10,000 records per 30-day period for free.

   Data collection
   If in-person meetings are permitted, then tablet or smartphone assisted interviews can be conducted by project staff, with the proper precautions. I recommend that your interviewers meet with respondents one at a time in a well-ventilated space, remain six feet apart, wear a face covering and use hand sanitizer.

2. Paper data collection: While paper data collection may sound basic, going back to this reliable method can work under social distancing guidelines, provided you take the proper precautions.

   Technology
   I recommend that you prepare your paper survey tools using an optical mark recognition (OMR) software. For those of you not familiar with the term OMR, you are probably familiar with the actual product, as this is just the technical name for those machine-readable, fill-in-the-bubble surveys. I have used one of the more popular products called Remark Office OMR, but open source options exist for those of you with an inclination to avoid licensing fees. If you already know the names of the individuals you would like to reach, then you can use the mail merge function to print your surveys in bulk, each already labeled with a name and unique ID number or scannable bar code.

   Data collection
   Once you print the surveys, drop them off at an accessible distribution site in the community, such as an empty school classroom or community center. Spread out the surveys across a table or the floor and organize them into piles by the name of the participant or the scheduled pick-up time. Create a schedule to stagger the pick-up and drop-off of the surveys to avoid overcrowding at the site. Someone from the project should supervise the pick-up and drop-off site and provide hand sanitizer to anyone who enters the site. If you use OMR, data entry is as simple as scanning each paper survey and then reading the results with that software. If not, your staff will need to manually enter the results into a computer.

Text-based remote monitoring

• Can you physically reach your target audience? No
• Does your target audience have access to a phone or computer? Yes
• Is your audience literate? Yes

Remote monitoring refers to data collection that you can conduct without physically reaching your audience. If your audience can read and has access to a phone or computer, then you can proceed with either an online or SMS survey.

3. Online surveys: Online surveys are increasingly common and with good reason. Unlike offline mobile forms, online surveys do not require you to download new software to your device just to answer some questions. They can also be a lot cheaper than paper because you do not need to budget for printing or data entry.

   Technology
   Popular commercial products like SurveyMonkey and Qualtrics offer a lot of great features, but these come with a cost. Google Forms and Microsoft Forms offer more basic functions, but they can support most surveys for free. If you already have an offline mobile form, and you are worried about the time that it will take to convert it to an online form, you can use a product called Enketo. Enketo can take your existing ODK or KoboToolbox mobile form and turn it into a webform. You can purchase Enketo as a standalone product, install the open-source software yourself or use it for free through KoboToolbox.

   Security
   If your primary concern is not the cost, but data security, then you will be happy to know that each of these online survey options can be secure. Data security is less about picking the right product and more about how you manage the data. The GDPR, which stands for General Data Protection Regulation, is an EU data protection law that went into effect in May 2018. To ensure that your monitoring complies, use this helpful checklist.

   Data collection
   If you already have contact information for your desired respondents, then you can share a link to your survey through existing communication channels, such as WhatsApp, email or SMS. If not, you can post a link on a project or community website.

4. SMS surveys: SMS data collection is an option if the person you are trying to reach has a phone and is literate but cannot access the internet. Instead of sharing an entire survey at once, SMS surveys send respondents one question at a time in the form of a text message.

   Data collection
   If you already have the phone numbers of the people you would like to reach, you can load their contact information to your platform. Once loaded, you can send the same survey to everyone at once or segment your audience and send different SMS questions to each group. However, if you do not have a list of contacts, you can use existing communication channels, such as radio or community leaders, to encourage people to text a keyword to your dedicated survey number to initiate the survey themselves. This method ensures that all participants consent and allows you to compile phone numbers for follow up questions.

   

   Click to expand an example of a user-initiated interactive SMS survey, created using Twilio; Source: Brian Dooley, 2020

   Technology
   While interactive SMS is intuitive for most end-users, it can be expensive. First, you will need to set up an SMS platform. Just like with the previous technology approaches, there are open source options for SMS surveys, most notably RapidPro. However, the technology required to run an SMS survey is much more complicated, and hosting a solution yourself is not even recommended by RapidPro. Instead, I would recommend signing up for an SMS survey provider. The Listening Post Collective created a useful review of SMS survey products, and CartONG, with assistance from UNHCR, developed a similarly helpful SMS survey primer in 2015. When evaluating SMS products, first ensure the company has a gateway in the country where you work. This gateway allows the SMS survey system to connect with the phones on the local networks and can take weeks to set up if not already in place.

   Costs
   Each product has a slightly different billing method, which makes comparing the total cost to run your survey difficult. You can expect to see varying combinations of charges for: users reached, questions sent, surveys completed and a monthly platform fee.

   

   These costs to the survey provider do not include the cost paid to the mobile network provider for each SMS sent. If you are not sure what the price per SMS in your context is, this pricing tool from Twilio can give you an idea for each network. Since your response rates will likely suffer if participants need to pay for each reply message, you should also budget for either reverse billing or reimbursing your participants with mobile money. This cost per SMS sent and received is likely to exceed the price paid to the SMS survey provider. Therefore, you will need to strategically reduce a combination of your sample size and the number of questions in your survey to fit within your budget.

Voice-based remote monitoring

• Can you physically reach your target audience? No
• Does your target audience have access to a phone or computer? Yes
• Is your audience literate? No

If your population has low literacy rates, you should consider telephone interviews. Like SMS surveys, the primary cost will be the per-minute cost of the phone calls placed to each respondent. To plan for these costs, you should first determine the number of people that you need to reach with your phone calls. Then, based on your available budget, you should decide how many minutes of airtime you can afford per survey. With those parameters, try role-playing your phone interview, and record the time required to complete each question. Using this time per question information, you can focus your survey on the essential questions that fit within your time limit. Once your survey is ready, you can either hire enumerators to place the phone calls or use an automated voice survey tool called interactive voice response (IVR).

5. Phone surveys: The easiest way to transition from in-person to remote monitoring is to ask your existing enumerators to conduct phone surveys. Provide each enumerator with a list of names and phone numbers to call, a script to follow, a form to record responses and mobile credits.

   Technology
   If enumerators have personal smartphones or tablet devices provided for previous in-person data collection, then you can create an offline mobile form that includes both the phone script and fields for data entry. If enumerators have a computer or phone with internet access, then you can create an online webform to serve the same purpose. KoboToolbox works well if you have a mix of enumerators with phones and computers since you can create one form and then deploy it either as an offline mobile form with KoboCollect or as a webform using Enketo. The data from both methods of entry will still aggregate in one central place in your Kobo account.

6. Interactive voice response (IVR): You should consider interactive voice response software if you do not have enough people to place calls manually. Like SMS, IVR shares survey questions one at a time, but as voice recordings instead of text messages. You should first ask for the preferred language of instruction then consent to proceed. Some products allow you to record the audio files in any language that you need, while other platforms only offer a limited number of languages

   Data collection
   If you have the phone numbers of your target audience, you can load a contact list to the platform and schedule calls for a time that you expect people to answer the phone. For those that do not respond, the system can call at different times until you find an optimal time for each person. If you do not have contact numbers, you can attempt to find your target audience through random-digit dialing and screening questions.

   Technology
   Many of the same companies that offer SMS survey platforms also offer IVR. One of those companies, Magpi, provides a helpful overview of SMS and IVR data collection. I am currently working with Viamo (formerly VOTO Mobile), to use their IVR service to gather biweekly updates on the security and COVID-19 situation in Northeast Nigeria, and to share COVID-19 information through a toll-free call-in line.

COVID-19 has created diverse and pressing needs that demand we proactively pivot to stay informed. There are many monitoring tools at your disposal to respond to your unique COVID-19 context, and hopefully this decision tree will help you find your way around the toolbox. We’d love to hear your experience on picking the right tools for your programs. Join our Implementing MEL in COVID-19 group to join the conversation!


A version of this post originally appeared on Brian Dooley’s LinkedIn profile.

Photo caption: School Support Officer conducting a pupil assessment in an empty classroom in Northern Nigeria
Photo credt: Brian Dooley/FHI 360

Sharing is caring!