15 innovations in data collection methods: Broadening the financial inclusion survey toolkit

Posted on 15th Jun 2020

Written by Grant Robertson

Advances in technology have fundamentally changed the way in which survey and behavioural data is collected. There has been a steady decline in face-to-face and telephonic data collection over the past 10 years, with a corresponding and rapid increase in digital data collection. The widespread use of the mobile phone and other smart devices (watches, Fitbits, Scales, TVs, etc.) has presented researchers with a golden platform to collect data in more innovative and creative ways.

Broadly speaking, there are two categories of data collection: active data collection (where one needs to respond to some stimulus, like a questionnaire) and passive data collection (in which the data is collected without any action required from that person). These methods should not be treated as substitutes but rather as complements to collecting rich data regarding people’s financial lives.

Although Computer-Aided Personal Interviewing (CAPI) has been around for decades and its benefits well documented, it is still not a mainstream data collection technology among financial inclusion researchers. The main advantage of CAPI is that it allows the interviewer to focus on the interview and leave the sophisticated routing and filtering to the laptop, tablet or mobile phone. It also allows for the implementation of more sophisticated research methods, such as Brand Price Trade-off (BPTO) surveys.

• Advantages: Multi-media elements and gamification can be incorporated in the survey.
• Disadvantages: The initial investment can be high.

Online/web surveys are perhaps the data collection method that has developed most rapidly over the past 10 years; and in many developed countries, online surveys are the primary tool for data collection. They lend themselves to all forms of gamification, which make them more engaging than other survey methods. However, sampling is a critical issue, as self-selection is always a concern. Sampling providers usually supply samples from access panels, through bespoke recruitment or techniques like river sampling, which randomly selects people in real time as they browse the web. Online surveys can be used on a variety of devices, including PCs, tablets and mobile phones.

• Advantages: These include the low costs of, automation of and real-time access to data, the fact that they take less time to execute, are convenient for respondents and offer design flexibility and creativity, and that interviewer bias is eliminated.
• Disadvantages: Online surveys may result in limited sampling and respondent availability, possible low response rates, a high number of drop-outs and the danger of professional respondents – those who regularly take part in surveys to earn money. It also requires that respondents have access to a smartphone, tablet or PC with an internet connection.

Mobile phone surveys are becoming increasingly popular, despite concerns around self-selection bias, because they are cost-effective, quick to deploy and generally provide “good enough” reading for decision-making. Mobile surveys can be deployed by using either USSD and SMS technology for markets in which feature phones predominate, or web questionnaires where smartphones are prevalent. In some markets, mobile surveys could be a viable method to collect high-level financial inclusion metrics during years in which the main demand-side survey is not run.

• Advantages of SMS/USSD surveys: These surveys are cost-effective, work on all phones, are considered an unobtrusive contact method, deploy well-used and well-understood technology, have quick turnaround times and can have large samples.
• Disadvantages of SMS and USSD surveys: They support 160 characters only and one to five questions at most, as there are high drop-off rates when more questions are included. They are vulnerable to a time-out of the service, verbal and technological literacy is assumed, and there is a cost to the respondent in the case of SMS surveys as well as a cost for coding open-ended responses.

Even though Computer-Aided Telephonic Interviewing (CATI) is regarded as an old-school technique and many research call centres have closed their doors, it is still a viable option in markets where mobile phone penetration is high and where there are no major cultural reasons for not sharing information telephonically. It is important that the interview be kept as short as possible – preferably under 15 minutes. CATI can be useful for tracking or dipstick data collection, as it is quick and cost-effective and can result in representative samples. Because demand-side surveys are not generally conducted annually, due to the high costs involved, CATI is a viable method to get a pulse reading on the important high-level financial inclusion metrics.

• Advantages: Low costs, faster, less interviewer bias and better for sensitive questions than face-to-face surveys
• Disadvantages: Universe limited to those that have phones

The use of Interactive Voice Recording (IVR) surveys is increasing. They can be quite impersonal and need to be short to be effective. Self-selection is usually a major concern, and the technique requires that the respondent uses the keypad to respond to questions asked by the IVR system.

GPS tracking. Tracking the movement patterns of consumers is becoming increasingly important as researchers realise that location and in-context dynamics are key drivers of consumer decision-making.

Web tracking technologies (like cookies or meters) allow researchers to monitor websites in terms of how long users spend on them, when they visit, which links are clicked through and so forth. Through this, researchers can gain a deep understanding of how consumers behave online, which content interests them and which products and services they research and purchase. This type of data is also used to improve advertising effectiveness.

A chatbot is an artificially intelligent software program that uses natural language processing to hold a conversation with its users. They are programmed to interpret and understand natural language to complete common tasks but were originally developed with the intent to use them in surveys to ask and direct open-ended questions. Some more sophisticated bots can assess sentiment and direct follow-up questions based on the direction of the sentiment. Survey bots make for a more conversational style, which should lead to better-quality responses and can be deployed in online surveys or through messaging apps. Technically, this technology could also be deployed using CAPI to standardise the way in which open-ended questions are asked or probed. Click here for examples.

Functional Magnetic Resonance Imaging (fMRI) is a technique used by research to monitor in which regions and to what extent the brain responds to various stimuli. It does this by monitoring blood flow in the brain. Marketers have been using fMRI extensively to see how consumers respond to advertising and which product mix factors influence decision-making.

Electroencephalography (EEG) is a new technique used by researchers to map brain electrical activity and emotional response to marketing material. Both fMRI and EEG methods have been used to understand financial investor behaviour and financial decision-making.

Galvanic skin response. Galvanic skin measurement is done to detect a change in the electrical resistance of the skin caused by emotional stress, and this is measured with a sensitive galvanometer. For example, we can measure the reaction of a consumer to a brand, advertisement, web page, in-store experience, fragrance and so forth. The advantage of this method is that it strips out the perception bias and measures the actual emotional response to a stimulus.

Eye-tracking technology allows researchers to track eye movements in real time. Knowing what people are seeing and what they pay attention to assists with advertising, web and product design. This data is quantified and an aggregated visual analysis is presented depicted the pattern and strength of gazes.

Social media monitoring/listening. There is a plethora of tools that allow researchers to extract and analyse social media data (Twitter, Facebook, etc.). These kinds of techniques can be useful to gauge sentiment towards a company, brand, product or political issue.

Other passive measurements. Almost any modern electronic device (such as a fridge, TV, Fitbit, scale or watch) has the ability to capture data from the user. These can be useful for studying specific research questions or for nudging consumer behaviour in a particular way.

Transactional data. Financial service providers (FSPs), mobile network operators (MNOs), retailers, etc. are collecting many types of transactional data. This data is extremely important, as it provides real behaviour data as opposed to the self-reported data collected through surveys. For example, it can state which financial products are being used and how consumers use them.

Conclusion
Presently, there are numerous ways to collect data. It is important to assess the relevance of the data to the problem at hand, as well as its quality. Quality can be considered a function of various dimensions such as the age, size or completeness of the data and coverage of the population of interest.Broadly speaking, there are two categories of data collection: active data collection (where one needs to respond to some stimulus, like a questionnaire) and passive data collection (in which the data is collected without any action required from that person). These methods should not be treated as substitutes but rather as complements to collecting rich data regarding people’s financial lives.