The use of mobile applications for data collection is becoming increasingly common in the social sciences. There are many examples of their use, such as mapping where people feel happy, Mappiness app, or where people feel unsafe, like the Fear of Crime Application and iExperience app. These tools allow social scientists to tap into people’s experiences as and when they take place, and can address many of the issues faced by traditional survey methods.
The most common justification for social scientists incorporating apps into their research is their ability to capture data about social phenomena in situ. Social scientists are interested in people’s “everyday real-world behavior”. Yet, as Shiffman et al argue, behavior is “seldom studied, assessed, or observed as it unfolds in the real world.” Instead, researchers often rely on summary and self-reports of behavior. For example, in the case of fear of crime, many national and local victimization surveys employ retrospective worry measures, such as asking people: “How often have you worried about being attacked in your neighborhood over the last month?” These approaches, inherently limit our understanding of immediate crime concerns that may be encountered during day-to-day movements in space and time.
One research design that addresses the above challenges of collecting information about experiences and perceptions as and when they occur is the approach of Ecological Momentary Assessments (EMAs) and the Experience Sampling Method (ESM). These designs allow participants to report repeatedly on their experiences in real-time, ‘in world’ settings, over time and across contexts. The traditional tool to implement this is to use diaries (paper-pencil, electronic etc.) or interaction diaries. However, another approach is to implement these designs into smartphone assessments or ambulatory physiological monitoring.
Such designs allow new approaches to the study of social phenomena. In the case of fear of crime research, instead of asking people to remember complex sets of emotions, they can provide a tool with which participants self-report their levels of safety, as and when they encounter situations that make them feel worried about potential crime. The collection of spatially and temporally explicit data on people’s experiences in this way enables a much finer grained analysis of social phenomena.
So what are some considerations to keep in mind when designing such studies? Here is a list of 4 things to consider based on our experiences of using apps for social research:
It is important that the concept being studied is conceptualized and operationalized in a meaningful and measurable way. For example, our review of 27 studies using apps to measure fear of crime identified significant variations in the operationalization of “fear”; ranging from choosing between happy and scared emojis, to locating fearful places on a map. Interestingly, there was less of a systematic engagement with past work operationalizing this construct than is usually seen with traditional measurement tools. It is important to keep validity and rigor in mind and not get distracted by the new opportunities offered up by using an app.
Other factors that we describe in our recent webinar hosted by the UK Data Service in relation to study design are considerations around the unit of analysis (ie., individual, place, time); the sample (i.e., compliance and engagement with smartphone EMAs, sample bias), and the duration of the study (i.e., survey triggering schedules).
2.Choosing Your Research Tool: Purpose Built vs. Pre-Existing Apps
There are a number of ways that surveys can be delivered via smartphones. The first, is to build the app from scratch. The benefits of this approach are that it provides the researcher with flexibility and control over features. However, creating a mobile app from scratch requires significant technological expertise and time. The second, is to use a pre-existing commercial platform. There are currently a number of companies that have developed apps for survey data collection (i.e., MetricWire). Because these apps are already built, they require no technological expertise or maintenance by a researcher, and surveys can be easily uploaded to the existing app. However, researchers are restricted in terms of the features and capabilities available on these pre-existing app platforms. We recommend that the decision to use a purpose built or pre-existing app be guided by the research design and capabilities of the researcher (i.e. technological expertise).
Because the data collected using apps contains a number of dependencies (i.e, surveys are completed in different places and times), traditional statistical approaches to analyzing the data collected may not be appropriate. Fortunately, there are alternative techniques available for analyzing quantitative data collected by apps including multi-level modelling, tree modeling, survival analysis, , and other time-series approaches. New techniques for responding to small sample size issues have also recently been proposed (i.e., Bayesian modelling).
Regardless of the type of data analysis used (quantitative or qualitative), we recommend that researchers make sure that information is collected from participants about their device, such as device ID to allow identification of unique responders, and multiple responses by the same person. This assists with analysis and any aggregations that may be undertaken with data that is collected repeatedly from the same respondent.
4. Ethical Considerations
Finally, there are a number of ethical considerations to keep in mind. Ranging from the mere-measurement effect (affecting what is measured by repeatedly asking about it), privacy/personal identification of respondents, and data storage requirements. We mention these at length in our webinar.
In the context of fear of crime research, we also note the recent proliferation of safety and crime reporting apps, and potential ethical issues that may arise from the use of such apps. For example, using safety mapping as a service may stigmatize places and the people living or frequenting those places as dangerous. It is important for researchers to remember that this data is based on individual subjective opinions of their environment and that opinions may vary from person to person, or be impacted by prior experiences (i.e., victimization, personal biases, etc).
Give it a go!
Overall, the use of mobile applications in social science research represents an increasingly popular methodological approach to collecting repeatedly sampled data about people’s situation-specific experiences and perceptions of the environments which they frequent during their daily routine activities. The granular spatial and temporal data generated with such tools, and the ease of reporting for participants can provide unprecedented insight into social phenomena such as happiness, fear of crime, and many more attributes of interest to social science researchers. For anyone wishing to get started, we recommend a read into ESM/EMA methods, a read of this systematic review of app-based studies of fear of crime, and a listen to our webinar.