Digital methods I

1 Maximizing understanding in qualitative research

Triangulation is the practice of employing multiple techniques to find correspondence and/or complementarity of empirical signals across multiple sources so as to maximize understanding and strengthen the substantiveness of qualitative research findings and interventions being made (Longhurst, 2016; Clifford et al., 2010). Triangulation may include combining qualitative and quantitative data in mixed-methods research – for example, in-depth interviews and census data (Rose, 1993) – but may equally consist of drawing on multiple qualitative materials such as narrative texts, focus groups, and selective surveys.

The imperative for triangulation is perhaps most pronounced in research that engages with human subjects via ethnographic methods (Flick, 2007), which include but are not limited to interviews, participant observation, focus groups, open-ended questionnaires, and participatory research. Here, triangulation enables researchers to reconcile what people say about their behaviours and preferences with how they actually behave and demonstrate those preferences as they go about their daily lives outside of a research context of interaction with researchers. In a focus group or semi-structured interview setting, research participants may adjust their actions and structure their responses to please the researchers, navigate power relationships, maintain or gain social capital, or protect themselves or their social group. This is compounded by problems of subject positionality, reflexivity, and subject awareness of what motivates actions and behaviours.

2 Limits to triangulation in the field

In the context of social practices enacted with and through digital media, stalwart approaches to triangulating reported phenomena with observations of those phenomena ‘in the field’ quickly come up against three key limitations. First, the small size of mobile interfaces, the deeply intimate essence of personal devices, and the highly individualized nature of mobile media practices means that it does not make epistemological sense to rely on the direct observation of these practices in situ (Pink et al., 2016). Second, the highly atomistic nature of individual interactions with personal technologies such as smartphones renders participant observation and participatory research unviable as a method for data collection (a rare exception being social gaming environments, such as multiplayer games). And third, digital platforms such as mobile applications (apps) and the personal devices on which they run constitute relatively closed ecosystems (Light et al., 2016). These limitations render traditional approaches to triangulation epistemologically incommensurate with and insufficient for studying the ways in which society is reconciling with the consequences of the pervasiveness of digital presences in the spaces and practices of everyday life (Taylor et al., 2014; Couldry and Powell, 2014).

These limitations may be illustrated with reference to research that looks to develop an understanding of how individuals contend with the erosion of locational privacy concomitant with the adoption of mobile media such as smartphones. Researchers have used questionnaires to query respondents about how they feel about being locationally tracked (e.g. Leszczynski, 2015; Ricker et al., 2015); large-scale surveys as well as small-scale studies that ask respondents about actions that they have taken to protect their personal locational privacy or to minimize location tracking (e.g. enabling or disabling locational functionalities on smartphones; Madden et al., 2013; Leszczynski, 2015; Zickuhr, 2013); and studies that present participants with hypothetical scenarios to determine how much they value locational privacy (Staiano et al., 2014). The methodological challenge lies in triangulating how individuals report that they feel about location tracking or indicate that they value locational privacy with their actual digital practices. For instance, even where respondents indicate that they have taken actions to militate against location disclosures (e.g. disabling location functionalities on devices or within apps), it is difficult to identify the scope, regularity, and contexts of these practices.

These difficulties of grappling with the unobservable, intimate, and highly individualized nature of digital practices have catalysed methodological refinement and innovation to gain access to the closed ecosystems of mobile digital praxes. Two such techniques are particularly noteworthy. The first of these is the app walkthrough method developed by Light et al. (2016), which presents a structured approach for systematically engaging with a digital app from the moment of initial installation, through exploratory and interactive uses of the app, to leaving a service and/or deleting the app. The technique as proposed by Light et al. (2016) involves walkthroughs conducted by researchers who assume the position of a potential app user, but walkthroughs could also be conducted in tandem with research subjects to elicit their digital practices. Continuing with locational privacy as an illustrative example, researchers could walk through an app with study participants to identify how they grapple with location permissions during the app installation process, grant or deny apps access to location data within the app, and whether or not they adjust privacy settings outside of the app. In a semi-structured interview setting, app walkthroughs could be combined with in-depth discussion to evoke study participants’ feelings about and reflections on their locational privacy-specific interactions with apps to generate rich narrative data for qualitative analysis.

The second technique is that of walking interviews to distil ethnographic ‘moments’ that shed light on the everyday experiences of mobile spatial media unique to individual research subjects. Wilmott (2016) walked with study participants as they used mobile applications and services to navigate the urban environments of Hong Kong and Sydney, asking them questions about their use of location-based services and wayfinding apps in situ. Her walking ethnography captures the ways in which engagements with mobile locative media constitute embodied practices of placing oneself in space, or, conversely, of feeling the acuity of being unable to meaningfully place oneself in space.

App walkthroughs and walking ethnographies provide entry points into the closed ecosystems of mobile devices and the intimate ecologies of quotidian digital practices, but they too come up against the limitations of triangulation. Engaging digitally-mediated social practices through qualitative and mixed-methods research does not negate possibilities for triangulation, but it does necessitate broadening the horizon of what counts, methodologically speaking, as ‘watching what happens [and] listening to what is said’ (Pink et al., 2016: 3).

3 Triangulation beyond ‘the field’

While digital practices may be difficult to study in situ in ways described above, they also constitute new mediums through which to actually conduct research, decoupling the conduct of ethnographic research in particular from requirements that researchers be physically present in ‘the field’. Events can be live-streamed on Facebook; political discourse may be followed in real-time on Twitter; forums and chat rooms afford opportunities for participant observation and participatory research (Pink et al., 2016). Returning to the example of locational privacy, mixed methods approaches extend potentials for ‘small’ ethnographic data to be brought into conversation with big data (Ford, 2014). Returning to the example of locational privacy, Perentis et al. (2017) designed a study in which they gave participants a smartphone with a pre-installed ‘personal data store’ through which subjects controlled disclosures (sharing) of their personal mobile data, including location information, which the researchers monitored for a period of 15 weeks. Information disclosure data captured as transactions through the personal data store encode the contexts and directionalities of mobile data sharing (when participants share data, and with whom) from which broader locational privacy practices may be distilled through data-analytic techniques. The results of this research could be brought into conversation with qualitative data from studies, such as those profiled earlier on, which get at the affective dimensions of location sharing to inform nuanced understandings of how people are collectively grappling with the reconfiguration of norms and expectations around locational privacy.

1 Big social data, spatial big data

In addition to comprising new research mediums, digital media are further unique in that they generate reams of ‘big data’ as by-products of mundane digital praxes enacted with and through engagements with digital platforms. This is particularly characteristic of social media, which involves users in the generation of content (Facebook likes, tweets, Instagram posts) as a functional requisite for meaningful participation on these platforms. Big social data constitute an unprecedentedly extensive, rich trove of data for social sciences research (Felt, 2016; King, 2011; Lupton, 2014; Manovich, 2012), opening up new lines of inquiry and driving the innovation of new data collection and analysis techniques. Where available, APIs ¹ enable researchers to access and collect social media data through-driven techniques such as mining, scraping, and streaming content that may then be analysed in near real-time. However, it is important to note that such content may be programmatically collected for only a few dominant platforms, namely Twitter and Instagram (and at the height of their popularity, Flickr and Foursquare) where users are or have historically been more likely to maintain public accounts and to post to the public domain (Poorthius et al., 2016). Social data are characteristically high in both volume (data production is so substantial that it is measured in petabytes rather than the n of data events) and velocity (its continuous generation means that it may be analysed in near real-time) (Kitchin, 2014), and they are, moreover, extensive in that practices of contributing content to social media platforms are pervasive across a range of mundane practices without being unique to any one activity or set of routines (Pink et al., 2017).

Social media content is of particular relevance for human geography research because much of this data is inherently geographical. In has been estimated that up to 80 per cent of big data are spatial (see Leszczynski and Crampton, 2016): they are from, about, and make reference to places. The built-in locational utilities of digital devices (e.g. smartphone GPS) have extended geotagging ² functionalities to digital applications, platforms and interfaces such that social data productions may be natively associated with geographical referents, ‘placed’ by users via map-based interfaces, or spatial referents may be extracted from the content of data productions themselves (Abernathy, 2017; Crampton et al., 2013; Honan, 2009; Leetaru et al., 2013; Poorthius et al., 2016). Geographers and others have used geocoded social content to study a range of socio-spatial phenomena, including mobilities both global and local (Cranshaw et al., 2012; Hawelka et al., 2014; Shelton et al., 2015), urban income inequality and segregation (Indaco and Manovich, forthcoming; Shelton et al., 2014, 2015), urban/rural divides (Hecht and Stephens, 2014), and even the happiness of city dwellers (Manovich et al., 2014).

Although geographers have proposed approaches that go ‘beyond the geotag’ by, for example, accounting for the relational spatialities of social content production beyond the ability to merely bring data events up as points on a map (Crampton et al., 2013; Shelton, 2016), the use of geocoded social media data for research comes up against epistemological limitations posed by the indeterminacy of the representativeness of these data. Here, representativeness signals both the extent to which data trails may be considered representative of the socio-spatial phenomena that their analyses purport to inform, and the representativeness of the populations and subjectivities that are abstracted into these data flows.

2 Representativeness I

The vast plumes of data generated by individuals as they engage on social media through digital platforms are produced within the contexts of embodied and spatially contingent spatialities, but they are not ‘direct records of human activity in space and time’ (Hawelka et al., 2014: 260). With very rare exceptions such as the social sharing of quantified self-activity logs via platforms such as Strava, social media data do not code for socio-spatial practices, activities, or broader processes. This is a function of the nature of social data production, geotagged or otherwise. These data are generated by networks of largely unaffiliated individuals who participate on social platforms without any intentionality of contributing to a structured or collective data resource, and for whom contributing content via social media platforms is attendant but secondary to an array of mundane practices, processes, and activities. This has significant implications for the use of social content in research and the validity of analytical outcomes for illuminating or informing an understanding of social processes, practices, and inequalities. At best, geotagged social media data may legitimately be enrolled only as a synthetic proxy for socio-spatial processes and practices; at worst, in the pessimistic view of Wilson (2015), they are epiphenomena in and of themselves with little utility value for social sciences research.

The polemics of misappropriating social media data as indicators rather than proxies of human activity plague a not insignificant number of studies that enlist social content within data-driven and programmatic approaches to the analysis of spatialities. An illustrative example is Hawelka at al.’s (2014) extrapolation of global mobility patterns from the analysis of a subset of natively geocoded tweets. The researchers assert sweeping conclusions about mobility trends, claiming that their analysis shows that, for example, ‘increased mobility is…characteristic of West European and other developed countries’, and that ‘Europeans [travel] more within Europe and to Asia that to other continents’ (Hawelka et al., 2014: 269, 266). But because social media data are secondary (although attendant to) social practices, geocoded tweets are not derivatives of mobilities; rather, they code for where individual users identified within this subset of geocoded tweets were usually resident, and from where else they were likely to geocode their tweets within the time horizon of the Twitter activity analysis. The ontological distance between social media data and socio-spatial phenomena renders the legitimacy of identified global mobility patterns claimed by the researchers epistemologically questionable if not spurious.

3 Representativeness II

If representativeness in the first sense asks how well social data may be understood to represent socio-spatial practices, processes and related phenomena, then a second axis of representativeness questions the extent to which content enrolled within data analytics is representative of the broader host of social content productions from which it is derived and of populations that produce those content flows. It has been widely estimated that only 1 per cent of all tweets are natively geocoded (Hawelka et al., 2014; Poorthius et al., 2016; Crampton et al., 2013). Even though this 1 per cent is comprised of billions of tweets (Zook, 2013), geocoded tweets are not statistically representative of Twitter activity; i.e. they do not constitute a representative sample of tweets. Within the remit of studies that are limited to the analysis of geotagged tweets, given the volume of even this miniscule subset of Twitter content, any selective sample of tweets, even where randomized, is unlikely to be statistically non-representative of the dataset of all natively geocoded tweets because of issues of n (number of data events). What percentage of natively geocoded tweets, of which 4 million are contributed in every 24-hour period (Sloan and Morgan, 2015), would be considered representative of geocoded twitter activity? Even 1 per cent of this 1 per cent is equivalent to 40,000 data events.

These polemics of representativeness extend beyond the statistical representativeness of extracted datasets to issues around the representativeness of the statistical populations from which any such subsets of data are drawn. The aggregate of individuals who use social media is distinctive in that it is a self-selecting rather than select (or non-selective) population, comprised of users who have intentionally subscribed to participating on social platforms. While research organizations such as the Pew Center regularly sample the population (in Pew’s case, the American populace) to identify statistically significant demographics of social media participation and use, it is methodologically difficult to ascertain which segments of a population are captured within a dataset scraped from a social media platform (a sample from a self-selecting population), or which social groups are described by the results of any subsequent analyses of these data.

Pew survey data indicate that social media users are on the whole younger, wealthier, better educated, and urban/suburban rather than rural; that women are more active than men; and that Hispanic Americans lead other ethnic and racial groups in social media participation (Greenwood et al., 2016). But these aggregate demographics obscure differences across and within platforms. For instance, only 24 per cent of American adults use Twitter (Greenwood et al., 2016). While young African American adults are proportionately more active on Twitter compared to other demographics (Smith, 2014), a similar study of teenagers and social media use identified that Hispanic American youth are most likely to geotag their social media posts irrespective of platform (Zickuhr, 2013).

The challenge lies in disaggregating these summary population-level demographics to individual data points, of which there may be millions in a sample of geocoded tweets. In the absence of access to the detailed user profile databases held and maintained by digital platform entities, academic researchers cannot validly assume that the demographics of Twitter activity in a sample of geocoded tweets mirror or cleave in tandem with these broader, aggregate-level demographic trends. Yet, because of known cross-platform differences (certain demographics are more likely to participate on particular platforms) and within platform differences (the leading geocoding demographics within a platform ecosystem versus the leading demographics of platform adoption), the results of these analyses cannot be extrapolated to the populace at large. It is not valid to estimate the universal volume of international travel by country of residence on the basis of geocoded tweets (Hawelka et al., 2014). Not only is the aggregate of individuals in a country who geotag their tweets not representative of the population of that country, it has also been shown that users who publish location metadata with their tweets are not statistically representative of the Twitter population at large (Sloan and Morgan, 2015). The issue around representativeness in this second sense is not only that extracts of social content available to researchers are non-representative of the universe of all data contributed through social platforms, but also that their representativeness vis-à-vis the populations involved in the production of these data is indeterminate.