Crowdwork and the mobile underclass: Barriers to participation in India and the United States

Abstract

Online crowdwork platforms have been praised as powerful vehicles for economic development, particularly for workers traditionally excluded from the labor market. However, there has been insufficient scrutiny as to the feasibility of crowdwork as an income-source among socioeconomically deprived populations. This article examines device requirements and differential access to digital infrastructure, both of which act as potential barriers to not only basic participation but also to economic success online. Given the increasing prevalence of mobile-first and mobile-only populations, research on this topic aids in understanding the crowdwork ecosystem among differing socioeconomic sectors. Based on a survey of 606 crowd workers across the United States and India, this article uses both quantitative and qualitative data to explore whether reliance on mobile devices is detrimental for economic outcomes of crowdwork. The results point to substantial inequalities in device use and received benefits from crowdwork, within each country and between the two contexts.

Keywords

Amazon Mechanical Turk crowdwork digital inequalities mobile mobile underclass

Introduction

A recent typology by Howcroft and Bergvall-Kåreborn (2018) distinguishes four types of crowdwork. One such type, namely online task crowdwork or “microwork,” has become the focus of increasing academic attention in recent years, both as a site for cross-disciplinary data collection and as a research context in its own right (Irani, 2015; Kittur et al., 2013; Martin et al., 2014). Crowdwork platforms, which offer globally distributed workers the opportunity to earn additional income through completing small tasks online, have attracted interest as potential providers of economic development opportunities among traditionally excluded populations (Alkhatib et al., 2017; Bucher and Fieseler, 2017; Kittur et al., 2013; Paolacci et al., 2010). Participation on the leading crowdwork platform Amazon Mechanical Turk (AMT), for instance, has been presented as an option for mass job creation and income generation in the Palestinian territories (Kuek et al., 2013), as well as and among female Syrian refugees in Jordan (Hunt et al., 2017). However, scholars have also begun to critique crowdwork from different angles, pointing to power asymmetries, exploitation (Bergvall-Kåreborn and Howcroft, 2014), and other access barriers, such as disability and age (Brewer et al., 2016; Zyskowski et al., 2015).

Although crowdwork is frequently depicted as digital and remote, framed as a form of disembodied artificial intelligence or “ghost work” (Gray and Suri, 2019; Irani, 2015), performing crowdwork still necessitates local physical infrastructure, such as a laptop, PC, tablet, or smartphone. This is in addition to requiring access to a stable Internet connection, electricity, and a place to work. Such prerequisites, however, currently exclude half of the global population who lack access to the Internet through any device (GSMA, 2018; ITU, 2017). Focusing on the every-day materialities of crowdwork, this article therefore examines physical barriers to crowdwork participation, focusing specifically on the device/s used by workers to access, find, and complete crowdwork tasks. More specifically, while adopting a digital inequalities lens, we explore whether crowdwork can be an effective economic opportunity for mobile-only or mobile-first users, since membership in this “mobile underclass” closely corresponds with membership in traditionally excluded populations (Castells et al., 2007; Hjorth et al., 2012; Napoli and Obar, 2014). The three central research questions of the article are therefore: (1) How mobile friendly is crowdwork? (2) What are the barriers to mobile crowdwork participation? and (3) Does mobile-first crowdwork participation result in tangible advantages or disadvantages for crowdworkers?

We conducted this initial exploration by focusing on one platform, AMT, but with an internationally comparative approach. The two countries selected for analysis were India and the United States, representing, respectively, a high-income and a lower-middle-income country, while also constituting the two most significant markets of AMT crowdworkers globally (Gray and Suri, 2019; Hara et al., 2019; Ross et al., 2010). The bi-national comparison provides a valuable contrast given their country differences in smartphone and Internet adoption. The United States, for instance, hosts more than 300 million unique mobile subscribers and, according to a 2018 Pew Report, 95% of Americans own a mobile phone of some kind, with 77% owning a smartphone (Poushter et al., 2018). India, by contrast, is the second-largest mobile market and the third-largest smartphone market in the world, with 26% of all mobile users having a smartphone in 2018 (eMarketer, 2018; GSMA, 2015). By examining two countries, we can therefore better identify how device usage shapes the experience of crowdwork.

In this article, we surveyed 606 crowdworkers on AMT across India and the United States, generating both quantitative and qualitative input around the experiences, materialities, and economic outcomes of crowdwork from a device perspective. Our findings support the mobile underclass argument, whereby mobile-first and mobile-only workers are practically excluded from this type of crowdwork due to functional and requester-related barriers. Thus, we contribute to digital inequalities research from a theoretically founded perspective, adding novel insights to research on Internet access, uses, and benefits in the work context (Lutz, 2019).

Theoretical background: the mobile underclass

As a consequence of rapidly expanding smartphone adoption worldwide, a growing proportion of the online population is “mobile-first” or “mobile-only,” with a growing population of “mobile natives” leapfrogging traditional forms of Internet access (Chircu and Mahajan, 2009; James, 2009). Such users can be mobile-first or mobile-only due to affordability constraints as well as for reasons of autonomy and mobility, particularly among those with unstable housing (Cotten et al., 2009; Gonzales, 2014). Tsetsi and Rains (2017) look at the sociodemographics of mobile-only Internet users, finding that individuals marginalized by race, income, and education are more likely to rely on smartphones for Internet access. Importantly, since socioeconomic limitations are pan-global and are not restricted by international borders, mobile-first or mobile-only users can be found in both the developing and developed world (Mossberger et al., 2012; Smith, 2015; Tsetsi and Rains, 2017).

The potential for mobile technologies to reduce socioeconomic inequalities remains an open research question (Castells et al., 2007; Marler, 2018; Pearce and Rice, 2013; Ureta, 2008). Some scholars, for instance, have explored whether mobile Internet access can alleviate social exclusion in the developing world (Chigona et al., 2009). However, other researchers have questioned the emancipatory potential of mobile devices. In ethnographic research on mobile phone use in poor neighborhoods of Santiago, Chile, Ureta (2008) demonstrates that the alleged mobility of mobile devices has to be weighed against constant exclusionary factors. As an emerging stream of discussion, critical accounts to the notion of mobile emancipation have been summarized within the concept of the mobile Internet underclass, as presented by Napoli and Obar (2014). They claim that mobile Internet access is a form of second-class Internet access due to limitations in functionality, restricted usability, and less open protocols (Napoli and Obar, 2014).

One key difference is in functionality, where complex tasks are more difficult to accomplish on mobile devices (Donner et al., 2011; Tsetsi and Rains, 2017; Wang and Liu, 2017; Wyche et al., 2018). Usability studies conducted in the developing world have shown that mobile phones are functionally difficult to use (Gitau et al., 2010; Medhi et al., 2011). Entering information, for instance, is easier to accomplish on a keyboard than on a mobile keypad (Yesilada et al., 2010) and the creation of large and complex documents is a highly uncommon activity on mobile phones (Yesilada et al., 2010). In research undertaken by Donner et al. (2011) among a women’s cooperative in South Africa, the limited functionality of mobile devices prevented users from uploading resumes or job applications, even though mobile email had permitted initial contact with potential employers.

Yet, mobile devices do not exist in a vacuum. Rather, they operate within an array of infrastructure, such as data plans, server farms, battery power, the regulatory environment, and signal availability—all factors which can help or hinder mobile Internet access (Donner, 2008; Thompson, 2018). The instability of mobile phone access, for instance, can be a legitimate constraining factor. Gonzales (2014), for example, finds that mobile access among low-income residents of New York City is dependably unstable and users expect periodic disconnection from their devices. Indeed, for low-income families in the United States, access to the Internet is constrained by slow or interrupted service, outdated devices, or having to share devices.

As a result of these limitations, mobile Internet use has been characterized as primarily “extractive” or as a form of “skimming” (Humphreys et al., 2013; Isomursu et al., 2007), in contrast to the more “immersive” PC experience. Thus, mobile-only or mobile-first users do not fully exploit the socioeconomic benefits of mobile Internet access, due to a tendency toward social rather than instrumental and capital-enhancing activities (Marler, 2018; Pearce and Rice, 2013). These arguments connect to the literature on the second- and third-level digital divide (Hargittai, 2002; Zillien and Hargittai, 2009). However, it is nevertheless important to be aware of the situation of mobile devices within the broader technological ecosystem of working, since workers may use multiple devices simultaneously and thus overcome the restrictions of mobile devices (Ruppert et al., 2013; Thompson, 2018; Tungare and Pérez-Quiñones, 2009).

Mobile affordances, such as portability and locatability, have enabled the rise of on-demand “gig work” platforms such as for instant-delivery or ride-hailing (Griffiths, 2007; Schrock, 2015; Thompson, 2018). In these cases, a worker’s mobile device operates simultaneously as the tracker, communication portal, and central conduit of platform management (Shapiro, 2018; Veen et al., 2020). As a result of a strict labor process, these on-demand “gig workers” must use pre-specified models of smartphone, thus negating potentially hierarchical inter-worker differences based on device choice (Gandini, 2019; Shapiro, 2018).

Crowdworkers, however, have more freedom and can select which device or devices they use to complete their work. In a comprehensive study of crowdwork across five major platforms (AMT, CrowdFlower, Clickworker, Microworkers, and Prolific), only Clickworker offers a mobile app (Berg et al., 2018). However, AMT proclaims itself to be “mobile friendly,” which means that workers can “search, browse, preview, accept, and submit human intelligence tasks (HITs; Wikipedia, 2019) using mobile devices as well as their desktop personal computers” (AMT, 2017). This element of choice thus introduces internal variability within the crowdwork labor process and consequently raises the question of whether crowdworking through mobile devices, either through preference or though necessity, can result in tangible advantages or disadvantages for crowdworkers.

Methods

Data collection

To provide an initial exploratory assessment of device usage among AMT crowdworkers, we conducted an online survey among two population groups: US-based AMT crowdworkers and India-based AMT crowdworkers.

The surveys were conducted in late September and early October 2018 through AMT, receiving a total of 293 completed responses in the United States and 313 in India. For the US-based sample, the survey took 6 minutes to complete on average, whereas the Indian-based sample took 15 minutes. US-based workers were compensated with US$1.5, while the Indian-based sample were compensated with US$1. The TurkPrime interface was used to launch the survey and administer payments. Except for location, no additional screening functions were applied to ensure maximum inclusivity.

The questionnaires included both open and closed questions. The open questions invited respondents to answer in an open text box on topics related to mobile device usage, their experience of working on AMT from a device perspective, and their device-investment attitudes. The closed questions related to demographic characteristics, work-related aspects, the use of mobile devices for the human intelligence tasks (HITs), and finally the use of mobile devices for task-related activities such as browsing HITs and communicating with requesters. Respondents were first asked demographic questions and questions related to their AMT work. After this, respondents were grouped into one of three response streams, depending on their answer to a filter question about their use of mobile devices (smartphones and/or tablets) for carrying out HITs: (1) laptop or PC only; (2) mobile devices only; and (3) mixed.

The average age in our US sample is 35 years (median = 33; standard deviation [SD] = 10) and the average age in our Indian sample is 29 years (median = 28; SD = 6). Fifty-five percent of the US sample identified as male, 44.5% as female, and 0.5% as other; 69.5% of the Indian respondents identified as male and 30.5% as female. The reported average annual household income (before taxes) in the United States was in the category US$50,000–59,999 (median income = 40,000–49,999). In India, the reported average annual household income (before taxes) was in the category INR 140,001–160,000 (median income = 80,001–100,000). Forty-one percent of the US sample report having a Bachelor degree as their highest degree, 8% a Master, and 2% a Doctorate. Thirty-four percent have a higher secondary education, 10% a lower secondary education, and 5% finished school after primary school. In India, the education level is higher, with 73.5% reporting a Bachelor degree and 23.3% a Master. Only around 3% report no Bachelor or Master degree. Overall, our sample seems to consist of a broad cross-section of AMT workers in the United States and India (though with more heterogeneity in the United States).

Measures

Due to a lack of established measures for all device-related aspects, we had to rely on newly developed questionnaire measures for most part. First, we queried the following AMT-related aspects: Experience (“How long have you been using Amazon Mechanical Turk?” with 8 response categories ranging from less than half a year to 8 years or more); Masters qualification (Binary: Yes or No); Estimated average wage per hour (11 response categories in the United States, 18 in India); Nature of the income from working on AMT (3 response categories: main source of income; supplementary income; non-needed side-earning); AMT working time in hours per week (10 response categories from 0 to 1 hour; to 10–61 hours or more); and AMT investment amounts (9 response categories in the United States and 12 in India). We included an open text box with the following prompt: “Please discuss in the text box below how and why you have invested (or not invested) in upgrading or purchasing equipment for the purpose of Turking.” Respondents were asked to what percentage they perform seven different types of HITs (academic survey; categorization; business feedback; sentiment; content; data processing; tagging), with individual slider questions.

Regarding device usage for HITs, we asked for the use frequency of four devices: PC, laptop, tablet, and smartphone. The question prompt was, “For each device, please indicate how often you use it to perform HITs on Amazon Mechanical Turk?” Use frequency was assessed on a 5-point scale, including 1 = never, 2 = rarely, 3 = sometimes, 4 = frequently, and 5 = very frequently. This question served as our filter question to stream the respondents into the (1) laptop or PC only; (2) mobile devices only; or (3) mixed group. Qualtrics’ survey flow and branching options were used to stream the respondents into the respective group based on their responses to the filter question. We decided, for the purpose of survey filtering, to group tablet and smartphone users together. While there are significant differences in functionality between smartphones and tablets, their commonalities in terms of touch-screen predominance and mobile operating system justified their combination here. Moreover, AMT provides requesters with templates targeted at “mobile devices including tablets and smartphones,” arguing that they cluster them as such internally (AMT, 2016).

Within the PC and/or laptop-only group, we queried respondents on their reasons for not using a mobile device for performing HITs, based on 11 closed 5-point Likert-type questions and 1 open question. The closed questions included task-specific barriers (e.g. “The HITs I usually perform cannot be done on a tablet or smartphone”; “The quality of the HITs I usually perform would be lower”), connectivity and affordability barriers (e.g. “I lack a reliable Internet connection on my tablet or smartphone”; “Using mobile data to perform HITs would be too expensive”; “I share my mobile devices with other people”), platform barriers (e.g. “MTurk is not mobile friendly”), and device barriers (e.g. “The keypad on mobile devices is impractical for performing HITs”). The open text box prompt was, “Please use the following text box to explain why you use a laptop/PC instead of mobile devices to perform HITs”.

Within the Mobile devices-only group, we queried respondents about the barriers to using a PC or laptop. The open text box prompt was, “Please use the following text box to explain why you use a mobile device rather than a desktop PC or laptop to perform HITs”.

Within the Mixed group, we included a slider question to determine the ratio of HIT performance between mobile devices and PC/Laptop (Please use the slider to indicate the rough ratio of your HIT performance between mobile devices and desktop PC/laptop: 1—“almost exclusively mobile devices” to 99—“almost exclusively desktop PC/laptop”). Respondents who scored 51 or higher were asked the same barrier questions as the PC and laptop-only group but with a slightly different prompt (“I use my desktop PC or laptop to perform HITs more often than my mobile device because . . .”). Respondents who scored 49 or lower were queried about their rationale for using a mobile device more often than a PC or laptop. Respondents scoring 50 exactly (zero respondents in the United States and two respondents in India) were asked whether they prefer to perform HITs on a mobile device or PC and/or laptop. Those who preferred performing HITs on a mobile device were queried about their reasons for their preference (“I prefer to use my mobile device to perform HITs because . . .”) and the same for those who preferred performing HITs on a PC or laptop, but conversely (“I prefer to use my desktop PC/laptop to perform HITs because . . .”).

We assessed second-screening with one item: “How often do you use your mobile device simultaneously with your desktop PC or laptop to perform HITs?” The five response options were as follows: 1 = never, 2 = sometimes, 3 = about half the time, 4 = most of the time, and 5 = always.

Finally, respondents were queried about the frequency of device use for non-HIT activities: browsing HITs, communicating with requesters, communicating with AMT, performing profile administration (e.g. payment information), and communicating with other Turkers (e.g. through online forums). In line with device use for the actual performance of HITs, we included four devices (desktop PC, laptop, tablet, smartphone) and five response options (1 = never, 2 = rarely, 3 = sometimes, 4 = frequently, and 5 = very frequently).

Method

We used a combination of quantitative data analysis, including principal component analysis (PCA) and linear regression analysis, and qualitative content analysis of the open text boxes to analyze the data. The quantitative data analysis was mostly conducted with IBM SPSS Statistics (v.25), except for the regression, where we used Stata (v.15). The qualitative content analysis was conducted with NVivo (v.12) by a trained coder, adopting two rounds of coding.

Results

Device use distribution and second screening

Within the US sample, 44 respondents (15.0%) reported using only their laptop or PC to perform HITs, matched by 37 respondents in India (11.8%). Only one respondent in the United States (0.5%) and one respondent in India (0.3%) reported using only their smartphone or tablet to perform HITs. Accordingly, the vast majority of respondents in both the United States (248–84.5%) and India (275–87.9%) reported that they used both a PC or Laptop as well as a mobile device to perform HITs. However, among this mixed user group, only 24 respondents in the United States (8.2%) and 18 respondents in India (5.8%) reported using their mobile device more frequently than a laptop or PC to carry out HITs. These initial descriptive results indicate a cross-country norm of performing a majority of HITs through a PC or laptop, with mobile devices being a supplementary option.

This aspect was expanded upon in the open text-fields, where respondents noted that the portability affordances of mobile devices made them a useful supplementary device in instances when a PC or laptop was not available. However, the mobility of mobile devices, while providing some workers with additional opportunities when “on-the-go,” often acted more as a notification device and as a tether to the stationary workspace:

A lot of Turking is spent waiting for HITs to drop. You can’t just sit down and work nonstop for X hours, because the work isn’t available. You have to set up scripts to notify when a good HIT is available, and you need to grab it fast. So, if I am not near my laptop when a good HIT drops, I will have to do it [the HIT] on my Smartphone. If I am near my Laptop and have internet, I will Turk on there because the screen size is larger than the phone. (US respondent, 29, male, mixed category)

In terms of mobile devices being supplementary, the data further support this with regard to the practice of “second screening,” whereby users are not only performing HITs on different devices interchangeably, but performing HITs on multiple devices simultaneously. Although second screening was uncommon among the US mixed user group (arithmetic mean of 1.49, SD = 0.64), it was relatively common among the Indian mixed user group (arithmetic mean of 2.43, SD = 1.11), with 35% of Indian mixed users indicating that they were second-screening about half the time or more.

The value of combining devices to enhance workflow capabilities when performing crowdwork was discussed among respondents in the open text-fields, with one US respondent commenting,

Sometimes it’s just easier to use mobile devices in combination with a desktop to maintain workflow. For instance, it might be troublesome to keep up multiple tabs on a desktop (if you don’t utilize multiple monitors), so having some things on a mobile device makes keeping track of work or messages easier. (US respondent, 32, male, mixed category)

Barriers to mobile crowdwork

Looking more specifically at the key barriers to using mobile devices for performing HITs, the PC or laptop-only group in the United States (N = 44) agreed most strongly that it would take longer to perform HITs on a mobile device (arithmetic mean = 4.39, SD = 0.69), and that the screen (4.25, SD = 1.04) and keypad (4.02, SD = 1.15) on mobile devices were less functional. In India (N = 37), PC or laptop-only users also agreed strongly with the impracticality of the keypad (3.81, SD = 1.02) and the screen (3.59, SD = 1.07), followed by the impossibility of doing HITs on a mobile device (3.49, SD = 1.22). Task completion time, however, was less of a concern for Indian workers compared with US workers (3.30, SD = 1.31).

Figure 1 displays key factors that inhibit performing HITs on mobile devices in the United States and India among the mixed user groups. As depicted in Figure 1, the mixed user groups both agreed that the key barriers to using mobile devices for performing HITs were that it takes longer to do HITs on a mobile device, that the screen and keypads on a mobile device were impractical for performing HITs, and that the HITs usually performed cannot be done on a mobile device. Figure 1 also demonstrates how affordability constraints, namely a lack of a reliable Internet connection, high price of mobile data, and having to share a mobile device with someone else play a subordinate role in the United States. By contrast, in India, the affordability constraints are much more pronounced as barriers for using mobile devices to perform HITs.

Figure 1.

Barriers to performing mobile hits in the mixed group (N = 248 in United States; 275 in India, arithmetic means are shown on top of bars).

A PCA among the US-based mixed user group reveals three components among the barriers to using mobile devices for performing HITs (see Table 1): (1) device usability and practicality, (2) affordability, and (3) task and software constraints. No significant correlations between these factors and demographic characteristics (age, gender, income, education) could be identified, indicating that barriers are perceived similarly across demographic groups. In India, the PCA resulted in two components (Table 2): (1) affordability and (2) functionality. Here, the affordability factor was significantly correlated to age (r = −.32, p = .000) and income (r = −.28, p = .000), indicating that younger and low-income users score higher on affordability barriers than older and higher income users. Gender and education were not significantly correlated to the affordability component but we found a significant correlation between gender and functionality barriers (r = .18, p = .005). Given the coding of the variable (1 = female, 2 = male), the positive coefficient indicates that male AMT crowdworkers report higher functionality barriers than female AMT crowdworkers.

Table 1.

Principal component analysis for mixed users in the United States.

Items	Component 1: Device usability and practicality	Component 2: Affordability	Component 3: Task and software constraints
The keypad on mobile devices is impractical for performing HITs.	0.84	−0.05	0.09
The screen on mobile devices is impractical for performing HITs.	0.80	−0.02	−0.05
The HITs I usually perform would take longer to do on a tablet or smartphone.	0.63	−0.15	0.21
The quality of the HITs I usually perform would be lower (more mistakes) on a tablet or smartphone.	0.63	0.19	0.08
I lack a reliable Internet connection on my tablet or smartphone.	0.05	0.82	0.05
Using mobile data to perform HITs would be too expensive.	0.07	0.75	−0.01
I share my mobile devices with other people.	−0.13	0.70	−0.04
The HITs I usually perform cannot be done on a table or smartphone.	0.07	0.00	0.79
My tablet or smartphone does not have the software (including APIs) I need to perform HITs.	0.12	0.01	0.73

AMT: Amazon Mechanical Turk; HIT: human intelligence task; API: application programming interface. Bold-faced values indicate that the item refers to the specific component in the column.

N = 248; Varimax rotation; Kaiser criterion; item “AMT is not mobile friendly” removed because it formed an own component; standardized loadings displayed.

Table 2.

Principal component analysis for mixed users in India.

Items	Component 1: Affordability	Component 2: Functionality
I lack a reliable Internet connection on my tablet or smartphone.	0.82	0.07
Using mobile data to perform HITs would be too expensive.	0.82	0.09
I share my mobile devices with other people.	0.76	0.20
My tablet or smartphone does not have the software (including APIs) I need to perform HITs.	0.68	0.27
The screen on mobile devices is impractical for performing HITs.	−0.02	0.75
The HITs I usually perform would take longer to do on a tablet or smartphone.	0.04	0.73
The keypad on mobile devices is impractical for performing HITs.	0.13	0.66
The HITs I usually perform cannot be done on a table or smartphone.	0.22	0.62
The quality of the HITs I usually perform would be lower (more mistakes) on a tablet or smartphone.	0.29	0.55
MTurk is not mobile friendly.	0.30	0.53

HIT: human intelligence task; API: application programming interface. Bold-faced values indicate that the item refers to the specific component in the column.

N = 275; Varimax rotation; Kaiser criterion; standardized loadings displayed.

These quantitative results are mirrored in the open text-field comments provided by the respondents, who discussed the numerous functional and infrastructural barriers to using mobile devices to perform HITs. A key theme was hindrances on efficiency, with respondents commenting that mobile devices caused lower efficiency and speed leading to a lower income. Since crowdworkers earn on a per-HIT basis, greater speed in completing tasks can result in more income-generating opportunities and thus more income:

Aside from mobile device specific HITs, working on MTurk on mobile devices is not only impractical, it’s unprofitable, and likely a tremendous waste of time. It’s extremely difficult to catch HITs, the interface is difficult to use, and most users wouldn’t be able to get the necessary qualifications to access better HITs while working on a mobile device. You’re basically forced to work on HITs using a desktop or laptop up until a certain threshold (1000 - 5000 HITs completed) before you can even get access to better paying HITs. (US respondent, 32, male, mixed category)

I use it ONLY when HITs are REQUIRED to be done on a mobile device. Otherwise I find it way too frustrating, slow, and impractical. The screen is too small. The keyboard is a pain to use. Navigating MTurk and copying/pasting completion codes is a nightmare. It’s just much, much more efficient and pleasant to do MTurk HITs on a PC or laptop. I want more speed, a larger screen and keyboard, and a more physical comfortable working situation, and that’s what the PC (in particular) gives me. (US respondent, 40, male, PC/laptop-only category)

One of the most important barriers to using mobile devices which emerged from the text comments was the role of the requester in preventing crowdworkers from selecting a mobile device. Certain tasks were impossible to be completed on a mobile device, but others were requested not to be completed on mobile devices even if it was functionally possible. Among respondents, this requester requirement was greeted with mixed responses, from apathy and understanding to frustration and annoyance:

Most of the time, it makes no difference if you use a mobile device or not, but some requesters put a requirement to NOT use a mobile device, which is infuriating and makes me not wanna work on their HITs. (US respondent, 26, male, mixed category)

However, conversely, it is also the requester who is often responsible for determining mobile device choice. Certain HITs can be specifically designed for mobile use, in which cases the use of a mobile device provides greater opportunities and a broader selection of HIT types. Nevertheless, among our respondents who discussed completing mobile-only HITs due to requester preference, respondents across both the United States and India commented that they would complete these requests on mobile devices as an exception and on-demand rather than pursue mobile-only HITs as a deliberate choice:

The only reason I do use my phone sometimes is because some hits will require them to be taken from a cell phone. When this happens I usually accept the HIT from my computer and then go on my mobile device to complete the HIT. This is a rare occasion though. (US respondent, 29, male, mixed category)

HIT-related activities

While performing HITs constitutes the central income-generating activity for crowdworkers, HIT performance exists within a broader range of non-HIT activities which crowdworkers must also perform, such as browsing for HITs, profile administration, and communicating with requesters, AMT, or other crowdworkers. Based on the responses from both samples, we found that these non-HIT tasks are carried out more frequently on a PC or laptop than on mobile devices. The smallest difference between mobile device and non-mobile device use is for “communicating with other AMT workers (e.g. through online forums),” thus suggesting that mobile devices fulfill social rather than functional and administrative needs. Focusing specifically on smartphones and laptops as two contrasting devices, we find pronounced differences in use frequency for non-HIT activities (Table 3). Overall, all task-related activities are performed more frequently on a laptop compared with a smartphone. However, the difference in device use for such activities is larger in the United States than in India, where these activities are overall more frequent.

Table 3.

Arithmetic means for task-related activities by device type.

Task-related activity	Average laptop use frequency	Average smartphone use frequency
Browsing for HITs: USA	3.61	2.10
Browsing for HITs: India	4.06	3.19
Communicating with a requester: USA	3.19	1.87
Communicating with a requester: India	3.98	3.05
Communicating with AMT: USA	2.83	1.77
Communicating with AMT: India	3.93	2.97
Performing profile administration: USA	3.22	2.09
Performing profile administration: India	3.95	3.02
Communicating with other AMT workers: USA	2.68	1.94
Communicating with other AMT workers: India	3.66	3.02

AMT: Amazon Mechanical Turk; HIT: human intelligence task; API: application programming interface.

N = 293 for United States and 299 for India (14 missing values)

Wage impact

To investigate how demographic, AMT-related, and device-related factors affect economic outcomes from crowdwork, we conducted a linear regression analysis. This tests the mobile underclass argument in the context of crowdwork and also connects the findings to the literature on the third-level digital divide (Blank and Lutz, 2018; Van Deursen and Helsper, 2015).

Table 4 shows the results of the linear regression analysis for both the United States and India. We find demographic impacts, particularly in terms of age and income, on average per-hour earnings in both sample countries. Thus, younger crowdworkers in both countries report earning more than their older counterparts and crowdworkers with a higher household income have a higher hourly wage on AMT. AMT-specific characteristics, such as Master status, income reliance, and working hours, have a stronger effect on average per-hour earnings in the United States than in India. Crowdworkers with a Master qualification and those who are working more hours on AMT report earning more per hour on average in the United States. The positive effect of experience in the United States had a p value of exactly .050 and was thus considered non-significant. In India, however, neither experience nor seniority led to higher per-hour earnings. In the United States, crowdworkers who are less reliant on their AMT income also report earning more per hour. More importantly, for questions of device impact, we found that device choice plays a small but significant impact on per-hour earnings in the United States. Namely, the use of smartphones to perform HITs had a negative and significant effect. In India, however, no significant device effects could be detected.

Table 4.

Linear regression of estimated hourly earnings on AMT.

Variable	USA	India
Education	.02 (0.08)	−.06 (0.27)
Gender	.01 (0.20)	.05 (0.35)
Age	−.31*** (0.01)	−.15** (0.02)
Income	.23*** (0.03)	.18** (0.03)
Use frequency on PC	.01 (0.12)	.07 (0.18)
Use frequency on laptop	.25* (0.13)	−.12 (0.24)
Use frequency on smartphone	.04 (0.12)	−.03 (0.17)
Use Frequency on tablet	−.02 (0.10)	.02 (0.18)
AMT experience	.12 (0.07)	−.12* (0.08)
Master status	.15** (0.23)	.08 (0.38)
AMT income reliance (Ref.: Fully)
AMT income reliance: partly	.20* (0.32)	−.04 (0.43)
AMT income reliance: little	.25** (0.38)	−.08 (0.57)
AMT hours working per week	.32*** (0.08)	−.04 (0.08)
Equipment investment	.11 (0.04)	.04 (0.05)
AMT work frequency on PC	.12 (0.13)	−.08 (0.20)
AMT work frequency on laptop	−.14 (0.11)	.10 (0.25)
AMT work frequency on tablet	.07 (0.16)	.07 (0.21)
AMT work frequency on smartphone	−.12* (0.11)	.05 (0.15)
Constant	1.22 (1.12)	51.63 (17.07)
R ²	.31	.10

AMT: Amazon Mechanical Turk.

Dependent variable: Hourly earnings in USD for United States and INR for India; N = 292 for United States and 303 for India; standardized regression coefficients shown; ***p < .001, **p < .01, *p < .05, no star = not statistically significant; Robust standard errors in brackets.

Overall, we are able to explain substantially more variance in the United States than in India. In the United States, 30% of the overall variance in estimated hourly earnings could be explained by the independent variables considered, while in India only 10% could be explained.

Summing up, the results from the regression analysis imply that there might be an economic penalty for carrying out HITs on a mobile device, specifically a smartphone. Other effects, for example, in terms of age and income, also point to stratification along socioeconomic categories in the sense that more privileged users benefit more from working on AMT than less privileged users.

Investment

In terms of device investment, we found that there were key differences among our respondents which reaffirmed structural inequalities in device use. In the United States, 55% of our respondents had not invested anything for the purpose of crowdworking. The average investment in the United States was in the category US$26–50, with the median being US$0. Based on text-field comments, the US respondents reported investing more as an additive measure, to gain extra comfort or speed. New office furniture, for instance, or faster peripherals in the form of ergonomic keyboards and mice were key investment choices. In addition, several respondents invested in specific technologies to garner access to higher paying HITs. However, this focus on additive investments was correlated with a preponderance of structural requirements already being catered for in terms of already owning or having access to the basic requirements for crowdwork. Rarely did US respondents discuss investing in fundamentals such as a computer or Internet connection, except in a few cases where respondents upgraded to a higher quality PC or laptop. Indeed, many respondents remarked that crowdwork was a low-investment activity and investment in additional equipment would negate the low-income gained from crowdwork. In essence, the cost–benefit ratio was not sufficient for investment except among high-frequency users:

I haven’t invested in upgrading or purchasing equipment to work on Mturk because I don’t think it would be worth it. My computer and internet speed is already good and I don’t think I need anything else to make any more money. I also don’t think the amount of money I would make in return would be worth me investing any significant amount of money into special equipment. (US respondent, 35, female, mixed category)

In contrast, 85% of the India-based sample reported investing specifically for the purposes of crowdwork, focusing more often on structural investments such as broadband Internet connections, mobile data plans, and computers:

When I used my old Internet connection, I was unable to accept some of the good HITs. So, I thought to change my Internet provider with high-speed Internet and hence I purchased a new high-speed Internet connection for the purpose of turking. (Indian respondent, 42, male, mixed category)

The average investment in India was in the category INR 3001–4000 (between US$43 and US$58), with the median being INR 2001–3000 (between US$29 and US$43). In the United States, the average investment was in the category US$26–50 and the median US$0. On average, AMT workers in India and the United States invest similar amounts. However, looking at the distributions more closely, investments are much more skewed in the United States, with a minority of 12.6% in the top category (more than US$500) and a majority of 54.5% investing US$0. In India, only 3.3% are in the top category (more than INR 50,000) but a substantial share of respondents is in the categories immediately below the maximum (12.9% in the 20,001–50,000 category, 6.3% in the 10,001–20,000 category and 5.9% in the 7501–10,000 category). Thus, in proportion to their income, workers in India invest much more in equipment for AMT. However, in India, the proportion of workers who report that the work on AMT is their main source of income is substantially higher than in the United States (42.2% vs 17.4%), so that such initial investments might be more necessary. Financial investments are not correlated with demographic characteristics in the United States, but in India, investments and household income correlate significantly and positively (r = .20, p = .000), indicating a rich-get-richer effect.

Discussion and conclusion

In this contribution, we provided an initial exploration into device use by online crowdworkers, focusing on AMT. The goal of this study was to examine how mobile devices could act as both an enabling and constraining factor in crowdwork. Our findings indicate that crowdworkers face both opportunities and barriers when using mobile devices, but that using mobile devices overwhelmingly constitutes a minority activity undertaken either as a last resort or for their particular affordances such as portability. In particular, mobile devices act as a valuable complement in a broader device ecosystem, aiding workflow and for additional task access. The practice of second screening, in particular, became apparent as a mode of use among the India-based sample. However, this might have to do with the differences in tasks undertaken between the US-based and India-based samples: In the United States, most respondents primarily carried out surveys whereas the types of tasks undertaken were more varied in India, where respondents also tag images, transcribe audio and write content with greater frequency.

Mobile-first or mobile-only Internet use is increasingly common, especially for entertainment and social purposes. However, in the domain of work, including new forms of work, such practices seem marginalized and preferences remain firmly attuned toward traditional PC or laptop devices. The functional constraints of mobile devices, compared with PC or laptop devices, acted as significant barriers to adoption, in particular, the impracticality of accessing and entering information on the smaller screens and keyboards of mobile devices. For the often data-intensive tasks made available, the processing speeds and Internet connection speeds of mobile devices proved insufficient for requirements, while the lack of access to specific software such as computer scripts to “catch HITs” limited the efficiency of mobile devices. Since efficiency and speed are central to income generation on crowdworking platforms such as AMT, even minor differences in speed and efficiency between devices could result in reduced income over time. Indeed, for the US-based sample, we were able to show a negative but weak device effect for smartphone use for carrying out HITs, showing that relying on mobile devices too heavily might result in being financially penalized. Based on these findings, crowdwork—at least of the type offered through AMT—does not represent an effective economic opportunity for the mobile-only or mobile-first underclass. In fact, the mobile-unfriendliness of AMT and similar platforms presents a distinct barrier to digital inclusion of mobile-only and mobile-first groups. Since our findings allow only limited insights into the specific hardships of mobile-only and mobile-first groups, future research on mobile crowdwork might focus on these groups. Qualitative methods could elicit whether and how such groups have tried to engage in both online task crowdwork and on-demand types of platform labor.

From an investment point of view, we noted that there were inequalities between those who were able to invest financially to gain access to better paid and more tasks and those who had to invest out of necessity to meet the basic requirements of participation. In this case, we noticed a rich-get-richer effect, where differential access to device “assemblages” could be connected to differential access to income opportunities. In this case, access to only a mobile phone would result in a poorer experience and vastly reduced income opportunities, not only as a result of reduced efficiency but as a result of reduced task availability.

One of the most striking factors which emerged from this study was the role of the requester as a restricting force. While crowdwork has a connotation of flexibility and mobility, with workers being able to do tasks in their free time and whenever they prefer, by restricting tasks to a specific device due to requester preference, the flexibility of workers is reduced not as a result of functional limitations but as a result of task design and availability. Without mobile-accessible tasks, discussion around mobile-readiness of crowdworkers is rendered largely moot. Interestingly, AMT (2016) announced a more mobile-friendly experience in 2016 and actively encouraged requesters to make their HITs mobile friendly. Yet, despite such encouragement, it remains unclear as to what extent requester behavior shifted and whether AMT will take further steps to make the experience even more mobile friendly in the future.

In comparative terms, we found considerable differences between the United States and India. Affordability constraints (e.g. lack of a reliable Internet connection, having to share a device) were much more pronounced in India than in the United States, while investments in infrastructure for crowdwork were more evenly distributed in India and proportionally much higher. Second screening was also more prevalent in India. Overall, mobile devices played a more important role among AMT workers in India, compared with those in the United States. Taken together, these findings show how device-related aspects play out differently in shaping or overcoming inequalities, depending on the national context.

To make crowdwork more inclusive and mobile friendly, effective policies need to take into account the specific barriers in a given context. In both countries, platforms should address functional barriers. In India, platforms and requesters could support crowdworkers by helping them overcome the affordability constraints, for example, by subsidizing or sponsoring data plan upgrades or by providing workplaces. Samasource and similar initiatives in the mobile crowdsourcing literature show promising examples of mobile crowdwork (Kumar et al., 2014; Narula et al., 2011; Vaish et al., 2014). In the United States, platforms and requesters could design more mobile-friendly interfaces and tasks, making the case how the combination of mobile devices and non-mobile devices could lead to increased productivity.

Our study comes with several limitations, such as the self-selection of respondents and the inability to recruit crowdworkers that specialize in non-survey tasks through our research design. We encourage future researchers to conduct qualitative research with workers specializing in non-survey tasks (image tagging, content production, sentiment) to explore the potentials and constraints of mobile devices in more depth. Moreover, our findings emerge from a sample of crowdworkers who are predominantly not mobile-only users. As such, future research into mobile-only users who have tried and/or considered crowdwork would no doubt yield valuable insights.

Our surveys were also cross-sectional, inhibiting strong causal claims over time. Especially for investigating third-level digital divides and the tangible outcomes from device use, longitudinal surveys or a combination of different data sources (including log data) would be useful. Controlled human–computer interaction (HCI) experiments, where workers carry out tasks either through a mobile device, a laptop/PC, or through a combination of both, would provide solid evidence about the device-outcome link.

Finally, we only looked at one platform with specific affordances. In the digital labor and crowdwork ecosystem, other players have emerged and seem to cater more actively to mobile users. For example, Clickworker has an app specifically targeting mobile functionalities such as taking pictures on the go, Gigwalk focuses on distributed management through mobile devices and location-based capabilities, and in HCI research, different solutions for mobile crowdsourcing have been developed (e.g. Chi et al., 2018; Yan et al., 2009). Future research could compare similar purpose platforms that differ in their mobile friendliness in terms of inclusivity and divides.

Footnotes

Authors’ note

All authors have agreed to the submission and the article is not currently being considered for publication by any other print or electronic journal.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research received funding by the Research Council of Norway within grant agreement 275347 “Future Ways of Working in the Digital Economy”.

ORCID iD

Christoph Lutz

Author biographies

Gemma Newlands is a PhD Candidate at the University of Amsterdam and a doctoral research fellow at the Nordic Centre for Internet and Society, BI Norwegian Business School. Her research interests include the platform economy, artificial intelligence and human-computer interaction in the workplace. Gemma has published in top-tier journals such as New Technology, Work and Employment, Internet Research, Journal of Business Research, First Monday, and Research in the Sociology of Organizations.

Christoph Lutz is an associate professor at the Nordic Centre for Internet & Society, BI Norwegian Business School (Oslo). His research interests include online participation, privacy, the sharing economy, and social robots. Christoph has published widely in top-tier journals in this area such as New Media & Society, Mobile Media & Communication, Information, Communication & Society, Social Media + Society, and International Journal of Communication.

References

Alkhatib

Bernstein

Levi

, et al. (2017) Examining crowd work and gig work through the historical lens of piecework. In: CHI’17: Proceedings of the 35th annual ACM conference on human factors in computing systems, Denver, CO, 6–11 May, pp. 4599–4616. New York: ACM.

Amazon Mechanical Turk (AMT) (2016) What’s New with Amazon Mechanical Turk. Available at: https://blog.mturk.com/whats-new-with-amazon-mechanical-turk-dc3d982e063a (accessed 5 August 2019).

Amazon Mechanical Turk (AMT) (2017) Now you can complete HITs on the new Worker website. Available at: https://blog.mturk.com/now-you-can-complete-hits-on-the-new-worker-website-6fab0da9ca80 (accessed 5 August 2019).

Berg

Furrer

Harmon

, et al. (2018) Digital Labour Platforms and the Future of Work: Towards Decent Work in the Online World. Geneva: International Labor Office.

Bergvall-Kåreborn

Howcroft

(2014) Amazon Mechanical Turk and the commodification of labour. New Technology, Work and Employment 29(3): 213–223.

Blank

Lutz

(2018) Benefits and harms from Internet use: a differentiated analysis of Great Britain. New Media & Society 20(2): 618–640.

Brewer

Morris

Piper

(2016) Why would anybody do this? Understanding older adults’ motivations and challenges in crowd work. In: Proceedings of the 2016 CHI conference on human factors in computing systems, San Jose, CA, 7–12 May, pp. 2246–2257. New York: ACM.

Bucher

Fieseler

(2017) The flow of digital labor. New Media & Society 19(11): 1868–1886.

Castells

Fernandez-Ardevol

Qiu

, et al. (2007) Mobile Communication and Society: A Global Perspective. Cambridge, MA: MIT Press.

10.

Chi

PYP

Batra

Hsu

(2018) Mobile crowdsourcing in the wild: challenges from a global community. In: Proceedings of the 20th international conference on human-computer interaction with mobile devices and services adjunct, Barcelona, 3–6 September, pp. 410–415. New York: ACM.

11.

Chigona

Beukes

Vally

, et al. (2009) Can mobile Internet help alleviate social exclusion in developing countries? The Electronic Journal on Information Systems in Developing Countries 36(7): 1–16.

12.

Chircu

Mahajan

(2009) Revisiting the digital divide: an analysis of mobile technology depth and service breadth in the BRIC countries. Journal of Product Innovation Management 26(4): 455–466.

13.

Cotten

Anderson

Tufekci

(2009) Old wine in a new technology, or a different type of digital divide? New Media & Society 11(7): 1163–1186.

14.

Donner

(2008) Research approaches to mobile use in the developing world: a review of the literature. The Information Society 24(3): 140–159.

15.

Donner

Gitau

Marsden

(2011) Exploring mobile-only Internet use. International Journal of Communication 5: 574–597.

16.

eMarketer (2018) More than a quarter of India’s population will be smartphone users this year. Available at: https://www.emarketer.com/content/more-than-a-quarter-of-india-s-population-will-be-smartphone-users-this-year

17.

Gandini

(2019) Labour process theory and the gig economy. Human Relations 72(6): 1039–1056.

18.

Gitau

Marsden

Donner

(2010) After access: challenges facing mobile-only Internet users in the developing world. In: Proceedings of the SIGCHI conference on human factors in computing systems, Atlanta, GA, 10–15 April, pp. 2603–2606. New York: ACM.

19.

Gonzales

(2014) Health benefits and barriers to cell phone use in low-income urban U.S. neighborhoods: indications of technology maintenance. Mobile Media & Communication 2(3): 233–248.

20.

Gray

Suri

(2019) Ghost Work. New York: Houghton Mifflin Harcourt.

21.

Griffiths

(2007) Future assemblies: theorizing mobilities and users. New Media & Society 9(6): 1029–1036.

22.

GSMA (2015) The mobile economy: India 2015. Available at: https://www.gsmaintelligence.com/research/?file=4113a57d43a9e93968e7ed00123ba4b2&download

23.

GSMA (2018) The mobile economy: North America 2018. Available at: https://www.gsmaintelligence.com/research/?file=1edb46b8f8d86187a7508bad348c3e87&download

24.

Hara

Milland

Hanrahan

, et al. (2019) Worker demographics and earnings on Amazon Mechanical Turk: an exploratory analysis. In: CHI EA 2019—Extended abstracts of the 2019 CHI conference on human factors in computing systems, Glasgow, 4–9 May.

25.

Hargittai

(2002) The second-level digital divide: differences in people’s online skills. First Monday 7(4). Available at: https://firstmonday.org/article/view/942/864

26.

Hjorth

Burgess

Richardson

(2012) Studying the mobile: locating the field. In: Hjorth

Burgess

Richardson

(eds) Studying Mobile Media: Cultural Technologies, Mobile Communication and the Iphone. London: Routledge, pp. 1–7.

27.

Howcroft

Bergvall-Kåreborn

(2018) A typology of crowdwork platforms. Work, Employment and Society 33: 21–38.

28.

Humphreys

Von Pape

Karnowski

(2013) Evolving mobile media use: uses and conceptualizations of the mobile Internet. Journal of Computer-Mediated Communication 18(4): 491–507.

29.

Hunt

Samman

Mansour-Ille

(2017) Syrian women refugees: opportunity in the gig economy? Overseas Development Institute. Available at: https://www.odi.org/sites/odi.org.uk/files/resource-documents/11742.pdf

30.

Irani

(2015) The cultural work of microwork. New Media & Society 17(5): 720–739.

31.

Isomursu

Hinman

Isomursu

, et al. (2007) Metaphors for the mobile Internet. Knowledge, Technology & Policy 20(4): 259–268.

32.

ITU (2017) ICT Facts and figures 2017. International Telecommunications Union (ITU). Available at: https://www.itu.int/en/ITU-D/Statistics/Documents/facts/ICTFactsFigures2017.pdf

33.

James

(2009) Leapfrogging in mobile telephony: a measure for comparing country performance. Technological Forecasting and Social Change 76(7): 991–998.

34.

Kittur

Nickerson

Bernstein

, et al. (2013) The future of crowd work. In: Proceedings of the ACM conference on computer supported cooperative work, San Francisco, TX, 23–27 February, pp. 1301–1318. New York: ACM.

35.

Kuek

Paradi-Guilford

Linden

, et al. (2013) Microwork for the Palestinian Territories (Feasibility study for the World Bank). Available at: http://siteresources.worldbank.org/INTWESTBANKGAZA/Resources/Finalstudy.pdf

36.

Kumar

Yadav

Dev

, et al. (2014) Wallah: design and evaluation of a task-centric mobile-based crowdsourcing platform. In: Proceedings of the 11th international conference on mobile and ubiquitous systems: Computing, networking and services, London, 2–5 November, pp. 188–197. New York: ACM.

37.

Lutz

(2019) Digital inequalities in the age of artificial intelligence and big data. Human Behavior and Emerging Technologies 1(2): 141–148.

38.

Marler

(2018) Mobile phones and inequality: findings, trends, and future directions. New Media & Society 20(9): 3498–3520.

39.

Martin

Hanrahan

O’Neill

, et al. (2014) Being a Turker. In: Proceedings of the 17th ACM conference on computer supported cooperative work & social computing, Baltimore, MD, 15–19 February 2014, pp. 224–235. New York: ACM.

40.

Medhi

Patnaik

Brunskill

, et al. (2011) Designing mobile interfaces for novice and low-literacy users. ACM Transactions on Computer–Human Interaction (TOCHI) 18(1): 2.

41.

Mossberger

Tolbert

Hamilton

(2012) Measuring digital citizenship: mobile access and broadband. International Journal of Communication 6: 2492–2528.

42.

Napoli

Obar

(2014) The emerging mobile Internet underclass: a critique of mobile Internet access. The Information Society 30(5): 323–334.

43.

Narula

Gutheim

Rolnitzky

, et al. (2011) Mobileworks: a mobile crowdsourcing platform for workers at the bottom of the pyramid. In: Workshops at the twenty-fifth AAAI conference on artificial intelligence, San Francisco, CA, 27–31 July, pp. 121–123. Menlo Park, CA: AAAI.

44.

Paolacci

Chandler

Ipeirotis

(2010) Running experiments on Amazon Mechanical Turk. Judgment and Decision Making 5(5): 411–419.

45.

Pearce

Rice

(2013) Digital divides from access to activities: comparing mobile and personal computer Internet users. Journal of Communication 63(4): 721–744.

46.

Poushter

Bishop

Chwe

(2018) Social media use continues to rise in developing countries but plateaus across developed ones. Pew Research Center Report. Available at http://www.pewglobal.org/2018/06/19/social-media-use-continues-to-rise-in-developing-countries-but-plateaus-across-developed-ones/#table

47.

Ross

Irani

Silberman

, et al. (2010) Who are the crowdworkers? Shifting demographics in Mechanical Turk. In: CHI’10 extended abstracts on human factors in computing systems, Atlanta, GA, 10–15 April, pp. 2863–2872. New York: ACM.

48.

Ruppert

Law

Savage

(2013) Reassembling social science methods: the challenge of digital devices. Theory, Culture & Society 30(1): 22–46.

49.

Schrock

(2015) Communicative affordances of mobile media: portability, availability, locatability, and multimediality. International Journal of Communication 9: 1229–1246.

50.

Shapiro

(2018) Between autonomy and control: strategies of arbitrage in the “on-demand” economy. New Media & Society 20(8): 2954–2971.

51.

Smith

(2015) U.S. smartphone use in 2015. Pew Research Center. Available at: http://www.pewinternet.org/2015/04/01/us-smartphone-use-in-2015/

52.

Thompson

(2018) The making of mobilities in online work-learning practices. New Media & Society 20(3): 1031–1046.

53.

Tsetsi

Rains

(2017) Smartphone Internet access and use: extending the digital divide and usage gap. Mobile Media & Communication 5(3): 239–255.

54.

Tungare

Pérez-Quiñones

(2009) Mental workload in multi-device personal information management. In: CHI’09 extended abstracts on human factors in computing systems, Boston, MA, 4–9 April, pp. 3431–3436. New York: ACM.

55.

Ureta

(2008) Mobilising poverty? Mobile phone use and everyday spatial mobility among low income families in Santiago, Chile. The Information Society 24(2): 83–92.

56.

Vaish

Wyngarden

Chen

, et al. (2014) Twitch crowdsourcing: crowd contributions in short bursts of time. In: Proceedings of the SIGCHI conference on human factors in computing systems, Toronto, ON, Canada, 28 April-1 May 2014, pp. 3645–3654. New York: ACM.

57.

Van Deursen

Helsper

(2015) The third-level digital divide: who benefits most from being online? In: Robinson

Cotton

Schulz

, et al. (eds) Studies in Media and Communications. Bingley: Emerald, pp. 29–52.

58.

Veen

Barratt

Goods

(2020) Platform-capital’s “App-etite” for control: a labour process analysis of food-delivery work in Australia. Work, Employment and Society 34(3), 388–406. DOI: 10.1177/0950017019836911.

59.

Wang

Liu

(2017) The mobile Internet underclass: reality or hyperbole? Journal of Information Science 44(5): 569–579.

60.

Wikipedia (2019) Amazon Mechanical Turk. Available at: https://en.wikipedia.org/wiki/Amazon_Mechanical_Turk (accessed 19 September 2019).

61.

Wyche

Simiyu

Othieno

(2018) Understanding women’s mobile phone use in rural Kenya: an affordance-based approach. Mobile Media & Communication 7: 94–110.

62.

Yan

Marzilli

Holmes

, et al. (2009) mCrowd: a platform for mobile crowdsourcing. In: Proceedings of the 7th ACM conference on embedded networked sensor systems, Berkeley, CA, 4–6 November, pp. 347–348. New York: ACM.

63.

Yesilada

Harper

Chen

, et al. (2010) Small-device users situationally impaired by input. Computers in Human Behavior 26: 427–435.

64.

Zillien

Hargittai

(2009) Digital distinction: status-specific types of Internet usage. Social Science Quarterly 90(2): 274–291.

65.

Zyskowski

Morris

Bigham

, et al. (2015) Accessible crowdwork? Understanding the value in and challenge of microtask employment for people with disabilities. In: Proceedings of the 18th ACM conference on computer supported cooperative work & social computing, Vancouver, BC, Canada, 14–18 March, pp. 1682–1693. New York: ACM.