Abstract
This article scrutinises the potential of the existing regulatory apparatus in Union law to tackle the social, technical, and legal challenges inherent in deploying automated systems in high-risk settings such as the workplace, with a view to setting out key lessons for the proposed EU Artificial Intelligence Act. Surveying data protection and discrimination rules as well as the social acquis, it highlights key areas for further development, from coherence between different regulatory regimes to the role of social partnership in shaping key standards and monitoring their implementation.
Keywords
Introduction
The European Commission's proposal for a Regulation setting out harmonised rules on artificial intelligence, published in the spring of 2021 and styled the Artificial Intelligence Act (‘AI Act’),
2
has thrust the deployment of algorithmic management systems firmly into the European regulatory spotlight. Annex III to the proposed legislative instrument lists a number of contexts in which the deployment of ‘AI systems’, defined very broadly to encompass all software incorporating a wide range of machine learning techniques, logic based systems, and other statistical computational methods,
3
shall be ‘considered high-risk’.
4
Crucially, this includes ‘Employment, workers management and access to self-employment’, covering both AI systems intended to be used for recruitment or selection of natural persons, notably for advertising vacancies, screening or filtering applications, evaluating candidates in the course of interviews or tests; [and] AI intended to be used for making decisions on promotion and termination of work-related contractual relationships, for task allocation and for monitoring and evaluating performance and behavior of persons in such relationships.
5
The recognition that algorithmic systems can pose significant risks when deployed at work is to be welcomed, as are the broader discussions about regulatory approaches to fast-emerging technologies which have been triggered by the proposed AI Act. 6 At the same time, however, it is important to recognise that Union law already regulates various facets of digital management systems. The proposed AI Act does not exist in a vacuum: relevant regimes range from data protection rules and the social acquis through to internal market measures. 7
This article sets out to scrutinise the potential of the existing regulatory apparatus in Union law to tackle the social, technical, and legal challenges inherent in deploying automated systems in high-risk settings, identifying promise as well as a number of gaps and broader considerations. As the AI Act is debated and forged in the Union's legislative process, it will be important to consider its interaction with the full range of relevant instruments, and to heed lessons learnt from regulatory successes and failures to date.
To this end, discussion is structured as follows: an initial section introduces a number of case studies of algorithmic management (sometimes also known as ‘people analytics’ or ‘Big Data HR’), including both control exercised by gig economy platforms, viz companies engaged in digital work intermediation, and the increasing augmentation of management functions in firms across the socio-economic spectrum. Section three sets out the ensuing regulatory challenges, identifying three particularly salient domains of risk: data collection, novel forms of processing, and changing ways of exercising employer control. Section four tackles each of these in turn, scrutinising the success of existing Union norms both in the individual and collective sphere in addressing the problems identified, with section five drawing together key lessons for (algorithmic) regulatory debates. Section six offers a preliminary analysis of the AI Act to explore the extent to which, if at all, the issues identified in previous sections are addressed in the current proposal. The concluding section takes a step back from this detailed scrutiny and addresses the potential conflict between the digital internal market and the social acquis in the context of the European Union's digital strategy.
Algorithms at Work
Until recently, the rise of algorithmic management has been relatively neglected amidst heated scholarly debates about the impact of technology on the future of work. 8 The focus has tended to be on the role of digitalisation and automation in shaping the future of work through their macro-level impacts: from widespread unemployment to the more subtle effects of the ‘wiring the labour market’, as David Autor has put it. 9 Debates about the future of labour market regulation in a world without work are a crucial challenge for the discipline; 10 but so is the deployment of automated management systems – if not even more urgently so.
The legal, economic, and social incentives are complex. 11 In short, as dramatic increases in computing power and data availability have led to a renaissance of ‘artificial intelligence’, the use of machine learning and similar computational techniques are increasingly deployed in workplaces across the socio-economic spectrum, from gig-economy platforms to major professional service companies and the public sector. As the cost of data collection and processing continue to fall, platforms are able to deploy technology to monitor – and control – their workforce to a hitherto unimaginable degree. 12 We are witnessing a ‘(re-) wiring of the firm’.
Given the fast pace of innovation and deployment, algorithmic management to date has defied precise definition or typology: for present purposes, the label encompasses the digitalisation of the full range of traditional employer functions, from hiring workers and managing the day-to-day operation of firm-internal markets through to engaging with enterprise-external markets (e.g. in price setting) and terminating employment relationships. 13
This automation happens on a number of different spectra: in terms of data, first, different software packages might offer to use employers’ existing workplace monitoring infrastructure or provide novel ways of collecting detailed workforce information. In terms of processing, second, traditional rule-based algorithms are increasingly supplemented or even supplanted by recourse to machine learning and other advanced statistical techniques. Finally, in terms of control, algorithmic management builds on received models of workplace monitoring, whilst potentially going significantly further by processing workforce data with a view to providing detailed recommendations as to how employer functions should be exercised (augmenting algorithmic management) – or indeed, at least in principle, to automate their exercise (fully automated algorithmic management). Different software applications will combine different aspects of each spectrum, depending on different management functions and local applications.
Platform Control 14
Algorithmic management first came to prominence in the context of the gig economy: nearly all platforms rely on a rating system of stars or points, designed at first glance to elicit consumer feedback on worker performance. In reality, however, ratings soon turned out to be an important source of data for early models of algorithmic management, substituting ‘for a company management structure’. 15 Even though specific approaches vary wildly across different platforms and time, the goal is often the same: rather than simply ‘matching’ well-informed workers and customers, most platforms are digital work intermediaries, in the business of tightly managing a large, invisible workforce. 16 Nearly every aspect of on-demand work is shaped by rating algorithms: from vetting potential entrants and assigning tasks to controlling how work is done and remunerated and sanctioning unsatisfactory performance—often without any transparency or accountability.
Algorithmic control in the gig economy is exercised in myriad ways along the three spectra identified above, often eschewing direct orders or explicit instructions, 17 and driven by sophisticated technology. Clients commissioning work through the online platform Upwork, for example, can check up on their workers through a so-called ‘work diary’: whenever a freelancer is engaged on an hourly-paid job, the platform's software captures regular screenshots of the freelancer's screen, counts keystrokes and records work completed to enable clients’ monitoring whether the freelancer is working for the whole of the time that she has billed for. 18 Control and surveillance encompass physical as well as digital aspects of work: Uber's app taps into the GPS, gyrometer, and acceleration sensors in each driver's iPhone to detect drivers’ speeding or abrupt braking. Whilst information about such control mechanisms is usually provided, it is often couched in terms of consumer or user protection, rather than as a key facet of employer control. 19
Alex Rosenblat and Luke Stark's study of these platform control mechanisms demonstrates how crowdwork conditions can easily be ‘shaped by the company's deployment of a variety of design decisions and information asymmetries via the application to effect a “soft control” over workers’ routines’. In practice, such automated control mechanisms will often be hard to escape or ignore – even though instructions are ‘carefully designed to be indirect, presumably to avoid the appearance of a company policy’. 20
Beyond the Gig Economy
Over a decade on, algorithmic control is by no means limited to platform-based work. Employees across the socio-economic spectrum, from factories and offices to universities and professional services firms, are increasingly managed by, or with assistance from, algorithms. Start-ups and established software providers promise to automate employer functions across the life cycle of the employment relationship, from hiring and managing workers through to firing them.
Perhaps the most visible application today is automated, or automation-augmented hiring, with tools ranging from online reputation screening 21 and CV filtering all the way through to video-supported hiring systems promising to use ‘candidates’ computer or cellphone cameras to analyze their facial movements, word choice and speaking voice before ranking them against other applicants based on an automatically generated “employability” score.’ 22
Once candidates are hired to become employees, algorithmic control may continue across most facets of the workplace. In terms of day-to-day management, for example, scheduling software increasingly automates the assignment of shifts and tasks. The changes are significant. 23 As Alexandra Mateescu and Aiha Nguyen explain, ‘these systems differ from manual scheduling practices because they base scheduling decisions on a wider range of historical data, such as weather and seasonal patterns, customer foot traffic, and past sales data.’ 24 The ensuing ‘accurate labour forecasting’ promises significant cost-savings for employers – whilst also leading to increased unpredictable and instable working conditions. 25
Scheduling is but one example of automated management functions: the full range of managerial tasks, from performance evaluation to pay setting, is increasingly prone to automation. 26 As Ajunwa, Crawford, and Schultz have argued, workers ‘must now contend with an all-seeing Argus Panoptes built from technology that allows for the trawling of employee data from the Internet and the employer collection of productivity data and health data.’ 27
In principle (though not within the European Union, as explained in section three, below), algorithmic management may even enable the fully automated termination of employment relationships. When faced with allegations of retaliatory dismissals in response to concerted trade union activity in one of its warehouses, Amazon revealed its extensive use of algorithmic management: the claimant's employment had been terminated for a lack of productivity, as determined by an automated system. Local warehouse management, the company's defence asserted, had had no input, control, or understanding of the details of the system deployed. 28 Litigation is pending before the Dutch courts to establish whether popular ridesharing platforms such as Uber have similarly relied on automated systems to terminate drivers, for example in cases of alleged fraud. 29
The Covid-19 Pandemic
The Covid-19 pandemic proved to be a turning point in terms of both scale and scope of algorithmic management. The rapid reordering of labour markets across the world led to an explosion in the deployment of algorithmic management software, in both white-collar and blue-collar professions. As regards the former, with traditional control and management options quickly disappearing as work shifted from offices to the home, software solutions to monitor staff working remotely became increasingly popular. 30 The potential for data collection and processing increased significantly: with nearly all staff interactions taking place through collaborative work software, information about workflows, interactions, and productivity could suddenly be collected at a comprehensive level. Perhaps unsurprisingly, many of these platforms offer various algorithmic management integration functions as a result. 31
Similar trends can be observed in workplaces that have been unable to shift to working from home, such as warehouses and factories. Here, algorithmic management systems were deployed for health and safety purposes, for example, by recording worker movements and providing live feedback and warnings through an AI ‘distance assistant’ if workers’ paths in the warehouse get too close. 32
* * *
Even from this brief survey of algorithmic management, three important points can be highlighted: first, the fact that (in principle at least) the full ambit of traditional employer functions can be automated, thus raising difficult questions about the applicability of the full range of domestic and European worker-protective norms, designed around traditional workplace structures in which the exercise of the managerial prerogative ultimately fell to a hierarchy of individuals. It should also be noted, second, that the development and deployment of such technologies is no longer limited to any particular sector of the economy: algorithmic management today is firmly entrenched in workplaces across the socio-economic spectrum. This implies, third and perhaps most importantly, that the rise of algorithmic management thus represents both a domestic as well as a transnational regulatory challenge: software developed in one jurisdiction can easily be used across different countries as increasing numbers of employers adopt existing ‘off-the-shelf’ solutions instead of developing their own, proprietary, software. It is to this question that discussion now turns.
High-risk Technology at Work: social, technical, and legal challenges
Annex III of the proposed AI Act is not the first time that the potential risks in deploying algorithmic systems in the workplace have been recognised at the Union level. In Opinion 2/2017 on data processing at work, 33 the European Union's (then) Article 29 Data Protection Working Party undertook a comprehensive review of many of the early technologies underpinning the rise of algorithmic management. On this basis, the working party issued a series of stark warnings. As rapidly falling cost and growing technical capabilities make algorithmic management ever less visible and more pervasive, 34 the Opinion suggests, emerging technologies might have a chilling effect on trade union organising, create increasing conformity through surveillance, invite incompatible further processing, and hamper anonymous whistle-blowing. 35 A few years on, increasing media reports suggest that these are not abstract dangers. Many of the challenges highlighted have already begun to play out in real life: 36 ‘sophisticated methods for collecting and analyzing employee data [ensure that] pinpointing of employees who are likely to unionize is a capability within reach for any employer’. 37
Abuses of the managerial prerogative are not a new phenomenon, of course – nor are they limited to technologically sophisticated workplaces. And yet, there are a number of factors which explain why the advent of automated management decisions raises particularly high-risk concerns. The systems involved differ from existing approaches in three fundamental ways: the collection and organisation of data, including both digital and real-world information, as well as its preparation in machine-readable formats; their processing, relying on machine learning and other ‘artificial intelligence’ techniques, which – broadly speaking – focus on identifying patterns and correlations in data sets; and the way control is exercised at a highly granular level, and sometimes even in the absence of clear orders.
Each of these features poses a series of particular challenges in the employment context, which can be clustered as follows: In terms of In terms of The granularity and intensity of
As I have argued in detail elsewhere, these key features of algorithmic management cut right across traditional legal assumptions of how to ascribe responsibility in domestic legal systems. 40 A detailed analysis of this point lies beyond the scope of the present paper; discussion in subsequent sections will focus instead on how elements of existing EU law could be harnessed to address each of the challenges identified in the application of algorithmic systems in high-risk contexts.
Regulating Algorithmic Systems at Work
In looking at relevant elements of the acquis, subsequent paragraphs identify potential avenues and current shortcomings in each of the three areas just highlighted. Beginning with the individual rights dimension, first, discussion will place particular emphasis on three relatively recent legislative instruments: the 2016 General Data Protection Regulation (‘GDPR’), 41 the 2019 Directive on Transparent and Predictable Working Conditions (‘TPWC’), 42 and the 2019 Regulation on promoting fairness and transparency for business users of online intermediation services (also known as the Platform to Business or ‘P2B Regulation’), 43 as well as the equal treatment acquis. Scrutiny then turns to the collective sphere and the role of the social partners in negotiating the introduction of algorithmic management practices.
The Individual Employment Dimension
Before turning to the specific discussion, an important caveat should be noted as regards the personal scope of these measures. The traditional approach of the social acquis has been to leave the determination of a measure's applicability to Member State law within boundaries set to varying degrees by different measures and the Court of Justice, 44 whereas the Article 29 Working Party has emphasised a very broad interpretation of employment, to ‘cover all situations where there is an employment relationship, regardless of whether this relationship is based on an employment contract’, and specifically including ‘new business models served by different types of labour relationships, and in particular employment on a freelance basis’. 45 Litigation under the GDPR can, in any event, be brought in certain circumstances without necessarily establishing employment status: the already-mentioned Dutch litigation, for example, thus managed to avoid otherwise heated (if, by now, consistently settled) preliminary debates as to gig economy platform workers’ legal classification. 46
The P2B Regulation explicitly defines its scope to exclude workers through an emphasis on the relationship between platforms and their ‘business users’. 47 It is nonetheless included in subsequent discussion, as substantive norms contained with the Regulation provide insight into contrasting regulatory approaches; discussion will return to the competing scope of different regulatory measures in the final section.
Data
In terms of data collection, first, the TPWC Directive appears to be silent on the point. With the rise of algorithmic management, however, it could be argued that data collection is increasingly becoming one of the ‘essential aspects of the employment relationship’ about which workers ought to be informed. 48 The P2B Regulation is more explicit: all terms and conditions are to be available in ‘plain and intelligible language’, ‘including in the pre-contractual stage’. 49 Articles 8 and 9 specifically address the question of data access, including crucially both information ‘users or consumers provide … or which are generated through the provision of [platform] services.’ 50
The strongest protection can be found in the GDPR. The ‘collection, recording, organisation, structuring, [and] storage’ of information are caught under the umbrella notion of ‘processing’: 51 all facets of the deployment of algorithmic management would therefore clearly fall into its scope. As a result, the introduction and operation of such systems will only be lawful where the employer can show a legal ground for doing so. 52 As the recitals make clear, however, consent, the primary lawful reason in many areas, ‘should not provide a valid legal ground for the processing of personal data in a specific case where there is a clear imbalance between the data subject and the controller’. 53 Recognising the specific features of the employment relationship, the WP29 Opinion therefore concludes that ‘[u]nless in exceptional situations, employers will have to rely on another legal ground than consent—such as the necessity to process the data for their legitimate interest. However, a legitimate interest in itself is not sufficient to override the rights and freedoms of employees.’ 54
Using algorithmic management software to optimise the day-to-day running of a business may, in principle, constitute an employer's legitimate interests.
55
In order to collect any required data, however, employers must first engage in a detailed proportionality assessment, considering whether:
the processing activity is necessary, and if so, the legal grounds that apply; the proposed processing of personal data is fair to the employees; the processing activity is proportionate to the concerns raised; and the processing activity is transparent.
56
Even where workforce information is ‘collected for specified, explicit and legitimate purposes’;
57
there are further safeguards in place for sensitive data, including data revealing ‘racial or ethnic origin, political opinions, religious or philosophical beliefs, or trade union membership.’
58
The WP29 Opinion illustrates the operation of these provisions in practice, explaining that ‘Only if it is necessary for the job to review information about a candidate on social media, for example in order to be able to assess specific risks regarding candidates for a specific function, and the candidates are correctly informed (for example, in the text of the job advert) the employer may have a legal basis … to review publicly-available information about candidates.’
59
The more intrusive the data collection, the less likely that a proportionate justification will be available to the employer: when recording keystrokes or mouse movements, ‘the processing involved in such technologies [is] disproportionate and the employer is very unlikely to have a legal ground under legitimate interest’. Bar a few ‘fringe exceptions’, facial recognition at work is ‘generally unlawful’. 60
The GDPR furthermore provides protection against ‘mission creep’ in algorithmic management, where data once collected could be stored and used for future, possibly as of yet unknown, applications: Article 5(1)(b) explicitly stipulates that data many not be ‘further processed in a manner that is incompatible with [the original] purposes.’
Processing
In terms of processing, the first challenge identified was the frequent changes leading to unpredictable working conditions which are inherent in the evolving nature of algorithmic control. Under the TPWC Directive, as long as the work pattern is at least mostly unpredictable (which it may be as a result of the very deployment of algorithmic management), the use of scheduling software should fall under the information provisions in Article 4(2)(m), with the worker entitled to information about reference hours, notice, and cancellation periods. The general predictability protection in Article 10 (including the right to refuse work assignments without adverse consequences and compensation in certain circumstances) is similarly limited to scheduling and work patterns. On the other hand, the TPWC Directive also provides that ‘any change in the aspects of the employment relationship referred to in Article 4(2) … shall be provided in the form of a document by the employer to the worker at the earliest opportunity and at the latest on the day on which it takes effect’. 61 These provisions, however, are less generous than the parallel protection in the P2B Regulation, which mandates a ‘notice period which is reasonable and proportionate to the nature and extent of the envisaged changes and to their consequences … [the] notice period shall be at least 15 days.’ The period may even be longer ‘if necessary to allow … technical or commercial adaptations’. 62
Several of the key principles in Article 5 GDPR could also be relied on to protect workers against constant and unnotified changes, notably the principle of transparency: 63 the WP29 Opinion explains how the workplace context mandates a high level of transparency and predictability. Where automated e-mail surveillance is in place, for example, ‘the rules that the system follows to characterize an e-mail as potential data breach should be fully transparent to the users, and in cases that the tool recognises an e-mail that is to be sent as a possible data breach, a warning message should inform the sender of the e-mail prior to the e- mail transmission, so as to give the sender the option to cancel this transmission.’ 64
The other major challenge clustered under this heading is algorithmic discrimination, arising in a number of different ways both from biased data sets and certain computational techniques: platform ratings, for example, might discriminate against non-traditional drivers, 65 whereas recruitment algorithms could disproportionately discount applications from particular sub-populations sharing protected characteristics. 66 Whilst present space limitations prohibit an in-depth engagement with this fast-growing area of research, 67 two important questions should be highlighted: first, whether existing anti-discrimination provisions in the employment sphere are in fact capable of combatting such discrimination. 68
Short of deliberately encoded biases, the discriminatory impact of an algorithm is usually assumed to fall within the scope of indirect discrimination. Algorithmic discrimination might thus, at least in principle, open the space for employer justification if the algorithm can be shown to be an effective and proportionate way of achieving a legitimate aim, such as the recruitment of the best candidates. 69 Technical intricacies, notably in the case of proxy discrimination, make it difficult to assess which party would succeed in the ensuing litigation, leaving some scholars to conclude that ‘victims of algorithmic discrimination are therefore in an essentially hopeless legal position’ under the EU acquis, 70 whereas others are more hopeful about a stringent proportionality enquiry before the Court of Justice. 71
As the potential for biased algorithms has become clearer, providers are increasingly investigating technical avenues to counteract bias. 72 This, in turn, raises a second major challenge for discrimination law: could the deployment of such an algorithm stray beyond the permitted boundaries of positive action, into the territory of positive discrimination? 73 The applicability of the Court of Justice's case law to this question is unclear: 74 if the algorithmic modification leads to ‘rules which guarantee [a protected group] absolute and unconditional priority’, 75 it will likely be ruled illegal unless ‘it provides … a guarantee that the candidatures will be the subject of an objective assessment which will take account of all criteria specific to the individual candidates and will override the priority accorded to [protected groups]’. 76 Once more, precise technical details will be key to the applicability of Union law: as Hacker concludes, whilst the ‘positive action doctrine of anti-discrimination law … clearly limits the scope of corrective powers’, it may ‘permit to a greater extent positive measures … before the actual selection stage of the decision procedure’. 77
Control
In terms of control, the first challenge identified was a lack of transparency and predictability, up to and including suspension and termination of the employment relationship. The TPWC Directive mandates that the worker be informed of ‘the procedure to be observed … including the formal requirements and the notice periods, where their employment relationship is terminated’. 78 It also introduces protections against retaliatory dismissals ‘and its equivalent’ – crucially defined to include the withholding of future work assignments. 79
Under the P2B regime, on the other hand, platforms have to ‘set out the grounds for decisions to suspend or terminate or impose any other kind of restriction upon … the provision of their online intermediation service’. They also need to provide (with certain exceptions, e.g. for violation of terms and conditions) a ‘statement of reasons’ for access restriction or suspension, with termination decisions furthermore subject to a minimum of 30 days’ notice. 80 The P2B Regulation furthermore provides for predictability in a crucial issue for platform workers – their ranking, which often determines access to the best or highest-paying jobs, 81 and which, with the rollout of algorithmic management to workplaces across the economic spectrum, will increasingly become an important factor in evaluating employees’ performance and job prospects more generally. Article 5(1) mandates that platforms set out ‘the main parameters determining ranking and the reasons for the relative importance of those main parameters as opposed to other parameters’. The information provided has to be ‘sufficient to enable … an adequate understanding’ of how the ranking mechanism takes various characteristics into account. 82
This approach could potentially overcome one of the key (arguable) weaknesses identified in the GDPR's provisions on information and transparency. 83 The literature is divided as to whether the GDPR gives data subjects a ‘right to explanation’ as to how a particular set of algorithms has arrived at a decision – and which would therefore quite dramatically reduce the range of solutions available. On a broad reading, the GDPR is said to mandate a ‘legibility test that data controllers should perform in order to comply with the duty to provide meaningful information about the logic involved in an automated decision-making;’ 84 whereas a narrower approach would provide workers (and other data subjects) with little more than a ‘right to be informed’, leaving them with a lack of ‘explicit and well-defined rights and safeguards against automated decision-making, and therefore runs the risk of being toothless.’ 85
Perhaps the strongest safeguard can be found in Article 22(1) GDPR, which grants workers the ‘right not to be subject to a decision based solely on automated processing … which produces legal effects concerning [them] or similarly significantly affects [them].’ As the WP29 Opinion makes clear, this provision is to be interpreted widely: it will not be sufficient, for example, merely to ‘fabricat[e] human involvement’; ‘if someone routinely applies automatically generated profiles to individuals without any actual influence on the result, this would still be a decision based solely on automated processing.’ 86 Reviewers need to be empowered to change the algorithmic decision, and indeed do so on a regular basis.
In the employment context, a broad interpretation will furthermore be required for legal or similarly significant effects: the termination of a contract of employment is clearly included, as is the automation of hiring processes: Recital 71 explicitly lists ‘e-recruiting practices without any human intervention’. The automation of workplace-internal processes, such as the use of scheduling software, is a more difficult issue. WP29 guidance suggests that for ‘data processing to significantly affect someone the effects of the processing must be sufficiently great or important to be worthy of attention.’
87
In other words, the decision must have the potential to:
significantly affect the circumstances, behaviour or choices of the individuals concerned; have a prolonged or permanent impact on the data subject; or at its most extreme, lead to the exclusion or discrimination of individuals.’
88
On that reading, it would appear that the deployment of automated scheduling software and other management tools would still fall within the Article 22 prohibition, as they will frequently involve ‘decisions that deny someone an employment opportunity or put them at a serious disadvantage’.
89
That said, Article 22 also provides for a number of exceptions, including necessity ‘for the performance of or entering into a contract’.
90
In the context of a very high volume of job applications for a particular position, this might permit the use of screening software to whittle numbers down to a shortlist – at least as long as there are no other effective or less intrusive methods available.
91
How, if at all, can the rights thus identified be enforced? Once more, there is considerable heterogeneity amongst the regimes under scrutiny. The GDPR provides for a range of remedial rights to data subjects (including access, rectification, erasure, processing restrictions, and portability), 92 though as discussed, their scope can be uncertain and continues to be the subject of extensive academic controversy, far beyond the employment context. Individual litigation, including the Dutch cases brought against Uber and Ola Cabs in 2021 discussed above, is beginning to shape the detailed application of relevant norms – including on the applicability of transparency provisions, and the ban on automated termination. 93
The TPWC Directive's Horizontal Provisions in Chapter IV provide a number of potential avenues, including a duty on Member States to ‘ensure that workers … have access to effective and impartial dispute resolution and a right to redress in the case of infringements of their rights arising from [the] Directive’ and protections against adverse treatment and victimisation. 94 The P2B Regulation, finally, includes a broader set of remedial tools, including detailed rules on internal complaint handling systems, ‘easily accessible and free of charge, … [ensuring] handling within a reasonable time frame’, as well as recourse to dedicated mediators. Crucially, the Regulation also provides for standing for ‘[o]rganisations and associations that have a legitimate interest in representing [involved parties], as well as public bodies set up in Member States’ in order to enforce its requirements. 95
‘Negotiating the Algorithm’: the collective dimension
In addition to the regulation of emerging algorithmic management practices through individual rights discussed thus far, there is also considerable scope for successful joint regulation through the social partners, coming together to ‘negotiate the algorithm’. 96 Given the fast pace of technological developments and considerable heterogeneity amongst different automated management products and services, the inherent flexibility and adaptability of social dialogue mechanisms might indeed be the most promising avenue, at least in the short term, permitting a high degree of reflexivity until the threats and opportunities involved become clearer. As De Stefano concludes, ‘even if it were possible to have automatic changes and updates in the operation of algorithms through self-learning artificial intelligence, the final decision to amend the criteria through which work performance is assessed should be taken by humans, made transparent and known to workers and also be subject to negotiation.’ 97
The extent to which such dialogue could take place within the frameworks provided by the Union acquis, however, is rather limited. The GDPR requires data controllers to conduct a Data Protection Impact Assessment (DPIA) ‘[w]here a type of processing in particular using new technologies, and taking into account the nature, scope, context and purposes of the processing, is likely to result in a high risk to the rights and freedoms of natural persons’ – as will frequently be the case in the context of introducing new algorithmic management tools.
98
There is space for worker representatives to be included in the DPIA process: Where appropriate, the controller shall seek the views of data subjects or their representatives on the intended processing, without prejudice to the protection of commercial or public interests or the security of processing operations.
99
A similar issue arises within the Information and Consultation Provisions in EU employment law: the introduction of algorithmic management should, in principle, be caught within the scope of Directive 2002/14, 100 as such a move can be characterised as a ‘decision likely to lead to substantial changes in work organisation’, thus requiring both information and consultation on point. 101 Given the very open-ended design of the Directive, its limitation to workers, and numerous carve-outs (for example, relating to enterprise size), even an explicit inclusion of algorithmic management would be unlikely to have a sustained impact across the Member States.
Lessons for Future Regulation
At first glance, then, EU law provides a number of promising avenues to help mitigate against the risks associated with algorithms at work. Whilst not contained in a single measure, the interplay of employment law, data protection, and anti-discrimination law has the potential to create a floor of rights across the Member States. At the same time, however, it is unlikely that existing regulatory structures will be able to ensure comprehensive worker protection in a world of fast-moving technological development. This is both due to specific shortcomings in individual measures or regulatory approaches as identified in the preceding section, and some overarching challenges which can provide important lessons for future regulatory efforts including the proposed AI Act.
As regards gaps between different legislative measures, first, the personal scope of different instruments needs to be carefully calibrated to avoid gaps which might leave vulnerable individuals without protection – or, even more problematic, create skewed incentives in certain areas of the labour market.
This is neatly illustrated in the context of platform-based work. There is a danger, for example, that certain gig economy workers might fall outside the scope of both the TPWC Directive and the P2B Regulation: platform workers who are employed appear to be excluded by the P2B Regulation's definition of ‘business users’ as ‘private individual[s] acting in a commercial or professional capacity’, 102 whereas those platform workers who are (genuinely) 103 self-employed will not be covered by the TPWC Directive. This opens up a potentially perilous gap, which could leave some of the most vulnerable platform workers without protection.
At the same time, the definition of ‘online intermediation services’ in Art 2(2)(a) of the P2B Regulation requires a platform to be an ‘information society service’ within the meaning of Directive 2015/1535 in order to come within the Regulation's scope. In a landmark ruling on the (materially identical) provisions of preceding legislation, 104 the Court of Justice ruled in December 2017 that the rideshare platform Uber was not a digital services provider for these purposes. The Grand Chamber examined the detailed algorithmic control exercised by Uber as a key factor in deciding that the platform's ‘intermediation service simultaneously offers urban transport services’ through its app. 105
A platform which exercises tight algorithmic employer control over its workforce, however legally styled, is therefore unlikely to fall within the scope of the P2B Regulation. As reaffirmed by a subsequent Grand Chamber ruling, in determining whether a service falls within the meaning of Directive 2015/1535, the Court needs to determine whether the digital ‘intermediation service forms an integral part of an overall service’. In answering that question, ‘the level of control’ exercised by a platform is crucial. 106 Elements of algorithmic control, including the setting of prices, are therefore amongst the key criteria in distinguishing platforms operating as ‘mere’ digital services intermediaries, and those intimately involved in the provision of the underlying service. As a result, however, some genuinely self-employed gig economy workers might find themselves beyond the scope of both regulatory measures – unable to qualify for worker protection under the TPWC Directive, whilst also ineligible for P2B protection given the nature of (labour) platforms’ business. This mismatch has the potential to create confusion at best, and potentially strong and counterproductive (mis-)classification incentives at worst.
A second important caveat should be entered as regards the interoperability of certain terms and related legal definitions. In the Union's regulatory and policy context, for example, there is considerable overlap in the designation of ‘platforms’, ranging from ‘content platforms’, such as Facebook and Youtube, and ‘online marketplaces’, such as Ebay, to labour or gig economy platforms of interest to the social acquis. A detailed definitional exercise is beyond the scope of the present discussion; one key distinguishing feature which could be explored to define labour platforms is the exercise of algorithmic employer control: responsibility for key employment-related functions, from hiring and firing workers through to setting pay and other terms of conditions, which therefore distinguishes labour or gig economy platforms from other business models such as online marketplaces.
A related concern specific to the judicial resolution of individual disputes is the question of transplanting interpretations of key terms across different regulatory domains. 107 If, for example, the Court of Justice were to adopt a narrow reading excluding a right of explanation from the right to be informed under Articles 13-15 GDPR, applying a similar standard to an employer's justification in indirect discrimination litigation would fundamentally alter the balance designed by the Union legislator. The overlapping use of key terms in different regulatory measures significantly increases such risks of inappropriate transplantation, particularly given the very different contexts in which litigation will arise.
The third point to note is a recognition of the tension between an omnibus approach to regulating new technological phenomena, and the very specific contexts of the world of work. Spiros Simitis has long warned of this tension between the ‘vagueness and ambivalence of an omnibus approach’ on the one hand, and ‘precise context-oriented demands’ on the other. 108 It was therefore reassuring that in its European Digital Strategy, 109 and the Commission's White Paper on Artificial Intelligence, the Union explicitly recognised the promise – and perils – of algorithmic management systems as identified in this article, suggesting that ‘the use of AI applications for recruitment processes as well as in situations impacting workers’ rights would always be considered “high-risk”.’ 110
Beyond this acknowledgment, and brief references to algorithmic discrimination as well as the role of the social partners amidst other stakeholders, however, there is scant acknowledgment of the specific challenges of the workplace. This stands in stark contrast with the White Paper's overarching economic emphasis on harnessing AI in the internal market, such as ensuring that small and medium-sized enterprises ‘can access and use AI’. 111 Given the powerful role that artificial intelligence is coming to play in the workplace, and the central role which work plays in most citizens’ lives, it is paramount that the particular challenges of algorithms at work are recognised and addressed in greater detail: algorithmic discrimination in hiring, for example, is but the tip of the iceberg. A successful regulatory strategy must address the implications of automated monitoring and control across the entire lifecycle of the employment relationship, from day-to-day management through to the termination of work.
In developing details of the Union's artificial intelligence policy, a particularly important site for future work is the intersection of equality law and data protection. As earlier discussion has shown, both legal regimes are of significant importance to the regulation of artificial intelligence at work; and yet their very distinct goals can make it difficult to find clear congruence once bias and discrimination become automated. The evolution of a successful artificial intelligence strategy should therefore focus further on the interoperability of the Union's anti-discrimination acquis and the GDPR in order to ensure that the regimes become mutually reinforcing and complementary in the workplace.
One important step in that direction could come from closer alignment with the Pillar of Social Rights. 112 Many of the key principles endorsed by the Pillar – from gender equality and equal opportunities (principles 2 and 3, respectively) through to social dialogue and health, safety, and a well-adapted work environment (principles 7 and 9) – are directly relevant to the challenges identified in this article. If the development of the digital single market is guided by these social principles, many of the potential conflicts between different areas of market regulation could therefore be successfully addressed at the design stage, thus avoiding subsequent clashes in litigation or policy debates.
* * *
The detailed interaction of different measures, definition of core terms, and recognition of context-specificity might not appear, at first glance, to be amongst the most pressing concerns when facing the major challenges of ‘creating EU global leadership’ in regulating AI policy and development. 113 The costs of getting it wrong, however, are significant – both in terms of hampering the operation of new proposed instruments, and interfering with the existing degree of protection provided by Union and domestic law. It is against this context that the proposed AI Act must be evaluated.
The Proposed AI Regulation
The AI Act proposal published in April 2021 envisages a multi-tiered approach to regulating artificial intelligence. 114 At a first level, a number of artificial intelligence practices such as public authority social scoring or real-time remote biometric identifications are prohibited altogether. 115 At a second level, the proposed Regulation identifies a series of ‘high-risk AI systems’ for which it sets out a detailed list of requirements, 116 whereas all other AI systems are subjected only to minimal transparency and disclosure obligations. 117
In line with the Commission's earlier White Paper, the recitals recognise the importance of including algorithms at work in the measure's scope: AI-systems used in employment, workers management and access to self-employment, notably for the recruitment and selection of persons, for making decisions on promotion and termination and for task allocation, monitoring or evaluation of persons in work-related contractual relationships, should also be classified as high-risk, since those systems may appreciably impact future career prospects and livelihoods of these persons. Throughout the recruitment process and in the evaluation, promotion, or retention of persons in work-related contractual relationships such systems may perpetuate historical patterns of discrimination, for example against women, certain age groups, persons with disabilities, or persons of certain racial or ethnic origins or sexual orientation. AI systems used to monitor the performance and behaviour of these persons may also impact their rights to data protection and privacy.
118
This recognition is translated into the inclusion of employment as a high-risk context in Annex III, as set out in the introductory section.
Once a system is within the scope of Title III, a series of detailed obligations apply to ‘providers’ (those who develop AI systems or have them developed with a view to placing it on the internal market), 119 ‘users’ (those deploying the AI system), 120 and other related parties. The ensuing obligations are modelled on EU product regulation for devices ranging from forestry vehicles to medical implants, also known as the New Legislative Framework (NLF). As Veale and Borgesius explain, ‘[u]nder NLF regimes, a manufacturer must undertake pre-marketing controls undertaken to establish products’ safety and performance, through conformity assessment to certain essential requirements laid out in law. Manufacturers then mark conforming products with “CE”; marked products enjoy EU freedom of movement.’ 121 Chapter 2 of the proposed AI Act sets out a comprehensive list of requirements for high-risk AI systems, from the need to establish risk management and data governance systems, 122 documentation and record-keeping obligations, 123 to transparency and human oversight rules. 124
Upon closer inspection, however, these standards are unlikely to be applied in any particularly stringent way. First, because compliance certification through conformity assessment is left – for the majority of high-risk systems, including those deployed in the employment context – to providers (i.e. developers) themselves. 125 Second, because the very standards listed in Chapter 2 might never play a role in practice, as private standardisation organisations are likely to ‘adopt a standard relating to the Draft AI Act, [which] providers can follow …, rather than interpreting the essential requirements. If following the standard, providers enjoy a presumption of conformity.’ 126
As a result of these and other shortcoming beyond the scope of the present discussion, the draft AI Act has been subject to trenchant criticism, 127 including from the perspective of labour law. 128 Indeed, the Act as proposed is unlikely to be able to tackle the very risks identified in its own recitals – not least, because it falls into several of the traps identified in the preceding analysis of existing regulatory approaches. The proposal does not address the gaps identified between different regulatory regimes such as data protection and discrimination law or clarify important terminology; indeed, it may well compound the problem. The adoption of the NLF regime, originally designed for physical goods which may cause tangible harm, is hardly appropriate for a digital technology which, on the Commission's own account, may pause significant risk to the protection of non-tangible values, including in particular fundamental rights. An additional, and fundamentally distinct, layer of regulation is added without necessarily changing much, if anything, in substance: indeed, some of the most promising provisions (such as the ban on certain forms of facial recognition) are, if anything, less stringent than what can already be found in the GDPR. 129
More fundamentally, the proposed AI Act shows the real limits of an omnibus regulatory approach in the employment context, with significant shortcomings in both the individual and collective dimension. As regards the former, for example, whilst many of the parties in the AI development ecosystem are identified and defined, the eventual subjects of algorithmic decision-making are left without any specific recourse. There is similarly no space for collective representation or social partnership: given the proposed Treaty basis in Article 114 TFEU, the usual, extensive legislative consultation process is not applicable, 130 nor do the substantive proposals envisage any role for social dialogue. This is a missed opportunity: even within the NLF framework, social partners could have been assigned important roles, both in terms of standard setting and compliance assessment and certification. This would allow Union-level development of algorithmic management systems with the benefit of worker and management representatives’ on-the-ground experience.
The proposed Act, finally, does not contain any equivalent provision to Article 88 GDPR, which would permit Member States to lay down more specific domestic provisions for the employment context, thus further inhibiting domestic regulatory experimentation.
Conclusion: Towards a ‘European approach to AI’?
As political debates over the last few months have made clear, the proposed AI Act is likely to undergo significant debates and amendments in the months, if not years, ahead. The debate about the future ‘European Approach to AI’ is therefore far from over. As the foregoing discussion of the promise and perils of algorithmic management has shown, significant amendments and further regulatory effort will be required in order to achieve the Commission's vision of ‘AI [that] works for people and is a force for good in society’, whilst also promoting AI research, innovation, and adoption. 131
In that sense, broader debates about technology regulation are delving into a potential tension between economic and social incentives long debated in EU (employment) law. But ‘measures in support of innovation’ need not necessarily be only deregulatory, such as the ‘regulatory sandboxes’ envisaged in the AI Act. 132 Coherent standards and reliable enforcement are just as important. As the Commission's 2020 communication on ‘Shaping Europe's Digital Future’ emphasised, ‘[i]n the digital age, ensuring a level playing field for businesses, big and small, is more important than ever. This suggests that rules applying offline – from competition and single market rules [… to] workers’ rights – should also apply online.’ 133
The proposed AI Act is but the latest step in a series of developments which have brought labour regulation at the Union (as well as domestic) level increasingly into close contact, and potentially even enmeshment, with broader regulatory questions in the Digital Single Market. The recognition of a need for joined-up regulation is to be welcomed. However, as the Union moves towards the creation of a Digital Single Market, it will be just as important to carve out a clear and protected space for employee-protective measures, both at the national and European level, whether through specific instruments or a recognition of the specificities of the employment context in omnibus measures.
The tension between economic and social interests in the European Union is difficult to resolve, not least for a host of constitutional reasons. Extensive discussions in the aftermath of the Court of Justice's decisions in Viking Lines and Laval and the Commission's Monti II legislative proposal bear important lessons as the fundamental economic freedoms move into the digital realm. 134 Creating structures which encourage digital innovation and competitiveness does not have to come at the cost of labour protection. The empirical evidence is clear: in some of the world's most innovative jurisdictions, strong worker-protective standards have fuelled economic and technological development. 135 A renewed emphasis on social rights will be key in achieving the Commission's vision of excellence and trust in a distinctly European approach to artificial intelligence, at work and beyond.
Footnotes
Funding
I am grateful to the European Commission and the European Research Council for funding under the European Union's Horizon 2020 research and innovation programme (grant agreement No 947806), and to Reuben Binns, Valerio De Stefano, Aislinn Kelly-Lyth, Michael Veale, Bernd Waas, and colleagues at the European Commission and the European Labour Law Network for feedback and discussion. The usual disclaimers apply.
1.
European Commission, Communication from the Commission to the European Parliament, The Council, The European Economic and Social Committee and the Committee of the Regions: Fostering a European Approach to Artificial Intelligence (COM(2021) 205 final.
2.
European Commission, Proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain Union legislative acts (COM(2021) 206 final (‘AI Act Proposal’).
3.
AI Act Proposal, Art 3(1) and Annex I (a)-(c).
4.
AI Act Proposal, Art 6(2) and Annex III.
5.
AI Act Proposal, Annex III 4(a)(b).
6.
For a detailed overview, see M Veale and F Zuiderveen Borgesius, ‘Demystifying the Draft EU Artificial Intelligence Act’ (2021) 22 Computer Law Review International (forthcoming).
7.
Space limitations prohibit a detailed discussion of further rules at the domestic level, such as Member State legislation on co-determination, though such norms will reflect many of the challenges raised in the concluding section.
8.
Though cf, for example, a special issue on ‘Automation, Artificial Intelligence, & Labor Law’ of the Comparative Labor Law & Policy Journal (2019) 41(3).
9.
D Autor, ‘Wiring the Labor Market’ (2001) 15 Journal of Economic Perspectives 25.
10.
C Estlund, ‘What Should We Do After Work? Automation and Employment Law’ (2018) 128 Yale Law Journal 254.
11.
For detailed discussion, see J Adams-Prassl, ‘What if your Boss was an Algorithm? Economic Incentives, Legal Challenges, and the Rise of Artificial Intelligence at Work’ (2019) 41 Comparative Labor Law and Policy Journal 123, especially sections 2&3.
12.
A Adams, ‘Technology and the Labour Market’ (2018) 3 Oxford Review of Economic Policy 349.
13.
For a full definition of these employer functions, see J Prassl, The Concept of the Employer (OUP 2015) 31-36.
14.
J Prassl, Humans as a Service: The Promise and Perils of Work in the Gig Economy (OUP 2018) chapter 3.
15.
T Slee, What's Yours Is Mine (O/R Books, New York 2015) 100-1.
16.
17.
Indirect control through nudging or gamification may well be more effective than direct orders: ‘To put the matter sharply, A may exercise power over B by getting him to do what he does not want to do, but he also exercises power over him by influencing, shaping or determining his very wants. Indeed, is it not the supreme exercise of power to get…others to have the desires you want them to have—that is, to secure their compliance by controlling their thoughts and desires?’ S Lukes, Power: A Radical View (OUP 2005).
19.
A Beinstein and T Sumers, ‘How Uber Engineering Increases Safe Driving With Telematics’ Uber Engineering, 29 June 2016: https://eng.uber.com/telematics/, archived at 
20.
A Rosenblat and L Stark, ‘Algorithmic Labor and Information Asymmetries: A Case Study of Uber's Drivers’ (2016) 10 International Journal of Communication 3758, 3775.
21.
J Jacobson and Gruzd, ‘Cybervetting job applicants on social media: the new normal?’ (2020) 22 Ethics and Information Technology 175.
22.
23.
A Mateescu and A Nguyen, Algorithmic Management in the Workplace (Data&Society, New York 2019) 9-10.
24.
Alexandra Mateescu and Aiha Nguyen, Algorithmic Management in the Workplace (Data&Society, New York 2019) 9-10.
25.
Ibid.
27.
A Ajunwa, K Crawford and J Schultz, ‘Limitless Worker Surveillance’ (2017) 105 California Law Review 735, 735.
28.
29.
30.
32.
33.
https://ec.europa.eu/newsroom/article29/item-detail.cfm?item_id=610169 (‘WP29 Work Opinion’). Today, the Article 29 Working Party has been replaced by the European Data Protection Board, which has ‘acknowledge[d] the continuity of the work provided by the predecessor Article 29 Working Party’ (Endorsement 1/2018): see
34.
Ibid 3-4.
35.
Ibid 9-10.
36.
37.
38.
For an accessible introduction, see e.g. N Polson and J Scott, AIQ: How Artificial Intelligence Works and How We Can Harness its Power for a Better World (Bantam Press, 2018).
40.
Adams-Prassl, ‘What if your Boss was an Algorithm? Economic Incentives, Legal Challenges, and the Rise of Artificial Intelligence at Work’ (n 11).
41.
Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation), OJ L 119/1, 4.5.2016 (‘GDPR’). See further Marta Otto, 'Workforce Analytics v Fundamental Rights Protection in the EU in the Age of Big Data' (2019) 40 Comparative Labor Law and Policy Journal 389.
42.
Directive (EU) 2019/1152 of the European Parliament and of the Council of 20 June 2019 on transparent and predictable working conditions in the European Union, OJ L 186/105, 11.7.2019 (‘TPWC’). For a detailed analysis, see B Bednarowicz, Delivering on the European Pillar of Social Rights: The New Directive on Transparent and Predictable Working Conditions in the European Union (2019) 48 Industrial Law Journal 604.
43.
Regulation (EU) 2019/1150 of the European Parliament and of the Council of 20 June 2019 on promoting fairness and transparency for business users of online intermediation services, OJ L186/57, 11.07.2019 (‘P2B Regulation’).
44.
N Countouris, ‘The Concept of “Worker” in European Labour Law: Fragmentation, Autonomy and Scope’ (2018) 41 Industrial Law Journal 192.
45.
WP29 Work Opinion, 4.
46.
Ekker Legal (n 29).
47.
P2B Regulation, Art 1.
48.
TPWC Directive, Art 4(1). The list provided in Art 4(2) is non-exhaustive.
49.
P2B Regulation, Art 3(1)(a) and (b).
50.
P2B Regulation, Art 9(1).
51.
GDPR, Art 4(2).
52.
GDPR, Art 6(1).
53.
GDPR, Recital 43.
54.
WP29 Work Opinion, 4.
55.
GDPR, Art 6(1)(f).
56.
WP29 Work Opinion, 10-11.
57.
GDPR, Art 5(1)(b).
58.
GDPR, Art 9(1).
59.
WP29 Work Opinion, 11.
60.
Ibid 16, 19.
61.
TPWC Directive, Art 6(1).
62.
P2B Regulation, Art 3(2).
63.
GDPR, Art 5(1)(a).
64.
WP29 Work Opinion, 15.
65.
M Kullmann, ‘Platform Work, Algorithmic Decision-Making, and EU Gender Equality Law’ international (2018) 34 Journal of Comparative Labour Law and Industrial Relations 1.
66.
For an early overview of the technical challenges, see Solon Barocas and Andrew D. Selbst, ‘Big Data's Disparate Impact’ (2016) 104 California Law Review 671.
67.
For a full overview, see A Kelly-Lyth, ‘Challenging Biased Hiring Algorithms’ (2021) 41 Oxford Journal of Legal Studies 899.
68.
Directive 2000/43/EC, implementing the principle of equal treatment between persons irrespective of racial or ethnic origin OJ L 180/22; Directive 2000/78/EC of 27 November 2000 establishing a general framework for equal treatment in employment and occupation OJ L 303/16; Directive 2006/54 on the implementation of the principle of equal opportunities and equal treatment of men and women in matters of employment and occupation, OJ L 204/23.
69.
See e.g. Directive 2000/43/EC Art 2(2)(b) (n 68), and corresponding provisions.
70.
P Hacker, ‘Teaching Fairness to Artificial Intelligence: Existing and Novel Strategies Against Algorithmic Discrimination under EU Law’ (2018) 55 Common Market Law Review 1143.
71.
Raphaele Xenidis & Linda Seden, ‘EU Non-discrimination Law in the Era of Artificial Intelligence: Mapping the Challenges of Algorithmic Discrimination’ in Ulf Bernitz, Xavier Groussot, Jaan Paju, and Sybe A. de Vries (eds), General Principles of EU Law and the EU Digital Order (Wolters Kluwer 2020).
73.
See e.g. Directive 2000/43/EC Art 6 (n 43), and corresponding provisions.
74.
The leading discussion on point to date can be found in Hacker (n 70), 1180-1181.
75.
C-450/93 Kalanke EU:C:1995:322 at [22].
76.
C-409/95 Marschall EU:C:1997:533 at [33].
77.
Hacker (n 70) 1181.
78.
TPWC Directive, Art 4(2)(j).
79.
TPWC Directive, Art 18(1), read in conjunction with Recital 43.
80.
P2B Regulation, Arts 3(1)(c), 4.
81.
J Prassl, Humans as a Service (n 14) 61-2.
82.
P2B Regulation, Art 5(5).
83.
GDPR, Art 13-15.
84.
G Malgieri and G Comandé, ‘Why a Right to Legibility of Automated Decision-Making Exists in the General Data Protection Regulation’ (2017) 7 International Data Privacy Law 243, 243.
85.
S Wachter, B Mittelstadt, L Floridi, ‘Why a Right to Legibility of Automated Decision-Making Does Not Exist in the GDPR’ (2017) 7 International Data Privacy Law 1, 1.
86.
Article 29 Working Party, Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679 (17/EN WP251rev.01) (‘WP29 Automated Opinion’) 21.
87.
Ibid 21.
88.
Ibid 21.
89.
Ibid 22.
90.
GDPR, Art 22(2)(a).
91.
WP29 Automated Opinion, 23.
92.
Arts 15, 16, 17, 18, and 20 GDPR, respectively.
93.
Ekker Legal (n 29).
94.
TPWC Directive, Arts 16 – 18.
95.
P2B Regulation, Arts 11(1), 12, 13, 14.
96.
V De Stefano, ‘Negotiating the algorithm’: Automation, artificial intelligence and labour protection’ (2019) 41 Comparative Labor Law and Policy Journal 15.
97.
Ibid 31.
98.
There is space in principle for worker representatives to be included in the DPIA process: Art 35(9) GDPR. See further WP29 working party, ‘Guidelines on Data Protection Impact Assessment (DPIA) and determining whether processing is "likely to result in a high risk" for the purposes of Regulation 2016/679 (wp248rev.01), 19, 13; WP29 Work Opinion, 8.
99.
GDPR, Art 35(9)
100.
Directive 2002/14/EC of the European Parliament and of the Council of 11 March 2002 establishing a general framework for informing and consulting employees in the EC OJ L 80/29, 23.03.2002.
101.
Ibid, Article 4(2)(c).
102.
P2B Regulation, Art 2(1).
103.
Case C-413/13 FNV Kunsten ECLI:EU:C:2014:2411.
104.
Directive 98/34/EC of the European Parliament and of the Council of laying down a procedure for the provision of information in the field of technical standards and regulations and of rules on Information Society services, OJ L 204/37, 22 June 1998 (as amended).
105.
Case C-434/15 Asóciacion Profesional Elite Taxi v Uber Systems Spain SL ECLI:EU:C:2017:981 [34-36], [37], [38].
106.
Case C-390/18 Airbnb Ireland ECLI:EU:C:2019:1112 [67], [66].
107.
A long-standing challenge for comparative labour law in particular: O Kahn-Freund, ‘On Uses and Misuses of Comparative Law’ (1974) 37 Modern Law Review 120.
108.
Spiros Simitis, ‘Reconsidering the Premises of Labour Law: Prolegomena to an EU Regulation on the Protection of Employees’ Personal Data’ (1999) 5(1) European Law Journal 45, 53.
110.
European Commission, White Paper on Artificial Intelligence - A European approach to excellence and trust (Brussels, 19.2.2020), COM(2020) 65 final, 17-18.
111.
Ibid 6, 7, 10, 11, 13.
113.
European Commission, Fostering a European Approach to Artificial Intelligence (n 1) 7-8.
114.
For a detailed overview, see Veale and Borgesius (n 6).
115.
AI Act Proposal, Title II. Note, however, the potentially extensive carve-outs in Article 5.
116.
Ibid, Title III, chapter 1 (classification) and chapter 2 (requirements).
117.
Ibid, Title IV.
118.
Ibid, recital 36.
119.
Ibid, Art 3(2).
120.
Ibid, Art 3(4).
121.
Veale and Borgesius (n 6) 10ff.
122.
AI Act Proposal, Arts 9 and 10.
123.
Ibid, Arts 11 and 12.
124.
Ibid, Arts 13 and 14.
125.
Ibid, Art 19, Art 43(2).
126.
Veale and Borgesius (n 6) 14.
127.
Ibid 26-7.
128.
129.
See text surrounding note 60.
130.
Article 154-155 TFEU.
131.
European Commission, Fostering a European Approach to Artificial Intelligence (n 1) 8.
132.
AI Act Proposal, Title V, Art 53.
133.
European Commission, Communication from the Commission to the European Parliament, the Council, the European economic and social Committee and the Committee of the regions: Shaping Europe's digital future (Brussels, 19.2.2020) COM(2020) 67 final.
134.
Cf M Freedland and J Prassl, EU Law in the Member States: Viking, Laval and Beyond (Hart 2014), including in particular the contributions by A Bogg and S Weatherill.
135.
See e.g. S Deakin, ‘Shares for rights’, FT Economists Forum (12 February 2013).