Abstract
To avert the twin threats of isolation and marginalization, we argue that European legal research should embrace the methodology of the social sciences to a much greater extent than is currently the case. To fit the hybrid – academic and professional – character of the law school, research should emphasize questions of broad interest to lawyers and legal reformers. We outline two lines of research, under the header of ‘Empirical Jurisprudence’, that, we believe, should be of fundamental interest to members of the legal community at large: (i) law as the art of persuasion; and (ii) law as social product and instrument of social planning. We show that the questions demarcated by these two research programmes are, and have always been, of interest to lawyers, claims to the autonomy of the legal discipline notwithstanding. We also argue that the rapidly expanding and increasingly eclectic array of empirical research techniques – from text mining to network analysis and machine learning – makes the turn to Empirical Jurisprudence especially promising.
1. Introduction: The challenges facing European legal research
The European legal academy is at a crossroad. It faces challenges on multiple fronts, forcing it to reconsider its teaching and research practices. First, the restructuration of the legal services industry, precipitated by globalization, new technologies and changing demands, is affecting the job prospects of law graduates. 1 Not surprisingly, this has led commentators to ask pointed questions about the value and pertinence of the approach to legal education that continues to dominate the law school curriculum throughout the continent. 2 Second, the research output of law schools is being questioned. 3 Legal academics, at least in non-Anglophone countries, are less likely to publish in English and less likely to do so in peer-reviewed journals than scholars in most other disciplines. The cause of legal scholars is not helped by the fact that their discipline appears to lack an explicit methodology. 4 At a time when the panels of research funding agencies are increasingly turning interdisciplinary, 5 legal academics seem to lack criteria to evaluate their own research. Replicability, hypothesis testing, external validity, sample representativeness – the criteria commonly used to evaluate the quality of scientific research – are largely foreign to the scholarship they produce. 6
To many a lawyer, comparisons with other disciplines, when done along such lines, come across as profoundly unfair. This is not only because the argument seems to overlook the existence of a burgeoning European empirical legal literature 7 or, in some areas, the increasing influence of political philosophy as a source of interdisciplinary insights. 8 Rather, what the comparison seems to miss is a crucial function of legal scholarship, which is not purely academic in character. While officially part of academia, law schools also serve a vocational function. Training future legal professionals is part of their mission statement. They are embedded in the community formed by professional jurists and this proximity to the world of legal practice is reflected in the large number of legal practitioners found among their instructors. In consequence, it is not absurd to view legal scholarship as being primarily there to serve the practical needs of legal practitioners, of judges adjudicating real-world disputes and attorneys advising demanding clients rather than for the amusement of an insular coterie of academics. 9 A practice-oriented, no-nonsense lawyer may rightly point out that these are not audiences likely to have much appreciation, let alone time, for arcane considerations such as replicability and falsifiability – not to mention the interpretation of regression coefficients. So, to require that legal scholars follow the same scientific practices as researchers in other disciplines would risk disconnecting their work from the community of legal practitioners in which law schools, by their very nature, are embedded. Instead, doctrinal analysis, as it has been practised for almost two centuries, remains the most relevant approach to what members of the legal community do and need.
This objection, we believe, raises a fair point and should not be brushed aside as the product of a philistine world view. Indeed, a strong case can be made that doctrinal legal scholarship, even when methodologically uninformed or epistemologically naïve, 10 fulfils an important function that those who criticize its conventional modus operandi do sometimes overlook. 11 The existence, pre-eminence and focus of doctrinal legal scholarship in continental European law schools say something important about how the legal academy understands itself and its relation to legal practice. For that reason, our point is not to argue that law schools should stop producing doctrinal work altogether. However, acknowledging the relevance of doctrinal scholarship does not remove the problem confronting European legal research. In essence, the problem is one of impact and prestige – but it is also one of money. Whether we like it or not, there is no research without funding. Yet the evidence suggests that law schools and law departments attract less third-party funding than other university departments. 12 We seriously doubt that, in the age of machine learning and artificial intelligence, legal scholars can improve their scientific credibility vis-a-vis researchers in other disciplines – who are increasingly called on to evaluate their research output – if they only produce doctrinal work. Moreover, even if one accepts the necessity of maintaining close bonds with legal practice, a big chunk of law schools’ output sits uncomfortably between academic research and legal practice. Consider the doctoral dissertation in law, of which law schools around Europe churn out thousands every year. Does it suit the needs of legal practitioners? On both counts, we feel inclined to say no. Start with practitioners. Practitioners typically lack time. For them, concision is of the essence – as is attested by the terse contributions and brief case notes that tend to populate the pages of the professional periodicals they actually read. 13 PhD theses going over several hundred pages, therefore, are unlikely to rank high on the readings list of busy judges and in-house counsels. Is it because a PhD in law is primarily meant to be an academic enterprise? If that is so, we must admit that doctoral dissertations in law rarely exhibit the level of methodological rigour and know-how commonly associated with the scientific method and, in many disciplines, with PhD training. A systematic evaluation of a sample of 90 doctoral dissertations defended at Dutch law schools has found that jurists justify their methods ‘largely implicitly, marginally in an explicit manner, or even not at all’. 14 The upshot is that law schools produce dissertations that tend to be low on both practical and academic impact. This is not without consequences for the skill set that a doctorate in law signals to potential employers. ‘What is the added value of a PhD in law in terms of research skills and know-how?’ is a question to which the answer should be straightforward. It is not – at least not in the positive. More worrying still, a growing share of the cutting-edge research on ‘legal’ institutions, which used to be the preserve of law professors, is now conducted by people from outside the legal discipline. 15 Economists, for example, have meticulously analysed how changes in tax and sports regulations (including the CJEU Bosman ruling) have affected the career and mobility of top-flight football players. 16 They have curated and crunched terabytes of data on judges. 17 Political scientists have analysed the decision-making process of international adjudicatory bodies 18 and asylum courts, 19 examined the influence of legal counsels on judicial outcomes 20 and mapped two centuries of constitution writing around the world. 21 Meanwhile, psychologists have started to investigate the quirks of the judicial mind. 22 The risk is thus real that researchers in other disciplines be seen as outclassing their legal colleagues in areas where such competition used to be absent.
In this article, we argue that, in order to avert the twin threats of isolation and marginalization, European legal research should embrace the methodology of the social sciences to a much greater extent than is currently the case. 23 In doing so, however, it should seek to maintain practical and educational relevance by focusing on questions that are important and interesting to lawyers, law students and legal reformers. 24 To be clear, we have nothing against blue-skies research. Yet, legal scholarship cannot simply become sociology-plus or political science-plus, if only because legal research cannot easily ignore the hybrid nature – academic and professional – of the European law school. In that regard, the discussion on the future of legal research relates to the debate on the evolution and transformation of the legal profession and their implications for legal education reform. 25 While legal researchers should have the opportunity to pursue whatever questions they find interesting, we should also be realistic that they will not easily avoid the ‘what is the point?’ question. Being part of a practitioners’ community, they will be expected to explain how their work bears on the demands of legal practice and the training of future lawyers. The present paper delineates two broad research programmes that have been, are and – we are convinced – will continue to be of fundamental interest to members of the legal community at large: (i) law as the art of persuasion; and (ii) law as social product and instrument of social planning. Contrary to claims asserting the autonomy of the legal discipline, we contend that the questions demarcated by these two lines of inquiry are, and have always been, of interest to lawyers. The cognitive interests of lawyers have, in that sense, always been interdisciplinary. Only the methodology of European legal scholarship did not reflect this eclecticism. We also argue that the rapidly expanding and increasingly eclectic array of empirical research techniques now available to legal researchers makes the turn to ‘Empirical Jurisprudence’ – as we label this approach to legal research – especially attractive and promising. Text mining, web scraping, network analysis and machine learning methods promise to greatly enrich and augment the possibilities of legal research. To be sure, some European legal scholars have already embraced the empirical turn and, as we shall point out, studies using some of these techniques have begun to appear in European law reviews. But we contend that empirical methods should and can be broadly embraced by the European legal academy. Law schools should promote empirical jurisprudential research and make it a central component of their research agenda. We also believe that Empirical Jurisprudence is a more promising research programme than Political or Critical Jurisprudence. Continuing a long tradition in legal philosophy, EU and international law studies have increasingly incorporated insights and ideas from political philosophy in recent years. 26 Our argument does not intend to diminish the merits of this literature. Not only do we see familiarity with political theory as eminently relevant to thinking about law and legal phenomena, but we also regard drawing out the normative implications of empirical findings as eminently desirable and as good practice. 27 Yet we doubt that any injection of political theory will suffice to tackle legal scholarship’s methodological crisis. For one thing, unless it is conversant with an empirical research programme, any theorizing enterprise is bound to wither over time. Without empirical research to raise new questions and problems, theory can rapidly become mired in narrow, sterile controversies, as classical jurisprudence arguably has. 28 For another, without the disciplining effect of a constant confrontation with empirical facts, normative thought is vulnerable to utopian impulses. As Amartya Sen stressed in his critique of Rawls, political theory, too, must be realistic. 29
Our argument is structured as follows. We first delineate the two lines of inquiry encompassed by Empirical Jurisprudence. In so doing, we take pains to relate these research programmes to existing strands of research in social science and legal thought as well as to the questions and challenges that legal professionals face in their daily practice. A fundamental premise of our argument is that while empirical jurisprudential scholarship needs to be sufficiently detached from practical legal discourse in order to avoid being overly influenced by the interests and agendas which shape that discourse, it must nonetheless bear some relevance to what judges and legal counsels do and law students prepare for. The research programme we sketch out meets these two criteria: while social-scientific in its methodology, it is still ‘legal’ in its substantive focus. Next, we illustrate the potential of new research methods for legal analysis with a set of examples. We then take issue with some of the objections that this research agenda is likely to elicit, starting with the claim that law is, or ought to be, an autonomous discipline. As we shall see, we do not deny that it is possible to articulate a coherent research programme separating law from other disciplines. However, we argue that, coherent as it may be, such a project is fundamentally uninteresting to lawyers; practitioners and legal scholars do, in fact, implicitly understand law in interdisciplinary terms. So, far from confining the study of law to a new academic ivory tower, Empirical Jurisprudence actually promises to reconcile the academic and professional dimensions of legal education and research. We conclude with some thoughts on the institutional incentives and practical obstacles to the adoption of this research agenda.
2. Empirical Jurisprudence: A brief outline
Empirical Jurisprudence seeks to leverage the full panoply of social-scientific research methods to answer questions that are generally relevant to judges, legal counsels, trial lawyers, litigants, compliance officers and legal reformers. 30 What motivates our choosing the label ‘Empirical Jurisprudence’ is the desire to link theorizing about law, something lawyers associate with classical jurisprudence, to an empirical research programme, following the model of other scientific disciplines. 31 Whether in economics, sociology, political science, physics or biology, theory is always conversant with empirical observation and experimentation. Theories generate hypotheses that can then be tested against empirical data, while empirical analysis generates new questions for theorists to answer. Adopting this scientific model, we believe, will benefit legal research.
Empirical Jurisprudence comprises two strands of research. One relates to the rhetorical dimension of lawyering and approaches law as a specialized form of political communication. The other is concerned with law and legal rules as instruments of social planning and sets out to explain how legal rules emerge and how they then modulate patterns of social conduct.
A. Law as the art of persuasion
The first strand of research deals with law as the art of persuasion. It starts out from the observation that rhetoric and advocacy are central to the law in action. Judges, legal counsels and litigators deal in persuasion and use legal discourse to influence the beliefs and actions of their audiences. 32 The great legal advocates are, like Cicero, verbally agile men and women who excel at persuasion. In that sense, learning to think, speak and write like a lawyer is in no small measure about acquiring and honing the ability to construct and deconstruct arguments on the spot and to deliver them with eloquence, as a counsel is often required to do in the courtroom or in her briefs. At the risk of drawing a controversial comparison, an instructive parallel here is with the work of lobbyists, PR advisers and politicians. 33 As with lobbyists, trial lawyers use argumentation to try and persuade a public decision maker – in that case a judge – to make a decision favouring the interests of the individuals, groups or corporations they represent. 34 Judges, too, are, in their own way, advocates. Not unlike elected politicians, judges strive to present their decisions in the best light so as to avert criticism while maximizing social acceptance. The art of writing effective party manifestos or government press releases and the art of penning persuasive judicial opinions thus share important similarities. To the extent that both are concerned with the optimal way to communicate a piece of information to a given audience, the task of a judicial opinion writer is not very different from that of a PR consultant drafting a campaign manifesto. A similar analogy can be drawn between lobbying and doctrinal legal scholarship. As noted above, doctrinal legal scholarship ordinarily takes a perspective that is continuous with, rather than detached from, the discourse of legal practitioners. Being itself part of that discourse, it does not only observe but also takes side – just as judges and attorneys do – in normative disputes. The typical law review article seldom consists of a mere description of what the law is. Instead, it is usually infused with value judgments and penned with a view to advance a certain vision of how the law ought to develop. In fact, much of doctrinal legal scholarship can be viewed as an attempt either to persuade the legal community to accept the policy determinations of judges or to persuade judges to change these policy determinations. 35
To be sure, there are not only commonalities but also important differences between advocacy in the legal sphere and advocacy in the political sphere. A not insignificant one is the role authorities – that is, legislative language, judicial precedents, and so on – characteristically play in legal discourse. Public expectations regarding what constitutes acceptable judicial conduct also impose specific constraints on legal discourse which jurists cannot easily ignore. These commonalities and differences suggest that the most fruitful way for legal researchers to study legal rhetoric is as a specialized form of political communication. Does that sound incongruous? It should not, because this approach does, in fact, go back a long tradition, starting with the Sophists of Ancient Greece, who – before Plato gave them a bad name – acted as the lawyers, spin doctors and PR gurus of Athenian society. It cuts through the work of Aristotle, 36 Cicero, 37 Perelman 38 and Toulmin 39 and finds an echo in the Law as Literature movement. Like us, Aristotle saw many parallels between legal and political rhetoric. 40 These parallels invite legal researchers to borrow from the stock of concepts and theories accumulated by modern research on political communication. Whether it is cherry-picking facts and precedents 41 or using positive framing (presenting the law ‘in its best light’ 42 ), 43 legal discourse – most notably, though not only, in the courtroom race for persuasion – tends to resort to the same rhetorical tactics as those familiar to political communicators. 44 Like Aristotle too, we see the science of the mind, namely psychology, as central to the study of persuasion in the legal context. 45 Research in cognitive psychology has shown how skilled rhetoricians can exploit quirks in the wiring of the human brain to achieve persuasion. For example, storytelling – that is, presenting facts and legal materials in the form of a coherent narrative – is an effective discursive technique that can take advantage of the human brain’s fondness for cognitive fluency and promptness to jump to conclusions. 46
Legal researchers should aim to map the rhetorical strategies deployed by judges, attorneys and even sometimes law professors and to establish how persuasion succeeds or fails depending on characteristics of the audience, the attributes of legal rhetoricians and the interests they seek to advance. 47
B. Law as product and instrument of social planning
The second line of inquiry in Empirical Jurisprudence builds on the notion that law is the product of human decision making and effectively functions as an instrument of social planning. Although persuasion is a fundamental dimension of the work of a large set of legal practitioners, legal practice is not solely about persuasion. Legal rules and legal doctrines are instruments of social planning designed to channel human behaviour in a certain way, and those who enact them do so with a view to engineering certain social outcomes. Lawmakers and legal reformers need to discern the preferences, incentives and constraints that will determine how human agents respond to the rules they have designed. Equally, before thinking about the best way to package their decisions, judges – particularly those sitting at the top of the judicial pecking order – have to consider the effect their rulings might have on the behaviour of litigants and other public and private decision makers. 48 So do in-house lawyers when they draft employment contracts, liability disclaimers or privacy notices.
Sometimes the more pressing issue in legal practice is less to fathom out the impact of legal rules than to predict which rules will emerge from the deliberations of judicial agents, regulators and legislators. When advising clients, lawyers are often called on to assess the probability that a contemplated course of action will trigger litigation and to estimate the odds of prevailing in the event that the client is actually taken to court. From compliance to legal risk management, lawyers frequently act as risk analysts, weighing the likelihood of possible scenarios and drawing out their concrete implications.
Legal engineering, legal forecasting and legal risk analysis require skills and insights that have less to do with eloquence and communication and more to do with understanding causal pathways, incentive structures and social interaction. Here the research agenda of Empirical Jurisprudence shows a clear overlap – to the point of being coextensive – with research programmes in institutional economics, political economy, decision theory, law and economics, forecasting, machine learning and quantitative political science. Does that mean legal researchers will have to compete with researchers in those disciplines? Our answer, unsettling as it may sound, is a resounding yes. Predicting the impact of legal rules or forecasting judicial outcomes are questions that are of direct relevance to practising lawyers and law students. Legal research, therefore, cannot afford to ignore them, even if that involves direct competition from researchers coming from other disciplines.
3. Emerging new methods
The two lines of inquiry we just marked off cover a broad spectrum of questions and concerns which, we believe, are central to the practice of law. Research focusing on these questions promises to yield insights that have direct bearing on what lawyers do and need to know in order to be successful in their line of business. What we want to show now is that suitable empirical methods exist to study these questions fruitfully. When empirical research was essentially synonymous with tedious hand coding of case outcomes and regression analysis – a useful but limited statistical tool – scepticism that empirical methods could be of any help in investigating legal phenomena was not wholly unjustified. In the meantime, though, a whole array of methods has emerged, which promises to significantly augment our understanding of legal discourse, persuasion and argumentation as well as of the impact and production of legal rules.
Start with network analysis. Network analysis is a useful tool to map relationships among legal actors. In the law, as in other professions, networks matter. Litigation or even career success may correlate with where and with whom one went to law school or which judge one clerked for. Yet network analysis can also serve to illuminate legal argumentation. Due to their referential character, precedents, legal texts and legal authorities can be fruitfully conceptualized as network, as a burgeoning literature on the European Court of Justice and the European Court of Human Rights already attests. 49 Plotted in Figure 1 is a small-scale network of the Court of Justice’s case law on family reunification. 50 The size of a dot (called a ‘node’ in network analysis jargon) indicates how much it is cited by other family reunion cases. The arrowed lines (edges) denote the direction of the citation (for example, Hadj Ahmed cites both Xhymshti and Baumbast & R). Citations are a proxy for the authority carried by precedents (perhaps Metock & Others is an authoritative precedent?) as well as for the pressure put on judges to justify particular rulings (maybe McCarthy is such a case?). Network analysis is useful in helping to determine which precedents are regarded as (at least prima facie) relevant to a dispute, which precedents the judges believe will lend authority to their ruling, and which they think need distinguishing. For lawyers who argue cases, network analysis can assist in identifying the cases that judges are likely to regard as relevant or ought to regard as relevant given the court’s past decisions. Much of the added value of network analysis, of course, lies in the possibilities it opens up for data visualization. Summarizing a large amount of information in a simple graph, the case law network in Figure 1 is much easier to process for the human brain than a verbal description of the relationships among these cases would be. The increasing availability of legal data is inversely proportional to our cognitive capacities to map out its structure comprehensively and systematically.
Network analysis of the Court of Justice’s case law on family reunion. Dot size is proportional to the number of outward citations.
Another useful computer-based technique to investigate legal discourse comes from the field of text mining. Topic modelling, for example, allows one to get an overview of the topics discussed in large collections of legal documents, whether law review articles, judicial opinions or legislative bills. Documents are assigned to topics generated by the topic modelling algorithm on the basis of semantic similarities. Illustrated in Figure 2 are 20 topics 51 summarizing the Court of Justice’s entire preliminary ruling activity (CJEU opinions were scrapped from the EUR-Lex website) which we generated using Latent Dirichlet Allocation. Each topic is defined by words characteristic of the documents it identifies. These words result from the application of the algorithm. They are not selected by the researcher. For each topic, the 10 most characteristic words are shown. Bars denote the degree to which words are characteristic of the corresponding topic (the higher the beta value the more characteristic of the topic the term is). Topic models can greatly ease what would otherwise be a haystack search. For example, where would one look for cases on family reunion? Topic 16 (with words like ‘residence’, ‘family’ and ‘nationals’) looks like a promising lead. Well, it turns out that most cases in Figure 1 can be found under this topic. From the plot in Figure 2 it is only a small step to charting the evolution of the proportion of these various topics in CJEU rulings over time.
Topic model of Article 267 judgments, 1961–2014. Displayed are the 10 words most characteristic of the corresponding topic.
Topic modelling can serve to study large collections of judgments to explore what issues litigants bring to the courts 52 or streams of law review articles to determine what questions legal scholars debate. 53 If one believes that classifying and summarizing legal materials, which busy practitioners may struggle to navigate, constitutes an important function of legal research, then these techniques hold out great promise. All the more so, as data-harvesting techniques make it possible to scrape the entire case law of a court (as we did here with the Court of Justice) automatically.
Moving from the analysis of topics and legal domains to mapping the stance taken by legal actors on the same topic, techniques such as sentiment analysis, Bayesian naïve classification and correspondence analysis can help get a quick overview of the position taken by legal scholars on a given question or trace the evolution of a court’s position on a given issue over several decades. Depicted in Figure 3 is a correspondence analysis of 19 opinions rendered by the German Federal Constitutional Court on European integration. 54 Spanning six decades, the jurisprudence of the German Court has alternated more and less pro-integration opinions. The algorithm appears to make a reasonably good job of summarizing the fluctuating jurisprudential signals sent by the Bundesverfassungsgericht, with the Banana and Honeywell rulings scoring high and the Maastricht and Lisbon rulings low on pro-integration. 55 Consistent with the views of the legal commentariat, the last iteration of the German Court’s see-sawing stance on European integration, its Identity decision, also appears significantly less Eurosceptic than its reference in the OMT case a year earlier. 56 Of course, a specialist of the Court’s case law may quibble that Solange I’s score should be lower or Solange II’s score higher. But this cannot take away the remarkable fact that the chart in Figure 3 has been generated without a human agent actually having to read a single decision. What this demonstrates is that these techniques can aid legal scholars better to trace out the twists and turns of a court’s case law.
Correspondence analysis. German Constitutional Court opinions on European integration.
Unsupervised text-scaling techniques such as correspondence analysis are based on the simple assumption that word usage varies not only from topic to topic but also depending on the stance speakers (here the same judicial institution at different points in time) take within the same topic. The greater the disparity in word usage, the greater the underlying ideological or jurisprudential difference is presumed to be. Examining what differences in word usage drive the scores generated by the algorithm, therefore, can be very instructive for a study of legal persuasion. Figure 4 shows how words in the corpus – there are thousands of them (hence the dense cloud) – correlate with the first and second dimension along which the texts have been arrayed. We can see that dimension 2, which is the dimension depicted in Figure 4, correlates with words like ‘democratic’ (demokratischer), ‘democratic principle’ (Demokratieprinzip), ‘statehood’ (Staatlichkeit) and ‘EEC Treaty’ (EWGV). Here EU scholars will recognize distinctive elements of the phraseology and doctrines invoked by the German Federal Constitutional Court to justify its landmark rulings on integration. More strongly correlated with dimension 1 are the words ‘criminal matters’ (Strafsachen), ‘criminal procedure’ (Strafverfahren), ‘rule of law’ (Rechsstaatsprinzip) and ‘review’ (Überprüfung) – to mention but a few. They reflect the extent to which the Court’s rulings focus on the application of the European Arrest Warrant.
Correspondence analysis. Correlations of terms occurring in German Constitutional Court opinions on European integration. The two dimensions explaining the largest variance are displayed. Words closer to the diagonal line have greater correlation.
There are multiple ways to visualize patterns of lexical usage and their relationship to the dimensions identified in a collection of documents (some, like interactive 3D graphs, though, do not render well on the 2D page of a journal article). In any case, these methods open up a whole set of new possibilities to explore legal discourse along with the rhetorical strategies of legal actors. 57
No less rich is the battery of techniques available to study behaviour and its causal determinants in legal settings. Thanks to modern online survey tools, conducting randomized experiments has never been easier. Students can be efficiently divided into a treatment and a control group while sitting in the classroom and be served different versions of the same case on their laptop or smartphone. For example, a well-researched cognitive bias in decision making is anchoring – the tendency to rely too heavily on an ‘anchor’, one trait or piece of information, when making a decision, even if the anchor is absurd or irrelevant. Using our law students as volunteers, we were able to replicate the findings of a US study showing that in tort disputes subjects (in that case real-world judges) can be influenced by an extreme anchor when determining what amount of damages should be awarded to the claimant. 58 Randomization ensures that law student attributes (such as gender, ideology or legal knowledge) are equalized across treatment and control groups. The effect of the treatment – the extremely low damage-award anchor in our classroom experiment 59 – can be established by a simple mean comparison or with the help of a box-and-whiskers plot, 60 as in Figure 5. Students exposed to the anchor were less generous (median value denoted by the bar inside the box is EUR 25,000) than students in the control group (median = EUR 60,000). 61
Randomized experiment: anchoring effect in damage award determination (N = 41). Plot shows distribution of damage awards in each group. Each distribution is divided in quartiles with bar in the middle of the boxes indicating with the median.
Equally promising are all the techniques associated with big data, artificial intelligence and machine learning. These include neural networks, decision tree learning (including Conditional Inference Tree), support vector machine and random forest. As demonstrated by a flurry of recent applications, these techniques hold out great promise in the area of legal forecasting. 62 Obviously, they tend to work better with large datasets. They generally involve partitioning the available data in separate training and test sets. The training set serves to train the algorithm to identify patterns in the data that predict the variable of interest, whereas the test set is used to evaluate its forecasting accuracy. Figure 6 illustrates a Conditional Inference Tree we trained to predict referral activity across EU Member States over the lifetime of the European project. Our target variable – that is, the outcome we want to predict – is the annual number of Article 267 references submitted by each Member State. 63 Building off from the empirical literature on the preliminary ruling mechanism, 64 our dataset includes various attributes thought to influence the rate at which domestic judiciaries refer questions to the Court of Justice: public support for EU membership, political fragmentation, length of membership, intra-EU trade, monism, population, power to exert judicial review under domestic law, the existence of a constitutional court and enlargement. In the end, the tree generated by the algorithm on the basis of our training set indicates that, aside from country-specific effects, only three attributes are predictive of referral activity at country level: population, length of membership and intra-EU trade (measured as Export + Import to and from other Member States).
Predicting annual referral rates with Conditional Inference Tree. Numbered nodes show variable and statistical significance. Tree is grown using leave-one-out cross-validation. Population is in thousand inhabitants. End leaves show predicted distribution of referral rates.
It is easy to follow the branches of the tree and see what combination of factors accounts for the predicted distribution of annual referral rates in each end leaf. For example, countries other than the Netherlands that do a middling amount of intra-EU trade (EUR 45 billion < x ≤ EUR 114 billion) but have a population larger than 56,363,000 will produce between 7 and 30 references a year, with a median of 18 (terminal leaf shows higher density between 10 and 20). More generally, the tree tells us how many Article 267 references to expect from Member State x in year y knowing its population, length of membership and level of intra-EU trade. Machine learning applications like this can yield high predictive accuracy. The tree in Figure 6, for example, correctly predicts 70% of referral rates. 65
These examples are certainly not meant to exhaust the list of empirical techniques potentially relevant to the research agenda expounded in this article. Field experiments, 66 qualitative interviews, expert surveys 67 and case studies also have their place in the jurisprudential empiricist’s toolkit. What these examples are meant to convey, however, is not comprehensiveness but a sense of the potential that all these techniques hold for legal research.
4. Objections to Empirical Jurisprudence: Disciplinary autonomy, tradition and relevance
The previous sections have demonstrated that there is now a wide range of empirical methods available to investigate questions that are of central interest to the legal community. However, sceptics attached to the idea of law as an autonomous discipline are unlikely to be swayed simply by being shown a handful of charts, and our argument that lawyers implicitly operate with an interdisciplinary concept of law may also fail to convince them. In this section, therefore, we try to address some of the objections that the outlined research programme is bound to elicit.
A. Why attempts to codify the doctrinal method are a blind alley
Some scholars appear, at least implicitly, to reject the view that adopting the methods and theoretical insights of the social sciences is the proper way to go about modernizing European legal scholarship. Instead, what they believe is that the credibility deficit of legal scholarship in the broader scientific community stems not from its lacking a defining method but, rather, from its failure to explain and articulate its implicit methodology. To put it in a nutshell, the solution is not methodological reorientation but methodological codification. The recommendations of these scholars range from simple guidelines such as being careful to spell out the problem at issue and correctly handling sources to the requirement that assumptions, including normative ones, be made explicit. 68 Some doubt that such recommendations really add up to a scientific methodology. 69 Others argue that greater emphasis on what are essentially basic rules of careful academic writing can nonetheless significantly raise the quality of doctrinal legal scholarship. 70
With this group of academics, we share a sense of dissatisfaction with the current state of European legal research. Yet we think that methodological codification does not go far enough to address its credibility deficit. Because most of the legal scholarship, including PhD dissertations, produced by European law schools routinely makes assertions or relies on assumptions that are part of the object of inquiry of other disciplines, methodological codification inevitably raises the interdisciplinarity question. We shall return to this issue below when we discuss the autonomy thesis. At the same time, though, and somewhat paradoxically, by implying that all doctrinal scholarship should follow its methodological precepts, the methodological codification movement may also go too far. Indeed, for legal authors who see practitioners rather than members of a scientifically trained research community as their primary audience, methodological codification may be antithetical to their rhetorical strategy. As pointed out above, this doctrinal work tends to conceive of itself as continuous with, rather than detached from, the discourse of legal actors and as part of the same struggle for persuasion. Insofar as doctrinal authors aim to engage the same practical debates in the same terms as judges and attorneys, they will resist attempts to curtail their rhetorical freedom. They will resist methodological norms that constrain their ability to creatively reconstruct precedents, laws, facts and canons of interpretation, just as they will resist methodological codification if its effect is to reduce their ability to use informal fallacies and to exploit cognitive biases. 71 Nor will they want to observe rules that require them to make their normative assumptions explicit (instead of suggesting that they are so self-evident as to obviate the need for justification) 72 or to acknowledge and represent competing normative views charitably and objectively (instead of constructing caricatures of them). Why, in short, would practice-oriented doctrinal legal scholarship want to uphold a methodological code that makes it less likely that it will achieve its ultimate aim: persuasion? When and where possible, practice-oriented doctrinal writers may be happy to make their normative claims in the name of ‘legal science’ – if only because the label ‘science’ is an effective rhetorical device – but they will resist any methodological precept that risks undermining the rhetorical effectiveness of their assertions. Deeply embedded in practice-oriented legal discourse is an ‘antitheoretischer Affekt’, 73 which is at the same time an ‘antimethodischer Affekt’. This disposition is a direct and logical consequence of the practice it purports to engage. 74 Practice-oriented doctrinal scholarship simply plays a different game, and neither methodologically codified doctrinal work nor Empirical Jurisprudence will ever be the only game in town. For that reason, what we propose is not the adoption or imposition of our methodological precepts across the board. Instead, what we advocate is a form of discursive bifurcation and differentiation. Legal research along the lines of the research programme presented in this paper can coexist with practice-focused doctrinal writing within law schools. Insights from empirical-jurisprudential research, notably on legal persuasion, may occasionally serve to inform the argumentation strategies of doctrinal scholars. In that sense, the two scholarships would not be entirely separate. Yet, governed by distinct logics, they would also be subject to distinct evaluation criteria, reflecting the two sides – the professional and the academic – of the law school’s hybrid institutional identity. To a certain extent, the European legal academy, with the emergence of Anglophone interdisciplinary peer-review journals, is already evolving in this direction.
B. Why autonomy is not desirable
The single most frequent objection – certainly in oral exchanges with law professors in continental Europe – levelled at the research programme set forth in this paper, and possibly at interdisciplinary approaches to law in general, appeals to the notion of disciplinary autonomy. One elaboration of the autonomy thesis characterizes it as consisting of four related claims. Legal reasoning is distinct from other forms of reasoning Legal decision making diverges from other types of decision making Understanding (i) and (ii) does not require insights from other disciplines than law Legal scholarship itself is not, nor should be, involved with other topics than legal doctrine.
75
Befitting a field where the dominant style of scholarly engagement continues to be rhetorical rather than analytical, the autonomy thesis is often asserted without justification to support it, as a kind of petitio principii. In similar fashion, interdisciplinary approaches to legal research have been dismissed as a passing fad 76 – a characterization tantamount to an argumentum ad odium. Whether the methodological codification movement is also premised on some variant of the autonomy thesis is unclear; its proponents do not mention it explicitly. In the end, the most elaborated defence of the autonomy thesis in the current discussion is presumably Ino Augsberg’s critique of empirical-legal approaches, which variously draws on neo-Kantian thinking, Niklas Luhmann and postmodern philosophy. 77
Remarkably, this invocation of autonomy runs against the tide of contemporary scientific research, where the general trend is towards greater interdisciplinarity and growing cross-fertilization across fields. 78 Even in the social sciences and the humanities, techniques – from Bayesian statistics 79 to text mining – and theoretical insights – whether from cognitive psychology, economics or even physics – are spreading quickly across disciplines, blurring the boundaries between previously unrelated research communities. 80 The digital revolution has made vast swathes of data – from government archives to all manner of documents, legal or not – available to researchers, and efforts to analyse these textual corpora are collapsing traditional distinctions between quantitative and qualitative or between causal and philological studies. 81
That other disciplines are embracing interdisciplinarity, however, does not, in and of itself, prove that the autonomy thesis is wrong (to suggest so would amount to an argumentum ad populum). This said, we note that there have previously been serious attempts to articulate a research programme establishing legal science as an autonomous discipline. Indeed, this was very much the aim of the Kelsenian project. Hans Kelsen and his disciples sought to establish a science of law distinct from sociology and moral theory focusing on the ‘Eigengesetzlichkeit des Rechts’ – the distinctive dynamic of law. 82 Aside from being premised on a well-specified concept of law, the research programme defined by the Pure Theory of Law is, we are willing to believe, both logically and epistemologically consistent. Unfortunately, it is also a research programme that is not very interesting. The Pure Theory of Law reduces law to a set of formal rules, without regard for the effect these rules have on human conduct or the social forces that caused them to emerge. A consistent Kelsenian scholar, therefore, is supposed to focus exclusively on the description and cognition of legal rules. 83 While the operational methodology of Kelsenian legal research would seem to be a form of applied linguistics, the results of research conducted in accordance with these methodological precepts will be idiographic, yet decontextualized, descriptions of the meaning of legal provisions. Such descriptions may be fully replicable, but most people – academics included – will find them of little direct use.
That rules considered in isolation do not make for a very attractive object of study is evidenced by the fact that very few scholars have actually adopted the Kelsenian approach. 84 Now, legal academicians who oppose the social-scientific turn advocated by Empirical Jurisprudence may be tempted to invoke the authority of the Austrian legal theorist to lend their position a greater aura of intellectual respectability. 85 However, the chance is these scholars’ own academic work is anything but Kelsenian. Legal scholars in general neither follow the methodology, nor stick to the carefully demarcated object of inquiry of the Pure Theory of Law. Their work is, in the meaning Kelsen gave to that term, impure. Far from confining themselves to the description and cognition of legal rules, legal writers constantly encroach upon the sphere of other disciplines. Legal scholarship is replete with implicit empirical assumptions – whether on the judicialization of politics and the effect of constitutional courts on de facto human rights protection, 86 the impact of CJEU rulings on the behaviour of domestic courts, 87 the impact of globalization on income disparities, 88 the impact of international treaties and human rights conventions, 89 state compliance with international law, 90 the influence of legal rules on judicial decision making, 91 the virtues of rulemaking by judges over rulemaking by legislators, 92 or the propensity of specialized courts to submit preliminary references. 93 Instances where doctrinal writers make empirical and behavioural assumptions are not hard to find. Such assumptions are frequently made without acknowledging the contribution or even relevance of the corresponding disciplines. But, more importantly, they demonstrate that legal scholarship is, in regard to the assertions legal scholars actually make, already interdisciplinary.
At this point, it is useful to go back to the methodological codification movement. What conception of legal scholarship does it purport to codify? If it is the restrictive, Kelsenian one, we doubt that it will get any traction. But if the goal is to codify doctrinal analysis more or less as it is actually practised by legal scholars, 94 then it inevitably runs into the problem of interdisciplinarity. Eventually, a choice will have to be made. Either legal researchers take interdisciplinarity seriously, as we contend they should, or they keep peddling naïve folk economics, folk psychology and folk political science. Choosing the latter course of action makes perfect rhetorical sense – again, complex, nuanced arguments tend to fare poorly in the courtroom – but it will do little to raise the scientific credibility of legal scholarship vis-a-vis other disciplines.
C. Empirical approaches do not ignore legal rules and doctrines
Other objections do not negate the possibility of legal-empirical research per se but reflect scepticism about existing empirical legal research. Empirical studies, it is contended, either do not address the doctrinal and normative questions that are of interest to lawyers or fail to do so in a convincing manner. 95 Nor do they address the need to classify and organize legal sources and materials. 96
It is true that in many empirical studies doctrinal variations do not appear either as dependent or as independent variable. The large political science literature on judicial decision making, for example, has long focused exclusively on explaining case disposition, although lawyers are usually more interested in the content of the accompanying judicial opinion. 97 It is also true that interdisciplinary studies are not all good or, for that matter, methodologically flawless. However, this is not an argument against empirical research, but actually one for producing more of it. Examples of good empirical legal analysis attentive to doctrinal variability and its effects or causes do exist. 98 The fact that not all interdisciplinary work lives up to that standard cannot justify dismissing the entire field – at least no more than the debunking of cold fusion can justify dismissing the entire discipline of physics. 99 Indubitably, empirical-jurisprudential research raises plenty of challenges. But taking up these challenges – by building theories, identifying adequate methods and developing appropriate empirical tests – is the very point of research. Furthermore, Empirical Jurisprudence does not rule out normative considerations. As pointed out in the introduction, drawing out the normative implications of empirical analysis is good practice and legal researchers should seek to bring empirical findings to bear on normative debates and policy discussions. As for the classificatory function of legal scholarship, we believe that computer-based methods such as the ones described above can greatly enhance the development of legal summaries and taxonomies.
D. Don’t say we haven’t been trained for it
Another of these objections usually made in oral rather than written form is that lawyers lack the requisite methodological training to conduct empirical-jurisprudential research. 100 It is true that the average European law student gets very little exposure, if any, to interdisciplinary approaches to law in a curriculum that spans five years or more in most countries. 101 Yet, most of the empirical techniques presented in this paper barely existed or rarely featured in social-scientific research 20 years ago; researchers, at some point, had to learn to master them. In fact, empirical methodology is itself a vast and vibrant research field and new methods are constantly being invented and refined, which, in turn, means the methodological bar for the publication of empirical results is getting higher and higher. Social scientists thus have no choice but to upgrade their methodological know-how in order to keep pace with the evolution of their discipline. This contrasts with the legal academy, where the prevailing wisdom seems to be that the old legal dog cannot learn new tricks. Yet we do not see why it ought to be so. Legal scholars and law students, after all, are at least as smart as their counterparts from history, sociology, economics and political science. There is no reason why life-long learning could not also become the norm in legal research.
5. Conclusion: A choice between reinvention and splendid isolation
We hope to have demonstrated the potential as well as the relevance of Empirical Jurisprudence as a research agenda for the legal academy. What is specifically ‘jurisprudential’ in this research programme is not the methodology but the questions it sets out to investigate. The spirit of this research programme is very pragmatic. Legal researchers should use whatever method works best to answer the questions that members of the legal community care about – whether it is finding the most effective persuasion strategy, forecasting judicial outcomes, assessing the impact of legal reform or assessing legal risk.
We believe that adopting this research programme can greatly benefit the European legal academy, and this on the research as well as on the teaching front. While making it easier to convince research funding agencies that legal research is worth the investment, it will also help foster a stronger research culture within European law schools. In more empirical disciplines, the endless struggle to improve theory building and methodological rigour means scholars have greater incentives to engage their peers, to seek feedback and to encourage a culture of constructive criticism and creative thinking. At the same time, if law schools are serious about training advocacy experts, insights from this research can be used to teach students how to discern the rhetoric behind doctrinal constructions, how law can be used as instrument either of social change or of social stability, and how new techniques can be deployed to navigate a complex, rapidly evolving and increasingly multi-jurisdictional environment. On that score, Empirical Jurisprudence chimes in well with a vision of the 21st-century law school that seeks to reconcile its vocational and academic missions while responding to the challenges brought by globalization and technological disruption. 102
What are the odds of such a research programme being actually implemented across the continent? Our view is that European legal research faces a choice between reinvention and increasing isolation. This should be an incentive to embrace change. But it would be foolish to underestimate the cultural and institutional obstacles. 103 Pressure from funding agencies together with competition from other disciplines will induce more jurists to embrace interdisciplinarity. Leading law schools, by recruiting hybrid researchers with degrees in law as well as in another discipline, as top US law schools have, 104 may accelerate the pace of change. Yet, in all but the most advanced and internationalized corners of the European legal academy, change will come more slowly. But come, we are pretty sure, it will.
Footnotes
Funding
All authors acknowledge financial support from ERC Starting Grant 638154 (EUTHORITY).