Abstract
The purpose of this article is to explore the effectiveness of entrepreneurship education (EE) programmes through the lens of realist evaluation (RE). The interest of the authoring team – a practitioner–academic mix with professional experience including developing EE in primary and secondary schools – lies with EE competitions, a type of intervention recommended for and delivered to students and pupils of all ages. RE is a theory-driven philosophy, methodology and adaptable logic of enquiry with which to conceptualize and analyse such programmes. In this study, we undertake an act of ‘organized scepticism’, as described by evidenced-based policy academic Ray Pawson, to identify and question the declared outcomes of EE competitions in European policy over a 10-year period. However, our contribution goes beyond the application of an evaluation approach, novel to EE. We argue that, while education generally, and EE specifically, appears committed to emulating ‘gold standard’ scientific evaluation approaches (e.g. randomized controlled trials, systematic review and meta-analysis), the field of evidenced-based policymaking has moved on. Now, alternative methodological strategies are being embraced and RE in particular has evolved as an approach which better aligns knowledge production with the reality of complex, socially contingent programmes. By using this approach, we not only establish that education and psychology theories challenge the outcomes of EE competitions declared in policy, but also demonstrate the wider relevance of RE to the appraisal and refinement of the theorizing and practice of entrepreneurship programmes and interventions.
What is entrepreneurship education?
The propositions put forward in answer to the above question are numerous and contested. Hannon (2005) describes several competing categorizations. Entrepreneurship education (EE), he summarizes, is often presented as learning ‘about’ (as academic study), ‘through’ (as learning core capabilities embedded across curricula) and ‘for’ (preparation for an entrepreneurial life and business start-ups or ventures) entrepreneurship. An alternative concept he presents highlights different foci to be learned through EE (process focus, client focus, outcome focus and vision focus). Yet another model proposes three basic categories for EE: the contextual application of entrepreneurial characteristics and qualities (entrepreneurship); a state of being (entrepreneurial) and the creation of an entrepreneurial climate and support structure (entrepreneurism). Matlay (2006) extends this thinking about what EE is by summarizing what it is meant to achieve, noting that it has been presented as a panacea for economic stagnation, a method for facilitating the education to work transition and a route for creating an entrepreneurial culture inside and outside education.
Such debates are ongoing and continue to reflect Gibb’s (2002) statement that there is no common agreement on definitional terms. He asserted that meaning can only be inferred from the focus and purpose of public policy ‘initiatives’ – for example, ‘the emergence of more small businesses; associated higher rates of small business creation; more fast-growth firms and technology-based businesses; social entrepreneurship, enterprise in public organizations and, increasingly, a basis for tackling social exclusion’ (Gibb, 2002: 235). This suggestion indicates another response, then, to the question of what constitutes EE: EE is an idea, or set of ideas, that is packaged into a programme or intervention and is prescribed by policymakers, delivered by providers or recommended to educators in schools, colleges and universities as particularly effective in ameliorating or solving certain problems. Problems such as economic stagnation have been given different names; for example, social messes (Horn, 2001), wicked problems (Head, 2008) and complex social problems (Chatterji, 2016). These problems are characterized as complex, open-ended and intractable and, as such, prompt complex responses in the form of programmes and interventions designed to address the critical situation in some way (e.g. countering economic stagnation by creating more and better entrepreneurs through EE). Chatterji (2016) has defined these responses as complex social programmes which share certain characteristics, including: they are socially mediated; they are delivered by people with varying levels of organization and autonomy; they have many moving parts; and they operate in larger, multilevel communities with multiple agendas and actors that may directly or indirectly influence the functioning of the programme and its outcomes. For the purpose of this article, then, we conceptualize EE programmes generally, and EE competitions specifically as complex social programmes (Pawson, 2006; Wong et al., 2013). Such a characterization is reflected by Pittaway and Cope (2007), who illustrate the complexity of EE when they present it as nested within a number of layers (e.g. the programme and university context, the general enterprise infrastructure and the wider policy environment), in addition to pointing to the additional complexity emanating from the fact that individual actors in a programme (students, businesses or faculty) will also have their own individual capacities and inclinations.
An important question to ask is how complex social programmes should be evaluated. If public money is spent on such programmes because they are presented as solutions to social problems, it is reasonable to ask whether they are a good investment – indeed, this is part of the rationale for evidenced-based policymaking (Pawson, 2006). Rideout and Gray aim to answer such a question in their 2013 paper ‘Does entrepreneurship education really work?’, conducting a review of empirical studies that evaluate EE programmes. They draw a clear line between methodological rigour and the scientific model of evaluation, citing approaches involving treatment and matched comparison groups and quasi-experimental controls as the foundation for drawing causal conclusions. However, the authors concede that such studies appear to unable to answer the question ‘Does it work?’ and, furthermore, are nowhere close to answering the more significant question, ‘How does it work?’ By way of remedy, they suggest that future research should utilize the ‘gold standard’ of evaluation research – quasi-experimental and experimental design – in order to move on the debate (Rideout and Gray, 2013: 345–347). However, we contend that such approaches can only ever tell us what has worked they can only provide a score for a particular programme at a particular time. The crucial elements of increasing scientific knowledge – cumulation, theory testing and improving and deriving knowledge from tacit understanding and experience – are ruled out by models that prejudge what is to be proven (Hammersley, 2001). Realist evaluation (RE) has been developed specifically to explore and evaluate complex social programmes (Pawson, 2006, 2013; Pawson et al., 1997; Wong et al., 2013). It aims to address weaknesses such as the incomplete knowledge developed through experimental methods, systematic review and meta-analysis, where findings can be mixed or inconclusive and do not provide sufficiently useful explanations to practitioners or policymakers about what might work, for whom, and why. When applied to evaluating social programmes, such methods can obfuscate complexity and lead to artificial and misleading results: …hypotheses are abridged, studies are dropped, programme details are filtered out, contextual information is eliminated, selected findings are utilized, averages are taken, estimates are made…this is all done in an attempt to wash out ‘bias’…however, in this purgative process the very features that explain how interventions work are eliminated from reckoning. (Pawson, 2006: 42)
RE as an alternative approach to navigating complexity
Weiss (1987) notes that knowledge generated through the evaluation of programmes can serve different functions: a warning (that things are going wrong), guidance (direction for improving policies and programmes), reconceptualization (making sense of activities and outcomes) and mobilization of support (stiffen support for, or weaken adherence to, a particular position). Our lived experience as practitioners in the field of EE is that competitions and competitive pedagogies are ubiquitous. The expansion of competitions has been charted in historical accounts of EE in higher education (HE) and schools (Katz, 2003; Sukarieh and Tannock, 2009). Competitions have been identified as a staple part of the extra-curricular EE menu in universities (Preedy and Jones, 2015), and the most frequent and familiar activity delivered in schools (Mann et al., 2017). For us, they are an example of ‘taken-for-granted’ practice, which Fayolle (2013) argues should receive more critical attention. As a result, we employ an RE approach to counteract the taken-for-granted assumptions and ‘musty sameness’ of existing policy (Weiss, 1987: 16) by assessing EE competitions in European guidance over a 10-year period.
RE (sometimes known as ‘realistic evaluation’) is an approach and logic of thinking evolved specifically for researching and evaluating complex, socially contingent programmes (Pawson & Tilley, 1997). From the RE perspective, whenever a programme is implemented it is testing a ‘theory’ about what might change and how (Westhorp et al., 2011). As such, it aims to surface and track ideas inherent in the design and delivery of social programmes and to explain why complex interventions work (or fail), in order to provide the policy and practice community with new ideas and knowledge to achieve or improve outcomes for intended beneficiaries. Researchers have developed protocols and standards for realist review and realist synthesis (Pawson et al., 2005; Wong et al., 2013), identifying 19 steps that can be followed systematically. But the logic of RE and methodologies is also a resource that ‘imbues a way of thinking’ (Astbury, 2018: 75) and can be adapted and applied flexibly – for example, to isolate and investigate a particular element of a policy or a programme, and its underlying theories (Pawson, 2006).
Pawson (2006) describes a number of assumptions and expectations inherent in the RE approach that are relevant to the adaptive approach undertaken in our study and that have wider implications for the evaluation of EE programmes and interventions:
the ‘same’ intervention will meet with success and failure (and everything in between) when applied in different contexts and settings;
tracking successes and failures in programmes will lead to elements of explanation about the reasoning and reactions of different stakeholders; and
we should examine which intermediate outputs need to be in place for successful outcomes to occur.
As practitioners involved in the design and delivery of EE programmes, these assumptions align well with experience. Interventions are received differently, aspects appear to ‘work’ for some and not for others and there are always contextual issues – at individual and institutional levels and in the wider environment – which influence the reactions of participants and the patterns of outcomes. We were therefore keen to adapt the logic of RE to make explicit the assumed outcomes of EE competitions and purposefully search for alternative and unintended outcomes so as to critically enrich perspectives and encourage a reconceptualization. Criticism of such adaptive approaches are based on the lack of clarity and reproducibility of studies (Pawson et al., 2004). But from a realist philosophical standpoint, standardization and reproducibility in the evaluation of complex social programmes – as they are presented in scientific experimentation – are an impossibility. Furthermore, researchers using RE question whether objectivity in science even stems from procedure, standardization and reproducibility, and instead assert that ‘validity rests on refutation rather than replication’ (Pawson et al., 2004: 38). The consequence is that researchers should surface the logic of their reasoning, and try to show their workings; but ultimately all findings should be considered tentative and fallible and as existing to expose to criticism a theory about how a programme works (Pawson et al., 2004: 38). In the next section, we describe how the realist logic of thinking was applied in the context of our chosen policy setting – European policy and guidance on EE over the 10-year period from 2006 to 2016.
Methodology
Our approach is inspired by the logic of RE, which recommends researchers undertake an act of organized scepticism, in which claims about a programme (in this case, EE competitions) are exposed to critical scrutiny through ‘precise, vexatious cross-examination’ (Pawson, 2013: 108). Such an approach ensures that alternative theories etc etc that alternative theories and unintentional outcomes receive as much research attention as do the widely promised benefits. We surface the logic of our reasoning by describing the process of our study. Table 1 summarizes the stages of our research and related questions, which are detailed below.
Research stages and related questions.
Note: EE: entrepreneurship education.
Stage 1: Identifying a focus for study
RE does not have the same hierarchy of evidence that exists in scientific research; instead, it seeks out any specific wisdom about how a programme may (or may not) work. This opportunity to study everything can lead to the researcher becoming overwhelmed with data, and therefore identifying a focus is important. As an authoring team that spans England and Spain and has current and historical involvement in European projects to develop enterprise, entrepreneurship and social innovation projects in school settings, we were interested to explore how EE competitions were positioned in policy and guidance issued by the European Commission (EC). In the last decade, the EC has acted as a catalyst and a facilitator in the promotion of EE and social cohesion), and in 2004 it communicated the launch of an action plan for the ‘European Agenda for Entrepreneurship’ (EC, 2004). One of the outcomes of this communication was a conference - ‘The Oslo Agenda for Entrepreneurship Education in Europe’ (EC, 2006), - and it signalled the start of of a decade of EE policy and practice promotion. Reviewing these documents provides a valuable overview of the policies and good practice that have been recommended to schools over the last decade, as well as capturing the national strategies, curricula and learning objectives of member state countries. Furthermore, we reason that this EC guidance is promoted to national governments across 28 member states and aims to influence those who have responsibility for developing policy for entrepreneurship and enterprise in education, and those involved in providing EE experiences (e.g. teachers, charities or enterprise promotion groups). By way of example, there are more than 42 million young people in lower and upper secondary education across Europe who are the target of ‘entrepreneurship and creativity’ policies (EC, 2015). The body of policy and guidance we chose to focus on will be used by a variety of actors to inform and justify decisions made at many levels: by teachers in the classroom, by school leaders at an institutional level, by programme commissioners at a district level, and finally, by national governments who decide how public money is spent on EE. We identified that the Oslo Agenda (2006) offered a useful line in the sand at which to start our study and extend up to and include the report Entrepreneurship Education at School in Europe (2016). This body of work consists of 11 publications, including state-of-the-art reports on policy and practice, guidance and case studies focused on EE in non-HE settings (see Online Supplemental Appendix).
Stage 2: Data extraction and initial synthesis
Based on the scope we identified, school-focused EC policy documents from 2006 to 2016 were studied, searching for the inclusion of the terms ‘competitions’, ‘contests’, ‘prizes’ and ‘awards’. When these terms were found, the context of their inclusion was logged and direct comments were collated. Pawson et al. (2004) describe how researchers may make use of ‘data extraction forms’ to assist the sifting and sorting of materials, and we used four initial headings to organize information: year of publication, policy document name, context of a term’s entry in the document and direct quotations. This process equates to what Pawson et al. (2004) call ‘theory tracking mode’, in which documents are scoured for ideas on what an intervention is and what it is meant to be doing. These ideas are then highlighted, noted and given an approximate label. Our data extraction form (available in the Online Supplemental Material), included text which related to competitions, contests, prizes and awards. Whole direct quotations were harvested and ascribed an approximate label (these are identified in Tables 3 and 4). In some cases (EC/CEDEFOP, 2011), the use of the word competition was so frequent, that judgements were made to extract representative text which would most add to theory development. We provide an example of our form in Table 2, with sample extractions from different years. Then we present an initial synthesis of how competitions are characterized in policy before cross-examining the assumed short and intermediate outcomes.
Selected examples from data extraction form – terms of search: competition, contest, award, prize in (non-HE) European policy and guidance 2006–2016.
Note: EE: entrepreneurship education; HE: higher education; JEP: Junior Entrepreneur Programme.
We were struck by how the positioning of competitions shifts dramatically after their first mention. They are initially described as an ‘effective communications activity’ in The Oslo Agenda, but within a decade the method is qualified as ‘a learning form where competitive elements are used in order to achieve better learning outcomes’ (Komarkova et al., 2015: 84). The summary presented in Table 3 demonstrates how competitions are characterized.
Characterization of competitions in European policy and guidance, 2006–2016.
Note: EE: entrepreneurship education.
Competitions feature as an integral part of strategy, a model of good practice and a teaching method. National, regional and local competitions successfully engage students as well as incentivizing teachers. They are the vehicle that drives young people into performing to the best of their ability and the nature of competition itself is assumed to transform learning and outcomes: Business planning/ideas competitions…help young people pursue their entrepreneurial ideas and ambitions. (CEDEFOP, 2011: 16) …[students]…discover and develop their abilities through school or national competitions. (EC/EACEA, 2012: 41) …competitive elements are used in order to achieve better learning outcomes…(EC/EACEA, 2016: 84)
Competitions are also characterized as an assessment method and pedagogical approach. Assessment is achieved through the critique and evaluation provided by business people, measurement against the performance of peers and summative assessment provided by performance in competitions and pitches. Finally, a significant number of reports (73%) included competitions and competitive learning in sections on ‘teacher development’, ‘teacher support’ or ‘teaching materials’, recommending the method in content or case studies as a technique which educators should apply in classroom situations to achieve entrepreneurial learning outcomes. Such advice may have contributed to the extensive promotion and adoption of the competition method. This is reflected in one state-of-play report which notes that ‘…traditional start-up methods (pitches, competitions, events, business or idea plan) are, to some extent, and often in an adapted way, applied across all levels of education…’ (EC/JRC, 2015: 65). According to this report, learning-by-doing combined with collaborative and competitive teaching methods is the most common pedagogical approach in EE. Indeed, successive reports position competition as a teaching method in its own right, describing it as something delivered not just by providers, or as part of an extra-curricular activity, but by entrepreneurship educators at all phases of education as part of their entrepreneurial pedagogical toolkit.
Table 4 presents the declared benefits of competitions, with four of them (better employability, better start-up rates, higher earning and economic growth) describing significant long-term impact.
Benefits of competitions in European policy and guidance, 2006–2016.
Note: Short-term and intermediate outcomes expected for students in bold type were the subject of the realist cross-examination.
Among all the benefits and positive outcomes declared for competitions, there is just one case where concern is reported. Promoters of the Junior Entrepreneur Programme in Ireland report that, ‘Initially, the JEP programme was based on a competition with one winner. During the pilot phase, the feedback showed this competitive environment had negative effects, creating unhappiness among teachers and pupils’ (EC/JRC, 2015: 59). For the first time in European policy, a note of caution is sounded about the widespread use of entrepreneurship competitions, at least for lower education levels. The next stage in our research process was to purposefully search for more such instances, and to look for theories and research that would add to this alternative perspective. The results are presented in the next section.
Stage 3: Cross-examining short-term and intermediate outcomes declared in policy and guidance
Pawson describes the importance of organized scepticism, in which claims about a programme are exposed to critical scrutiny through ‘precise, vexatious cross-examination’ (Pawson, 2013: 108), so that alternative theories and unintentional outcomes receive as much research attention as the widely promised benefits. To focus this element of our study, we chose to limit our cross-examination to the most frequently cited short and intermediate benefits declared for students. Policy and guidance most commonly declares that competitions develop students’ skills, that they are motivating, that students are inspired by their peers in competitions and that students find competitions rewarding.
We focus our cross-examination on these outcomes for two reasons. First, as an authoring team including teacher educators, we are sensitive to Guskey’s (2002) model of evaluating professional development. He identifies impact on students as the bottom line for educators. Does an activity benefit students? Does it achieve its stated goals? Were there unintended outcomes? If educators do not see a positive impact on students, he suggests, they will not be motivated to continue with and develop a practice (Guskey, 2002). Second, we are minded of the realist call for greater inspection of the ‘implementation chain’ – that is, what intermediate outcomes should be in place to lead to longer-term impacts (Pawson, 2006: 29). We speculated how likely it was that long-term benefits would be realized if short and intermediate benefits were not secured. The inherent logic of a programme and its theory may start to break down if such outcomes are not achieved. As such, we purposefully sought out research from other fields (e.g. education and psychology) which would offer alternative theories on the short-term and intermediate outcomes declared in policy and guidance. We present our cross-examination in the following subsections and conclude with a discussion of social context, which emerged as an important factor to consider.
Skill development
Seven reports identified that one of the benefits of competitions is that they are successful in developing students’ skills. Positive outcomes are facilitated by the nature of the competitive model itself, which enables young people to ‘…gain specific…skills for handling various social situations…’ (EC/EACEA, 2012: 41), ‘pursue entrepreneurial ideas and ambitions’ (CEDEFOP, 2011: 16), which promotes ‘creative ideas, teamwork, solving of real problems’ (EC/EACEA, 2012: 57) and which means that ‘students score higher on questions about self-confidence, cooperation skills and motivation in school’ (EC, 2009: 36).
One argument for competition is that it is part of the DNA of society, evident in relationships and dynamics at home and at work, and in hobbies and entertainment (Fülöp, 2009; Vansteenkiste and Deci, 2003), and therefore learning to compete is learning the skills required for life as well as for work. However, educational research has looked at the effects of different tasks and structures on students’ learning and outcomes and calls into question the uncritical use of competitions as a vehicle for such skill development. While competition can increase productivity or performance on rote, speed or basic tasks (Johnson et al., 1981; Slavin, 1977), it can undermine performance in problem solving and creative tasks (Amabile, 1983; Butler, 1989; Johnson et al., 1981). ‘Creativity’ and ‘problem solving’ often feature on the entrepreneurial competencies wish list. For example, the recent European Entrepreneurship Competence Framework (EntreComp), includes creativity as a specific competency, and problem-solving features in descriptors for other skills, such as taking the initiative and spotting opportunities (Bacigalupo et al., 2016). Further research is required then to clarify the declared benefits and actual effects of competition in EE on the development of such skills. Theorists distinguish between ‘performance goals’ and ‘mastery goals’ and the different ways these conceptions influence the development of skills. Central to a performance goal is the idea that one’s skill is evidenced by doing better than others, and that this performance is publicly recognized (Ames, 1992; Dweck, 1986). As a result, learning and skills development is viewed as a way to achieve a desired goal, rather than as an end in itself. As a consequence, if considerable effort is invested but does not lead to ‘success’ it can lead to a negative evaluation of competences (Ames, 1992), and disengagement from developing that skill. In contrast, mastery goals focus on the intrinsic value of learning and using efforts to develop skills and competences (Ames, 1992; Dweck, 1986). So, to summarize, a competitive process may incentivize a prioritization of performance outcomes over skill development. For example, in a group working on a competitive pitch, those who might most benefit from developing presentation skills are least likely to take the lead, despite being most in need of development (McCollough et al., 2016). Competition is not then, in and of itself, a strategy that guarantees the development of skills and competencies. There is a risk that competitions will focus students on the ends – winning (or losing) – rather than on the valued building of new skills and competences (Bergin and Cooks, 2000; McCollough et al., 2016). Furthermore, if students’ efforts do not lead to success, their sense of competency may be diminished and their future interest put in jeopardy, potentially leading to the opposite of developing ‘can do’ entrepreneurial skills and competences (can’t do, won’t do).
Motivation
A positive outcome, commonly cited in European policy documents regarding the use of competitions in EE, is that they motivate students. Such benefits are described generically – for example, ‘Business planning ideas/competitions…motivate young people’ (CEDEFOP, 2011: 14) – and specifically, in that they are seen as motivating young people to ‘perform to the best of their ability’ (CEDEFOP, 2011: 108) and to ‘perform to the best of their ability’ (CEDEFOP, 2011: 108), and in that young people report higher self-perceptions of motivation (EC, 2009: 36). But Self-Determination Theory defines the act of motivation as requiring the subject to be ‘moved to do something’ (Ryan and Deci, 2000a: 69). Proponents Deci and Ryan (1985) recognize that there are different types of motivation. For example, intrinsic motivation leads to the performance of an activity for its inherent satisfaction, for fun, challenge and out of curiosity, whereas extrinsic motivation refers to doing something because it leads to a separable outcome (Ryan and Deci, 2000a: 55). Competitions are a special type of extrinsic activity per se as they necessitate measuring one’s own performance against that of others, which can tend to decrease intrinsic motivation (Ames, 1984a; Ames and Ames, 1984).
Arguments for competition in education suggest that it can play a role in catalysing increases in performance (Slavin, 1977), but the use of competitive pedagogy as an effective strategy to motivate pupils in classroom environments remains largely contested in educational research (Deci et al. 1981; Good and Brophy, 2008). Ultimately, though, competitive processes will be qualitatively different depending on cultural, situational and personal/individual factors, such as the way competitors view each other, the process they are involved in and the way they cope with winning and losing (Fülöp, 2000, 2009). A crucial process element we wish to draw attention to is the difference between the motivations (and subsequent experiences and derived meaning) of those participating in voluntary business contests compared to those participating in compulsory competitions embedded in the curriculum. The former participants may introduce ‘volunteer bias’, where the nature of the volunteers causes the positive outcome as opposed to the intervention itself – an effect that is difficult to control for (Goldstein et al., 2015; Heiman, 2002; Keiding and Louis, 2016). However, being forced to participate in a competition through a compulsory curriculum activity changes the dynamic, and the potential effects, of the act itself. Competition can be experienced as ‘coercive’ (Good and Brophy, 2008) and may be ineffective as a motivational strategy for all pupils (Meece et al., 2006). Motivation may be diminished through the focus placed on winning (Ames, 1984b; Butler, 1989; Deci et al., 1981) and the exposure to public failure (Rahal, 2010). Students who repeatedly perform poorly in comparison to peers will find little appeal in competitions (Good and Brophy, 2008). A crucial insight to heed is the extent to which competition is experienced in different ways by different participants. The dimensions that are seen to underlie intrinsic motivation (perceived competence and perceived effort, enjoyment and interest, pressure and tension) will result in different motivations being derived by different participants (Ryan and Deci, 2000b), according to their performance and the perceived meaning they take from it, and these individual perceptions can have a greater impact on intrinsic motivation than competition outcomes.
Inspired by peers
Another declared benefit of competitions is the opportunity to learn from and be inspired by peers. This includes the ‘inspiration’ young people gain from each other (EC, 2010: 45), and how ‘valuable learning’ is achieved by observing and imitating those whose ‘techniques and skills are greater’ (CEDEFOP, 2011: 77). But psychologists have identified that peer excellence can have a demoralizing effect if students believe that their peers’ excellent level of performance is out of their reach. Rogers and Feller’s experiments (2016) showed that incidental exposure to exemplary peer performances could undermine motivation and success, leading to de-identification with the relevant domain and, finally, quitting. The ‘discouragement-by-peer-excellence-effect’ challenges the notion that students will automatically be inspired by and learn from their peers, since being exposed to their excellent performance may make them feel less capable of performing at that level. Crucially, this changed belief appears to decrease student performance. Social Comparison Theory (Festinger, 1954) states that our sense of self is determined by making comparisons between ourselves and others in order to evaluate ourselves, and can help explain these reactions. If students compare themselves and their performance unfavourably with others, it threatens, rather than inspires, their self-worth and motivation (Meece et al., 2006). This effect is reflected in the findings of one empirical study which identified the negative impact of participating in the regional finals of an EE competition on students from lower socio-economic backgrounds: ‘When meeting other groups at regional meetings or at competitions, the pupils from the lower socio-economic background felt underprivileged, backward and less capable’ (Heilbrunn and Almor, 2014: 8). These students scored lower in terms of self-efficacy, and perceived entrepreneurship as less feasible and less desirable after the intervention.
Rewards
A number of reports identified that competitions provide important rewards for students (EC, 2009: 41, 2010: A45; EC/EACEA, 2016). The assumed benefit is that competitions reward the ‘best’ (EC, 2009: 41), provide recognition (EC, 2010: A45) and that rewards in competition ‘keep motivation high’ (EC, 2009: 40). Deci et al. (1999) distinguish between different types of rewards and their effects. Rewards that are perceived to be controlling (e.g. contingent on task engagement, task completion or quality of performance) can have negative effects on intrinsic motivation, whereas rewards that are informational (providing feedback or recognition) can provide satisfaction and have positive effects on intrinsic motivation. In competition, the reward – ‘winning’ – is extrinsic to the activity itself, and is dependent on beating opponents (Vansteenkiste and Deci, 2003). In many cases, rewards have conflicting effects and are dependent on context, so many factors must be taken into account. A meta-analysis of the effects of extrinsic rewards showed that they had a substantial undermining effect on intrinsic motivation (Deci et al., 1999). Reinforcing a previous potential red flag about the significance of student age, tangible rewards were more detrimental for children than college students, and verbal rewards were less enhancing for children than college students. This underscores the role of personal context and that interpretations of competitive outcomes must be considered from the actor’s perspective, rather than simply taking stock of who wins or loses (McAuley and Tammen, 1989). Deci et al. (1999) conclude that understanding the effects of rewards requires a consideration of the interpretation that recipients will give to the rewards in relation to their feelings of self-determination and competence. Of course, the opposite of being rewarded by winning is losing, and this can have significant negative effects (Good and Brophy, 2008; Vansteenkiste and Deci, 2003). Good and Brophy (2008) discuss the development of a loser’s psychology, where individuals and teams feel embarrassed or humiliated, and those who consistently lose may suffer losses in confidence, self-concept and enjoyment.
Finally, and perhaps more dramatically, winners are more likely to engage in unethical behaviour due to an inflated sense of entitlement and, as a result, unethical behaviour may cascade from being rewarded in competitive settings, as noted by Schurr and Ritov (2016). While that finding was from research conducted with adults, and therefore the effects have not been demonstrated in the classroom, it can be taken as a provocation for critical thought on unintended consequences, the influence on social relationships and possibilities for future enquiry.
Social context
Social context emerged out of our cross-examination as a factor that is likely to influence outcome patterns for participants in EE competitions. It has previously been argued that EE can be considered a success if it dampens unrealistic expectations and fulfils a type of ‘sorting’ according to aptitude and ability (Von Graevenitz et al., 2010). However, research in mainstream education has shown that such processes are rarely neutral in school settings, and children and young people from lower socio-economic groups are more likely to be failures due to the expectations of others and the opinions and actions of decision makers (Boaler et al., 2000). Heilbrunn and Almor (2014) showed that an overall positive statistical effect for a competitive intervention was misleading when social context was taken into account. Lower socio-economic students were practically (as well as perceptually) disadvantaged in the organization and experience of competing, with less personal and institutional support and less individual capability to complete tasks (such as phoning sponsors), as well as feeling inadequate compared to better equipped peers at regional finals.
The significance of these observations is that, clearly, the field on which teams and schools play in EE competitions is far from level. Suggesting that competition in these circumstances is a fair and effective ‘sorting’ process may well result in young people being alienated from entrepreneurship according to context (not according to ability and interest), reproducing social inequalities rather than ameliorating them. Consider a key component of competitions – the public presentation, or pitch. This is identified in three reports as a valuable element of competitions, through which students are evaluated by others and assess their own performance. The pitch represents a litmus test for finalists, but may well favour teams from socially advantaged backgrounds. Patterns of talk and interaction constitute a manifestation of class differences (Bernstein, 2009; Savage, 2015), and elevator pitches and other forms of interaction with the jury mean that socially advantaged teams with the social skills to make the right impression may be more likely to be crowned winners.
Summary
In summary, our cross-examination indicated that competitions can lead to unforeseen outcomes, especially for those in ‘at risk’ groups (e.g. students from lower socio-economic backgrounds). In particular, two distinct but closely related characteristics of competitions can result in unintended effects: Unhealthy competitions implicitly reward advantaged students (Shindler, 2009).In the same way that Petersen and O’Flynn (2007) observed that Duke of Edinburgh Award participation could be viewed as a neo-liberal technology which enabled further benefit for already advantaged students, Heilbrunn and Almor (2014) showed how the satisfaction and benefit that higher socio-economic students reported masked the deterioration experienced by lower socio-economic students. The resources which advantaged students are able to bring to bear, as individuals and in terms of school organization, teacher commitment and family capital, result in a competition that is skewed, and more likely to reward them. Winners are able to use their victory as social or educational capital at a later time (Shindler, 2009). Bold claims are made about the positive impact on employability and personal, professional and entrepreneurial success following participation in competitions. However, such new educational strategies may constitute a constraint rather than an opportunity, depending on social class (Van Zanten, 2005). Families vary in the extent of their resources to take advantage of such strategies, and those with existing advantage can therefore consolidate and increase their position in relation to others. Researchers have noted that students from independent schools are over-represented in enterprise competitions (Athayde, 2012; Huddleston et al., 2012), and a similar picture emerges when looking at the state sector in the United Kingdom, with grammar school pupils outnumbering alumni of the non-selective sector (Mann and Kashefpakdel, 2014). Essentially, EE competitions may enable confident, socially and culturally advantaged young people to gain additional social and educational capital that will benefit them further at a later time and thus, in effect, create greater disadvantage for their less well equipped peers.
Limitations
Our study is limited in its scope in that we use existing research to surface and challenge theory. But we believe it points to the usefulness of RE as a theoretically driven approach which is committed to exploring, critiquing and evaluating programmes to improve outcomes for participants.
Conclusions
An aim of RE is to provide a deeper and fuller account of reality so that practitioners and policymakers have better explanations on which to base their policy decisions and practice. An underpinning principle of the paradigm is that complex, socially contingent interventions will always have different effects on different participants in different circumstances. As well as supporting a critical evaluation of competitions, the theory-driven way of thinking that characterizes RE is especially useful for those designing, delivering, evaluating or researching complex social programmes. In this article, the identification of unintended outcomes from competitions and competitive learning processes makes it clear that the positive benefits declared in policy and guidance are far from assured. We have gone on to develop this strand of thinking and identify and communicate crucial factors that appear particularly significant in influencing outcome patterns (Brentnall et al., in press). An issue that emerged from this research as important is that of social context. We are reminded of widely cited education researcher Diane Reay, who asserts that the iniquitous effects of social class in schooling constitute a ‘monster that grows in proportion to its neglect’ (Reay, 2006: 289). Social context is often stripped out of EE, rendered invisible in research and absent in the policy picture, where one-size-fits-all enterprise policy initiatives are wielded blindly (Athayde, 2012). This sentiment might prompt EE researchers to reflect that the wider social and economic context of education is inescapable for children and young people in schools, and so is crucial for ethical research and practice which might want to challenge (or, at the very least, not reproduce) disadvantage.
This novel application of RE to EE competitions has revealed the extent to which competitions and competitive pedagogies are handed down as a method which delivers significant short-term and long-term benefits. The taken-for-granted position presents competitions as an effective model for developing skills, and motivating, inspiring and rewarding students, but applying a realist logic of enquiry reveals an alternative scenario. It illustrates that competitions can diminish competency and interest, and alienate young people from entrepreneurship before anyone can reasonably guess what their futures might hold. Of most concern is that competitions and competitive pedagogies are promoted in school-focused policy and guidance as an effective approach for all students, of all ages, in all contexts; as a result, young people will increasingly be conscripted into such activity through compulsory curricula or extra-curriculum activities. We hope that the act of organized scepticism undertaken here enables practitioners, researchers and policymakers to look beyond the intuitive appeal of competitions and competitive pedagogies and scrutinize and test theory which might add a sense of caution to the current uncritical recommendations and widespread application of such interventions. While competitions in EE are presented as engaging and effective interventions, our cross-examination demonstrates that the declared benefits and positive outcomes are by no means guaranteed.
Supplemental material
Supplemental Material, IHE_-_Brentnall_et_al_supplementary_material - The contribution of realist evaluation to critical analysis of the effectiveness of entrepreneurship education competitions
Supplemental Material, IHE_-_Brentnall_et_al_supplementary_material for The contribution of realist evaluation to critical analysis of the effectiveness of entrepreneurship education competitions by David Higgins, Laura Galloway, Paul Jones, Pauric McGowan, Catherine Brentnall, Iván Diego Rodríguez, and Nigel Culkin in Industry and Higher Education
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
