The contribution of realist evaluation to critical analysis of the effectiveness of entrepreneurship education competitions

Abstract

The purpose of this article is to explore the effectiveness of entrepreneurship education (EE) programmes through the lens of realist evaluation (RE). The interest of the authoring team – a practitioner–academic mix with professional experience including developing EE in primary and secondary schools – lies with EE competitions, a type of intervention recommended for and delivered to students and pupils of all ages. RE is a theory-driven philosophy, methodology and adaptable logic of enquiry with which to conceptualize and analyse such programmes. In this study, we undertake an act of ‘organized scepticism’, as described by evidenced-based policy academic Ray Pawson, to identify and question the declared outcomes of EE competitions in European policy over a 10-year period. However, our contribution goes beyond the application of an evaluation approach, novel to EE. We argue that, while education generally, and EE specifically, appears committed to emulating ‘gold standard’ scientific evaluation approaches (e.g. randomized controlled trials, systematic review and meta-analysis), the field of evidenced-based policymaking has moved on. Now, alternative methodological strategies are being embraced and RE in particular has evolved as an approach which better aligns knowledge production with the reality of complex, socially contingent programmes. By using this approach, we not only establish that education and psychology theories challenge the outcomes of EE competitions declared in policy, but also demonstrate the wider relevance of RE to the appraisal and refinement of the theorizing and practice of entrepreneurship programmes and interventions.

Keywords

Enterprise entrepreneurship competitions entrepreneurship education realist evaluation

What is entrepreneurship education?

The propositions put forward in answer to the above question are numerous and contested. Hannon (2005) describes several competing categorizations. Entrepreneurship education (EE), he summarizes, is often presented as learning ‘about’ (as academic study), ‘through’ (as learning core capabilities embedded across curricula) and ‘for’ (preparation for an entrepreneurial life and business start-ups or ventures) entrepreneurship. An alternative concept he presents highlights different foci to be learned through EE (process focus, client focus, outcome focus and vision focus). Yet another model proposes three basic categories for EE: the contextual application of entrepreneurial characteristics and qualities (entrepreneurship); a state of being (entrepreneurial) and the creation of an entrepreneurial climate and support structure (entrepreneurism). Matlay (2006) extends this thinking about what EE is by summarizing what it is meant to achieve, noting that it has been presented as a panacea for economic stagnation, a method for facilitating the education to work transition and a route for creating an entrepreneurial culture inside and outside education.

Such debates are ongoing and continue to reflect Gibb’s (2002) statement that there is no common agreement on definitional terms. He asserted that meaning can only be inferred from the focus and purpose of public policy ‘initiatives’ – for example, ‘the emergence of more small businesses; associated higher rates of small business creation; more fast-growth firms and technology-based businesses; social entrepreneurship, enterprise in public organizations and, increasingly, a basis for tackling social exclusion’ (Gibb, 2002: 235). This suggestion indicates another response, then, to the question of what constitutes EE: EE is an idea, or set of ideas, that is packaged into a programme or intervention and is prescribed by policymakers, delivered by providers or recommended to educators in schools, colleges and universities as particularly effective in ameliorating or solving certain problems. Problems such as economic stagnation have been given different names; for example, social messes (Horn, 2001), wicked problems (Head, 2008) and complex social problems (Chatterji, 2016). These problems are characterized as complex, open-ended and intractable and, as such, prompt complex responses in the form of programmes and interventions designed to address the critical situation in some way (e.g. countering economic stagnation by creating more and better entrepreneurs through EE). Chatterji (2016) has defined these responses as complex social programmes which share certain characteristics, including: they are socially mediated; they are delivered by people with varying levels of organization and autonomy; they have many moving parts; and they operate in larger, multilevel communities with multiple agendas and actors that may directly or indirectly influence the functioning of the programme and its outcomes. For the purpose of this article, then, we conceptualize EE programmes generally, and EE competitions specifically as complex social programmes (Pawson, 2006; Wong et al., 2013). Such a characterization is reflected by Pittaway and Cope (2007), who illustrate the complexity of EE when they present it as nested within a number of layers (e.g. the programme and university context, the general enterprise infrastructure and the wider policy environment), in addition to pointing to the additional complexity emanating from the fact that individual actors in a programme (students, businesses or faculty) will also have their own individual capacities and inclinations.

An important question to ask is how complex social programmes should be evaluated. If public money is spent on such programmes because they are presented as solutions to social problems, it is reasonable to ask whether they are a good investment – indeed, this is part of the rationale for evidenced-based policymaking (Pawson, 2006). Rideout and Gray aim to answer such a question in their 2013 paper ‘Does entrepreneurship education really work?’, conducting a review of empirical studies that evaluate EE programmes. They draw a clear line between methodological rigour and the scientific model of evaluation, citing approaches involving treatment and matched comparison groups and quasi-experimental controls as the foundation for drawing causal conclusions. However, the authors concede that such studies appear to unable to answer the question ‘Does it work?’ and, furthermore, are nowhere close to answering the more significant question, ‘How does it work?’ By way of remedy, they suggest that future research should utilize the ‘gold standard’ of evaluation research – quasi-experimental and experimental design – in order to move on the debate (Rideout and Gray, 2013: 345–347). However, we contend that such approaches can only ever tell us what has worked they can only provide a score for a particular programme at a particular time. The crucial elements of increasing scientific knowledge – cumulation, theory testing and improving and deriving knowledge from tacit understanding and experience – are ruled out by models that prejudge what is to be proven (Hammersley, 2001). Realist evaluation (RE) has been developed specifically to explore and evaluate complex social programmes (Pawson, 2006, 2013; Pawson et al., 1997; Wong et al., 2013). It aims to address weaknesses such as the incomplete knowledge developed through experimental methods, systematic review and meta-analysis, where findings can be mixed or inconclusive and do not provide sufficiently useful explanations to practitioners or policymakers about what might work, for whom, and why. When applied to evaluating social programmes, such methods can obfuscate complexity and lead to artificial and misleading results:

…hypotheses are abridged, studies are dropped, programme details are filtered out, contextual information is eliminated, selected findings are utilized, averages are taken, estimates are made…this is all done in an attempt to wash out ‘bias’…however, in this purgative process the very features that explain how interventions work are eliminated from reckoning. (Pawson, 2006: 42)

A basic realist assumption is that programmes are ‘complex interventions, introduced into complex social systems’ (Pawson, 2013: 33); and, as such require a different approach to evaluation which takes this into account. In the next section, we reflect on our motivation to evaluate and introduce relevant principles of RE.

RE as an alternative approach to navigating complexity

Weiss (1987) notes that knowledge generated through the evaluation of programmes can serve different functions: a warning (that things are going wrong), guidance (direction for improving policies and programmes), reconceptualization (making sense of activities and outcomes) and mobilization of support (stiffen support for, or weaken adherence to, a particular position). Our lived experience as practitioners in the field of EE is that competitions and competitive pedagogies are ubiquitous. The expansion of competitions has been charted in historical accounts of EE in higher education (HE) and schools (Katz, 2003; Sukarieh and Tannock, 2009). Competitions have been identified as a staple part of the extra-curricular EE menu in universities (Preedy and Jones, 2015), and the most frequent and familiar activity delivered in schools (Mann et al., 2017). For us, they are an example of ‘taken-for-granted’ practice, which Fayolle (2013) argues should receive more critical attention. As a result, we employ an RE approach to counteract the taken-for-granted assumptions and ‘musty sameness’ of existing policy (Weiss, 1987: 16) by assessing EE competitions in European guidance over a 10-year period.

RE (sometimes known as ‘realistic evaluation’) is an approach and logic of thinking evolved specifically for researching and evaluating complex, socially contingent programmes (Pawson & Tilley, 1997). From the RE perspective, whenever a programme is implemented it is testing a ‘theory’ about what might change and how (Westhorp et al., 2011). As such, it aims to surface and track ideas inherent in the design and delivery of social programmes and to explain why complex interventions work (or fail), in order to provide the policy and practice community with new ideas and knowledge to achieve or improve outcomes for intended beneficiaries. Researchers have developed protocols and standards for realist review and realist synthesis (Pawson et al., 2005; Wong et al., 2013), identifying 19 steps that can be followed systematically. But the logic of RE and methodologies is also a resource that ‘imbues a way of thinking’ (Astbury, 2018: 75) and can be adapted and applied flexibly – for example, to isolate and investigate a particular element of a policy or a programme, and its underlying theories (Pawson, 2006).

Pawson (2006) describes a number of assumptions and expectations inherent in the RE approach that are relevant to the adaptive approach undertaken in our study and that have wider implications for the evaluation of EE programmes and interventions:

the ‘same’ intervention will meet with success and failure (and everything in between) when applied in different contexts and settings;

tracking successes and failures in programmes will lead to elements of explanation about the reasoning and reactions of different stakeholders; and

we should examine which intermediate outputs need to be in place for successful outcomes to occur.

As practitioners involved in the design and delivery of EE programmes, these assumptions align well with experience. Interventions are received differently, aspects appear to ‘work’ for some and not for others and there are always contextual issues – at individual and institutional levels and in the wider environment – which influence the reactions of participants and the patterns of outcomes. We were therefore keen to adapt the logic of RE to make explicit the assumed outcomes of EE competitions and purposefully search for alternative and unintended outcomes so as to critically enrich perspectives and encourage a reconceptualization. Criticism of such adaptive approaches are based on the lack of clarity and reproducibility of studies (Pawson et al., 2004). But from a realist philosophical standpoint, standardization and reproducibility in the evaluation of complex social programmes – as they are presented in scientific experimentation – are an impossibility. Furthermore, researchers using RE question whether objectivity in science even stems from procedure, standardization and reproducibility, and instead assert that ‘validity rests on refutation rather than replication’ (Pawson et al., 2004: 38). The consequence is that researchers should surface the logic of their reasoning, and try to show their workings; but ultimately all findings should be considered tentative and fallible and as existing to expose to criticism a theory about how a programme works (Pawson et al., 2004: 38). In the next section, we describe how the realist logic of thinking was applied in the context of our chosen policy setting – European policy and guidance on EE over the 10-year period from 2006 to 2016.

Methodology

Our approach is inspired by the logic of RE, which recommends researchers undertake an act of organized scepticism, in which claims about a programme (in this case, EE competitions) are exposed to critical scrutiny through ‘precise, vexatious cross-examination’ (Pawson, 2013: 108). Such an approach ensures that alternative theories etc etc that alternative theories and unintentional outcomes receive as much research attention as do the widely promised benefits. We surface the logic of our reasoning by describing the process of our study. Table 1 summarizes the stages of our research and related questions, which are detailed below.

Table 1.

Research stages and related questions.

Stages	Research question
1. Identifying a focus	To what extent do competitions and competitive pedagogies feature in European policy and guidance?
2. Data extraction and initial synthesis	What are the benefits and outcomes declared in policy and guidance?
3. Cross-examination of short and intermediate outcomes declared in policy and guidance	What rival theories can help explain the mixed results of competitive EE experiences?

Note: EE: entrepreneurship education.

Stage 1: Identifying a focus for study

RE does not have the same hierarchy of evidence that exists in scientific research; instead, it seeks out any specific wisdom about how a programme may (or may not) work. This opportunity to study everything can lead to the researcher becoming overwhelmed with data, and therefore identifying a focus is important. As an authoring team that spans England and Spain and has current and historical involvement in European projects to develop enterprise, entrepreneurship and social innovation projects in school settings, we were interested to explore how EE competitions were positioned in policy and guidance issued by the European Commission (EC). In the last decade, the EC has acted as a catalyst and a facilitator in the promotion of EE and social cohesion), and in 2004 it communicated the launch of an action plan for the ‘European Agenda for Entrepreneurship’ (EC, 2004). One of the outcomes of this communication was a conference - ‘The Oslo Agenda for Entrepreneurship Education in Europe’ (EC, 2006), - and it signalled the start of of a decade of EE policy and practice promotion. Reviewing these documents provides a valuable overview of the policies and good practice that have been recommended to schools over the last decade, as well as capturing the national strategies, curricula and learning objectives of member state countries. Furthermore, we reason that this EC guidance is promoted to national governments across 28 member states and aims to influence those who have responsibility for developing policy for entrepreneurship and enterprise in education, and those involved in providing EE experiences (e.g. teachers, charities or enterprise promotion groups). By way of example, there are more than 42 million young people in lower and upper secondary education across Europe who are the target of ‘entrepreneurship and creativity’ policies (EC, 2015). The body of policy and guidance we chose to focus on will be used by a variety of actors to inform and justify decisions made at many levels: by teachers in the classroom, by school leaders at an institutional level, by programme commissioners at a district level, and finally, by national governments who decide how public money is spent on EE. We identified that the Oslo Agenda (2006) offered a useful line in the sand at which to start our study and extend up to and include the report Entrepreneurship Education at School in Europe (2016). This body of work consists of 11 publications, including state-of-the-art reports on policy and practice, guidance and case studies focused on EE in non-HE settings (see Online Supplemental Appendix).

Stage 2: Data extraction and initial synthesis

Based on the scope we identified, school-focused EC policy documents from 2006 to 2016 were studied, searching for the inclusion of the terms ‘competitions’, ‘contests’, ‘prizes’ and ‘awards’. When these terms were found, the context of their inclusion was logged and direct comments were collated. Pawson et al. (2004) describe how researchers may make use of ‘data extraction forms’ to assist the sifting and sorting of materials, and we used four initial headings to organize information: year of publication, policy document name, context of a term’s entry in the document and direct quotations. This process equates to what Pawson et al. (2004) call ‘theory tracking mode’, in which documents are scoured for ideas on what an intervention is and what it is meant to be doing. These ideas are then highlighted, noted and given an approximate label. Our data extraction form (available in the Online Supplemental Material), included text which related to competitions, contests, prizes and awards. Whole direct quotations were harvested and ascribed an approximate label (these are identified in Tables 3 and 4). In some cases (EC/CEDEFOP, 2011), the use of the word competition was so frequent, that judgements were made to extract representative text which would most add to theory development. We provide an example of our form in Table 2, with sample extractions from different years. Then we present an initial synthesis of how competitions are characterized in policy before cross-examining the assumed short and intermediate outcomes.

Table 2.

Selected examples from data extraction form – terms of search: competition, contest, award, prize in (non-HE) European policy and guidance 2006–2016.

Year	Document	Context in which term is used	Direct quotations	Approximate labels
2006	Oslo Agenda for Entrepreneurship	Appears in Section F, ‘Communication Activities’ as a method which could be applied to any of the activities/programmes throughout the document.	‘F2 Celebrate entrepreneurship education activities and programmes that work well, by organising awards and competitions’. (p. 4)	Part of an EE strategy. Dissemination activity.
2012	Entrepreneurship Education at School in Europe (EC/EACEA), National Strategies, Curricular and Learning Outcomes	In the section ‘National strategies, current initiatives’, competition is included in the description of current activities in different member states.	Czech Republic: ‘…the teachers offer students professional help. They help students to discover and develop their abilities through school or national competitions and include activities in instruction that familiarise students with career opportunities’. (p. 41)	Develops skills.
2013	Entrepreneurship Education: A Guide for Educators (EC)	In the chapter ‘Examples of practice in Initial Teacher Education’, within the entrepreneurial or innovative teaching methods and pedagogies section, competition and its benefits are included within case studies.	‘Competition engages the community and motivates teachers. JA-YE sees competition as crucial to engaging with the local community and media. Judges of these competitions are typically from the business community, politicians and local/national education authority, which help to get their support behind the project. Several awards are given out to mini-companies; teachers and schools are also awarded, and this kind of recognition often creates positive publicity both for the activities but also for the schools and the teachers’. (p. 83)	Engages teacher participation. Dissemination activity. Private sector engagement. Rewards students. Rewards teachers.
2015	Entrepreneurship Competence: An Overview of Existing Concepts, Policies and Initiatives (EC/JRC)	The report aims to define the state of play in entrepreneurship education, combining insights from a literature review, an inventory of selected initiatives and in-depth case studies.	JEP Programme (Ireland) ‘Besides, using a competitive environment as the key pedagogical approach proved to be inadequate at primary education level creating unhappiness among students and teachers’. (p. 126)	Teaching method.
2016	Entrepreneurship Education at School in Europe (EC/EACEA), National Strategies, Learning Outcomes and Curricula	Included in member states’ strategies for developing entrepreneurship in education, features in the Lithuanian national information sheet under ‘explicit integration into the curriculum’.	‘Entrepreneurship education is a cross-curricula objective at all levels of education through the “National Programme of Economics and Entrepreneurship” for basic education. This is based on the key competences including entrepreneurship, communication, learning to learn and citizenship. The programme provides possibilities for teaching and learning in different ways, e.g. student companies and business competitions’. (p. 165)	Part of an EE strategy. Develops skills. Teaching method. Good practice model.

Note: EE: entrepreneurship education; HE: higher education; JEP: Junior Entrepreneur Programme.

We were struck by how the positioning of competitions shifts dramatically after their first mention. They are initially described as an ‘effective communications activity’ in The Oslo Agenda, but within a decade the method is qualified as ‘a learning form where competitive elements are used in order to achieve better learning outcomes’ (Komarkova et al., 2015: 84). The summary presented in Table 3 demonstrates how competitions are characterized.

Table 3.

Characterization of competitions in European policy and guidance, 2006–2016.

Approximate label	EC (2006)	EC (2009)	EC (2010)	CEDEFOP (2011)	EC (2011)	EC/EACEA (2012)	EC (2012)	EC (2013)	EC (2015)	EC/JRC (2015)	EC/EACEA (2016)	No./11 reports
1. Part of an EE strategy	*	*	*	*	*	*	*	*	*	*	*	11/11
2. Good practice model for delivery		*	*	*	*	*	*	*	*	*	*	10/11
3. Teaching method		*	*	*	*		*		*	*	*	8/11
4. Dissemination activity	*	*	*	*			*		*		*	7/11
5. Private sector engagement		*		*	*			*		*	*	6/11
6. Engages teacher participation		*						*	*		*	4/11
7. Incentivizes student participation			*	*		*						3/11
8. Assessment method				*		*				*		3/11
9. Facilitates cash prize/finance		*		*								2/11

Note: EE: entrepreneurship education.

Competitions feature as an integral part of strategy, a model of good practice and a teaching method. National, regional and local competitions successfully engage students as well as incentivizing teachers. They are the vehicle that drives young people into performing to the best of their ability and the nature of competition itself is assumed to transform learning and outcomes:

Business planning/ideas competitions…help young people pursue their entrepreneurial ideas and ambitions. (CEDEFOP, 2011: 16)

…[students]…discover and develop their abilities through school or national competitions. (EC/EACEA, 2012: 41)

…competitive elements are used in order to achieve better learning outcomes…(EC/EACEA, 2016: 84)

Competitions feature as a method for engaging the private sector. They are a means of raising the profile of activities, attracting media interest and therefore increasing the commitment of the private sector. The competition itself is seen as crucial (rather than simply holding a celebration event), because it enables the recruitment of judges. These are people from the business community, local politicians or education authorities who help to get support behind their project (EC, 2013: 87).

Competitions are also characterized as an assessment method and pedagogical approach. Assessment is achieved through the critique and evaluation provided by business people, measurement against the performance of peers and summative assessment provided by performance in competitions and pitches. Finally, a significant number of reports (73%) included competitions and competitive learning in sections on ‘teacher development’, ‘teacher support’ or ‘teaching materials’, recommending the method in content or case studies as a technique which educators should apply in classroom situations to achieve entrepreneurial learning outcomes. Such advice may have contributed to the extensive promotion and adoption of the competition method. This is reflected in one state-of-play report which notes that ‘…traditional start-up methods (pitches, competitions, events, business or idea plan) are, to some extent, and often in an adapted way, applied across all levels of education…’ (EC/JRC, 2015: 65). According to this report, learning-by-doing combined with collaborative and competitive teaching methods is the most common pedagogical approach in EE. Indeed, successive reports position competition as a teaching method in its own right, describing it as something delivered not just by providers, or as part of an extra-curricular activity, but by entrepreneurship educators at all phases of education as part of their entrepreneurial pedagogical toolkit.

Table 4 presents the declared benefits of competitions, with four of them (better employability, better start-up rates, higher earning and economic growth) describing significant long-term impact.

Table 4.

Benefits of competitions in European policy and guidance, 2006–2016.

Approximate label	EC (2009)	EC (2010)	CEDEFOP (2011)	EC/EACEA (2012)	EC (2012)	EC (2013)	EC (2015)	EC/JRC (2015)	EC/EACEA (2016)	No./11 reports
A. Develops skills	*	*	*	*			*	*	*	7/11
B. Motivates students	*		*				*		*	4/11
C. Rewards teachers	*	*				*	*			4/11
D. Students learn from/are inspired by peers		*	*		*					3/11
E. Rewards students	*					*	*			3/11
F. Better start-up rates	*						*			2/11
G. Increased entrepreneurial intention							*			1/11
H. Better employability							*			1/11
I. Higher earnings							*			1/11
J. Economic growth							*			1/11

Note: Short-term and intermediate outcomes expected for students in bold type were the subject of the realist cross-examination.

Among all the benefits and positive outcomes declared for competitions, there is just one case where concern is reported. Promoters of the Junior Entrepreneur Programme in Ireland report that, ‘Initially, the JEP programme was based on a competition with one winner. During the pilot phase, the feedback showed this competitive environment had negative effects, creating unhappiness among teachers and pupils’ (EC/JRC, 2015: 59). For the first time in European policy, a note of caution is sounded about the widespread use of entrepreneurship competitions, at least for lower education levels. The next stage in our research process was to purposefully search for more such instances, and to look for theories and research that would add to this alternative perspective. The results are presented in the next section.

Stage 3: Cross-examining short-term and intermediate outcomes declared in policy and guidance

Pawson describes the importance of organized scepticism, in which claims about a programme are exposed to critical scrutiny through ‘precise, vexatious cross-examination’ (Pawson, 2013: 108), so that alternative theories and unintentional outcomes receive as much research attention as the widely promised benefits. To focus this element of our study, we chose to limit our cross-examination to the most frequently cited short and intermediate benefits declared for students. Policy and guidance most commonly declares that competitions develop students’ skills, that they are motivating, that students are inspired by their peers in competitions and that students find competitions rewarding.

We focus our cross-examination on these outcomes for two reasons. First, as an authoring team including teacher educators, we are sensitive to Guskey’s (2002) model of evaluating professional development. He identifies impact on students as the bottom line for educators. Does an activity benefit students? Does it achieve its stated goals? Were there unintended outcomes? If educators do not see a positive impact on students, he suggests, they will not be motivated to continue with and develop a practice (Guskey, 2002). Second, we are minded of the realist call for greater inspection of the ‘implementation chain’ – that is, what intermediate outcomes should be in place to lead to longer-term impacts (Pawson, 2006: 29). We speculated how likely it was that long-term benefits would be realized if short and intermediate benefits were not secured. The inherent logic of a programme and its theory may start to break down if such outcomes are not achieved. As such, we purposefully sought out research from other fields (e.g. education and psychology) which would offer alternative theories on the short-term and intermediate outcomes declared in policy and guidance. We present our cross-examination in the following subsections and conclude with a discussion of social context, which emerged as an important factor to consider.

Skill development

Seven reports identified that one of the benefits of competitions is that they are successful in developing students’ skills. Positive outcomes are facilitated by the nature of the competitive model itself, which enables young people to ‘…gain specific…skills for handling various social situations…’ (EC/EACEA, 2012: 41), ‘pursue entrepreneurial ideas and ambitions’ (CEDEFOP, 2011: 16), which promotes ‘creative ideas, teamwork, solving of real problems’ (EC/EACEA, 2012: 57) and which means that ‘students score higher on questions about self-confidence, cooperation skills and motivation in school’ (EC, 2009: 36).

One argument for competition is that it is part of the DNA of society, evident in relationships and dynamics at home and at work, and in hobbies and entertainment (Fülöp, 2009; Vansteenkiste and Deci, 2003), and therefore learning to compete is learning the skills required for life as well as for work. However, educational research has looked at the effects of different tasks and structures on students’ learning and outcomes and calls into question the uncritical use of competitions as a vehicle for such skill development. While competition can increase productivity or performance on rote, speed or basic tasks (Johnson et al., 1981; Slavin, 1977), it can undermine performance in problem solving and creative tasks (Amabile, 1983; Butler, 1989; Johnson et al., 1981). ‘Creativity’ and ‘problem solving’ often feature on the entrepreneurial competencies wish list. For example, the recent European Entrepreneurship Competence Framework (EntreComp), includes creativity as a specific competency, and problem-solving features in descriptors for other skills, such as taking the initiative and spotting opportunities (Bacigalupo et al., 2016). Further research is required then to clarify the declared benefits and actual effects of competition in EE on the development of such skills. Theorists distinguish between ‘performance goals’ and ‘mastery goals’ and the different ways these conceptions influence the development of skills. Central to a performance goal is the idea that one’s skill is evidenced by doing better than others, and that this performance is publicly recognized (Ames, 1992; Dweck, 1986). As a result, learning and skills development is viewed as a way to achieve a desired goal, rather than as an end in itself. As a consequence, if considerable effort is invested but does not lead to ‘success’ it can lead to a negative evaluation of competences (Ames, 1992), and disengagement from developing that skill. In contrast, mastery goals focus on the intrinsic value of learning and using efforts to develop skills and competences (Ames, 1992; Dweck, 1986). So, to summarize, a competitive process may incentivize a prioritization of performance outcomes over skill development. For example, in a group working on a competitive pitch, those who might most benefit from developing presentation skills are least likely to take the lead, despite being most in need of development (McCollough et al., 2016). Competition is not then, in and of itself, a strategy that guarantees the development of skills and competencies. There is a risk that competitions will focus students on the ends – winning (or losing) – rather than on the valued building of new skills and competences (Bergin and Cooks, 2000; McCollough et al., 2016). Furthermore, if students’ efforts do not lead to success, their sense of competency may be diminished and their future interest put in jeopardy, potentially leading to the opposite of developing ‘can do’ entrepreneurial skills and competences (can’t do, won’t do).

Motivation

A positive outcome, commonly cited in European policy documents regarding the use of competitions in EE, is that they motivate students. Such benefits are described generically – for example, ‘Business planning ideas/competitions…motivate young people’ (CEDEFOP, 2011: 14) – and specifically, in that they are seen as motivating young people to ‘perform to the best of their ability’ (CEDEFOP, 2011: 108) and to ‘perform to the best of their ability’ (CEDEFOP, 2011: 108), and in that young people report higher self-perceptions of motivation (EC, 2009: 36). But Self-Determination Theory defines the act of motivation as requiring the subject to be ‘moved to do something’ (Ryan and Deci, 2000a: 69). Proponents Deci and Ryan (1985) recognize that there are different types of motivation. For example, intrinsic motivation leads to the performance of an activity for its inherent satisfaction, for fun, challenge and out of curiosity, whereas extrinsic motivation refers to doing something because it leads to a separable outcome (Ryan and Deci, 2000a: 55). Competitions are a special type of extrinsic activity per se as they necessitate measuring one’s own performance against that of others, which can tend to decrease intrinsic motivation (Ames, 1984a; Ames and Ames, 1984).

Arguments for competition in education suggest that it can play a role in catalysing increases in performance (Slavin, 1977), but the use of competitive pedagogy as an effective strategy to motivate pupils in classroom environments remains largely contested in educational research (Deci et al. 1981; Good and Brophy, 2008). Ultimately, though, competitive processes will be qualitatively different depending on cultural, situational and personal/individual factors, such as the way competitors view each other, the process they are involved in and the way they cope with winning and losing (Fülöp, 2000, 2009). A crucial process element we wish to draw attention to is the difference between the motivations (and subsequent experiences and derived meaning) of those participating in voluntary business contests compared to those participating in compulsory competitions embedded in the curriculum. The former participants may introduce ‘volunteer bias’, where the nature of the volunteers causes the positive outcome as opposed to the intervention itself – an effect that is difficult to control for (Goldstein et al., 2015; Heiman, 2002; Keiding and Louis, 2016). However, being forced to participate in a competition through a compulsory curriculum activity changes the dynamic, and the potential effects, of the act itself. Competition can be experienced as ‘coercive’ (Good and Brophy, 2008) and may be ineffective as a motivational strategy for all pupils (Meece et al., 2006). Motivation may be diminished through the focus placed on winning (Ames, 1984b; Butler, 1989; Deci et al., 1981) and the exposure to public failure (Rahal, 2010). Students who repeatedly perform poorly in comparison to peers will find little appeal in competitions (Good and Brophy, 2008). A crucial insight to heed is the extent to which competition is experienced in different ways by different participants. The dimensions that are seen to underlie intrinsic motivation (perceived competence and perceived effort, enjoyment and interest, pressure and tension) will result in different motivations being derived by different participants (Ryan and Deci, 2000b), according to their performance and the perceived meaning they take from it, and these individual perceptions can have a greater impact on intrinsic motivation than competition outcomes.

Inspired by peers

Another declared benefit of competitions is the opportunity to learn from and be inspired by peers. This includes the ‘inspiration’ young people gain from each other (EC, 2010: 45), and how ‘valuable learning’ is achieved by observing and imitating those whose ‘techniques and skills are greater’ (CEDEFOP, 2011: 77). But psychologists have identified that peer excellence can have a demoralizing effect if students believe that their peers’ excellent level of performance is out of their reach. Rogers and Feller’s experiments (2016) showed that incidental exposure to exemplary peer performances could undermine motivation and success, leading to de-identification with the relevant domain and, finally, quitting. The ‘discouragement-by-peer-excellence-effect’ challenges the notion that students will automatically be inspired by and learn from their peers, since being exposed to their excellent performance may make them feel less capable of performing at that level. Crucially, this changed belief appears to decrease student performance. Social Comparison Theory (Festinger, 1954) states that our sense of self is determined by making comparisons between ourselves and others in order to evaluate ourselves, and can help explain these reactions. If students compare themselves and their performance unfavourably with others, it threatens, rather than inspires, their self-worth and motivation (Meece et al., 2006). This effect is reflected in the findings of one empirical study which identified the negative impact of participating in the regional finals of an EE competition on students from lower socio-economic backgrounds: ‘When meeting other groups at regional meetings or at competitions, the pupils from the lower socio-economic background felt underprivileged, backward and less capable’ (Heilbrunn and Almor, 2014: 8). These students scored lower in terms of self-efficacy, and perceived entrepreneurship as less feasible and less desirable after the intervention.

Rewards

A number of reports identified that competitions provide important rewards for students (EC, 2009: 41, 2010: A45; EC/EACEA, 2016). The assumed benefit is that competitions reward the ‘best’ (EC, 2009: 41), provide recognition (EC, 2010: A45) and that rewards in competition ‘keep motivation high’ (EC, 2009: 40). Deci et al. (1999) distinguish between different types of rewards and their effects. Rewards that are perceived to be controlling (e.g. contingent on task engagement, task completion or quality of performance) can have negative effects on intrinsic motivation, whereas rewards that are informational (providing feedback or recognition) can provide satisfaction and have positive effects on intrinsic motivation. In competition, the reward – ‘winning’ – is extrinsic to the activity itself, and is dependent on beating opponents (Vansteenkiste and Deci, 2003). In many cases, rewards have conflicting effects and are dependent on context, so many factors must be taken into account. A meta-analysis of the effects of extrinsic rewards showed that they had a substantial undermining effect on intrinsic motivation (Deci et al., 1999). Reinforcing a previous potential red flag about the significance of student age, tangible rewards were more detrimental for children than college students, and verbal rewards were less enhancing for children than college students. This underscores the role of personal context and that interpretations of competitive outcomes must be considered from the actor’s perspective, rather than simply taking stock of who wins or loses (McAuley and Tammen, 1989). Deci et al. (1999) conclude that understanding the effects of rewards requires a consideration of the interpretation that recipients will give to the rewards in relation to their feelings of self-determination and competence. Of course, the opposite of being rewarded by winning is losing, and this can have significant negative effects (Good and Brophy, 2008; Vansteenkiste and Deci, 2003). Good and Brophy (2008) discuss the development of a loser’s psychology, where individuals and teams feel embarrassed or humiliated, and those who consistently lose may suffer losses in confidence, self-concept and enjoyment.

Finally, and perhaps more dramatically, winners are more likely to engage in unethical behaviour due to an inflated sense of entitlement and, as a result, unethical behaviour may cascade from being rewarded in competitive settings, as noted by Schurr and Ritov (2016). While that finding was from research conducted with adults, and therefore the effects have not been demonstrated in the classroom, it can be taken as a provocation for critical thought on unintended consequences, the influence on social relationships and possibilities for future enquiry.

Social context

Social context emerged out of our cross-examination as a factor that is likely to influence outcome patterns for participants in EE competitions. It has previously been argued that EE can be considered a success if it dampens unrealistic expectations and fulfils a type of ‘sorting’ according to aptitude and ability (Von Graevenitz et al., 2010). However, research in mainstream education has shown that such processes are rarely neutral in school settings, and children and young people from lower socio-economic groups are more likely to be failures due to the expectations of others and the opinions and actions of decision makers (Boaler et al., 2000). Heilbrunn and Almor (2014) showed that an overall positive statistical effect for a competitive intervention was misleading when social context was taken into account. Lower socio-economic students were practically (as well as perceptually) disadvantaged in the organization and experience of competing, with less personal and institutional support and less individual capability to complete tasks (such as phoning sponsors), as well as feeling inadequate compared to better equipped peers at regional finals.

The significance of these observations is that, clearly, the field on which teams and schools play in EE competitions is far from level. Suggesting that competition in these circumstances is a fair and effective ‘sorting’ process may well result in young people being alienated from entrepreneurship according to context (not according to ability and interest), reproducing social inequalities rather than ameliorating them. Consider a key component of competitions – the public presentation, or pitch. This is identified in three reports as a valuable element of competitions, through which students are evaluated by others and assess their own performance. The pitch represents a litmus test for finalists, but may well favour teams from socially advantaged backgrounds. Patterns of talk and interaction constitute a manifestation of class differences (Bernstein, 2009; Savage, 2015), and elevator pitches and other forms of interaction with the jury mean that socially advantaged teams with the social skills to make the right impression may be more likely to be crowned winners.

Summary

In summary, our cross-examination indicated that competitions can lead to unforeseen outcomes, especially for those in ‘at risk’ groups (e.g. students from lower socio-economic backgrounds). In particular, two distinct but closely related characteristics of competitions can result in unintended effects:

Unhealthy competitions implicitly reward advantaged students (Shindler, 2009).In the same way that Petersen and O’Flynn (2007) observed that Duke of Edinburgh Award participation could be viewed as a neo-liberal technology which enabled further benefit for already advantaged students, Heilbrunn and Almor (2014) showed how the satisfaction and benefit that higher socio-economic students reported masked the deterioration experienced by lower socio-economic students. The resources which advantaged students are able to bring to bear, as individuals and in terms of school organization, teacher commitment and family capital, result in a competition that is skewed, and more likely to reward them.

Winners are able to use their victory as social or educational capital at a later time (Shindler, 2009). Bold claims are made about the positive impact on employability and personal, professional and entrepreneurial success following participation in competitions. However, such new educational strategies may constitute a constraint rather than an opportunity, depending on social class (Van Zanten, 2005). Families vary in the extent of their resources to take advantage of such strategies, and those with existing advantage can therefore consolidate and increase their position in relation to others. Researchers have noted that students from independent schools are over-represented in enterprise competitions (Athayde, 2012; Huddleston et al., 2012), and a similar picture emerges when looking at the state sector in the United Kingdom, with grammar school pupils outnumbering alumni of the non-selective sector (Mann and Kashefpakdel, 2014). Essentially, EE competitions may enable confident, socially and culturally advantaged young people to gain additional social and educational capital that will benefit them further at a later time and thus, in effect, create greater disadvantage for their less well equipped peers.

Limitations

Our study is limited in its scope in that we use existing research to surface and challenge theory. But we believe it points to the usefulness of RE as a theoretically driven approach which is committed to exploring, critiquing and evaluating programmes to improve outcomes for participants.

Conclusions

An aim of RE is to provide a deeper and fuller account of reality so that practitioners and policymakers have better explanations on which to base their policy decisions and practice. An underpinning principle of the paradigm is that complex, socially contingent interventions will always have different effects on different participants in different circumstances. As well as supporting a critical evaluation of competitions, the theory-driven way of thinking that characterizes RE is especially useful for those designing, delivering, evaluating or researching complex social programmes. In this article, the identification of unintended outcomes from competitions and competitive learning processes makes it clear that the positive benefits declared in policy and guidance are far from assured. We have gone on to develop this strand of thinking and identify and communicate crucial factors that appear particularly significant in influencing outcome patterns (Brentnall et al., in press). An issue that emerged from this research as important is that of social context. We are reminded of widely cited education researcher Diane Reay, who asserts that the iniquitous effects of social class in schooling constitute a ‘monster that grows in proportion to its neglect’ (Reay, 2006: 289). Social context is often stripped out of EE, rendered invisible in research and absent in the policy picture, where one-size-fits-all enterprise policy initiatives are wielded blindly (Athayde, 2012). This sentiment might prompt EE researchers to reflect that the wider social and economic context of education is inescapable for children and young people in schools, and so is crucial for ethical research and practice which might want to challenge (or, at the very least, not reproduce) disadvantage.

This novel application of RE to EE competitions has revealed the extent to which competitions and competitive pedagogies are handed down as a method which delivers significant short-term and long-term benefits. The taken-for-granted position presents competitions as an effective model for developing skills, and motivating, inspiring and rewarding students, but applying a realist logic of enquiry reveals an alternative scenario. It illustrates that competitions can diminish competency and interest, and alienate young people from entrepreneurship before anyone can reasonably guess what their futures might hold. Of most concern is that competitions and competitive pedagogies are promoted in school-focused policy and guidance as an effective approach for all students, of all ages, in all contexts; as a result, young people will increasingly be conscripted into such activity through compulsory curricula or extra-curriculum activities. We hope that the act of organized scepticism undertaken here enables practitioners, researchers and policymakers to look beyond the intuitive appeal of competitions and competitive pedagogies and scrutinize and test theory which might add a sense of caution to the current uncritical recommendations and widespread application of such interventions. While competitions in EE are presented as engaging and effective interventions, our cross-examination demonstrates that the declared benefits and positive outcomes are by no means guaranteed.

Supplemental material

Supplemental Material, IHE_-_Brentnall_et_al_supplementary_material - The contribution of realist evaluation to critical analysis of the effectiveness of entrepreneurship education competitions

Supplemental Material, IHE_-_Brentnall_et_al_supplementary_material for The contribution of realist evaluation to critical analysis of the effectiveness of entrepreneurship education competitions by David Higgins, Laura Galloway, Paul Jones, Pauric McGowan, Catherine Brentnall, Iván Diego Rodríguez, and Nigel Culkin in Industry and Higher Education

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

Supplemental material

Supplemental material for this article is available online.

References

Amabile

(1983) The social psychology of creativity: A componential conceptualization. Journal of Personality and Social Psychology 45(2): 357.

Ames

(1984a) Achievement attributions and self-instructions under competitive and individualistic goal structures. Journal of Educational Psychology 76(3): 478.

Ames

(1984b) Competitive, cooperative, and individualistic goal structures: a cognitive-motivational analysis. Research on Motivation in Education 1: 177–207.

Ames

(1992) Classrooms: goals, structures, and student motivation. Journal of Educational Psychology 84(3): 261.

Ames

(1984) Systems of student and teacher motivation: toward a qualitative definition. Journal of Educational Psychology 76(4): 535–556.

Astbury

(2018) Making claims using realist methods. In: Emmel

Greenhalgh

Manzano

Monaghan

Dalkin

(eds) Doing Realist Research. Thousand Oaks: Sage, pp. 59–78.

Athayde

(2012) The impact of enterprise education on attitudes to enterprise in young people: an evaluation study. Education + Training 54(8/9): 709–726.

Bacigalupo

Kampylis

Punie

. (2016) Entrecomp: The Entrepreneurship Competence Framework. Luxembourg: Publication Office of the European Union.

Bergin

Cooks

(2000) Academic competition among students of color: an interview study. Urban Education 35(4): 442–472.

10.

Bernstein

(2009) Class, Codes and Control. London: Routledge.

11.

Boaler

Wiliam

Brown

(2000) Students’ experiences of ability grouping – disaffection, polarisation and the construction of failure. British Educational Research Journal 26(5): 631–648.

12.

Brentnall

Diego

Culkin

(in press) Enterprise education competitions: a theoretically flawed intervention? In: Higgins

Jones

McGowan

(eds) Creating Entrepreneurial Space: Talking through Multi Voices, Reflections on Emerging Debates. Bingley, UK: Emerald Publishing Limited, pp. 25–48.

13.

Butler

(1989) Interest in the task and interest in peers’ work in competitive and non-competitive conditions: a developmental study. Child Development 60: 562–570.

14.

CEDEFOP (2011) Guidance Supporting Europe’s Aspiring Entrepreneurs: Policy and Practice to Harness Future Potential. Thessaloniki: European Centre for the Development of Vocational Training.

15.

Chatterji

(2016) Causal inferences on the effectiveness of complex social programs: navigating assumptions, sources of complexity and evaluation design challenges. Evaluation and Program Planning 59: 128–140. DOI: 10.1016/j.evalprogplan.2016.05.009.

16.

Deci

Betley

Kahle

. (1981) When trying to win: competition and intrinsic motivation. Personality and Social Psychology Bulletin 7: 79–83.

17.

Deci

Koestner

Ryan

(1999) A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin 125(6): 627.

18.

Deci

Ryan

(1985) Intrinsic Motivation and Self-Determination in Human Behaviour. New York: PlenumPress.

19.

Dweck

(1986) Motivational processes affecting learning. American Psychologist 41(10): 1040–1048.

20.

EC (2004) Communication from the Commission to the Council, the European Parliament, the European Economic and Social Committee and the Committee of the Regions Action Plan. Brussels: European agenda for Entrepreneurship.

21.

EC (2006) The Oslo Agenda for Entrepreneurship Education in Europe. Brussels: European Commission.

22.

EC (2009) Entrepreneurship in Vocational Education and Training. Final Report of the Expert Group. Brussels: European Commission, DG Enterprise and Industry.

23.

EC (2010) Towards Greater Cooperation and Coherence in Entrepreneurship Education. Report and Evaluation of the Pilot Action High Level Reflection Panels on Entrepreneurship Education Initiated by DG Enterprise and Industry and DG Education and Culture. Brussels: European Commission, DG Enterprise and Industry.

24.

EC (2011) Entrepreneurship Education: Enabling Teachers as a Critical Success Factor. A Report on Teacher Education and Training to Prepare Teachers for the Challenge of Entrepreneurship Education. Brussels: European Commission, DG Enterprise and Industry.

25.

EC (2012) Building Entrepreneurial Mindsets and Skills in the EU. A Smart Guide on Promoting and Facilitating Entrepreneurship Education for Young People with the Help of EU Structural Funds. Brussels: European Commission, DG Enterprise and Industry.

26.

EC (2013) Entrepreneurship Education: A Guide for Educators. Brussels: European Commission, DG Enterprise and Industry.

27.

EC (2015) Entrepreneurship Education: A Road to Success. 13 Case Studies Prepared for the Study ‘Compilation of Evidence on the Impact of Entrepreneurship Education Strategies and Measures’. Brussels: European Commission, DG Enterprise and Industry.

28.

EC/EACEA/Eurydice (2012) Entrepreneurship Education at School in Europe. National Strategies, Curricula and Learning Outcomes. Brussels: Education, Audiovisual and Culture Executive Agency Eurydice and Policy Support.

29.

EC/EACEA/Eurydice (2016) Entrepreneurship Education at School in Europe. Eurydice Report. Luxembourg: Publications Office of the European Union.

30.

Fayolle

(2013) Personal views on the future of entrepreneurship education. Entrepreneurship & Regional Development 25(7–8): 692–701.

31.

Festinger

(1954) A theory of social comparison processes. Human Relations 7(2): 117–140.

32.

Fülöp

(2000) Teachers’ perception of the role of competition in their country: Hungary, Japan and the USA. Citizenship, Social and Economics Education 4(3): 142–158.

33.

Fülöp

(2009) Happy and unhappy competitors: what makes the difference? Psihologijske teme 18(2): 345–367.

34.

Gibb

(2002) In pursuit of a new ‘enterprise’ and ‘entrepreneurship’ paradigm for learning: creative destruction, new values, new ways of doing things and new combinations of knowledge. International Journal of Management Reviews 4(3): 233–269.

35.

Goldstein

Lynn

Muniz Terrera

. (2015) Population sampling in longitudinal surveys. Longitudinal and Life Course Studies 6(4): 447–475.

36.

Good

Brophy

(2008) Looking in Classrooms, 10th ed. Boston: Allyn & Bacon.

37.

Guskey

(2002) Professional development and teacher change. Teachers and Teaching 8(3): 381–391.

38.

Hammersley

(2001) On ‘systematic’ reviews of research literatures: a ‘narrative’ response to Evans & Benefield. British Educational Research Journal 27(5): 543–554.

39.

Hannon

(2005) Philosophies of enterprise and entrepreneurship education and challenges for higher education in the UK. The International Journal of Entrepreneurship and Innovation 6(2): 105–114.

40.

Head

(2008) Wicked problems in public policy. Public Policy 3(2): 101.

41.

Heilbrunn

Almor

(2014) Is entrepreneurship education reproducing social inequalities among adolescents? Some empirical evidence from Israel. The International Journal of Management Education 12(3): 445–455.

42.

Heiman

(2002) Research Methods in Psychology. New York: Houghton Mifflin.

43.

Horn

(2001) Knowledge mapping for complex social messes. Presentation to the ‘Foundations in the knowledge economy’ at the David and Lucile Packard Foundation.

44.

Huddleston

Mann

Dawkins

(2012) Employer Engagement in English Independent Schools. London: Education and Employers Taskforce.

45.

Johnson

Maruyama

Johnson

. (1981) Effects of cooperative, competitive and individualistic goal structures on achievement: a meta-analysis. Psychological Bulletin 89(1): 47–62.

46.

Katz

(2003) The chronology and intellectual trajectory of American entrepreneurship education: 1876–1999. Journal of Business Venturing 18(2): 283–300.

47.

Keiding

Louis

(2016) Perils and potentials of self-selected entry to epidemiological studies and surveys. Journal of the Royal Statistical Society: Series A (Statistics in Society) 179(2): 319–376.

48.

Komarkova

Gagliardi

Conrads

Collado

(2015) Entrepreneurship competence: An overview of existing concepts, policies and initiatives – Final Report (JRC96531). Luxembourg: Publications Office of the European Union. doi: 10.2791/067979

49.

McAuley

Tammen

(1989) The effects of subjective and objective competitive outcomes on intrinsic motivation. Journal of Sport and Exercise Psychology 11(1): 84–93.

50.

McCollough

Devezer

Tanner

(2016) An alternative format for the elevator pitch. The International Journal of Entrepreneurship and Innovation 17(1): 55–64.

51.

Mann

Dawkins

McKeown

(2017) Towards an Employer Engagement Toolkit: British Teachers’ Perspectives on the Comparative Efficacy of Work-related Learning Activities. (Occasional Research Paper 9). London: Education and Employers Taskforce.

52.

Mann

Kashefpakdel

(2014) The views of young Britons (aged 19–24) on their teenage experiences of school-mediated employer engagement. In: Mann

Stanley

Archer

(eds) Understanding Employer Engagement in Education: Theories and Evidence. London: Routledge, pp. 143–162.

53.

Matlay

(2006) Researching entrepreneurship and education. Education Training 48(8/9): 704–718.

54.

Meece

Anderman

(2006) Classroom goal structure, student motivation, and academic achievement. Annual Review of Psychology 57: 487–503.

55.

Pawson

(2006) Evidence-Based Policy: A Realist Perspective. Thousand Oaks: Sage.

56.

Pawson

(2013) The Science of Evaluation: A Realist Manifesto. Thousand Oaks: Sage.

57.

Pawson

Greenhalgh

Harvey

. (2004) Realist Synthesis: An Introduction. Manchester: ESRC Research Methods Programme, University of Manchester.

58.

Pawson

Greenhalgh

Harvey

. (2005) Realist review – a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy 10(1): 21–34.

59.

Pawson

Tilley

(1997). Realistic Evaluation. Thousand Oaks: Sage.

60.

Petersen

O’Flynn

(2007) Neoliberal technologies of subject formation: a case study of the Duke of Edinburgh’s Award scheme. Critical Studies in Education 48(2): 197–211.

61.

Pittaway

Cope

(2007) Entrepreneurship education: A systematic review of the evidence. International Small Business Journal 25(5): 479–510.

62.

Preedy

Jones

(2015) An investigation into university extra-curricular enterprise support provision. Education + Training 57(8/9): 992–1008.

63.

Rahal

(2010) Identifying and Motivating Underachievers. Focus On . Alexandria, VA: Educational Research Service.

64.

Reay

(2006) The zombie stalking English schools: social class and educational inequality. British Journal of Educational Studies 54(3): 288–307.

65.

Rideout

Gray

(2013) Does entrepreneurship education really work? A review and methodological critique of the empirical literature on the effects of university-based entrepreneurship education. Journal of Small Business Management 51(3): 329–351.

66.

Rogers

Feller

(2016) Discouraged by peer excellence: exposure to exemplary peer performance causes quitting. Psychological Science 27(3): 365–374.

67.

Ryan

Deci

(2000a) Intrinsic and extrinsic motivations: classic definitions and new directions. Contemporary Educational Psychology 25(1): 54–67.

68.

Ryan

Deci

(2000b) Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist 55(1): 68.

69.

Savage

(2015) Social Class in the 21st Century. London: Penguin.

70.

Schurr

Ritov

(2016) Winning a competition predicts dishonest behaviour. Proceedings of the National Academy of Sciences 113(7): 1754–1759.

71.

Shindler

(2009) Transformative Classroom Management: Positive Strategies to Engage All Students and Promote a Psychology of Success. Hoboken: Jossey-Bass.

72.

Slavin

(1977). Classroom reward structure: an analytical and practical review. Review of Educational Research 47(4): 633–650.

73.

Sukarieh

Tannock

(2009) Putting school commercialism in context: a global history of junior achievement worldwide. Journal of Education Policy 24(6): 769–786.

74.

Vansteenkiste

Deci

(2003) Competitively contingent rewards and intrinsic motivation: can losers remain motivated? Motivation and Emotion 27(4): 273–299.

75.

Van Zanten

(2005) New modes of reproducing social inequality in education: the changing role of parents, teachers, schools and educational policies. European Educational Research Journal 4(3): 155–169.

76.

Von Graevenitz

Harhoff

Weber

(2010) The effects of entrepreneurship education. Journal of Economic Behavior & Organization 76(1): 90–112.

77.

Weiss

(1987) Evaluation for Decisions: Is anybody there? Does anybody care? Plenary address for meeting of American Evaluation Association, Boston, MA.

78.

Westhorp

Prins

Kusters

CSL

. (2011) Realist Evaluation: An Overview. Wageningen: Centre for Development Innovation.

79.

Wong

Greenhalgh

Westhorp

. (2013) RAMESES publication standards: realist syntheses. BMC Medicine 11(1): 21.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.30 MB