Validating an individual innovation competence assessment tool for university

Abstract

University–industry collaboration produces networks that may be capable of innovations, such as novel products and services. The collaboration projects also need to benefit student learning, yet teachers have little clarity with regard to innovation competence development. Individual innovation competence is a set of personal characteristics, knowledge, skills and attitudes that are connected to create concretised and implemented novelties via collaboration in complex innovation processes. The paper reports on the findings from the development and validation of an individual innovation competence assessment tool. The aim is to determine which individual innovation competences are significant in university–industry collaboration and which of these competences are sensitive to educational interventions. The study used a three-phase method involving development of the questionnaire items, validation in teacher and student panels, and a pilot pre- and post-survey study. All seven domains of individual innovation competences were significant and sensitive to educational intervention (a multidisciplinary innovation project conducted with industry). The most responsive competence domains regarding change were concretisation and implementation planning skills, and project management skills. The paper concludes with application opportunities for the tool and recommendations for further research.

Keywords

Assessment entrepreneurship education individual innovation competence multidisciplinary innovation project university–industry collaboration

This paper reports the findings of the development and validation of a competence assessment tool that can be used to define individual innovation competence. University–industry innovation collaboration produces open research, development and innovation (RDI) networks that may be capable of generating innovations, such as novel products and services (Ankrah and Al-Tabbaa, 2015; Mäkimattila et al., 2015; Rantala and Ukko, 2018; Slotte and Tynjälä, 2003). Collaboration projects also need to benefit learning, and not only the organisations looking for innovations. This collaboration may promote participant innovativeness, which can be referred to as ‘individual innovation competence’ (Hero et al., 2017; IIC). However, a fundamental issue in innovation and entrepreneurship education is the hidden nature of the foundations that underpin its delivery and assessment (Fayolle et al., 2016; Neck and Corbett, 2018; Seikkula-Leino et al., 2010).

Previous research has found a need to understand and develop self-report assessment tools that can be integrated into innovation pedagogy (e.g. Keinänen et al., 2018; Nielsen 2015). The aim of this study was to define, develop and validate an IIC scale for the higher education–industry collaboration context. An IIC scale and an assessment tool based on the scale is needed as authentic university–industry innovation processes have very limited clarity in terms of teachers being able to assess projects and competence development at the individual student level (Helle et al., 2006). Thus, the pedagogy is very hard to improve.

According to Mitchelmore and Rowley (2010), if competence frameworks are to be used it is important to be able to measure competencies before and after any intervention and to prioritise those that would benefit from development for specific individuals. Self-assessment tools can be used to determine the respondent’s situation both before and after an intervention or work project to enable insights to be gained into the experience during the intervention (cf. Andrade, 2019). In the context of industry project work, it can be used to discover employee competence development during a project. In higher education, pre–post and post–pre assessment tools can be used to evaluate the impact of an instruction; that is, a course, programme or workshop (Hiebert et al., 2011). Teachers involved in innovation and entrepreneurship programmes can use the tools to determine the impact of their project-based pedagogy and teaching. By promoting rigour in the development of the scale in an authentic innovation project context, it is also possible to put it to wider use with industry partners and develop innovation competence in companies and public organisations.

Towards individual innovation competence

According to Mulder (2012), Mulder and Gulikers (2011) and Sturing et al. (2011), competence is defined as an entity resulting from the integration of knowledge, skills and attitudes that manifests itself in performance in a specific context and in concrete, authentic tasks. The competence needed in innovation processes relates to knowledge, skills and attitudes (see Zhuang et al., 1999), but the influence of individual characteristics also seems to be significant (Da Silva and Davis, 2011). Mulder (2012) has distinguished three perspectives for competence: behavioural functionalism, integrated occupationalism and situated professionalism. In this study, we follow Mulder (2012) in his definition of competence as situated professionalism, as it indicates that competence holds meaning only in a specific context in which professionals interact with each other.

The terms ‘competence’ and ‘competency’ are intertwined but distinct. Competence is the evaluation of performance in a specific activity, whereas competency is a class of factors that can be used to characterise individual abilities and their behaviours (see Michelmore and Rowley, 2010). Competences are attainable through experience, learning and coaching (Volery et al., 2015). Competence shows as behaviour in an activity in an authentic context and it should have an intention related to action (Spencer and Spencer, 1993).

Differences between innovation and entrepreneurial competences

Innovation and entrepreneurship competences seem to intertwine and overlap. Entrepreneurship competence has been defined as a part of innovation competence, and vice versa (Bjornali and Støren, 2012; Cerinšek and Dolinšek, 2009; Chell and Athayde, 2011; Edwards-Schachter et al., 2015; Gundry et al., 2014; Kasule et al., 2015; Santandreu-Mascarell et al., 2013; Waychal et al., 2011). According to Waychal et al. (2011), entrepreneurial abilities (along with creativity and achievement orientation) are factors of innovation as a competence. According to Gundry et al. (2014), innovation (in addition to risk-taking and proactive behaviour) is a central dimension of entrepreneurship. Proactive entrepreneurs who adopt a strategic orientation that permits flexibility and responsiveness are more likely to innovate. Entrepreneurial competence relates to actions where a business is started, transformed (Mitchelmore and Rowley, 2010) and grown (Bird, 1995). Entrepreneurial competences have been defined in many ways – including risk-taking, positive thinking, vision, intuitive decision making, creative problem-solving, managing interdependency, tolerating ambiguity and innovation (Bissola et al., 2017). In higher education, the ultimate purpose of entrepreneurship education is to help potential entrepreneurs launch new ventures and understand the consequences of their decisions, whereas the purpose of innovation programmes is to enhance the innovative performance of individuals and organisations (Maritz and Brown, 2013; Maritz and Donovan, 2015). However, IIC as a part of entrepreneurship competence often lacks a clear definition and differs from entrepreneurial competence.

Innovation competence manifests itself in context

There are several meanings for the term ‘innovation competence’, according to the context in which it is used. Most of the relevant research has examined the innovation competence of organisations (e.g. Kodama and Shibata, 2014; Wang, 2014), the country-, region- or area-level innovation competences of organisations (e.g. DiPietro, 2009), the innovativeness of things, such as innovative software (e.g. Lim et al., 2011) or innovativeness as consumer technology adoption (e.g. Manning et al., 1995), but not creating the technology or product. This study focuses on the individual-level human competence related to the development of innovations; that is, the creation of innovations as the collaborative work of several individuals.

Competence manifests itself in contexts and tasks (Mulder, 2012; Sturing et al., 2011). More specifically, competence is measured through behaviour – as an individual’s ability to act in an authentic situation. Innovation development, the context in which IIC is needed, relates to actions where concretised and implemented novelties are created via collaboration in complex innovation processes. These outcomes are understood as novelties that are made concrete, useful and implemented to convey value (mainly following Peschl et al., 2014; Quintane et al., 2011; Sawyer, 2006, 2009). They can take such forms as new services, products and processes, and marketing and organisational innovations (Oslo Manual, 2005).

Innovation development is often associated with teams of diverse individuals and networked multi-professional collaborations (e.g. Nandan and London, 2013; Van Der Vegt and Bunderson, 2005). The motivation for such organisation often springs from the need to solve complex problems that benefit from diverse perspectives and the needed versatile talent (Kurtzberg, 2005; Van der Vegt and Janssen, 2003). The ambitious goal of producing an innovation requires multidisciplinary collaboration to produce a large number of high-quality original ideas and to develop the competence needed in such versatile and multistage work. The multidisciplinary composition of teams in innovation networks allows for the complementarity of competence (Miettinen and Lehenkari, 2016).

Individual innovation competence

IIC is understood as a set of personal characteristics, knowledge, skills (or abilities) and attitudes that are connected to create concretised and implemented novelties via collaboration in complex innovation processes. Previous empirical research on innovation competence development in university–industry innovation projects relates primarily to single-discipline higher education contexts (e.g. Gilbert, 2011; Keinänen and Oksanen, 2017; Liebenberg and Mathews, 2012; West and Hanafin, 2010). Only a few studies have specifically addressed the multidisciplinary learning that novel innovations seem to require (e.g. Heikkinen and Isomöttönen, 2015; Johnsen, 2016; Muukkonen et al., 2013). There are several validated innovation competence assessment scales in the research literature – for example, there are scales concentrating on domains such as creative problem solving, systems thinking, goal orientation, teamwork and networking (Keinänen et al., 2018), or on creativity, critical thinking, initiative, teamwork and networking (Keinänen and Butter, 2018; see also Edwards-Schachter et al., 2015), but not, for instance, on the concretisation and implementation requirements included in many innovation definitions (e.g. Peschl et al., 2014; Quintane et al., 2011).

According to a recent systematic review (Hero et al., 2017) and its complementary empirical studies (Hero, 2017; Hero and Lindfors, 2019), those requirements are (a) personal characteristics such as self-esteem (e.g. Avvisati et al., 2013; Santandreu-Mascarell et al., 2013), self-management (e.g. Bjornali and Støren, 2012; Chatenier et al., 2010), achievement orientation (e.g. Mathisen et al., 2008; Montani et al., 2014), motivation and engagement (e.g. Chatenier et al., 2010; Chell and Althayde, 2011; Edwards-Sachter et al., 2015; Montani et al., 2014; Waychal et al., 2011), flexibility (e.g. Nielsen, 2015) and responsibility (Hero and Lindfors, 2019); (b) skills such as future orientation (Montani et al., 2014; Vila et al., 2014; Waychal et al., 2011), creative thinking skills (e.g. Chatenier et al., 2010; Edwards-Schachter et al., 2015), social skills like networking, collaboration and communication (e.g. Avvisati et al., 2013; Bjornali and Støren, 2012; Santandreu-Mascarell et al., 2013), development project management skills (e.g. Chatenier et al., 2010; Hero and Lindfors, 2019; Nielsen, 2015), implementation planning skills such as making, productisation, sales, marketing and entrepreneurship planning (Arvanitis and Stucki, 2012; Bruton, 2011; Hero, 2017, 2019; Hero and Lindfors, 2019); and (c) knowledge, such as one’s own and others’ discipline content knowledge (e.g. Avvisati et al., 2013; Bjornali and Støren, 2012).

Similar to other competences, innovation competence can be learned and developed (Bruton, 2011; Peschl et al., 2014). The progress or lack of progress towards such competence needs to be discovered so that teaching can be adjusted to match industry needs, student-experienced competence gaps and the authentic contexts in which learning projects are conducted.

Aim, materials and methods

This study defines, develops and validates an IIC scale for the higher education–industry collaboration context. It set out to explore which individual innovation competences are significant in university–industry collaboration and which of these competences are sensitive to authentic project-based educational intervention. Based on discovered need (Seikkula-Leino et al., 2010), the aim was to develop an IIC scale to be used as a self-assessment tool in an authentic project collaboration context for innovation. In higher education, self-rating questionnaires are applicable because they are relatively cheap and easy to administer (Braun et al., 2012). The research questions are: (a) what IICs are significant in a university–industry collaboration, and (b) which of these competences are sensitive to educational interventions in a multidisciplinary context? These questions are important because, in today’s higher education programmes, collaboration projects and project-based pedagogies offer an opportunity to be more practical and to focus on developing concrete outcomes and professional competences. Teaching staff are involved in an advisory rather than an authoritarian role (Helle et al., 2006). These pedagogical processes are often authentic open innovation projects that may result in real multidisciplinary RDI networks being formed, producing incremental or even radical new solutions and promoting student entrepreneurship. Thus, a multidisciplinary innovation pedagogy in higher education institutions promotes competence for students and new concrete products, services or other authentic, practical and usable solutions for industry or society (Heikkinen and Isomöttönen, 2015; Ness and Riese, 2015).

We used a three-phase method to develop the IIC scale, as summarised in Figure 1. Each phase consisted of research activities and their outcomes that developed the scale step-by-step towards a survey questionnaire that could later be used to unveil the impact of educational interventions on student innovation competence development.

Figure 1.

Methods and outcomes in the development phases of the IIC scale.

Development of the questionnaire items

Initially, we used the findings from a systematic literature review by Hero et al. (2017) as well as from its complementary empirical studies (Figure 2; Hero, 2017, 2019; Hero and Lindfors, 2019) to uncover the factors linked with IIC. The benefits of the systematic review method as a base study here were its contribution to rigour in material collection through the use of strict inclusion criteria and bias assessment, and the ability to report findings in a transparent way (Higgins, 2008; Petticrew and Roberts, 2006). Altogether, 74 IIC factors were identified and these were grouped into 21 sub-categories and further to seven domains.

Figure 2.

Sub-categories and domains of individual innovation competence identified for the development of statements. Source: Hero et al. (2017); Hero (2017); Hero and Lindfors (2019).

Similar to the instrument development process reported by Nilsson and colleagues (2014), we transformed the factors into statements, which eventually became items in the IIC scale. The items described the respondents’ behaviours rather than their characteristics. Specifically, the items were operationalised statements of the students’ self-assessed ability to act in an authentic collaborative and networked innovation development process. The items were designed by following the recommendations of Braun et al. (2012). Vague terms, retrospective estimation and double-barrelled questions were avoided (double-barrelled questions were split into two questions). Therefore, by the end of Phase I, there were 79 items in the IIC scale.

Validation with the first and second panels

In the second phase, as suggested by Braun et al. (2012), we invited a panel of teachers experienced in multidisciplinary innovation project tutoring to comment on the questionnaire items to confirm the content and construct validity of our scale. Of the 33 potential participants, 11 participated in a workshop. Approximately half of the items were further developed. Most of the problems concerned wording. For example, in the ‘Concretisation and implementation planning skills’ category, regarding esthetical and psychomotor skills, the item ‘I know how to use my psychomotor skills that are required in the realisation of a new concrete product’ was changed to ‘I know how to use my crafting skills that are required in the realisation of a new concrete product’. ‘Psychomotor’ was considered a term with which the target group would not be familiar.

The common denominators for innovation development conditions were discussed, and there were several statements in which the conditions had to be described in more detail to delimit a context for the behaviour in question: ‘I know how to use my sense of beauty’ was extended with ‘…in the realisation of a quality product’ and the word ‘team’ was added in the statements, ‘I can work actively to add value to my team to achieve our goals’ and ‘I am capable of leading a team’. Several of the problems concerned the double-barrelled issue (Braun et al., 2012) – for example, ‘Openness to experiences’ was initially formulated as ‘I am curious and open to new experiences’, but was sharpened by the panel so that it contained only one adjective. Eight items had to be added based on the problems in the items described above. After this phase, the IIC scale comprised 87 items.

To ensure face validity, a student panel was also invited to test the tool. A group of undergraduate students who had recently completed innovation studies were sent an email invitation to participate in the panel. Of the 172 potential students, nine participated by filling in the survey consisting of the 87 items. An extra column was added next to the rating scale to allow for open comments. Their feedback resulted in minor language modifications to four items. Finally, the IIC scale consisted of 87 items in seven domains relating to Personal characteristics (17 items), Future orientation (10 items), Creative thinking skills (13 items), Social skills (14 items), Project management skills (21 items), Content knowledge (2 items) and Concretisation and implementation planning skills (10 items).

The measurement was carried out using a 6-point ordinal Likert-type scale (0 = cannot say, 1 = not at all, 2 = weakly, 3 = moderately, 4 = very well, 5 = excellent). In addition to the survey, we assessed age, gender, degree programme, language of instruction, study year and the participants’ understanding of what the term ‘innovation’ meant. This background information was examined through open questions and nominal scale variables when appropriate.

Outline of the pilot study

Finally, the IIC scale was pilot tested to ensure its consistency and reliability (Braun et al., 2012). Furthermore, the findings of the pilot study were used to identify which IICs were significant in a university–industry collaboration and which were sensitive to the innovation project type of educational intervention.

The self-assessment tool was pilot-tested in a multidisciplinary innovation pedagogy context in one University of Applied Sciences (UAS) in Finland. This pedagogical intervention was a 7-week university–industry MINNO® Innovation Project implemented at Metropolia UAS.

During their second or third year of study, every student in all undergraduate programmes completes a MINNO® Innovation Project comprising 10 credits, which is equal to approximately 270 hours of study time. The project’s explicit aim is for students to develop novel solutions, products, services or processes in response to authentic challenges presented by companies and other professional organisations (for further information on the implementation of the pedagogy, see Hero, 2020; Hero and Lindfors, 2019; Metropolia UAS, 2020). First, students get to choose their preferred project challenge. Thereafter, to solve the challenge, students from different disciplines team up and form their own network of teachers, company representatives and other relevant stakeholders.

The instructive process includes orientation and theory along the way in the form of an innovation toolbox, team project work, concept presentations (i.e. pitches for the customer companies), customer caching, prototyping, research and testing, implementation and entrepreneurship planning, followed by a final public event and delivery to the customer. Teams are normally tutored for 1–2 days a week, and the customers typically give feedback on the solutions 2–5 times. Typically, a team’s project outcome includes concept papers, a prototype and its test results with productisation and a go-to-market sales and marketing implementation plan. The teachers act as facilitators and offer tools for innovating. The teachers also help in networking and finding new partners from working life, if necessary. Grades are based on all project outcomes, customer and teacher observations, diaries and assessment discussions. The self-assessment questionnaire does not have an effect on the grade.

In a single-group pre-test–post-test investigation, the pre-test data were collected during the first days of the 7-week MINNO® Innovation Project of the spring semester. The enrolled students received an invitation and a link to an electronic survey document. The responses were automatically directed to the archive of the e-document system, in which they were also stored. Only the researchers had access to the archive. The post-test data were collected at the end of the 7-week project. The survey instrument and the arrangements for the data collection remained the same.

While being aware of the weaknesses relating to the lack of a control group, this quasi-experimental single-group pre-test–post-test design was considered appropriate for our study. There are two primary reasons for this decision: first, the pedagogical solutions used to facilitate the development of the learners’ innovation competences at Metropolia are practically oriented rather than lecture-based, and so an equivalent control group was not available (Bowling, 2003; Campbell and Stanley, 1963); second, finding an equivalent control group from another university with a similar programme offering was not seen as an option due to the lack of control with intervening variables, which would jeopardise the requirement for identical conditions (Bowling, 2003; Campbell and Stanley, 1963). Instead, in our study, the participants were used as their own control, which is a characteristic of repeated-measures designs (Loiselle and Profetto-McGrath, 2004).

A total of 430 students had enrolled in the project course, and they were all invited to participate in the pilot study. While not all the students were eligible, willing or able to participate, the sample consisted of 138 students. The response rate was 32.1%. Of the 138 participants in the pre-test survey, 56 (41%) also participated in the post-test survey.

Ethical clearance to conduct the study was granted by the Director of Research, Development and Innovation at Metropolia UAS in January, 2020. The process also includes procedures to ensure adherence to the European Union General Data Protection Regulations, GDPR (2016/679) (European Union, 2016), as well as to the national Data Protection Act (1050/2018) in Finland. As outlined by the Finnish National Board on Research Integrity (2019), measures were taken to protect the dignity, rights and safety of the participants. The potential participants received information about the study, as well as an invitation to participate in it during the week before the kick-off of the MINNO® Innovation Project. The voluntary nature of participation and the measures taken to ensure anonymity were explained, as was the fact that the self-assessment scores were not in any way linked to the grading of the course. Only the researchers had access to the electronic archive that was used to store the original data. The students used individual ID codes to access the data collection instrument; an individual cannot be identified via the codes, however. They were only used to allow the researchers to match the pre- and post-test the replies of the same participant.

We analysed the quantitative data using SPSS version 25. Descriptive statistics (percentage, frequency mean, range, standard deviation (SD)) served to characterise the sample. The domains of the IIC scale were confirmed through exploratory factor analysis and described with means and SDs. Following the factor analysis, the Cronbach’s alpha coefficient was computed to measure the internal consistency reliability of the IIC scale (Plichta and Kelvin, 2013). To compare the IICs before and after the intervention, we used paired t-tests (Nummenmaa, 2011; Plichta and Kelvin, 2013). In this stage, we used the seven domains of the IIC scale as sum variables.

To analyse the effects of the independent variables (age, study year, gender, field of study), we conducted a repeated measures analysis of variance (Plichta and Kelvin, 2013). The language of instruction was not included in this analysis due to the low number of participants taught in English during post-testing (n = 3). This method was chosen as it allows for the measurement of the dependent variable (the sum variables illustrating the seven domains of the IIC scale) over two or more time points and the exploration of the interaction between the independent and dependent variables. The following necessary assumptions for the analysis were met: the dependent variable was measured at the continuous level; the sum variables were matched pairs; there were no outliers in any combination of the related groups; the dependent variable was normally distributed in each combination; and the sphericity between all combinations of related groups was equal. Whenever the independent variable had more than two categories, the Bonferroni correction was used to counteract multiple comparisons.

Results of the pilot study

The IIC scale comprised 87 statements. Each competence domain comprised several statements expressing the sub-domain (i.e. the factor) found in the systematic review and the complementary studies (see Table 1 for an example of one competence domain).

Table 1.

Example of one competence domain in the IIC scale, its sub-domains and questionnaire statements.

Domain	Sub-domain	Questionnaire statement
Concretisation and implementation planning skills	Making skills	I know how to design practical items from abstract ideas.
Concretisation and implementation planning skills	Making skills	I know how to make a working prototype.
		I know how to make a functional product by hand.
		I have crafting skills required for making a new concrete product.
		I know how to use my sense of beauty in the realisation of a quality product.
	Productisation planning skills	I know how to turn an idea into a product.
	Marketing, sales and entrepreneurship planning skills	I know how to make a marketing plan.
	Marketing, sales and entrepreneurship planning skills	I know how to make a sales plan for a product.

To test our theoretical understanding of the IIC scale, we assessed its internal consistency with the Cronbach’s alpha reliability coefficient. The alphas ranged from 0.689 to 0.931 for the different domains (Table 2). All items in each domain appeared to be worthy of retention.

Table 2.

Testing the internal consistency of the IIC scale (n = 138).

Domain	Mean	Variance	Number of items	Chronbach’s alpha
Personal characteristics	3.84	0.582	17	0.888
Future orientation	3.72	0.707	10	0.878
Creative thinking skills	3.70	0.717	13	0.912
Social skills	3.69	0.886	14	0.917
Project management skills	3.40	1.107	21	0.931
Content knowledge	3.36	0.962	2	0.689
Concretisation and implementation planning skills	2.59	1.732	10	0.926

The stability and precision of the IIC scale over time was also examined by measuring the correlation of all seven domains before and after the intervention. In this test–retest examination, the Pearson correlations were as follows: Personal characteristics, 0.623; Future orientation, 0.520; Creative thinking skills, 0.704; Social skills, 0.708; Project management skills, 0.708; Content knowledge, 0.524; Concretisation and implementation planning skills, 0.787. All results were statistically significant (p < 0.001). We also tested the content validity of the IIC scale by conducting an exploratory factor analysis to confirm its structure. In this pilot phase, the data for analysis were too limited to make a reliable interpretation of the scale factors compared to the tested competence domains.

All seven innovation competence domains proved to be sensitive to change. All innovation competences post-assessed by students were higher than their pre-assessment levels. Most sensitive to change were the capabilities that enabled students to learn practical operational skills, such as managing their project work better, developing practical solutions, turning an idea into a product, or evaluating the threats and opportunities associated with entrepreneurship.

To compare whether students’ IIC had significantly increased between measurements before and after the intervention, we used paired t-tests. The results of the domain-based paired t-tests are presented in Table 3. The sum variables are used here to illustrate the seven domains of the IIC scale. In each domain, a significant difference in the scores was found between the before and after measurement outcomes. These outcomes suggest that the intervention had a positive effect on the development of IIC. Specifically, our results suggest that the positive effect is highest for Project management skills (t(55) = −9.361, p < 0.001) and Concretisation and implementation skills (t(55) = −10.279, p < 0.001).

Table 3.

Students’ self-assessed IIC before and after the intervention in the domains of the IIC scale (n = 56).

Domain	Mean	T	df	p value
Personal characteristics	−0.37	−7.65	55	<0.001
Future orientation	−0.34	−4.67	55	<0.001
Creative thinking skills	−0.48	−7.67	55	<0.001
Social skills	−0.41	−7.18	55	<0.001
Project management skills	−0.58	−9.36	55	<0.001
Content knowledge	−0.64	−6.88	55	<0.001
Concretisation and implementation planning skills	−0.90	−10.28	55	<0.001

The weak effect of the multidisciplinary innovation project course in the domain of Personal characteristics was expected, as personal characteristics are supposed to be relatively stable patterns of thoughts, feelings and actions (cf. Costa and McCrae, 1988). Also noteworthy is the relatively small change in Future orientation.

The most significant background determinant affecting self-assessment was the field of study. The respondents were distributed across different fields: nursing (n = 17), specialist healthcare (n = 9), rehabilitation (n = 11), social services (n = 6), culture (n = 10) and technology (n = 3). The students in specialist healthcare assessed themselves most critically and had the lowest pre-scores in all competence domains. The post-test indicated, however, that they had the highest increase in their competences. It seems that the starting level of one’s own competence influences the assessment of the growth of one’s competence after the intervention. The students of technology and social services most often assessed their skills as the highest both in the pre- and post-tests. Such patterns were statistically significant for the domains of Social skills (p = 0.009) and Project management skills (p > 0.001). As the number of students in each group was small, the results may be considered indicative.

Male (n = 18) and female (n = 37) students seemed to have similar assessments of their competence development in all domains. Students under 25 years of age (n = 22) rated their innovation competence as slightly higher than those over 25 (n = 34) both before and after the intervention. In line with this was the observation regarding the year of study: students in their first or second year (n = 16) rated their innovation competence as slightly higher than those in their third or fourth year (n = 40). None of these results was statistically significant, however.

Discussion and conclusions

As innovation competence is one of the key targets of higher education and an important part of entrepreneurship education, this study defines, develops and validates an IIC scale for university–industry learning interventions. To serve this purpose, the aim of this paper is to determine which IICs are significant in university–industry collaboration and which of these competences are sensitive to educational interventions in a multidisciplinary context. The paper reports the findings of the IIC scale development and its pre–post survey pilot tests. All seven domains of IICs defined in the first phases of the study were significant and sensitive to the piloted multidisciplinary innovation project educational intervention. An increase was found in each competence domain based on the students’ pre and post self-assessments of their innovation competences. Previous IIC assessment scales in the educational context have not included implementation-related competences, although innovations by definition most often include the concrete form of the solution (i.e., the novel product, service, etc) and its implementation requirements (e.g. Peschl et al., 2014; Quintane et al., 2011). Most responsive to change were the competence domains of Concretisation and implementation planning skills and Project management skills. The weakest effect of the educational intervention was in the domain of Personal characteristics, which was expected as personal characteristics are relatively stable patterns of thoughts, feelings and actions (Costa and McCrae, 1988). There were relatively low changes in the Future orientation domain (cf. Montani et al., 2014; Vila et al., 2014; Waychal et al., 2011). It is possible that the intervention did not include and develop future orientation, and it should be added more explicitly to pedagogy by teachers. Of the background variables, only the field of study seemed to be associated with the change that occurred between the pre and post measurements. This finding must be considered with caution, however, due to the small number of participants in the study.

The scale seems to mirror the characteristics of university–industry innovation collaboration and the IIC related to it. If innovations are novelties that are made concrete, are useful and are implemented to convey value (e.g. Peschl et al., 2014; Quintane et al., 2011), competences such as creativity may support the novelty requirement, and concretisation and implementation planning skills may support the requirement of concrete usefulness and go-to-market readiness. If innovation development is associated with teams of diverse individuals and networked multi-professional collaborations (Nandan and London, 2013; Sloep et al., 2014; Van Der Vegt and Bunderson, 2005), it is justifiable that the scale should highlight social collaboration, communication and networking skills. If the ambitious goal of producing an innovation requires multidisciplinary collaboration to produce a large number of high-quality original ideas and to collect the competence in a team that is needed in such versatile and multistage work (e.g. Jonassen et al., 2006; Kurtzberg, 2005), the scale should also measure flexibility, responsibility, self-esteem, creativity and the development of project management skills.

The strength of this study is the transparency of the validation process and the ethical conduct of the empirical tests. Based on the rigorous and transparent validation process in an authentic innovation development context, the bias-assessed systematic review background of the competency variables and domains and the pilot test results, we are able to postulate that this scale is already usable in authentic innovation project contexts. However, there are limitations. The empirical validation tests were conducted with a limited number of participants. Nevertheless, these participants highlighted the development needs of the scale. One weakness of a pre–post survey may be that the ‘measuring stick’ changes during the intervention as the respondent develops greater knowledge. At the beginning of the instruction phase, students may not know what they did not know, so they may give themselves higher ratings than they would at the end of the learning experience; that is, they may rate themselves as lower post-instruction (Howard, 1980). This limitation should be taken into consideration before drawing broader conclusions by using this quantitative survey method with some material collection and analysis to provide qualitative insight into the learning experience (e.g. diaries written during the project or assessment workshops at the beginning, middle and end of the project). Another weakness is the length of the scale. It still comprises 87 questions and thus takes 15 to 20 minutes to answer.

The findings indicate that, despite the rigour of the validation and because of the limited materials, further research and tests are recommended. First, we recommend that the tool should be refined based on the pilot test analyses. Second, it should be tested with large participant groups in the same context. Third, it should also be validated for other contexts – namely, for entrepreneurs and corporate employees – to be able to compare the student results with industry results. This could increase understanding of the innovation potential as assessed by people participating in innovation development networks. Fourth, the results raise the need to deepen our understanding of the relationship between pedagogy, industry targets and student learning experience in a multidisciplinary team to enable more efficient collaborations with working life to be designed. This also requires qualitative research.

In conclusion, the IIC scale differs from other innovation competence scales in that it focuses on individual competence (cf. e.g. Kodama and Shibata, 2014; Lim et al., 2011; Wang, 2014), comprises a large number of items, is based on a systematic review, and takes the implementation and exploitation phases of innovation development into consideration (cf. e.g. Edwards-Schachter et al., 2015; Keinänen et al., 2018). The major impact of this study is in the distribution opportunity following the pre and post survey validation of the IIC scale. Future research with larger groups is possible after this initial validation study. As a practical implication, now that the development of the scale has been made transparent, it is possible to test and refine it with larger participant groups. It can then be distributed in different countries to compare the impact of best practices and pedagogical excellence in university–industry innovation and entrepreneurship education.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Laura-Maija Hero

References

Andrade

(2019) A critical review of research on student self-assessment. Frontiers in Education 4(87): 1–13.

Ankrah

Al-Tabbaa

(2015) Universities–industry collaboration: a systematic review. Scandinavian Journal of Management 31: 387–408.

Arvanitis

Stucki

(2012) What determines the innovation capability of firm founders? Industrial and Corporate Change 21(4): 1049–1084.

Avvisati

Jacotin

Vincent-Lancrin

(2013) Educating higher education students for innovative economies: what international data tells us. Tuning Journal for Higher Education 1(1): 223–240.

Bird

(1995) Towards a theory of entrepreneurial competency. Advances in Entrepreneurship Firm Emergence and Growth 2: 51–72.

Bissola

Imperatori

Biffi

(2017) A rhizomatic learning process to create collective knowledge in entrepreneurship education: open innovation and collaboration beyond boundaries. Management Learning 48(2): 206–226.

Bjornali

Støren

(2012) Examining competence factors that encourage innovative behaviour by European higher education graduate professionals. Journal of Small Business and Enterprise Development 19(3): 402–423.

Bowling

(2003) Research Methods in Health. Investigating Health and Health Services. New York, NY: Open University Press.

Braun

Woodley

Richardson

JTE

, et al. (2012) Self-rated competences questionnaires from a design perspective. Educational Research Review 7(1): 1–18.

10.

Bruton

(2011) Learning creativity and design for innovation. International Journal of Technology and Design Education 21(3): 321–333.

11.

Campbell

Stanley

(1963) Experimental and Quasi-Experimental Designs for Research. Boston, MA: Houghton Mifflin Company.

12.

Cerinšek

Dolinšek

(2009) Identifying employees’ innovation competency in organisations. International Journal of Innovation and Learning 6(2): 164–177.

13.

Chatenier

Verstegen

Biemans

, et al. (2010) Identification of competencies for professionals in open innovation teams. R&D Management 40(3): 271–280.

14.

Chell

Athayde

(2011) Planning for uncertainty: soft skills, hard skills and innovation. Reflective Practice 12(5): 615–628.

15.

Costa

McCrae

(1988) Personality in adulthood: a six-year longitudinal study of self-reports and spouse ratings on the NEO Personality Inventory. Journal of Personality and Social Psychology 54(5): 853–863.

16.

Da Silva

Davis

(2011) Absorptive capacity at the individual level: Linking creativity to innovation in academia. Review of Higher Education 34(3): 355–379.

17.

Data Protection Act (2018) Act 1050/2018. Available at: https://www.finlex.fi/en/laki/kaannokset/2018/en20181050.pdf (accessed 10 October 2020).

18.

DiPietro

(2009) Country innovativeness and the difficulty of doing business. Journal of Global Business Issues 3(2): 69–75.

19.

Edwards-Schachter

García-Granero

Sánchez-Barrioluengo

, et al. (2015) Disentangling competences: interrelationships on creativity, innovation and entrepreneurship. Thinking Skills and Creativity 16(0): 27–39.

20.

European Union (2016) Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Available at: http://data.europa.eu/eli/reg/2016/679/oj (accessed 10 October 2020).

21.

Fayolle

Verzat

Wapshott

(2016) In quest of legitimacy: the theoretical and methodological foundations of entrepreneurship education research. International Small Business Journal 34(7): 895–904.

22.

Finnish National Board on Research Integrity (2019) The ethical principles of research with human participants and ethical review in the human sciences in Finland: TENK guidelines. Available at: https://tenk.fi/en/advice-and-materials/guidelines-ethical-review-human-sciences (accessed 10 October 2020).

23.

Gilbert

(2011) From chalk and talk to walking the walk: facilitating dynamic learning contexts for entrepreneurship students in fast-tracking innovations. Education + Training 54(2/3): 152–166.

24.

Gundry

Ofstein

Kickul

(2014) Seeing around corners: how creativity skills in entrepreneurship education influence innovation in business. International Journal of Management Education 12(3): 529–538.

25.

Heikkinen

Isomöttönen

(2015) Learning mechanisms in multidisciplinary teamwork with real customers and open-ended problems. European Journal of Engineering Education 6(40): 653–670.

26.

Helle

Tynjälä

Olkinuora

(2006) Project-based learning in postsecondary education: theory, practice and rubber sling shots. Higher Education 51(2): 287–314.

27.

Hero

L-M

(2017) Innovation tournament as an activity system to promote the development of innovation competence. Journal of Professional and Vocational Education 19(4): 8–31.

28.

Hero

L-M

(2019) Learning to develop innovations: individual competence, multidisciplinary activity systems and student experience. Doctoral Dissertation, Annales universitatis Turkuensis 475, University of Turku, Finland.

29.

Hero

L-M

(2020) MINNO® innovation project: a multidisciplinary way to develop innovation competences. In: Remenyi

(ed) 6th Teaching Innovation & Entrepreneurship Excellence Awards 2020: An Anthology of Case Histories. Reading: Academic Conferences International, pp. 57–70.

30.

Hero

L-M

Lindfors

(2019) Students’ learning experience in a multidisciplinary innovation project. Education + Training 61(4): 500–522.

31.

Hero

L-M

Lindfors

Taatila

(2017) Individual innovation competence: a systematic review and future research agenda. International Journal of Higher Education 6(5): 103–121.

32.

Hiebert

Domene

Buchanan

(2011) The power of multiple methods and evidence sources: Raising the profile of Canadian counselling psychology research. Canadian Psychology 52: 265–275.

33.

Higgins

(ed) (2008) Cochrane Handbook for Systematic Reviews of Interventions, vol. 5. Chichester: Wiley-Blackwell.

34.

Howard

(1980) Response-shift bias: a problem in evaluating interventions with pre/post self-reports. Evaluation Review 4(1): 93–106.

35.

Johnsen

(2016) Learning to create new solutions together: a focus group study exploring interprofessional innovation in midwifery education. Nurse Education in Practice 16(1): 298–304.

36.

Jonassen

Strobel

Lee

(2006) Everyday problem solving in engineering: Lessons for engineering educators. Journal of Engineering Education 95(2): 139–151.

37.

Kasule

Wesselink

Noroozi

, et al. (2015) The current status of teaching staff innovation competence in Ugandan universities: perceptions of managers, teachers, and students. Journal of Higher Education Policy and Management 37(3): 330–343.

38.

Keinänen

Butter

(2018) Applying a self-assessment tool to enhance personalized development of students’ innovation competences in the context of university–company cooperation. Yliopistopedagogiikka 25(2): 19–28.

39.

Keinänen

Oksanen

(2017) Students’ perception of learning innovation competences in activity-based learning environment. Journal of Professional and Vocational Education 19(4): 48–61.

40.

Keinänen

Ursin

Nissinen

(2018) How to measure students’ innovation competences in higher education: evaluation of an assessment tool in authentic learning environments. Studies in Educational Evaluation 58: 30–36.

41.

Kodama

Shibata

(2014) Strategy transformation through strategic innovation capability: a case study of Fanuc. R&D Management 44(1): 75–103.

42.

Kurtzberg

(2005) Feeling creative, being creative: an empirical study of diversity and creativity in teams. Creativity Research Journal 17: 51–65.

43.

Liebenberg

Mathews

(2012) Integrating innovation skills in an introductory engineering design-build course. International Journal of Technology and Design Education 22: 93–113.

44.

Lim

Moriarty

Huthwaite

(2011) ‘Being-in-role’: a teaching innovation to enhance empathic communication skills in medical students. Medical Teacher 33(12): 663–669.

45.

Loiselle

Profetto-McGrath

(2004) Canadian Essentials of Nursing Research. Philadelphia, PA: Lippincott Williams & Wilkins.

46.

Mäkimattila

Junell

Rantala

(2015) Developing collaboration structures for university–industry interaction and innovations. European Journal of Innovation Management 18(4): 451–470.

47.

Manning

Bearden

Madden

(1995) Consumer innovativeness and the adoption process. Journal of Consumer Psychology 4(4): 329–345.

48.

Maritz

Donovan

(2015) Entrepreneurship and innovation: setting an agenda for greater discipline contextualisation. Education + Training 57(1): 74–87.

49.

Maritz

Brown

(2013) Illuminating the black box of entrepreneurship education programs. Education + Training 55(3): 234–252.

50.

Mathisen

Martinsen

Einarsen

(2008) The relationship between creative personality composition innovative team climate and team innovativeness: an input–process–output perspective. Journal of Creative Behaviour 42(1): 13–31.

51.

Metropolia University of Applied Sciences (2020) Minno® Innovation Projects. Available at: http://www.metropolia.fi/en/research-development-and-innovation/innovation-projects/ (accessed 10 October 2020).

52.

Miettinen

Lehenkari

(2016) Encounters and extended collaborative creativity: mobilization of cultural resources in the development of a functional food product. In: Glaveanu

(ed) The Palgrave Handbook of Creativity and Culture Research. London: Palgrave Macmillan, pp. 263–283.

53.

Mitchelmore

Rowley

(2010) Entrepreneurial competencies: a literature review and development agenda. International Journal of Entrepreneurial Behaviour and Research 16(2): 92–111.

54.

Montani

Odoardi

Battistelli

(2014) Individual and contextual determinants of innovative work behaviour: proactive goal generation matters. Journal of Occupational and Organisational Psychology 87(4): 645–670.

55.

Mulder

(2012) Competence-based education and training. The Journal of Agricultural Education and Extension 18(3): 305–314.

56.

Mulder

Gulikers

(2011) Workplace learning in East Africa: a case study. In: Malloch

Cairns

Evans

O’Connor

(eds) The SAGE Handbook of Workplace Learning. London: SAGE, pp. 307–318.

57.

Muukkonen

Kosonen

Marttiin

, et al. (2013) Pedagogical design for knowledge creating inquiry in customer projects. Knowledge Management and E-Learning 5(3): 278–297.

58.

Nandan

London

(2013) Interdisciplinary professional education: training college students for collaborative social change. Education + Training 55(8/9): 815–835.

59.

Neck

Corbett

(2018) The scholarship of teaching and learning entrepreneurship. Entrepreneurship Education and Pedagogy 1(1): 8–41.

60.

Ness

Riese

(2015) Openness, curiosity and respect: underlying conditions for developing innovative knowledge and ideas between disciplines. Learning, Culture and Social Interaction 6: 29–39.

61.

Nielsen

(2015) Assessment of innovation competency: a thematic analysis of upper secondary school teachers’ talk. Journal of Educational Research 108(4): 318–330.

62.

Nilsson

Johansson

Egmar

, et al. (2014) Development and validation of a new tool measuring nurses self-reported professional competence: the Nurse Professional Competence (NPC) Scale. Nurse Education Today 34(4): 574–580.

63.

Nummenmaa

(2011) Käyttäytymistieteiden tilastolliset menetelmät [Statistical Methods in Behavioral Sciences], 3rd edn. Helsinki: Tammi.

64.

Oslo Manual (2005) Guidelines for Collecting and Interpreting Innovation Data, 3rd edn. Paris: OECD Publishing. Available at: http://www.oecd.org/sti/inno/oslomanualguidelinesforcollectingandinterpretinginnovationdata3rdedition.htm (accessed 10 October 2020).

65.

Peschl

Bottaro

Hartner-Tiefenthaler

, et al. (2014) Learning how to innovate as a socio-epistemological process of co-creation: towards a constructivist teaching strategy for innovation. Constructivist Foundations 9(3): 421–433.

66.

Petticrew

Roberts

(2006) Systematic Reviews in the Social Sciences: A Practical Guide. Malden: Blackwell.

67.

Plichta

Kelvin

(2013) Munro’s Statistical Methods for Health Care Research, 6th edn. Wolters Kluwer Health. Philadelphia, PA: Lippincott, Williams & Wilkins.

68.

Quintane

Casselman

Reiche

, et al. (2011) Innovation as a knowledge-based outcome. Journal of Knowledge Management 15(6): 928–947.

69.

Rantala

Ukko

(2018) Performance measurement in university–industry innovation networks: implementation practices and challenges of industrial organisations. Journal of Education and Work 31(3): 247–261.

70.

Santandreu-Mascarell

Garzon

Knorr

(2013) Entrepreneurial and innovative competences, are they the same? Management Decision 51(5): 1084–1095.

71.

Sawyer

(2006) Educating for innovation. Thinking Skills and Creativity 1(1): 41–48.

72.

Sawyer

(2009) The collaborative nature of innovation. Journal of Law and Policy 30: 293–324.

73.

Seikkula-Leino

Ruskovaara

Ikävalko

, et al. (2010) Promoting entrepreneurship education: The role of the teacher? Education + Training 52(2): 117–127.

74.

Sloep

Berlanga

Retalis

(2014) Introduction to the special issue on Web-2.0 technologies in support of team-based learning for innovation. Computers in Human Behavior 37(1): 342–345.

75.

Slotte

Tynjälä

(2003) Industry–university collaboration for continuing professional development. Journal of Education and Work 16(4): 445–464.

76.

Spencer

(1993) Competence at Work: Models for Superior Performance. New York, NY: John Wiley & Sons.

77.

Sturing

Biemans

HJA

Mulder

, et al. (2011) The nature of study programmes in vocational education: evaluation of the model for comprehensive competence-based vocational education in the Netherlands. Vocations and Learning 4(3): 191–210.

78.

Van Der Vegt

Bunderson

(2005) Learning and performance in multidisciplinary teams: the importance of collective team identification. Academy of Management Journal 48(3): 532–547.

79.

Van der Vegt

Janssen

(2003) Joint impact of interdependence and group diversity on innovation. Journal of Management 29: 729–751.

80.

Vila

Pérez

Coll-Serrano

(2014) Innovation at the workplace: Do professional competencies matter? Journal of Business Research 67(5): 752–757.

81.

Volery

Mueller

von Siemens

(2015) Entrepreneur ambidexterity: A study of entrepreneur behaviours and competencies in growth-oriented small and medium-sized enterprises. International Small Business Journal 33(2): 109–129.

82.

Wang

(2014) A longitudinal study of innovation competence and quality management on firm performance. Innovation: Management, Policy & Practice 16(3): 392–403.

83.

Waychal

Mohanty

Verma

(2011) Determinants of innovation as a competence: an empirical study. International Journal of Business Innovation and Research 5(2): 192–211.

84.

West

Hannafin

(2010) Learning to design collaboratively: participation of student designers in a community of innovation. Instructional Science 39(6): 821–841.

85.

Zhuang

Williamson

Carter

(1999) Innovate or liquidate: Are all organisations convinced? A two-phased study into the innovation process. Management Decisions 37(1): 57–71.

Validating an individual innovation competence assessment tool for university–industry collaboration

Abstract

Keywords

Towards individual innovation competence

Differences between innovation and entrepreneurial competences

Innovation competence manifests itself in context

Individual innovation competence

Aim, materials and methods

Development of the questionnaire items

Validation with the first and second panels

Outline of the pilot study

Results of the pilot study

Discussion and conclusions

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

References