Graduates’ Attitudes to Research Skill Development in Undergraduate Media Education

Literature Review

Willison and O’Regan (2007) demonstrate that students may not understand the relevance of research skill development to future careers and are frequently left to construct their own sense of coherence from varied research experiences during their studies. Educators may often struggle to articulate how and why they deliver graduates with capacity to conduct research that has value in their subsequent careers (Brew, 2006; Jenkins et al., 2003). This creates significant problems for educators, because students’ uncertainties regarding the value of research training for career outcomes may affect their subsequent formal and informal evaluations of course quality, which directly impacts on estimations of university reputation made by governments, industry bodies and prospective domestic and international students (Moogan, Baron & Harris, 1999).

The value of university education has been vehemently debated in media and communication studies (e.g., Camp, 2012; Curran, 2013; Dickson, 2000). In Australia enrolments in Communications and Media Studies increased by almost 50 per cent between 2007 and 2012, significantly higher than for enrolments in undergraduate courses as a whole (Dept. of Education and Training, 2012). These high relative increases have been matched in the USA (Camp, 2012) and the UK (Buckingham, 2013). Lee notes, ‘In many Asian universities … communication programmes are usually among the most popular’, but that increasing competition and ‘the onslaught of the market imperative’ (2008, p. 61) raise questions about the quality of the educational outcomes delivered to graduates.

Disproportionally large increases in enrolment in media degrees and the increasing costs of higher education require universities to demonstrate how courses produce learning outcomes that cannot be effectively delivered by other means and value for their graduates when seeking appropriate career outcomes (Bridgstock, 2011). Prospective students, in particular must be convinced that their course of study will produce a return on investment, which requires that the value of each element of the curriculum, including research skill development, is clearly understood. This is not to advocate a valuation based only on financial remuneration after graduation, although this has often been the focus of public debate (Daly, Lewis, Corliss, & Heaslip, 2012). Value accrues in many ways and less tangible returns (e.g., career satisfaction) may be of equal importance to graduates (Ball, Pollard & Stanley, 2010). Qualitative data analyses, such as that utilized in this study, are particularly helpful in understanding the different types of value judgements made by students and graduates regarding their learning experience.

Media education must also take into account the varied needs of media industries, each of which is defined by different types of content, production and distribution technologies, and audiences or markets. Organizations working in particular industry segments require complex divisions of labour that combine input from ‘creatives’ and a host of support staff. Universities offering media courses face the risk that they may fail to cater adequately to the skill demands of this complex media labour market (Buckingham, 2013). Analysis of recent graduates’ opinion may, therefore, be particularly helpful with regard to our understanding of aspects of curriculum design related to employability.

It is possible to simplify understanding of the complex career and employment landscape facing media graduates through ‘triad analysis’ (Higgs & Cunningham, 2008; Higgs, Cunningham & Pagan, 2007), which categorizes potential employment as:

Media work for a media industry organization,

Consequently, the diversity of career and employment outcomes for Media graduates means that educators must provide general research capabilities and skills that relate to specific areas of work and further study. Hanney (2013) also notes that universities must provide research skills for employability; that is, the ability to understand the industries and environments within which graduates will work in order to maintain their career trajectories. These employability research skills include the ability to develop knowledge about particular industries, their processes and the organizations working in them; the skills necessary for entrepreneurship and risk analysis; the capacity for self-awareness and other metacognitive skills required to recognize one’s abilities (Jones & Defillippi, 1996); and the articulation of the skills they bring to a particular organizational context or industry environment (Bridgstock, 2009).

The development of an explicit statement of the stages through which undergraduates come to think and act as researchers can potentially help students become aware of these differing uses of their research skills. Scaffolded and explicit instruction, rather than implicit expectation and variation learning design, is needed for many students to see substantial gains in research skill development (Willison & Buisman Pijlman, in press; Willison & O’Regan, 2007). Wass, Harland and Mercer (2011, p. 322) found that, ‘research activity was central to knowledge-making and the primary scaffold for a student’s … critical thinking’. However, the use of open-ended research tasks based on varying pedagogic principles throughout a degree may lead some students to apply only the same degree of sophistication in their final year as in their first year of study (Chaplin, 2003).

The RSDF was designed to provide a systematic description of the stages of learning through which students develop the knowledge and skills required to function as effective researchers in their chosen disciplines that can be applied in different educational contexts (Willison & O’Regan, 2007). Its application in learning design and assessment engages students in systematic action to solve problems and also assists in the development of metacognitive awareness of the ways that research skills developed in specific disciplines can be applied in other contexts.

The RSDF describes research in terms of six interdependent facets:

Embarking on research and clarifying the need for knowledge;

The RSDF describes the development of these research skills at undergraduate level in terms of five levels of increasing autonomy, starting from Level 1, Prescribed Research that requires students to receive considerable guidance from experienced researchers, through to Level 5, Open Research, at which point students are able to operate relatively independently of expert guidance. These facets of research and levels of autonomy are organized sequentially, but it is important to recognize that they are seldom encountered in a linear fashion due to the complex, iterative nature of research practice (Willison, 2013).

Methodology

The essential question addressed in the wider study, of which this investigation of media graduates is a subset, is this, does the RSDF meet the requirements of all or only some university disciplines? In this case, we wanted to know whether the RSDF provides sufficient scaffolding for learning in the context of media education at undergraduate level. We also wished to evaluate graduates’ self-awareness of their capacities as researchers (Jackson, 2014) by examining how they explain the use of research skills in employment. Such metacognition encompasses both an abstract awareness of epistemology and a grounded understanding of how research principles can be applied in diverse contexts to solve unique problems.

As noted above, the media degree included in the wider study provided a robust evaluation of the credibility and usefulness of the RSDF, because a generally applicable approach to research skill development may not appear to be relevant to media students, graduates or employers given the apparent diversity of media industries and graduates’ career outcomes. Indeed, research skill development in media programmes may even be questioned if it strays too far from areas of apparently immediate professional relevance and into academic critique (Herkman, 2008). Therefore, a goal of our research was to investigate how recent graduates evaluated their experience of research skill development and the use of the RSDF to facilitate this during given their different employment destinations.

The RSDF was used in two compulsory upper-level subjects coordinated by the first author. One of these was a subject that provided direct training in media research methods and the other one was a subject examining media industries in a global context. It was used in these subjects to design curriculum structure and assessments, including rubrics used in marking.

One-hundred and thirty graduates (76 female; 54 male) who completed the Media degree in 2011 were invited to provide contact details in order to be interviewed, in keeping with the ethics committee-approved protocol that required that there would be no email contact from an interviewer without participant invitation to do so. Fourteen graduates responded and were contacted by the project’s research officer via email and invited to participate; 10 graduates finally participated (four female; six male).

Most studies of graduate employment use survey methodologies (So, Lam & So, 2013), which do not provide scope for detailed reflection on learning experiences. Few studies have successfully gathered fine-grained data of graduate’s perspective on their degree program and its relevance to employment (Bandaranaike & Willison, 2015). The current study focused on a specific aspect of education in a media degree—the development of research skills, so interview questions were both targeted and analysed using criteria suitable for qualitative data (Guba & Lincoln, 1998; Schwandt, Lincoln & Guba, 2007). We obtained interviews that reflected a reasonable gender balance and the variety of employment types indicated by Higgs and Cunningham’s work on employment in the creative industries (Higgs & Cunningham, 2008). These types of qualitative data on graduates’ perspective have been notoriously difficult to generate, due to problems with maintaining current contact details of graduates, and because ethics committee requirements typically require opt-in strategies for education research, which typically have low response rates (Hewison & Haines, 2006). ¹

The interviews followed a semi-structured approach (Wengraf, 2001) to enable common elements to be identified but allow interviewees to express ideas or opinions. The schedule for the interviews was based on that used for other cohorts interviewed during the wider OLT-funded project on RSDF evaluation (Willison, 2014) and the interviews were conducted by a research officer who had extensive qualitative research experience. They were recorded, transcribed by a professional transcription service, de-identified by the research officer and then passed on to the authors for analysis.

The authors read each transcript independently to identify sections relevant to the research questions, the initial organization of data was compared for areas of disagreement, and then multiple analyses were conducted. The first stage of analysis targeted student statements that related to each of the facets of the RSDF in employment, and determined both in a general sense and facet-specific sense how students related to the research processes articulated by the RSDF. The second stage of analysis examined the division into employment group types, and determined participants’ experience of both general and media-specific research skills. Finally, contradictory data were noted and the researchers continued to use constant comparison of these emerging findings until agreement was reached about the final interpretation of the data given the initial research questions.

For the purposes of analysing the interview data, participants were sorted according to their employment using categories based on Cunningham and Higgs triad criteria (Higgs & Cunningham, 2008). Participants were variously engaged in:

Media work for a media industry organization (n = 3; students identified below as C, D and E)

It is important to recognize several limitations in this study. Post hoc accounts may be biased towards ‘positive reporting’, therefore our data may be skewed in a positive direction. Those interviewed do not, as previously stated, represent the entire cohort of graduates in this year group, and the results do not generalize to other graduation cohorts. Moreover, only those who graduated were interviewed, saying little about those who did not complete their course of study. While the numbers for most employment groups were low, a sense of diverse perspectives, rather than representative perspectives, within these groups emerged; however, only one graduate from the ‘non-media work/non-media industry’ segment was interviewed, potentially restricting understanding of this category. A further limitation is that we were only able to capture a ‘snap-shot’ of opinion from recent graduates, which means we have not yet been able to fulfil our ambition to examine long-term changes in graduates’ opinion as their careers develop and change over time.

Graduates’ Reflections on the Use of Research Skills in Employment

Looking first at attitudes to research education in general, irrespective of employment type, participants often reported that the research skills learned in their studies were ‘directly applicable to what you’re currently doing’ (participant F). In the words of another participant, ‘independent inquiry is actually very paramount [sic.] to what we end up doing or hopefully what we end up doing’ (participant G). Another said, ‘[I] use it every day now’ (participant C). There was also consensus that the six facets utilized in the RSDF clearly mapped onto the cognitive processes they were required to perform in their different work contexts. Participants frequently provided detail about specific facets of research that emerged in their degree finding applicability in the workplace. For example, in relation to the facet ‘Evaluate and reflect’ one (participant A) stated:

When you’re actually in uni’, they nail you in everything you say that you need evidence for it, you need intro evidence and explaining of how you came to this result. It’s kind of like that in the workplace.

Another (participant F) noted,

One of the things I found most helpful was being asked to contextualise what you had researched possibly in isolation. So always looking at something compared to a bigger picture or something else, so comparing and contrasting, which we did a lot through the undergrad degree.

Nevertheless, there was less agreement among graduates about the need for the use of the structured RSDF. Some graduates stated that the use of the RSDF facets aided their learning. One participant (H), for example, stated that their use enabled them to develop an understanding of ‘what you need to know’ that persisted into employment, resulting in knowing ‘better now how to go about it’. For this graduate, knowing the facets meant, ‘we can tell if we’ve improved’: a use of the RSDF in employment as a kind of benchmark to determine performance effectiveness. Another graduate reflected on a missed opportunity:

[A]t the time I didn’t really take it in as much that this was a step-by-step guide that I’d be using in the future, because I guess I kind of thought that it was common sense, but looking back, yes, I use it every day now (participant C).

By contrast, some participants perceived the explicit articulation of the facets to be unnecessary for university-level study: ‘I think it’s way too over the top. It’s common sense, that you need to find things out and analyse the way you find things out’ (participant E). For this participant the RSDF provided an articulation of cognitive skills that accurately reflect those demanded in their work, but which was unnecessary due to their perception that these did not need to be taught in an explicit manner.

Looking now at the perceived value of training in the particular research skills for specific professional contexts, we see a distinction between participants employed in the ‘Media Work/Media Industry’ category and the others. Interviewees in the former category frequently communicated concerns regarding their research education. Two of the three participants doing media work for media organizations, distinguished between the research skills learned in the course and ‘the research skills I guess I use as a journalist, of course, though in terms of just reading council documents and ringing people up, none of these sorts of different research methods we learnt in [the course] I generally apply in my day-to-day work’ (participant E). One of these participants noted that they would have preferred to learn journalism-specific skills, such as ‘learning how to interview as a journalist as opposed to as a researcher’ (participant D).

One participant engaged in media work for a non-media employer also said that in their employment context, ‘We haven’t really done any sort of market research, which is what I feel [the course] really provided an education in’ (participant B). They added that, ‘there’s probably scope to do that, but more so when we might be planning for how we communicate and engage our members more effectively’. This participant raises the possibility that the present lack of application was not necessarily the whole story. One participant (H) doing non-media work for a non-media employer found that a specific subject (Media Research Methods),

… was the course [meaning subject] that sort of—well, whilst the other subjects sort of helped me to do general research, this subject in particular helped me to learn a bit more about research, like different types of research, and what the advantages and disadvantages that are associated with that. So obviously that’s more specific and sort of more helpful, although I haven’t really used much of it yet.

While this participant was willing to accept that research skills might be useful in the future, despite not being used at the present time, perceived lack of current application led some participants to conclude that the discipline-specific media research methods subject did not provide useful skills. Participants did, however, also need time to reflect in interviews; a student who had generally had been negative about the RSDF-informed learning process said towards the end of the interview that skills developed in the course ‘have come in handy, now that I think about it. With the stuff down in [the town where I work], media laws definitely come into place because we have to interview minors, and as far as researching questions that are applicable for young people and old, drawing up contracts and that kind of stuff, understanding exactly what all our contracts imply probably helped’ (participant G).

Finally, we turn to the development of metacognition, the capacity for self-awareness about one’s own thinking and learning processes, which is particularly important for graduates when they are first learning how their various skills and study experiences might be applied in different employment contexts. Such metacognitive abilities were identified at various points in the interview data. One participant found that ‘the strength is it really introduced me to this idea of research. It really introduced the basics of what you need to know, and then you can just sort of apply it according to what you do’ (participant H). In agreement, another interviewee stated that there was ‘not really any facts or any specific knowledge that it’s teaching you; it’s basically scheming up for the skills that you need when you’re actually in the workplace, how to solve problems, how to communicate with others, and how to deal with communicating your ideas’ (participant A).

A participant (G) from the ‘Media work/Non-media Employer’ group expressed this metacognitive thinking very clearly:

… really look back on what you’ve done and what you’ve learnt and how you could do it better, all that kind of stuff, was always really rammed into us when we were studying, definitely, to be quite introverted into what you were doing and why, and how it could be improved in the future.

A ‘Non-media/Media’ participant (F) stated the concept thus:

... eventually it becomes not just autonomous learning but a questioning of the process and system itself, where you can actually review your own process of learning or system of whatever you’re doing, so you can actually change it and improve it. I think that’s something which is going to be really important for me in my career going forward.

They saw that awareness of one’s learning through the facets of the RSDF enabled autonomous learning and critical thinking. Metacognition and critical thinking went hand in hand and were perceived as important for current and future employment.

By contrast, participants in the ‘Media work/Media industry employer’ segment often had a practical, time-constrained approach that influenced the cognitive skills used (or at least their perspective of them):

We get a press release. It says the government are cutting back 30 jobs at the local electricity office. We then want to find relevant information to that as to who is losing their jobs, what sort of staff are losing their jobs and how it’s going to affect local people, where the jobs are going, why are they cutting them back … (participant E)

Thinking of information in terms of who, what, how, where and why is a familiar approach to journalistic research, but the participant went on to say:

You don’t sit back and think: how am I going to approach this research? How am I going to work out who lost their job at the electricity company? How am I going to tell everybody how they did that? When you work in journalism, in particular, you do everything in a matter of six hours. You’ve got a day to pool this so-called research; you’ve got a day to put it all together and get it out there.

This perspective gives little consideration to reflection, due to practical constraints, and reflects the more immediate media production cycle. This indicates that students who perceived the six facets of research in the RSDF to be intuitive and ‘common sense’ tended to disparage the need to think explicitly about these skills.

One of the most interesting comments with relevance to the development of students’ metacognitive abilities came from a participant (G) who stated that the aims we sought to achieve through its explicit use would instead ‘have to be hidden under a veil of something much cooler’ in order to appeal to media student:

I think students tend to not like the concept of a super organised learning system, particularly when half of them would consider themselves like very, very creative. So yes, if you try and get creative people to work within parameters, then a lot of them don’t like that.

The implication is that ‘research’ and ‘research skills’ may not be perceived by students to be relevant to life after university, because they are not deemed to be relevant to creative thinking. This may be of concern given the extent to which content and product development in different segments of the media industries is heavily reliant on different forms of audience and market research (Napoli, 2003).

Discussion

The comment that the RSDF facets might be more appealing to students if they were hidden under ‘a veil of something far cooler’ provides an interesting perspective on curriculum alignment (Biggs, 1999), because it appears to challenge the current emphasis in higher education on the development of explicit statements of learning outcomes. Indeed, it may often be tempting to try to maintain student engagement by concealing our true intentions as educators. However, following Wiggins and McTighe (2005), we argue that there are risks inherent in this approach, because it obscures the need for research to be conducted as an active cognitive process that rigorously investigates and reveals areas of uncertainty. In addition, ‘veiling’ does not take into consideration the needs of interviewees who stated that they benefited from explicit articulation of research as a type of activity that utilized distinctive practices that could be applied across a variety of contexts. ‘Research’ is the most appropriate term for this activity, but its relevance and use must be introduced early in students’ education and frequently revisited if we wish to reveal, rather than obscure its usefulness in employment and the transferability of research skills to other contexts.

The way that participants from several employment categories articulated their own metacognitive processes, facilitated by the use of the RSDF facets, supports this requirement to be clear about the need for research to investigate uncertainty and challenge orthodoxy. One participant conveyed a sense of incredulity about the relevance of research skills when studying at university, but later surprise when these, ‘might actually be directly applicable to what you’re currently doing’. Even one of the more sceptical graduates quoted earlier confirmed that, ‘I didn’t really take it in… but looking back, yes, I use it every day now’. These comments indicate a need for students to develop a heightened awareness of the relevance of these skills for future careers and career development earlier in the degree, in agreement with findings in the broader study (Willison, 2014). A course designed without a coherent conceptual framework may provide variety, but may also prevent students from generalizing ‘to their learning experience as a whole’ (Adcroft, 2011, p. 417). In other disciplinary contexts, there is evidence that multiple exposures to explicit research skill development consistently framed by the RSDF in a variety of courses were important for developing increased metacognitive awareness through increased coherence (Willison, 2014).

Our participants indicated diversity in the research skills needed in different employment destinations and in the type of learner they perceived themselves to be. They appreciated the need for research skills, but as indicated by previous research (Ashton, 2011) invariably valued them most highly if they were clearly seen to be applicable in their current employment context. One study on outcomes of quantitative research courses found that less than half of students surveyed (n = 168) thought that research skills would be useful in subsequent employment (Murtonen, Olkinuora, Tynja & Lehtinen, 2008). However, in a study (surveys, n = 601; interviews, n = 46) of outcomes of explicit research skill development using the RSDF in content-focused (non-research methods) subjects, the sense of usefulness of research skills for actual or anticipated employment was prevalent (Willison, 2012).

This misalignment appeared to be most acute in relation to the experience of those graduates who found media employment in media organizations and were working in entry-level journalism positions, which confirms observations made by Callaghan and McManus (2010) that employers in the journalism industry are most likely to emphasize the need for ‘basic journalistic skills’ (p. 11), as opposed to ‘higher learning’. However, if one goal of media education is to challenge existing practices, then ‘critical thinking’ is a prime consideration for graduates and the development of metacognitive abilities that enable analysis of one’s own thinking is important (Callaghan & McManus, 2010). The participant who linked thinking critically about the system to metacognition demonstrated this: ‘a questioning of the process and system itself, where you can actually review your own process of learning’. Others struggled to achieve the metacognitive capacity required to understanding the relevance of more generic research skills to different employment contexts. Given the diversity of participants’ employment, even in this limited sample, it is not surprising that these tensions were evident in the interview data.

Conclusion

The diversity of employment outcomes and complexity of the graduate work environment in the media industries, presents significant challenges for research skill development in undergraduate media courses. Nevertheless, qualitative feedback from graduates can provide educators with invaluable insights on how they can meet this challenge. Our data suggest that an increased focus on students’ metacognition through the systematic application of a framework for research skill development may be an useful way to bridge potential gaps between undergraduate study and employment, because this helps to prevent the development of ‘asymmetries…[that] may constrain the realization of a boundary less career’ (Defillippi & Arthur, 1994, p. 320) and facilitates the transfer of learning between different employment situations. This could prove to be more effective than attempting to create courses of study that are highly specialized in an attempt to address every nuance of every skill, especially in fields where rapid technological change risks making skills learned in a degree program obsolete even before graduation (Berkeley, 2009). Use of the RSDF may assist in this by helping students to develop a more enduring skill set suitable for the uncertainties of employment immediately after graduation and for the future, when increased autonomy and critical thinking are vital for success in different media careers (Dickson, 2000, p. 173).