Abstract
The BBC’s online news provision has been increasingly pressured by forces that are both internal and external to the corporation within recent years. In the light of this, the concern here is twofold. First, the aim is to produce a more comprehensive framework for the monitoring and evaluation of the quality of specific, key aspects of the corporation’s online hard news output than currently exists. It is intended to be usable not only by expert outsiders looking in at the BBC’s standards, but, with a little basic training, by the organisation’s editorial staff, should the corporation choose to consider its merits and agree its adoption. The second aim is to provide a means of helping maintain and increase the quality of the same key aspects of BBC news output at the stages of reporting and editorial decision-making within ongoing stories.
Introduction
The starting point for this article is the increasing pressure that technological change, tight budgets and perceived shifts in audience interest are placing on those wishing to maintain what they see as traditional news quality standards within the BBC as one of the world’s flagship news producers. Such pressures have raised serious doubts about key aspects of the future of the corporation’s news provision.
In the light of this, the concern here is twofold. First, the aim is to produce a comprehensive framework for the monitoring and evaluation of the quality of specific, key aspects of the corporation’s online hard news output. Our intention is that it should be usable not only by expert outsiders looking in at the BBC’s standards, but, with a little basic training, by the organisation’s editorial staff, should the corporation choose to consider its merits and agree its adoption. Having discussed current BBC quality monitoring procedures with a member of senior newsroom staff during our research we are convinced that what we set out here would represent a considerable upgrading of these. Part of what we suggest in this article does already occur within the newsroom, in the form of rigorous discussions between relevant journalists and editors. However, existing procedures stop well short of the formalised structure and comprehensiveness of the sophisticated, but easy to use, ‘colour coded’ monitoring and evaluation framework contained within the ‘matrix’ approach that is outlined here.
The second aim is to provide a means of helping maintain and increase the quality of the same key aspects of BBC news output at the stages of reporting and editorial decision-making within ongoing stories.
The intention ultimately is to produce a measuring device that marries the practical needs and limitations of online platforms with a clear articulation of what best-quality hard news coverage could look like within such boundaries. The latter is grounded not in unrealistic ‘academic theorising’, as the news industry might see it, but in a combination of two things: first, the types of hard news and analysis 1 that are needed if democratic societies are to function effectively; and second, the types of hard news content and storytelling that are needed to get the levels of audience attention that will attract and keep advertisers or, in the case of the UK output of the BBC, justify licence fees.
There is space only to address the first of these concerns here. The second will be covered in a future article by one of the authors.
Appropriate quality standards
Before proceeding any further it is necessary to acknowledge that quality is a highly contested topic when discussed in the context of news journalism, 2 in terms of both how it should be defined and how it should be measured. Frequently, for example, it is alleged to be lacking as a result of the presence of alleged bias within news reports (see for example the carefully qualified argument in Anderson and Weymouth, 1999). Such bias can be argued to result from what is excluded from reports as well as what is included (Anderson and Weymouth, 1999; Fairclough, 2001: 58). But this perspective on quality is countered by others who see declaredly biased journalism as ethically sound and realistic quality news reporting (particularly given the problems with achieving balance and objectivity), as long as it is based on rigorous sourcing and evidence building with a dedication to fact checking (see for example, Starkey, 2007: 59–60).
Furthermore, different understandings of quality journalism exist within different continents, even when all of the states involved are liberal democracies. In the case of Kenya, for example, Anderson and Ogola argue that quality can be measured in terms of the extent to which journalism provides the kind of information and forums that facilitate the continuing development of democratic and economic/societal structures (2013). This contrasts noticeably with several prominent traditions of western-shaped preferences, one of which, for example, stipulates simply that journalism should meet the democratic need for rigorous, in-depth information providers or watchdogs with regard to the platforms and records of political parties and the key political players (Stromback, 2005). The structural components of the Kenyan definition derive obviously from the specific needs of a developing society.
Equally, different levels and qualities of news information are argued to be required for different types of democracy. Stromback (2005), for example, has produced a neat, four-fold typology of democracy which illustrates how, at one end of the spectrum, ‘procedural democracy’ simply requires of journalism that it monitors the carrying out of election procedures and reports on any malpractice or incompetence in this regard. Near the other end he identifies what he terms ‘participatory democracy’. This requires that journalism should not just help monitor the fairness of election procedures and report adequately on the competing records and platforms of rival political parties, but also provide citizens with a level and quality of information that will enable them to become participants in the political system between, as well as during, elections. If governments or local authorities under-perform, or break their promises, or if order within the society they are supposed to govern breaks down, then, for example, it is the job of quality news journalism to provide the citizenry with the key details of what is going wrong, together with the information that is necessary for the electors to attempt to hold those elected and their officials to account. Participatory democracy requires journalism to frame politics as a process open for the active participation of everyone.
Looking at what the best of BBC journalism has tried to do over the years, then this to us would seem to approximate to the type of journalism expected within this type of participative democracy. For the purposes of this article, the participative democracy level of expectation of news provision will be the one that is implicit within our definition of quality news journalism (outlined shortly). Our grounds for this are that, bearing in mind the regulatory requirement for the BBC to be impartial across the range of its coverage, it would seem logical to take what it itself presents as its highest quality of journalism (BBC, 2010a, 2010b; Hockaday, 2011) and use that as the yardstick against which its performance should be measured. It is this kind of journalism that the brand and reputation of BBC news journalism has been built on, so our question then becomes, ‘what is needed to monitor and guarantee this type of quality journalism online?’ It is in answer to this that we have constructed the matrix approach, which we explain shortly.
However, the clarity of this approach to the definitional problem does not in itself resolve the issue of what it is that constitutes the various essential parts of this high quality form of journalism and which of them a cost-constrained BBC should be expected to monitor when evaluating its own performance. Clearly, one of the key factors is the set of news values that are applied in the selection of stories for coverage (and the extent to which they fit with the needs of participative democracy) and the way that the hierarchy of news agendas is decided as a result.
Other key aspects of the kind of news quality necessary for participative democracy include the nature and range of the sources that are used, the range of angles that are covered, the extent to which background context is provided, the degree to which reports are based on accurate and well-researched facts and the skill with which stories are written in order to communicate maximum understanding and interest across a target audience. The BBC’s news output has in the recent past come under attack regarding its performance in various of these quality category areas from a range of prominent sources, including Wilson of Dinton (2005), Lloyd (2004) and Davies (2008).
A variety of research methods is available for measuring these various aspects of quality. The extent to which the rigorous and accurate reporting of facts has been distorted by ideological bias within news reports, for example, has been frequently identified via the use of content analysis, most prominently and controversially perhaps through the work of the Glasgow Media Group, and critical discourse analysis (see Anderson and Weymouth, 1999; Fairclough, 2001), with the most well-funded studies using both. The skill with which stories are written to communicate maximum understanding and interest can be assessed by the kind of focus group methods used in a pivotal 2001 study of UK television news reporting of the developing world, which included involvement from senior BBC journalists (Glasgow Media Group, 2001). All of the available methods are subject, to one extent or another, to the limitations of subjectivity.
So, in theory, all of the above components of quality news journalism could be assessed, subject to the inevitable limitations of subjectivity characteristic of each of the available research methods. However, the obvious question arises as to how much it would be reasonable to expect of a news organisation that is subject to tight public controls on its overall budget. The level of cutbacks the BBC’s news operation has endured in recent years automatically means that such time- and labour-intensive research methods as critical discourse analysis have to be ruled out and left as the province of well-resourced university research teams. However, what is ‘affordable’ will be affected not only by the overall news budget but also by the importance of the reputation of the BBC news brand – something the corporation cannot afford to lose. We would argue that, financial constraints or not, reputation in this context is so valuable a commodity that the quality monitoring that is funded must be able to assess in meaningful depth a sufficient number of core quality indicators (relevant to participative democracy) for the evidence it produces to be regarded as a credible portrait of news performance on specific issues. In the event of a serious controversy over BBC coverage, the likely key critics and stakeholders who will need to be convinced will include parliamentary specialist committees and viewers’ and listeners’ watchdog groups. It is the quality and detail that will be necessary to most effectively respond to their questions that should be kept in mind.
This is the type of quality monitoring that will underlie the framework or ‘matrix’ that we are proposing here.
Quality and the specifics of the online news environment
The quality of an online news site can of course be monitored with regard to several dimensions. Besides the main straightforward text format of news reporting which makes up a major part of the content currently on the BBC online site, it is possible to monitor and assess the quality of the ‘audio only’ content, the use of still photographs, the audio-visual content, the ‘moving visuals only’ content and the ways in which user-generated content (UGC) is managed and utilised. The detailed quality monitoring of all of this is a considerable task for any organisation, particularly one with the huge range of output of the BBC, and from a practical point of view the devising and explanation of a suitable quality monitoring framework of the necessary scale and detail also is beyond the capacity and scope of a single journal article. It is proposed, therefore, to focus here on the two most crucial parts of the content within the current design configuration of the BBC site, namely the text material, together with the most appropriate aspects of the audio-visual content.
In defining the text and audio-visual content as the most crucial on the site we have used simple but effective criteria of judgment. One approach employed was simply to measure the total area of space given to the visual image, including graphics, still images and video – and the total amount devoted to text. With regard to the stories used in the case study for this article, it was found that if site users navigate away (click through) from the BBC home page to the stories themselves then they are predominantly text based – typically around 80 per cent text (measured in square mm of page coverage.) However, this is not always the case. Much seems to depend on the availability of footage and images from across the BBC. Measured in the same way, the home page varies a little from day to day, but is (marginally) predominantly text based (we should add that our measurements were made using printed screen dumps). However, if the measurement includes ‘heat mapping’ (a process that assesses where eyeballs first fall, how long they spend digesting an image, etc.) then the perception of which media dominates a page is likely to be different again. There are problems therefore in declaring either of these formats to be the dominant one across every context so our approach has been to produce methodologies that will deal with each. Because, for the reasons stated above, it is not possible to cover all visual images here, it was decided to focus on the audio-visual, given the long-existing evidence that it remains the dominant format for news reception within the developed world (OFCOM, 2010; Pew Research Center for the People and the Press, 2010) and our assumption is that the take-up of the audio-visual sections of the website, while not necessarily exactly mirroring this preference, will nevertheless be significant.
It is important to stress that on-site linear content, in the form of direct and continuous access to the BBC News Channel, is not what is being referred to here. By definition, the BBC online news site is designed in a manner that allows users to choose for themselves which news items they will access and within the context of such a site design linear news programming is an ‘add-on’ rather than an integral part of the dominant site framework.
A great deal of attention is paid to user-generated content within the BBC itself, other major global news brands, such as the Guardian, and within recent academic literature on journalism (e.g. Harrison, 2009; Singer and Ashman, 2009). However, the available evidence shows that, despite all of the attention given to UGC and, indeed, the BBC’s avowed intention to make full use of it where appropriate (BBC, 2010c), only a minority of BBC online news site users engage in anything bar the most simple forms of interaction, such as the sending in of uncontroversial digital photographs (this was established simply by examining the number of users’ comments relative to the number of known unique users over a constructed week). From a practical point of view, therefore, UGC will not be covered within this article, given the limitations of space and the fact that, whatever the hype, at the moment it remains far down the scale of audience priorities. The concern here is to produce a quality monitoring framework that will focus on the areas where most users are likely to notice and judge the quality of what they receive from the BBC.
Having outlined these initial key considerations, the intention now is first to elaborate the quality monitoring procedures that we propose with regard to the textual content of BBC online news reports and second to explain our proposals concerning audio-visual content.
The quality analysis of the textual content of online news reports and our definition of the core essence of news quality
What we have in mind for the quality monitoring of the textual content of online news reports is a framework, or ‘matrix’, that will enable editorial teams working in online journalism to both monitor the quality of news output on particular hard news topics and provide journalists with yardsticks against which to check the quality of their own coverage across the running length of a story. From both an editorial monitoring point of view and a correspondents’ ‘on the job’ point of view, the matrix approach will help them establish to what extent the range of relevant angles has been covered relating to a particular ongoing story, together with how well that story has been covered in terms of the discussion of context or possible causal factors and the provision of links. This kind of matrix is designed to be both a tool that can be used across a range of hard news topic areas to help ensure quality in the coverage of issues that arise so quickly that there is little time for thought as to how they should be covered, and a template that can be produced rapidly when a workable ‘standard operating procedure’ becomes desirable.
The fact that online journalism allows for the continuous updating of stories means that different sections of the audience may view different drafts of stories on particular aspects of an issue (in the case of the BBC, it should, of course, be noted that continuous updating is standard practice across all platforms). Because not all users will necessarily go to the final draft, having read earlier versions, the question arises as to whether the earlier drafts should all be monitored and evaluated as well as the final version. Our answer is that, ideally, they should if an adequate picture is to be gained of the level of quality received by the audience as a whole, but ultimately it is a resource decision for BBC management as to whether such a comprehensive process is affordable.
In the case of the BBC the overall online coverage of a story should provide what might reasonably be determined as:
The range of key facts and relevant most credible (in terms of the available evidence) causal relationships relating to the issues at its heart.
Where relevant, possible solutions for problems caused by the latter, or which caused them.
The full range of key implications arising from the story.
Crucial links to other reputable information sources on the topic.
This is the core essence of news quality relevant to ‘participative democracy’, as defined for the purposes of this article.
The provision of this type of quality news coverage (in the context of claims to be a quality news provider) requires periodic story brainstorming conferences around specific on-running thematic areas. These should involve journalists from a range of specialisms who can identify angles on a story that impact on their subject areas that a lone specialist on the core issue concerned might easily miss. Such conferences, therefore, will list a variety of key angles from which an issue might be viewed, as they might be defined by a competent range of journalists and news editors covering these specialist areas. Competence here implies not only professional good practice in the business of news gathering, but a recognition of the importance of keeping abreast of developments in academia and other centres of ideas generation, as well as the wider outside world, that will ensure that journalists stay at the leading edge of their game. It is only this type of forum that will allow the full picture surrounding an issue to be built up and key news and views issues that might otherwise be missed to be picked up.
In line with current editorial practice, it is important that news production systems like the BBC’s ENPS are used to formally minute these conferences and that the minutes are circulated rapidly to all involved for three reasons. First, so that, editorially, a precise record can be kept of what it was that was decided – it is only then that meaningful monitoring can be conducted of the extent to which that which was decided was delivered; second, so that the journalists concerned have something to check their own record of what was decided – this should help them stick to the quality standard of coverage agreed and produce better stories; and third, in the event of outside criticism of the coverage, the BBC then has documentary evidence of its own attention to detail to produce in its defence.
Applying the quality standards
On the news quality promotion side of the fence we are acutely aware that if a complex story is to be covered well then journalists need something that can be produced quickly and which is simple to use. On the quality monitoring side of the fence, we are aware also of the severe pressures that have been placed on the BBC news budget in recent years, particularly as a result of the costs of the digital switchover, and of the coming cut in the online news budget of 25 per cent by 2013 (BBC, 2010b). Our proposal, therefore, is for a matrix that is both simple to use for quality promotional purposes (in the context of rigorously minuted editorial conferences, populated mostly by people of high levels of experience and expertise) and simple to apply for subsequent quality monitoring and assessment purposes.
The principle on which the conferences should be run – a systematic debate and discussion amongst specialist staff to identify all of the key angles relating to a specific ongoing issue – has been identified earlier. We emphasise ‘systematic’ to distinguish what we are proposing from examples of best practice that occur already. The ‘systematic’ matrix approach starts with the identification of the most obvious key questions and issues that a particular story raises across the range of the relevant specialisms upon which it might impact. It then goes a stage further and identifies and dissects the range of sub-questions and issues that arise in each of those areas. The importance of these is then explained, debated, decided and minuted within the conference format. This then helps guard against the possibility of, for example, the kind of major ‘white van man’ issues that were ignored by the BBC prior to the 2001 fuel protests being missed (Drury, 2007). In the process a checklist is produced, against which journalists can monitor their own performance and which editorial staff can also use to monitor the overall coverage of the story at various points. The extent to which individual questions are pursued, explained and analysed should depend upon the decisions made about their relative importance during editorial conferences (bearing in mind that developing events may require alterations to the matrix), but at a minimum the full range of key issues identified within the matrix should have been covered by the end of a story’s run. Equally, adequate context should have been provided and subsequently referred to in reputable links by the story’s end. It needs to be stressed that the assumption that these two tasks will be undertaken satisfactorily is at the heart of the matrix process.
The quality monitoring of the coverage that we suggest is tuned to the limited resources likely to be available and is facilitated directly by the structure of the matrix. Whilst electronic news production systems may include sophisticated software to host the matrix, even within a programme as simple as MS Word, a minuted version of the matrix could be colour coded in the manner demonstrated below, i.e. with each of the overall key questions highlighted in a different colour. The minuted version will, of course, first need to be reorganised into the kind of matrix structure shown here. In the absence of emerging meta or semantic search technologies used increasingly in online environments and at relatively small expense, editorial could employ a small team of monitors to go through the text coverage of a story over its running length (or at various points within it) and highlight in the relevant colour each portion of text in which specific overall questions are referred to. It is explained later in this article how the colour coding can be handled for portions of text to which several category areas of the matrix apply. The sub-issues and questions identified as deriving from each overall question within the matrix would be assigned a number from 1 onwards and the monitors would then go through each highlighted section and, for example, mark (via the insertion of the relevant number) in a larger bold font each point at which the sub-questions are mentioned. They would also note those questions which were not addressed.
Subjectivity, is of course, always the key issue, and not least among the problems that it throws up is the need to ensure that the monitors used are skilled enough to identify the less obvious matches between the matrix questions and content and to handle sensibly passages in which several key questions might come into play. But what is being suggested here is a refinement of and a ‘step up’ from the quality monitoring procedures previously applied at the BBC, or at other news producers who rely on high quality reputations (e.g. Palmer, 2003). To illustrate what we have in mind we have produced a quality matrix that could be devised by relevant correspondents within the BBC within the format described earlier and which in turn could be applied across a wide range of issues relating to Russia, including the 2008 military confrontation with Georgia. Russia is chosen as our example so that we can show how the matrix can be used to meet even such challenges as the need to ensure adequate coverage (and monitoring of the coverage) of highly complex issues relating to enormous and powerful states. A full matrix is drawn up for this purpose and, as is explained shortly, is also applied in a linked case study example to demonstrate fully how it can be used in practice. While one of the authors is sufficiently well informed regarding the chosen case study area to identify most of the key points that need to be identified and has relevant academic expertise, he would not claim to be an expert Russianist and it is important to understand, therefore, that the material presented here is for illustrative purposes and does not claim to cover every angle that subject specialists might identify as being important. The example does, nevertheless, in itself demonstrate what editorial conferences within an organisation with the level of expertise of the BBC should be able to produce without great difficulty.
To avoid confusing readers at this point we delay the final, more detailed elaboration of the matrix until its basic outline has been presented below. Because journal articles are restricted to monochrome fonts and tabulation we have indicated in brackets the colour that each section of the matrix would be coded by.
After it has been outlined and elaborated on we then proceed to explain our proposals for a simple but effective means of monitoring online audio-visual quality.
Matrix produced as a quality reference point for coverage of stories about Russia
It is important to note that we have included more detail here than probably would need to be set down by experienced correspondents to make the principles underlying the application of the matrix more fully apparent to a wider readership than they otherwise would be.
When Russia is reported as having done something, either allegedly negative or positive, to what extent should the relevant factors below be taken into account to help evaluate in a fair and balanced manner whether its government’s actions match the claims made for or against them?
a) The actions of other states, business corporations, international organisations, etc. (red in matrix when it is applied)
To what extent should the actions, interests, ideologies and wider motives of states and individuals, political parties, etc. that criticise, praise or are alleged to be victims of Russia, be taken into account in stories concerning the Russian government and its actions?
1) For example, to what extent should the possibility be taken into account that, in the light of one or more of the historical factors outlined in section (b) below, specific actions of the USA in Poland, Georgia or Ukraine might automatically be seen as threatening to the security of the Russian people – inside and outside of Russia’s boundaries – by Moscow?
2) To what extent should attempts by the USA and its NATO allies to advance their economic interests in the former USSR and contiguous territories be considered as possible automatic generators of suspicion, or the need for defensive actions, on the part of the Russian authorities’ given traditional Russian concerns about the West and its ‘economic imperialism’?
3) To what extent should the fact that the attitude of the EU towards Russia can be seen in Moscow as signifying only a partial acceptance of it as a member of the European society of states be taken into account?
4) To what extent should the fact that the western expansion of influence, economic power and military power into and around much of the former USSR has been felt as a national humiliation – and something that Russia would be justified in seeking to at least partially reverse – be taken into account in stories about Russian actions towards its Eastern neighbours or the West? Should the West be surprised if, where it senses that they would be unwilling to take any effective action to prevent them, the Russian government from time to time acts assertively to roll back some of that influence, re-establish lost ‘dignity’ and draw clear lines in the sand?
b) Historical context (turquoise in matrix when it is applied)
To what extent should the following factors be mentioned and offered as part of the explanatory function of a story on Russia, where relevant?
5) The fact that for several centuries there have been traditional Russian concerns about security deriving from the difficulty of defending large borders with areas of significant vulnerability in terms of the geography of the land. These extend to a belief periodically in the need to intervene in adjoining states that threaten to destabilise areas around its borders, or provide its rivals or enemies with bases, etc.
6) Deriving from this, the fear of encirclement.
7) The historical preference to establish as much as possible of a buffer zone in Eastern Europe as a result of the massively destructive lessons of invasions by Napoleon and Hitler using the ‘Polish corridor’.
8) The historical evolution of the idea of the Russian nation and Russian nationalism with two key trends within it – i) expansionism, stretching back to the original city state of Muscovy and ii) the idea of the protection of the dignity of the Russian nation. Expansionism became a self-fulfilling prophecy as each new expansion of the borders by definition increased the size of the territory that needed to be defended and introduced new areas of geographical vulnerability, which produced an incentive to expand further to try and acquire more secure borders and bring under direct control peoples who may threaten Russian security, etc. The acquisition of new borders often brought new areas of vulnerability rather than greater security, helping to keep the process of expansion going.
9) The argument that inherent within the centuries of expansion outwards from the Muscovy city state there have frequently also been imperialist tendencies that keep resurfacing and may be resurfacing now. This imperialism can be economic, political and/or military in form as circumstances and the preferences of Russian governments permit.
10) The evolution of the idea of the need for a strong leader to hold Russia and its diverse and extensive territories and its people together. The word Czar, for example, is the Russian version of Caesar. This can lead to pressures on Russian leaders to demonstrate how strong they are abroad as well as at home and that may be part of the explanation for Putin’s actions.
11) The fact that Russian social evolution was slower than in Western Europe due to the persistence of a peasant society and feudalism, and the fact that industrialisation progressed a lot more slowly than in the West during the 19th century. This in turn led to a much more limited experience of democratic ideas than in, for example, Britain and France. The Russian experience of democracy therefore has been very different to these two countries and was very short-lived prior to the takeover of Lenin and then Stalin.
12) The fact that even after Peter the Great and Catherine the Great tried to modernise Russia and learn from Western Europe, Moscow was only half accepted as part of Europe by the other European states. Russia therefore has had a split identity historically – as part of Europe and yet outside of the European mainstream. This has been maintained by the post-Cold War attitude of the USA and the EU states towards Russia.
13) The fact that the Russian experience of greater democracy (than has existed before or since) under Yeltsin coincided with weak (in the sense of being incoherent and illogically impulsive) government that introduced market reforms too quickly and with too little consideration of their consequences, together with massive economic dislocation and the considerable suffering and social disadvantage of millions of people who had benefited economically under the old authoritarian Soviet system, together with the disproportionate growth in power and wealth of the oligarchs. Media freedom, for example, became seen by many as freedom for the oligarchs to advance their own interests through the media that they owned. All of this has been argued by some to be part of the explanation for why people have been prepared to sacrifice significant aspects of democracy in return for the strong governance under Putin and Putin/Medvedev that has apparently dealt with a number of these problems to the satisfaction of key parts of the population.
14) The fact that the USA has pursued a policy of what might be perceived as expansionism towards Moscow during and since the Reagan administrations of the 1980s can be seen not just as a humiliation (see 4 above), but also as a threat to security. Accelerated US arms racing crippled the Soviet economy and hastened the fall of the Soviet state and empire. Thereafter the twin policies of extending NATO and EU membership right up to Russia’s borders during Russia’s period of economic and military weakness (actively supported by some of the USA’s key allies in Europe) and the political chaos under Yeltsin, together with the direct military interventions in Bosnia and Kosovo, have impinged directly on Russia’s traditional border security and encirclement fears and its preference for buffer zones around its territory. This expansionism could be seen as having continued up until recently with US policy in Georgia and Ukraine, for example – and indeed, during the Georgian–Russian conflict.
15) The possible continuing influence of the old Soviet idea of the correlation of forces – the belief that the balance of economic/military/political/ideological forces can be made to shift in a direction that gives Russia additional freedom of manoeuvre and that Russia should take advantage of this when it happens to further its security and ideological preferences, interests, etc.
c) The economic and international political context (green in matrix when it is applied)
To what extent should the following factors be mentioned and offered as part of the explanatory function of a story on Russia, where relevant?
16) The Russian economy has been reinvigorated by vast revenues from the exploitation of its oil and gas resources, which have also funded military reconstruction.
17) Putin also ultimately restored the confidence of the Russian military via his policy in the Chechen Republic and sent a warning to other Russian republics contemplating separation and those in the ‘near abroad’ at the same time.
18) Significantly as a result of the reinvigoration of the Russian economy via oil and gas revenues, and overseas investments facilitated by these, the standard of living of politically key parts of the Russian urban population has risen substantially, providing greater social stability and confidence in, and support for, the post-Soviet system of governance.
19) The position of being a substantial supplier of gas and oil at a time when much of Europe is short of these resources has given Russia increased leverage in respect of the independent territories of the former USSR and Western Europe, together with increased influence in the global system of states and international organisations as a whole, on two levels – the economic and political.
20) This oil and gas supplier power – with a demonstrated willingness to turn off the supply in the case of Ukraine – has invisible impacts in terms of causing the EU states, the USA and the former Soviet states to forego some policy options and actions which might upset Moscow because of their fear of possible disruptions to supply, etc.
21) This new found wealth and the influence it brings, together with the reinvigoration of the confidence and credibility of the Russian military, has enabled Russia to start re-establishing its sphere of influence and buffer zones in Georgia and some other contiguous states.
22) The fact that Russia remains a nuclear power gives it the continuing and often forgotten ability to constrain significantly western foreign and defence policy options.
23) The fact that Russia’s continuing possession of a seat on the UN Security Council is combined with a renewal of its confidence in saying no to the West means that the USA has to start listening much more to its policy preferences.
24) The fact that a reinvigorated Russia, together with cooperation from its intelligence services, is needed in the fight against global terrorism means that, again, on some key issues, the West needs it as much as it needs them.
d) The domestic political system (pink in matrix when it is applied)
To what extent should the following factors be mentioned and offered as part of the explanatory function of a story on Russia, where relevant?
Russia is not a western-style democracy in terms of several key components of its domestic polity and society. This is shown by a number of factors, including:
25) The influence of organised crime and corruption amongst some of the forces of law and order.
26) Limited media freedom – a) opposing and exposing the wrong people in politics can result in journalists being murdered with little chance of their killers being caught; b) Putin moved to curtail drastically media freedom and forced out of the country powerful media owners and personalities who opposed him; those who want to stay in business have to operate within clear parameters and key elements of the media have simply been taken over by interests closely linked to Putin; c) Putin, who is a skilled user of the media, has used this new control to promote his own image domestically as a trustworthy strongman who gets things done and who provides stability and prosperity for many of the key urban classes; and d) Putin has also used this new found power to limit the access of the opposition to the media at election times and ensure that his message, personality and politics and that of his protégés, such as Medvedev, get the most and the most favourable coverage overall. However, Russian skills in manipulating the domestic media are not always carried over into the international context and they sometimes under-appreciate the importance of the foreign media in the global diplomatic game.
27) The fact that electoral politics in Russia works on a rather less democratic basis than in the countries of Western Europe. Putin has engineered a change to the system of governance that, in the eyes of some analysts, has allowed him to continue to be the key figure in government beyond his second presidential term via the election of a ‘puppet’ president. Putin’s placements are seen as having taken power in key parts of Russia.
28) Putin created a system in which he was able to build a significant amount of popular support for, or at least acceptance of, his domestic and foreign policies through a variety of means including media manipulation and control, constitutional and political engineering, his personal charisma and his exploitation of a continuing preference in the Russian psyche for strong leaders. This system continued under the Medvedev/Putin axis.
29) The fact that it can be seen as a matter of national, even moral obligation, pride and dignity for Russia to intervene in other states where its citizens’ lives are threatened and, if necessary, to support Kosovo-like claims for independence on the part of those citizens.
e) Personality factors (yellow in matrix when it is applied)
To what extent are the following factors mentioned and offered as part of the explanatory function of a story on Russia, where relevant?
30) The dominant personality of Putin within Russian politics, combined with and partially resulting from his charismatic effect internally and his political skill in promoting his strong man image and neutralising opponents.
31) The personal history and socialisation of Putin within the KGB. He is a traditional Cold War warrior brought up in a tradition of competing vigorously with the West and what, from a Russian perspective, are western expansionist interests and policies.
32) His visible personal pride and his commitment to Russian nationalism.
33) His ruthlessness, as demonstrated by leaving until the last minute the calling in of outside help to rescue a stricken nuclear submarine, on which everyone on board ultimately perished and his probable knowledge of, if not direct involvement in, the polonium poisoning of a ‘troublesome dissident’ in London.
The matrix in more detail
Now that the reader is familiar with the basic outline of the matrix additional elaborations can be introduced. With regard to the assessment of the quality of explanation within the ongoing coverage of stories (including the discussion of background context and relevant causal factors), a simple but nevertheless useful indicative four-tier system would be used. Alongside each number indicating the presence of particular issues and questions within stories, the letters a, b and c would also be included: ‘a’ would signify an outstanding level of explanation which explores both relevant nuances and the debate over possible causalities; ‘b’ would signify an adequate degree of explanation in terms of what would be needed for the average audience member to understand the key possible causal factors; ‘c’ would signify the provision of a basic level of explanation of an issue with no elaboration; while ‘d’ would indicate the presence of too little explanation for the average audience member to understand the fundamentals of the key issues involved within a story. Obviously the audience research available to the BBC would need to be used to help decide the information needs of the average member, but as this is done already our system would be imposing little in terms of an additional burden in this respect. The presence of reputable links (i.e. those to information providers with a reputation for fact checking and reliability) would be indicated simply by the insertion of a large font, bold ‘L’ wherever they are included within or alongside stories.
Finally, it can be just as useful to identify omitted issues and questions as well as those that are included, and if editorial wishes the matrix can also be used for this purpose. Where significant omissions are identified the relevant paragraph, colourings, etc. would be encased within a three-point shadow box, with the identifying numbers of omitted items being clearly indicated.
A detailed illustrative case study – an application of the matrix to samples of BBC online coverage of the 2008 Russian–Georgian military confrontation and how to access it
An 8000-word journal article format does not allow for a full demonstration of the ways in which the matrix could be used, so the intention here is to provide an illustrative and detailed taster showing how the matrix outlined above could be applied to the 2008 Russian–Georgian confrontation in a separate, dedicated website. 3 This demonstrates conclusively the ease with which the matrix can be applied to reports for post-event quality monitoring purposes, together with the high level of visibility of the various categories of content which the colour coding and numbering system provides.
Finally, now that our proposals for monitoring the quality of the textual content of the site have been fully explained, we can move on to look at procedures that are appropriate for audio-visual content.
Monitoring audio-visual quality on the BBC’s online site
Bearing in mind the considerable resource constraints facing the BBC, and that detailed monitoring of audio-visual content may well be something that has to be done on a sample basis as a consequence, the intention here is to produce an optional ‘add-on’ to the text quality monitoring framework that does not involve greatly complicated tasks, or increase significantly the burden on monitors, but which nevertheless should deliver meaningful results. In pursuing this ambition we have noted that significant amounts of the audio-video content on the online site are being funnelled in from elsewhere within the BBC. To avoid unreasonably over-complicating the monitoring process, therefore, it seems logical to specify that if that content is being used as part of the non-linear section of the site (i.e. we’re not referring to the continuously streamed 24-hour BBC News Channel at the top of the home page), then it should be judged in relation to its functions on that part of the site and not with regard to its point of origin. Once it hits the non-linear section of the site it is addressing a different audience and must be judged with regard to the extent that it meets their needs, not the viewers of the Ten O’clock News, or wherever it first appeared. So if it is used with regard to a particular text story on the site, then it should be judged by the extent to which it helps realise or add to the matrix for that story.
It is proposed, therefore, that sampled pieces of audio-video should be assessed in the manner shown below.
A simple table should be drawn up and entries for each piece of video/audio-video viewed entered within it. The content should be as shown in Table 1.
Monitoring online audio-visual news quality
The nature of most audio-video pieces generally used in the current non-linear sections of the site means that they are short enough for quick and relatively easy evaluation using Table 1. With regard to identifying and recording those categories and sub-categories of a relevant text story matrix that are covered in an audio-visual piece, for example, it would simply be necessary for monitors to note each point as they arise, pause the video, check where it fits (or doesn’t fit) within the matrix for the text story concerned and then note this within the relevant part of the table. As with the analysis of the text-based content, monitors would require training in the necessary evaluative techniques. Most obviously, these would include the ability to adequately assess evidence and argument that is conveyed within the visual and not just the audio content of a piece.
Conclusion
It is not possible to set down in a single academic article all of the detail that ideally could be provided relating to the above. However, it is the authors’ contention that enough information and explanation has been provided both to the make the case for quality monitoring, maintenance and promotion measures such as the above a strong one and to show that their introduction need not impose impossibly heavy burdens on time or resources. All that remains is to finish on two key points. First, it is important to reiterate that our article does not claim to deal with every type of quality measure that needs to be applied to BBC coverage, but only to the specific key angles that are discussed here. It is the intention of the authors to look at some additional quality dimensions in a future article, but to do so with the reality of their economic costs firmly in mind. Second, the importance of the matrix with regard to the role of journalism in a democracy should be remembered. The level and quality of coverage of serious issues which its relatively rigorous and systematic methodology is designed to encourage is absolutely fundamental with regard to the need to have an adequately informed electorate if democracies are to work in a meaningful fashion (Anderson, 2007: 40). While this point has been mostly implicit throughout, it is worth stating it explicitly at the end.
Ultimately, whether or not a matrix approach will be adopted for quality monitoring and/or maintenance purposes is dependent on the BBC’s senior management. BBC staff have told us that they believe they already implement parts of the process outlined here, but it was clear to us that the procedures that are in place are neither as structured nor as comprehensive as those we have outlined. The BBC lacks the matrix and that in turn reduces significantly the quality of information on its news performance that is available both to itself and those to whom it is responsible. We have balanced practicality, dictated by resource constraints, with a degree of comprehensiveness that will still deliver high grade results in terms of measuring and maintaining quality. The case study provides a detailed application of the matrix that has been set out here to demonstrate its effectiveness as a means of monitoring specific key aspects of online news quality. That lengthy case study demonstrates both the matrix’s ease of use in handling large quantities of material and the clarity of its visual presentation of news quality levels. At a time when the News International debacle has raised serious questions about the quality of UK journalism as a whole we would argue that the framework we have provided is an invaluable means of helping the BBC guarantee the reputation of its brand.