Qualitative monitoring of information infrastructures: A case study of INSPIRE

Introduction

The aims to establish a Spatial Data Infrastructure (SDI) for improving environmental data management in the European Community by 2020 (Commission of the European Communities, 2007). The basic objective of this ambitious initiative is to make harmonised high-quality geographic information readily available to support environmental policies along with policies or activities which may have an impact on the environment in Europe. It is a legally mandated programme managed by the EC's Environment Directorate General together with its Joint Research Centre and the European Environment Agency which brings the 28 Member States together to build an SDI based on 34 related data themes.

The EC began working on the Directive in 2001 and it was approved by its Council of Ministers and the European Parliament in 2007. The Directive builds upon the infrastructures for spatial information already operated by the Member States and its detailed implementation is in their hands (Masser and Crompvoets, 2015). The text of the INSPIRE Directive requires that common implementing rules are established for specific areas which are legally binding in their entirety. These include provision for monitoring and reporting of the progress of its implementation. Two sets of implementing rules are outlined in the Directive. One set is based on a quantitative approach based on performance indicators derived from a list of spatial data sets and services developed by the Member States submitted each year. Alongside these quantitative indicators, the Member States are required to provide qualitative information on their progress in the form of written reports every three years. These two approaches are essentially complementary in nature and have to be legally implemented alongside each other in practice.

The main objective of the paper is to examine the use of the qualitative monitoring approach to manage the building of an information infrastructure (II) because this deals with some of the broader issues involved with implementation while quantitative monitoring is largely concerned with the more specific issues of performance review. The paper considers the experience of the implementation of the ECs INSPIRE SDI Directive as a case study of the use of qualitative monitoring in the implementation of IIs. After a short introductory section, the second part considers the nature of IIs that underpin programmes of this kind. This is followed by a section discussing the approaches that can be used for the qualitative monitoring of II implementation. The rest of the paper summarises the findings of the case study of implementing the INSPIRE Directive. It discusses the monitoring and reporting requirements that are set out in the INSPIRE Directive itself and considers the outcomes of the first two rounds of reporting in the European Union Member States and briefly considers the use of these reports in the EC's own Mid-Term Review of the entire INSPIRE implementation process. The concluding section of the paper discusses the findings of the analysis and identifies some areas of research that might be carried out in future work on II implementation.

The nature of IIs

The INSPIRE Directive has many features in common with the notion of an II and this concept provides a useful general framework for the research described in this paper. This term has been used for more than 20 years to describe ‘a shared, evolving, open, standardised, and heterogeneous installed [data] base’ (Hanseth, 2002). Its main features have been described by Montiero et al. (2014: i) in the following way:

IIs are characterised by openness to number and types of users (no fixed notion of “user”), interconnections of numerous modules/systems (i.e. multiplicity of purposes, agendas, strategies), dynamically evolving portfolios of (an ecosystem of) systems and shaped by an installed base of existing systems and practices (thus restricting the scope of design, as traditionally conceived). IIs are also typically stretched across space and time: they are shaped and used across many different locales and endure over long periods (decades rather than years).

It is also worth noting that Georgiadou et al. (2005) have put forward a potential research agenda for monitoring SDI implementation based on four elements. The first of these elements, the dynamics of implementation, draws largely on the need to understand the nature of socio-technical networks as outlined above. The second calls for a participatory approach to the process of standardisation while the third element regarding the scope of the design process stresses the importance of understanding the existing installed base and the networks that underlie it. The final element calls for rigorous empirical research by interdisciplinary teams and longitudinal studies that follow the unfolding of the implementation process.

Most of the features that can be found in the criteria outlined by Aanested et al. (2007) and Georgiadou et al. (2005) can be seen in the development and implementation of INSPIRE. From the outset, it has been treated as a participatory process by those involved in order to facilitate its implementation (Craglia et al., 2005; Masser and Crompvoets, 2015). This participatory approach has been a major factor in the implementation process. For these reasons, Craglia (2014: 32) has described it as

an infrastructure built on those of 28 different countries in 24 languages by a truly democratic process, INSPIRE is a role model not only in relation to the developments of SDI but more generally to the formulation of public policy at the European level.

Approaches to qualitative monitoring

Most of the texts that are available on the subject of public sector implementation (see, for example, Van Dooren et al., 2015) deal with the prior design of such activities rather than the ongoing monitoring of such programmes. In this situation, only two approaches are available for the regular qualitative monitoring of II implementation. The first of these approaches is when the tasks are carried out internally within the countries by those most directly concerned with the implementation process. This is the approach adopted by the EC with respect to INSPIRE. The second approach involves some form of evaluation by external parties. The main strengths and weaknesses of these two approaches are set out in Table 1.

OPEN IN VIEWER

Table 1.

Characteristics of the two approaches to qualitative monitoring.

Internal approach Strengths – Carried out by people centrally involved in the project. – Takes account of diversity of circumstances where several cases involved – Helps to build up local stakeholder communities Weaknesses – Produces uneven results where several cases involved – Need some form of template to highlight issues to be addressed – Can be used to convey messages to commissioning bodies External approach Strengths – Reports prepared by an independent external body/bodies – Reports to single commissioning body – Produces consistent and comparable reports Weaknesses – Dependent on local contacts and available secondary sources – Does not handle diversity well where several cases involved – Costs must be borne by commissioning body

The great strength of the internal approach is that it places the onus for reporting on those organisations that are directly engaged in the implementation process. This approach involves some measure of self evaluation which can be defined as ‘an evaluation by those who are entrusted with the design and delivery of a development intervention [or programme]’ (Organisation for Economic Cooperation and Development, Development Assistance Committee, 2002: 35). The advantage of this form of monitoring in relation to large-scale legal programmes such as INSPIRE is that it is carried out by people with in depth knowledge who are sensitive to the needs of the project and the various organisations involved in its implementation. In situations such as the implementation of the INSPIRE Directive where more than one country is involved, the internal approach can also handle the diversity of circumstances that may exist between countries. It can also be used to raise overall awareness of the wider issues involved in local stakeholder communities. Consequently, its outcomes may help to identify weaknesses that need to be remedied during the implementation process. Most of the international aid agencies recognise the importance of stakeholder involvement in monitoring and evaluation (see, for example, United Nations Development Programme, 2009: 93).

However, there are a number of weaknesses associated with this approach that must be taken into account. Where several organisations are involved, it may result in uneven results unless some form of template is used to ensure some degree of uniformity in the responses. However, it must also be recognised that this may restrict the extent of their coverage. It should also be noted that the whole process can be used to convey messages about local circumstances to those commissioning the responses (Van Dooren et al., 2015: 72–73). For this reason, it is possible that some reports may be used to present an over optimistic (or a pessimistic) view of national circumstances to provoke attention.

Most of the weaknesses are automatically overcome where reporting is carried out by an external independent agency as was the case with the series of NSDI State of Play reports commissioned by the EC prior to the formal Committee Decision on the monitoring and reporting implementing rules in 2009 (Vandenbroucke, 2011). The outcomes of this approach are likely to be more consistent and compatible where several organisations are involved and there is no likelihood that the process might also be used to convey messages. However, it must be recognised that the reports generated in this way represent an outsiders' point of view of what is happening in each country and not all the relevant issues may have been taken into consideration. Those responsible for the preparation of the reports may not have been informed about the latest developments or are forced to rely on secondary sources that contain obsolete materials. This is particularly a problem in countries where some or even all strategic documents and key websites are only available in a language that is not understood by the external evaluators. The external approach also can be expensive and rather cumbersome to administer where several countries are involved and that the costs of reporting must be borne by the higher level commissioning bodies.

Both these approaches draw to some extent on both internal and external sources. Internal bodies need advice from external agencies and external agencies must draw on local sources in preparing their reports. Nevertheless, there is a clear distinction between them in terms of the primary responsibilities between the body that is directly concerned with local programme implementation in the internal approach and the independent agency that carries out the evaluation in the external approach.

Monitoring and reporting in the INSPIRE directive

The need to monitor the progress of INSPIRE implementation is discussed in the last section of the INSPIRE Directive: ‘Member States shall monitor the implementation and use of their infrastructures for spatial information. They shall make the results of this monitoring accessible to the Commission and to the public on a permanent basis’ (Article 21(1)).

From the outset it was clear that two sets of implementing rules would be required for this purpose. The first would be based on a quantitative approach based on indicators derived from the list of spatial data sets and network services. This approach is set out in as the definitive Commission Decision (Commission of the European Communities, 2009) regarding INSPIRE monitoring and reporting of 5 June 2009 (Articles 3 to 10). These indicators must include already conformant data sets and services as well as those that still had to be brought into conformity and reflect the Member State's plans for the implementation of INSPIRE. They cover the following topics:

Existence of metadata for spatial data sets and services

Alongside the substantial body of statistical material that is created by these quantitative indicators the INSPIRE Directive also recognised that it would be necessary for the Member States to provide qualitative information on their progress in the form of written reports every three years covering developments since the previous report. These country reports are the primary focus of this paper. The provisions for reporting are set out in Article 21(2) of the Directive itself.

No later than 15 May 2010 Member States shall send to the Commission a report including summary descriptions of five topics:

How public sector providers and users of spatial data sets and services and intermediary bodies are coordinated, and the relationship with the third parties and of the organisation of quality assurance

From this it can be seen that these topics cover a much wider range of issues than those covered by the quantitative monitoring indicators listed above. Two rounds of qualitative country reports have been completed so far: in 2010 and 2013. Some of the main features of these reports will be described in the next two sections of this paper.

It should also be noted that in April 2013, the Commission, with the agreement of the Member States, set up an INSPIRE Maintenance and Implementation Group with representatives from the Member States to facilitate the exchange of experience and good practice related to the implementation of the INSPIRE Directive and to identify and give advice about the priority issues to be addressed in the maintenance of the INSPIRE Directive as part of the rolling work programme for the Commission's maintenance and implementation for INSPIRE (INSPIRE, 2013a). The work of this group and its technical and policy sub groups focuses on this common work programme which is based on issues and change requests submitted by INSPIRE stakeholders. The establishment of this group coincided with the submission of the 2013 country reports and its impact should be reflected in the 2016 round of reports submitted by the Member States.

The 2010 country reports

Prior to the submission of the 2010 country reports, the Member States were given the detailed template setting out the main headings in the reports that were to be prepared by them (INSPIRE Monitoring and Reporting Drafting Team). This amplifies the five topics identified in the Commission Decision of June 2009: i.e. the coordinating structures that have emerged within each country, their functioning in practice, the use of the emerging infrastructure, the extent of data sharing and the estimated costs and benefits of its implementation (see Table 2).

OPEN IN VIEWER

Table 2.

Template for the 2010 country reports.

Table of Contents 1 Inspire Reporting – Overview of Requirements 2 How to Use This Template 3 Executive Summary 4 Abbreviations and Acronyms 5 Introduction 6 Coordination and Quality Assurance (ART. 12) 6.1 Coordination (ART. 12.1.)  6.1.1 Member State contact point  6.1.2 The coordination structure  6.1.3 Comments on the monitoring and reporting process 6.2 Quality Assurance (ART. 12.2.)  6.2.1 Quality assurance procedures  6.2.2 Analysis of quality assurance problems  6.2.3 Measures taken to improve the quality assurance  6.2.4 Quality certification mechanisms 7 Functioning and Coordination of the Infrastructure (ART.13) 7.1 General Overview Description of the SDI 7.2 Inspire Stakeholders 7.3 Role of the Various Stakeholders 7.4 Measures Taken to Facilitate Sharing 7.5 Stakeholder Cooperation 7.6 Access to Services Through the INSPIRE Geoportal 8 Usage of the Infrastructure for Spatial Information (ART.14) 8.1 Use of Spatial Data Services in the SDI 8.2 Use of the Spatial Datasets 8.3 Use of the SDI by the General Public 8.4 Cross-border Usage 8.5 Use of Transformation Services 9 Data Sharing Arrangements (ART.15) 9.1 Data Sharing Arrangements Between Public Authorities 9.2 Data Sharing Arrangements Between Public Authorities and Community Institutions and Bodies 9.3 Barriers to the Sharing and the Actions Taken to Overcome Them 10 Cost/Benefit Aspects (ART.16) 10.1 Costs Resulting From Implementing Inspire Directive 10.2 Benefits Observed 11 Conclusions 12 Annexes 12.1 List of Organisations – Names and Contact Details 12.2 List of References for the Compilation of the Report

The deadline for the submission of the first round of reports to the Commission was 15 May 2010. With the exception of Cyprus (and Croatia which was not a full member of the EU at that time), the other 26 countries more or less met this deadline and 20 of these reports submitted in 2010 were in the original language of the Member State. A year elapsed before translations of these reports into English were available on the INSPIRE website (http://inspire.ec.europa.eu).

This website shows that these 26 Member States submitted nearly 700 pages of text to the Commission in the first round of reporting. The length of the reports varies considerably in length from 88 pages of text and annexes in the Portuguese report to 12 pages in the case of Ireland and 13 for Poland and the United Kingdom. Most of the reports are between 20 and 40 pages in length and the average for the whole group is just over 26 pages. Two countries, France and the Netherlands, included material in their reports describing the methods used to calculate the statistical indicators that are an input to the monitoring task and the Portuguese report also included a substantial appendix consisting of the replies that had been received from each of the various stakeholders (Masser, 2011).

With respect to the five main topics itemised in the INSPIRE Directive, most reports contain a section of between 4 and 10 pages dealing with matters of coordination and quality assurance and a similar length section describing the functioning of their SDI and identifying the main stakeholders. The lead agency for coordination was the national mapping and/or cadastral agency in 14 cases. Most of the lead agencies in the other 12 cases were environmental agencies or Ministries with environmental responsibilities. Notable exceptions are the Executive Agency for Electronic Communication Networks and Information Systems in the Ministry of Transport, Information Technology and Communications which is the lead agency in Bulgaria and the Malta Information Technology Agency which is the lead agency in Malta.

All 26 countries were in the process of creating a structure to coordinate the implementation of the INSPIRE Directive. In most countries this took the form of either a National Council for Geographic Information (e.g. Finland) with a broad remit concerned with managing spatial information at a national level or an INSPIRE Coordination Committee (e.g. Czech Republic) with a more limited scope focussing essentially on INSPIRE implementation. In some cases this was based on a modified version of an existing committee such as the Conseil national de l'information géographique in France or the Advisory Council for the National Geographic Information System in Portugal. A special agency called Geonovum had been set up in the Netherlands to deal with the technical aspects of INSPIRE implementation.

In most cases either the national environmental or the national mapping and cadastral agency was the lead agency in coordinating implementation efforts but there were considerable differences between countries with respect to stakeholder involvement. Fourteen coordination bodies were restricted to central government agencies while six others contained members from all three levels of government (Austria, Belgium, Germany, Italy, Portugal and Spain). Some countries (Czech Republic, Denmark, Finland, Ireland, the Netherlands, Sweden and the United Kingdom) also involved representatives from the private sector and/or national geographic information associations. These differences reflect legal, political and administrative systems in operation in these countries. This was particularly the case in countries such as Germany and Spain where some or all the responsibilities for spatial data management is devolved to the regional level.

The response to the questions on usage and data sharing was much more limited in most cases and a number of countries claimed that they did not have the information available as yet or that it was too early in the INSPIRE implementation process for them to be in a position to provide useful information on these topics. However, there were a few notable exceptions to this rule. Every country apart from France provided some information on the costs and benefits aspects of INSPIRE implementation even though very few of them had much direct experience of this in practice.

Nevertheless, it appears from the material that is contained in these reports that most countries provided enough information for the Commission to evaluate their progress at this early stage of INSPIRE implementation, although two countries, Bulgaria and France, produced reports that contained a very limited amount of information on the five main topics identified in the template.

The 2013 country reports

Member States were required to submit their second round of qualitative country reports by 15 May 2013. As in the previous round, a template was prepared by the Commission to help the Member States in preparing their reports (INSPIRE Monitoring and Reporting Drafting Team, 2013). This was essentially the same as that used for the 2010 reports, although the section on costs and benefits was substantially expanded for this round.

Apart from Croatia which did not join the European Union until 1 July 2013, all the other 27 European Union Member States submitted reports within a reasonable period after the deadline (Masser and Crompvoets, 2015: chapter 4). A total of 23 of the reports submitted in 2013 were in the language of the Member State as against 20 in 2010. These were subsequently translated into English and made available on the INSPIRE website (http://inspire.ec.europa.eu).

The second round of country reports was substantially longer in most countries than those submitted in the first round and the total number of pages they contained added up to more than one thousand pages. Most of the reports were at least 30 pages in length and the average for the whole group was more than 39 pages as against 26 pages in the 2010 round. The shortest reports were those from Luxembourg and Ireland with 15 and 20 pages, respectively, and the longest reports came from Portugal and Spain with 96 and 93 pages, respectively. As in the previous round of reporting, the Portuguese report included a 50-page annex containing the replies received from all the main stakeholders.

In most cases at least three fifths of the text of these reports dealt with the experiences of the Member States with respect to the five topics listed in the INSPIRE Directive. As might be expected with an initiative which is still in the relatively early stages of implementation, the responses to the questions about coordination and functioning were twice as detailed as those relating to the other three topics regarding data usage, data sharing and the costs and benefits associated with INSPIRE implementation.

Most countries produced at least five pages of text on the arrangements that they have made for coordination and quality assurance. One change from the situation in 2010 is that in 2013 the Ministry with responsibility for environmental matters rather than the national mapping or cadastral agencies was typically taking the leading role in the coordination process. Although most of the arrangements reported in the 2010 reports are still in operation, the coordination structures are still evolving in some countries. In the United Kingdom, for example, a UK INSPIRE Compliance Board had replaced the UK Location Council as the body overseeing the implementation of the INSPIRE Directive from April 2013. Similarly, rapid technological growth in Latvia, together with reduced financing, had made it necessary to update the original coordination concept in 2012 and a new and updated structure was expected to be in place by mid 2013. In Portugal, the overall responsibility for INSPIRE implementation also changed when the national mapping agency became part of the Directorate of Spatial Planning in 2012.

As in 2010, the country reports show considerable differences between countries with respect to the involvement of stakeholders in the implementation of the INSPIRE Directive. In Finland, for example, ‘All parties have had the opportunity to participate in the implementation of the infrastructure for spatial data and its monitoring in the open INSPIRE network, in which 350 experts from 120 organisations’ (Finnish report: 3). However, some other countries had a more restricted view of their operations. In Bulgaria, for example, it was felt that, ‘At this stage, the national legislation does not require the establishment of other district and municipal level coordinating structures and the relations with third parties are not detailed’ (Bulgarian report: 8).

In countries with federal governments like Austria, Germany and Spain, there was also a strong regional dimension to INSPIRE implementation. The SDI Germany is a national network of coordination structures with contact points at the Federal government, Länder (states) and municipalities at the operational level. Similarly, ‘the central state administrative bodies involved in producing geographic data and the autonomous communities have both simultaneously developed their own organisational structures’ in Spain’ (Spanish report: 12) and the French report also made it clear that they had identified the regional level as the key level for the successful implementation of INSPIRE.

A number of countries also pointed out that INSPIRE implementation cannot be treated in isolation and must be seen as part of a more broadly based infrastructure. The Danish report, for example, makes it clear that ‘public authorities are cooperating on the development of a common public sector infrastructure for spatial information. The aim is for the infrastructure to support eGovernment with spatial information in a simple and effective way’ (Danish report: 5).

INSPIRE is ‘in line with the policy adopted in The Netherlands with regard to key registers’ containing the main national strategic sources of information. The United Kingdom report makes the point that

the UK wide SDI is openly available and does not only hold INSPIRE data. It is part of a wider agenda which includes making public data more open and more easily available, making government more transparent, and allowing citizens digitally to access all government services by default. (United Kingdom report: 19)

In the absence of quantitative indicators concerning data sharing, Member States were asked to give an overview of the arrangements that they had made for sharing data between public authorities. Some countries including Finland, the Netherlands and the United Kingdom, were developing some form of open data structures for government information to promote data sharing but most Member States reported limited progress on this front. This was largely due to the continued existence of a number of barriers to data sharing. These included restrictions on access, problems of finding common ground and interest for cooperation, and lack of applicable national standards. In Slovenia, for example, ‘despite well defined tasks in the national legislation, it still occurs that the data of particular ministries are not interconnected and are managed in different ways with different software tools. The result is poor interconnectivity of spatial data’ (Slovenian report: 15).

The final section on the costs and benefits in the 2013 reports is a significant improvement on the material contained in costs and benefits sections of the 2010 reports and the reports submitted by Belgium, Greece and Sweden contain a great deal of detailed statistical information on this topic. However, estimating the benefits in quantitative terms at this stage in the implementation of the INSPIRE Directive still presented problems for many countries. The German report notes, for example, that ‘it has to be taken into consideration that interoperable INSPIRE data will be made available, according to the INSPIRE timetable, only in the second half of the present decade by 2020’ (German report: 25).

The overall impression given by the country reports submitted to the EC in 2013 is that the implementation of the INSPIRE Directive is well under way and that the necessary steps are being taken by the Member States. They also highlight the great diversity of practices and institutional arrangements that has developed between the countries that are implementing the INSPIRE Directive. As always, there is still room for improvement. The German report states, for example, that ‘with the specifications assuming more concrete form and the associated financial and human resources efforts, it is precisely at municipal level that the added value and usability of the INSPIRE network services are questioned.’ (p. 22). The Spanish report also notes that the ‘SDI is developing quite unevenly between different … bodies … in Spain, so there is a need to continue focusing on promotion and stimulus’ (pp. 59–60) and the Belgian report also notes that

the four parties [i.e. Brussels, Flanders, Wallonia and the federal level] have not all made equal progress in the development of a geographical data infrastructure. This has to do mainly with (political) choices made in the past and the budget made available. (p.63)

External evaluation of the INSPIRE implementation process

An external valuation of the overall outcomes of the INSPIRE implementation process was carried out as part of the Mid-Term Review by the INSPIRE team for the EC to meet the requirement that the Commission itself presents a report on the implementation of this Directive which evaluates the extent to which the Directive has achieved its original objectives by 15 May 2014 (European Environment Agency, 2014). Overall, the outcomes of the assessment suggested that INSPIRE implementation has reached its half way stage with generally positive outcomes (Jensen, 2015).

The authors of the Mid-Term Review report drew heavily on the material contained in country reports in their analysis. In overall terms, they felt that the quality of the reports had improved between the first and second rounds. However, they also came to the conclusion that

these reports focus on the implementation of the technical infrastructure and do not yet consider the contribution of INSPIRE to improve the implementation of environmental policies or policies that affect the environment, which is the main role the infrastructure should have. (p. 16)

Conclusions

This final section considers the main findings of the case study and also identifies some potential research areas that are relevant for future work on implementing IIs such as INSPIRE. The brief descriptions of the outcomes of the two rounds of country reports in the previous sections give a useful overview of their experiences. The findings of the mid-term review also provide another perspective to this analysis. At the same time, they highlight the considerable diversity of the arrangements that have been made by the Member States to implement the provisions of the INSPIRE Directive within their existing administrative and organisational structures.

The findings of the case study shows how the five key issues identified in the Directive itself and elaborated upon in the template shown in Table 2 can be evaluated by qualitative monitoring as an example of an II. These are (1) the progress on coordination, (2) functioning, (3) data usage, (4) data sharing agreements, and (5) costs/benefits of INSPIRE. As noted above, considerably more information is available about the first two key issues listed above and the materials regarding the other three issues are also much more variable. This largely reflects the stage of evolution of the implementation process in the Member States and the balance between the first two and the other three sets of issues can be expected to change in the next two rounds of reports.

The reports show that the arrangements for the coordination of the implementation process are largely in place in most Member States. However, there is also a lot of evidence to suggest that these are not static in nature and that these are still evolving in some cases. These may reflect changes elsewhere in administrative structures such as the decision to move to an open data environment for government departments in the United Kingdom.

The country reports show that there is considerable diversity in the contributions that public authorities and third parties are making to the functioning of the implementation process. Some Member States report an extensive involvement of agencies from all levels of the public sector as well as third parties including universities and private sector companies in the implementation process as against others who see the tasks as essentially a task for central government bodies. In countries where responsibilities delegated to the regional and local levels of the public sector it has been necessary to create additional structures to oversee the process of implementation.

The sections on data usage in the country reports show that most Member States reported that they had created some form of national geoportal to facilitate the use of their INSPIRE data but this was very still much a work in progress. Nevertheless, the use of data and services has been increasing very rapidly in most Member States. Although it is clear that there is a long way together before the range of network services can be regarded as anything approaching fully operational and INSPIRE compliant, this seems to be one of the most important success stories of INSPIRE implementation.

In contrast, the progress that has been made towards promoting data sharing in most countries has been rather limited despite the efforts that some countries have made towards overcoming the institutional legal barriers that are involved and moving towards open data government.

With respect to the final issue listed in the Directive regarding the costs and benefits of implementation, the 2013 reports contains considerably more information about the costs that are involved but many countries felt that it is still too early in the process to be able to make quantitative estimates of the benefits accruing from implementation in other than the most general terms. This situation should improve in the next rounds of monitoring.

The findings of studies such as this inevitably raise almost as many questions about the nature of qualitative monitoring of IIs such as INSPIRE as they answer. It is not always clear in this case, for example, who was centrally involved in the preparation of each country's report despite the information provided in response to the coordination section of question on the template presented in Table 2. It is also not always clear how the other stakeholders were consulted in each country. The obvious implication of this finding is that more detailed longitudinal case studies of individual countries are needed to explore these matters in much greater depth than is possible from an analysis of the individual country reports. This represents a further stage of research on this topic.

The discussion in section ‘The nature of IIs’ of this paper highlighted the ways in which the implementation of the INSPIRE Directive reflects the four key elements of an II that underpin their implementation as outlined by Aanested et al. With this in mind, the detailed findings of the preceding analysis must be reconsidered in order to identify some more possible topics for future research on these topics.

The importance of adopting a socio-technical approach is borne out by the participatory approach that has been adopted by the EC to the whole process of developing the INSPIRE Directive and its implementation. This has been an important factor in the building up of a European-wide SDI community which has been instrumental in translating SDI concepts into practice. This opens up a number of opportunities for cross-national comparative research studies of INSPIRE implementation. It is also worth noting that the establishment of the INSPIRE Maintenance and Implementation Expert group to promote the cross national exchange of experience and good practices in the Member States coincided with the submission of the 2013 country reports and the full impact of its work will not be felt until the next round of country reports that are due in 2016.

The concept of the installed base is central to the analysis of any II. It builds upon the strengths and weaknesses of what already exists and their trajectory is ‘nurtured by a variety of human actors with their interests, agenda, preferences and limited as well as fluctuating control’ (Aanested et al., 2007: 16). This is evident in some of the research on the Flanders SDI that was carried out in the Spatialist project (see, for example, Dessers, 2013). This presents a particularly interesting area for further research in federal systems including the United Kingdom, as the result of the devolution of the similar powers to regions with different histories (see, for example, Masser, 2005: chapter 4).

The politics of representation highlight the extent to which IIs like INSPIRE cannot be treated in isolation from the political and institutional context within which they are implemented and the extent to which this permeates their usage. One interesting potential area of research in this respect is the impact of language of the infrastructure implementation process. It is interesting to note that 23 out of the 27 country reports in 2013 were submitted in one of the 24 languages that are used by the EU Member States. It should also be noted that most of the discussions that take place at the European Union level and the annual INSPIRE conferences take place in English and that people without a good working knowledge of English are effectively excluded from these discussions. The country reports that were submitted in the language used in the country are also translated into English and consequently not accessible to participants whose language skills are restricted to that used in their own country. Consequently, there is an important distinction between participants with or without English language skills in each member state, and it would be interesting to explore the way in which this has influenced the processes of INSPIRE implementation.

There is also a great deal of scope for research on the multiplicity of networks that are involved in INSPIRE implementation. Aanested et al. refer to the ‘spaghetti’ of overlapping systems that operates in practice. The nature of the networks that are likely to be involved in each Member State can be seen from the Finnish report discussed above which refers to the involvement of 350 experts from 120 organisations in the national implementation process. As each of these experts brings with them their own networks, the scale and complexity of the extent to which they influence the processes of implementation opens up some interesting potential possibilities for research.

The four research areas described above indicate some of the potential opportunities for future socio-technical research on the implementation of IIs such as INSPIRE. It is to be hoped that the findings of this analysis will stimulate more rigorous empirical research of the kind suggested by Georgiadou et al. (2005), thereby contributing to an improved understanding of the issues to be need to be taken into account with respect to the implementation of large-scale IIs.