Electronic records preservation in China – An exploratory inquiry

Introduction

Electronic records in China are defined as records that are created by digital devices and in a digital environment, exist in digital formats, rely on digital devices such as computers for reading and processing, and can be transmitted via communication networks. As a subset of electronic records, archival electronic records are those, based on their reference and use values, needing to be preserved as archives (China, GB/T18894–2002). The preservation of electronic records therefore refers to the keeping and management of archival electronic records, which is, in China, the responsibility of the various kinds of archival institutions collectively established by the Archives Act of the People’s Republic of China. The major types of archival institutions are those established at the levels of province and administrative region, that is, at the lower level than the province, yet function similarly; namely, provincial archives and secondary provincial archives.

This paper reports on the findings of a study that focused on the current status of electronic records preservation in both the provincial and secondary provincial archives throughout China. It was the first of its kind conducted in the geographical area of Mainland China in terms of scope and subject matter. It is believed that the findings generated by the investigation will be instructive and beneficial to the archival functions in other countries or regions.

Study origin

The study on the current status of electronic records preservation in China constituted one part of the research project entitled ‘Annual Reporting on the Forward-Looking Development of Electronic Records Management and Preservation’, supported jointly by China’s Fundamental Research Funds for Central Universities and the Funds for Research of the Renmin University of China, the university to which the author is affiliated. The project was designed to gain systematic understanding of the field of electronic records management and preservation in a continuous manner, through regularly tracing and analyzing the theoretical and practical advancements in the relevant fields both domestically and internationally. By doing so, it aims to distil principles and best practices applicable to electronic records management and preservation initiatives around the world, and at the same time provide directions and guidance for developing requirements for local situations. The project consists of a series of studies, including both literature reviews and empirical investigations, and the latter were typically identified based on the results of the former. The study reported in this paper falls into the realm of empirical investigation, which surfaced in the process of scanning literature relevant to long-term preservation of records in archival institutions. Because archival institutions in China have primarily a mandate of keeping government records, the literature search was limited to archives in the public sector, both in China and internationally. The review found that the Chinese literature did not address the topic of records long-term preservation as a specialized, independent topic, but instead as part of the development of digital archives (or electronic records centers, as preferred by some archives), which involves a broader range of activities. A number of archival institutions started experimenting with the development of digital archives and reported their design and initial achievements to the Chinese archival community (Li, 2001, 2003; Pan, 2006; Huang, 2006). Accompanying this experimental movement, archival professional metadata standards regarding records long-term preservation were issued in 2009 (State Archives Administration of China, 2009; Huang, 2009). However, the review also found that, when the entire country was considered, the actions towards records long-term preservation appear to be discrete and unsystematic, lacking comprehensiveness in terms of the number of archival institutions that reported their involvement in digital preservation, and standardization in terms of the approaches taken in developing digital archives. Moreover, this raised the question as to the usability of the metadata standards, which apply to all archival institutions in the country, yet their situations with respect to electronic records preservation were unclear. It was then decided that a study focusing on gaining a nation-wide understanding of the preservation activities in archival institutions, including, for example, electronic records accessioning, the types of records accessioned, file format handling, planning for sustainability, etc., was necessary.

To further frame the study, the literature search was extended to sources in English. It found that many countries or regions and international organizations have taken the step of gaining understanding of the landscape in their journeys of electronic records preservation. During the period 1994–1995, the Committee on Electronic Records of the International Council on Archives (ICA) conducted a survey on electronic records programs in national archives of its member states (i.e. Category A members), among which is China (International Council on Archives, 1996). The State Archives Administration of China (SAAC), the public body responsible for the countrywide administration of recordkeeping and archival preservation in China, responded to the survey; however, the information is brief and apparently outdated. In 2002, the ICA conducted another survey of its Category A member states, focusing, however, on the overall operation of national or federal archival institutions (Ogawa, 2002). Electronic records were included as one of the eight areas being investigated, the data collected being informative and relevant to the objectives identified by the present study. The information on China, however, was even briefer than that in the previous study, preventing a sufficient understanding of the current situation.

At the national level, the Library and Archives of Canada (LAC) conducted in 2005 a survey on the preservation of digital information in Canada, relying on both primary and secondary data (i.e. existing surveys, Internet searches, and contacts with experts) (McDonald and Shearer, 2005). This survey was broad in scope as it had primarily a goal of supporting the development of a national strategy that includes a variety of digital content producers and service providers, which is therefore not limited to public archives.

In 2008, a survey on ‘Digital preservation in local authority archive services’ was jointly conducted by the Association of Chief Archivists in Local Government in England and Wales, the Archivists of Scottish Local Authorities Working Group, and the Local Government Group of the Records Management Society of Great Britain (Boyle F, Eveleigh A and Needham H, 2008). This survey collected data from 38 institutions, covering the areas of digital preservation planning, general awareness of digital preservation, practical digital preservation, electronic information management, and infrastructure/issues for the future.

In 2009, the Technology Best Practices Task Force of the Society of American Archivists published a report on its initial work of identifying competencies and standards “relating to all areas of archival practice that are affected by electronic records and digital asset issues”, which, like the Canadian study, relied on secondary sources (Nelson, 2009). Although this survey of literature does not address the operation of archival institutions or, in particular, the preservation of electronic records, in a manner that collects and presents empirical data, the identified best practices (in the form of a recommended bibliography) are relevant to the actual carrying out of archival activities including long-term preservation. As such, the study possesses the potential of guiding specific, in-depth analysis. In 2011, the Council of State Archivists of the United States (CoSA) launched its State Electronic Records Initiative (SERI) as the first comprehensive national effort to improve the management and preservation of electronic government records in all states and territories. During Phase One of the initiative, a survey was conducted regarding the practices of 50 states and four territories, which was then followed up by telephone interviews with representatives from 48 states and three territories. The survey and interviews collected data about all aspects of program operation, including program development and implementation, funding, staffing, policies and procedures, technical infrastructure, records holdings, programmatic needs, and areas of possible collaboration (CoSA, 2012).

These studies collectively confirmed the need to conduct a similar study in China, focusing on gaining a nation-wide understanding of the landscape of electronic records preservation and aiming at laying foundations on which action plans can be based. Furthermore, the review of the relevant literature inspired a future study that will focus on comparing these studies.

Research methodology

To accommodate the objective of the investigation, i.e. to gain a countrywide understanding, the investigation decided on the research methodology of survey. The sample selection attempted to be as broad as possible, and as a result, all archival institutions at both the primary and secondary provincial levels in Mainland China were targeted, i.e. 31 provincial archives and 16 secondary provincial archives, a total of 47 institutions.

A questionnaire was developed as the data collection tool, featuring both closed and open-ended questions to enable both quantitative and qualitative analyses. The guiding framework under which the questions were formulated came from the Open Archival Information System (OAIS) Reference Model, as it provides, for the general purpose of preserving digital information, a coherent set of terminology and a comprehensive provision of the functions of operating digital archives. Moreover, an Open Archival Information System (OAIS), defined by the reference model as an archive that consists of an organization of people and systems and has accepted the responsibility to preserve information and make it available for a user community (ISO 14721:2003), corresponds largely to the mandate of archival institutions in China. The questionnaire was thus organized in accordance with the OAIS reference model, in particular, its section 4.1 Functional Model. According to this model, the functions of an OAIS should constitute Ingest, Archival Storage, Data Management, Administration, Preservation Planning, and Access. To reduce the burden for respondents of interpreting the OAIS terminology, the six functions were adapted to the Chinese archival environment as follows:

This investigation was supported strongly by the State Archives Administration of China. In June 2010, the questionnaire was sent out to the entire sample group, i.e. the 47 archival institutions, which yielded an almost full response rate: all institutions except one provincial archives responded to the questionnaire.

Findings

The findings were generated by both quantitative and qualitative analyses and are reported in the order of the OAIS functions listed in the methodology section.

Ingest

The function of Ingest was revealed by the aspects of the time when acceptance of archival electronic records into the archival institutions began, the manner by which records were accepted, and the quantity and type of records accepted.

According to the survey results, more than half of the responding institutions, i.e. 26 out of 46 (56.5 percent), had started ingesting electronic records. Among the 26 are 17 provincial archives and 9 secondary-provincial archives, making the ratio of ingesting archives nearly equal at both levels, i.e. 17 out of 30 (57 percent) for the provincial archives and 9 out of 16 (56 percent) for the secondary provincial archives. This indicates that the level of archival institution does not have a visible impact on their decisions regarding accepting digital records.

The beginning times of ingestion, however, vary. Among the 26 archives are three pioneers which started in 1984, 1995 and 1999 respectively; the rest all started after 2002. Again, the level of archives did not display any impact on the beginning time of ingestion, as the one that started in 1995 is indeed a secondary provincial archives. For all the archives, a pattern emerged demonstrating that the time period 2002–2006 was the peak time for archives to start ingestion, which corresponds to the wide deployment of information technologies by all walks of life in China. Figure 1 exhibits the analyses in graphics.

OPEN IN VIEWER

Figure 1.

Number of archives having completed first digital ingestion.

Two types of records transfer commonly exist in China: regular and irregular, which are applicable to electronic records. Irregular transfers are typically triggered by unexpected or one-time occurrences including the ceasing of an organization’s existence, the completion of special task forces and the hosting of significant events such as the Olympic Games. Irregular transfers thus take place in a random, unplanned manner. Regular transfers, on the contrary, require the establishment of formal transfer-acceptance agreements between the records creating agencies and the archival institutions, which are then executed in a routine, continuous manner. The research found out that a high percentage, i.e. 80.8 percent, of the archives that had received electronic records fall into the category of regular transfer, indicating that regularly accepting electronic records as part of archival holdings had become normal. The technical manners by which the transfer had taken place included offline (i.e. relying on physical transfers of media) and online (i.e. relying on organizational networks). Among the accepting archives, 65.4 percent used mainly the offline approach and 34.6 percent used both. The specifics regarding online transfer require future investigation.

The major type of accepted records is administrative records existing in text or image formats, as all accepting archives received this type and only 33.3 percent reported that their archives also received operational records. All except three archives accepted both digitized (mainly scanned) and born digital records. The quantities of accepted records displayed less consistency and varied from one archives to another. The size of digital records collections ranged from a few thousand to tens of thousands in terms of the number of records, or tens to hundreds of gigabytes in terms of storage capacity. The largest one contained 7.11 million records or 2000 gigabytes, and the smallest one included only 151 records or one gigabyte. It is worth noting that the archives that possessed the largest collection of digital records was not one of the three pioneers, i.e. it does not have the longest history of receiving digital records; similarly, the archives that possessed the least amount of digital records was not one of those who started most recently, i.e. in 2010. This result demonstrates the need to study further the operations of both the records creating agencies and the archival institutions.

Archival Storage

The quality of electronic records determined for long-term presentation depends highly on the functionality of the archival storage system where the transferred records are managed and accessed. Before the advent of regular transfers of electronic records, archives in China typically had computer-assisted systems that managed paper records. This type of system was not designed for long-term preservation of records in digital formats; it maintains only cataloguing information, not actual records. In other words, the management of born digital records, including the assurance of their authenticity and accessibility, goes beyond the capacity of this type of ‘archival system’, which required the development and implementation of new preservation systems.

Over half of the accepting archives, i.e. 51.9 percent or 14, reported that their institutions had been developing records preservation systems targeting born digital records. Nine vendors participated in these developments and the systems they configured were distinct from each other. These systems included the Archival Resource Management Centre for Electronic Records, Electronic Records Management Centralized System, E-Government Archival Information Management System, Records and Archives Integrated System, Electronic Records Filing and Transfer Platform, Online Archives, and Digital Archives, etc. The various distinct names reveal to some extent the different functional orientations that the archives had decided upon, which in turn, refle-cted their different focuses in fulfilling their mandates. Currently, development centers on the functions of ingestion, storage and backup, retrieval and access, with IT system common functionality of security and audit. This indicates that at this stage, archives tend to focus on functions that are of an immediate usage nature (i.e. ingestion and access), and leave long-term preservation related functions to the next stage of development.

A wide range of media types and file formats were accepted by the surveyed archives. Seven media types included text, image, graphics, database, audio, video and 22 file formats included DOC, TXT, WPS, CZ, RTF, PBD, PDF, CEB, XML, HTML, XLS, DBF, MBD, JPEG, TIFF, BMP, DWG, MPEG, AVI, WAV, MP3, and DAT. In a particular archives, the media type could range from one to six and the file format from two to 16. Text and image were the most common ones, with 88.5 percent and 76.9 percent respectively among the accepting archives. The three that appeared to be the least common were video (30.8 percent), audio (26.9 percent) and graphics (19.2 percent). For file format, DOC, PDF and XML emerged as the most common ones with respectively 76.9 percent, 53.8 percent and 42.3 percent among accepting archives. Only one archives reported to have accepted the two unique file formats developed locally in China, PBD and SZ.

The preservation strategies utilized by the archives included four types, all of which are recommended by the State Archives of Administration of China (China. GB/T18894–2002). They are: 1) normalization of original file formats into standard ones; 2) regular checking of the media for readability; 3) periodical refreshing of media; and 4) keeping the older versions of software and hardware. Not all of the strategies were employed by all of the accepting archives: 65.4 percent used the first type, 53.8 percent used the second type, 50 percent used the third type and 26.9 percent used the fourth type. Only 34.5 percent used both the second and the third types despite the fact that all of them acknowledged that there was a need to utilize both. Only one archives – a secondary provincial archives – employed all four types of strategy. Other strategies such as emulation have not yet found their way into any of the accepting archives.

Despite the development of systems and established preservation strategies, the analysis of the qualitative data revealed the existence of preservation related issues. Almost half of the accepting archives (46.2 percent) had encountered difficulties in rendering electronic records, indicating a serious problem caused by ineffective long-term preservation. The situation has worsened when compared to a similar study conducted in 2007, which reported that only 10.3 percent among the 29 surveyed archival institutions had encountered the unreadable problem (Zhang, 2008). Figure 2 below presents the discovered preservation issues.

OPEN IN VIEWER

Figure 2.

Electronic records preservation issues.

Preservation planning

The survey showed that there was wide acceptance among the responding archives of the notion that a set of mechanisms needed to be established and enhanced in order to respond to the challenges of electronic records preservation. These mechanisms include the development of a societal supporting system, renewal of traditional notions and methodologies, research on preservation systems, personnel training, organizational structure reform, policy formulation and implementation, and technological infrastructure construction. Nearly all the mechanisms (except the development of a societal supporting system) were agreed upon by more than 70 percent of respondents, and the mechanisms of policy formulation and implementation and technological infrastructure construction were agreed upon by up to 90 percent. These two mechanisms correspond indeed to the two most active kinds of initiative currently taking place in archives. This result indicated that, first, comprehensive recognition of preservation challenges had been established and secondly, that the journey towards effective long-term preservation had just begun. Figure 3 presents the agreed necessary mechanisms for preservation.

OPEN IN VIEWER

Figure 3.

Necessary Mechanisms for Digital Records Preservation.

Discussion

Two themes emerged in the data analysis that warrant further discussion: one regarding the foundation of electronic records preservation, i.e. electronic records management in records creating agencies, and the other regarding the Chinese approach to archives management.

As indicated strongly by the qualitative data, the issues surrounding electronic records management in records creating agencies were considered most challenging for long-term preservation by the absolute majority of respondents. These issues included, for example, the diversity of records creating systems, the associated variety of electronic records, the insufficiencies in records management systems that were designed with a ‘paper mind’, the lack of integration between records creating systems and the records management system, and limited metadata capture. Compounding the problem was the fact that there was a serious shortage of qualified records personnel for electronic records management in agencies and a lack of the knowledge and skills required for electronic records preservation in archives. As the research demonstrated, the seven archives which stated that their preservation systems were based on the OAIS model did not appear to fully understand what an OAIS-compliant system entails. One compelling piece of evidence is that two archives, one claiming to be OAIS-based and the other not, both implemented the same system from the same vendor. It is worth pointing out that this shortage was recognized by many archivists. As one informant stated, “our willing is strong yet our capability is weak”.

The issues surrounding electronic records management had direct impact on the effectiveness and quality of long-term preservation. The areas being impacted include the function of appraisal, the availability of records contextual information when ingesting and data management after ingestion. Over a half of the received electronic records were not appraised, which raises the question as to whether they indeed require long-term preservation. The goal of appraisal is to facilitate decision-making on what to keep, because long-term preservation is both resource and labor intensive. The lack of appraisal adds a burden to archival processing and reduces the value of kept archives. Poor electronic records management caused the unavailability of records contextual information, which was indispensable for continuous records management, including long-term preservation. The insufficiency of metadata captured in the stages of records creation and management contributed directly to the lack of sufficient information for developing comprehensive, detailed descriptions with respect to the function of data management in accepting archival institutions. These findings confirm the findings of the InterPARES project, which investigated the relationships between electronic records management and their long-term preservation for 12 years. According to the most recent reporting of the findings of its third phase, poor electronic records management had been the fundamental cause to ineffective digital preservation (Xie, 2012).

The other theme revealed some features unique to the Chinese tradition of managing archives, most noticeably, the acceptance of the same records in both paper and digital formats and the strong desire among responding archives for a nation-wide strategy for tackling the long-term preservation challenges. The first feature corresponds to the finding that the majority of transferred records are administrative rather than operational ones, which are typically in the formats of text and image. According to the survey, 73.1 percent followed this tradition and only 23.1 percent stopped receiving paper records for which digital equivalents exist. The idea of maintaining two sets of the same (sometimes similar) records requires further investigation as the decisions may be case-specific and/or record type dependent. The second feature reflects the frustration that the individual archives encountered when facing the lack of direction, limited standardization, and piecemeal solutions. As clearly demonstrated by this research, archives were taking different approaches for long-term preservation and the strong administrative relationships (i.e. the levels of archival establishment) did not appear to influence the initiatives taken. While there was a considerable number of provincial and sub-provincial archives that had started receiving digital records, the central archives (i.e. the national archives, which also assumes the role of the State Archives Administration) remained inactive. This had caused difficulties for lower level archives, which on the one hand faced pressure from records creating organizations and on the other lack of directions, methodologies and resources to tackle the challenges. As a result, the majority of respondents called for a national strategy with a top-down approach, and recommended the following specific measures:

Conclusion

As a survey research with a self-administered questionnaire, the findings of the study are limited to the general understanding of the digital preservation of records in China. However, it has served the purpose of exploring a field that is still new and emerging in China and laid the foundation by which further and future studies can be conceived. The selected sample consisted of the most representative type of archives in China, and the situations they were in and the experiences they had generated would no doubt be instructive to other types of archives in China, and potentially to archives in other countries or areas that operate in similar or comparable environments. As such, the study contributed effectively to the achievement of goals set by its parent project, which aimed at fundamental solutions for records management and digital preservation. The challenges of managing digital records and of ensuring their long-term preservation will continue to present themselves to records professionals as new digital technologies emerge at a rapid rate. Digital records will eventually become the dominant type of transferred archives, which is widely acknowledged by the archival institutions who do not currently accept digital records or do not do so in a regular manner. Recognition of the necessity of digitally preserving records exists and the enthusiasm for embarking on a new journey of caring for non-traditional records is evident. What are needed to follow up are solid methodologies that are theoretically sound and practically pertinent. This is the goal that not only the Chinese but also the international records/archival community is striving to achieve. It is hoped that the efforts of this study could assist in moving closer to that direction.