Crowdsourcing transcription of historical manuscripts: Citizen science as a force of revealing historical evidence from Croatian Glagolitic manuscripts

Abstract

This paper is a report on the project Civil Science in the Field of Glagolitics: from crowdsourcing to knowledge and it describes its first phase. The project is being conducted by the scientific Centre for Research in Glagolitism of the University of Zadar, Croatia, from 2021 to 2022. The researchers come from the Centre, as well as from the Department of Information Sciences of the University of Zadar and State Archive in Zadar, Croatia.

The main objective of the project is to examine the possibilities and benefits of citizen participation in the scholarly projects in humanities, particularly the projects whose object of research are manuscripts written in historical script that present a valuable source for local history. The term historical script refers to a script that is not used nowadays as an official script in any country or community but was in use a particular period of history on a certain territory.

The corpus for the pilot study conducted within this project consists of manuscripts and their fragments written in cursive form of the Croatian Glagolitic script. Glagolitic script is the oldest known Slavic script, introduced in the 9th century and being used in Croatia up until the 19th century, simultaneously with Latin and Cyrillic scripts. The citizen participation is researched on the example of crowdsourcing transcription of manuscripts written in cursive form of the Croatian Glagolitic script.

In the first phase of the project, the pilot study was conducted. The aim of the pilot study presented in this paper is to create a solid basis for involving the public in scientific projects within the disciplines of humanities whose object of research are documents written in historical scripts, namely within the field of the Croatian Glagolitics.

Keywords

Crowdsourcing citizen science digitization digital humanities collaborative manuscript transcription glagolitic script historical script

1. Introduction

Manuscripts are an important research corpus for various disciplines of humanities. Therefore, the humanities scholars, when conducting research, usually need access to manuscripts that are to be found in libraries, archives, and other heritage institutions. To meet the needs of those scholars concerning their access to material, GLAM (Galeries, Libraries, Archives, Museums) institutions are usually digitizing collections of old and rare books, including manuscripts. Consequently, by imposing changes in research methodologies, massive digitization of manuscript material potentiated new research possibilities in humanities and thus fostered the emergence of Digital Humanities (DH).

The library community shows a great interest in DH, primarily in assisting the DH scholars in identifying, gathering, organizing, analyzing, and interpreting data, as well as in identifying available tools and software for data organization, visualization, and other activities involved in conducting research. One of such activities is manuscript text transliteration.

Done in a traditional environment, transliteration was usually done by a single scholar, then revised by one or several colleagues, and finally published in printed form as an edited edition. Nowadays, manuscript transliteration is done in a digital environment, with the use of digital tools for transliteration that facilitate collaborative work of several scholars working on the same document. Besides being the main resource for scholars, manuscripts, especially historical manuscripts of local importance, are of public interest, and as such should be presented to the public, namely local inhabitants, citizens, etc. In addition to public presentation of digitized local manuscripts, many DH-based projects and initiatives encourage and support the involvement of citizens as active partners in the research of local historical manuscripts. They present an important part of activities and endeavors gathered under the common term Citizen science, which means involving the public, usually amateur groups, users of GLAM institutions, and civil associations in scholarly research.

One of the common methods of involving citizens in scholarly research is crowdsourcing. In the field of research of historical manuscripts, participation of the public usually takes place in the transliteration of manuscripts. Crowdsourcing campaigns of that type are conducted by a number of GLAM institutions and research centers with an aim of involving citizens in scientific activities and using the knowledge and motivation of the crowd to transliterate more manuscripts in less time. Crowdsourcing became one of the common practices directed to involving citizens in an institution’s projects, whether heritage institution’s project or scientific one.

In this paper, the idea of using crowdsourcing to engage citizens in scholarly projects, namely in the transliteration of historical manuscripts of local value written on historical script, will be presented on the example of the pilot study on the participation of citizens in the transliteration of historical manuscripts from the Zadar islands, written in the Croatian Glagolitic script.

The Glagolitic script is a historical script, one of the three scripts used in Croatia, but it is not in use anymore. It was introduced in the 9th century, and it is believed that its creators were the Saint brothers Cyril and Methodius, Christian missionaries from Thessalonica in the Byzantine Empire. It was used in the Slavic countries until the 16th century. However, in Croatia Glagolitic script was used up until the 19th century, simultaneously with predominantly used Latin and sporadically used Cyrillic scripts. In the 19th century, it was fully replaced by the Latin script. Public presentations and workshops whose aim is to promote this historically important script for the Croatian culture and identity, are usually focused on the angular variant of the script, which presents a unique Croatian type. On the other hand, manuscripts written in the Glagolitic script on the territory of the Zadar Archdiocese mostly use the cursive form, which is less legible, especially for non-experts. However, these manuscripts are extremely important for local history and culture, and sometimes are the only source of information on the local history of the Zadar islands and the surrounding area. As such, their content is supposed to be of huge interest to local citizens. This pilot study will research the feasibilities, benefits, and obstacles of crowdsourcing campaigns concerning manuscripts written in historical scripts.

2. Citizen science and crowdsourcing

2.1 Citizen science and crowdsourcing: Definition

Crowdscourcing, called also “crowd science”, “networked science”, or “massively-collaborative science” (Franzoni & Sauerman, 2014; Young, 2010; Wiggins & Crowstone, 2011), is based on the idea of enabling citizens to participate in scientific research, “as observers, funders, in identifying images or analysing data, or providing data themselves” (European Commission Website, n. d.), while Citizens Science (CS) is described either as a method, a movement, or a social capacity, always referring to “scientific work undertaken by members of the general public, often in collaboration with professional scientists and scientific institutions” (Sis.net, 2017). Crowdsourcing is closely related to GLAMs (Galleries, Libraries, Archives and Museums) which have a long history of involving citizens deeply in the wide range of activities and engagement with members of the public (Owens, 2012; Holley, 2010; Slater Reese, 2016). Both Crowdsourcing and Citizen Science are recognized by the European Commission as means of the democratization of science, and as such are represented in the EU Strategy. The Citizen Science movement in Europe is organized through the European Citizen Science Association (ESCA), which is enhancing the participation of citizens in scientific research, as well as supporting and performing citizen science projects (ESCA, n.d.).

Crowdsourcing, together with Citizen Science, has become an unavoidable method in many academic disciplines. Although some contend that the idea itself dates back to the 1880s when so-called gifted amateurs were engaged to voluntarily supply the Oxford English Dictionary with spellings, etymologies, and definitions, Brabham (2014, p. 9) argues that, although similar to it, crowdsourcing is a new phenomenon that relies on the technology of the Internet. It flourished in natural sciences in Galaxy Zoo1

¹
Galaxy Zoo. Retrieved from: http://www.galaxyzoo.org/.

project where over 250.000 amateurs contributed in bringing forward scientific discoveries of new classes of galaxies by collecting astronomical data.

The term crowdsourcing itself was first coined by Jeff Howe, editor of Wired magazine, in his article “The Rise of Crowdsourcing”, and in his blog Crowdsourcing: Tracking the Rise of Amateur. The term was adopted very quickly by the popular press and bloggers. Although over 40 definitions of crowdsourcing are found (Estellés-Arolas & Gonzales-Ladron-de-Guevara, 2012), it is usually defined as “an online, distributed problem-solving and production model that leverages the collective intelligence of online communities, to serve specific organizational goals” (Brabham, 2013, p. XIX).

To fulfil their role in the community, libraries are promoting inclusion of the public in various activities organized by the libraries or by other stakeholders and institutions. Some of those activities are based on social engagement in adding value to existing library data, whether by tagging, by commenting data, information or collections, rating or reviewing publications, or by creating and uploading content to add to library collections (Holley, 2010). Thus, social engagement of users is supported, encouraged and promoted by libraries, especially since the development of the Internet. Crowdsourcing is similar to, but not the same as social engagement. Crowdsourcing “uses social engagement techniques to help a group of people achieve a shared, usually significant, and large goal by working collaboratively together as a group” (Holley, 2010). This means that crowdsourcing involves more effort from the crowd. In the framework of this paper, it is important to explain the term crowd, which could be misleading as it refers to an anonymous mass of people, which is usually far from the truth in the crowdsourcing projects, especially those conducted within scientific projects on specialized topics like those connected to historical manuscripts and historical scripts. Owens claims that the most successful crowdsourcing project in libraries, archives, and museums have not involved crowds, but engaged members of the public (Owens, 2013, p. 121).

Crowdsourcing in humanities became possible with the growth of DH, namely with the rapid and creative deployment of online platforms and virtual meeting places for humanists to collaborate remotely. Those platforms are nowadays adjusted to crowdsourcing tasks, to be able to give the citizens a possibility to interact with scholars, to learn more about scholars’ tasks, methods and research results, to contribute to projects with their own knowledge, and perform tasks on a corpus far larger than a single scientist could process alone. Although there is a range of crowdsourcing projects within DH, it is still mostly used as a method of using the power of the crowd to transcribe handwriting (i.e. Transcribe Bentham project). Crowdsourcing in the humanities focuses on the encouraging of a wide, and different, audience to engage in processes of humanistic inquiry. It is important to note that crowdsourcing develops the potential of an online community to add, transform or interpret data that are important to the humanities and thus help humanists to deliver data that will facilitate their research process (Terras, 2016).

Estellés-Arolas and González-Ladrón-de-Guevara (2012) found around 40 different interpretations of crowdsourcing, but what is in common to all of them is what Brabham (2013, p. 3) synthesizes as: (1) an organization that has a task to be performed; (2) a community (crowd) that is willing to perform the task voluntarily; (3) an online environment that allows the work to take place and the community to interact with the organization and (4) mutual benefit for the organization and the community.

2.2 Crowdsourcing – benefits

The benefits of involving citizens in scientific research are twofold and can be divided into the benefits to institutions (projects) and benefits to citizens involved. Benefits to libraries, in particular, are listed by Holley (2010) and range from achieving goals the library would never have the time or financial support to achieve by itself, to adding value to and improving data, and demonstrating the value of libraries in the community and strengthening and building the trust of the users to the library.

Holley researched a number of crowdsourcing projects in libraries and made a Volunteer Profile, which introduces the idea that volunteers in such projects are loyal, hardworking and long, are very interested in the subject, appreciate the possibility of learning and contributing, find the work personally rewarding and want to achieve the project/group goals, etc., and they are motivated by love, interest, cause, possibility to give something back to the community, etc. (Holley, 2010). The research showed that the number of work volunteers usually exceeds the expectations of the managers, which is such a loud call for the GLAM sector and humanities scholars to go out with crowdsourcing campaigns, to achieve their goals quicker and to give the opportunity to the citizens to collaborate – both sides will grow and connect more closely.

Owens sees participants as amateurs, with the word amateur meaning “lover of”, who have been contributing to public good ever since, and suggests “to think of crowdsourcing not as extracting labor from a crowd, but as a way for us to invite the participation of amateurs… in the creation, development, and further refinement of the public good” (Owens, 2013, p. 123). We could not agree more with his idea about crowdsourcing as a powerful tool used not only for improving and enhancing data related to cultural heritage collections but also as a way of deep engagement with the public.

Brumfield, who has conducted a series of crowdsourcing transcription projects, noticed that participants of his project were very satisfied with that work, it contributed to their self-image, and they wanted to work as much as possible. It corresponds to the findings of Ridge, who claims that people, after covering their basic needs, tend to be motivated by things that do not bring financial benefit, rather provide them with a sense of purpose. Through those actions, they actually establish and sustain their identity and find a sense of belonging by being a part of something bigger than themselves. Those projects usually provide a way for them to connect with each other and make meaningful contributions to the public good (Ridge, 2014).

Advantages, disadvantages, and challenges of citizen science and crowdsourcing can vary from project to project. In literature, the advantages and disadvantages of citizen science are most often discussed in the context of a particular field. For example, Dickinson et al. (2010) in their work Citizen Science as an Ecological Research Tool: Challenges and Benefits talk about the challenges and benefits of citizen science in the field of ecology, and Mitchell et al. (2017) in their paper explain the positive and negative sides of the inclusion of citizen science in higher education. Some of the challenges in citizen science projects can be placed under organizational and motivational challenges. The organizational ones certainly include matching the project and the participants, division of labor, and project management, while the motivational ones include maintaining the motivation to participate in the project and dealing with the conflicting motivations of the participants. (Franzoni & Sauermann, 2014, pp. 13–17).

Schoenenberger, Zenzerović and Tolić (2020) also talk about the challenges of citizen science in their handbook Priručnik za graansku znanost. According to them, the main shortcomings in citizen science projects may be: (Schoenenberger, Zenzerović & Tolić, 2020, p. 15)

•
data bias if volunteers do not have appropriate training,
•
the accuracy of individual volunteers, which varies depending on motivation, difficulties in tasks and experience, and
•
complex research methods that are sometimes not suitable for citizen science.

Brabham (2008, p. 86), when talking about the challenges of crowdsourcing, focuses specifically on its negative impact on the crowd. The public often sees crowdsourcing as a way for institutions or companies to use the crowd to make a profit or some outcome that benefits them. Given that crowdsourcing is based on the web, the Internet, and technology, the author emphasizes that this can be a problem because many people are still without internet access, and even those who have internet access often do not have enough internet speed to participate with. This brings us, as the author says, to the problem that diversity of opinion cannot be ensured.

Analyzing the advantages and disadvantages of citizen science and crowdsourcing, it can be concluded that the above benefits and challenges are generalized and can serve citizen science practitioners as a basis for project development and certainly as elements to keep in mind when planning such projects. The literature has certainly shown that challenges vary from project to project, so when planning them, one should focus on analyzing possible challenges in a particular field of science.

Since in crowdsourcing campaigns crowds/citizens are undertaking small portions of much larger tasks, it is very important to be able to divide the tasks into smaller portions that will be assigned to volunteers. The quality of the campaign, and consequently of the project itself, depends on those tasks, which have to be clearly defined, feasible and suitable to the volunteers, and they have to be well integrated with the project in which the campaign is organized.
3. Research background

3.1 The written heritage project

In collaboration with Vestigia Manuscript Research Centre of the University of Graz, the scientific Centre for Research in Glagolitism of the University of Zadar is running an interdisciplinary cross-institutional project called Digitization, bibliographic description and research of texts written on Glagolitic, Croatian Cyrillic, and Latin scripts until the end of 19th century in Zadar and Šibenik area (hereinafter referred to as Written Heritage).2

²
The project started in 2017 at the Department of Information Sciences of University of Zadar, Croatia, in cooperation with Vestigia Manuscript Research Centre of University of Graz, Austria, and was transferred to Centre for Research in Glagolitism of University of Zadar upon its foundation in 2020.

Throughout the rich Croatian history, three scripts, namely the Glagolitic, Latin and Bosnian variant of Cyrillic script, called bosanica or bosančica, and three languages, namely the Church Slavonic, Croatian redaction of the Church Slavonic and Latin language, were used simultaneously (Hercigonja, 1994, 2009). Although only Latin script is in use nowadays in Croatia, Glagolitic script is strongly connected with the Croatian identity and a substantial amount of documents and literature are written in it. The main focus of the Written heritage project remains on the manuscripts written on Glagolitic script, predominantly those that originated from or are linked to the jurisdiction of the Zadar Archdiocese.

Origins of Glagolitic script are linked with the work of St Methodius and his disciples and date back to the 9 ${}^{\text{th}}$ century. The script was created for the Slavs of Moravia to be used for teaching Christian faith in Slavic language. According to one of a few theories, it was St Cyril who created the Glagolitic alphabet in the period between 861 and 863 AD. To do so, he used three basic geometrical figures, circle, triangle and square, symbolizing Divinity, Holy Trinity and Earth. It was used in Moravia, Pannonia, Bohemia, Bulgaria and Macedonia until the 11 ${}^{\text{th}}$ century, while in Croatia the Glagolitic script was used as early as from the 10 ${}^{\text{th}}$ century but continued to develop independently. It even created its own stylization there, with its graphic system adjusted to the Croatian phonological system. Besides the angular form, typical for Croatia, from the 14 ${}^{\text{th}}$ century a cursive form was developed. Glagolitic writing laid the foundations for the subsequent development of the Croatian literature and the Croatian literary language. It was also used for the first books printed in Croatian (as early as in 1483), as it flourished in Croatia in 14 ${}^{\text{th}}$ and 15 ${}^{\text{th}}$ centuries, in the period known as the Golden Age of the Croatian Glagolitics. The Croatian Glagolitic clergy are to be thanked for transferring Glagolitic script to print in such an early age of printing in Europe, but also for nurturing and preserving the Croatian Glagolitism as a symbol of the Croatian identity, culture and independence. Although the Glagolitic script is strongly connected with the Croatian history and identity, only a small number of people can read the script, especially when it is written in the cursive form.

Local historical manuscripts written by priests in parishes throughout the Zadar Archdiocese are often written in cursive Glagolitic script, which is palaeographically described as the Glagolitic script of Northern Dalmatian style, developed in scriptorium in Tkon, on the island of Pašman in the Zadar archipelagos (Štefanić, 1969, p. 20; Vigato, 2014, 2017). In contrast to its angular form, the cursive form of Glagolitic script is much more complicated for reading. An important part of the corpus of Glagolitic manuscripts written in cursive Northern Dalmatian Glagolitic script is nowadays part of collections of heritage institutions in Zadar and present a valuable source for local and national history. Although they are sometimes the only source of information about local history, they remain in archives, out of the public’s reach, waiting for scholars to read and interpret them, to fill the gaps in local historiography.

3.2 GlagoLab portal

Within the Written heritage project, manuscripts, predominantly those written in Glagolitic script, were digitized, cataloged, presented, and researched. GlagoLab3

³
GlagoLab portal. URL: https://glagolab.eindigo.net/.

– Portal and digital laboratory for collaborative research and promotion of the Croatian Glagolitic alphabet was founded with an intention of becoming a central reference point for the Zadar Glagolitism. This corresponds to the aim of Written heritage project, which is to provide a digital scholarly platform for the research of historical manuscripts, predominantly those written in Glagolitic script, as well as to create and provide a suitable platform to collect, publish and share digital reproductions and data on Glagolitic manuscripts (Tomić, 2018). In the first phase of the project, Glagolitic manuscripts of local provenance were digitized and cataloged according to international rules and standards in the manuscript description. Additionally, watermarks found in the Glagolitic manuscripts were digitized and cataloged, while both catalogs of manuscripts and watermarks were integrated, in order to provide a broad access not only to the contents but also to the various evidence on the local history of the Zadar Archdiocese being concealed in those valuable codecs (Tomić et al., 2020). Glagolitic manuscripts digitized within the project include register books (birth, christenings, matrimonies, or death registers), land records, brotherhood registers, matriculas, homily and lecture collections, and other manuscripts written in parishes of the Zadar archdiocese, including the city of Zadar and its surrounding area, and, predominantly, the Zadar islands. The texts of those manuscripts are usually the only written trace of everyday life, customs, personal and family histories, etc. (Franov Živković, 2018, 2020).

The Glagolitic manuscripts of the Zadar Archdiocese were mostly gathered by late Msgr. Pavao Kero, the researcher of Glagolitic script, former director of world-famous Permanent Exhibition of Church Art in Zadar. For more than 40 years, Msgr. Pavao Kero was visiting parishes of the Zadar Archdiocese, islands, towns and villages around Zadar, and combed through old dusty chests in the attics of parish houses, old utility rooms packed with boxes and forgotten things, as well as old closets and shelves, in a search for historical manuscripts, predominantly those partly written in Glagolitic script. His search lasted for years and besides field research, it involved a research of scattered lists of Glagolitic manuscripts of the Zadar provenance. The research resulted in an inventory of 429 codecs and their fragments of the Zadar Archdiocese provenance, written partly or in full in Glagolitic script. Kero finally published a bibliographic list of Glagolitic manuscripts of the Zadar Archdiocese (Kero, 2015). Parts of those Glagolitic manuscripts are kept in the Archive of Zadar Archdiocese. Additionally, Msgr. Pavao Kero initiated Monumenta Glagolitica Archidioecesis Iadertinae, a series in which transcriptions together with accompanying texts on more than 25 Glagolitic codecs coming from parishes of Zadar Archdiocese were published. Transcriptions of Glagolitic texts were made, while the accompanying texts were written by Msgr. Pavao Kero and several researchers from the Zadar academic institutions, namely the University of Zadar and the Institute of Historical Sciences of the Croatian Academy of Science and Art in Zadar. Each volume of the series attracted the interest of researchers of the Croatian Glagolitics, as well as historians. Msgr. Pavao Kero is responsible for the gathering of scholars researching Zadar Glagolitics informally known as Zadar Glagolitic Circle which served as a solid base for the founding of the scientific Centre for Research in Glagolitism of the University of Zadar. Also, it is upon his efforts that the Glagolitic Association of Zadar was founded, with the following objectives: to promote Glagolitic script and Zadar Glagolitism among the public, to gather amateurs interested in reading and interpreting Glagolitic documents, and to organize activities aimed at promotion of Glagolitics among the public, mainly school children. The Association actively organizes annual assembly Sabor glagoljaša Slovo rogovsko intended for students and the interested public and publishes annual journal Slovo rogovsko. The journal covers topics from the Zadar Glagolitics, including the transliterations of selected local Glagolitic manuscripts, reviews of books and activities related to the Zadar Glagolitics and information on activities organized by members of the Association, mainly consisting of primary school teachers, for and with students on the topic of the Zadar Glagolitics.

Many of the scholars researching Zadar Glagolitics are members of the Zadar Glagolitic Association and regularly take part in activities organized by Association, which mainly include public talks, book presentations and events related to the Zadar Glagolitics. Therefore, engagement in public promotion of Glagolitic script is an important part of activities undertaken by scholars whose object of research is Zadar Glagolitics.

However, those activities, although highly appreciated and commendable, are one-way activities, where scholars present their research results in a popular manner. General public is rarely offered space for getting involved in the research themselves, although the researched manuscripts come from their birthplaces, or places of their ancestors, which means they are highly motivated to read, understand and interpret those texts.

4. Pilot study Let’s research Glagolitic heritage together

To change the position of citizens towards scientific projects and to promote them as equal partners in the research of the Zadar Glagolitic, the project Civil Science in the Field of Glagolitics: from crowdsourcing to knowledge (2021–2022) is conducted as part of the wider Written heritage project. The main goal of the project is to research the possibilities and benefits of citizen participation in the scholarly projects in humanities, particularly those whose objects of research are historical manuscripts written in the historical script. A historical script in this project is a script that is no longer used as an official script in any country or community, but was in use in a particular period of history on a particular territory. Glagolitic script and Glagolitism fit well within this definition. Therefore, the emphasis of the project is placed on historical manuscripts written in Glagolitic script in the 17 ${}^{\text{th}}$ and 18 ${}^{\text{th}}$ century by Glagolitic priests in the Zadar parishes and the Zadar islands.

The idea of the project is to actively engage the citizens in research of Zadar Glagolitics. The goal is to create conditions for scholars of the humanities and information sciences, librarians, and other information practitioners to collaborate with each other and with citizens on a project of historical manuscripts digitization, transcription, bibliographic description and research. Each of the project goals, namely transcription, bibliographic description and research, are planned so as to offer the community the chance to engage in selected activities through an online collaborative platform.

In the first phase of the project, the pilot study was conducted with a goal to engage the public in the transcription of local historical manuscripts written in the cursive form of Glagolitic script, to test the feasibility, main features, benefits, and potentials of such crowdsourcing campaigns.

The hypotheses of the pilot study were as follows: (1) crowdsourcing campaign based on manuscript transliteration is a suitable method of involving citizens in scholarly projects concerning Glagolitic manuscripts, as it includes tasks that citizens are used to and for which they express motivation; (2) the target group of citizens who would be involved in the campaign are the members of Zadar Glagolitic Association – they are skilled in reading local Glagolitic manuscripts, they play an active role in promoting Glagolitics within the community, and they are highly motivated in reading local Glagolitic manuscripts; (3) if designed properly, a crowdsourcing campaign will benefit both the project and to the volunteers. A proper design of crowdsourcing campaigns includes appropriate choice of manuscripts/texts, well planned and clearly explained tasks for amateurs and appropriate easy-to-use research infrastructure, as well as clarity and visibility of the planned achievements and goals, for scholars and amateurs alike.

The pilot study was conducted in three phases. In the first phase, a scenario for a crowdsourcing campaign based on crowdsourcing transcription of Glagolitic manuscripts was created and the campaign Let’s research Glagolitic heritage together was launched. The goal of the second phase was to determine the willingness of project participants to involve citizens in selected project activities in the field of Croatian Glagolitics, while the goal of the third phase, which is yet to be conducted, is to determine attitudes, motivation, and expectations of amateurs towards crowdsourcing campaigns in the field of Croatian Glagolitics. Results of all phases of this pilot study will be used to create a model of involving citizens in scientific projects within the disciplines of the humanities, namely within the field of Croatian Glagolitics. The plan is to make the future model as general as possible, so that it can be used for including citizens in other, similar projects. This paper presents the results of the first phase, together with the first part of the second phase.

4.1 Crowdsourcing campaign Let’s research Glagolitic heritage together – the crowdsourcing scenario

The first phase of the pilot study focused on creating of a scenario for a crowdsourcing campaign based on crowdsourcing transcription of Glagolitic manuscripts and the launching of the campaign.

The crowdsourcing scenario included the following: (a) determination of the degree of openness of the campaign, (b) the plan for disseminating information on the campaign, (c) the plan for communicating with amateurs (d) the list of Glagolitic texts with an argumentation that guided the selection in terms of their suitability for the project goals, as well as their suitability for the amateurs, (e) description of tasks and activities to be assigned to the amateurs within the campaign, namely type and size of tasks as well as the model of explaining the tasks to the amateurs, (f) decision on infrastructure to be used to carry out the tasks (digital tools and research infrastructure), and (g) the evaluation of crowdsourcing campaign, including the evaluation of the quality of the transcriptions made by the amateurs and the evaluation of the adequacy of the crowdsourcing plan.

4.1.1 Determination of the degree of openness of crowdsourcing campaign

The degree of openness of a crowdsourcing campaign refers to how open the campaign is to a specific and targeted group of people and how open the campaign is to the general public. In accordance with the hypothesis of the pilot study, the target group consisted of the members of the Zadar Glagolitic Association (ZGA). However, in order to test the hypothesis, the decision was made not to limit the campaign only to the ZGA members, but to include the wider public, while the ZGA members will only be additionally encouraged to join.

4.1.2 The plan for dissemination of information on the campaign

In the plan for disseminating the information on the campaign, it was decided for the pilot study to address the members of the ZGA by posting information on their mailing list through the director of the ZGA, by posting information on their Facebook page, and by promoting and explaining the campaign personally to the members on the Annual Assembly of ZGA which took place only a few weeks before launching the campaign.

The other methods for dissemination of information included posting information on the campaign on social networks, namely the Facebook page of the Centre of the Research in Glagolitism, as well as on other Facebook pages dedicated to Glagolitics, cultural heritage, or Croatian language, including Facebook pages of other Croatian associations in the field of Glagolitics. Also, it was decided to post the information on web pages of the Centre, and of the University of Zadar.

4.1.3 The plan for communicating with the amateurs

One of the methods of communication with participants was e-mailing. A text of a “welcome e-mail” was composed, in order to be sent to each amateur after logging in to the system. The second step includes posting challenges via social networks, to attract interest from potential new amateurs, but also to create a platform for communication with the existing participants.

4.1.4 The selection of Glagolitic manuscripts

The Glagolitic manuscripts to be crowdsourced have to be selected in a manner that ensures that they are suitable to the goals of the Written Heritage project, and that they are suitable for the amateurs. Therefore, manuscripts digitized within the Written Heritage project were selected based on the suitability of their content and the legibility of the script to amateurs.

4.1.5 Description of tasks and activities to be assigned to amateurs within the campaign

Concerning the type and size of tasks assigned to the amateurs through the crowdsourcing campaign, it was decided that in the first phase, amateurs would be encouraged to transliterate texts, while in the next project phases, the task will be expanded to mark and resolve personal names and toponyms. The methods of explaining the tasks to the amateurs include simple written instructions on how to register in the system, as well as the instructions on how to transliterate Glagolitic text.

4.1.6 Decision on infrastructure to be used to carry out the tasks

The choice of the infrastructure to be used for the crowdsourcing campaign should be of high importance. The engagement of the amateurs largely depends on how intuitive and easy-to-use the infrastructure is, as well as on the possibilities it offers for collaborative work, including posting questions, comments, and other methods of active engagement and collaboration of participants. On the other hand, it should provide the project with the opportunity to engage amateurs, to monitor their work and quality of the results, as well as to benefit from their work.

Figure 1.

The Transkribus portal with transcription entered by the amateur.

4.1.7 The evaluation of the crowdsourcing campaign

The evaluation of the crowdsourcing campaign included the evaluation of the quality of the transcriptions made by the amateurs and the evaluation of the adequacy of the overall crowdsourcing plan.

4.2 Transkribus – the tool in use

After describing the task given to the amateurs and detailed research and testing of several digital tools, the decision was made to use Transkribus,4

⁴
Transkribus. URL: https://readcoop.eu/transkribus/.

the same infrastructure which is planned to be used for collaborative transliteration by scholars on the Written heritage project. Transkribus is a platform developed by the University of Innsbruck in collaboration with leading research groups from all over Europe as part of the Horizon 2020 EU research project READ (Fig. 1). The aim of Transkribus is to support all those involved in the transcription of historical, printed, or handwritten documents. It provides a range of tools for automated document capture, including AI-powered handwritten text recognition (HTR), image recognition (layout analysis), and structure recognition (document understanding). Besides being a tool for collaborative transcription in digital environments, Transkribus offers a range of integrated tools intended for annotating personal names, places, and chosen concepts in the text, enables commenting by the users/transcribers, and allows the owner of the project to monitor user activities. It offers a service called Model training enabling users to train a Handwritten Text Recognition model (HTR) in order to be able to automatically process a collection of documents. The model needs to be trained to recognize a certain style of writing. Between 5.000 and 15.000 words of transcribed material are required for the training of a model, while a greater amount of training data is usually required for handwritten texts. As there is no model trained so far for cursive Glagolitic script, it is planned to train that model within the Written heritage project. Amateurs gathered through crowdsourcing campaigns could help achieve such a large number of transcribed words.

Transkribus proved to be the right platform for the campaign, amateurs registered themselves easily, and started to transcribe texts with no complaints concerning the platform.

4.3 Discussion

The campaign was announced on the International Day of Glagolitic Script, February 22, 2021, and launched on March 30, 2021, under the name Let’s research Glagolitic heritage together! This paper discusses the results of the first month of the campaign.

The decision on whether the campaign would be open to a targeted groups of citizens or it should include a wider public was the first challenge faced by the campaign creators. In accordance with one of the hypotheses of the pilot study, the determined target group were members of the Zadar Glagolitic Association (ZGA). The decision for their selection was based on the fact that they are most skilled in reading the local Glagolitic manuscripts and are capable to progress in reading various forms of the script used in different manuscripts by different scribes. In addition, they play an active role in promoting Glagolitics within the community, and they are active individuals who are willing to research and contribute to the knowledge of Zadar Glagolitics in their spare time. Finally, they are highly motivated and skilled for reading the local Glagolitic manuscripts, and they are generally ready to put in a lot of effort and spend their spare time in reading and researching Glagolitics. However, the decision was eventually made to spread the information to a wider public, while the ZGA members would be additionally encouraged to join via email.

In contrast to expectations, the hypothesis proved to be wrong and members of the ZGA showed no interest in the campaign. They did not publish the call on the Facebook page and the director of the ZGA did not post the call via its mailing list. Although the campaign was promoted and explained by the head of the project on the Annual Assembly, the interest of the members was very low, although it is to be taken into account that due to the COVID-19 epidemic the Assembly was attended by only a few members.

However, the call for joining the campaign was launched widely, via portal GlagoLab – the University of Zadar and Centre for Research in Glagolitism web pages, and the Facebook page of the Centre. The call was also sent to other Facebook pages that focus on the Glagolitics or written heritage. Unexpectedly, the call published on the Facebook page of the Association of Friends of Glagolitics from Zagreb attracted the biggest attention. The director of the Association even posted special information on the campaign in their monthly bulletin. The members started to comment on the campaign, welcoming it and expressing satisfaction for its launching.

Soon, the call published on the University of Zadar web pages attracted the attention of local media and they published the call on their portals.

The same day when the campaign was launched, the first amateurs registered into the system and started with transcriptions. The amateurs who registered for the project were coming from different backgrounds, some from the scientific community. Although most amateurs were not affiliated with any association, some came from the Zagreb Glagolitic Association, which can be explained by the promotion of the campaign on their FB pages and bulletin.

After logging into the Transkribus, amateurs received a welcome email. The address of each amateur was collected through the Transkribus information system, as it was one of the few information amateurs had to provide through the process of registration. The e-mail contained a “Thank you” message for registering and cooperating, and presented the first step towards opening a direct communication canal with individual amateurs. In the email, it was announced that their involvement in the project consisted of transliteration of the local Glagolitic manuscripts, but that they were also expected to be involved in the analysis of texts and discussions about their paleographic characteristics, content, stories they convey. The participants were also given information on project partners and contact information and they were asked to provide their consent for the publishing of their names, achievements, number of transcribed lines, etc., on the FB pages.

The participants’ reactions to the welcoming e-mail and their comments on Facebook were very positive. They expressed the need to introduce themselves, as well as to commend the initiative. Here are some of the comments written in their emails:

“I am very glad that this project has been launched, even the digitization of the material itself is a great thing. I hope you’ll succeed.” “Thank you for the welcome and many compliments to the initiatives you launched! Apart from transliterations, I am also interested in the research of watermarks, which I read about.” “Thank you for the welcome and instructions for participating in the project. First of all, I would like to present myself in the context of potential project competencies…”

These reactions suggest that the amateurs, after noticing the call to join the campaign, researched and familiarized themselves with other segments of the project and the Centre for Research in Glagolitism. We may conclude that crowdsourcing campaigns could also serve as a marketing strategy for the project and a solid platform for disseminating project results.

In their first emails, they immediately started to ask questions related to transliteration. Those inquiries were sometimes very detailed, pointing to some general problems with transliterations of cursive Glagolitic script or even specific linguistic issues concerning dialects used in the manuscripts. Those questions showed the need for a communication platform that should be extended in the future. It is also interesting to note that no one asked questions related to the technical part of the platform. It seems that the participants had no problems using Transkribus, thanks to the instructions provided on the GlagoLab portal. The comments of the amateurs are enthusiastic, positive, and open. The welcome e-mail proved to be the right first step in starting communication with individual amateurs.

In the second round of information dissemination, the first challenge on the FB web page was posted. It was a question about an old Slavic word “kapitul” and its meaning. The word was frequently used in Glagolitic madriculas but is not used today. Two answers were offered, although not by the amateurs registered on the project. Afterwards, the organizers posted the correct answer, in a form of a short answer together with the link to the article explaining the word. As the post was viewed and commented on, we may assume that posting challenges and information on FB could be a proper format for attracting new amateurs, but also for attracting attention to the project, its results, and the Glagolitics itself.

As historical Glagolitic manuscripts are digitized, described, and researched within the Written Heritage project, it was decided to use those manuscripts for the crowdsourcing campaign. That way, the bigger number of transcribed and researched manuscripts within the project can be reached with the help of the amateurs. It is very important to note here that the number of Glagolitic manuscripts of the Zadar provenance is significantly larger than the number of scholars involved in the project. On the other hand, transcription of the Glagolitic manuscripts is a hard and time-consuming job, although to scholars it usually presents only the preparatory part of the research. By involving amateurs in the process, it is to expect that the bigger number of transcribed manuscripts could be reached in less time and that the researchers of the project could recognize the benefits of involving amateurs in project activities.

The selection of the manuscripts was guided also by the notion of the adequacy of the selection to the amateurs. Therefore, the selection was based on the adequacy of the content and the legibility of the script. Glagolitic manuscripts digitized within the Written Heritage project are historical manuscripts of local provenance rich in information. It includes descriptions of local customs and everyday life, description of rules within local brotherhoods, texts of sermons or recommendations for proper life given by local priests, as well as registry books containing names and surnames of people living in local communities, mainly at Zadar islands, toponyms, etc. They date from the 17 ${}^{\text{th}}$ to 18 ${}^{\text{th}}$ century and their content is valuable and important for the local community. Sometimes they even present the only source about the history of Zadar islands and Zadar hinterland. However, since they are kept in the Archives of the Zadar Archdiocese and are written in the cursive form of historical Glagolitic script and use archaic Croatian language, which only a minority of people can read and properly understand, the manuscripts are rarely taken into account when interpreting the history of Zadar county. Based on such criteria, it is expected that the selected manuscripts would be of interest to the citizens and that they will be highly motivated to do it. Except for the content, the selection was based on the legibility of the texts. The manuscripts are mainly written in the cursive form of Glagolitic script, with differences in legibility depending on the form of the script and the manuscript’s condition. More legible texts were selected for this pilot study.

Based on these criteria, ten manuscripts were selected for the project (Fig. 2). Five manuscripts are parish registers, three are madriculas (rules) of the parish’s brotherhoods, one is the text of a sermon, and one is a collection of lessons for a decent life written by a priest. Although written in the cursive form of Glagolitic scripts, the selected manuscripts have slightly angular forms of the letters, thus enabling the legibility, especially for people who are not experts in cursive Glagolitic. Furthermore, in the selection, we aimed for a diversity of manuscripts’ provenance; therefore, manuscripts from eight different locations from the Zadar area and its islands (Fig. 3).

Figure 2.

The list of manuscripts selected for the campaign, as shown in Transkribus portal, together with the name of the holding institution and indication of the Call Number of each manuscript.

Figure 3.

The map showing the provenance of the manuscripts selected for the campaign.

All selected manuscripts are kept in the Archives of the Zadar Archdiocese. The name of the holding institution and the call number of each manuscript are indicated in the portal used for the campaign. The permission to use selected manuscripts in the crowdsourcing project was given by the Zadar archbishop.

To describe their tasks and to help amateurs, instructions were posted on the web pages of the campaign. They explained how to register and start the transliteration, and how to transliterate a Glagolitic test. Instructions included examples of a transliteration line by line. Additionally, the written presentation of the Glagolitic alphabet was published, including short descriptive instructions focusing on special characters and ligatures. The instructions were published on the GlagoLab portal, together with the instructions on how to use Transkribus, a digital tool for transcription (Fig. 4).

Figure 4.

Instructions for registration and transliteration on the GlagoLab portal.

Figure 5.

The time of the activities of the amateurs as recorded in the Transkribus system.

By monitoring their working habits, it can be concluded that the amateurs usually work in the late afternoon, or even night hours, and that they do not interfere with each other’s texts, but rather choose one text and work on it solely (Fig. 5). It was noticed that some of them started to post comments by the text, pointing to problems with transliterating obvious mistakes done by the scribe or transliterating special characters that do not have an equivalent in the Latin script (Fig. 6). The evaluation of quality of the transcriptions made by the amateurs was performed on a sample of four pages, and the transliterations proved to be of satisfactory quality, with very few errors, mostly connected with ambiguities typical for the transliteration of cursive Glagolitic letters, including those letters which do not have their surrogates in the contemporary Croatian script. However, it is obvious that some transcribers transcribe certain characters and ligatures differently, which is especially the case with the transcription of characters that do not have equivalents in the Latin script. This points to the need for further clarification of transliteration rules, as well as the need for a discussion on transliteration issues among transcribers and members of the project.

Figure 6.

The record on activities by the amateur on a single manuscript.

It was noticed that the amateurs react to encouragements. Participants took a break in between two notices sent by the organizer of the campaign.

The pilot study campaign plan proved to be well prepared, as the campaign successfully attracted interested amateurs who actively transcribed manuscripts’ pages and started discussions. The chosen digital tool, Transkribus, proved to be easy to use. Also, selected manuscripts attracted interest. The campaign was planned in such a way to enable testing different features of the project itself, which proved to be useful for planning future campaigns.

5. Future research

In the second phase, to determine the willingness of project participants to involve citizens (amateurs) in project activities, the interviews will be made with project participants whose responsibilities and tasks within the project partly or fully overlap with the tasks planned to be crowdsourced.

In the third phase, whose goal is to determine attitudes, motivation, and expectations of amateurs towards crowdsourcing campaigns in the field of Croatian Glagolitics, the interviews would be made with amateurs who made some active contributions to the project. These interviews are supposed to shed new light and provide ideas on methods for stimulating and motivating amateurs and thus enhancing their work.

Further research is planned to design a reward system for amateurs, as well as an activity monitoring system. Also, the plan is to research methods and forms of group discussions among amateurs, as well as forms of the most appropriate public presentation of their work.

6. Conclusion

This pilot study demonstrates that the citizens are highly motivated to research their own history through original material. However, such material is usually, for various reasons, out of their reach, traditionally intended only for scholars to access and research. When getting a chance to research it, they gladly accept the opportunity and are very enthusiastic about it. Despite the fact that historical scripts, in this case, Glagolitic script, is not in use anymore and that it is not expected that lots of people are able to read them, this pilot study proves that there are amateurs who are capable of reading them and eager to do it when they get an opportunity. Also, the pilot study proved that those amateurs are not always members of associations and that crowdsourcing campaigns in this field must be directed towards the general public, not exclusively to members of specific associations. Considering their motivation, but also their knowledge on local history, familiarity with the contexts of the documents, as well as familiarity with the events and persons from their local communities, and their motivation to gain even deeper knowledge from local manuscripts, citizens’ contribution to local manuscripts’ research is incredibly valuable, although still underrepresented within scholarly projects. This is especially the case in the field of research of local manuscripts written in historical scripts, namely scripts that are not in use anymore and are thus legible only by a minority of people. Opening to the public and bringing amateurs into the research arena, where they can join efforts with scholars in the interpretation of evidence and traces of local history hidden in local manuscripts, could bring nothing but a spectacular victory to any scholarly project based on historical manuscripts’ research. To achieve that, the research should be well planned, the roles of amateurs and scientists must be defined thoughtfully and clearly, while the tasks that each participant would have to fulfill should be planned in detail and in accordance with their roles and possibilities. Also, a good collaborative platform should be provided, which enables collaborative work and the sharing of comments. It is to conclude that communication with amateurs is of crucial importance, because these participants need communication and guidance, as well as feedback on their work. Also, they put in additional efforts when they receive encouragement and feedback.

The pilot study showed that the project itself benefits from the crowdsourcing campaign, too. The campaign provides an opportunity to foster project activities, in this case, to train the model of HTR more quickly, but also to disseminate information on the overall project, its results, publication, etc., and to draw the attention of the public to the project and institution conducting it.

Results of this pilot study provide a solid basis for creating a future model of involving citizens in scientific projects within the humanities’ disciplines, especially text-centered research projects whose object of research are texts written in historical scripts. It is expected that an even more solid foundation for the model will be created, after receiving the results from the second and the third phases of the pilot study, i.e., after determining the tendencies towards involving citizens (amateurs) in project activities, and after defining the interest of amateurs in crowdsourcing.

References

Brabham

. (2013). Crowdsourcing. Cambridge: Massachusetts Institute of Technology.

Estellés-Arolas

, & Gonález-Ladrón-de-Guevara

(2012). Towards an Integrated Crowdsourcing Definition. Journal of Information Science,38(2), 189-200.

Brabham

D. C.

(2008). Crowdsourcing as a Model for Problem Solving. Convergence: The International Journal of Research into New Media Technologies, 14(1), 75-90.

Dickinson

J. L.

Bonter

D. N.

, & Zuckerberg

(2010). Citizen Science as an Ecological Research Tool: Challenges and Benefits. Annual Review of Ecology and Systematics, 41(1), 149-172.

European Citizen Science Association. (n.d.). https://ecsa.citizen-science.net/.

European Commission Website. (n.d.). Citizen Science. https://ec.europa.eu/digital-single-market/en/citizen-science.

Franov-Živković

(2018). The Glagolitic Confraternities Books of the Zadar Area and their Value as Historical Documents. In: Willer

Gilliland

A. J.

& Tomć

(Ed.). Authority, Provenance, Authenticity, Evidence. Zadar: Sveučilište u Zadru; Odjel za informacijske znanosti, 2018, pp. 383-415.

Franov-Živković

, & Deković

(2020). Bratovštine Polače i okolnih mjesta: Nadina, Rštevića, Tinja i Jagodnje u XVIII. i XIX. st. In B. Dšen (Ed.). Polački kraj u prošlosti i sadašnjosti. Zadar: Ogranak Matice hrvatske u Zadru; Ilirija d. d., Biograd, 2020, pp. 247-271.

Franzoni

, & Sauermann

(2014). Crowd science: the organization of scientific research in open collaborative projects. Research Policy, 43, 1-20.

10.

Hercigonja

(1994). Tropismena i trojezična kultura hrvatskoga srednjovjekovlja. Zagreb: Matica hrvatska.

11.

Hercigonja

. (2009). Tisućljeće hrvatskoga glagoljaštva. Zagreb: Hrvatska sveučilišna naklada.

12.

Holley

(2010). Crowdsourcing: How and Why Should Libraries Do It? D-Lib Magazine, 16 (3/4). Retrieved from http://www.dlib.org/dlib/march10/holley/03holley.htm.

13.

Kero

(2015). Popis glagoljskih kodeksa Zadarske nadbiskupije. Zadar: Stalna izlžba crkvene umjetnosti, Sveučilište u Zadru.

14.

Mitchell

Liberatore

Triska

, & Ashcroft

(2017). Benefits and challenges of incorporating citizen science into university education. PLoS ONE. 12(11), 1-15.

15.

Owens

(2012, May 20). The Crowd and the Library. Retrieved from: http://www.trevorowens.org/2012/05/the-crowd-and-the-library/.

16.

Owens

(2013). Digital Cultural Heritage and the Crowd. Curator: The Museum Journal, 56(1).

17.

Ridge

(2014). Crowdsourcing our cultural heritage. Surrey, England: Ashgate Publishing Limited.

18.

Schoenenberger

Zenzerović

, & Tolić

(2020). Prirčnik za gra\djansku znanost. Zagreb: Institut za razvoj i inovativnost mladih. https://izradi.croatianmakers.hr/wp-content/uploads/2020/09/Prirucnik-za-gradjansku-znanost-cijela-knjiga-i-korice.pdf.

19.

Sis.net. (2017). Citizen science policies in the european commission: research agendas towards issues of concern to citizens.http://www.sisnetwork.eu/media/sisnet/Policy_brief_Citizen_Science_SiSnet.pdf.

20.

Štefanić

(1969). Glagoljski rukopisi I. Zagreb: JAZU.

21.

Terras

(2016). Crowdsourcing in the Digital Humanities. In Schreibman

Siemens

& Unsworth

(Ed.). A new companion to digital humanities, UK: Wiley & Sons, Ltd, pp. 420–439.

22.

Tomić

(2018). Project Digitization, Bibliographic Description and Research of Texts Written in Glagolitic, Croatian Cyrillic and Latin Scripts Until the End of the 19th Century in the Zadar and Šibenik Area (Written Heritage). Cataloging & Classification Quarterly, 56(4), 1-5.

23.

Tomić

Grzunov

, & Eškinja

. (2020). Mogućnosti i pretpostavke filigranolških istrživanja u kontekstu istraživanja zadarske glagoljske bštine. Vjesnik bibliotekara Hrvatske, 63(1-2), 1-30.

24.

Vigato

(2017). Glagoljske oporuke sa sjeverozapadnih zadarskih otoka. Zadar: Sveučilište u Zadru.

25.

Vigato

(2014). Jezik i pismo najstarije sačucvane silbenske glagoljske matice kštenih. Zadar: Sveučilište u Zadru.

26.

Wiggins

, & Crowston

(2011). From conservation to crowdsourcing: a typology of citizen science. Proceedings of the Annual Hawaii International Conference on System Sciences, 1-10. Retrieved from: https://crowston.syr.edu/sites/crowston.syr.edu/files/hicss-44.pdf.

27.

Young

J. J.

(2010, May 28). Crowd science reaches new heights. The Chronicle of Higher Education. Retrieved from: https://www.chronicle.com/article/The-Rise-of-Crowd-Science/65707/.