Feasibility documents as critical structuring objects: An approach to the study of documents in digital research production

Introduction

One of the most interesting changes in the Digital Humanities, a young field that applies digital tools and methods to the humanities, in the past few years is the emergence of new voices of research software engineers (RSEs), research software analysts and data scientists, who have introduced different epistemological and methodological approaches. Their experimental collaborations with humanists have led to taking on more risky and large-scale scholarly projects as exemplified by ‘Shakespeare in the Royal Collection’ conducted by King’s College London, Birkbeck, the University of London, the King’s Digital Lab, the Royal Collection Trust and CogApp; and ‘Living with Machines’ by the British Library, the Alan Turing Institute and other academic partners. Cross-institutional projects involve a large team of people who are responsible for particular tasks and their delivery at the right time and within budget. The outcomes of digital projects extend beyond a single output to produce a set of technologically and methodologically advanced artefacts. For example, in the case of ‘Shakespeare in the Royal Collection’, the team has produced a publicly accessible database of all the Shakespeare-related holdings, 3D visualisations of key spaces at Windsor Castle where Shakespeare’s plays were performed, and an exhibition and timeline presenting the history of Shakespeare in the Royal Collection (SHARC, 2021).

Digital historical and cultural artefacts are no longer only objects of humanities research analysis and interpretations, but they have become moving objects between different actors – universities, cultural institutions, creative industries, funding bodies – who represent different priorities, policies and opinions. Digital artefacts are therefore the objects of negotiations about their copyright clearance, task responsibilities, technical functionality, long-term sustainability and time delivery. Humanists are not accustomed to seeing scholarly research projects in terms of multi-layered project management and risk assessment. However, large-scale digital projects require a new way of approaching and delivering them.

Discussions about the roles of RSEs and data scientists have become more visible in the UK-based Digital Humanities (DH) community since 2019 when the Society of Research Software Engineering was founded. In 2020 and 2021, three significant reports were released, explicitly articulating the challenges and need for the future development of digital research: The Challenges and Prospects of the Intersection of Humanities and Data Science, a white paper by the Alan Turing Institute (McGillivray et al., 2020), Sustaining Digital Humanities: Important Developments in the UK Landscape by the Software Sustainability Institute (Bergel et al., 2020) and Capacity Enhancement in Digital Humanities in the United Kingdom and Ireland: Training and Beyond, a discussion paper of the UK–Ireland DH Network (Romanova et al., 2021). These reports constitute the outputs of ongoing debates, extending beyond the UK DH environment, which aim to draw attention to the following two key issues in the field of DH. I will briefly outline them as a framework for further analysis.

The first aspect highlighted by the reports is to strengthen collaborations between humanists, RSEs and data scientists as a prerequisite for creating meaningful, critically driven and risk-assessed digital research (Peirson et al., 2016). However, to improve these partnerships, there is a need for common terminology and methodological understanding so that ‘methodological decisions can be fully explained, motivated and connected with the original research questions and interpretation of the research results’ (McGillivray et al., 2020: 15). The main challenge of collaborative projects results from the fact that they involve task divisions, skills and expertise and those are often used as a reference point to differentiate between intellectual contributions and technical work.

The study of work practices is therefore crucial for a better understanding of the role of technical contributors whose work is often neglected, as Steven Shapin states in the widely known article The Invisible Technician: ‘The price of technicians’ continued invisibility is an impoverished understanding of the nature of scientific practice’ (1989: 563). In digital practices, we can observe similar ways of technical work being rendered invisible as a result of the aforementioned juxtaposing of intellectual and practical work. The latter tasks with their focus on engineering objects, producing digital artefacts and maintaining digital systems are often obfuscated and go unrecognised (Lischer-Katz, 2019). It is therefore necessary to improve the understanding of RSE work as research-driven engineering that constitutes an articulation of Agre’s critical technical practice whereby the craftwork is integrated with and informed by the reflexive work of critique (1997).

The second aspect addressed by the UK reports mentioned above is the sustainability issue that is often associated with maintaining infrastructure and systems, but which is rarely connected with research management and workflow. An explicitly articulated workflow is, however, an assurance of the practical implementation of digital research projects. As Bergel et al. (2020: 17) argue, ‘The sustainability of DH is directly linked to the impact of the activities within which digital resources and methods are involved. Within those activities, DH is variably recognised’. Therefore, there is a need for more awareness about the importance of project management practices that along with feasibility and risk assessment can ensure the delivery of critically robust and sustainable artefacts.

To this end, for instance, the work of King’s Digital Lab in engaging RSE practices adapts the software industry’s approaches of Dynamic Systems Development Method (DSDM), an agile project delivery framework and the Software Development Lifecycle (SDLC) to design and produce high-quality products. This is a new perspective, and is not without its critics, as will be discussed later in this article. It also has implications for how DH may be considered within the commercial world. It is enlightening to consider that digital work does not only produce digital artefacts (e.g., software, tools, digital collections, databases) but it also ‘delivers change’ (Smithies, 2011): cultural, methodological and technical.

DH artefacts – digital collections, archives, 3D models, mapping and visualisations – are co-constructed by human, technical systems, operational methods and institutional bodies and constitute new forms of critical expression. They require us to understand and reveal the ways that humanists combine critical approaches in making digital objects, how they translate scholarly questions into technical solutions and how they collaborate with RSEs. This, in turn, demands empirical studies for the social unfolding of Digital Humanities practices.

The call for ethnographic and critical work within the Digital Humanities by the Software Sustainability Institute (Bergel et al., 2020: 7) is not the first attempt to draw attention to engagement in ethnographic research within DH. The calls for social studies by Christine L. Borgman (2009) and Alan Liu (2013) constituted an explicit articulation of the need to investigate how knowledge is produced by human beings in relation to, for example, instruments, technologies and institutions. Liu proposed the study of Digital Humanities centres as laboratories, similar to the ethnography of scientific laboratories developed in science and technology studies (Hess, 2001). A Digital Humanities lab is a site where humanities thinking is integrated with computational, engineering and organisational practices (Malazita et al., 2020; Pawlicka-Deger, 2020; Smithies and Ciula, 2020). It is therefore a technologically and socially dense organisation where an observer can uncover the practices and patterns behind task-oriented projects and team interactions and is a place of culture informed by local settings and policy. This article aims, therefore, to contribute to the development of ethnographic research in DH by providing a new methodological approach that is the analysis of documents produced in DH work. So far, the ethnographic studies in DH have been conducted based on interviews and observations (Antonijević, 2015). Documents, as the article shows, can in turn provide new insight into social and management practices that cannot be captured by observations. This approach, however, requires ethnographers to gain access to labs documents. This is not a straightforward process but an effort that demands receiving an ethical clearance and above all, building trust between ethnographers and lab members.

These discussions are a starting point for my reflections about the notions of critical technical practice (Agre, 1997), critical computational thinking (Berry and Fagerjord, 2017) and critical production (Smithies, 2017) in the context of Digital Humanities work. The goal of this article is to provide an analytical approach to investigate how humanities inquiries are translated into methodological and technical practices, and how technological and operational methods contribute to humanities knowledge production. The analysis is based on the ethnography of King’s Digital Lab (KDL), a lab comprising RSEs who work on technical research solutions for digital research in the humanities and social sciences. Although the case study represents a lab located in the United Kingdom, the methodological and theoretical analysis can contribute to the development of DH research conducted in other research environments.

KDL’s project management is based on an Agile approach to delivering solid and sustainable digital products, seen as ‘moving targets’ resulting from the coordination of ‘many moving parts, differing opinions and priorities’ (KDL, 2020a). Part of KDL’s Software Development Lifecycle is to prepare ‘Feasibility’ documents, which inform the lab’s decision to progress with a project proposal. Feasibility documents aim to assess technical and design requirements based on research questions and to organise a project workflow. They provide an interesting insight into the components, layers and management of digital products. The prevalent perception of the technical work of RSEs within DH as a ‘mechanistic and uncritical procedural activity’ (Ciula et al., 2020) reflects long-standing social and cultural attitudes that demonstrate a lack of knowledge of technical work. This essay will challenge this perspective and present interrelated and multi-layered actions of commitments, requirements and expectations as recorded in the lab Feasibility documents.

Documents are artefacts that include references to the social processes through which they were produced and reproduced (Shankar et al., 2017: 59). They have been increasingly recognised as important objects of investigation in science and technology studies (STS), as Kalpana Shankar et al. states: ‘Documents are often, maybe always, at the centre of efforts to achieve coordination and control, so greater attention to the complexity of their role should result in better accounts of how this is accomplished’ (2017: 70). I propose therefore to use the method of the ‘STS of documents’ (Shankar et al., 2017) and analyse Feasibility documents produced by KDL in a manner similar to STS-based studies of scientific labs’ protocols (Crabu, 2014) and kits (Neresini and Viteritti, 2014). I suggest Feasibility documents (1) inform ethnographic work about lab workflow and management and in doing so, are able to capture the interconnectedness of work layers and practices; (2) enable an empirical analysis of digital research projects and the process of translation from research questions, to methods, to technical solutions and (3) are critical structuring objects that structure the research process and relationships between involved actors and are structured by local institutional strategies and decisions. This article aims therefore to provide a methodological framework for the analysis of documents produced in the DH that have the potential to unearth new questions about the socio-technical nature of digital research production.

An Agile approach in a research software engineering-based digital humanities lab

King’s Digital Lab (KDL) was founded in 2015 to provide software development and infrastructure to departments in the faculties of Arts and Humanities, and Social Science and Public Policy at King’s College London. At the time of writing, it has 13 permanent full-time staff: research software analysts, research software engineers, a research software systems manager, UX/UI designers, project managers and a director. It is a unique lab entirely focused on research software engineering (RSE) to ‘create digital tools to explore academic research in new ways’. KDL has been evolving at the intersections of two communities: the Society of Research Software Engineering and Digital Humanities. These two groups seem to be divergent but are connected by the digital dimension of their work.

The perception of software engineering in DH is that it provides technical support and infrastructure for sharing resources in a similar manner to institutional IT services. RSE is, however, more than a service – it is a research service. While it is concerned with the ‘production, use, and maintenance of large, complex software systems’ (Gold, 2009), it does so by addressing research questions. The manufacture of research-enabling software systems has become part of the research workflow where software and tools are tailored specifically to scholarly questions, and incorporated into the lifecycle of a research project, from conceptualisation, to deployment, to preservation.

RSE has been changing the perception of technical work as purely mechanical labour devoid of intellectual elements. It is an engineering activity led by critical, methodological and epistemological concerns. KDL is devoted to technical development but above all, it aims to ‘explore the epistemic and methodological implications of digital humanities development’ (Smithies et al., 2019). Software is seen as an ‘epistemology engine’ (Smithies, 2017: 154) that shapes the way humanists gather, produce and transfer knowledge. Research software engineers draw attention to the process of production – of methods, software and infrastructure – showing how the manufacture of these layers underpinning scholarly research can affect the way scholars conceptualise and conduct that research. As Gold says, ‘Similar approaches have been used to great effect in scientific discovery where the process of reaching a result can be as important as the result itself’ (2009).

KDL is committed to the Software Intensive Humanities where ‘research proceeds through the development and use of tools designed to enhance (or in some cases make possible) certain kinds of research’ (Smithies, 2017: 154). Over the last 6 years (2016–21), the lab has delivered and inherited from the Department of Digital Humanities more than 150 projects involving more than 200 partners from higher education, cultural institutions and creative industries. The digital works are greatly diverse and range from digital archives and digital scholarly editions to data visualisations and analysis. Each work is built upon close collaboration with partners to ensure the delivery of high-quality digital outputs at the right time within budget. While the mixture of academic and business approaches and its implementation to the humanities domain is innovative and risky (e.g., the risk of adopting business perspectives on management, quality and success of products), the principles of collaboration and ‘being more than a digital service’ have a long tradition in Digital Humanities at King’s College London.

Harold Short in one of his interviews reflected on the development of the Centre for Computing in the Humanities (later the Department of Digital Humanities and closely affiliated to KDL) that was rapidly evolving at King’s College London in the early 2000s. The new centre began offering undergraduate programmes, launched a master’s programme, and coordinated large numbers of research projects.

The crucial thing from the very outset is that we were not a service, we didn’t have a service role in our projects, we only took on projects if it was as equal partners … someone wanted a database built, we’ll say ‘well, the Computing Services Department can do that for you’. If they want to talk to us about how to address research questions, then that’s where we started. (Short et al., 2012)

It is fascinating to read about the development of Digital Humanities by their pioneers who built the foundation for such a new field, and who have been tasked with articulating the epistemology of building that is knowing and thinking through making (Bradley, 2019; Nyhan and Flinn, 2016; Ramsay and Rockwell, 2012; Thaller, 2017).

KDL was therefore built upon a long tradition of King’s Digital Humanities and has evolved from the Centre for Computing and the Humanities (1991), the Centre for eResearch in the Humanities (2008) and the Department of Digital Humanities (2011–). In 2011, at the moment of institutional transformation, the Department explicitly articulated its goal for future years to ‘more clearly identify it[self] as an academic department, rather than a service centre’ as well as to undertake the issue of digital preservation and sustainability that had not yet been resolved (Maron, 2011). One of the main goals of KDL at its first year of activity was to archive around 100 legacy Digital Humanities projects, delivered over a twenty-year period. This immense task turned out to be a challenge on many fronts: orphaned projects were left without funding for maintenance, among other things, and the obsolete PHP-based frameworks posed an infrastructural security risk (Smithies et al., 2019). To ensure a sustainable and scalable approach to digital research projects, KDL has adapted the Software Development Lifecycle (SDLC) which is built upon an Agile method.

SDLC refers to a methodology and production process with clearly defined tasks and requirements for delivering high-quality software (KDL, 2021a). This approach is widely used in software engineering where an artefact is built by an interconnected team responsible for providing functional and sustainable products within budget and time constraints. Figure 1 presents the SDLC adapted by KDL and which consists of nine stages: 1) initial contact (partners contact the lab with their project ideas); 2) internal assessment (KDL review the project ideas at the weekly lab Project Pipeline meeting and produce a ‘Terms of Reference’ document); 3) requirements assessment (the lab analyses the feasibility, risks and impact of the project, considers technical solutions, and designs development tailored to the project questions; prepares a Feasibility document; and makes the final go/no-go decision); 4) funding applications (the project proposal is submitted to grant funders); 5) kick off (if the project is funded, the lab organises a meeting with partners to discuss the work process); 6) evolutionary development (the process of developing artefacts based on timebox planning, communication and close collaboration with partners); 7) deployment (each increment is reviewed with the partners after which they are officially deployed); 8) release (the project goes live and partners can discuss with the lab how to make the project sustainable beyond the project lifespan) and 9) post-project (the lab assesses if the requirements and expectations have been met and assess the maintenance options of the project) (KDL, 2021b).OPEN IN VIEWER

The KDL workflow is built upon a business-driven framework called the Agile approach. It is a method driven by the principles of iterative development (where the software production is broken down into smaller tasks tested and delivered incrementally), flexibility (a quick response and adaptation to changing conditions) and delivering a high-quality product on time (Agile Business Consortium, 2017). This approach contrasts with traditional project management, wherein the features of the project are fixed, meaning that all of them are equally important. Therefore, in order to deliver them, we can manipulate both the time frame and the cost to keep the project on track. Since all features are required and time and cost are variables, there is a risk that the project will be managed poorly, which leads to a lowering of the quality of the artefact. In the Agile approach, however, time, cost and quality are fixed, which means that we can negotiate features to deliver a high-quality product within a given project time frame. To determine which features are essential and which can be left out, the lab team uses the MoSCoW prioritisation technique to specify ‘must’, ‘should’, ‘could’ and ‘won’t have’ requirements. This technique will be explained further in the next section.

The Agile-based SDLC process is focused on delivering the software in the form of incremental pieces, which are specified in advance but can change in an ‘agile’ way during the process of design and development. While it is a useful approach that helps the team to have control over the production, it is necessary to be conscious that behind this rigid business structure, there are human aspects. Ultimately, the lab deals with individuals who have their own research ideas, expectations and aspirations. The lab is aware that they work on software but above all with people who are attached to their ideas. ‘Their research is their baby’, we often hear at the lab meetings. At moments of disagreement between the different parties, however, the SDLC seems to be used to salvage the situation: ‘Just stick to the SDLC’, one research software analyst said after a frustrating meeting with the partners.

The SDLC can help the lab to keep a project on track when unpredictable human aspects disturb the work process. This business model is just a framework within which the lab seeks to tame the complexity of digital production. The flexibility of the Agile method is therefore useful to find the balance between expectations and requirements, innovation and sustainability and experimentation and functionality. The resistance to structured work processes such as the SDLC model that is sometimes experienced stems from the assumption that there is no place for creativity and individualism. However, at the Feasibility document preparation stage, the human elements of the project can come to the surface alongside technical aspects and institutional priorities.

Feasibility documents

Documents are rich empirical materials for ethnographic scholars as they document social practices (What is the subject of documentation?) as well as the conditions of documenting the process (How has this documentation been carried out?). Although ethnographers deal with a fixed and completed artefact, documents refer to the action taking place in the past; the text enables them to extract information about factors that determined the way the work was conducted and to reproduce the documentation practice if any additional information, such as authors’ comments, is left in the margins. Documents are dense artefacts that refer to people, places, objects and events and become a relevant source of knowledge about organisational forms and practices (Shankar et al., 2017). They also make practices and factors – sometimes invisible in a daily observation – visible and explicitly articulated. In lab meetings, used by the STS ethnographers as one of the ‘epistemologies of vision point’ (Garforth, 2012: 281), observers can see how a lab’s work is progressing with the use of specific tools. However, it is not always empirically obvious why this tool was chosen – social motivations and decisions are hidden behind the vision point. The project documentation – if it is done well – can contain information about the stages of developing the work, from its conceptualisation (the moment of selecting tools and their justification) to development, to deposition.

Empirical materials reveal themselves as powerful objects that can illuminate organisational processes, social practices and a lab’s culture and aims in structuring and controlling the relationships between the actors involved in the production of artefacts. Tracking the term ‘responsibility’ in documents, for example, is an interesting way to identify the points of tension and uncertainty in the work process. As Lisa Garforth notes, ‘Seeing is dynamic and practiced in the social science field, and as such, its negotiation and management has consequences for our understanding of the ethnographic gaze and what is at stake in observational methods’ (2012: 266). The same practice can be applied to studying documents which record the processes the authors wanted to register. It is therefore important to be critical about what is being seen in these documents; the points of exclusion and invisibility should be treated as a part of the ethnographic narrative in the form of expert treatment and reflexivity.

Feasibility documents constitute a part of the SDLC model and aim to ‘gather information to help decide if the project should be stopped, if unlikely to be viable’ (KDL’s Feasibility documents and project templates are openly available at GitHub (KDL, 2020b)). After the initial contact with partners and several meetings, the lab team prepares the Feasibility document to assess the project for the lab. This involves activities such as the elicitation of requirements, reviewing partners’ data samples and datasets, sketching technical solutions and checking whether the budget is sufficient for such work.

The aim of using a Feasibility document is to plan the work process by breaking down large segments of activities into items of high-level requirements following the MoSCoW technique. The document is also intended to assess risk and impacts and by doing so, it seeks to provide justification to the university for taking on the project. The document is prepared by the lead research software analyst in a collaboration with the research software engineers and designers who together outline possible technical solutions, taking into consideration a range of potential risks, from data set quality and accessibility, to methodological uncertainty, to technical security. The Feasibility documents are important artefacts that aim to be both fixed and flexible at the same time. They constitute a rich resource for the ethnographer who can follow which parts are under control and which are open to negotiation.

I propose to conduct a ‘feasibility analysis’ that can reveal the project management and development stages: 1) the analytical process (the translation of research questions into technical solutions); 2) the production process (the move from technical and design practices to research answers) and 3) the infrastructure and management process (project workflow and sustainability solutions). The analysis will also unpack the complex, iterative processes – along with the factors behind them – of moving from research questions to software analysis and design methods, to technical requirements. The Feasibility document represents the accumulation of expectations that are structured into the logical process of the translation of inquiries into material feasible items. I intend therefore to conceptualise Feasibility documents as critical structuring objects that aim to organise the process of digital production into iterative and interdependent stages.

Based on the analysis of 40 Feasibility documents produced by KDL, I first sought to understand how they inform the lab management work and how they contribute to structuring the digital research process. Thinking about the work structure, it is helpful to illustrate it by means of a diagram, in order to show the multi-layered structure of digital research production where all elements are interconnected. Figure 2 represents the feasibility model and the lab workflow. The lab work consists of the following processes and layers.OPEN IN VIEWER

First is the ‘Analytical process’ which is the intellectual inquiry, meaning the research ideas submitted by partners (e.g., humanists, social scientists, archivists) to KDL and a methodological layer that aims to assess by the lab members which digital methods and software would be suitable for addressing the scholarly questions. This step is led by a research software analyst in close contact with partners and the rest of the lab team (developers and designers) who can help to ensure that the technical and methodological solutions are feasible and functional.

Next is the ‘Production process’ which entails the technical development and delivering increments to partners. This layer corresponds to the so-called ‘black box’ of production, meaning that partners receive the finished pieces of work without observing how the production was carried out. Having said that, the Feasibility documents provide transparent information about the development process; therefore, partners can gain an understanding of how the work will be conducted. The ‘degree’ of black box access depends, however, on partners’ technical knowledge and engagement. For instance, the requirement: ‘Minimal specification VM for Django set-up for data entry’ might be understood by some partners and unclear for others.

The ‘Maintaining process’ takes us to an infrastructure layer coordinated by the software system managers. This substrate of digital production reminds us how digital work is very much material – storage, backup solutions and random-access memory – and how humanists put less emphasis on the underlying processes of their work which is crucial for its accessibility, safety and sustainability.

The last, ‘Monitoring process’ constitutes a backbone management layer of the lab run by the project and lab managers. This is a cornerstone of the lab work responsible for planning, executing and controlling projects.

The diagram of the feasibility model (Figure 2) consists of the main part called the ‘Research case’ that is variable due to projects. Therefore, while the projects are treated individually, the lab organisation and Agile-based workflow around the ‘Research case’ stays relatively fixed and stable. This confirms how the SDLC approach aims to provide a strong foundation for the entire process.

The research cases, with their methodological, technical and infrastructural requirements, constitute solid empirical materials which enable ethnographers to follow the process of translation, from research objectives, to methods, to technical solutions. In Figure 3, I present an example of the KDL digital project, ‘Radical Translations: The Transfer of Revolutionary Culture between Britain, France and Italy (1789–1815)’ led by the King’s Department of French and the Department of English and Comparative Literature, which seeks to study translation as a political intervention in the period following the French Revolution (RT, 2021). The KDL contribution has been to produce a set of research/discovery visualisations, such as a timeline that would allow the research team to better understand the documents, the meaning of ‘being radical’, and its connection to translation strategies. The project requirements were broken down into eight high-level requirements devoted to separate but interconnected objectives, methods and technical solutions.OPEN IN VIEWER

Figure 3 can be read from top to bottom to see how the methods and technical development are tailored to each objective. For instance, one research question is to study transnational radical ideas and translations during the French period and their impact on cultural exchange. To do so, the team has chosen the prosopographical and bibliographical data model with associated data structure and data storage solution to collect information about translators and translated texts. This is a ‘must’ task (called M1: a ‘must’ requirement) as it aims to provide a core database for the research upon which the following tasks are built. The resource has been created using Django, an open-source web publishing framework that has been widely adopted by the KDL team. The document justifies the selection of this solution as a ‘stable, powerful, and scalable’ web platform that gives control over the interface and web copy.

Another task (called C7: a ‘could’ requirement) is devoted to representing the movement of people (translators and other activists) and for this objective, a visualisation method (e.g., maps) has been selected as a suitable technique. This requirement has been assigned as a ‘could’ task that strongly depends on M1, meaning that C7 is only feasible if biographical events and associated locations are collected and structured at the M1 assignment stage. For visualisations, the team chooses LeafletJS, which is a widely used open-source JavaScript library used for web mapping applications.

Going through the Feasibility documents, ethnographers can gain deep insight into the lab organisation and workflow processes. However, what is even more interesting is the analysis of their content to understand not only what the documents record but also why the processes were planned to be carried out that way, and noting that they might have diverged since then (e.g., C7 became a ‘would’ during the evolution of the project) as well as how the lab decisions were made.

Feasibility documents as critical structuring objects

In the section that follows, I intend to conceptualise Feasibility documents as critical structuring objects by investigating two different relationships between 1) data, methods, and technical practices and 2) the lab team and partners. By doing so, I aim to identify factors that determine the way those relations and the following decisions are shaped. Figure 4 illustrates the action and decision model distilled in the process of Feasibility document preparation. Research questions, data sources and tech stack accumulate and undergo SDLC-driven logical operations performed by considering three indicators: requirements (How can the project be conducted?), data (What is the subject of modelling and production?) and factors (Why are decisions made this way?).OPEN IN VIEWER

These three components are interconnected and determined by many aspects represented in the form of the grey triangle in the diagram. The decision of whether to proceed with the project is a complex process whereby the lab team considers overlapping or conflicting relationships: the university management and infrastructure, lab management, strategy, time, research questions, capacity, budget, software stack, impact and risk. By analysing all those aspects filtered also through the culture of the lab, the team comes to the final go/no-go decision represented by a carefully assessed list of technical solutions, the division of responsibility, feasibility conditions and restrictions.

If we look again at Figure 3, we can explore how high-level humanities research questions (‘Intellectual inquiry’ section) are translated into technical solutions (‘Methodology’ and ‘Technical development’ layers) and what infrastructure is required to complete and maintain the digital products (‘Maintaining’ part). The Feasibility document, however, also provides insight into a decision-making process. The partners contacted the lab team to discuss digital solutions that would allow them to study transnational radical ideas and translations during the French period and to better understand what it means to be radical and how translation strategies are essential to this. As the next step, a research software analyst has explored methodologies to tackle the research questions and, after consultation with the sub-team assigned to the project idea, proposed the data modelling and associated data storage approach as well as user interface functionalities to identify recurring radical concepts and ideas. While assessing the possible technical solutions, the lab team has investigated preliminary data collected by the partners and decided whether they are sufficient to produce a representative data model. A research software analyst noted that the data have still been underdeveloped and identified them as one of the risks that needs to be addressed in the form of improving the structure of data collection. The lab team has proposed, therefore, to work on building a solid data structure that would be a prerequisite for creating discovery tools and conducting the research analysis. They have further identified high-level technical requirements and decided which one is ‘must’, ‘could’ or ‘won’t have this time’. These decisions are significant since they have further implications on research directions as well as costs. The lab team provides high-level information about what costs are needed to deliver the technical solution. As the research team has a specific amount of money, limited by grant funding, then it becomes clear what it can be done within the budget. This is a part of the negotiation and refinement of the research questions as the partners can propose which requirements should be prioritised. Having said that, the lab team decides the extent of shuffling the requirements as they might be mutually dependent, i.e., meaning that one task is only feasible if another one is completed.

Time is also another crucial factor in planning digital production. The lab team allocates time for infrastructure setup, accessibility testing and change freeze, and this entails that less time can be devoted to the design and development as well as other research explorations. This is clearly communicated to the partners and further; the teams together decide how to reconcile the time the research team gets to complete the project and the time proposed by the lab. Another factor taken into consideration by the lab team is impact. As the document specifies, ‘The resource been envisaged constitutes for now the only project idea KDL is involved with in collaboration with the department of Comparative Literature at KCL and could be an interesting challenge to extend current conceptual models and related tools that aim at intersecting historical space and time’. The research partnership turned out to be one of the key elements in making a decision on whether to proceed with the project.

This multistage and multi-layered decision-making process shows how the lab treats digital production and research software engineering seriously, as James Smithies stated in presenting KDL software stack: ‘This level of organisation helps us manage technology, but also promotes critical awareness: we don’t only use technology, we choose technology’ (2016). Digital research projects are more than employing tools and they involve, above all, a responsibility towards cultural heritage, artefacts, history and society. This is why the lab stresses the ‘design first philosophy’ that resonates well with Agre’s sensibility of critical technical practice: ‘According to Agre, design in a critical vein should itself be a form of inquiry into the systematic failures, limitations, and confusions that arise in the design of digital systems’ (Vertesi et al., 2017: 177). Reflective design and critical production approaches are visible in the lab work at the Feasibility document preparation stage where many voices, inquiries, and expectations are accumulated. I will next consider more closely the two previously mentioned relationships, beginning with the connections between data, methods and technical practices.

Conclusion

Digital production is driven by curiosity and creativity, but as projects grow in size and complexity, the risk and responsibility increase. Kirk Woolford et al. (2010: 203), while discussing a critical technical practice, argue that the software craftsperson or artist-engineer undertakes exploratory experiments and investigations with new tools but ‘it is not purely driven by curiosity’: ‘The craftsperson has a commitment to a client, and to the pragmatics of a commissioning situation. It is not ethical to take risks with a client’s money, and deadlines must be honored’. A similar cautious approach to critical technical practices has been observed at KDL. As the lab says, ‘Whilst our passion is to create and develop exciting new tools, we can’t be driven by these appetites alone’ (KDL, 2020a).

The lab’s responsibility is dispersed over several different actors: project partners, cultural institutions and the university. The team deals with many risks of different natures: technical (e.g., mechanical failure, security issues), financial (e.g., budget limitations), methodological (e.g., flaws in data modelling, inaccurate methods) and reputational risks (e.g., poor project management and development can lead to weakening the lab and the university’s reputation). The risk assessment as part of the Feasibility documents is therefore a significant task carried out by the lab to mitigate and protect themselves from any unexpected mishaps. The lab approaches each project cautiously and seriously since from the moment of a project being accepted, the team gives its full commitment to provide a robust and sustainable artefact.

To this end, the lab adapts the Agile DSDM approach to its SDLC framework to offer a carefully structured project workflow. As part of this model, the lab prepares the Feasibility documents that, as has been shown, play a significant role in assessing the methodological, technical and design requirements by considering a broad set of factors. I have sought to show that the Feasibility documents constitute important ethnographic materials which can provide insights into a lab workflow and management process as being a complex and interconnected set of layers and practices. Further, I have conceptualised the documents as critical structuring objects to indicate how they help align different relationships, responsibilities and expectations as formed by local strategies, decisions and risk assessment.

This article has also aimed to offer a methodological framework for the empirical analysis of documents produced in the Digital Humanities within an RSE context. Drawing on the method of the ‘STS of documents’, I intended to show that documents can be studied as ethnographic objects that can help to reveal critical and socio-technical practices entangled with operational methods and local requirements. So far, much less attention has been given to the study of documents produced in DH; however, given a diverse and large number of materials being the outputs of DH work (e.g., feasibility documents, kits, protocols, white papers), it has become timely and significant to approach them to investigate the socio-technical practices of digital research production. The DH can therefore benefit greatly from the use of the STS-based method of reflective discourse on documents. Documents can enable critical empirical analysis of the production of DH outputs, including the investigation of social relationships, task divisions, labour issues, the workplace culture, technical practices and infrastructural values. They are the result of efforts aiming to reach consensus, coordination, and control by considering the nature of personal and organisational accountability. Thinking more deliberately about documents produced in DH can shift attention from end-product digital objects towards the complex process of their creation, which can unpack a range of social, technological and management issues.