Evaluating and extending the Informed Consent Ontology for representing permissions from the clinical domain

1. Introduction

Informed consent is a foundational requirement in both clinical care and research studies. Informed consent forms serve as the primary source of evidence that an informed consent process occurred, and that the consenter has received enough information to make an informed decision regarding the permissions they are being asked to either grant or deny (The Joint Commission, Division of Healthcare Improvement, 2016). While efforts are underway for tracing such permissions for biospecimens collected during research studies, significantly less attention has been given to clinical consent forms and permissions to reuse residual clinical biospecimens. Residual clinical biospecimens are the portions or derivatives of blood or tissue collected during clinical care process that remain after their clinical indications are fulfilled. These specimens are increasingly recognized as a valuable resource.

There is a need for information systems to facilitate discovery, access, and responsible reuse of stored biospecimens and data, and to facilitate data integration and knowledge discovery within a contemporary connected research environment. This vision requires development of technology which supports expectations around FAIR (Findable, Accessible, Interoperable, Reusable) Principles and includes metadata to support discovery of biospecimens and data according to their permitted and restricted uses (Wilkinson et al., 2016). Semantic web technologies such as ontologies hold great promise as infrastructure solution for scalable, interoperable approaches in health care and research (Kock-Schoppenhauer et al., 2017). Ontologies structurally enable integration of heterogeneous data sources by semantically representing core entities and relationships of a given domain and have been widely successful in biomedical data sciences (Smith et al., 2007).

The Basic Formal Ontology (BFO) is a realism-based, upper level ontology comprised of terms that represent objects (i.e., continuants) and processes (i.e., occurrents) (Arp et al., 2015). This common structure is the basis for enforcing logical rules across all ontologies which import or refer to BFO, including those of the Open Biomedical Ontologies (OBO) Foundry. All OBO Foundry ontologies share core principles, including (1) use of a shared logical structure (i.e., BFO) as the source for common classes and relationships, (2) open access and use, (3) community-based collaborative development, (4) and non-overlapping, strictly scoped content. The OBO Foundry enforces these design principles to achieve scientific accuracy and semantic interoperability through a formal logical basis (Smith et al., 2007).

The Informed Consent Ontology (ICO) is part of the OBO Foundry and serves as a reference ontology for representing informed consent (He, 2015/2020; Lin et al., 2014). ICO was designed to represent documents and processes specifically relevant to biomedical and health research. ICO contains representations of processes such as signing an informed consent form and Institutional Review Board (IRB) approval of the consent form, as well as for the investigator and participant roles. Although ICO was not developed to represent the nuanced information specific to informed consent in health care processes, ICO offers the ability to model individuals’ decisions during a process of informed consent. While some of the existing classes may be transferable to this new domain, ICO must be formally evaluated for reuse in clinical consent processes and extended or revised as necessary.

2. Objectives

The purpose of this study was to evaluate ICO for its completeness in expressing clinical permissions, including the reuse of residual clinical biospecimens and health data. ICO was subsequently revised and extended to broaden the reference terminology artifact and to make interoperable representations of clinical permissions available.

OPEN IN VIEWER

Fig. 1.

Multistep, iterative ontology evaluation process.

This study follows a formative evaluation design in which we examined an information resource (ICO) under development (Friedman and Wyatt, 2006). Figure 1 depicts our multistep evaluation process which includes: identification of study aim(s), development of research or evaluation questions, the iterative investigative loop – comprised of data collection, data analysis, and reflection and reorganization – synthesis of findings, community negotiation of revisions, and ontology revision and extension. Our analysis was also guided by evaluation methods and questions abstracted from the literature, including the National Institute of Standards and Technology’s (NIST) Ontology Summit’s guidance for evaluating ontologies across ontology life cycles (Neuhaus et al., 2014). Stakeholder involvement, including ICO developers and the OBO foundry community members, was solicited throughout the evaluation process.

3.3. Iterative investigative loop

The Iterative Investigative Loop was comprised of three steps: data collection, analysis and modeling, and reflection and negotiation (Friedman and Wyatt, 2006). These steps were performed both iteratively and in tandem, enabling continuous revisiting of the data, revision of the models, and collaboration with stakeholders.

3.3.1. Data collection

Data collected and analyzed for this evaluation study included permissions for clinical processes generally and reuse of residual clinical biospecimens and health data specifically. These permissions were abstracted from US federal regulations and clinical consent forms. First, permissions to reuse residual clinical biospecimens within US federal regulations were identified through a review of biomedical, legal, and health policy literature. The full methods and results of this review are described elsewhere (Umberfield et al., 2021b). By identifying these permissions through the literature, we reduced introducing our own biases into ICO by instead extracting the interpretations of what is legally permissible by experts in their respective fields and incorporating a range of perspectives. The included regulations were:

21st Century Cures Act, 2016

Americans with Disabilities Act of 1990

Clinical Laboratory Improvement Act, 1992

Food and Drug Administration Amendments Act of 2007

Genetic Information Nondiscrimination Act of 2008

Health Insurance Portability and Accountability Act, 1996

Patient Protection and Affordable Care Act, 2010

Protection of Human Subjects (i.e., Common Rule), 2019

Rehabilitation Act of 1973

Research and Investigations Generally (2011) (as cited in Public Health Services Act, 2019)

Second, an annotation study of clinical consent forms was conducted to identify permission-sentences. Permission-sentences are statements within the consent form that, when the form was signed by the patient or their legally authorized representative, permitted the health care facility or its clinicians to do some action or activity. Clinical consent forms were collected via direct contribution by health care facilities and systematic web searching, the methods of which are described elsewhere (Umberfield et al., 2021a).

OPEN IN VIEWER

Table 1

List of permission-sentences from clinical consent forms which were included in this evaluation. Note: by agreement with recruited facilities, we obscured facility names whose consent forms were not publicly available using “XXX”. For all facilities whose consent forms were publicly available, we did not remove identifiers

Purposively Sampled Permission-Sentences
Reuse of Residual Clinical Biospecimens and/or Health Data
1.	I hereby authorize XXX to retain, preserve and use for scientific or teaching purposes, or to dispose at its discretion or convenience, any specimen or tissues taken from my body during my visit.
2.	I am a New York state resident and I give permission for GeneDx to retain any remaining sample longer than 60 days after completion of testing and use my de-identified data for scientific and medical research purposes.
3.	I DONATE and authorize XXX to own, use, retain, preserve, manipulate, analyze, or dispose of any excess tissues, specimens, or parts of organs that are removed from my body during the procedures described above and are not necessary for my diagnosis or treatment.
4.	I agree that any excess tissue, fluids or specimens removed from my body during my outpatient visit or hospital stay (my specimens) that would otherwise be disposed of by the Hospital may be used for such educational purposes and research, including research on the genetic materials (DNA).
5.	I authorize the pathologist, at his or her discretion, to retain, preserve, use, or dispose of any tissues, organs, bones, bodily fluid or medical devices that may be removed during the operation(s) or procedure(s).
6.	I hereby consent to the use and disclosure of my protected health information as described in the Notice of Privacy Practices.
Other Clinical Procedures and Activities
7.	By signing this form, I am requesting and giving my consent for MHSM and the doctors and/or nurses to give me blood and/o [sic] blood products during this admission or series of treatments.
8.	Your signature below indicates that you understand to your satisfaction the information about the genetic testing ordered by your health care provider and that you consent to having this testing performed.
9.	I consent to the Facility videotaping, photographing, video monitoring, or taking other recordings of me or parts of my body for diagnosis, treatment, research, or patient safety purposes.
10.	I also consent to diagnostic studies, tests, anesthesia, x-ray examinations and any other treatment or courses of treatment relating to the diagnosis or procedure described herein.
Randomly Sampled Permission-Sentences
11.	I, _____ , request and consent to the start or induction of my labor by my provider and other assistants as may be selected by him/her.
12.	I voluntarily consent to receive medical and health care services that may include diagnostic procedures, examination, and treatment.
13.	I consent to the use of closed-circuit television, taking of photographs (including videos), and the preparation of drawings and similar illustrative graphic material for scientific purposes providing my identity is not revealed.
14.	I hereby consent to engaging in virtual health/telemedicine services, where available, as part of my treatment.
15.	In the event a healthcare worker is exposed to my blood or body fluids in connection with my procedure, or during my hospital stay, I agree to the collection and testing of my blood for HIV.

OPEN IN VIEWER

Fig. 2.

Context for a permission directive (permission-sentence).

Permission-sentences were eligible for inclusion if they were positively identified by all three annotators of the consent forms. A sample of 15 permission-sentences were included in this evaluation (Table 1). Four sentences were selected to for the heterogeneity of the activities they permitted (e.g. videotaping, surgery, anesthesia). Six permission-sentences were purposively selected because they explicitly permitted sharing or reuse of residual clinical biospecimens or health data. Five additional permission-sentences were randomly selected to reduce bias potentially introduced through purposive sampling. The sample was not further extended as the design pattern for permission-sentences (Fig. 2) remained stable when modeling the five randomly selected sentences.1

¹

While these sentences pointed to a range of processes (e.g., telemedicine, induction of labor), all processes fit under the umbrella class of ‘health care process’ (OGMS:0000096) or its parent, ‘planned process’ (OBI:0000011). Similarly, although a new role, ‘Pathologist role’ (OBI:0000145) further specified who might also bear the ‘permission role’ (ICO:0000199). The list of all permitted processes and all roles of those granted permission in all instances of consent forms is innumerable. Inclusion of these processes and roles is more apt in application ontology, rather than in ICO or other reference ontology. All permission-sentences included in this analysis are listed in Table 1.

3.3.2. Analysis and modeling

The goal of informal modeling is to identify all relevant ontological entities (i.e., classes and relationships), the entities’ important attributes, and appropriate terminology for the new domain. By informally modelling in concept maps, the models’ content is made understandable to both ontologists and domain experts (Neuhaus et al., 2014).

First, we identified all entities and their relationships within each permission from the source data. We then referenced key parent classes modeled in ICO – including information content entity, material entity, process, and role – and began sorting the identified entities into these categories without defining them or their hierarchies. At the same time, we graphically modeled the loosely defined classes from the consent forms using Mindjet MindManager and PowerPoint.2

²

For regulations, we focused just on identifying classes rather than graphically modeling their relationships, as there is not yet a legal ontology in the OBO Foundry. While important, a formal legal ontology is out of scope for the present work and is beyond the domain scope of ICO.

We carefully examined ICO and OBO Foundry ontology classes in terms of their labels, textual definitions, and formal definitions for fidelity to their use in the context of US federal regulations and clinical consent forms. These steps were iteratively performed until a final model for each permission from the source data was developed and vetted by team members and collaborators.

3.3.3. Reflection and negotiation

Ontology evaluation methods are not yet standardized. Therefore, we abstracted evaluation questions from the literature to guide verification (i.e., did we build it right?) and validation (i.e., did we build the right thing?) of ICO revisions and extensions. These reflection questions were used to guide negotiation during regular meetings with ICO team members and biweekly calls with members of the Ontology for Biobanking (OBIB) community. The list of evaluation questions includes:

Verification. 

Are the revisions adherent to OBO Foundry principles (OBO Foundry, 2020)?

Are classes consistent across the ontology’s hierarchy (Zhu et al., 2009)?

Are classes non-redundant (Zhu et al., 2009)?

Does the model capture only entities within the specified scope of the ontology (Neuhaus et al., 2014)?

Is the documentation sufficiently unambiguous to enable a consistent use of the terminology (Neuhaus et al., 2014)?

Validation. 

Does the ontology contain the necessary and sufficient information (identified through the source data) to make it fit for our particular purpose? Are all entities within the scope of the ontology captured (Neuhaus et al., 2014; Zhu et al., 2009)?

Do the domain experts agree with the ontological analysis (Neuhaus et al., 2014)?

3.3.4. Ontology revision and extension

ICO was revised and extended following all evaluation steps, synthesis of findings, and negotiations of proposed revisions with members of the OBO Foundry community. We categorize these revisions into three categories: (1) classes or relations within ICO that needed modification (2) classes or relations required for modeling clinical permissions, but whose content more appropriately belonged to the domains of other OBO Foundry ontologies, and (3) classes required for modeling clinical permissions that were not available in ICO or another OBO Foundry ontology. For the first category, we worked with ICO developers to revise the semantic labels, definitions, and the logical position of the entity within the ontology. Once completed, we performed automated artifact evaluation including lightweight reasoning using ROBOT, an open-source ontology tool (Jackson et al., 2019), to ensure structural consistency with OBO Foundry design principals. For the second category, we submitted requests to other OBO Foundry ontologies or worked directly with the developers to implement updates necessary to cover clinical permissions. Finally, for the third category, we added these classes to ICO’s import specification before running automated evaluation using ROBOT.

4. Results

Prior to this evaluation, ICO contained 893 classes and 95 object properties (relations). Table 2 provides summary counts of all classes used in the modeling process, including those ICO classes that were transferable to this new domain and those that should either be imported or added to ICO or the OBO Foundry suite of ontologies for representing clinical permissions.

Modeling permissions abstracted from US federal regulations and clinical consent forms used 52 classes and 12 object properties already within ICO, demonstrating the appropriateness of extending ICO for the clinical domain rather than developing a new ontology. Five of these classes from ICO were flagged for revision. Revisions were regarding either the classes’ formal definitions (i.e., position within the hierarchy) or human-readable definitions, which were ambiguous or inaccurate.

Evaluation also revealed that extension of ICO is necessary to represent permissions from the source data. Twenty-six classes were recommended for import into ICO from other OBO Foundry ontologies. Additionally, we recommended twelve new classes to either be added to ICO or another OBO Foundry ontology in order to express all content within the included source data. These changes are summarized in Appendix A, which is available in Umberfield et al. (2021c). Appendix A includes tables summarizing:

Following these changes, ICO contains 20 new classes or relations via import and 9 de novo ICO classes. Remaining classes not imported required further discussion and are documented as requests. In total ICO contains 1000 classes and 93 relations. Table 3 provides the ontology abbreviations used throughout this paper as well as the full PURL for that ontology. Appendix B contains supplementary tables reporting the respective classes and relationships used for each of the fifteen permission-sentences from clinical consent forms and mapping classes to sentence content (Umberfield et al., 2021c).

Early in the modeling process, a design pattern for the context of permission-sentences emerged. Figure 2 is the informal model of this base design pattern: The individual who is granting permission (‘homo sapiens’ (NCBITaxon:9606); ‘consenter role’ (ICO:0000086)) participates in a process of consenting (‘informed consent process’ (OBI:0000810)) by using a consent form (‘informed consent form’ (ICO:0000001)) which contains a permission-sentence (‘permission directive’ (ICO:0000244)) that prescribes some process (‘planned process’ (OBI:0000011)). It should be noted that this is the most simplified version of this design pattern, and there is significant heterogeneity and added complexity as each permission-sentence was modeled. As an example, the real-world person (i.e., instance) who is the consenter may also have a range of other important roles including being the patient (‘patient role’ (OBI:0000093)) or the patient’s legally authorized representative (‘legal guardian role’ (OMRSE:00000038)). Likewise, the processes that are prescribed by permission-sentences also varied widely, but most often included a ‘health care process’ (OGMS:0000096) like surgery or blood product administration, a ‘specimen collection process’ (OBI:0000659), or an ‘act of data sharing’ (ICO:0000228).

OPEN IN VIEWER

Fig. 3.

Graphic model for the following permission sentence: “…I give permission for GeneDx to retain any remaining sample longer than 60 days after completion of testing and use my de-identified data for scientific and medical research purposes.”.

As the permission-sentences became more complex, so too did their graphic models. Figure 3 demonstrates this complexity. In this example, not only was some process prescribed, but also specifications on how the data that emerged from that process may be used (‘data use limitation’ (DUO:0000001)) and the timeline by which these processes may occur (‘temporal restriction directive’ (new class, ICO:0000363)). Additionally, the flow of a given biospecimen becoming a residual clinical biospecimen which bears an ‘excess material role’ (ICO:0000313) was fleshed out to ensure that all necessary classes were modeled.

As the source data were modeled and reviewed with team members, we asked the following evaluation questions to evaluate the revisions and extensions of ICO in terms of verification and validation:

5. Discussion

This evaluation revealed substantial overlap and therefore appropriateness of using ICO to represent permissions expressed in clinical consent forms. It further identified gaps and inconsistencies for representing such permissions and summarizes necessary extensions and revisions of ICO and other OBO Foundry ontologies. In their study to identify the minimum metadata necessary for biobank information systems to share their samples and data, Norlin et al. (2012) recognized the need for ontologies which represent the “ethical standards under which the samples are collected, any restrictions on research use, and access requirements to the samples” (p. 344). This work addresses a critical gap in formally representing how residual clinical biospecimens and health data can be responsibly reused, which is of increasing value in data-intensive health sciences (e.g., population health, translational science, precision health, etc.) which require access to such information. This work produces a machine-readable semantic resource, grounded in real-world sources included regulations and clinical consent forms. It further demonstrates the importance of continued development of reference terminologies to solve real-world problems.

One important feature of this work is demonstration of a bottom-up modeling approach to evaluate and extend an existing information resource using real-world source data. Such an approach has historically been used to map out the structure of a domain as a step towards developing information infrastructure (Harris et al., 2015; Schleyer et al., 2007). It enables discovery of the information needs of users (Shang et al., 2018) and – by fitting such modeling within an existing top-level structure – builds out content that is interoperable with existing, in-use classes and promotes infrastructure reuse (Freimuth et al., 2012). By using real world source data (regulations and consent forms), we were able to identify content coverage gaps in ICO and other OBO Foundry ontologies that may not have been otherwise identifiable. It also adds this missing content to the suite of ontologies, enabling use alongside widely adopted biomedical ontologies and acting as a valuable contribution for biobank information management.

It is our hope that ICO will serve as a reference terminology to facilitate responsible reuse of residual clinical biospecimens and health data, by both increasing their discoverability by entities that will advance scientific knowledge related to health and also protecting the agency and rights of patients, whose expressed choices regarding the disposition of their biospecimens and health data should be respected. Revising and extending ICO to include representation of this new domain enables future system interoperability of permissions and obligations for sharing and use which can be adopted by a range of systems and applications. As examples, ICO may also be used for assessing informed consent tools such as eConsent forms and as a resource for mapping and annotating text in future natural language processing tasks. It may also be used to build query tools or decision support systems to support covered entities, biorepositories, federated research networks, institutional review boards, and other individuals in identifying eligible biospecimen and data resources that meet their needs or deciding if and when certain biospecimen and data resources should be shared.

Ontology evaluation is a well-recognized challenge. Despite more than a decade of publication on the issue, there remains no standard methodology for ontology evaluation (Burton-Jones et al., 2005; Gelernter and Jha, 2016). Among the criteria by which ontologies are evaluated are “its coverage of a particular domain and the richness, complexity, and granularity of that coverage; the specific use cases, scenarios, requirements, applications, and data sources it was developed to address; and formal properties such as the consistency and completeness of the ontology and the representation language in which it is modeled” (Obrst et al., 2007). Ontology evaluation is underutilized, leading to the release of poor ontologies and ultimately hindering “the successful deployment of ontologies as a technology” (Neuhaus et al., 2014). Currently, systematic evaluation of ontologies requires that evaluators assemble methods from across various evaluation schemas. One strength of this study is its systematic evaluation of ICO, including both verification and validation.

Importantly, our evaluation and modeling process provides a roadmap for improving and expanding domain knowledge within ontologies, and addressing knowledge-representation gaps that hinder their successful uptake (Harris et al., 2015). This role is particularly well-suited to clinical research informaticists, an emerging and recognized specialty that leverages informatics to discover and manage new knowledge relating to health and disease and their use in research (American Medical Informatics Association, 2020). We also wish to amplify the need for collaboration and transparency in ontology development. There are hundreds to thousands of ontologies across repositories, but these ontologies are not necessarily interoperable. It is only with a shared semantic structure and collaborative negotiation of ontology structures that dynamic and changing knowledge can successfully be modeled in interoperable families of ontologies. This work must occur in open and collaborative communities to achieve transparency in knowledge representation.

Several limitations of this evaluation are recognized. First, deliberation with other OBO Foundry ontology communities is ongoing to determine the most appropriate home for some of the new terms which may be out of scope for ICO. Citations for correspondences regarding proposed and actual changes to ICO and other OBO Foundry ontologies are listed in Appendix C (Umberfield et al., 2021c). Second, while we have worked with ontologists and clinical research informaticists, we have not yet presented our models to domain experts from the legal and compliance, health information management, or biobanking domains. However, the extensive literature review and systematic collection and analysis of clinical consent forms mitigates this limitation; the literature and approved consent forms are a form of the collective voice of domain experts. Future refinements will include further checking with experts to ensure we have correctly interpreted the literature and the permissions.

6. Conclusion

Representing permissions to reuse residual clinical biospecimens and health data in an information resource that is interoperable within a family of recognized biomedical ontologies is valuable for facilitating the responsible reuse of these resources at scale. By evaluating and extending ICO, we make a meaningful step in this direction. Our methods demonstrate the use of a bottom-up approach to modeling content from their respective domains’ perspectives. We propose such methods as a valuable way for clinical research informaticists and domain experts to engage in the development and revision of semantic information resources.