Assessment of Static Telecytological Diagnoses' Reproducibility in Cervical Smears Prepared by Means of Liquid-Based Cytology

Introduction

Telecytology is a component of telemedicine and is simply defined as the practice of transmitting digital cytology images through telecommunication networks to remote viewing locations for diagnosis, storage, or education. ¹ It can be used for obtaining expert opinions on difficult cases from distant laboratories, as well as for training and educational purposes. ^2,3 Telecytological diagnosis can be achieved either with the use of cytological pictures viewed in real time from the microscope (dynamic telecytological systems) or with the use of cytological pictures that are first captured in a digital format and then transmitted using a store-and-forward approach to distant observers (static telecytological systems). ⁴ Few studies have focused on the possible impact of static telecytology in the everyday laboratory's work flow. Most of these studies have found a high (90–95%) concordance between telecytological and glass slide diagnoses. ^{5 –10}

Still, there are not enough studies available referring to the method's diagnostic reproducibility, especially in the field of gynecologic cytology. The aim of this study was to confirm static telecytological systems' diagnostic reproducibility by measuring interobserver and intraobserver agreements among four cytopathologists who reviewed through an e-learning platform static, digitized, representative images from cervical smears.

Materials and Methods

The current study was carried out on 404 consecutive cervical smears: benign, 135; atypical squamous cells of undetermined significance, 92; low-grade squamous intraepithelial lesion, 62; high-grade squamous intraepithelial lesion, 87; squamous cell carcinoma, 26; adenocarcinoma, 2. The material collected was prepared by the ThinPrep 2000 automated slide processor (Cytyc Co. [now Hologic^®, Bedford, MA]). From each case, one slide was prepared, stained by the Papanicolaou method, and examined by a skilled cytopathologist. Copies of the original cytological reports were kept for further evaluation and comparison with the final histological diagnoses.

All cytological slides were retrospectively collected from the department's registry in sequence and randomly numbered from 1 to 404. One cytopathologist imaged 10 representative fields from each case. Images were captured with a Hamamatsu C4742-95 digital camera (Hamamatsu Photonics, Herrsching am Ammersee, Germany) mounted on a Leica DMLB microscope (Leica Microsystems, Wetzlar, Germany). The camera was connected via an SCSI interface to a 1,200-MHz Pentium CPU running Windows XP (Microsoft, Redmond, WA). The fields selected provided valuable information that is routinely used in cytological diagnosis, such as background, cellularity, and nuclear and cytoplasmatic details. All areas of interest were imaged at×100 and×400 magnifications. Images were captured at 1,124×1,120 resolution at 24-bit color depth and 300 dpi. JPEG compression was applied to all pictures. Each case's pictures were segregated and uploaded into the CytoTrainer4 e-learning platform.

CytoTrainer4 is a Web-based distance education system, developed in the Department of Cytopathology, “ATTIKON” University General Hospital, Athens, Greece. The system is used for the continuous education and scientific support of cytopathologists. The system is designed and developed in such a way that would provide users with a flexible time and place of access. The core is the clear separation between trainers and trainees. Trainees can browse through the educational material, collaborate in synchronous and asynchronous mode, practice their skills through problems and tasks, and test their knowledge using a self-evaluation tool. Trainers manage the learning material, attend students' progress, and organize problem and task-based scenarios. The training system enables person-to-person interaction and encourages trainers' active participation in students' learning activities. CytoTrainer courses focus on separate organs and ThinPrep cytology. Tests from each organ (i.e., Pap test, breast, or thyroid) constitute unique lessons in the system's environment.

CytoTrainer is equipped with a self-assessment module in which the trainer presents real cases to the trainee with all the information required to give a conclusive diagnosis. The self-assessment cases are divided into four parts: case presentation, discussion of the presentation, clinical case resolution, and discussion of the resolution. The trainee can view a set of illustrative photographs of each case, contact with a tutor or another student online, and leave messages in the Message Board. After submitting the diagnosis, the trainee receives a feedback regarding the correct diagnosis. The feedback is not just a confirmation of a correct or an incorrect diagnosis, but a short text with a rationale behind the selection, including links to the corresponding images of the case. Moreover, the feedback includes several sections that the student should explore, accompanied by additional explanation and external resources.

The self-assessment module of CytoTrainer was involved in this study; however, the cytopathologists were not given any clinical or patient history information, and case discussion was not allowed. Only the case images and the option to select among various diagnoses were provided. In addition, in the case of diagnosis failure, the cytopathologists were given the opportunity to view images of similar cases, in order to enable improvement of their knowledge.

All 4,040 representative images were transferred to the CytoTrainer platform as assessment material and were reviewed remotely by the four independent board-certified cytopathologists on workstations using Microsoft's Internet Explorer Web browser. Each account was password-protected. Each of the four cytopathologists had a sufficient but different level of training and experience in liquid-based gynecologic cytology.

A detailed discussion of diagnostic criteria was not conducted between the cytopathologists, nor was clinical information provided. The diagnostic approach of digital images concerning criteria of diagnosis, terminology of lesions, requirements of adequacy, and recommendations for further management ere similar to those applied in conventional cytology (the 2001 Bethesda System for reporting cervical or vaginal cytologic diagnoses). The Bethesda System for reporting cervical or vaginal cytologic diagnoses was revised in 2001 to establish uniform terminology and standardize diagnostic reports. In addition, it introduced a standardized approach for reporting if an individual specimen is adequate for evaluation. ¹¹ In our study, the cytological reports were classified in basic diagnostic categories. Diagnostic categories and their subclassification are presented in Table 1.

To confirm the reproducibility of the telecytological diagnosis and to study the interobserver agreement further, 12 and 24 months after the first checking round, the same representative digital images were presented in random order to the same cytopathologists and were reviewed again by them. The intraobserver variability was studied among the checking round and all subsequent review rounds.

Diagnostic agreement was analyzed for intraobserver agreement by using the diagnosis given by the same cytopathologist between the two separate diagnostic rounds and for interobserver agreement by using the diagnoses given by the different cytopathologists on each one of the diagnostic rounds. All analyses were performed using the SAS statistical package (SAS Institute Inc., Cary, NC). Cohen's κ statistic was used for the calculation of interobserver and intraobserver reproducibility. ¹² To determine the significance of the intraobserver κ values, we used the Svanholm formula. ¹²

Results

Telecytological diagnoses in checking and review rounds by distant cytological assessment are presented in Table 2. The κ statistic was used to analyze the interobserver agreement between telecytological diagnoses and the initial cytological diagnoses available (Table 3).

The κ statistic was used to analyze the interobserver agreement among the cytopathologists during the checking and review rounds (Table 4). Initially, during the checking round, the diagnostic agreement was nearly perfect among all cytopathologists. Interobserver agreement did not alter significantly during the following review rounds. The κ statistic also was used to analyze the intraobserver agreement of the diagnosis by the same cytopathologist in the different diagnostic review rounds based on the three main diagnostic categories and their subcategories. The results are shown in Table 5. More specifically, the diagnostic agreement among all diagnostic rounds for the five cytopathologists ranged from perfect (for the more experienced cytopathologist) to nearly perfect (for the other less experienced cytopathologists) with the corresponding overall κ values ranging from 0.76 to 1.

Discussion

To the best of our knowledge, this is one of the first published studies to examine the diagnostic accuracy of telecytology for the diagnosis of digital images from cervical–vaginal smears prepared by a liquid-based thin-layer preparation.

Telecytology, when integrated into the daily work flow, can provide special consultation opportunities to distant laboratories. ² We suggest that static telecytology systems are preferred because of their low cost by laboratories that cannot afford the high cost of buying and maintaining dynamic systems. ^1,3 In addition, the telecommunication costs for the operation of static systems is lower as the usual common ADSL Internet connection is adequate to operate a static system.

Appropriate field selection, sufficient image quality, and especially diagnostic expertise are the most crucial parameters ensuring the good function of a static telecytological system. ² Static telecytological systems' reliability can be assessed by continuous monitoring of interobserver or intraobserver diagnostic reproducibility in selected digital images.

Agreement is the total or proportional number of cases that were given the same diagnosis between or within observers, including the part of the agreement that may be attributed to chance. ¹²

Reproducibility is the part of the agreement that cannot be explained purely by chance. Reproducibility is measured by the κ statistic. Within the positive κ values and in accordance with the study by Landis and Koch, ¹² the agreement was interpreted as follows: a range of 0.00–0.20 indicated slight agreement, a range of 0.21–0.40 indicated fair agreement, a range of 0.41–0.60 indicated moderate agreement, a range of 0.61–0.80 indicated substantial agreement, and a range of 0.81–1.00 indicated nearly perfect agreement.

In the current study, 404 cases (4,040 digital images from cervical smears) were diagnosed within 12 months in two review rounds. This procedure enabled the cytopathologists to become more familiar with the process and to gradually increase slightly the diagnostic agreement in relation to the experience progressively acquired in the later rounds.

Regarding the intraobserver variability, the results of the current study demonstrated that the agreement concerning each diagnostic category by the same cytopathologist improved progressively throughout the different review rounds as the observers reassessed previous specimens. More specifically, the intraobserver variability was nearly perfect among the five cytopathologists and increased gradually during the diagnostic review rounds with κ values ranging from 0.76 to 1 (Table 3).

The most common manifestation of interobserver discrepancy is upgrading of the telecytological diagnosis to a definitive carcinoma diagnosis or downgrading of a suspicious telecytological diagnosis to a rather benign lesion because of image deficiencies. ^12,13 High digital image resolution and good color reproduction in this study contributed to the agreement of the four experienced pathologists. As a consequence, the agreement among the four experienced cytopathologists was perfect for the diagnosis of squamous carcinoma, nearly perfect for the diagnostic categories of adenocarcinoma and intraepithelial lesions, and substantial for epithelial cell abnormalities (atypical squamous cells of undetermined significance).

Still, the use of digital images for diagnostic purposes in the absence of a conventional slide was a new experience for all participants, and some of them needed some additional time to become familiarized with the whole process.

In conventional cytology, specific diagnostic criteria and pitfalls are already described and can be observed in cervical smears. During static telecytological diagnosis, the cytopathologist has to use the same diagnostic criteria and to avoid the same pitfalls. ¹⁴ What makes the telecytological diagnosis more demanding is the uncertainty about the real specimen's adequacy or the representativeness of the selected digital images. The role of the person appointed to capture and transmit images is of paramount importance for the success of a static telecytological system. In our study, the person who captured and transmitted the digital images was an already certified cytopathologist with adequate experience in cytological diagnosis of cervical smears. Our study suggests that cytological diagnoses proffered on the basis of digitized images (from cervical smears prepared by means of liquid-based cytology) can be as reliable as those of conventional cytology. The diagnostic reliability of this method makes possible its use for further upgrading of the laboratory services, by producing digital educational material for use in Web-based training systems. ¹⁵ Those programs can improve the professional skills of the participating medical staff and make them feel more confident in their daily work. Furthermore, digital images can be used in proficiency testing programs for assessing the diagnostic expertise of the laboratory's medical staff.

Summarizing the possible applications of static telecytology in the years to come, we can predict its further application for remote primary diagnosis, second opinion consultation, archiving interesting cases through slide replication, implementation of remote quality assurance programs, educational purposes, proficiency testing, and board exams. The future of telecytology is promising, yet additional studies are recommended to further evaluate the usefulness, reproducibility, and diagnostic accuracy of this innovative method in all fields of diagnostic cytopathology.