Glenn M. Preminger, Giovanni Pagliuca, and Jean de la Rosette

The Clinical Research Office of the Endourological Society (CROES) is the research arm of the Endourological Society. The Office promotes, initiates and manages a series of large-scale international multicenter clinical studies. Its mission is to improve our knowledge of several critical issues in endourology. All the CROES studies are global and on a large scale in terms of number of patients recruited and amount of data collected. Because of the nature of the studies conducted, a core aspect of the CROES activities is to ensure that data provided are valid and reliable and adhere to high-quality standards for clinical research. Data audit is therefore a necessary and essential step needed to guarantee the quality of the different clinical databases currently managed.

The CROES has just completed the first data audit of its innovative GreenLight™ Laser Study. This newsletter will describe this experience and the audit process that has been implemented and will soon be extended to all our clinical studies.

Data collection and quality audit overview

The GreenLight Laser Study assesses the current indications for HPS GreenLight laser treatment and the treatment outcomes in terms of objective and subjective improvement. A large amount of data has been collected in this study since its start, from historical patient data with regards to preoperative, intraoperative, and postoperative information, including follow-up at different stages after intervention. A dedicated online data collection system is in place to facilitate data collection globally. ¹ Twenty-eight centers have been actively participating in the study, across 21 different countries, recruiting more than 1000 patients to date, and regularly adding data to the CROES online data management system, using dedicated Electronic Case Report Forms (eCRF).

Because data are collected remotely and online, a crucial question that inevitably arises is: How to validate and finally “lock” the clinical database after study closure? A general data audit is the most common practice to ensure that data are validated and safely moved to the analysis stage. A full audit of a large scale clinical database would, however, require resources that are beyond the means of most international organizations and is rarely performed in the context of these studies. One common practice is to assess a small proportion of the data (usually 10%); however, this methodology has been shown to be problematic also. The random data audited might suffer from over or under sampling (depending on the scale of the trial) or might not target the critical issues that the study is trying to address. ²

An alternative approach is to focus the audit on those critical data points that are sensible enough to show data variability. This can be done by identifying data both at a patient level (single variables such as “ASA score” or “Laboratory values”), but also at center level (overall patient complication rates) that appear out of the norm and/or suspicious. This approach does not solve all the problems, though, because it is only concerned with verification of online data via selected queries. Data source verification (original transcripts or copies or source documents) is another issue that should be carefully investigated when performing a quality audit.

The CROES data audit combines these last two approaches in one procedure that aims at verifying both online data and source data in one single procedure.

Audit process

Each center participating in the study was audited and received an audit form with a list of 10 queries (Appendix 1). Each query targeted a number of patients and a set of critical variables such as American Society of Anesthesiologists score, transrectal ultrasonography volume, and prostate-specific antigen value. Data points were selected according to their status of outliers, out of range, or otherwise suspicious data. Each principal investigator was asked to confirm the data points for each patient in the audit form. In total, more than 500 individual data points were audited.

Each center was also asked to provide two copies of original source data for two variables: Laboratory values for preoperative creatinine and operating room report for surgery time (Appendix 2). Each principal investigator was asked to anonymize the documents, scan them, and mail them to the CROES Office attached to the center query form. In summary, each center was asked to return the audit query form and provide two copies of original documentation.

General outcomes

All but three centers returned the audit form, which represents a 90% return rate. The centers that did not return the form had problems with their individual institutions in accessing the source data (ie, they could not get a copy of the formal Operative Report). A similar problem was encountered by a few centers that could not provide copies of original source data because of local restrictions and/or country privacy policies.

More than 90% of the data was successfully validated. Mismatches were found in a minority of cases and were usually because of human error (mistyping) and were amended by the principal investigators. A few cases will have to be resolved by contacting the investigators again and inquiring in more depth about the data provided. The CROES Audit Committee ³ will oversee the process and ensure a timely reconciliation of the data.

General remarks and further developments

Auditing a large-scale clinical database is an enterprise that requires vast resources and an established methodology. Given the possibilities that the CROES has and the extent of the CROES GreenLight Study (28 centers globally, more than 1000 patients in the database), this first clinical data audit has proven to be a rather effective way of validating the data. Ninety percent of centers gave their full support, with only a minority having issues with the audit process, mostly because of local institutional restrictions. More than 90% of the data audited was validated and proven to be reliable. Copies of original documents are the most compelling and unequivocal evidence of the validity of the data. The occasional mismatches found will have to be reconciled by contacting the local principal investigators again to understand the nature of the mismatch. This task will involve an active role of the audit committee and a second check from the clinical data manager to resolve the mismatches and close the case.

As it has been designed, the CROES Quality Audit can be effectively applied to the other clinical trials currently managed, in much the same format. This first CROES experience in data auditing has shown that it is possible to provide a high level of confidence in the data even when that is collected globally and remotely, with limited resources available, and that the levels of control implemented are adequate enough to detect problems in the dataset itself that can then be timely resolved.