Statical: Evaluation of an open access biostatistics tool

Abstract

Background

Statistical analysis is crucial in clinical research, but many clinicians lack training or support. StatiCAL, a free and intuitive tool, was created to address this gap without needing coding skills. This study evaluates its effectiveness and usability by analyzing physicians’ performance and perceptions during real-world statistical tasks.

Methods

We conducted a cross-sectional study evaluating StatiCAL’s usability and effectiveness. Surgical physicians analyzed a clinical dataset using StatiCAL after completing a background questionnaire. Usability was measured with the French SUS, and all analyses were independently verified in R.

Results

Fourteen physicians with limited prior experience in statistical analysis participated. All successfully completed the assigned tasks using StatiCAL, with an average completion time of 28 minutes. Interestingly, participants without prior experience completed tasks significantly faster than those with experience (p = 0.044). The average F-SUS score was 85.33, indicating high usability above the standard threshold.

Conclusions

StatiCAL showed high usability and effectiveness, allowing clinicians, regardless of prior experience, to perform accurate statistical analyses efficiently. Its intuitive interface supports broader access to biostatistics, encourages collaboration with specialists, and simplifies research workflows. Further studies with larger, more diverse populations are needed to confirm and expand these findings.

Keywords

medical statistics data analysis statistical software data management validation study

Introduction

In clinical research, and more generally in the field of scientific research, statistics play a crucial role in confirming results.¹ Conducting studies with rigorous statistical methodology is essential for publication.^2–6 However, statistical skills - or access to an expert with these skills - are not widely available.⁷ To address this gap, StatiCAL was developed as: a free, user-friendly statistical software solution and to meet the need for a tool dedicated to researchers and clinicians. The novelty of StatiCAL compared to other statistical software was discussed in our first published work. It enables users to perform various statistical analyses (descriptive, univariate and multivariate), conduct basic data management (variable manipulation, deletion, creation, and data filtering), and retrieve modified datasets along with detailed analysis reports. Moreover, the tool supports multiple file formats⁸ (link to the software: https:// debds. Shiny apps. io/StatiCAL/).

The practicality of such a solution is key, and this aspect need to be evaluated to strengthen the tool relevance. However, we did not find similar usability studies for other dedicated statistical tools (R-Commander, SPSS, Stata, MEPHAS, R++, or EasyMedStat). Therefore, the objective of this work was to conduct a validation study to assess the reliability and effectiveness of our application. This step allows us to test our work under real-life conditions with naïve users who are unfamiliar with the application and do not necessarily have basic programming skills.

Materials and methods

Study design

This investigation employed a cross-sectional study design to evaluate the efficacy of StatiCAL, a novel, open-access, web-based biostatistics tool, among a cohort of physicians.

Tool description

StatiCAL is designed to parallel the functionalities of proprietary biostatistics software, offering a range of statistical analysis options without the need for user registration or fees. Its development in R programming language and architecture has been detailed previously in a first publication.⁸ The tool distinguishes itself by its accessibility and ease of use, aiming to democratize biostatistical analysis for clinicians.

Preliminary testing of the StatiCAL tool was conducted in collaboration with the Department of Statistics and clinician-researchers to ensure its functionality and relevance to clinical practitioners.

Recruitment and participants

Participants were recruited via email invitations sent to residents, fellows, and senior surgeons in oral surgery, maxillofacial surgery, ENT (Ear Nose Throat) specialist, HNS (head and neck surgery), and plastic surgery. All invited individuals agreed to participate

Data collection instruments

Before engaging with the study, participants were provided with a comprehensive description of StatiCAL and access to an online tutorial, which was also available on the tool’s website. This preparatory phase was designed to familiarize participants with the tool and maximize the efficiency of its use during the study.

The study commenced with a 20-items online questionnaire administered via Microsoft Forms® to evaluate participants’ publication history, practical biostatistical knowledge, and programming skills. Questions covered authorship position, number and quality of publications, and formal training in statistics.

Participants then conducted a biostatistical analysis (Supplementary data 1) using the StatiCAL tool on a customized French version of the Head-Neck-CT-Atlas dataset,⁹ tailored to align with the tool’s data organization requirements. The analysis encompassed univariate and multivariate analyses, including survival analysis, with tasks designed to reflect real-world biostatistical challenges.

Participants were asked to complete a comprehensive evaluation of their experience using the tool. This assessment was done through the French version of the F-SUS questionnaire (System Usability Scale),¹⁰ a widely recognized questionnaire designed to measure the perceived ease of use of interactive systems.

Statistical analysis

Analyses were performed within StatiCAL and independently verified using R Studio by an expert in biostatistics. Categorical data were presented as relatives and absolute frequencies. Continuous data were presented with mean, median, min, max and standard deviation. Missing data were presented as absolute number and percentage. Chi² test or Fisher test, in the event of non-compliance with the Chi² application conditions, were used to evaluate the data repartition between groups for categorical variables. T-test or Wilcoxon rank sum test, in the event of non-compliance with the T-test application conditions, were used to evaluate the distribution similarity of continuous variables. Pearson or Spearman coefficient, test, in the event of non-compliance with the Pearson correlation application conditions, were used to measure the dependency between continuous variables. All analyses adhered to a 5% alpha risk, with results presented at a 95% confidence interval using the R version 4.3.1 on a Windows platform.

Ethics and data anonymity

In compliance with French legislation, all participant data were processed anonymously, negating the need for ethical board approval.

Study duration

Between designing the study, recruiting participants, performing the practical work, and analyzing the results, the study lasted 6 months.

Results

Description of the population

As shown Table 1, 14 participants were evaluated, mainly residents (50%). Mean age was 30 and ranged from 23 to 45 years old. ENT/HNS specialist was the most represented speciality (57.1%). Half of the participant had already published as first or last author and 71.4% of those only published less than 5 articles in that position. A majority of the participant had never been listed as co-author in a publication (71.4%). The participants had basic training in statistics for medical studies but only 35.7% had already performed a statistical analysis. Among them BisotatGV and SPSS were the most used statistical analysis tool as 40% of the participants knew them. However, 79.6% of the participants had heard of statistical analysis tool such as R (36.4%), BiostatGV (27.3%) and Excel (45.5%). A total of 6 tools were named. Finally, only 1 participants knew how to code and in a language which didn’t allow for statistical analysis (C).

Table 1.

Participants caracteristics.

Variables	Total	Missing DAta
Total	N= 14 (100%)
Status		0 (0%)
Fellow	2 (14.3%)
Junior doctor	2 (14.3%)
Resident	7 (50%)
Professor	2 (14.3%)
Senior doctor	1 (7.1%)
Age (years) _{mean(sd) median [min-max]}	30.08 (5.8) 29 [23-45]	1 (7.14%)
Speciality		0 (0%)
Oral surgery	1 (7.1%)
Plastic and reconstructive surgery	3 (21.4%)
CMF	2 (14.3%)
ORL => (ENT/HNS specialist)	8 (57.1%)
Time to perform practical work (mins) _{mean(sd) median [min-max]}	28(10.8) 26.5[15-45]	4 (28.57%)
Score F-SUS _{mean(sd) median [min-max]}	85.18 (10.35) 87.5 [65-100]	0 (0%)
First or last author ?		0 (0%)
No	7 (50%)
Yes	7 (50%)
If the participant answered “yes” to the above question:
Number: First or last author		7 (50%)
Less than 5	5 (71.4%)
More than 10	2 (28.6%)
Co-author ?		0 (0%)
No	10 (71.4%)
Yes	4 (28.6%)
If the participant answered “yes” to the above question:
Number: Co-author		10 (71.43%)
Less than 5	1 (25%)
Between 5 and 10	2 (50%)
More than 10	1 (25%)
Already performed statistical analysis ?		0 (0%)
No	9 (64.3%)
Yes	5 (35.7%)
Do you know statistical tools?		0 (0%)
No	3 (21.4%)
Yes	11 (79.6%)
Do you know how to code?		0 (0%)
No	13 (92.9%)
Yes	1 (7.1%)
Can you plot graphics ?		0 (0%)
No	11 (78.6%)
Yes	3 (21.4%)
Can you perform an univaraite analysis?		0 (0%)
No	10 (71.4%)
Yes	4 (28.6%)
Can you perform a mulitvariate analysis?		0 (0%)
No	13 (92.9%)
Yes	1 (7.1%)
Can you perform a survival analysis?		0 (0%)
No	14 (100%)

Statistical programming knowledge

Before performing the statistical analysis, participants answered about their statistical knowledge. Only 21.4% among them knew how to plot graphics. Only few of them could perform univariate and multivariate analysis with proportion of respectively 28.6% (univariate) and 7.1% (multivariate). None of them could perform survival analysis (Table 1). Precision about participant knowledge about statistical tools and coding language was shown in Table 2. Among the participant who did know a statistical software the majority didn’t heard of the most used software: R (36.4%), SPSS (9.1%) and Stata (9.1%). None them mentioned SAS, the most used in the industrial phamaceutic field.

Table 2.

Participants knowledge about statistical or programming tools.

Variables	Total	Missing data
Total	N = 14 (100%)
For participants who answered “Yes” to the question: “Have you ever performed a statistical analysis?”
Use: SPSS		9 (64.29%)
No	3 (60%)
Yes	2 (40%)
Use: BiostatGV		9 (64.29%)
No	3 (60%)
Yes	2 (40%)
Use: Excel		9 (64.29%)
No	4 (80%)
Yes	1 (20%)
For participants who answered “Yes” to the question: “Do you know statistical tools?”
Known: R		3 (21.43%)
No	7 (63.6%)
Yes	4 (36.4%)
Known: Excel		3 (21.43%)
No	6 (54.5%)
Yes	5 (45.5%)
Known: SPSS		3 (21.43%)
No	10 (90.9%)
Yes	1 (9.1%)
Known: BiostatGV		3 (21.43%)
No	8 (72.7%)
Yes	3 (27.3%)
Known: Stata		3 (21.43%)
No	10 (90.9%)
Yes	1 (9.1%)
Known: EeasyMedStat		3 (21.43%)
No	9 (81.8%)
Yes	2 (18.2%)
For participants who answered “Yes” to the question: “Do you know how to code?
Known: C		13 (92.86%)
Yes	1 (100%)

Biostatistical analysis

Among the participants, 14 performed the analysis. All of them answered correctly the questions. The mean duration of the analysis was 28 ± 10.8 minutes with a range of 15 to 45 minutes (Table 1). In comparison, the expert statistician coded the analysis and report of the same work with R in 35 minutes.

Participant which didn’t already performed statistical analysis were faster than the other: 21 ± 8.02 minutes vs 35 ± 9.45 minutes (p = 0.044) (Supplementary data 2).

A tendency was observed in the correlation of age and duration of the analysis: r=0.64 (p = 0.063) (Supplementary data 3). The participant evaluated the tool with the F-SUS questionnaire, mean result was a score of 85.33 with a range of 65 to 100. (Figure 1).

Figure 1.

Position of StatiCAL in the SUS scale. Graph taken from the publication SUS: A Retrospective [10].

Discussion

StatiCAL was designed to simplify statistical analyses for researchers and clinicians. Our study demonstrated that clinicians, with elementary statistical training from medical studies and no programming experience, could perform accurate statistical analyses using the tool.

However, a formal validation of its utility was necessary to assess its broader applicability. The goal of this evaluation study was to test the tool with participants who had diverse levels of experience in statistical analysis, research, and the use of analytical tools. Our study population was highly heterogeneous, ranging from residents to professors. Almost none of participants had experience with statistical analysis programming. The high success rate among participants highlights StatiCAL’s effectiveness, moreover when the time to perform the analysis was faster for the participants than for the expert biostatistician on R. Although the work involved in using StatiCAL and coding in R is not identical, this comparison demonstrates that the usability of StatiCAL enables users to outperform an expert statistician on this specific task. This finding underscores the tool’s potential to bridge the gap between clinicians and complex statistical methodologies. It will also be essential to evaluate StatiCAL in routine research practice. Its efficiency and ease of use could facilitate preliminary analyses, enhance communication with expert biostatisticians, and reduce their workload, ultimately improving overall research workflow.

Interestingly, while half of the participants had experience as first or last authors, very few had been listed as co-authors. This likely reflects the fact that many clinicians engage in research primarily during personal academic projects, such as theses or university dissertations. They are less frequently involved in collaborative research teams. Consequently, investing time in learning statistical programming is often impractical for this population, reinforcing the need for easy-to-use tools like StatiCAL to apply their statistical knowledge effectively.

The F-SUS score further supports the usability of StatiCAL, with a mean score of 85.33 - classified as a rank B - high and the mean above the threshold of 68 which is considered as “above the average”¹¹ demonstrating that the tool is easy to use for every-body. To our knowledge, there is no other statistical tools which have been tested with similar methodology, making a rigorous comparison difficult. We believe that conducting such studies is crucial to confirming the value of these tools,¹² as many articles showcasing dashboards developed in clinical contexts have done, recognizing the importance of such evaluations.^13–22 Mean SUS-score for those 10 tools was 77.6 and ranging from 67.6 to 89.7 showcasing the interest of StatiCAL.

However, this study has several limitations. The sample size was relatively small, which may limit generalizability. Additionally, the structured nature of the task did not fully reflect real-world statistical investigations, as participants were guided throughout the process.

Conclusion

This study highlights the usability and effectiveness of StatiCAL as a statistical tool tailored for clinicians with varying levels of statistical expertise. Our findings indicate that even users with minimal prior experience in statistical analysis were able to complete biostatistical tasks accurately and efficiently. Notably, participants without previous statistical experience performed the analysis faster than those with experience, suggesting that the tool’s intuitive design enhances usability, particularly for younger users familiar with digital tools.

StatiCAL provides a valuable solution for clinicians, enabling them to perform statistical analyses autonomously without requiring extensive prior knowledge in programming. By simplifying access to statistical analysis, we believe StatiCAL has the potential to enhance clinical research autonomy, promote collaboration between clinicians and statisticians, and ultimately contribute to more rigorous and accessible data analysis practices. However, taking into account the small sample size, future studies with larger and more diverse cohorts are needed to further validate its effectiveness, explore additional usability factors and conclusion of this article.

Supplemental material

Supplemental material - Statical: Evaluation of an open access biostatistics tool

Supplemental material for Statical: Evaluation of an open access biostatistics tool by Tanguy Pace-Loscos, Jocelyn Gal, Sara Contu, Renaud Schiappa, Emmanuel Chamorey, and Dorian Culié in Health Informatics Journal.

Supplemental material

Supplemental material - Statical: Evaluation of an open access biostatistics tool

Supplemental material

Supplemental material - Statical: Evaluation of an open access biostatistics tool

Footnotes

ORCID iD

Tanguy Pace-Loscos

Ethical considerations

All procedures involving human participants in this study were conducted in accordance with the ethical standards of the institutional research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. In accordance with French legislation, this study did not require formal approval from an ethics committee, as all data were collected anonymously and no intervention or collection of sensitive personal data was involved.

Consent to participate

Informed consent was obtained from all participants prior to their inclusion in the study, through their agreement to participate in the practical work, with explicit information that the data could be used for the current publication.

Author contributions

D.C. and T.P.L. conceived the projects. D.C. designed the practical work. T.P.L. realised the statistical analysis. D.C. and T.P.L. drafted the manuscript that was revised by J.G., S.C., E.C. and R.S. All authors read and approved the final manuscript. All authors take responsibility for the accuracy and integrity of the work.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The authors declare that they have no competing interests.

Data Availability Statement

The datasets generated and analyzed during the current study include: (i) a customized version of the Head-Neck-CT-Atlas dataset adapted for the practical statistical tasks using the StatiCAL tool, (ii) the results of the practical analyses performed by the participants, (iii) responses to the F-SUS usability questionnaire evaluating the tool, and (iv) questionnaire data describing participant characteristics, including statistical background and publication history. The practical analysis dataset was generated by modifying the original Head-Neck-CT-Atlas dataset to align with the data format requirements of StatiCAL, simulating real-world clinical research scenarios. All participant evaluations and characteristics were collected through online surveys administered via Microsoft Forms®. All datasets are available from the corresponding author upon reasonable request.*

Supplemental material

Supplemental material for this article is available online.

Appendix

References

Davidian

Louis

. Why Statistics? Science 2012; 336(6077): 12. https://doi.org/10.1126/science.1218685

New guidelines for statistical reporting. N Engl J Med 2019; 381(16): 1597–1598.

Hess

. A guide to official statistics. Lancet 1930; 216(5585): 588.

Gamble

Krishan

Stocken

, et al. Guidelines for the content of statistical analysis plans in clinical trials. JAMA 2017; 318(23): 2337–2343. https://doi.org/10.1001/jama.2017.18556

Lee

. A collection of statistical methods for precision oncology. JCO Precis Oncol 2019; 3: 1–3. https://doi.org/10.1200/po.19.00308

Livingston

. AMA Manual of Style Committee. Study design and statistics. In: AMA Manual of Style: A Guide for Authors and Editors. Oxford University Press, 2020.

Windish

Huot

Green

. Medicine residents’ understanding of the biostatistics and results in the medical literature. JAMA 2007; 298(9): 1010–1022. https://doi.org/10.1001/jama.298.9.1010

Pace-Loscos

Gal

Contu

, et al. StatiCAL: an interactive tool for statistical analysis of biomedical data and scientific valorization. BMC Bioinformatics 2024; 25(1): 210. https://doi.org/10.1186/s12859-024-05829-z

Grossberg

Mohamed

Mulder

, et al. HNSCC Version 4. The Cancer Imaging Archive 2020; https://doi.org/10.7937/k9/tcia.2020.a8sh-7363. [dataset].

10.

Gronier

Baudet

. Psychometric evaluation of the F-SUS: creation and validation of the French version of the System Usability Scale. Int J Hum Comput Interact 2021; 37(16): 1571–1582. https://doi.org/10.1080/10447318.2021.1898828

11.

Brooke

. SUS: a quick and dirty usability scale. In: Jordan

Thomas

Weerdmeester

, et al. (eds). Usability Evaluation in Industry. : Taylor & Francis, 1996, pp. 189–194.

12.

Almasi

Bahaadinbeigy

Ahmadi

, et al. Usability evaluation of dashboards: a systematic literature review of tools. Biomed Res Int 2023; 2023: 9990933. https://doi.org/10.1155/2023/9990933

13.

De Croon

Klerkx

Duval

. Design and evaluation of an interactive proof-of-concept dashboard for general practitioners. In: Proceedings of the International Conference on Healthcare Informatics, 2015.

14.

Schall

MCJ

Cullen

Pennathur

, et al. Usability evaluation and implementation of a health information technology dashboard of evidence-based quality indicators. Comput Inform Nurs 2017; 35(6): 281–288. https://doi.org/10.1097/CIN.0000000000000325

15.

Dowding

Merrill

Russell

. Using Feedback Intervention Theory to Guide Clinical Dashboard Design, AMIA Annu Symp Proc 2018; 2018: 395–403.

16.

Yoo

Jung

Kim

, et al. A real-time autonomous dashboard for the emergency department: 5-year case study. JMIR Mhealth Uhealth 2018; 6(11): e10666. https://doi.org/10.2196/10666

17.

Dowding

Merrill

Barrón

, et al. Usability evaluation of a dashboard for home care nurses. Comput Inform Nurs 2019; 37(1): 11–19. https://doi.org/10.1097/CIN.0000000000000484

18.

DTY

Vennemeyer

Brown

, et al. Usability testing of an interactive dashboard for surgical quality improvement in a large congenital heart center. Appl Clin Inform 2019; 10(5): 859–869. https://doi.org/10.1055/s-0039-1698466

19.

Richter Lagha

Burningham

Sauer

, et al. Usability testing a potentially inappropriate medication dashboard: a core component of the dashboard development process. Appl Clin Inform 2020; 11(4): 528–534. https://doi.org/10.1055/s-0040-1714693

20.

Roa Romero

Tame

Holzhausen

, et al. Design and usability testing of an in-house developed performance feedback tool for medical students. BMC Med Educ 2021; 21(1): 354. https://doi.org/10.1186/s12909-021-02788-4

21.

Xiao

Simon

Wityk Martin

, et al. Advance care planning dashboard: quality indicators and usability testing. BMJ Support Palliat Care 2024; 14: e462-e465. https://doi.org/10.1136/bmjspcare-2021-003071

22.

Khanbhai

Symons

Flott

, et al. Enriching the value of patient experience feedback: web-based dashboard development using co-design and heuristic evaluation. JMIR Hum Factors 2022; 9(1): e27887. https://doi.org/10.2196/27887

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.42 MB

0.00 MB

0.21 MB

0.09 MB