The Impact of Virtual Site Visits on Programmatic Accreditation Through the Commission on Accreditation of Allied Health Education Programs (CAAHEP)

Abstract

Objective:

This research was designed to investigate the effectiveness of programmatic accreditation, when performing a virtual site visit, compared with a traditional on-site visit.

Materials and Methods:

This study used a quasi-experimental, mixed-methods design with the virtual site visits as the independent variable and the site visit outcomes (number of citations, level of the citation, and recommendation for continuing accreditation) as the dependent variable. For this study, there were 28 diagnostic medical sonography programs throughout the United States, applying to the Commission on Accreditation of Allied Health Education Programs (CAAHEP), for reaccreditation status. Those programs that experienced a virtual site visit were compared with their counterpart programs (N = 109), which had a traditional on-site visit.

Results:

Based on these comparisons, there was no significant difference in diagnostic medical sonography programmatic accreditation effectiveness in the United States when performing a virtual site visit.

Conclusion:

These study results would highly suggest that there was no significant difference in the level of the citations or the accreditation outcomes, based on a virtual site visit compared with those programs having an on-site visit. This research points to the effectiveness of the accreditation process, when is it conducted through a virtual site visit.

Keywords

Commission on Accreditation of Allied Health Education Programs (CAAHEP)virtual site visit on-site visit programmatic accreditation education diagnostic medical sonography

While programmatic and institutional accreditation remains the vital components of a quality education, the associated costs of a site visit can be excessive, especially when the technology is available to accomplish this process virtually. In the wake of the SARS-COVID-2 (COVID-19) global pandemic, it also became evident that a virtual site visit was the only viable means to accredit diagnostic medical sonography (DMS) programs, given safety concerns for site visitors. Based on a virtual site visit pilot project, conducted by The Joint Review Committee on Education in Diagnostic Medical Sonography (JRC-DMS) in 2014, accrediting bodies and academic institutions were encouraged to consider a virtual site visit.

In an era of cutting costs and maximizing efficiency, while still upholding the integrity of the programmatic accreditation, there was a need to maintain the effectiveness and efficiency of the accrediting process, utilizing a virtual review. The Commission on Accreditation of Allied Health Education Programs (CAAHEP) is a programmatic postsecondary accrediting agency recognized by the Council for Higher Education Accreditation (CHEA). This commission carries out its accrediting activities in cooperation with 25 review committees on accreditation.¹ At this time, CAAHEP accredits more than 2200 entry-level education programs, in 32 health science professions.¹ CAAHEP’s vision is to be the innovative leader for the accreditation of health sciences education.¹ CAAHEP’s mission is to provide quality assurance in education in the health sciences and health professions, by setting standards for program accreditation and evaluating program performance.¹ One of the committees on accreditation, under CAAHEP, is JRC-DMS. Since 1983, the organization has provided a mechanism for committee review to recognize educational programs, throughout the United States, that instruct students in the disciplines related to DMS through education consistent with standards for entry into practice.²

While CAAHEP programmatic accreditation remains a necessary aspect of providing quality education in DMS, the associated costs can be excessive, especially considering budget cuts for higher education. Since the JRC-DMS continuously evaluates its process for recommending programs for continuing accreditation to CAAHEP, it was proposed in late 2014 that a subcommittee be appointed to develop a virtual site visit procedure that could be accomplished remotely in contrast to the conventional on-site visit. The committee examined how existing communication technologies could be used to potentially decrease accreditation costs, without sacrificing the integrity of the process.

Although there is an abundance of research available on the validity and the essential components of programmatic and institutional accreditation, there was a scarcity of information found in the literature on using the virtual site visit method. Quality assurance, more rigorous accreditation standards, and improvement in higher education continue to present institutions with a dynamic and challenging environment. As McEwan and Walsh concluded,³ a virtual accreditation has the potential to transform the process of external accreditation, making it less stressful, more convenient, less costly in terms of time and money, but equally rigorous and effective. The research indicated the possibility of implementing technology enhanced accreditation processes that are equally effective and efficient, but far less expensive than the traditional on-site visits.³ The Public Health Accreditation Board (PHAB) released a report in their Guide to National Public Health Reaccreditation in 2016, which highlights their efforts of cost containment in the reaccreditation process by conducting their site visits virtually by using meeting video software.⁴

Further research in this area was necessary to uncover the impact of virtual site visits on programmatic accreditation of DMS educational programs through the CAAHEP. The specific problem was to determine whether the consistency of an on-site visit was maintained when conducted virtually. There has not been previous research found to analyze this specific topic; therefore, data were needed to serve as a foundation for other inquiries in the future.

The objective of this study was to examine the effectiveness of the accreditation process, using the features of an online video conferencing and screen sharing tools, which could transform the on-site accreditation process.

Materials and Methods

This study began in November 2014 (pre-COVID), as a pilot study, to evaluate a virtual site visit, for a single learning concentration program, at a Midwestern university. This pilot was declared a success and virtual site visits, for educational programs, expanded over the next few years. These virtual site visits include all learning concentrations (e.g., general sonography, adult cardiac sonography, and vascular technology) along with programs offering a multiconcentration educational program (i.e., combinations of any of the listed learning concentrations).

This research was based on the following three quantitative questions and a fourth qualitative question:

Research Question 1: How does the effectiveness of virtual site visits for programmatic accreditation in DMS education in the United States differ from traditional on-site visits?

Research Question 2: How does the level of the citations affecting programs having a virtual site visit for programmatic accreditation in DMS education differ from programs having a traditional on-site visit?

Research Question 3: Will virtual site visits impact the outcomes for programmatic accreditation?

Research Question 4: How do program directors feel about having a virtual site visit?

Institutional Board Review approval was obtained by the University of St. Francis for this research study which was titled, “The Impact of Virtual Site Visits on Programmatic Accreditation through CAAHEP.”⁵

The study followed a mixed-method, quasi-experimental design, by identifying the virtual site visits as the independent variable and the site visit outcomes, including the number of citations, table for additional information, and recommendation for continuing accreditation, as the dependent variable. The convenience sample placed 28 DMS educational programs in a treatment group, due to their willingness to host a virtual site visit, at their institution. This group also met the established criteria set by the JRC-DMS board of directors, governing such visits. Data from the control group of 109 programs that had the traditional on-site visit starting in November 2014 also included all the aforementioned learning concentrations, along with programs offering multilearning concentrations, which was a combination of those listed. The award granted by the sponsoring institutions included programs offering a certificate, diploma, associate, or baccalaureate degree.

By comparing the number of citations, which included their hierarchical level in a program undergoing a traditional site visit to one conducted virtually, accreditors could be better able to understand the accountability of the virtual site visit. For this study, the possible citations a program could receive, at the site visit, were tiered into three categories: level 1 (low-level), level 2 (mid-level), and level 3 (high-level). The levels placed the citations within a hierarchical system depending on the significance of the citation issued during the virtual and on-site visits:

Level 1 (low-level)—citations based on noncompliance with the standards that may take the program less than 30 days to correct. Reference: Standard II. Program Goals, B. Appropriateness of the Goals and Learning Domains.

Level 2 (mid-level)—citations based on noncompliance with the standards that may take the program more than 30 days to correct. Reference: Standard III. Resources, B. Personnel, 5. Faculty and Instructional Staff, a. Didactic Instructors, 2) Qualifications (e.g., A didactic instructor is not appropriately credentialed for the course he/she is teaching).

Level 3 (high-level)—citations based on noncompliance with the standards that may take the program a significant amount of time to correct. Reference: Standard IV. Student and Graduate (Outcomes), Evaluation/Assessment, B. Outcomes, 1. Outcomes Assessment (e.g., A program exceeded the outcomes assessment threshold of 20% for programmatic attrition for the past two graduating classes).

The convenient sample for this study comprised CAAHEP-accredited DMS programs seeking continuing accreditation in one or more of the learning concentrations offered by the program. This sample also included programs seeking a 10-year accreditation cycle. This study had a control group of programs that had the established on-site visit, and a treatment group for which programs were selected for the virtual site visit. The following criteria were used as inclusion criteria for the treatment group:

Meets the established JRC-DMS threshold outcomes on their annual report.

Is not on probation status.

Is not on reactivation status.

Does not have a formal complaint submitted through CAAHEP.

The program currently uses a clinical management system (e.g., Trajecsys).

The program currently uses a learning management platform (e.g., Canvas, Blackboard).

Program faculty are technologically savvy and have the support of an on-site information technology (IT) department, during the virtual site visit.

The treatment group contained 28 DMS programs. The control group consisted of 109 DMS programs. The programs in the control group, undergoing the on-site visit for continuing accreditation, were not subject to additional criteria, which the treatment group had met. An additional exclusion criteria for this study were any programs seeking initial accreditation.

The anticipated data to be collected came from the educational program’s site visit report that had been completed by the site visit team, evaluating the program. In addition, data were gleaned from the reviewer’s assessment and the letter of findings, sent to the program following the comprehensive review for continuing accreditation. The programs were evaluated using the 2011 Standards and Guidelines for the Accreditation of Educational Programs in Diagnostic Medical Sonography.⁶

Anticipated Statistical Analysis

The first three research questions required quantitative data to determine whether any significant differences existed. The data were processed with IBM SPSS Statistics v 19.0, statistical software.⁷ The fourth research question needed a qualitative data analytics to explore the open-ended responses in a constant comparative approach to develop common themes.

Results

The first research question was: How does the effectiveness of having a virtual site visit for programmatic accreditation in DMS education in the United States differ from traditional on-site visits? The hypothesis posed was that there will be no difference in DMS programmatic accreditation effectiveness in the United States when performing a virtual site visit.

Descriptive crosstabs were used to compare the frequencies of citations among programs having a virtual site visit compared with programs having an on-site visit. Likewise, crosstabs were run for on-site and virtual site visits based on the number of citations identified at the site visit. The total number of citations issued during the site visits for programs seeking continuing accreditation was 530, with 90 citations given to programs experiencing a virtual site visit.

The one-way analysis of variance (ANOVA) determined whether there was a statistically significant difference when comparing the number of concentrations for programs either having an on-site visit or the virtual site visit. Tukey’s honest significant difference test was used to detect a significant difference between the on-site and virtual visits by the concentration(s) offered by the program based on the multiple nominally coded variations of the concentrations. In Table 1, there were eight (16.5%) of the programs that had an on-site visit and received one citation, in contrast to eight (28.6%) of the programs undergoing a virtual site visit and having two citations (see Table 2).

Table 1.

Descriptive Statistics for On-site and Virtual Site Visits, Based on the Number of Citations (Control Group).

Control Group1 = On-site		Frequency	%	Valid Percent	Cumulative Percent
.		2	100.0
1 = Control Group On-site		7	6.4	6.4	6.4
		18	16.5	16.5	22.9
		15	13.8	13.8	36.7
		16	14.7	14.7	51.4
		16	14.7	14.7	66.1
		10	9.2	9.2	75.2
		11	10.1	10.1	85.3
		2	1.8	1.8	87.2
		3	2.8	2.8	89.9
		5	4.6	4.6	94.5
		2	1.8	1.8	96.3
		1	.9	.9	97.2
		2	1.8	1.8	99.1
		1	.9	.9	100.0
	Total	109	100.0	100.0

Table 2.

Descriptive Statistics for On-site and Virtual Site Visits by the Number of Citations (Treatment Group).

Treatment Group		Frequency	%	Valid Percent	Cumulative Percent
2 = Treatment Group Virtual	Total	2	7.1	7.1	7.1
		3	10.7	10.7	17.9
		8	28.6	28.6	46.4
		5	17.9	17.9	64.3
		3	10.7	10.7	75.0
		3	10.7	10.7	85.7
		1	3.6	3.6	89.3
		1	3.6	3.6	92.9
		2	7.1	7.1	100.0
		28	100.0	100.0

As shown in Tables 1 and 2, a frequency test identified how often the number of citations occurred for the control and treatment groups during the site visits, along with the percent of the occurrences. A total of 109 programs had an on-site visit for their continuing accreditation and served as the control group. The treatment group comprised 28 programs having a virtual site visit for their continuing accreditation. Seven programs in the control group did not have any citation compared with the two programs in the treatment group. The overall frequency of citations for the control group decreased after six citations compared with the frequency of four citations given to the treatment group. One control program received the highest number of citations at 19 compared with two treatment programs each receiving eight citations.

The results of an independent sample test group were conducted on a sample of 137 DMS programs undergoing a continuing site visit for reaccreditation using the 2011 CAAHEP Standards and Guidelines for the Accreditation of Educational Programs in Diagnostic Medical Sonography.⁶ The control group consisted of 109 DMS programs experiencing the traditional on-site visit compared with the treatment group of 28 programs undertaking the virtual site visit approach. The on-site group had a mean of 4.04 and a standard deviation of 3.3 when compared with the virtual site group with a mean of 3.21 and a standard deviation of 2.16 to measure the significance of the impact of programs having a virtual site visit in comparison with programs having an on-site visit for the number of citations per program.

The independent samples t test was conducted on a sample of 137 sonography programs experiencing either a traditional site visit (109 programs) or a virtual site visit (28 programs) to determine whether there was a statistically significant difference in the number of citations received for their reaccreditation through the JRC-DMS. There was no significant difference since the significant two-tailed P values are greater than .05. The t test results yielded a value of 1.247 (equal variances assumed) and 1.589 (equal variances not assumed) with 135 degrees of freedom (equal variances assumed) and 63.35 (equal variances not assumed) with a P = .215 (equal variances assumed) and .117 (equal variances not assumed).

Consulting Table 3, it shows the comparison between groups as df = 5, mean square = .296, and F = 1.863. The comparison within the groups shows the df = 131 and the mean square at .159. These scores demonstrate that there was no significant difference at the P ≤ .05 level (P = .105).

Table 3.

One-Way Analysis of Variance (ANOVA) for On-site and Virtual Visits by the Types of Learning Concentrations.

GroupsControl = 1—On-site; Treatment = 2—Virtual
	Sum of Squares	df	Mean Square	F	Significance
Between Groups	1.479	5	.296	1.863	.105
Within Groups	20.799	131	.159
Total	22.277	136

Tukey’s honest significant difference analysis noted whether there was a significant difference between the on-site and virtual visits by the concentration(s) offered by the program (see Table 4). There was no significant difference as the sig. mean values for the on-site and virtual visits are greater than .05. In this table description, the nominally coded areas were as follows: General = 1, Cardiac = 2, Vascular = 3, General/Vascular = 4, General/Cardiac = 5, General/Cardiac/Vascular = 7.

Table 4.

Tukey’s Honest Significant Difference Test for On-site and Virtual Visits, by Coded Learning Concentrations.

Coded Concentrations	N	1 = On-site; Means	2 = Virtual; Means
5 = General/Cardiac	17	1.12	1.15
1 = General	72	1.15	1.20
4 = General/Vascular	15	1.20	1.32
7 = General/Cardiac/Vascular	22	1.32	1.38
2 = Cardiac	8	1.38	1.67
3 = Vascular	3	.730	.066
Sig. P ≤ .05

No programs offered a Cardiac/Vascular combination. The number of programs offering each type of concentration were as follows: 72 General, 22 General/Vascular/Cardiac, 17 General/Cardiac, 15 General/Vascular, 8 Cardiac, 3 Vascular. The means for on-site visits ranged from 1.12 to 1.38 and the means for virtual visits ranged from 1.15 to 1.67.

Tukey’s honest significant difference showed no significant differences between the control/on-site visits and the treatment/virtual site visits based on the multiple coded variations of concentrations in the programs included in the visits (see Table 5). This analysis adds additional corroboration for accepting the null hypothesis stating that there were no significant differences between on-site and virtual visits for any part of the data.

Table 5.

Tukey’s Honest Significant Difference Test for Multiple Comparisons of On-site and Virtual Visits, by Coded Learning Concentrations.

Dependent Variable: Group Control/Treatment: 1 = On-site; 2 = Virtual
Tukey HSD
(I) Coded Concentrations	(J) Coded Concentrations	Mean Difference (I-J)	Std. Error	Sig.	95% Confidence Interval
(I) Coded Concentrations	(J) Coded Concentrations	Mean Difference (I-J)	Std. Error	Sig.	Lower Bound	Upper Bound
1	2	−.222	.148	.667	−.65	.21
	3	−.514	.235	.250	−1.19	.17
	4	−.047	.113	.998	−.37	.28
	5	.035	.107	.999	−.28	.35
	7	−.165	.097	.532	−.45	.12

Abbreviation: HSD, honest significant difference.

The second research question was: How does the level of the citations affecting programs having a virtual site visit for programmatic accreditation in DMS education differ from programs having a traditional on-site visit? The hypothesis posed was whether there would be no difference in the level of the citations (level 1 = low-level; level 2 = mid-level; level 3 = high-level) a program will receive when having a virtual site visit in comparison with programs having an on-site visit.

Utilizing Pearson’s correlation coefficient allowed for determining whether there was a positive or negative direction of a relationship between the levels of citations and the on-site of virtual site visits based on the following: Groups (1 = Control/On-site Visits; 2 = Treatment/Virtual Visits); Level of Citations (Level 1 = low-level; Level 2 = mid-level; Level 3 = high-level); Programs with 1, 2, or 3 concentrations.

Table 6 shows the use of Pearson’s correlation coefficient as a means to assess the relationship between levels of the citations between the programs having an on-site visit and programs having a virtual site visit. There was a correlation between the two variables r = .978, n = 3, with no significant differences since P = .134 as measured against P ≤ .05. A scatterplot summarizes the results (see Figure 1). Overall, there was a strong, positive curvilinear correlation between the level of citations and programs having either an on-site or virtual site visit. Increases in the number of citations for each level (level 1, level 2, level 3) for programs having an on-site visit were correlated with increases in all levels for programs undergoing the virtual site visit. Level 2 (mid-level) citations (220) for programs experiencing the on-site visit had the highest number of citations which correlated with the highest number of citations noted for level 2 (mid-level) citations for the programs having a virtual site visit.

Table 6.

Pearson’s Correlation Coefficient Used to Determine the Strength of Association Between Levels of Citations for On-site Compared With Virtual Site Visits.

	On-site Visit	Virtual Visit
On-site Visit
Pearson Correlation	1	.978
Sig. (two-tailed)		.134
N	3	3
Virtual Visit
Pearson Correlation	.978	1
Sig. (two-tailed)	.134
N	3	3

Figure 1.

The scatterplot of Pearson’s correlation coefficients for the levels of citations for programs with an on-site visit in comparison with programs experiencing a virtual site visit.

Table 7 illustrates the CAAHEP Standards for the Accreditation of Educational Programs in Diagnostic Medical Sonography that are identified within a hierarchical system by this researcher. The tiered citations are placed into three categories (level 1 = low-level; level 2 = mid-level; level 3 = high-level) depending on the timeframe to address the citation issued during the virtual and on-site visits. Level 1 (low-level) citations are based on noncompliance with the Standards that may take the program less than 30 days to correct, while level 2 (mid-level) citations may take the program more than 30 days to correct. Level 3 citations are based on noncompliance with the Standards that may take the program a significant amount of time to correct.

Table 7.

The CAAHEP Standards for the Number of Citations Per Level of Citations (Level 1 = Low-Level; Level 2 = Mid-Level; Level 3 = High-Level) for On-site vs Virtual Site Visits.

CAAHEPStandard Levels	On-site VisitNumber of Citations	Virtual Site VisitNumber of Citations
Level 1 (low)
II. Program Goals	61	5
V. Fair Practices
V.A.2—V.A.3. Accreditation status, College & Program Policies	21	8
Totals—Level 1	82	13
Level 2 (mid)
III. C. Curriculum	125	26
IV. Student and Graduate (Outcomes) Evaluation/Assessment	40	6
V. Fair Practices
V.A.- V.A.1 Publications and Disclosures, Announcements & Publications	12	6
V.B.—V.F. Lawful and Non-discriminatory Practices, Safeguards, Student Records, Substantive Change, Agreements	43	17
Totals—Level 2 (mid)	220	55
Level 3 (high)
I. Sponsorship	0	0
III. Resources	99	20
III.D. Resource Assessment	9	0
IV.B Outcomes
IV.B.1.—IV.B.2. Outcomes Assessment & Outcomes Reporting	26	2
V.A.4. Publications—outcomes assessment	4	0
Totals—Level 3 (high)	138	22
Totals—Levels 1, 2, 3	440	90

Abbreviation: CAAHEP, Commission on Accreditation of Allied Health Education Programs.

In Table 8, the combined numerical breakdown for the on-site and virtual site visits is provided. The overview compares the number of programs, number of concentrations, number of concentrations per program, the total number of citations, and their levels for on-site and virtual site visits.

Table 8.

Combined Values for On-site and Virtual Site Visits.

Site Visits	On-site	Virtual
Number of programs	109	28
Number of concentrations	166	47
Number of general concentrations	102	23
Number of cardiac concentrations	36	12
Number of vascular concentrations	28	12
Number of programs with one concentration	67	16
Number of programs with two concentrations	27	5
Number of programs with three concentrations	15	7
Number of programs with no citations	7	2
Total number of citations	440	90
Level 1 (low-level) citations	82	13
Level 2 (mid-level) citations	220	55
Level 3 (high-level) citations	138	22

The third research question was: Will virtual site visits impact the outcomes for programmatic accreditation? The hypothesis was that there would be no difference in the JRC-DMS recommendation to CAAHEP to accredit programs having a virtual site visit for programmatic accreditation in comparison with programs that have an on-site visit.

To determine whether programs having a virtual site visit will be impacted for recommendation to CAAHEP for their continuing accreditation, this researcher examined the JRC-DMS board of directors’ recommendation to CAAHEP for continuing accreditation and the review of the CAAHEP Award Letters for continuing accreditation of the 28 programs having a virtual site visit. As a result of this study’s investigation, all 28 programs were recommended to CAAHEP for continuing accreditation, and all were awarded continuing accreditation through CAAHEP.

The final research question was: How do program directors feel about having a virtual site visit? To answer this question, it was important to review the JRC-DMS Site Visit Questionnaire. This questionnaire is provided after each site visit and program directors were asked to fill out an electronic version. The questions focused on the virtual site visit arrangements, the site visit team’s performance, and the accreditation process. Respondents were also invited to share their ideas for improving the accreditation process. The return of 23 questionnaires out of 28 programs that experienced the virtual site visit process was the basis for these findings. The response provided by the program directors is confidential and not seen by the site visit team. Post-site visit questionnaires are reviewed quarterly by the JRC-DMS chair-elect and reported to the board. Consult the figures provided for the qualitative, constant comparative, thematic approach to the four open-ended questions posed on the JRC-DMS Site Visit Questionnaire. Program directors were surveyed to seek comments regarding arrangements for the virtual site visit, the site visit team’s performance, and the accreditation process. They were also invited to share ideas for improving the accreditation process. Some respondents did not answer every question on the questionnaire.

In the comparison of the responses, the theme for this question, as shown in Figure 2, related to the arrangements of the virtual site visit which resulted in a very positive experience for the program directors.

Figure 2.

The qualitative constant comparative thematic approach to responses of the arrangements for the virtual site visit.

In a comparison of the significant findings regarding the virtual site visit team’s performance, as shown in Figure 3, the themes implied that the site visitors were very professional, well-prepared, knowledgeable, and organized.

Figure 3.

Qualitative constant comparative thematic approach to the responses for the performance of the virtual site visit team.

The major theme noted regarding the accreditation process, as shown in Figure 4, revealed the virtual site visit worked very well for the programs and was less expensive for the institution.

Figure 4.

The qualitative constant comparative thematic approach to the responses to the accreditation process.

The overarching theme shared by the program directors, as shown in Figure 5, suggested the virtual site visit should be an option for programs because the process works well. Some directors remarked that engagement and interpersonal communication are sometimes excluded in the virtual setting.

Figure 5.

The qualitative constant comparative thematic approach to the responses for improving the accreditation process.

The virtual site visit became a reality for many DMS programs during the pandemic, thus allowing for programs seeking initial accreditation, initial to continuing accreditation, and reaccreditation to experience a virtual visit. Because of the virtual site visit, the timeline from submission of the self-study to a virtual visit was shortened but still provided an effective assessment.

Discussion

Based on feedback The Joint Commission received from both customers and surveyors who have been involved in the virtual survey process, the belief was that they could effectively provide a meaningful assessment virtually.⁸ This statement aligns with current research analysis of the data which supports the virtual site visit’s effectiveness for CAAHEP accreditation, for educational programs in DMS.

The data analyses confirmed the McEwan and Walsh³ theory that the virtual accreditation site visit is a viable option given the many possibilities of communication technologies available today. This cost-saving alternative has the potential to reshape the rigorous accreditation process while saving educational institutions the expense of the on-site visit which was noted by the program directors. In this study, it was determined there was no significant difference in the level of the citations or the outcomes when having a virtual site visit for programmatic accreditation in comparison with programs having an on-site visit. The thorough accreditation process is maintained when having a virtual site visit.

In March 2020, the Network of Schools of Public Policy, Affairs, and Administration (NASPAA) had to adapt their on-the-ground accreditation site visit to a virtual one due to the public health outbreak.⁹ This Commission on Peer Review & Accreditation (COPRA) took a stance on the virtual site visit as an opportunity for the program to showcase its strengths, the site visit team to confirm and clarify the self-study report, and COPRA to gain insight into each program.⁹

In the wake of COVID-19, not only sonography educational programs seeking accreditation were affected by the pandemic. The Commission for Accreditation of Park and Recreation Agencies (CAPRA) was also faced with conducting their accreditation virtually. The CAPRA Commission received positive feedback from staff at many of the agencies that participated in virtual visits during the COVID-19 pandemic.¹⁰ The collected feedback from everyone involved will ensure the process of accrediting park and recreation agencies will continue to improve in 2021 and beyond.¹⁰

Based on the findings from this study, the virtual site visits for CAAHEP accreditation of educational programs in DMS were just as effective as the traditional on-site visits.

Limitations

As with any study, some factors contribute to limitations discovered after the study was completed. The selection of programs for the virtual site visit had to meet the additional criteria established by the JRC-DMS’ board of directors. This included meeting threshold outcomes on their annual report, not on reactivation, did not have a formal complaint submitted through CAAHEP, and was not on probation status. To ensure a virtual site visit’s success, programs also had incorporated an electronic clinical management system, a learning management system, and program faculty who were technologically savvy. Treatment fidelity was a concern as a limitation; however, this was reduced by the fact that programs not meeting threshold outcomes on their annual reports, programs on reactivation, having or receiving probation status, or had a formal complaint submitted through CAAHEP were excluded from the study.

The limitation of a small sample size, which included programs that had a virtual site visit, was an additional concern at this study’s planning stage. The JRC-DMS Strategic Plan includes increasing the number of virtual site visits performed; therefore, this researcher anticipated an increased sample size for the data analysis. Also, the rapid onset of COVID-19 impacted the ability of the JRC-DMS to conduct in-person site visits. The JRC-DMS suspended all on-site accreditation visits on March 16, 2020, indefinitely due to the site visitors’ health and safety concerns. Consequently, campus closures due to the pandemic halted the virtual site visit process until Fall 2020.

As with most studies, the design of the current study is subject to participant exclusion. Programs seeking initial accreditation and initial to continuing accreditation were excluded from the study. Also excluded were programs seeking continuing accreditation but not utilizing a clinical management system or learning management system in their programs.

Conclusion

Based on the findings of this study, this section provides recommendations for future research and practice. A future study of this kind could be enhanced by using a larger sample size than 28 programs that participated in this study. Due to this study being conducted and the ongoing pandemic, the JRC-DMS has instituted a virtual site visit process for programs seeking initial CAAHEP accreditation. Future research could focus on programs applying for initial programmatic accreditation through CAAHEP that meet the technological requirements for a virtual site visit.

A correlational research study could also be done to measure the degree of association between programs experiencing a virtual site visit for initial accreditation and programs undergoing a virtual site visit for continuing accreditation. Investigation could also continue to identify cost savings to accreditation organizations and the programs seeking accreditation when implementing virtual instead of on-site visits.

Although this study’s literature review found no articles that focused on a virtual site visit for programmatic accreditation of educational programs in DMS, these results concluded that the virtual site visit for accreditation was just as effective when compared with the traditional on-site visit.

Footnotes

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

Ethics Approval

Institutional Board Review approval was obtained by the University of St. Francis for this research study which was titled: The Impact of Virtual Site Visits on Programmatic Accreditation through CAAHEP.⁵

ORCID iD

Catherine E. Rienzo

References

Commission on Accreditation: CAAHEP. https://www.caahep.org/About-CAAHEP.aspx. Accessed March 27, 2022.

Company & Mission: JRC. https://www.jrcdms.org/mission.htm. Accessed March 27, 2022.

McEwan

Walsh

Jr.

Virtual Site Visits: The Future of Institutional Accreditation? in: Barton

Hedberg

Suzuki

. (eds): Proceedings of Global Learn Asia Pacific 2011–Global Conference on Learning and Technology. Melbourne, Australia: Association for the Advancement of Computing in Education (AACE). 2011, pp. 605–612.

Guide to reaccreditation final for distribution. https://www.phaboard.org/wpcontent/uploads/Guide-to-Reaccreditation-FINAL-for-distribution.pdf. Accessed March 27, 2022.

Rienzo

: The impact of virtual site visits on programmatic accreditation through the commission on accreditation of allied health education programs (CAAHEP). University of St. Francis; 2021.

Commission on accreditation of allied health … - jrcdms.org. https://www.jrcdms.org/pdf/DMSStandards.pdf. Accessed April 15, 2022.

IBM Products. http://www.ibm.com/products/spssstatistics2019. Published November 9, 2020. Accessed January 27, 2022.

The Joint Commission Pilots Virtual Surveys during pandemic. Health Facilities Management. https://www.hfmmagazine.com/articles/3952-the-joint-commission-pilots-virtual-surveys-during-pandemic. Accessed March 27, 2022.

Virtual Site Visits 2020–NASPAA: https://www.naspaa.org/sites/default/files/docs/2020-03/Advice%20for%20moving%20SV%20online%203.13.pdf. Accessed March 27, 2022.

10.

How CAPRA learned to Love Zoom and maintained continuity in the wake of COVID-19: we are parks and recreation: Parks & recreation magazine. NRPA. https://www.nrpa.org/parks-recreation-magazine/2021/february/how-capra-learned-to-love-zoom-and-maintained-continuity-in-the-wake-of-covid-192/. Accessed March 27, 2022