Content Validity of the National Board Dental Hygiene Examination

Abstract

This article describes a protocol that was employed to validate the content of the National Board Dental Hygiene Examination (NBDHE) using two separate practice analyses. The protocol consisted of the following phases: (1) the careful definition of the domain of knowledge and skills required for successful entry-level dental hygiene practice, (2) the conduct of a survey to gather information regarding the judgments of practicing dental hygienists on the important knowledge, abilities, and skills required of entry-level dental hygiene practice, (3) the analysis of the survey data, (4) the integration of the survey findings with the examination content using a two-dimensional matrix, (5) development of the updated specifications of the examination based on the practice analysis data, and (6) implementation of the updated specifications for subsequent item and test development. Survey data from 1,284 and 1,388 full-time practicing dental hygienists for the 2002 and 2009 practice analyses, respectively, were analyzed. A two-dimensional matrix was used to map the content of the examination to the practice of dental hygiene based on the practice analysis data. The mapping showed that the distribution of items in the content specifications reflected actual dental hygiene practice. Thus, the adequacy of the content validity of the NBDHE was confirmed.

Keywords

validity practice analysis National Board Dental Hygiene Examination practice analysis survey Rasch model

The Standards for Educational and Psychological Testing (American Educational Research Association [AERA], American Psychological Association [APA], & the National Council on Measurement in Education [NCME], 1999) as well as various articles published in the literature (Allen & Yen, 1979; Fitzpatrick & Morrison, 1971; Messick, 1993; Millman & Greene, 1993) address the importance of building evidence of validity as the first concern in the development of all examinations. Validity refers to the degree to which logic and evidence support the use of an examination and interpretation of examination results for a specific purpose (AERA, APA, & the NCME, 1999, p. 9). In accordance with the Standards and the guidelines drawn from the literature, evidence to support the validity of the National Board Dental Hygiene Examinations (NBDHE) has been gathered through a variety of research studies. The NBDHE is developed and administered by the Joint Commission on National Dental Examinations. These studies vary significantly in type and scope. As concluded in these studies, however, the NBDHE is a valid assessment instrument for making decisions regarding the qualifications of individuals seeking dental hygiene licensure (Kramer & DeMarais, 1997; Kramer & Neumann, 2004, 2007; Tsai & Leatherman-Dixon, 2013; Yang, Neumann, & Kramer, 2009; Yang, Tsai, Kramer, & Neumann, 2007).

The Joint Commission continues to validate the NBDHE as an integrated evaluation, as suggested by Messick (1993). That is to say, the ongoing validation process ensures that the examination content reflects the most up-to-date knowledge and skills required of individuals seeking to practice dental hygiene. The ongoing validation process also supports the Joint Commission in its ongoing effort to provide a high-quality examination. While different sources of validity evidence are gathered, the focus in the present study is to provide evidence to support the current content validity of the NBDHE. Specifically, this study provides the validation protocol used to develop the content specifications for the examination. Item writers and test developers use the specifications for item and test development.

Although the content of the NBDHE is routinely monitored and refined by experts serving on test construction committees and is overseen by the Joint Commission, additional validation processes involving the empirical analyses of the judgments of practicing dental hygienists were conducted. These dental hygienists determine the critical content elements in the examination that are required for the safe practice of entry-level dental hygiene. This type of empirical analysis is often termed a practice analysis or job task analysis, which has been widely used by health care professions to identify important characteristics of content elements in the specifications of the examinations required of a job or task.

The first practice analysis for the NBDHE was conducted in 2002, and the overall process with the results was published (Kramer & Neumann, 2007). The second practice analysis associated with the NBDHE was conducted in 2009, the results of which have yet to be published. This article provides an overview of the two practice analyses, including the protocol used to develop the valid specifications of the examination based on the outcomes of the practice analyses. The similarities and differences in the overall designs of the two practice analyses as well as their impact on the results are also presented and discussed.

Protocol

Evidence of content validity for the NBDHE can be demonstrated by showing a strong relationship between the examination content and the judgments of practicing dental hygienists on what is critical to their practice (Kramer & Neumann, 2007). To accomplish this objective, a protocol was developed to validate the content of the NBDHE. The protocol includes the following validation phases: (1) the careful definition of the domain of knowledge and skills required for entry-level dental hygiene practice, (2) the conduct of a survey to learn the judgments of practicing dental hygienists on the important knowledge and skills necessary to safely and effectively practice dental hygiene, (3) analysis of the survey data, (4) integration of the survey findings with the examination content using a two-dimensional matrix (Lunz, Stahl, & James, 1989), (5) the development of the up-to-date content specifications for the examination based on the data from the practice analysis, and (6) the implementation of the updated specifications for development of future editions of the examination.

Method

NBDHE

The purpose of the NBDHE is to assist state boards in determining the qualifications of individuals who seek licensure to practice dental hygiene (National Board Dental Hygiene Examination guide, Joint Commission on National Dental Examinations, 2012). The examination assesses the ability to understand important information from basic biomedical, dental, and dental hygiene sciences and also the ability to apply such information in a problem-solving context (National Board Dental Hygiene Examination guide, Joint Commission on National Dental Examinations, 2012). The examination consists of 350 multiple-choice items. The items in the different disciplines are intermingled throughout the examination. Of the 350 items, 200 stand-alone items cover the areas of the scientific basis for dental hygiene practice, provisions for clinical dental hygiene services, and community health/research principles (National Board Dental Hygiene Examination guide, Joint Commission on National Dental Examinations, 2012). The remaining 150 case-dependent items address the knowledge and skills useful in assessing patient characteristics, obtaining and interpreting radiographs, planning and managing dental hygiene care, performing periodontal procedures, using preventive agents, providing supportive treatment services, and professional responsibility (National Board Dental Hygiene Examination guide, Joint Commission on National Dental Examinations, 2012). A typical case consists of a synopsis of a patient’s health and social histories, the patient’s dental charting, radiographs, and photographs of the patient (National Board Dental Hygiene Examination guide, Joint Commission on National Dental Examinations, 2012). A series of items are grouped with each case. To endorse the correct response to the item, the candidate must possess the requisite knowledge and skills to interpret the clinical case materials correctly and then identify the most appropriate alternative among the options provided (National Board Dental Hygiene Examination guide, Joint Commission on National Dental Examinations, 2012).

Design

For the purposes of these two studies, the domain of dental hygiene practice was defined by the Competencies for Entry into the Profession of Dental Hygiene, promulgated by the American Dental Education Association (2001), and supplemented by the Accreditation Standards published by the Commission on Dental Accreditation (1998) of the American Dental Association (ADA). Please note that a more recent set of competencies became available in 2009. This revised set was published in the July 2004 issue of the Journal of Dental Education (American Dental Education Association, 2004). In addition, the Accreditation Standards was revised in 2007, 2009, and 2012 (Commission on Dental Accreditation, 2013).

The Joint Commission deemed these two sources to be sufficiently descriptive of the full domain of dental hygiene. The Joint Commission further defined the purpose of dental hygiene patient care to be Health Promotion and Disease Prevention. The resulting 56 competencies were grouped into three categories: (1) professionalism, (2) patient/client care, and (3) community health involvement.

With the domain of knowledge and skills involved in entry-level dental hygiene practice clarified, the next logical phase was to develop and conduct surveys. In this validation phase, survey questions were developed by gathering demographic information, including general information, personal information, and information on the practice environment. The remaining survey consisted of a listing of the 56 competencies. For each competency, a sample of practicing dental hygienists were asked to rate the importance to patient care on a 5-point rating scale. The levels of the rating scale were defined as follows:

5: Critical to patient care. Without this competency, the resulting patient care would be clearly unacceptable.

4: Important to patient care. Without this competency, the resulting patient care would be compromised.

3: Moderately important to patient care. Without this competency, the resulting patient care would be clinically acceptable but less than ideal.

2: Unimportant to patient care. Without this competency, the resulting patient care would only be slightly affected.

1: Very unimportant to patient care. Without this competency, the resulting patient care would not be affected.

The ratings on the scale represented the levels of uniformly increasing importance. In some instances, a competency may be unrelated to survey participants’ practice of dental hygiene. For these instances, participants circle N/A, for not applicable. The ADA’s Survey Center developed the surveys using the 56 competencies and the questions related to the demographic information. The print format was used for both practice analysis surveys.

A sampling design was developed for both practice analyses. For each practice analysis, a stratified random sample of recently licensed dental hygienists was drawn from the Joint Commission’s application files, so that the number of survey participants from each licensing jurisdiction was proportionate to the number of candidates residing in that jurisdiction at the time of application for the NBDHE. The sample consisted of candidates who passed the NBDHE and graduated from an accredited dental hygiene program. For the 2002 practice analysis, the sample consisted of newly licensed dental hygienists who have been licensed within the last 5 years after passing the examination. For the 2009 practice analysis, the Joint Commission enlarged the sample size by including dental hygienists who have been licensed within the last 10 years after passing the examination. In order to obtain a reasonable number of returned surveys, the baseline percentage of candidates included in the survey per year was set at 10%. In practice, some of the mailing addresses included in the file were outdated. Therefore, the sample size for each year was expanded by increasing the percentage sampled for each year, that is, 1% to 10%. As a result, the numbers of candidates included in the survey were 3,939 (16%) and 7,875 (15%) sampled from the total numbers of candidates for the 2002 and 2009 practice analyses, respectively.

According to the sampling design, the ADA’s Survey Center arranged for the production and distribution of the surveys. A cover letter describing the importance of the practice analysis, along with the survey, was sent. Cover letters for the subsequent mailings of the survey were sent to remind those dental hygienists who had not returned their survey. For the 2002 practice analysis, a total of 1,841 dental hygienists responded to the survey. A final adjusted response rate was 75.6% after excluding unclaimed and undeliverable or new addresses. For the 2009 practice analysis, a total of 2,283 dental hygienists responded to the survey with a final adjusted response rate of 46.5%. Again, the final adjusted rate was computed after excluding unclaimed, undeliverable, or new addresses. The overall design for the 2002 and 2009 practice analyses for NBDHE is presented in Table 1.

Table 1.

The overall design for the NBDHE practice analyses.

	2002	2009
Domain of Entry-Level Dental Hygiene Practice (Competencies)	56 (ADA, 1998; ADEA, 2000)	56 (ADA, 1998; ADEA, 2000)
Survey	Print	Print
Rating Scale	5-point Importance	5-point Importance
Rating Analysis	Lunz et al. (1989)	Lunz et al. (1989)
Calibration	Rasch	Rasch
Sampling	Stratified random sample	Stratified random sample
Number of Dental Hygienists included in the Sample (Rate %)	3,939 (16.0%)	7,875 (15.0%)
Number of Survey Respondents (Final Adjusted Response Rate (%))	1,841 (75.6%)	2,283 (46.5%)

Note. NBDHE = National Board Dental Hygiene Examination.

Survey Data Analysis

Subsequent to distribution and data collection, the survey data were analyzed. For the 2002 practice analysis, of the 1,841 dental hygienists responding to the survey, 1,284 were full-time practitioners. For the 2009 practice analysis, of the 2,283 dental hygienists responding, 1,388 were full-time practitioners. All analyses used the valid survey responses from the full-time practitioners. The use of the data from full-time practitioners in all analyses was deemed appropriate by the Joint Commission because the experiences of full-time practitioners tend to be continuous and more extensive than those of part-time practitioners (Kramer & Neumann, 2007).

Analyses of the ratings involved two steps. First, calibrations were computed to place the importance rating for each competency onto the same scale of measurement through the use of the Rasch model. Ratings were calibrated using WINSTEPS, a Rasch-model computer program (Linacre, 2000/2006). The default value of 0.0 was used as a mean of the Rasch scale. The range of the scale is typically from −3.00 to +3.00. The estimation method is Joint Maximum likelihood estimation with initial starting values provided by the normal approximation algorithm. Rasch calibrations were reported in log-odds units. On this scale, lower values of Rasch calibrations reflected more important competencies, and, conversely, higher values reflected less important competencies. Second, Rasch calibrations for each competency were transformed to the number of items using linear transformation equations (Lunz et al., 1989) based on a 350-item examination. The higher the importance ratings, the greater the number of items assigned to that competency. For the 2002 practice analysis, Kramer and Neumann (2007) described a detailed computational process used with the results of the rating scale analysis. The 2009 practice analysis followed the same steps to analyze the importance ratings from the survey participants.

Results of the Survey Data Analysis

Results of the analyses of the survey data were based on the full-time practitioners responding to the surveys. Findings of the demographic data for both practice analyses showed that both samples were representative of all regions of the United States, including New England, Middle Atlantic, South Atlantic, East South Central, East North Central, West North Central, West South Central, Mountain, and Pacific. The survey ratings were found to be reliable with the reliability coefficients of above 0.90 for both practice analyses.

The number of practicing dental hygienists providing ratings (N), the average ratings ( $\overset{ˉ}{X}$ ), and the associated standard deviations (σ) for both practice analyses are presented in Table 2. As shown in Table 2, average ratings per competency for both practice analyses were very similar. Both practice analyses resulted in the same range in average ratings from a high of above 4.90 (Competency 15—“obtain medical histories” and Competency 28—“adhere to established infection control protocol) to a low of below 3.0 (Competency 8—“participate in community service activities”). It was noted that the average ratings from the 2009 survey participants who had been licensed within 5 years of passing the NBDHE were higher than those from the survey participants who had been licensed between 6 and 10 years on most of the competencies. As indicated at the bottom of Table 2, emboldened elements indicated that groups 2009: 0–5 years and 2009: 6–10 years were statistically significantly different at the .05 level.

Table 2.

Importance ratings by years of experience.

Professionalism	2002			2009			2009			2009
	0-5 years			0-5 years			6-10 years			0-10 years
	N	$\overset{ˉ}{X}$	σ	N	$\overset{ˉ}{X}$	σ	N	$\overset{ˉ}{X}$	σ	N	$\overset{ˉ}{X}$	σ
1. Apply a professional code of ethics to dental hygiene practice.	1,238	4.77	0.49	848	4.84	0.43	511	4.76	0.57	1,383	4.81	0.49
2. Adhere to state and federal laws.	1,237	4.85	0.42	850	4.88	0.44	511	4.84	0.52	1,385	4.86	0.47
3. Assume for responsibility for dental hygiene care.	1,229	4.73	0.52	848	4.81	0.45	509	4.76	0.50	1,381	4.79	0.47
4. Provide dental hygiene services based on accepted standards of care.	1,238	4.74	0.48	850	4.84	0.42	510	4.78	0.50	1,384	4.82	0.45
5. Evaluate scientific literature and other sources of information to make decisions about dental hygiene treatment.	1,234	4.09	0.76	845	4.18	0.77	507	4.12	0.77	1,376	4.16	0.77
6. Perform self-assessment for life-long learning and professional growth.	1,233	4.15	0.86	845	4.26	0.80	510	4.12	0.82	1,379	4.20	0.81
7. Participate in professional organizations.	1,213	3.14	1.16	833	3.23	1.17	496	3.01	1.16	1,352	3.15	1.17
8. Participate in community service activities.	1,163	2.80	1.11	804	2.97	1.19	479	2.84	1.15	1,305	2.92	1.17
9. Provide quality assurance mechanisms for health services.	1,198	4.13	0.85	821	4.23	0.85	479	4.04	0.93	1,321	4.16	0.88
10. Provide care to all clients using an individualized approach that is empathetic and caring.	1,237	4.71	0.55	850	4.76	0.53	509	4.73	0.57	1,383	4.75	0.54
11. Identify career options and practice settings.	1,144	3.30	1.23	796	3.46	1.16	472	3.27	1.19	1,291	3.39	1.18
Patient/Client Care
12. Provide dental hygiene care to promote patient/client health and wellness.	1,234	4.81	0.45	848	4.86	0.40	512	4.81	0.47	1,384	4.84	0.43
13. Provide evidence-based practice using critical thinking and decision-making skills.	1,233	4.58	0.63	849	4.69	0.56	511	4.62	0.60	1,384	4.67	0.57
14. Maintain accurate, consistent and complete records.	1,235	4.80	0.48	846	4.84	0.45	509	4.80	0.51	1,379	4.83	0.47
Collect and Analyze Data to Identify Patient Needs and Oral Health Problems
15. Obtain medical histories.	1,236	4.93	0.29	850	4.93	0.32	511	4.88	0.41	1,385	4.91	0.35
16. Obtain dental, psychosocial, and behavioral histories.	1,234	4.41	0.70	846	4.46	0.72	505	4.46	0.71	1,375	4.46	0.72
17. Perform head and neck examination.	1,226	4.44	0.74	844	4.55	0.72	504	4.55	0.72	1,372	4.55	0.72
18. Perform intra-oral examination.	1,239	4.80	0.47	847	4.83	0.44	512	4.82	0.48	1,383	4.83	0.45
19. Measure and record vital signs.	1,193	3.80	0.93	824	4.05	0.92	492	3.98	0.96	1,339	4.02	0.94
20. Perform periodontal examination.	1,235	4.83	0.45	845	4.86	0.43	509	4.86	0.43	1,378	4.86	0.43
21. Obtain and interpret radiographs.	1,237	4.83	0.44	848	4.85	0.43	508	4.88	0.38	1,380	4.86	0.41
22. Perform dental indices.	1,179	3.91	0.96	820	4.07	0.99	492	4.22	0.91	1,334	4.13	0.97
23. Perform a risk assessment for oral diseases and oral habits.	1,219	4.24	0.83	842	4.37	0.84	504	4.34	0.79	1,370	4.36	0.82
Establish Goals and Treatment Strategies to Facilitate Optimal Oral Health
24. Determine a dental hygiene diagnosis.	1,225	4.64	0.61	844	4.73	0.53	507	4.70	0.55	1,375	4.72	0.54
25. Develop a dental hygiene treatment plan.	1,229	4.66	0.59	848	4.76	0.51	508	4.72	0.56	1,380	4.75	0.53
26. Obtain informed consent.	1,223	4.67	0.67	842	4.79	0.52	504	4.66	0.64	1,369	4.74	0.57
27. Communicate a dental hygiene case presentation.	1,182	4.15	0.98	824	4.25	0.91	494	4.28	0.88	1,341	4.26	0.91
Provide Treatment to Achieve and Maintain Oral Health
28. Adhere to established infection control protocol.	1,241	4.94	0.28	850	4.95	0.28	512	4.92	0.35	1,385	4.94	0.31
29. Implement and monitor environmental safety programs.	1,186	4.34	0.87	828	4.45	0.78	498	4.41	0.77	1,348	4.43	0.78
30. Perform periodontal debridement and scaling procedures.	1,237	4.86	0.40	845	4.90	0.35	500	4.85	0.43	1,368	4.88	0.38
31. Incorporate pain management techniques.	1,226	4.52	0.66	842	4.61	0.60	503	4.54	0.66	1,368	4.58	0.62
32. Apply chemotherapeutic agents.	1,164	4.17	0.79	813	4.29	0.76	472	4.18	0.79	1,304	4.25	0.77
33. Administer fluoride therapy.	1,229	4.35	0.73	846	4.45	0.71	508	4.35	0.72	1,377	4.41	0.72
34. Apply pit and fissure sealants.	1,201	4.34	0.73	821	4.41	0.75	496	4.29	0.80	1,339	4.37	0.77
35. Perform coronal polishing.	1,211	3.83	1.05	836	4.00	0.99	506	3.97	0.97	1,364	3.99	0.99
36. Provide care of oral prostheses.	1,223	4.23	0.79	839	4.33	0.76	501	4.28	0.80	1,363	4.31	0.77
37. Provide care and maintenance of restorations.	1,187	4.23	0.84	821	4.33	0.82	491	4.29	0.83	1,335	4.31	0.83
38. Provide nutritional counseling.	1,197	3.78	0.84	836	3.96	0.92	502	3.95	0.87	1,361	3.95	0.91
Evaluate the extent to which Treatment Goals have been achieved
39. Use and compare indices.	1,148	3.75	0.94	788	3.89	0.98	471	3.95	0.88	1,281	3.91	0.95
40. Re-evaluate oral and periodontal health status.	1,236	4.73	0.53	845	4.75	0.52	507	4.71	0.54	1,375	4.73	0.53
41. Evaluate treatment needs.	1,233	4.69	0.52	846	4.72	0.52	508	4.69	0.54	1,377	4.71	0.53
42. Establish a recall schedule and determine necessary referrals.	1,233	4.75	0.48	847	4.79	0.45	507	4.74	0.53	1,377	4.77	0.48
43. Evaluate patient satisfaction with treatment outcomes.	1,235	4.43	0.68	844	4.46	0.70	504	4.44	0.70	1,371	4.45	0.70
44. Refer patients/clients who may have systemic, psychological, and/or social problem for comprehensive patient/client evaluation.	1,174	4.30	0.82	819	4.44	0.78	488	4.35	0.82	1,329	4.41	0.80
45. Perform basic cardiac life support.	1,160	4.79	0.58	795	4.78	0.56	478	4.76	0.60	1,296	4.77	0.58
Community Health Involvement
46. Promote oral and general health and wellness to the public.	1,169	4.14	0.83	826	4.23	0.85	471	4.10	0.93	1,317	4.18	0.88
47. Communicate effectively both verbally and in writing.	1,197	4.51	0.68	838	4.56	0.67	490	4.49	0.73	1,348	4.53	0.70
48. Assess the oral health needs of the community.	1,075	3.71	0.91	780	3.88	0.92	451	3.77	0.93	1,250	3.84	0.93
49. Identify the availability and quality of community resources and services.	1,077	3.72	0.84	771	3.89	0.90	455	3.76	0.92	1,246	3.84	0.91
50. Develop community-based disease prevention and health maintenance strategies.	1,024	3.68	0.93	743	3.87	0.95	434	3.68	0.97	1,196	3.79	0.97
51. Provide screening, referral, and educational services.	1,080	4.01	0.89	779	4.16	0.88	452	3.97	0.92	1,251	4.09	0.90
52. Provide community oral health services in a variety of settings.	1,013	3.78	0.92	727	3.91	0.95	429	3.73	0.99	1,176	3.84	0.97
53. Evaluate opportunities for improving access to care in a variety of settings.	1,013	3.81	0.91	743	3.98	0.92	426	3.76	1.00	1,189	3.89	0.96
54. Evaluate reimbursement mechanisms and their impact on the patient’s/client’s access to oral health care.	1,024	3.65	0.92	733	3.83	0.92	429	3.64	1.01	1,181	3.75	0.96
55. Evaluate the outcomes of community-based programs and plan for future activities.	982	3.56	0.94	707	3.75	0.95	409	3.54	0.98	1,133	3.66	0.97
56. Utilize professional and social networks and resources to increase access to care.	1,047	3.82	0.92	753	3.93	0.95	439	3.75	1.01	1,211	3.86	0.98

Data used are for those dental hygienists who responded that they are practicing full-time and had a valid response for number of years practicing.

Bolded elements indicate that groups 2009:0-5 years of experience and 2009:6-10 years of experience are statistically significantly different at the 0.05 level.

The numbers of items devoted to each competency for both practice analyses are presented in Table 3. As shown, the numbers were very similar for both practice analyses. Both practice analyses found that the greatest number of items was devoted to the Competency 28—“adhere to established infection control protocol” (13 items for the 2002 practice and 16 items for the 2009 practice analysis), whereas no items were devoted to Competency 8—“participate in community service activities.” The number of examination items devoted to each competency by years of experience for 2009, that is, 0–5 years, 6–10 years, and 0–10 years were computed and compared. The results showed that the numbers were almost identical forthese three groupings. The results along with the statistics in Table 2 were presented to the Joint Commission through its Committee on Research and Development. The results were also examined by independent research consultants. As noted by the Joint Commission and consultants, the number of items assigned to each competency was not impacted by years of experience.

Table 3.

Numbers of items devoted to the competencies based on the 2002 and 2009 practice analysis findings.

Category	Competency	2002	2009
Professionalism	1	9	11
	2	11	13
	3	9	11
	4	9	11
	5	5	4
	6	5	5
	7	1	1
	8	0	0
	9	5	4
	10	9	10
	11	2	1
Patient/Client Care	12	10	12
	13	7	9
	14	10	11
	15	12	14
	16	6	6
	17	7	7
	18	10	11
	19	3	3
	20	10	12
	21	10	12
	22	4	4
	23	5	5
	24	8	9
	25	8	10
	26	8	9
	27	5	4
	28	13	16
	29	6	5
	30	11	11
	31	7	7
	32	5	4
	33	6	6
	34	6	5
	35	4	3
	36	5	5
	37	5	4
	38	3	3
	39	3	2
	40	9	9
	41	9	9
	42	9	10
	43	6	6
	44	6	5
	45	10	7
Community Health Involvement	46	5	4
	47	7	6
	48	3	2
	49	3	2
	50	3	1
	51	4	3
	52	3	1
	53	3	1
	54	3	1
	55	2	1
	56	3	2

Developmental Activities for the Updated NBDHE Content Specifications

The content specifications of NBDHE were updated for subsequent item and test development. This was accomplished by mapping the importance ratings from survey respondents to the number of items to devote to the content elements in the examination. An ad hoc review committee was convened for this purpose. The Committee used a two-dimensional matrix (Lunz et al., 1989) to complete the mapping process. The two-dimensional matrix was found to be a valid model to conduct the mapping process (Lunz et al., 1989; Kramer & Neumann, 2007). This model was used for both NBDHE practice analyses. One dimension of this matrix consisted of the individual competencies underlying entry-level dental hygiene practice. The other dimension of the matrix consisted of the existing specifications of NBDHE (National Board Dental Hygiene Examination specifications, Joint Commission on National Dental Examinations, 2002; National Board Dental Hygiene Examination specifications, Joint Commission on National Dental Examinations, 2009). During the mapping process, the ad hoc review committee distributed the 350 items across competencies to the content elements that support the competencies using this matrix (Lunz et al., 1989). Each review committee was comprised of the following members: A Joint Commissioner appointed by the American Dental Hygiene Association, full-time practicing dental hygienists from various regions of the United States, and dental hygiene educators who are familiar with the dental hygiene school curricula and the content of the examination.

Upon the completion of the mapping, updated specifications for the NBDHE were developed. The distribution of 200 stand-alone items based on the 2002 and 2009 practice analyses are presented in Table 4. The distribution of 150 case-based items is not reported in Table 4 because the number varies due to the nature of patient cases. Some of the cases have more items in certain areas than in other areas. To avoid confusion, the Joint Commission does not report and release the distribution of 150 case-based items to the public. As shown, the number of items devoted to content elements was slightly increased or decreased depending on the judgments of the review committee and the data from the practice analysis. Insignificant changes were made to the specifications based on the findings of both practice analyses.

Table 4.

Updated specifications for the NBDHE with the distribution of stand-alone items based on the 2002 and 2009 practice analyses.

Category	Discipline	2002	2009
Scientific Basis for Dental Hygiene Practice: 60 Items	1.0. Anatomic Sciences	15	16
	2.0. Physiology	4	5
	3.0. Biochemistry and Nutrition	7	6
	4.0. Microbiology and Immunology	11	10
	5.0. Pathology	13	13
	6.0. Pharmacology	10	10
Provision of Clinical Dental Hygiene Services: 116 Items	1.0. Assessing Patient Characteristics	16	17
	2.0. Obtaining and Interpreting Radiographs	14	13
	3.0. Planning / Managing Dental Hygiene Care	34	34
	4.0. Performing Periodontal Procedures	19	18
	5.0. Using Preventive Agents	9	8
	6.0. Providing Supportive Treatment Services	7	7
	7.0. Professional Responsibility	17	18
Community Health/Research Principles: 24 Items	1.0. Promoting Health and Preventing Disease Within Groups	5	6
	2.0. Participating in Community Programs	11	10
	3.0. Analyzing Scientific Literature, Understanding Statistical Concepts, and Applying Research Results	8	8

Note. NBDHE = National Board Dental Hygiene Examination.

The final phase involved in validating the content of NBDHE was completed after the overall methodology, results of the analyses of the surveys, and the updated specifications with the distribution of items were reviewed and approved by the Joint Commission. Derived directly from the specifications of the NBDHE, items are developed and selected for the future editions of the examination. Comparability of editions of the NBDHE with regard to content is then insured as the editions of the examination are all constructed and assembled from the same specifications.

Discussion

The results of the analyses of the survey data showed that the 56 competencies adequately defined the domain of entry-level dental hygiene practice and were appropriate for use as a basis for the practice analyses and surveys. The important attributes required for entry-level dental hygiene practice were identified in the specifications of the examination by a series of validation phases as specified in the protocol. Results in Table 4 show that the content elements in the examination reflect the importance ratings provided by the practicing dental hygienists with regard to patient care. The major function of these validation phases is to ensure that there is agreement between the purpose of the examination and the actual requirements of entry-level dental hygiene practice. The agreement indicates that the examination is valid. Overall, consistent results from the practice analyses were demonstrated by the strong mapping between the examination content and the knowledge and skills associated with successful entry-level dental hygiene practice. And thus, such strong evidence supports and confirms the validity of the specifications for NBDHE. Furthermore, the protocol used for these practice analyses is deemed appropriate for examining, confirming, and evaluating validity. That is to say, the protocol is appropriate for content validation because it reflects importance perspectives from practitioners who safely perform the competencies in actual practice.

With regard to the representation of the 2002 and 2009 samples, they consisted of practicing dental hygienists who have been licensed within 5 years of passing the examination and 5–10 years, respectively. As noted in the results of the survey data analysis, the immediate impact on the average ratings for some competencies was shown in Table 2. The question arises as to whether there is an impact on the average ratings and an influence on the number of items devoted to the competencies in Table 3. As noted previously, the results of the analyses of the ratings by different groupings based on years of experience show that the numbers of items devoted to each competency were almost identical. The Joint Commission reviewed the results and determined to use a representative sample of dental hygienists who have been licensed within 10 years for the 2009 analysis. It is noteworthy that there is a challenge in delivering the surveys and reminders due to outdated mailing addresses for practicing dental hygienists who have been licensed more than 5 years.

This study provides a protocol that might prove useful to other professions wishing to gather sources of evidence related to the content validity for their licensure or credentialing examinations. However, practical constraints and challenges as described previously should be considered when applying the validation protocol. Extra developmental activities that were not used in the practice analyses for NBDHE might be considered in order to further enhance the validation protocol. First, add a separate scale to rate the competencies for their frequency of occurrence relevant to the dental hygiene practice environment. Judgments of importance and frequency of occurrence in practice provide an overall understanding of individual competencies (Kane, Kingsbury, Colton, & Estes, 1989). Second, change the survey format to online for cost efficiency considerations. Third, define a survey sample. That is, should individuals having more than 5 years in practice be included? Fourth, review the timeline for conducting a practice analysis. Currently, the Joint Commission conducts a practice analysis approximately every 5 years. Results of the two practice analyses showed that insignificant changes were made to the specifications. Given cost considerations, the dental hygiene and other professions might consider an expanded schedule for conducting a practice analysis.

Footnotes

Authors’ Note

The information and opinions contained in this article reflect and are solely the work of the authors and are not those of the American Dental Association or its employees or members.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Allen

M. J.

Yen

W. M.

(1979). Introduction to measurement theory (pp. 95–96). Monterey, CA: Brooks/Cole.

American Dental Education Association. (2001). ADEA 2000-2001 annual proceedings. Journal of Dental Education, 65, 662–666.

American Dental Education Association. (2004). Competencies for the new dentist. Journal of Dental Education, 68, 742–744.

American Educational Research Association (AERA), American Psychological Association (APA), and the National Council on Measurement in Education (NCME). (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Commission on Dental Accreditation. (1998). Accreditation standards for dental hygiene programs. Chicago, IL: American Dental Association.

Commission on Dental Accreditation. (2013). Accreditation standards for dental hygiene programs. Chicago, IL: American Dental Association.

Fitzpatrick

Morrison

E. J.

(1971). Performance and product evaluation. In Thorndike

R. L.

(Ed.). Educational measurement (2nd ed., pp. 237–270). Washington, DC: American Council on Education.

Joint Commission on National Dental Examinations. (2002). National Board Dental Hygiene Examination specifications. Chicago, IL: Joint Commission on National Dental Examinations.

Joint Commission on National Dental Examinations. (2009). National Board Dental Hygiene Examination specifications. Chicago, IL: Joint Commission on National Dental Examinations.

10.

Joint Commission on National Dental Examinations. (2012). National Board Dental Hygiene Examination 2012 guide. Chicago, IL: Joint Commission on National Dental Examinations.

11.

Kane

M. T.

Kingsbury

Colton

Estes

(1989). Combining data on criticality and frequency in developing test plans for licensure and certification examination. Journal of Educational Measurement, 26, 17–27.

12.

Kramer

G. A.

DeMarais

D. R.

(1997). Construct validity of the National Board Dental Hygiene Examination. Journal of Dental Education, 61, 709–716.

13.

Kramer

G. A.

Neumann

L. M.

(2004). Domain specifications and validity: National Board Dental Hygiene Examination practice analysis [poster]. Journal of Dental Education, 68, 920–944.

14.

Kramer

G. A.

Neumann

L. M.

(2007). Validation of the National Board Dental Hygiene Examination. Journal of Dental Hygiene, 81, 1–17.

15.

Linacre

J. M

. (2000/2006). WINSTEPS: Rasch-model computer programs [computer software]. Chicago, IL. MESA Press.

16.

Lunz

M. E.

Stahl

J. A.

James

(1989). Content validity revisited: Transforming job task analysis data into test specifications. Evaluation & the Health Professions, 12, 192–206.

17.

Messick

(1993). Validity. In Linn

R. L.

(Ed.), Educational measurement (3rd ed., pp. 13–103). Washington, DC: American Council on Education.

18.

Millman

Greene

(1993). The specification and development of tests of achievement and ability. In Linn

R. L.

(Ed.), Educational measurement (3rd ed., pp. 335–366). Washington, DC: American Council on Education.

19.

Tsai

T.-H.

Leatherman-Dixon

(2013). Setting and validating the pass/fail score for the NBDHE. Journal of Dental Hygiene, 87, 90–94.

20.

Yang

C.-L.

Kramer

G. A.

Neumann

L. M.

(2009). Examining the stability of candidate performance and item difficulty for dental hygiene examination. Poster presented at the 86th Annual Session and Exhibition of the American Dental Education Association, Phoenix, AZ.

21.

Yang

C.-L.

Tsai

T.-H.

Kramer

G. A.

Neumann

L. M.

(2007). Examining the stability of National Board Dental Hygiene performance: A 5-year study. Poster presented at the 84th Annual Session and Exhibition of the American Dental Education Association, New Orleans, LA.