Abstract
Welcome back! In this issue of Update, we have eight outstanding articles with one short-form article, one research-to-resource article, three reviews of literature, two studies, and one “extending the discussion” response article. The issue begins with a short-form article on concert programs by Freer, followed by a research-to-resource article on culturally responsive pedagogy by Kelley-McHale. The topics of the reviews of literature include collective free music improvisation by Ng, choral configuration by Adams, and popular, informal, and vernacular music classrooms by Mercado. The topics of the studies include music listening by Williams, Geringer, and Brittin and perceptions of adjudicated events by Rawlings. After the Rawlings article, we have an “extending the discussion” article by Millican that provides additional pragmatic ideas for teachers on the topic of adjudicated events.
Because research studies using questionnaires as measurement tools are prevalent in our field, I will be providing some best practices ideas that may assist authors: (a) obtaining the initial sample for a survey research study, (b) determining when the respondent list is complete enough, and (c) using a questionnaire strategically.
Using high-quality, logical, transparent sampling processes to obtain an initial sample from a well-defined population is an important part of survey research methodology. Descriptive research studies do not tend to use p values that can provide a conceptual link to a population, and so it is important for survey research studies to engage the population in a strategic way through sampling processes in order to represent the population as a part of the research process. If a population list is not comprehensive and representative, then any generalizations made from a sample to a broader population will not be meaningful. For instance, if a researcher wants to know teachers’ opinions but can only get a population list of teachers in private schools, generalizations cannot be made to public schools.
Probability sampling processes allow for the greater possibility that any population member will have an equal chance of being in the sample. Common probability sampling processes that are used are (a) simple random sampling, which involves randomly drawing participants from the population list, commonly through an online random number generator; (b) stratified sampling, which is commonly used when researchers want to represent specified subgroups in their sample, such as band, orchestra, or choir directors, either in equal numbers or proportional to the population; or (c) cluster sampling, which often involves a two-stage process, such as sampling clusters (e.g., schools) followed by sampling individuals from those clusters.
One of the possible challenges with survey research is obtaining responses from only a small segment of those sampled to participate in the study. Say I have a list of 100 individuals to whom I send an invitation to participate in the study and I only get 20 of them to respond. Is that really a substantive enough set of data points to be able to represent the sample and generalize back to the population? Maybe not. It may be that the 80 nonrespondents had a very different opinion on the topic than the 20 respondents did. Researchers must keep close tabs on this nonresponse issue or they may be publishing a biased result from a very positive or very negative subset of those who were invited to participate.
To get a feel for the extent that a set of responses may or may not represent the sample and broader population, researchers have some options at their disposal. They can
send a follow-up reminder to questionnaire nonrespondents from the sample trying to obtain a more complete set of data,
make direct contact with a sample of nonrespondents to obtain their responses over the phone and check the responses of those who responded traditionally to those who responded after direct contact, or
look at key markers for respondents and nonrespondents, such as demographic characteristics, to see if those who responded tend to be similar to those who did not.
In addition, Dillman, Smyth, and Christian (2014) cited a formula that can help researchers gauge the number of respondents necessary to have usable data.
Once a study is complete, it is a joy for readers when researchers can clearly describe in the article the use of a questionnaire in terms of methodology and results that align with the study’s purpose. For example, researchers can help readers by clearly stating in the purpose statement what the strategic use of a questionnaire will be in an article. Any data-gathering tool, such as a questionnaire, has many potential methodological uses in studies, including descriptive, relational, or comparative. Authors who can use the terms describe, relate, or compare, in their purpose statements are making the intent of the study clear for the reader right from the start.
The most common methodological use of a questionnaire is for descriptive studies. One common way that descriptive questionnaires are used is as a summed score to describe respondents’ overall skill levels or perceptions. When summing items to get an overall score, then internal consistency would be documented in the methodology section of the article as the reliability choice to estimate whether the items that are being summed load well together. Researchers would then present the overall respondent score descriptive statistics in the results section. A second common way that descriptive questionnaires are used is as a set of heterogeneous items to describe respondents’ perceptions about a diverse set of content. When using heterogeneous items, test-retest reliability would be documented in the methodology section to estimate whether respondents answer each item consistently over time. Researchers would then present descriptive statistics for the item means individually in the results section.
As a side note, if a questionnaire is being used as a variable in a relational or comparative study, it is advised that a summed, homogeneous questionnaire be used to avoid overuse of the data set that can happen when researchers run an excessive number of parametric statistics across each item of a questionnaire. When researchers run more than one comparative or relational statistic, a correction measure such as Bonferroni should be used to counteract the overuse problem.
In summary, researchers will help reader comprehension if they can clearly describe important components for studies that use a questionnaire: (a) obtaining the initial sample for a survey research study, (b) determining when the respondent list is complete enough, and (c) using a questionnaire strategically.