The Effect of Negation and Polar Opposite Item Reversals on Questionnaire Reliability and Validity: An Experimental Investigation

Abstract

This research was concerned with the effect of item wording on questionnaire reliability and validity. The subjects were 280 business undergraduate students who read a scenario describing a hypothetical leader's behavior and then completed one of four different questionnaires to describe that leader. Comprising the experimental treatments, the questionnaires included four different types of items: regular (e.g., "I am happy"), polar opposite (e.g., "I am sad"), negated polar opposite (e.g., "I am not sad"), and negated regular (e.g., "I am not happy") items. The results indicated significantly lower coefficient alpha internal-consistency reliabilities for the polar opposite and negated polar opposite items as compared with those for the regular and negated regular items. Using accuracy scores, one-way analyses-of-variance and posthoc directional t-tests showed the same results for validity of measurement. Implications for future questionnaire development and research are briefly considered.

Get full access to this article

View all access options for this article.

References

American Psychological Association (1982). Ethical principles in the conduct of research with human participants. Washington, D.C.

Anastasi, A. (1982). Psychological testing (5th ed.). New York: Macmillan.

Benson, J. and Hocevar, D. (1985). The impact of item phrasing on the validity of attitude scales for elementary school children. Journal of Educational Measurement, 22, 213-240.

Bentler, P. M. , Jackson, D. N. , and Messick, S. (1972). A rose by any other name. Psychological Bulletin, 77, 109-113.

Blaney, P. H. (1986). Affect and memory: A review. Psychological Bulletin, 99, 229-246.

Campbell, N. O. and Grissom, S. (April 1979). Influence of item direction on student responses in attitude assessment. Paper presented at the 63rd annual meeting of the American Educational Research Association, San Francisco, CA.

Hathaway, S. R. and McKinley, J. C. (1967). Minnesota multiphasic personality inventory: Manual for administration and scoring. New York: Psychological Corporation.

Jackson, D. N. and Messick, S. (1965). Acquiescence: The nonvanishing variance component. American Psychologist, 20, 498.

Jordan, L. A. (1977). The use of covariance structure analysis for the exploration of acquiescence and other response style issues in faceted test data. Unpublished doctoral dissertation, Department of Psychology, City University of New York (University Microfilms 77-28, 645).

10.

Matlin, M. W. and Stang, D. J. (1978). The Pollyana principle: Selectivity in language, memory, and thought. Cambridge, MA: Schenkman.

11.

Rorer, L. G. (1965). The great response style myth. Psychological Bulletin, 63, 129-156.

12.

Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.

13.

Schmitt, N. and Stults, D. M. (1985). Factors defined by negatively keyed items: The result of careless respondents? Applied Psychological Measurement, 9, 367-373.

14.

Schriesheim, C. A. and Hill, K. D. (1981). Controlling acquiescence response bias by item reversals: The effect on questionnaire validity. Educational and Psychological Measurement, 41, 1101-1114.

15.

Simpson, R. D. , Rentz, R. R. , and Shrum, J. W. (1976). Influence of instrument characteristics on student responses in attitude assessment. Journal of Research in Science Teaching, 13, 275-281.

16.

Stogdill, R. M. (1963). Manual for the leader behavior description questionnaire-Form XII. Columbus: Bureau of Business Research, Ohio State University.

17.

Winkler, J. D. , Kanouse, D. E. , and Ware, J. E., Jr. (1981). Controlling for acquiescence response set in scale development. Paper presented at the 90th annual meeting of the American Psychological Association, Los Angeles, CA, August.