Indexes of Validity and Reliability for Cross-Societal Measures

Abstract

Extracting measures from classical ethnographies has always necessitated proxy indicators in cross-societal research. Methods are given for calculating indexes of validity and reliability for both a low inference measure x and high inference proxy measure y of the same conceptual variable Z. Where two different ethnographic descriptions A and B exist for each society in a sample, low inference measures x _A and x _B are used to assess validity and reliability from the classical “true score” measurement model x = τ + ε _X , τ is the true score, ε _X is error, ξ[E _X ] = 0. Tests establish whether the model fits data. Reliability is defined to be (1 — Var[E _X ]/Var[X]). Validity is Var[T]/Var[X]. The validity and reliability of high inference measure y are increased by fitting a curve x = f(y) + ε _f(y) , ξ[E _f(y) ] = 0, to data. Standard cross-cultural sample examples are given. Solutions for cross-societal research may be considered for cross-cultural research.

Keywords

measurement validity reliability proxy conditional imputation cross-societal research

Get full access to this article

View all access options for this article.

References

Allison, P.D. (2002). Missing data. Thousand Oaks, CA: Sage.

Bollen, K.A. (2002). Latent variables in psychology and the social sciences. Annual Review of Psychology, 53, 605-634.

Carmines, E.G. , & Zeller, R.A. (1979). Reliability and validity assessment. Beverly Hills, CA: Sage.

Denton, T. (2007a). Behavioral relations for components of recent preindustrial modernization: Quantitative assessment. Ethnology , 45, 229-254

Denton, T. (2007b). Unit of observation in cross-cultural research: Implications for sampling and aggregated data analysis. Cross-Cultural Research, 41, 3-31.

Divale, W.T. (2004). World cultures CD data disk. World Cultures, 14, 194.

Hambleton, R.K. , Swaminathan, H. , & Rogers, H.J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.

Kutner, M.H. , Nachtscheim, C.J. , & Neter, J. (2004). Applied linear regression models (5th ed.). Boston: McGraw-Hill .

Levinson, D. , & Malone, M.J. (1980). Toward explaining human culture. New Haven, CT: HRAF Press.

10.

Lord, F.M. , & Novick, M.R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.

11.

Murdock, G.P. (1967). Ethnographic atlas: A summary. Pittsburgh, PA: The University of Pittsburgh Press .

12.

Murdock, G.P. (1981). Atlas of world cultures. Pittsburgh, PA: The University of Pittsburgh Press.

13.

Murdock, G.P. , & Provost, C. (1973). Measurement of cultural complexity. Ethnology, 12, 379-392.

14.

Murdock, G.P. , & White, D.W. (1969). The standard cross-cultural sample. Ethnology, 8, 329-369.

15.

Murdock, G.P. , & Wilson, S.F. (1972) Settlement pattern and community organization: Cross-cultural codes. Ethnology, 11, 254-295.

16.

Northrop, F.S.C. (1963). The logic of the sciences and the humanities. Cleveland, OH: World Publishing Co.

17.

Pryor, F.L. (1985). The invention of the plow. Comparative Study of Society and History, 27, 727-743.

18.

Pryor, F.L. (1986). The adoption of agriculture: Some theoretical and empirical evidence. American Anthropologist, 88, 879-897.

19.

Ross, M. (1983). Political decision making and conflict: Additional cross-cultural codes and scales. Ethnology, 22, 169-192.

20.

Siegel, S. (1956). Nonparametric statistics for the behavioral sciences . New York: McGraw-Hill.

21.

Spiegel, M.R. , Schiller, J.J. , & Srinivasan, R.A. (2000). Theory and problems of probability and statistics (2nd ed.). New York: McGraw-Hill.

22.

White, D.R. (1988). Rethinking polygamy: Co-wives, codes and cultural systems. Current Anthropology, 29, 529-572.

23.

White, D.R. (1990). Reliability in comparative and ethnographic observations . Journal of Quantitative Anthropology, 2, 109-150.

24.

Whyte, M.K. (1978). The status of women in preindustrial societies . Princeton, NJ: Princeton University Press.

25.

Winkelman, M. (1994). Subsistence and sociopolitical conditions: SCCS variable descriptions and reliability 1. World Cultures, 8, 15-18.