Multilevel researchers have predominantly applied either direct consensus or referent-shift consensus composition models when aggregating individual-level data to a higher level of analysis. This prevailing focus neglects both theory and empirical evidence, suggesting that the variance of group members' responses may complement the absolute mean level of group members' judgments. The goals of this article are to demonstrate the application of dispersion-composition models for capturing variability among group members' collective judgments and highlight the statistical challenges (and inherent constraints) of using group means and variances as predictors of study criteria. To this end, the authors present and illustrate a six-step sequential framework for applying dispersion-composition models using data from two independent field samples. The authors contend that the application of dispersion-composition models not only will strengthen a study’s conclusions by eliminating potential rival data interpretations but may also shed new light on past findings, potentially opening new doors to a more complete understanding of multilevel phenomena.
AllenN. J.StanleyD. J.WilliamsH. M.RossS. J. (2007). Assessing the impact of nonresponse on work group diversity effects. Organizational Research Methods, 10, 262–286.
2.
BassB. M.AvolioB. J. (2000). Manual for the Multifactor Leadership Questionnaire (Form 5X). Redwood City, CA: Mindgarden.
3.
BealD. J.DawsonJ. F. (2007). On the use of Likert-type scales in multilevel data: Influence on aggregate variables. Organizational Research Methods, 10, 657–672.
4.
BiemannT.KearneyE. (2010). Size does matter: How varying group sizes in a sample affect the most common measures of group diversity. Organizational Research Methods, 13, 582–599.
5.
BlieseP. D. (1998). Group size, ICC values, and group-level correlations: A simulation. Organizational Research Methods, 1, 355–373.
6.
BlieseP. D. (2000). Within group agreement, non-independence and reliability: Implications for data and analysis. In KleinK. J.KozlowskiS. W. J. (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 349–381). San Francisco, CA: Jossey-Bass.
7.
BlieseP. D.BrittT. W. (2001). Social support, group consensus and stressor–strain relationships: Social context matters. Journal of Organizational Behavior, 22, 425–436.
8.
BlieseP. D.HalversonR. R. (1998). Group consensus and psychological well-being: A large field study. Journal of Applied Social Psychology, 28, 563–580.
9.
BlieseP. D.HangesP. J. (2004). Being both to liberal and too conservative: The perils of treating grouped data as though they were independent. Organizational Research Methods, 5, 362–387.
10.
CarronA. V.BrawleyL. R. (2000). Cohesion: Conceptual and measurement issues. Small Group Research, 31, 89–106.
11.
CarronA. V.BrawleyL. R.EysM. A.BrayS. R.DorschK. D.EstabrooksP. A. (2003). Do individual perceptions of group cohesion reflect shared beliefs? An empirical analysis. Small Group Research, 34, 468–496.
12.
ChanD. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234–246.
13.
ChenG.MathieuJ. E.BlieseP. D. (2004). A framework for conducting multi-level construct validation. Research in Multi-Level Issues, 3, 273–303.
14.
ColquittJ. A.NoeR. A.JacksonC. L. (2002). Justice in teams: Antecedents and consequences of procedural justice climate. Personnel Psychology, 55, 83–109.
15.
CongerJ. A.KanungoR. N.MenonS. T. (2000). Charismatic leadership and follower effects. Journal of Organizational Behavior, 21, 747–767.
16.
CortinaJ. M. (1993). Interaction, nonlinearity, and multicollinearity: Implications for multiple regression. Journal of Management, 19, 915–922.
17.
DansereauF.ChoJ.YammarinoF. J. (2006). Avoiding the “fallacy of the wrong level”: A within and between analysis (WABA) approach. Group and Organization Management, 31, 536–577.
18.
DansereauF.YammarinoF. J. (2000). Within and between analysis: The varient paradigm as an underlying approach to theory building and testing. In KleinK. J.KozlowskiS. W. J. (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 425–466). San Francisco, CA: Jossey-Bass.
19.
DeRueD. S.HollenbeckJ.IlgenD.FeltzD. (2010). Efficacy dispersion in teams: Moving beyond agreement and aggregation. Personnel Psychology, 63, 1–40.
20.
DineenB. R.NoeR. A.ShawJ. D.DuffyM. K.WiethoffC. (2007). Level and dispersion of satisfaction in teams: Using foci and social context to explain the satisfaction and absenteeism relationship. Academy of Management Journal, 50, 623–643.
21.
GeorgeJ. M. (1990). Personality, affect, and behavior in groups. Journal of Applied Psychology, 75, 107–116.
22.
GeorgeJ. M. (2002). Affect regulation in groups and teams. In LordR. G.KlimoskiR. J.KanferR. (Eds.), Emotion in the workplace: Understanding the structure and role of emotions in organizational behavior (pp. 183–217). San Francisco, CA: Jossey-Bass.
23.
GeorgeJ. M.JamesL. R. (1993). Personality, affect, and behavior in groups revisited: Comment on aggregation, levels of analysis, and a recent application of within and between analysis. Journal of Applied Psychology, 78, 798–804.
24.
GootyJ.YammarinoF. J. (2011). Dyads in organizational research: Conceptual issues and multilevel analyses. Organizational Research Methods, 14, 456–483.
25.
HarrisonD. A.KleinK. J. (2007). What’s the difference? Diversity constructs as separation, variety, or disparity in organizations. Academy of Management Review, 32, 1199–1228.
26.
HirschfeldR. R.BernerthJ. B. (2008). Mental efficacy and physical efficacy at the team level: Inputs and outcomes among newly formed action teams. Journal of Applied Psychology, 93, 1429–1437.
27.
JudgeT. A.PiccoloR. F. (2004). Transformational and transactional leadership: A meta-analytic test of their relative validity. Journal of Applied Psychology, 89, 755–768.
28.
KleinK. J.ConnA. B.SmithD. B.SorraJ. S. (2001). Is everyone in agreement? An exploration of within-group agreement in employee perceptions of the work environment. Journal of Applied Psychology, 86, 3–16.
29.
KozlowskiS. W. J.KleinK. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In KleinK. J.KozlowskiS. W. J. (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 3–90). San Francisco, CA: Jossey-Bass.
30.
LanceC. E.ButtsM. M.MichelsL. C. (2006). The sources of four commonly reported cutoff criteria. What did they really say?. Organizational Research Methods, 9, 202–220.
31.
LeBretonJ. M.BurgessJ. R. D.KaiserR. B.AtchleyE. K.JamesL. R. (2003). The restriction of variance hypothesis and interrater reliability and agreement: Are ratings from multiple sources really dissimilar?. Organizational Research Methods, 6, 80–128.
32.
LeBretonJ. M.SenterJ. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11, 815–852.
33.
LindellM. K.BrandtC. J. (2000). Climate quality and climate consensus as mediators of the relationship between organizational antecedents and outcomes. Journal of Applied Psychology, 85, 331–348.
34.
LubinskiD.HumphreysL. G. (1990). Assessing spurious “moderator effects”: Illustrated substantively with the hypothesized (“synergistic”) relation between special and mathematical ability. Psychological Bulletin, 107, 385–393.
35.
MathieuJ. E.ChenG. (in 2011). The etiology of the multilevel paradigm in management research. Journal of Management, 37, 610–641.
36.
MathieuJ. E.MaynardM. T.RappT.GilsonL. (2008). Team effectiveness 1997-2007: A review of recent advancements and a glimpse into the future. Journal of Management, 34, 410–476.
37.
MeadeA. W.EbyL. T. (2007). Using indices of group agreement in multilevel construct validation. Organizational Research Methods, 10, 75–96.
38.
MossholderK. W.BedeianA. G. (1983). Cross-level inference and organizational research: Perspectives on interpretation and application. Academy of Management Review, 8, 547–558.
39.
NaumannS. E.BennettN. (2002). The effects of procedural justice climate on work group performance. Small Group Research, 33, 361–377.
40.
NewmanD. A.HangesP. J.DuanL.RameshA. (2008). A network model of organizational climate: Friendship, clusters, subgroup agreement, and climate schemas. In SmithD. B. (Ed.), The people make the place: Dynamic linkages between individuals and organizations (pp. 101–126). Mahwah, NJ: Lawrence Erlbaum.
41.
NewmanD. A.SinH. P. (2009). How do missing data bias estimates of within-group agreement? Sensitivity to SDwg, CVwg, rwg(j), rwg(j)*, and ICC to systematic nonresponse. Organizational Research Methods, 12, 113–147.
42.
QuigleyN.TekleabA. G.TeslukP. (2007). Comparing consensus- and aggregation-based methods of measuring team-level variables: The role of relationship conflict and conflict management processes. Organizational Research Methods, 10, 589–608.
43.
RiordanC. E.WeatherlyE. W. (1999). Defining and measuring employees' identification with their work groups. Educational and Psychological Measurement, 59, 310–324.
44.
RobersonQ. M.SturmanM. C.SimonsT. L. (2007). Does the measure of dispersion matter in multilevel research? A comparison of the relative performance of dispersion indexes. Organizational Research Methods, 9, 564–588.
45.
SinH.-P.NewmanD. A. (2005). Variance of means versus mean of variances: A contrarian view on operationalizing group dispersion. In WeaverK. M. (Ed.), Proceedings of the Sixty-Fourth Annual Meeting of the Academy of Management. Academy of Management Best Paper Proceedings. Briarcliff Manor, NY: Academy of Management.
46.
SpoorJ. R.KellyJ. R. (2004). The evolutionary significance of affect in groups: Communication and group bonding. Group Processes and Intergroup Relations, 7, 398–412.
47.
TerryP. C.CarronA. V.PinkM. J.LaneA. M.JonesG.HallM. (2000). Perceptions of group cohesion and mood in sport teams. Group Dynamics, 4, 234–243.
48.
TimmermanT. A. (2005). Missing persons in the study of groups. Journal of Organizational Behavior, 26, 21–36.
49.
Van KatwykP. T.FoxS.SpectorP. E.KellowayE. K. (2000). Using the Job-Related Affective Well-Being Scale (JAWS) to investigate affective responses to work stressors. Journal of Occupational Health Psychology, 5, 219–230.
50.
van MierloH.VermuntJ. K.RutteC. G. (2009). Composing group-level constructs from individual-level survey data. Organizational Research Methods, 12, 368–392.
