Abstract
During the last 30 years, significant debate has taken place regarding multilevel research. However, the extent to which multilevel research is overtly practiced remains to be examined. This article analyzes 10 years of organizational research within a multilevel framework (from 2001 to 2011). The goals of this article are (a) to understand what has been done, during this decade, in the field of organizational multilevel research and (b) to suggest new arenas of research for the next decade. A total of 132 articles were selected for analysis through ISI Web of Knowledge. Through a broad-based literature review, results suggest that there is equilibrium between the amount of empirical and conceptual papers regarding multilevel research, with most studies addressing the cross-level dynamics between teams and individuals. In addition, this study also found that the time still has little presence in organizational multilevel research. Implications, limitations, and future directions are addressed in the end.
Organizations are made of interacting layers. That is, between layers (such as divisions, departments, teams, and individuals) there is often some degree of interdependence that leads to bottom-up and top-down influence mechanisms. Teams and organizations are contexts for the development of individual cognitions, attitudes, and behaviors (top-down effects; Kozlowski & Klein, 2000). Conversely, individual cognitions, attitudes, and behaviors can also influence the functioning and outcomes of teams and organizations (bottom-up effects; Arrow, McGrath, & Berdahl, 2000). One example is when the rewards system of one organization may influence employees’ intention to quit and the existence or absence of extra role behaviors. At the same time, many studies have showed the importance of bottom-up emergent processes that yield higher level phenomena (Bashshur, Hernández, & González-Romá, 2011; Katz-Navon & Erez, 2005; Marques-Quinteiro, Curral, Passos, & Lewis, in press). For example, the affectivity of individual employees may influence their team’s interactions and outcomes (Costa, Passos, & Bakker, 2012). Several authors agree that organizations must be understood as multilevel systems, meaning that adopting a multilevel perspective is fundamental to understand real-world phenomena (Kozlowski & Klein, 2000). However, whether this agreement is reflected in practicing multilevel research seems to be less clear. In fact, how much is known about the quantity and quality of multilevel research done in the last decade? The aim of this study is to compare what has been proposed theoretically, concerning the importance of multilevel research, with what has really been empirically studied and published. First, this article outlines a review of the multilevel theory, followed by what has been theoretically “put forward” by researchers. Second, this article presents what has really been “practiced” based on the results of a review of multilevel studies published from 2001 to 2011 in business and management journals. Finally, some barriers and challenges to true multilevel research are suggested.
This study contributes to multilevel research as it describes the last 10 years of research. It quantitatively depicts the type of articles being written, and where we can find the majority of the publications on empirical and conceptual work related to multilevel thinking.
Multilevel Research: Aligned Theory, Measurement, and Analysis
Rousseau (1985) and Mathieu and Chen (2011) highlighted three fundamental aspects to multilevel research that must be aligned, to avoid level-related confusions or errors: the level of theory, the level of measurement, and the level of analysis.
Level of theory refers to the focal level: the entity about which the researcher draws conclusions (individuals, subunits, firms, etc.) and to which generalizations they are designed to apply. Although it apparently seems easy, establishing the boundaries dividing one entity from another (what defines a team, when its members belong to more than one team, for example), and defining when one moves from one level of analysis to another (from teams to organizations, as an example) must be done quite carefully. The same considerations should be applied when examining mixed teams, or individuals who belong to different projects in distinct organizations, as it makes it difficult to understand their membership and their contribution to higher levels. Another important aspect at this level is to actually explicit the multilevel theory, which means to outline how phenomena at different levels are linked. These links may be top-down or bottom-up. Top-down mechanisms express the influence of higher level contextual factors on lower levels of the organization. For example, the culture of an organization influences the more or less formal patterns of interaction between individuals. Bottom-up mechanisms explain how lower level dynamics shape the emergence of higher level phenomena, which are unique, and cannot be reduced to their lower level elements (Dansereau, Alutto, & Yammerino, 1984). In fact, there are team processes that necessarily imply the interaction and coordination between team members and cannot be reduced to their individual perceptions. One example is team reflexivity, which has demonstrated to yield team effectiveness (Graça & Passos, 2012; Schippers, Den Hartog, Koopman, & Van Knippenberg, 2008). Conceptually, these two mechanisms are both equally important. However, empirical research has focused more on downward (contextual) processes rather than upward (emergent) processes, suggesting that the larger context is more likely to influence lower level variables than the opposite. Nonetheless, some researchers have highlighted the importance of upward influences and claimed that empirical studies should also emphasize them (Chen, Kanfer, DeShon, Mathieu, & Kozlowski, 2009; Morgeson & Hofmann, 1999). This trend finds support in other research areas such as social networks and complex adaptive systems because these offer methodological tools to observe and analyze bottom-up effects in social structures. One example is a recent study on the dynamics of financial stability showing that market crashes can only be avoided when banks accept the loss of gains and behave in a resilient way. That is to say that when banks decide to lower their risk, the financial network will grow and the probability of a financial crisis increases (Cruz & Lind, 2011).
The level of measurement refers to the entities from which data are drawn and should reflect the theory level. The level of theory and the level of measurement should therefore be aligned to avoid possible misunderstandings and erroneous conclusions. When studying organizational climate, one must gather data at the organizational level whereas addressing individual-level motivation, researchers should gather data from individuals. In an area of research where individuals are often the main sources of information, researchers must justify why the process of data collection used is suitable for their particular research purposes. When the level of measurement is lower than the level of analysis, it is crucial to have a good justification for aggregating the data preceded by a theoretical rationale that explains how the higher level phenomenon comes into existence. Chan (1998) proposed a typology of composition models (see Table 1) that may guide researchers when working on theory building, data gathering, and the measurement instruments used. This typology, like others (Chen, Mathieu, & Bliese, 2004, for example), requires that researchers “have a theory about how data collected at one level of analysis should be combined to represent constructs at a higher level of analysis” (Mathieu & Chen, 2011, p. 617). It encompasses five models that describe how lower level data may be aggregated to represent higher level phenomena or constructs.
Composition Models.
Note. Adapted from Chan, 1998.
Adding to Chan’s (1998) composition models, Kozlowski and Klein (2000) suggested another form of emergence, which they named “compilation models.” On one hand, composition models reflect an equal contribution of each lower level entity to the higher level. For example, an organization’s service climate is theoretically a reflection of the members of the organizations’ shared perceptions of the extent to which organizational policies reward and encourage customer service (Schneider & Bowen, 1985). On the other hand, compilation models suggest that higher level phenomena may be more complex combinations of lower level contributions. Team performance, for example, is a complex function of specific individual contributions that are not the same from individual to individual: Some individuals may contribute more to team performance than others. Although this form of emergence (compilation) underlies many concepts, it is not widely recognized by researchers (Kozlowski & Klein, 2000).
Finally, theory and measurement levels should also be aligned with the level of analysis, that is, the level at which data are analyzed to test hypotheses. Statistically, several measures have been created to assess within-group agreement and justify data aggregation to the higher level, such as the within-group agreement index (Rwg(j); James, Demaree & Wolf, 1984, 1993), the intraclass correlation coefficient (ICC; Bliese, 2000), and the average dispersion index (ADI; Burke & Dunlap, 2002). However, the level of analysis goes beyond aggregation issues. In the 1980s, several techniques and methods for analyzing multilevel data emerged, such as ANCOVA (Mossholder & Bedeian, 1983); contextual analysis (Firebaugh, 1979); WABA (Within and Between Analysis; Dansereau et al., 1984); CLOP (cross-level operator; James, Demaree, & Hater, 1980); random-coefficient modeling (RCM) with HLM (hierarchical linear modeling; Burstein, Linn, & Capell, 1978); and there have been other more recent advances, such as the development of the Nonlinear and Linear Mixed Effects program for S-PLUS and R programs (Pinheiro & Bates, 2000). Moreover, Albright and Marinova (2010) have recently presented a brief review on how to estimate multilevel models using SPSS, Stata, SAS, and R, thus making important contributions to the advances of multilevel research. Mplus has also proved to be a valuable tool for analyzing multilevel data (Muthén & Asparouhov, 2011), especially for longitudinal designs. According to Kozlowski and Klein (2000), there is no single “best” technique. Researchers should base their choices on the “consistency between the type of constructs, the sampling and the data, and the research question; and on the assumptions, strengths, and limitations of the analytic technique” (Kozlowski & Klein, 2000, p. 51). There has been an evolution in the theory of multilevel, as well as interesting developments in the statistical procedures and software available. Nonetheless, how do researchers integrate these developments in their research? How are multilevel theory and practice reflected in recent peer-reviewed publications?
Multilevel Thinking in Theory and Research
Multilevel research has long caught researchers’ attention, at least in a theoretical sense. Recently, Rousseau (2011) summarized some developments that have occurred for micro- and macrobridging, and highlighted that multilevel research is being (successfully) done. Rousseau structured her argumentation by presenting some evidence. First, the existence of an organizational mode of thinking, introduced as a natural feature of organizational researchers and as a distinctive competence of organizational science. Due to its inherent interdisciplinary aspects (e.g., psychology, sociology, and economics), organizational science fosters a multilevel perspective. Other evidence is the use of multilevel or cross-level heuristics by researchers, like the rule of thumb of always considering one level up and one level below the focal construct the researcher is studying, and partitioning variance. Third, the development of multilevel concepts, like “emergence” or “embeddedness,” guides researchers in their multilevel theory and research. Also, the use of cross-level interventions is another evidence that proves that multilevel thinking is “inherent in the working lives of many organizational scientists” (Rousseau, 2011, p. 1). In a study by Hitt, Beamish, Jackson, and Mathieu (2007), the multilevel perspective was identified in approximately 25% of the articles published in the Academy of Management Journal and in 50% in the case of the Academy of Management Review, in a 12-month period. Thus, it appears that multilevel research is being done and developed.
In addition to Rousseau’s contribution, we would like to emphasize that there is further proof that shows the relevance and interest concerning multilevel thinking. Many researchers are conscious of the importance and practical appliance of multilevel thinking. The sharing that stems from multidisciplinary teams embedded in research departments and the discussion of knowledge that each team member can offer about distinctive and specific areas of expertise may lead to an effective multilevel thinking and to an integrative organizational science. In addition, increasingly more people combine different areas in their curricula. For instance, they have a degree in a specific area (e.g., psychology), and do their PhD and their research or work in another area (e.g., management). Thus, these people are likely to have more knowledge and broader skills to think, to analyze, and to understand organizational phenomena at different levels and from different perspectives. Moreover, in spite of acting and promoting change in only one area, they have the skills needed to act and promote change in different ones. More evidence of multilevel thinking and research is the development of multilevel research methods and statistical procedures, such as the ones mentioned above. These developments have led many universities to provide summer schools dedicated to multilevel statistical procedures. This shows that universities have made the more advanced organizational research methods available to students to allow them to develop their ability to analyze and to make interventions within a hierarchical system of organizations, groups, and individuals (for instance, Multilevel Modeling with R for Beginners at INGRoup Conference, 2012). What is more, several books on the importance and practice of multilevel have been published in the last decade (Klein & Kozlowski, 2000, for a reference). There are also articles and journal issues specifically dedicated to multilevel (Journal of Management—Special issue: Bridging micro and macrodomains, as an example). These publications discuss the recent advances in multilevel theory and research, and represent the effort of the experts (on management, human resources, social capital, workplace demography, etc.) to comprehend multilevel issues. Ultimately, their work encourages and stimulates researchers to make their contributions, thus multilevel progresses more and more.
Finally, the theoretical and methodological advances that have been mentioned so far allow us to say that the current reflections about research contexts (i.e., micro, macro), and change (i.e., time), as key variables to understand a whole system, express researchers’ concern to embrace this approach. This means that because time is present in all contexts—for some authors time is a component or a dimension of context (Johns, 2006; Porter & McLaughlin, 2006)—and the functioning of teams and organizations is dynamic, it is essential to consider time to understand the micro and macrocontexts. Even with all the considerations and advances concerning multilevel thinking, Kozlowski and Klein (2000), assumed that the influence of multilevel was “merely metaphorical” (p. 1). After 12 years, multilevel is widely known and valued by organizational researchers, but can we say that it is no longer a metaphor? Are the core problems of organizational dynamics really explained according to the multilevel research as has been done? To what extent did multilevel research add something to the understanding of these problems? Moreover, to what extent did multilevel research changed the management practices in organizations? At last, is there any further potential for developing multilevel thinking in research and practice?
Some Barriers to Multilevel Thinking
One of the foundations of multilevel thinking was the idea that micro phenomena are embedded in macrocontexts and that macrophenomena can emerge due to the interaction and dynamics of lower level elements (Kozlowski & Klein, 2000). Yet, organizational researchers are likely to highlight either a micro- or a macroperspective. Although an organization is an integrative system, organizational science has been having difficulties in integrating theories that explain phenomena at the individual and group level of analysis (e.g., goal-setting theory) with theories that explain phenomena at the organizational level of analysis (such as the resource-based view of the firm). Roberts, Hulin, and Rousseau (1978), referred to the need for an integrative effort of different disciplines in organizational science. The multilevel paradigm was born when a meso approach highlighted the fact that any phenomenon of interest was influenced by factors situated above and below that phenomenon. Nowadays, researchers are still trying to make an effort to bridge the micro−macro divide, as shown in some special issues of the Academy of Management Journal (Hitt et al., 2007), the Journal of Organizational Behavior (Griffin, 2007), and the Journal of Management (Aguinis, Boyd, Pierce, & Short, 2011). One of the reasons for multilevel thinking and research not being developed in organizational science may be due to this micro-/macrobridging that is still a challenge to researchers (Rousseau, 2011). First, it is a cognitive challenge as people need to reflect on a large amount of information when considering complex phenomena. It is also a social dilemma, because people wonder why they should invest their effort in studying complex models of multilevel research instead of focusing on specific topics that they can develop in depth. Finally, multilevel research may be a political process when some levels are viewed as more important than others. However, in accordance with Molloy, Ployhart, and Wright (2011), there is not just one divide between micro- and macroresearch, but there are indeed two different divides. The authors define “divide” as a conceptual and methodological separation in the literature within specific areas. These separations reflect a different focus on the vast economic and social systems where individuals and organizations are embedded. The first divide identified by Molloy et al. (2011) is called a “system level divide.” Within organizational science, researchers from different subdomains (organizational behavior, strategy, entrepreneurship, human resources management, to name a few) have historically focused their attention differently on one of three system levels: individuals and groups, organizations, and economic and social systems. As a consequence, depending on the level of the system that researchers are paying attention to, the operationalization of the micro- and macroconcepts themselves is different. For example, within the subdomain of the management of human resources, macrolevel refers to organizations, broadly defined, and micro levels refer to individuals, dyads, or groups. In the subdomain of strategy, however, micro refers to firms and corporations, whereas macro means industries, regional clusters, strategic groups, and so on. This makes it very difficult to merge or to bridge the different areas of multilevel literature and is related to the problem of defining the focal unit, as mentioned before: If the bridge were based on micro-/macrooperationalization, individuals and organizations would be put in the same basket.
The second divide is called “disciplinary divide.” Molloy et al. (2011) defined the “trinity” of disciplines within organizational research: economics, sociology, and psychology. Each discipline has its own theoretical approaches (how phenomena are viewed and conceptualized), specific methodologies (how phenomena are examined and measured), and particular assumptions. Indeed, there is not a shared epistemology within organizational science, and this leads to differences in the way important phenomena are viewed, conceptualized, examined, and measured. Even when the methodological procedures are similar, researchers from different domains use different symbols. Thus, this difference creates a communicative boundary between micro- and macroresearchers that leads to confusion and misinterpretation (Aguinis et al., 2011). For example, the concept of human capital in psychology is linked to individual differences in cognitive ability and to diverse psychological processes such as learning. For an economist, it is mostly an investment decision and for a sociologist it has to do with someone’s career history and structural prominence of prior employees. For Aguinis et al. (2011), the specialization domains of researchers enhance these divides: Some are micro, such as organizational behavior and management of human resources, while others are macro, such as business policy and strategy, and organization and management theory.
Journals may also be divided by micro (e.g., Journal of Applied Psychology) and macro (e.g., Strategic Management Journal) levels. This division is reflected in the articles’ characteristics, such as their length, acceptance rates, and the number of coauthors per article. Hitt et al. (2007) showed that despite some journals publishing micro- and macrostudies, the integration of these perspectives is a challenge that needs to be met for a greater acceptance of the concept of “multilevel.” Moreover, different researchers perceive the same journal as contrary to their position. Microresearchers perceive a journal as a macrojournal while macroresearchers perceive the same journal as a microjournal (Aguinis et al., 2011). Also, the membership of the Academy of Management distinguishes members between micro- and macroclusters. As a consequence, few members belong to micro- and macrojournals simultaneously (Aguinis et al., 2011). Considering the evidence for the flourishing of multilevel thinking and its observable obstacles, is “multilevel” just a keyword that is in fashion or does it really reflect multilevel thinking, theorizing, measuring, and analyzing?
To illustrate “what is really practiced” in the domain of multilevel research, we conducted a literature review to analyze whether researchers are really doing multilevel research and whether empirical studies or theoretical proposals are, in fact, multilevel—or whether they only intend to be multilevel.
Method
Sample and Procedure for Data Analysis
The literature review we performed was broad-based, but not a systematic one (we excluded proceedings and unpublished works and we analyzed the abstracts). We conducted the search on the “ISI Web of Knowledge” dataset, restricted the search to the “Business” and “Management” web of science categories, and limited the search to a 10-year period: from 2001 to 2011. We used the term multilevel and the topic “title.” With these criteria we obtained 141 articles. After reading the abstracts of all articles, nine articles were excluded because they were not related to multilevel definition (two examples are Frykfors & Jonsson, 2010 and Xiao, Kaku, Zhao, & Zhang, 2011). In the end, we identified 132 articles to analyze. Each article was classified based on its abstract. In a first step, the abstracts were equally divided among the raters and were classified individually. The abstracts were then subjected to a second blind analysis to check whether there was agreement on the classification. When there was no agreement among the raters, the abstracts were read and discussed in team meetings until all the authors reached an agreement.
First, articles were classified as conceptual or empirical. Within conceptual papers, each article could be classified as “theoretical” (a new model and/or propositions about specific topics), or as “research methods” (methodological developments, discussions about current and new methodologies, and their application in specific settings). Empirical papers were classified either as “single-level,” “cross-level,” or “homologous multilevel,” according to Kozlowski and Klein’s (2000) proposal. 1 We added a new possibility of classification a posteriori that emerged from the data analysis: When the researchers considered time as a level of analysis, we classified those empirical research articles as “time” (see Table 2).
Classification of the Abstracts.
Results
The results show that 43% of the abstracts correspond to empirical cross-level studies (see Figure 1). A relevant percentage of abstracts correspond to conceptual papers (40%): theoretical articles (23%) or conceptual research methods (17%) about multilevel. Only 4% correspond to a homologous multilevel model.
Types of “multilevel” articles.
Levels of Analysis
To deepen our understanding about what is really practiced by researchers, we performed another analysis: We enumerated which levels of analysis (individual, team, organizational, and industrial) were included in the empirical studies. Regarding the empirical single-level articles (n = 15), the majority (67%) were studies that analyzed the individual level, and 13% considered the team level; the remaining 20% analyzed other levels. In homologous multilevel studies (n = 5), we found that three studies included all levels: individual, team, and organizational levels. One focused on Multilevel Confirmatory Factor Analysis, studying individual and team levels and the other one analyzed individual and organizational levels. All studies that addressed time (n = 2) examined the individual level. Within the empirical cross-level articles (n = 57), the majority (47%) were studies that analyzed individual and team levels. A percentage of 19% studied individual and organizational levels and 7% referred to variables at team and organizational levels. In a lower quantity, organizational and country levels were studied in 5% of the papers and 3% focused on organizational and industrial levels.
Journals, Citations, and Years
We also considered the journals in which articles were published, the number of times cited on the Web of Science, and the year of publication. Sixteen articles were published in Organizational Research Methods (Impact Factor5years [IF5years] = 5.366), 10 in the Academy of Management Journal (IF5years = 10.565), 9 in The Leadership Quarterly (IF5years = 4.295), as well as in Small Group Research (IF5years = 1.582). Several journals only published one article between 2001 and 2011 (e.g., Human Relations, Human Resource Management Review, Journal of Business and Psychology). A closer look at the results clarifies what is happening in multilevel research. Figure 2 depicts the Journals with the most “multilevel” articles and the types of articles published, according to our classification.
Types of articles published in the Journals with more “multilevel” publications.
Considering the journals with the highest number of publications (n = 69), 33 were conceptual articles (16 theoretical and 17 on research methods). Only four reflected homologous multilevel empirical research (with one being a Confirmatory Factor Analysis), with the majority (27) presenting cross-level studies. Finally, five articles had single-level empirical analysis, despite the word “multilevel” in the title. The 132 “multilevel” articles were cited, on average, 19 times (SD = 28.90); however, 17 articles have not been cited yet. Five articles (Aguilera, Rupp, Williams, & Ganapathi, 2007; Hitt et al., 2007; Lapointe & Rivard, 2005; Liao & Chuang, 2004; Taggar, 2002) were cited more than 100 times. These five most cited articles were published in journals that are in the “top 3” of the management ranking (on the Web of Science): Academy of Management Review, the first one in the ranking (IF5years = 11.442; Aguilera et al., 2007), the Academy of Management Journal, the second (IF5years = 10.565; Hitt et al., 2007; Liao & Chuang, 2004; Taggar, 2002), and the Mis Quarterly, the third in the ranking (IF5years = 7.497; Lapointe & Rivard, 2005). Fifty-six articles were cited between one and 20 times, 25 were cited between 20 and 50 times, and eight were cited between 50 and 100 times. Within the most cited “multilevel” articles, two had a conceptual nature and the remaining three presented cross-level empirical research. None reflected empirical research using a homologous multilevel model. The two most cited “cross-level” articles were, indeed, cross-level studies.
In 2001, the year Kozlowski and Klein’s book was published, three “multilevel” articles appeared on the Web of Science and in 2002 the number increased to 10. So far, 2011 was the most productive year for “multilevel” as 30 articles were published (see Figure 3).
Temporal evolution of the number of “multilevel” articles published in the last decade.
Discussion
Multilevel research seems to be equally preached and practiced. Indeed, there are almost as many conceptual papers (i.e., theoretical and research methods) as empirical cross-level ones, reflecting the large number of theoretical proposals that are “preached.” Researchers believe it is important to develop multilevel research, and tend to theorize about it. Empirically, researchers mainly conduct investigations that analyze the relationship between variables at different levels of analysis—cross-level models—and mainly study top-down influences between teams and individuals. Research using homologous multilevel models is scarce. Finally, the least practiced multilevel methodologies seem to be considering time as a variable and multilevel confirmatory factor analysis. Authors might choose other techniques rather than multilevel modeling to access changes over time (e.g., growth modeling, repeated measures).
Biology suggests that facultative mutualism between species occurs when two individuals interact with each other for mutual benefit, with no real need to do so (Odum, 1971). Our findings lead us to propose that a similar effect is happening with multilevel theory and cross-level empirical research. However, the growth of multilevel research is mainly due to empirical studies with cross-level models. Therefore, multilevel “grows” in publications and in public recognition with the development of cross-level studies. On the other hand, cross-level research benefits multilevel thinking, as it must rely on models that consider relationships between two or more levels. In fact, it seems that researchers have been using the concepts of multilevel and cross-level almost as if they are completely interchangeable. However, for the sake of conceptual rigor, researchers must be more cautious with the use of the words multilevel and cross-level. Indeed, a multilevel study can be more than a cross-level study and cross-level models are by no means the only true multilevel ones (an alternative example is multilevel homologous models).
The number of so-called “multilevel” papers that focus solely on one level of analysis suggests that there is still some confusion regarding the difference between multilevel thinking (considering influences from upper and lower levels theoretically) and multilevel research (actually modeling the relationships between variables at different levels of analysis and measuring and statistically analyzing them accordingly). Multilevel thinking is not absent within the academic community. What is, perhaps, lacking is the operationalization of that multilevel thinking in more research that actually converts an encompassing vision of organizations in empirical studies. Aguinis et al. (2011) argued that there is also a science-practice divide. In accordance with the authors, the practitioners are interested in solving problems from all levels of analysis and sometimes researchers explore only one level. When this occurs, practitioners believe that the research produced is not relevant and cannot help them. To contribute not only to the understanding of organizational variables and their statistical relationships, conducting multilevel research would help bridge the gap between the science and the practitioner’s communities. Looking at the organizational reality from a practitioner’s viewpoint, with the typology of multilevel models in mind (Kozlowski & Klein, 2000), one can ask what would be more important: Knowing if the effect of one variable on another means the same thing across levels of analysis or understanding the effect of an important organizational, industrial, or company-level variable on a team or individual-level variable? According to the research being done, researchers apparently consider that the latter is more relevant. Yet, the few numbers of studies may be a consequence of the complexity in doing so.
Considering the amount of “multilevel” studies carried out in the last decade (n = 132) in the business and management areas, we can assume that, despite multilevel research being advocated by many researchers, it is not yet a very common practice. Even influential business and management journals are not explicitly asking for multilevel contributions. When we analyze journals’ aims and scope, we can see that some important journals in Business and Management (such as the Journal of Management, the Academy of Management Review, the Journal of Business Economics and Management) do not clearly mention that they intend to publish multilevel contributions. Other journals refer to levels of analysis, mentioning that they are “not tied to any particular discipline, level of analysis, or national context” (Academy of Management Journal); that they publish research about psychological phenomena that can be “at one or multiple levels—individuals, groups, organizations, or cultures” (Journal of Applied Psychology); or that “The journal will focus on research and theory in all topics associated with organizational behavior within and across individual, group and organizational levels of analysis” (Journal of Organizational Behavior). Nonetheless, the majority of multilevel articles are published in these top journals. It seems that journals with a history of having high-quality research standards are more open to complex studies.
In spite of the fact that a strong movement toward the development of multilevel theories and knowledge exists, much is yet to be done and various problems still have to be solved before multilevel can progress. In the following section, we outline some ideas that we hope may contribute for the substantiation of multilevel studies.
Roadmap to a Meso Paradigm—Many Challenges, Some Possible Answers
Mathieu and Chen (2011) argued that despite the multilevel paradigm being alive and well, it is also faced with some challenges. However, these can represent opportunities for the field to continue to evolve, if properly addressed. There is still much room for enhancing the possibilities of developing and conducting serious and valid studies with a more integrated approach. Some authors (House, Rousseau, & Thomas-Hunt, 1995) refer to this integrated science of organizations with the term meso, implying that organizational science is simultaneously macro and micro. As previously stated by other authors, we believe that the solution may indeed lie in meso thinking. Once we start assuming that companies and organizational systems are complex systems, constantly changing and interacting with outside systems (e.g., the market) and inside systems (e.g., departments), it becomes clear that a useful way for us to research in a multilevel scenario is to think and to do meso.
Level of theory
One first challenge concerns the existing models about organizational theory. Some authors have already identified aspects that may contribute to closing the gap between micro and macro and between the science-practice divide. Molloy et al. (2011) and Hitt et al. (2007) recommended that researchers focus on the real-world phenomena faced by practitioners, integrating their knowledge and promoting/facilitating multilevel thinking in organizational professionals. As those who are in the field do not analyze reality by thinking abstractly about levels or disciplines, they are likely to be able to describe important multilevel phenomena that researchers, embedded in their own disciplines and levels, may not be able to conceive. However, as most worldwide leaders and managers have not been trained to think “multilevel,” they lack the awareness that several variables at different levels may establish interactions that will lead to new arrangements of systems, and influence organizational dynamics. Academia should also provide students (future professionals and researchers) with the appropriate mind-set to think (with theories) and act “multilevel” (with tools and statistical analysis).
Level of measurement
The measurement processes/techniques/instruments present another considerable challenge for “multilevel”: there are no clear guidelines about the steps necessary to validate the transition of a construct that exists at one level of analysis to another level of analysis. Moreover, even when there is an attempt to do it, researchers tend to aggregate individual answers at higher levels. However, there are concepts that only make sense at the individual level, and the way they are measured at the individual level does not reflect the higher level. Studying a higher level construct is not just a question of methodology or data analysis, but is essentially a theoretical one. Collective constructs that are driven from individual level must have a solid theoretical base that supports their existence.
Level of analysis
Multilevel theories and methods assume that units are perfectly nested within higher levels, but this situation is not always true, considering the complexities of most organizational contexts. Thus, analytic techniques should consider the nonindependence of nesting arrangements, namely network approaches and qualitative data for further generating appropriate quantitative data. There are also important limitations in software development and an absence of mathematical principals and mature software instruments. Indeed, there are still current software limitations in making the model specification of some conceptual multilevel models and common metrics for model assessment. As multilevel analysis is complex and highly sensitive to internal and external variation and until the development or integration (from other sciences) of new mathematical procedures is possible, statistical procedure may be a limitation itself for effective multilevel analysis, making it very difficult for multilevel theory and methodology to progress.
The time variable
Finally, multilevel temporal issues challenge researchers to model nested and longitudinal relationships. This becomes more important as some units may change their higher level membership over time. So, future multilevel research should also address temporal elements. According to Rousseau (2011), simulations are extremely useful to study effects over time, as they reduce the typical complications associated with longitudinal designs and are being successfully adopted by some authors (Mathieu & Schulze, 2006; Santos & Passos, in press).
Once Multilevel, Always Multilevel?
One important final consideration is necessary. Despite the importance and the advantages of multilevel thinking, we should not be carried away by its enthusiastic developments and assume that we should always conduct multilevel research. In fact, in some situations, multilevel is not at all suitable. As Simon (1973) stated, even in real-world organizations, what happens at lower levels (like departments) is often ignored by the higher levels (e.g., CEO), except on some occasions (for instance, when the department misses a deadline or goes over the stipulated budget). In this sense, in some situations, researchers would benefit by focusing their study on parts of the organization instead of the whole. Mathieu and Chen (2011) have also explored this issue and wondered whether all research has to be multilevel. In accordance with the authors, it is valuable to adopt a bracketing strategy, meaning to include constructs of one level lower and one level higher in the conceptual and empirical analysis. So, researchers should justify why they selected specific variables from one level and why they excluded others.
In short, researchers not only have to worry about how to do multilevel research (and deal with the associated problems) but also about when to (or not to) do it. Before conducting a multilevel study, researchers need to take some aspects into account. First, they should only conduct a multilevel study when it will make a significant contribution within a given theoretical field. Moreover, researchers should only conduct a multilevel analysis when theory supports the multilevel relationship. If theory does not support it, they should change the variables, hypotheses, or redo the literature review. Finally, the researchers need to analyze whether there is appropriate theoretical work, methodological procedures, and instruments to conduct a multilevel analysis, and, only after that, proceed with their work. Otherwise, instead of contributing to accurate and useful knowledge, unregulated and ill-conceived multilevel practices will lead to inaccurate theory building.
Conclusion
The present study was conducted using a limited time range (2001-2011) and also restricted the search to the “Business” and “Management” web of science categories. Therefore, the results must be interpreted considering these limitations and may not be generalized to other knowledge areas. Moreover, we limited the keyword search to the article’s title and analyzed only the abstracts, not the full text, which may have led to missing some relevant information.
However, within our research criteria, it is clear that nowadays, 12 years after Kozlowski and Klein’s (2000) initial introduction, no one can say that “multilevel” is not alive and well: Multilevel issues are definitely experiencing an interesting moment, as the attention in this kind of methodology seems to be growing. Many researchers are committed to exploring the potential of multilevel research, as well as its limitations and weaknesses, or simply reinforcing its practice. However, it is assumed that there are some problems and challenges to be solved and some bridges to build to achieve an effective multilevel theory and practice. Our analysis demonstrates that researchers recognize the importance of multilevel research, but articles on conceptual models are almost as numerous as empirical ones. Moreover, the majority of the empirical papers focus on one specific type of multilevel model—cross-level models—but more research is still to be done in other kinds of multilevel models. Nevertheless, more problematic is the existence of single-level research under the definition of “multilevel.”
If we want to intervene and apply our conceptual models to real organizations, that are concrete multilevel systems, it will require more than small fragments of problems/phenomena. We need research that focuses on the dynamics between levels of observation and that unfold over time to understand how the different subsystems within organizations interact and evolve.
Footnotes
Acknowledgements
The authors thank Ana Raquel Soares for her contribution on the development of early versions of this article. The authors also thank Pedro J. Ramos-Villagrasa for his helpful comments in preparing the manuscript.
Authors’ Note
Equal authorship for the first four authors, names listed alphabetically.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a Project [Ref. PEst-OE/EGE/UI0315/2011] from the Portuguese Foundation for Science and Technology.