Abstract
A multitude of studies in the management literature are focusing on within-person phenomena. The study of such phenomena offers great promise as within-person research facilitates the capacity to enhance temporal precision, show change over time, and reveal the kinds of novel insights that are not possible if relying solely on a traditional between-person perspective. Drawing on the features of within-person research that comprise its unique value proposition, we conduct a quantitative and narrative review of within-person studies to ascertain the degree to which these studies are maximizing the contribution and impact that they can make to the field of management. We pose three research questions that we present as a holistic framework for assessing the contributions of within-person research. To answer our questions, we synthesize across studies and analyze variability data, correlational data, and researchers’ hypothesizing to show (a) the degree to which hypotheses in within-person studies incorporate temporality; (b) the differential within-person fluctuation and variability that exists based on construct, theoretical, and measurement-related factors; and (c) the degree to which within-person relationships are different from equivalent between-person relationships. While our data and conclusions offer insight into the contributions being made by the within-person literature at large, we also propose that our framework can be used at the individual study level of analysis to help optimize the contributions made in future within-person research.
Well over two millennia ago, Heraclitus argued that the only thing constant is change (Petit, Hu, & Dick, 2008). Indeed, over the course of a workday, employees experience fluctuations in their moods, situations, and behavior. The inquiry into employees’ varying experiences offers promise as a means to facilitate understanding about dynamism and change, the next evolution in management scholarship (Ployhart & Vandenberg, 2010). Management researchers have long recognized that time impacts nearly all organizational phenomena, yet even though individuals’ changing experiences over time are incredibly important, they are exceedingly difficult to study and assess (Bluedorn & Denhardt, 1988). Carrying the potential to address this problem, studies focused on within-person phenomena are ideally suited to make scientific contributions. Recently, research efforts have accelerated in this regard as hundreds of studies have focused on within-person phenomena. Called the “gold standard” for assessing people’s experiences in daily life (Kahneman, Krueger, Schkade, Schwarz, & Stone, 2004: 1777), these within-person studies have been labeled ecological momentary assessment (Beal & Weiss, 2003), event sampling (Elfering et al., 2005), diary studies (Heller & Watson, 2005), the daily diary method (Ferris, Spence, Brown, & Heller, 2012), everyday experience methods (Reis & Gable, 2000), or the experience sampling method (ESM; Beal, 2015). Regardless of the descriptor used, these studies feature repeated longitudinal measurement of constructs in context and over short periods of time.
Conventional, between-person research shows how individuals differ from one another and has not fully addressed time (Ancona, Goodman, Lawrence, & Tushman, 2001; Monge, 1990; Pitariu & Ployhart, 2010; Sonnentag, 2012). As such, predictions based on between-person relationships (e.g., employees of abusive bosses have lower performance than employees of effective bosses) are qualitatively different from predictions based on within-person relationships that account for changes over time (e.g., a given employee’s performance fluctuates daily in response to his or her boss’s behavior that day). Most phenomena fluctuate over time (Dalal & Hulin, 2008), so static research paradigms that do not account for change fail to capture complex and dynamic states, behaviors, and situations (George & Jones, 2000). This time-invariant perspective can inhibit research results, misrepresent reality, and limit the development of a comprehensive body of management knowledge (Dalal & Hulin, 2008; Ployhart & Vandenberg, 2010; Wickham & Knee, 2013). A focus on within-person variability offers a different paradigm for understanding organizational phenomena (Curran & Bauer, 2011; Dalal, Bhave, & Fiset, 2014; Fisher & To, 2012; Uy, Foo, & Aguinis, 2010), emphasizing that individuals are not static entities but, rather, that their states, behaviors, and environments change over days, hours, or even from one minute to the next. Revealing how management phenomena unfold over time, within-person research offers promise as a means to facilitate contributions that enhance temporal precision, elucidate dynamic phenomena, and provide novel insights about constructs and their relationships with one another that are not possible with a between-person perspective.
Although within-person studies are ideally suited to contribute to the understanding of change that occurs over time, it is possible for researchers to employ within-person tools and techniques without fully capitalizing on the opportunity to bring about new ways of understanding organizational phenomena. The problem is that although within-person techniques can play a powerful role in generating paradigm-shifting research (Colquitt & Zapata-Phelan, 2007), within-person studies can also be applied in ways that only incrementally advance research on management (e.g., by answering static questions, studying stable constructs, or mirroring between-person findings using a within-person design). Within-person studies can offer utility solely as a methodological tool (e.g., for reducing retrospective bias or increasing ecological validity; Scollon, Prieto, & Diener, 2009), but they also enable the potential for greater insights about change over time. Within-person research can be invasive and costly (Ilies, Aw, & Lim, 2016), and the resources it requires are more justified when it results in scientific contributions that significantly advance the field of management (Sternberg, 2006). For this reason, there is a need for a broad assessment of where and how within-person research most significantly changes scholars’ understanding of management phenomena.
In this study, we assess the impact of within-person research on management scholarship by connecting the literature on scientific contributions (e.g., Colquitt & Zapata-Phelan, 2007; Sternberg, 2006) to the core capabilities facilitated by a within-person perspective (e.g., Beal, 2015). Focused on the elements of within-person research that comprise its unique value proposition—the capacity to enhance temporal precision, show change, and reveal novel, within-person insights—we conduct a narrative and quantitative synthesis of the within-person literature to answer the following questions: To what degree are within-person researchers posing questions that facilitate temporal contributions? To what degree do constructs change over time, necessitating a within-person perspective? Does studying change over time offer new understanding relative to that which can be gained through between-person research? On the basis of our review, we propose that the impact of within-person research is most significant when researchers (a) incorporate time into their theorizing, (b) expose substantive variability that supports conceptual premises about constructs’ interrelationships, and/or (c) offer findings that extend beyond those that could be procured with between-person research.
Our study offers contributions that are notably different from the excellent resources about how within-person research should be carried out (e.g., books or narrative reviews; Beal, 2015; Bolger, Davis, & Rafaeli, 2003; Hektner, Schmidt, & Csikszentmihalyi, 2007; Ohly, Sonnentag, Niessen, & Zapf, 2010). Given the unique strengths of a within-person perspective (Bolger et al., 2003; Uy et al., 2010), we focus our contributions on when and how the study of within-person phenomena most advances the field of management. First, we identify a critical gap that can limit the contributions of within-person research, namely, that many within-person hypotheses do not offer the capacity to increase temporal precision. Without hypothesizing about time and change, a within-person study misses an opportunity to make a more significant contribution. Second, we show how construct variability, a core assumption of temporal theoretical models that describe the causes and effects of phenomena (e.g., George & Jones, 2000; Mitchell & James, 2001), differs from construct to construct and across moderators. These results about construct variability serve as a basis for guiding future theory building. Third, we uncover the extent to which within-person construct relationships are distinct from between-person construct relationships; for instance, for engagement and performance, is the between-person relationship (employees who are generally more engaged perform better than those who are less engaged) the same as the within-person relationship (employees perform better on days in which their engagement is greater)? This is a critical consideration as it highlights the need for precise theory that clearly delineates conceptual rationale specific to different levels of analysis, and it can also help determine whether organizational efforts should target training and development initiatives versus screening and selection procedures.
Within-Person Research and Contributions
The magnitude of a study’s contribution encompasses how impactful the study is to the field of management, and a literature exists that explains the importance of contributions (Bergh, 2003; Colquitt & Zapata-Phelan, 2007; Corley & Gioia, 2011; Rynes, 2002; Seibert, Kacmar, Kraimer, Downes, & Noble, 2017; Sternberg, 2006; Whetten, 1989). Whether researchers and reviewers laud, repudiate, or show indifference toward a study is largely determined by their assessment of the magnitude of the respective study’s contribution. 1 In our study, we seek to assess the degree to which the study of within-person phenomena has capitalized on its promise for making large, transformative contributions that propel the field of management forward.
Drawing from the distinctive characteristics of within-person research (Bolger et al., 2003; Uy et al., 2010), there are three value propositions that position within-person studies to make important contributions: enhancing temporal precision, showing variability over time, and providing novel insights. Accounting for the fact that “many, perhaps even most, research questions . . . are in reality within-person questions” (Dalal et al., 2014: 1399), within-person techniques facilitate the chance to add value to the field through the capacity to ask and answer substantively different research questions (Hamaker, 2012) about within-person fluctuation that takes place over time. Thus, the first value proposition of within-person research centers on its potential to contribute by improving temporal precision in the study of management phenomena. Enhancing temporal precision of theory provides new insight into how constructs relate to one another (e.g., Boswell, Shipp, Payne, & Culbertson, 2009; Fuller, Stanton, Fisher, Spitzmueller, Russell, & Smith, 2003).
The second value proposition for within-person research resides in its ability to elucidate within-person variability, which can guide researchers to new insights. Understanding the degree to which phenomena vary over time is central to theoretical models that describe how constructs should be expected to affect one another over time (George & Jones, 2000; Mitchell & James, 2001). Without specifying the timing of when causes lead to effects, theoretical explanations are incomplete (Gabriel, Diefendorff, Bennett, & Sloan, 2017; Shipp & Cole, 2015). The third value proposition is that within-person research can reveal novel information and enable a new understanding of workplace phenomena because the relationships between constructs can be different at the between-person level than at the within-person level (Curran & Bauer, 2011; Dalal et al., 2014; Dalal & Hulin, 2008). This point has been articulated using a number of compelling anecdotes; for example, Hamaker (2012) suggested a negative between-person relationship between typing speed and typing errors (i.e., people who type faster make fewer errors because they are more proficient) and a positive within-person relationship between typing speed and errors (i.e., a given person makes more errors as his or her typing speed increases).
Using these three key value propositions as a framework, we assess the scientific contributions of within-person research. Considered in isolation, each of these value propositions offers incomplete insight and is limited in its capacity to detect contribution. When the three elements of our framework are applied in concert with one another in a holistic way, they constitute a powerful way to ascertain the magnitude of contribution. While any given study’s contributions are nuanced and unique (Rynes, 2002), and the assessment of any single study’s contribution is a highly complex endeavor requiring intimate knowledge of a research stream, we propose that researchers and reviewers can leverage the three propositions of our framework to gain valuable insight into the potential contribution of individual within-person studies.
Facilitating Contributions With Time-Based Hypotheses
A fundamental strength of within-person research is that it can make contributions by enhancing the temporal precision of constructs’ relationships. Time has been incorporated into organizational research in many ways, and there are a number of theoretical perspectives related to time (Gersick, 1988; Mosakowski & Earley, 2000). In within-person research, time is primarily viewed as an underlying basis for change (Shipp & Cole, 2015). In this perspective, time does not cause change, and employees are not interpreting or reacting to time. Instead, time is an ongoing context (Johns, 2006) for understanding when one construct changing will lead to another construct changing. This aligns most closely with arguments that theories should specify when—in time—causes lead to effects (George & Jones, 2000).
There are many ways constructs can be related to one another (Mitchell & James, 2001). To see whether the questions posed in within-person research facilitate meaningful contributions to the understanding of change over time, we examine the hypotheses in within-person studies as these hypotheses are valuable in (a) specifying expectations related to organizational phenomena, (b) revealing the sophistication of research questions posed, and (c) guiding the decisions made during the process of conducting research (Aguinis & Vandenberg, 2014; Colquitt & Zapata-Phelan, 2007; Edwards, 2010; Schmidt, 1992). Because they are relatively consistent in form and highly specific in nature (Edwards & Berry, 2010; Pitariu & Ployhart, 2010), hypotheses reveal whether researchers posit within-person questions with temporal components or merely connect phenomena without specifying how their relationships change over time.
We leverage Pitariu and Ployhart’s (2010) framework for evaluating the temporal precision of hypotheses as a basis for contribution. Their framework offers three broad categories (time, shape, and duration) for how the dynamics of change over time can be incorporated into hypotheses. First, incorporating time acknowledges that a dynamic relationship exists and offers more temporal precision than ignoring time altogether. Second, duration emphasizes the idea that a dynamic relationship may change in magnitude over time. Finally, shape refers to a broad class of specifications for how a relationship between two constructs changes over time (e.g., curvilinear effects; see also trends, cycles, spirals, rhythms, etc.). Pitariu and Ployhart argue hypotheses exist along a continuum of temporal precision, offering a more meaningful contribution and greater theoretical advancement as they specify more temporal components. 2
Research Question 1: To what degree do within-person hypotheses incorporate temporal elements?
Facilitating Contributions by Elucidating Within-Person Variability
The most promising contributions of within-person research (e.g., illustrating cross-level interactions; showing within-person change; Uy et al., 2010) are predicated on within-person variability—the idea that people’s states, behaviors, and situations change over time (Scollon et al., 2009). Even though evidence about within-person variability is accumulating, unanswered questions remain, carrying the potential to limit the contributions of within-person research (e.g., by narrowing researchers’ questions or the methods used to investigate those questions). First, researchers have argued that almost all phenomena fluctuate over time (Dalal & Hulin, 2008; Roe, 2014) and that “even the most static of constructs are, in truth, quite variable over time and across situations” (Beal, 2012: 615). Variability over time is a critical minimum bar to clear when evaluating the need to study within-person phenomena, yet researchers have not examined variability on a large scale. Second, some constructs may exhibit greater variability than others when examined through a within-person lens. Thus, we seek to elucidate differential variabilities that may exist across different kinds of constructs, which could open avenues for future contributions to the field. Third, the multitude of study design decisions facing within-person researchers are often not discussed (Fisher & To, 2012; Shrout & Lane, 2012), raising a question about the degree to which observed within-person variability is substantive versus methodological. Could researchers’ choices systematically influence the observed within-person variance (WPV)? A quantitative synthesis of within-person variability would answer this question.
Construct variability is critical to understand because it underlies temporal theoretical models that specify the causes and effects of phenomena (George & Jones, 2000; Mitchell & James, 2001), and scholars have described how the effect that one construct exerts upon another takes some time to emerge, is stable for a time, and then decays (Mitchell & James, 2001). Rooted in this notion, researchers are now devoting significant conceptual attention to theorizing about within-person change over time (e.g., Methot, Lepak, Shipp, & Boswell, 2017). An understanding of how constructs change is fundamental to explaining time-bound relationships (e.g., increments, episodes, loops, spirals, cycles, etc.; Beal, Weiss, Barros, & MacDermid, 2005; Shipp & Cole, 2015) because in order to develop temporally precise theory, researchers must have a clear picture of how much constructs change over time.
A clear understanding of construct variability can guide researchers toward the level—between-person versus within-person—at which theoretical explanations are most likely to be valuable to the field of management. A refined view of the levels and time frames at which constructs vary can lead theorists to focus on levels at which causes and consequences may be particularly important, offering direction for designing a study to best capture the timing of causes and effects. From a practical standpoint, stable and variable constructs should be managed differently. When constructs are primarily stable, as demonstrated by high between-person variability, management practices emphasizing choices between individuals (e.g., staffing, team design) are particularly likely to be effective. Conversely, when constructs fluctuate, as demonstrated by high within-person variability, practices emphasizing employees’ states at a particular moment in time (e.g., training or interventions) may have more influence on outcomes.
Despite the potential benefits of more deeply understanding how constructs vary over time, the management literature has not synthesized the data that could examine researchers’ claims about construct variability. This lack of cumulative knowledge to some degree parallels the state of between-person research several decades ago. By quantitatively synthesizing findings across research studies (e.g., through meta-analysis), researchers have made substantial gains in building cumulative knowledge about between-person phenomena (Ones, Viswesvaran, & Schmidt, 2017). Such advances would not have been possible if researchers had solely been conducting individual empirical studies, because any single study is limited (Schmidt, 1992). A similar process focused on variability at the within-person level could be useful in advancing the field, as researchers might identify new insights when the variability of constructs falls outside the range observed in prior research; researchers may even may be able to identify issues that lead to paradigm-shifting contributions. For instance, if a researcher conducting a study of depletion found the within-person variation in depletion to be much higher than that reported in prior research, this finding would suggest the need to further probe the study context (e.g., to see what job characteristics may be fluctuating to change the stimuli employees encounter). This information could lead researchers to reevaluate and build new theoretical propositions.
Of course, these points presuppose that fluctuation in dynamic constructs over time is substantive and not affected by methodological artifacts. Yet, construct variability over time can be due to many sources, including systematic change or random or systematic sources of error (Mitchell & James, 2001). Thus, we also examine whether study-design factors impact construct variability. If so, the implication is that contributions researchers are making (a) may be a function of methods choices, chance, or unexpected conceptual drivers rather than a result of hypothesized substantive properties and/or (b) may be limited by the inability to truly capture and reveal all the substantive change that occurs over time. The quantitative synthesis that we propose could be useful in determining whether unconsidered factors (e.g., sample characteristics, study design features) are influencing construct variability. For example, if a researcher’s study of depletion found that study participants’ within-person variability in depletion was much lower than would be expected based on prior research, researchers may examine study design decisions (e.g., the timing of measurements or duration of the study) to assess the effects of a study design choice.
Assessing Within-Person Variability
Examination of variability parses out the within-person from the between-person variability. When WPV is large, constructs are relatively dynamic and change over time. When WPV is small, constructs are relatively stable and observed variability is primarily between—rather than within—individuals. For instance, Johnson, Lanaj, and Barnes (2014) studied managers’ organizational citizenship behavior (OCB), showing that 34% of the difference in OCB occurred within managers, while 66% of the difference in OCB occurred between managers. The upshot is that the majority of the difference in the OCB a manager performed was related to how that manager behaved relative to his or her peers—not based on the manager’s own fluctuating behaviors.
Because future research is likely to consider new constructs that are the focus of within-person studies, an analysis of higher-level categories of constructs may be beneficial in guiding future scholarship. For the higher-level analysis, we draw from sociocognitive theory (Bandura, 1986), which posits interrelationships between individuals, their behaviors, and their contexts. Using this sociocognitive lens, within-person constructs can be classified as individuals’ states (i.e., cognitive, affective, or biological personal experiences), their behaviors (i.e., actions individuals take to navigate their environment), or their situations (i.e., environmental conditions that influence and are influenced by actors). States (i.e., affect; emotion) are often described as being dependent on time and should thus exhibit considerable within-person variability (Weiss & Cropanzano, 1996); our expectations are similar for situations, which lie partially outside individuals’ control. Conversely, behavioral constructs are often rooted in more stable individual differences and should be less variant over time (Fleeson, 2001; Mischel & Peake, 1983). Even more specific than a categorical analysis of variability, there is also value in exposing the degree of within-person variability that exists for different constructs.
Assessing Moderating Factors
Researchers seeking to elucidate within-person phenomena are faced with a number of decisions pertaining to data gathering and study implementation (Stone & Shiffman, 2002). We investigate whether observed within-person variability varies as a function of some of these most critical decisions, including (a) conceptual factors (i.e., data collection technique and passage of time between measurements) and (b) measurement factors (i.e., number of items used to assess the focal construct, total measurement occasions, and study duration). It is important for researchers to be aware if choices made while conducting within-person studies affect the amount of WPV they observe.
It is critical to complement knowledge about how one construct’s impact on another varies over time (Mitchell & James, 2001) with an understanding of how a given construct (i.e., an employee’s positive affect [PA]) should be expected to fluctuate on its own irrespective of other forces acting on it. To answer this question, we look to a burgeoning field of theory and research that describes how circadian rhythms dictate human physiology, resulting in consistencies in people’s alertness, reaction, coordination, and sleepiness over the course of 24-hr cycles (i.e., Aschoff, 1965; Czeisler et al., 1999).
We examined two factors that could facilitate the manifestation of the effects of circadian rhythms. First, we categorized data collection techniques (event contingent, signal contingent, interval contingent; Scollon et al., 2009; Uy et al., 2010). Whereas event- and signal-contingent data collection require participants to describe their states, behaviors, and situations at varying, unpredictable times in a circadian cycle (e.g., when a beeper prompts them or when a specific event occurs), an interval-contingent data collection technique requires participants to describe their states, behaviors, and situations at consistent times (i.e., every day at the end of the workday). Because interval-contingent studies measure respondents at consistent points in their respective circadian cycles, WPV may be reduced.
A second means of assessing circadian effects pertains to the passage of time between measurements (e.g., how much time passes between each time a respondent’s stress levels are assessed). Given the consistency that people exhibit at similar times in their circadian cycles, we wondered whether construct measurements taken within 24-hr periods would exhibit more WPV than would construct measurements taken in intervals of 24 hours apart or more (e.g., 24 hr, 48 hr, 72 hr, etc.). Beyond the theoretical importance of considering the timing between measurements, there are also methodological considerations. Specifically, eliminating retrospective bias is a key strength of within-person research (Beal, 2015), so it is important to know if this strength is negated by a longer passage of time between measurements. Bolger et al. (2003) noted that within-person researchers cannot capture change with the same fidelity when longer intervals occur between repeated measures. If lengthening the time between survey administrations lowers fidelity and increases retrospective bias, it would seem that study participants would be more likely to report a stable, average level of a construct, thus increasing between-person variability and decreasing within-person variability.
Measurement factors comprise the other moderator dimension because within-person reviews have identified such factors as critical considerations (i.e., Scollon et al., 2009). We evaluated one factor that occurs within each measurement occasion (number of items used to assess a construct) as well as two factors that take place across measurement occasions (total number of assessments and study duration). Ohly et al. (2010) recommended limiting the number of items used to assess a construct when studying within-person phenomena, describing how increasing the number of survey items that participants are asked to complete could impact their responses. As such, we examined the total number of items used to assess a construct as a way of investigating whether fewer scale items impacts within-person variation. Bolger et al. (2003) described how within-person researchers need to consider how frequently to assess constructs, and Ohly et al. noted some of the drawbacks that occur as the number of assessments increase. Thus, we examined the number of assessment occasions to assess its impact on observed within-person variability. Scollon et al. (2009) identified a concern over within-person studies that go on too long, and Stone, Kessler, and Haythornwaite (1991) described how within-person studies that endure for more than 2 weeks can inhibit the quality of responses. Thus, we investigated whether within-person studies’ duration impacts WPV.
Research Question 2: To what degree do constructs exhibit variability over time?
Facilitating Contributions Across Levels
A basis for studying within-person phenomena is uncovering unique insights that could not be derived from between-person research. A within-person perspective treats the within-person variability that characterizes individuals’ changing experiences over time as meaningful and substantive versus being interpreted as error as it is in between-person research (e.g., Scott & Barnes, 2011). Between-person perspectives can be deficient in their explanations (Dalal et al., 2014), and many theories in management are, in reality, within-person theories (Dalal & Hulin, 2008). In short, one reason studying within-person phenomena can lead to new insights is because the causes and consequences of constructs at the between-person level may be different from those within person. As an example, the correlation matrix from Scott and Barnes (2011) shows a large negative between-person relationship between employees’ PA and their work withdrawal. Yet, there is a nonsignificant relationship between PA and end-of-day work withdrawal at the within-person level. This pattern of correlations indicates that although people who are usually higher in PA are less likely to withdraw from work, a person being in a positive affective state has no significant effect on that person’s likelihood of withdrawal later in the day.
Yet, there are other times in which homology exists between levels, suggesting that within-person and between-person construct relationships may be equivalent (Kozlowski & Klein, 2000). For instance, Ilies, Dimotakis, and De Pater (2010) showed moderately strong positive relationships between workload and affective stress at both the within-person and the between-person level. In this study, there exists a homology of theory: Individuals with greater workloads experience more stress than individuals with lesser workloads (between person), and a given individual experiences more stress on days with a greater workload (within person). Because the expected relationships are the same at both levels of analysis, a within-person study of these constructs does not fundamentally change scholars’ understanding of how workload and stress are related. When within-person relationships parallel between-person relationships, it becomes important to focus on other study aspects in order to highlight the need for the within-person study. For instance, researchers might specify how the timing or theoretical processes (i.e., mediators) differ across levels. Several scholars have suggested homologies are unlikely in the field and that similar within-person and between-person relationships are merely coincidental if they exist at all (Beal, 2015; Cervone, 2005; Dalal, Lam, Weiss, Welch, & Hulin, 2009; Ostroff, 1993; Uy et al., 2010). Yet the question of how often within-person and between-person construct relationships are equivalent has not been empirically examined.
Research Question 3: To what degree are a study’s within-person construct relationships equivalent to a study’s between-person construct relationships?
Method and Results
We examined our research questions through a quantitative review of within-person studies using search terms including experience sampling, event sampling, momentary assessment, diary study, everyday experience methods, and daily diary in either the PsycINFO or Business Source Complete database. 3 Given the scope and purview of Journal of Management, we retained articles that included either employee samples or at least measured constructs relevant to the workplace. This process resulted in 378 studies. 4 Indicative of the explosion in the popularity of within-person research, 80% of the studies were published since 2005.
We initially coded the hypotheses, constructs assessed, moderating factors, and the variance components reported for each study. Two of the study’s authors completed the initial coding, agreeing 89.7% of the time at the end of their calibration exercise. The two coders addressed discrepancies until reaching consensus in order to ensure consistency going forward. Later rounds of coding took place as well: the study’s second author recoded studies for additional moderating factors, a graduate assistant recorded the correlation coefficients in the studies that reported such data, and a graduate assistant coded for reliability coefficients.
Research Question 1: To What Degree Do Within-Person Hypotheses Incorporate Temporal Elements?
A key premise is that study contributions pertaining to time are unlikely to emerge when researchers do not pose time-related questions. We coded each of the 1,684 numbered hypotheses with respect to the presence of time, duration, and shape (Pitariu & Ployhart, 2010). Frequency results for this analysis are presented in Table 1. Supporting the notion that the perspective emphasizes time, we found that approximately 64% of the hypotheses (1,078/1,684) specified a time-based relationship. As a point of comparison, Edwards and Berry (2010) found that fewer than 25% of propositions in articles published in Academy of Management Review provided predictions with conditions such as time. Given this point of reference, within-person research seems to emphasize time to a greater extent than the broader management literature.
Temporal Sophistication of Study Hypotheses in Within-Person Research
As Table 1 reports, 582 of 1,684 hypotheses (34.6%) did not specify any temporal relationships, failing to capitalize on the full potential of studying within-person phenomena. Just six of 1,684 hypotheses (0.4%) specified more than one component of time, shape, or duration. Of the studies that included these temporally precise hypotheses (Grech, Neal, Yeo, Humphreys, & Smith, 2009; Judge & Ilies, 2004; Richard & Diefendorff, 2011; Rodríguez-Sánchez, Schaufeli, Salanova, Cifre, & Sonnenschein, 2011), we highlight one of Grech et al.’s (2009) hypotheses as an exemplar. In their study of the relationship between workload and fatigue, Grech et al. posed the following hypothesis: “The within-person relationship between perceived workload and fatigue changes over consecutive days of shift work. The relationship is negative at the beginning of the work period but becomes positive over the following days” (Grech et al., 2009: 233). As is apparent in the hypothesis, it includes both shape and time. Regardless of whether or not this hypothesis is supported (incidentally, support was found), the question posed by Grech et al. offers promise for improving theory through its increased temporal precision. An encouraging trend we found is that researchers are increasingly incorporating time into their theorizing. Specifically, for studies published prior to 2005, only 39% of the hypotheses included a time element. Yet, 66% of studies published after 2005 incorporated time. It appears researchers are converging on the notion that within-person studies can extend management knowledge by increasing scholars’ understanding related to the temporality of management phenomena.
Research Question 2: To What Degree Do Constructs Exhibit Variability Over Time?
Since study participants are repeatedly assessed on the same constructs in within-person research, variance can be parsed into within-person and between-person components. To facilitate our examination of variability, we relied upon primary studies’ use of the intraclass correlation (ICC) to quantify within-person variability. The ICC identifies the percentage of variance in a construct that is attributable to within-person and between-person differences. We calculated sample-size weighted averages of the within- and between-person variances and also evaluated whether observed within-person variability differed as a function of research design.
Assessing Within-Person Variability
In Table 2, we present the observed WPV averages for the constructs that have been most frequently studied in within-person studies. We also categorized the constructs into states, behaviors, or situations based on sociocognitive theory (Bandura, 1986), and categorizations for the most common constructs are provided in Table 2. Constructs reflecting individuals’ experiences in terms of their attitudes, feelings, and emotions (i.e., PA, negative affect [NA]) were coded as states; actions people engage in (i.e., OCB, counterproductive work behavior [CWB], sleeping) were coded as behaviors; and conditions that exist in people’s environments (i.e., job demands, social support) were coded as situations. Although within-person perspectives are touted as offering rich examination of people’s daily experiences, it seems the focus has been more on changes in the minds of individuals rather than on changes in the events individuals encounter in their lives. This gap in studying situations is particularly important to address given our finding in Table 2 that situations generally exhibited more within-person variability than states (54.53% observed WPV, 95% confidence interval [CI] [51.58, 57.48] vs. 49.34% WPV, 95% CI [47.91, 50.78]). We also highlight the incredible amount of within-person change that takes place regardless of category, revealing how much fluctuation remains unaccounted for in strictly between-person research and justifying scholars’ insistence that addressing change over time is necessary to advance organizational science (Bolger et al., 2003; Gabriel et al., 2017; Shipp & Cole, 2015).
Sample-Size Weighted Averages of Within- and Between-Person Variance for Most Frequently Reported Categories and Constructs
Note: OCB = organizational citizenship behavior; CWB = counterproductive work behavior.
Although the number reported in the Study Participants column reflects the total participants upon which the data are based, some overlap in study participants occurs. A given participant in a study may have had his or her positive affect and engagement measured. In that case, the participant would be counted two times in the States Overall row of the Study Participants column. Similarly, if participants had three different job-demands constructs measured in a given study, they would be counted three times in the Job Demands row of the Study Participants column. Within- and between-person variance may not sum to 100% due to rounding.
We also assessed within-person variability at the construct level, concluding that much within-person change exists across the entire range of constructs represented, which supports scholars’ contention that most constructs fluctuate over time (Beal, 2012; Dalal & Hulin, 2008). Beyond this overall finding, we posed the question of whether constructs would differentially vary over time. As shown in Table 2, we observed statistically significant differences in within-person change based upon the respective constructs being considered. With this finding, we uncovered empirical evidence about construct stability that can be applied to future theorizing and study design. For instance, constructs like sleeping (63.68% observed WPV and 36.32% between-person variance [BPV]) exhibited more within-person variability, while constructs like self-esteem (35.76% observed WPV and 64.24% BPV) exhibited more stability, showing that individuals, themselves, vary considerably in their sleep patterns, while employees’ self-esteem tends to differ more from person to person than from moment to moment.
We also highlight that prior research has shown constructs like job satisfaction and self-esteem to be quite stable over time (Judge & Hulin, 1993; Trzesniewski, Donnellan, & Robins, 2003), and our variability analysis showed these constructs to be among the variable constructs least frequently studied within person. Yet, these two constructs still exhibited meaningful within-person variation (self esteem, 35.76% observed WPV; job satisfaction, 37.65% observed WPV). Thus, meta-analytic research that has linked these constructs at the between-person level (e.g., Judge & Bono, 2001) may need to be supplemented with additional research that illustrates the nature of within-person connections between the constructs.
Another finding relates to WPV differences between PA and NA. Whereas 43.79% of the observed variance in PA is within person (95% CI [40.40, 47.18]), 54.89% of the observed variance in NA is within person (95% CI [50.94, 58.84]). The finding that NA has higher levels of WPV than PA can be particularly valuable for developing theory related to affect. For example, because NA exhibits greater variability over time, it may be that NA is more subject to situational influences than is PA, which varies more between individuals. Also, relevant to situations, interpersonal conflict varied substantially within persons (63.44% WPV; 95% CI [47.57, 79.30]), whereas social support demonstrated more stability (40.08% WPV; 95% CI [34.18, 45.97]). This would suggest that employees’ experience of conflict can vary quite considerably from day to day or from one workplace interaction to the next. Yet, a certain level of social support tends to be stable and endure more over time. Future theoretical development on how social interactions form and change over time may be useful. These specific examples reveal areas to focus on in the literature, and we hope Table 2 provides additional insights that will serve researchers in their respective domains. More generally, questions about the duration of effects are often not asked, and we recommend that researchers should make variability considerations central to the comprehensive understanding of constructs and the nomological networks that surround them.
Assessing Moderating Factors
As shown in Table 3, our understanding of the stability of phenomena over time may be at least partially contingent on study design choices. Given that researchers’ decisions can impact the amount of WPV observed, we next offer findings with respect to the conceptual and measurement-related moderating factors.
Averages of Within-Person Variance by Methodological Decisions, All Constructs
Note: CI = confidence interval.
Data collection techniques. 5
Although Table 3 shows the interval-contingent sampling technique is the most frequently used, constructs studied using interval-contingent sampling exhibited less intraindividual variance (WPV = 48.54%; 95% CI [47.36, 49.73]) than constructs measured with signal-contingent (WPV = 61.07%; 95% CI [58.12, 64.02]) or event-contingent (WPV = 72.86%; 95% CI [67.61, 78.12]) techniques. In line with how circadian rhythms impact human physiology in consistent, cyclical ways, this finding suggests that WPV is minimized to the degree that study participants are completing assessments on a schedule at consistent times. For instance, if an employee’s mood is assessed every day at the end of the workday (interval contingent), that employee may generally be in a positive mood. Conversely, if the same employee is assessed on Monday at lunch, Tuesday just before work, and Wednesday at the end of the workday (signal contingent), then the fluctuating moods experienced may be more apt to manifest in higher WPV in mood.
Time intervals between measurements
The examination of constructs from one workday to the next was the most common strategy deployed by researchers. Thus, in line with the theoretical significance of the 24-hr circadian cycle, we compared constructs assessed with multiple surveys per day against those assessed with longer time windows between measurements. We found that when measurements were separated by less than 24 hr, WPV was significantly higher (WPV = 54.08%; 95% CI [52.43, 55.72]) than when measurements were separated by 24 hr or more (WPV = 47.74%; 95% CI [46.20, 49.27]). The increased WPV that is present when constructs are measured multiple times within a 24-hr period could reflect the fluctuations that people experience in line with human physiological changes that occur naturally due to circadian rhythms. Although this issue can be managed using autoregressive estimation techniques, our broad interpretation is that people’s experiences generally change more within a day than they do between days.
Number of items used to assess the focal construct
Within-person survey administrations sometimes assess constructs with fewer items than are typically used in between-person research. In this analysis, we assessed whether the number of survey items used to assess a focal construct exerts an impact on the within-person variability that is observed. We found that when constructs were assessed with four or fewer items, there was significantly more WPV observed (WPV = 52.45%; 95% CI [50.88, 54.01]) than when there were five to nine items used (WPV = 47.46%; 95% CI [45.38, 49.53]) or 10 or more items used (WPV = 46.36%; 95% CI [42.93, 49.78]). Notably, the majority of relationships we examined (56.1%) were assessed with a scale of four or fewer items. Although this may be a function of the kinds of constructs that are typically measured with five or more items, it does suggest a need for expanded research on whether and how scale length influences observed within-person variation.
Number of measurements completed
Our findings indicate that less WPV is observed when five or fewer survey administrations are used (WPV = 43.24%; 95% CI [40.84, 45.64]) than when six to 10 administrations are used (WPV = 49.98%; 95% CI [47.92, 52.04]) or when 11 administrations or more are used (WPV = 54.06%; 95% CI [52.49, 55.63]). One reason that fewer administrations may detect less WPV is that they may be less able to capture the fluctuating experiences an employee faces; thus, they may be more apt to illustrate stable, between-person differences. This is consistent with our findings pertaining to time between measurements: It appears that more and more-frequent survey episodes show higher levels of observed WPV than do fewer and less frequent surveys.
Duration of the study
We found that this factor mattered, as less WPV was observed when within-person studies lasted less than a week (WPV = 45.39%; 95% CI [43.48, 47.30]) versus when they lasted 1 to 2 weeks (WPV = 53.17%; 95% CI [51.47, 54.87]) or more than 2 weeks (WPV = 52.37%; 95% CI [49.97, 54.78]). The advantages of detecting more WPV through studies that extend longer merit more research attention, particularly given the cautions that researchers have raised about within-person studies enduring for too long (Scollon et al., 2009; Stone et al., 1991).
Research Question 3: To What Degree Are a Study’s Within-Person Construct Relationships Equivalent to a Study’s Between-Person Construct Relationships?
All else equal, the potential for greater contribution from a within-person study exists when there is not homology across levels (e.g., when the relationship between X and Y is different within person than it is between person). For instance, detailing the timing, trend, and causal ordering of self-efficacy and performance, Sitzmann and Yeo (2013) conducted a meta-analysis that quite effectively showed how the relationship between self-efficacy and performance is much different within person than between person. In such cases, when homology is not present, the imperative for within-person findings is apparent since the same conclusions could not be derived from between-person studies. To gain insight into how often within-person relationships differ from between-person relationships, we coded the correlation coefficients in the studies that reported both within-person and between-person correlational data. These correlations were usually obtained using WABA (within and between analysis) or HLM (hierarchical linear modeling), and they separate a correlation between two constructs into its independent between- and within-person components (see Dansereau, Alutto, & Yammarino, 1984). As an example, Vogel and Mitchell (2017) reported the relationship between abusive supervision and organizational deviance, showing a correlation of .15 within persons and .47 between persons. Coding the relationships from all the other studies that reported such data resulted in 2,356 pairs of between- and within-individual correlations. 6
Because tests for homology can vary in their strictness (Chen, Bliese, & Mathieu, 2005), we applied multiple standards to assess homology. For the most stringent test, we assessed the frequency with which the relationship between two constructs exhibited both (a) the opposite sign (positive vs. negative) across the two levels and (b) a statistically significant different relationship across the two levels. To conduct these tests, we identified the within-person and between-person correlations in studies that differed in directionality. For these relationships, we determined whether the between- and within-person correlations exhibited a statistically significant difference from one another. We converted all correlations using a Fisher r-to-z transformation; then, we used procedures described by Bliese (2016) to conduct a z test to determine whether the within-person correlations were significantly different from the between-person correlations (Cohen & Cohen, 1983). In this strictest test of homology, 4.1% of construct relationships differed in directionality and were significantly different from one another. In the 4.1% of cases where homology (strictly defined) does not exist, the contribution is profound.
We also applied a more liberal standard of homology, investigating the degree to which within-person and between-person correlations were different from one another even if not exhibiting opposing directionality. As an example, the aforementioned pair of correlations from Vogel and Mitchell (2017) are significantly different from one another. Applying the same analytical procedures across the 2,356 pairs of between- and within-person correlations (Bliese, 2016; Cohen & Cohen, 1983), we found that 24.1% of the relationships were significantly different at the within-person compared to the between-person level (see Table 4). As a means of assessing our use of Fisher’s z test, we reanalyzed the aforementioned Sitzmann and Yeo (2013) studies—which demonstrated that self-efficacy and performance are differentially related across the within- and between-person levels—using Fisher’s z test procedures. Using Fisher’s z test procedures across the 32 pairs of between- and within-person correlations in their study, we found that 68% of the self-efficacy–performance relationships (and 45% of the reverse-causal relationships) are significantly different across levels. These percentages are much higher than those in the typical within-person study in our larger sample, where only 24.1% of correlations varied across levels. Moreover, the correlation of the between- and within-person effect sizes was .31 in Sitzmann and Yeo’s study and .83 in our larger sample. Thus, the conclusions that we would derive using Fisher’s z tests on their data (i.e., that there are different within- and between-person processes operating; Ostroff, 1993) would parallel the conclusions reached by Sitzmann and Yeo. Notably, since the correlations in their study differ across levels at 2 to 3 times the rate of the typical within-person study in our sample, Sitzmann and Yeo’s study serves as an exemplar for elucidating novel within-person relationships. We hope their model is followed by researchers in other content domains and provide Table 4 to assist in identifying domains where within- and between-person relations are most likely to differ.
Homology of Correlations Within and Between Persons
Note: Sig. = statistically significant; diff. = different; OCB = organizational citizenship behavior.
Pairs of correlations; only constructs with more than 30 correlations are included in the table.
Denotes the percentage of within- and between-person correlations of the same constructs that are statistically different from one level to another (within person to between person) at the p < .05 level of significance.
Denotes the percentage of within- and between-person correlations that both are statistically different from one another at the p < .05 level of significance and have opposite signs (positive vs. negative).
The significance tests we described assume that the within- and between-person correlations in a given study are independent from each other. Dansereau et al. (1984) argue that the two correlations are independent in the WABA procedure (see also Bliese, 2016). However, primary studies are sometimes not clear about whether they followed WABA or used another procedure to compute within- and between-person correlations. Another practice is to person-center each observation and then compute the correlation on the person-centered scores (i.e., within-person correlation) and the between-person correlation on the aggregated person means. According to Enders and Tofighi (2007), this procedure results in independent correlations because the person-level effects are removed from the within-person correlation. It is possible that a small number of primary studies used another procedure (e.g., did not person-mean-center before computing correlations) that resulted in potentially nonindependent correlations (Ostroff, 1993). 7 To minimize the impact of this potential issue, we reconducted our analysis using an alternate procedure where we computed meta-analytic effect sizes on each set of construct relationships and then compared the CIs based on between-person and within-person meta-analytic results. We computed bare bones estimates as well as estimates individually corrected for measurement error. 8 In either case, results from the meta-analytic procedure suggested correlations were more similar across levels (i.e., more cases of homology) than using the correlational procedure with a Fisher r-to-z transformation. Our results suggest that in practice it is common for correlates to be similar across levels, consistent with Ostroff’s (1993) claim that correlations will be the same across levels unless unique theoretical processes exist across levels. As such, given the degree to which correlations are similar across levels, our findings to some degree diverge from a common claim in published within-person research that within-person effects are significantly different from between-person effects.
For future theory building, we conducted a more fine-grained analysis categorizing the within- and between-person correlations by constructs to examine whether some constructs exhibited greater correlational differences across levels than others. Results of this analysis are presented in Table 4. Although multiple interpretations are possible within these results (and we hope future researchers use these as a basis for further inquiry), we highlight a few noteworthy findings. For the construct of happiness, relationships are often different at the within-person level than they are at the between-person level. This implies that the causes and consequences of an employee being happy at a given time are different from the causes and consequences of some employees being generally happier than others. Our finding suggests it is critical for researchers studying happiness to specify their expectations at the appropriate level of analysis; furthermore, it means that researchers should not assume that findings about happiness at one level of analysis generalize to the other level of analysis. For constructs like time pressure and coping, within-person relationships between the constructs and other constructs are frequently similar to between-person relationships. This shows that these constructs are homologous across levels—and suggests that the theoretical processes involved in these constructs may often be similar across levels.
Discussion
Given the meteoric rise in the number of management studies focused on within-person phenomena, we developed a three-question framework to (a) gain insight into the contributions of the collective body of within-person research studies and (b) provide guidance for how the contribution of individual within-person studies can be assessed. Leveraging the unique strengths of within-person research as a basis for evaluating contributions, our framework reveals how the potential of within-person studies can be optimized, resulting in the most impactful contributions to organizational science. Our study provides clear affirmation that within-person research has made important contributions while also noting ways in which the contribution of such research can be maximized in the future. We find that the contributions of within-person studies are maximized when researchers (a) integrate temporal elements into hypotheses, (b) effectively reveal the changes that employees experience over time, and/or (c) elucidate novel, within-person findings that do not simply mimic relationships at the between-person level.
Although each question we posed provides only limited insight when applied in isolation, the three dimensions of contribution offer a great deal of utility when simultaneously applied in a holistic way. There are instances in which it is neither plausible nor necessary for a given study to achieve contributions in all three of the focal areas we probe, and in some cases a strong enough contribution in one of the focal dimensions may compensate for a lack of contribution in the other categories. Conversely, a deficit or weakness in one of the three dimensions can adversely affect the potential strength and contribution of the other dimensions. For example, regarding Research Question 1, many within-person hypotheses can improve by positing temporal relationships. If a study is imprecise in the inclusion of time during hypothesizing, the need to study construct variability over time is less apparent, and the study’s capacity to detail differential theoretical processes that operate across levels may be inhibited. Yet, even when a within-person study is devoid of hypotheses with temporal elements it can still make a critical contribution if it successfully elucidates novel within-person construct relationships across levels. Regarding Research Question 2, choices made during the research design process (i.e., effective planning and execution that materializes from strong theorizing) can optimize the amount of within-person fluctuation that is detected. If a within-person study is focused on stable phenomena, there is less value in proposing within-person hypotheses (Bolger et al., 2003), and if a study is deployed in such a way that the degree to which constructs truly fluctuate is not fully detected, the study may be less likely to uncover differential relationships that exist across levels. Regarding Research Question 3, given the frequency with which homology exists across levels, it is incumbent on researchers to carefully consider how to design studies that capitalize on phenomena that fluctuate over time. When a within-person study is focused on constructs that are homologous across levels, it becomes even more crucial for that study to be investigating variable constructs or pursuing complex hypotheses that necessitate observing phenomena over time. However, even if a within-person study features constructs with homologous relationships across levels, it can still make an important contribution if the research is focused on a complex model (i.e., exploring a mediating process that differs at the within-person level).
Summary of Findings and Implications
Research Question 1: Temporal Hypothesizing
Scholars are increasingly emphasizing the importance of building time-related contributions (i.e., George & Jones, 2000). Within-person research is ideally suited to answer this call, and we probed the degree to which within-person research incorporates temporal elements into hypotheses, thus carrying the potential to meaningfully advance the field (Bergh, 2003; Mitchell & James, 2001). Adding temporal specification in a within-person study carries an enhanced contribution in that traditional (cross-sectional) between-person research does not theorize about changes over time (Ployhart & Vandenberg, 2010). Although it is encouraging that a majority of hypotheses in within-person research include an element of time—and positions within-person research well for making time-related contributions to the field—more than one third of hypotheses in within-person studies did not include time-related predictions. The even smaller number of within-person hypotheses that are specifying predictions related to duration or shape shows that much potential exists for the development of more sophisticated ideas about phenomena over time.
To the degree that it is plausible and conceptually appropriate, future researchers should employ temporal considerations when forming research questions and hypotheses. One challenge in this vein is that existing theories are often not temporally sophisticated enough to build temporally precise predictions. Affective events theory, for example, posits that employees experience cognitive and affective reactions to their daily experiences, which influence their job behaviors and attitudes. Yet the theory itself does not highlight the temporal context in which this occurs, leaving open the question of the timing of these predictions. Herein lies opportunity for within-person research. As an example, Ilies, Scott, and Judge’s (2006: 564) Hypothesis 1a was effective in that it included a time element: “Individuals’ daily positive affect is positively related to their daily citizenship behavior over time.” Yet, this hypothesis could go further by adding shape and duration components. For instance, the hypothesis could posit, “Individuals’ experience of daily positive affective events is positively related to their daily citizenship behaviors for the 1 hr after the positive affective events occur. At the end of an hour, there is no relationship.” This hypothesis—which suggests that affective events almost instantaneously reach an equilibrium condition with their correlates before entering an entropic period after an hour passes—builds a theory based on causes and effects between constructs over time (e.g., the MCC curve; Mitchell & James, 2001) and enhances temporal precision. In so doing, it bolsters the nomological net around affect and OCB and extends existing theory.
Research Question 2: Within-Person Variability
Knowledge about construct variability serves as the basis for contributions in within-person research. Judged collectively, the within-person literature makes a strong contribution by revealing how variable different constructs are over time. First, in support of scholars’ contentions (Beal, 2012; Dalal & Hulin, 2008; Roe, 2014), our analysis showed that all the within-person constructs studied with frequency vary over time. Although this fact certainly does not suggest that a large contribution will emerge simply because a study shows that a construct is variable, it does at least provide initial justification for conducting within-person research. Moreover, even small amounts of within-person variability can have meaningful—and even dire—consequences. For instance, even less than 1 hr’s loss of sleep can impact injuries at work after Daylight Savings Time (Barnes & Wagner, 2009).
Second, our synthesis of within-person data produced an empirically derived estimate of the variability of the most frequently studied constructs, revealing that significant differences exist in the degree to which constructs fluctuate over time. Just as validity generalization and meta-analysis have been quite useful to between-person research in overcoming the limitations of any single study (Schmidt, 1992), our construct variability data can be useful to researchers in multiple ways: (a) when theorizing, (b) when making determinations about assessment at the study design phase, and (c) when evaluating the nature of results during the latter stages of a study. As a boon to theory around change and dynamism, our empirical findings should serve as a means of confirming theoretical propositions about the degree to which constructs vary over time. Conversely, where extant theory has not adequately specified constructs, the results of our within-person variability analyses should serve to help researchers build theory about a given construct’s variability. For instance, OCB, although still quite variable, is one of the more stable constructs that within-person researchers study with frequency. Thus, if a researcher was monitoring study participants’ OCB and began to notice an uptick in within-person variability in OCB (e.g., WPV in OCB increased substantially over what had been observed in the past), this increased level of within-person variability could help the researcher to identify the timing and effects of causal mechanisms (e.g., positive affective events) potentially responsible for changes in OCB. Researchers could also cross-reference our within-person variability data in order to check to see whether the variability they have observed in their own data falls within the range that would be expected based on our cumulative findings and extant theory. When a single study’s construct variability does not appear within the expected range, the data may suggest that there is a need to extend theory to account for certain boundary conditions.
On a practical level, for constructs that exhibit more between-person variability than within-person variability, managers may be best served focusing on factors like selection and team design in order to facilitate the attainment of desired outcomes. Conversely, for constructs that exhibit more within-person variability, managers may be better served to use interventions (e.g., training) in order to lead employees to desired outcomes. Our variability analysis can thus be useful in determining what should be managed by getting the right people and what should be managed by putting people in the right situation.
Third, the data provided a clear answer to our question of whether moderators exert an impact on observed within-person variability, with differences appearing as a function of both theoretical and measurement factors. The fact that variability was different based on two proposed underlying theoretical “drivers”—data collection technique and passage of time between measurements—suggested that a given construct may appear substantively different depending on how it is assessed. In particular, we highlighted the possibility of circadian rhythms (i.e., Aschoff, 1965; Czeisler et al., 1999) as bearing some responsibility for the differences (or lack thereof) in a given construct’s variability, with repeated assessments of constructs over time on less predictable schedules at varying times in a circadian cycle correlated with increased levels of WPV. As pertains to measurement factors, we found that within occasions, shorter construct measures were associated with greater within-person variability. Conversely, across occasions, we found that within-person studies that lasted longer and featured more data collection administrations resulted in increased WPV. Perhaps the differences are artifacts of measurement error. It may be that some designs are more effective than others of capturing how a construct varies over time. Future research is needed to understand the factors driving these measurement-factor differences.
Fourth, relevant to researchers in their respective content areas, we have included an online supplement that provides fine-grained analyses of a construct-by-moderator breakdown of within-person variability. As an example of how these analyses may prove useful, one may notice the absence of within-person research using event-sampling techniques to study constructs like social support, exhaustion, and cross-domain conflict. Given the theoretical relevance of assessing such constructs when certain events occur (i.e., evaluating perceptions of social support following interactions with coworkers), this dearth of studies suggests that the means by which researchers are assessing these constructs may not fully capture the variability of the constructs to the degree that would be possible. 9
Research Question 3: Differences Between and Within Persons
Contributing to a conversation in the literature with respect to whether the use of between-person designs is appropriate for testing within-person theories (Dalal et al., 2014; Ployhart & Vandenberg, 2010), we evaluated the degree to which within-person research reveals new relationships at the within-person level that are not present at the between-person level. Although researchers have asserted that the within- and between-person levels are distinct (Cervone, 2005; Dalal et al., 2014; Dalal & Hulin, 2008; Uy et al., 2010), our results suggest within- and between-person bivariate relationships are often similar in within-person research. Table 4 elaborates on this conclusion for constructs that exhibited similar relationships across levels (e.g., time pressure). Even though the bivariate relationship is similar across levels, theory and empirical research can explore whether these comparable effects are a function of similar or dissimilar processes operating at each level. For example, is time pressure hindering performance because it produces strain both within person and between person, or is time pressure equivalently hindering performance through one theoretical process (i.e., strain) within person versus by another process (i.e., role overload) between person? Similar exploration could be conducted on the constructs in Table 4 that exhibit differential relationships across the levels (e.g., anger). For instance, does a construct like organizational commitment differentially carry the effects of anger on performance within person versus between person? Or, are the effects of anger on performance carried through one construct (i.e., organizational commitment) between person and an entirely different construct (i.e., emotional labor) within person? Answers to these questions will help to reveal the areas in which within-person studies are compulsory for building within-person nomological networks.
We do want to raise two points regarding the interpretation of our results. First, our analysis contrasted within- and between-person correlations from within-person studies and, as such, does not make comparisons with traditional (i.e., one-time assessment) between-person designs. Whereas a within-person study can disentangle the within- and between-person levels, between-person designs confound the levels in a single correlational estimate. Importantly, between-person estimates from traditional studies versus those derived from within-person studies may diverge given that there are other differences we did not systematically compare (e.g., single vs. multiple source; time-separated measurement). As an initial means of assessing whether between-person effect sizes differ based on study type (“one-shot” data collection versus repeated measures data collection), we derived a meta-analytic between-person effect size from our data set and then compared that effect size to a meta-analytic estimate from the published literature for three of the most frequently studied constructs in our sample: PA, NA, and job satisfaction. For PA and job satisfaction, Connolly and Viswesvaran (2000) reported a between-person meta-analytic effect size of .49 (95% CI [.19, .79]). Similarly, the meta-analytic between-person effect size that we computed on our sample for the same relationship is .51 (k = 5; 95% CI [.34, .68]). For NA and job satisfaction, Connolly and Viswesvaran reported a between-person meta-analytic effect size of –.33 (95% CI [–.51, –.15]). Similarly, the meta-analytic between-person effect size that we computed for the relationship is –.41 (k = 4; 95% CI [–.43, –.38]). For PA and NA, Kaplan, Bradley, Luchman, and Haynes (2009) reported a between-person meta-analytic effect size of –.23, and we computed a between-person meta-analytic effect size of –.30 (k = 27; 95% CI [–.39, –.21]). The overlap in confidence intervals and similarity in magnitude of the effect sizes suggest one-shot and repeated-measures designs of between-person relationships could yield similar conclusions, and we recommend expanding this approach in future research to assess its generalizability across constructs. We also recommend that within-person researchers examine published between-person effect sizes prior to conducting within-person studies in order to gain insight into potential theoretical extensions. Once within-person data have already been collected, we view utility in making comparisons with between-person findings in order to detect empirical distinctions across levels.
Second, Research Question 3 represents only one consideration in our framework. Indeed, when our three-item framework is applied holistically (as we would suggest), within-person studies can make a contribution even when homology exists and patterns of effects are similar across levels. In such cases, it is crucial to theorize about the importance of fluctuation over time, to pose nuanced temporal hypotheses that require within-person research, and/or to use theory to develop rationale for why processes at the within-person and between-person levels should be expected to differ even if the bivariate relationships between the levels are the same.
Limitations and Future Research
First, we assessed the collective within-person literature in an objective, data-driven manner by focusing our investigation on three focal issues that are directly linked to the strengths of a within-person perspective (Bolger et al., 2003; Uy et al., 2010): questions posed, change over time, and novel findings across levels. These three issues are fundamental to any researcher’s within-person studies, and the takeaways from them can be unilaterally applied to the study of within-person phenomena across many research domains. However, there are several published studies that cut across categories and cannot be easily placed within our three-part framework. For instance, within-person studies enable the pursuit of complex models (e.g., the capacity to elucidate cross-level interactions; Uy et al., 2010) that would not be possible without measuring respective constructs at both within-person and between-person levels. 10 Some within-person studies have made a contribution because they integrate previously separate theories; for these studies it is the theoretical arguments, rather than the use of within-person research per se, that comprise the contribution. Also, at times, within-person researchers study variable constructs or phenomena that are temporally bound and thus could not be studied without a within-person perspective. For example, Bledow, Rosing, and Frese (2013) studied how creativity is influenced by affective shift, which they defined as a process resulting from a person experiencing “an episode of negative affect that is followed by a decrease in negative affect and an increase in positive affect” (p. 432). The affective shift construct, which requires accounting for how individuals experience fluctuating levels of positive and negative affect over time, is inherently a within-person construct, and it would not be possible to measure and assess the construct if employing a traditional (i.e., one-time assessment) between-person study.
Second, we investigated five critical factors as potential moderators of the amount of WPV that is observed. Future research should also examine whether other factors impact observed within-person variability; for example, does the length of the data collection instrument affect participant fatigue, or does the passage of time that occurs from event occurrence to data collection increase retrospective bias as more time expires, thus decreasing within-person variation? “Rating source” (self vs. other) and “nature of rating” (subjective vs. objective) merit attention, too. We also recommend further probing the effects of circadian rhythms (e.g., by examining the precise time of day in which data collection occurs). We highlighted the importance of elucidating variability; future research is needed to see whether increasing within-person variability has trade-offs (e.g., as an impediment to validity).
Third, our analysis of homology to answer Research Question 3 was confined to the constructs that have been studied by within-person researchers. Although there are a multitude of states (e.g., affect, mood, emotion), behaviors (e.g., performance), and situations (e.g., job stressors) that have been the focus of considerable research attention at both within-person and between-person levels, it is possible that the degree of homology may be different than we found if investigating different constructs that are pursued in future research. Also, we compared within-person and between-person correlations within studies, which enables an “apples-to-apples” comparison given that the data were formed using the same measures on the same sample of study participants. Although researchers have noted the efficacy of using within-person research to assess between-person phenomena (Beal, 2015; Bolger et al., 2003) we cannot discount the fact that it is possible that between-person correlations that are formed from a strictly between-person study (e.g., from a one-time assessment of constructs) could be substantively different from a between-person correlation formed from repeated measures of a within-person study (e.g., because retrospective bias may be more present in between-person correlations derived from single-time assessment than from within-person studies; Uy et al., 2010). Thus, a conservative interpretation of our findings would suggest generalizing our findings to within-person conclusions rather than between-person conclusions—at least until future research extends the kinds of between-person findings we reported for PA, NA, and job satisfaction to all kinds of between-person construct relationships.
Fourth, by synthesizing across hundreds of within-person studies we were able to collectively assess the literature and provide data to support future research efforts. Yet, some constructs have been studied much more frequently than others, so our data are limited by the areas in which researchers have elected to invest their time and attention; relatedly, the “file drawer problem” could reduce our collection of studies and potentially bias results. If convention facilitated the inclusion of more extensive data reporting, we could have bolstered the coverage of variance components and correlational relationships of constructs. Although our requests for data from primary study authors embellished our data set, fewer than half of published within-person manuscripts reported data on variance components and correlational data for both within- and between-person relationships. If all of the studies had reported such data, researchers could develop a clearer picture of relationships among within-person constructs and learn more about which constructs vary within individuals over time, how much they vary, and if they vary in some conditions more than in others. Answers to these questions are important not only to develop cumulative knowledge but also to build theory and better understand individuals’ experiences at work. Thus, in the interest of building cumulative, within-person knowledge, we propose a series of reporting requirements; such standards would facilitate better future research while also ensuring that the strengths of meta-analysis (e.g., for providing empirical support to bolster theory; Schmidt, 1992) can be extended to within-person findings. We recommend reporting the following in within-person studies: (a) within-person and between-person correlations, including the procedure used to compute within-person correlations; (b) WPV and BPV components; (c) the approximate time required of study participants to complete each data collection instrument (to assess fatigue effects); (d) the data collection technique used (interval, signal, or event contingent); (e) the time of day in which data collection occurred; (f) the amount of time that passed between each data collection; and (g) the average amount of time that passed from data notification to participant data submission. Further, we suggest the importance of a qualitative assessment of the context in which the study takes place (e.g., during finals week for college professors) in order to offer insight into the degree to which a linear trajectory (Sitzmann & Yeo, 2013) may be occurring among the constructs being studied.
Fifth, measurement error may impact our variability and correlational analyses. For the variability analysis, we are not aware of guidance in the literature that describes corrections for measurement error on ICCs, and primary within-person studies are often unclear in their reporting of whether reliabilities are calculated on data between persons, within persons, or some combination of the two. It is possible that error may have inflated our ICCs and led to overestimates of within-person variability. If this is the case, the same phenomena would be influencing future primary within-person studies; thus, we think the ranges of results reported in the form of our CIs should be applicable to researchers. However, this limitation should to some degree temper our conclusion that most constructs exhibit within-person variability. Similarly, given the lack of consistent reporting as well as the lack of direction in the literature, we used raw correlations to conduct our within- and between-person correlational analysis across levels in Research Question 3. Future research is needed on how measurement error influences ICC and within- and between-person correlations.
To conclude, we assessed how the burgeoning within-person literature is making contributions to management. The greatest potential for contribution exists when researchers include temporal elements in their hypotheses, elucidate within-person change over time, and/or report novel findings that are not possible with between-person research. We look forward to the continued implementation of within-person studies and hope our study will prove useful.
Supplemental Material
JOM788435_DS – Supplemental material for Scientific Contributions of Within-Person Research in Management: Making the Juice Worth the Squeeze
Supplemental material, JOM788435_DS for Scientific Contributions of Within-Person Research in Management: Making the Juice Worth the Squeeze by Brian W. McCormick, Cody J. Reeves, Patrick E. Downes, Ning Li and Remus Ilies in Journal of Management
Footnotes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
