An Examination of the Influence of Implicit Theories,Attribution Styles,and Performance Cues on Questionnaire Measures of Leadership

Abstract

We examined the direct and interactive effects of respondents’ implicit leadership theories (ILTs), attribution styles, and performance cues on leadership perceptions. After first assessing respondents’ implicit leadership theories and attribution styles, the participants were randomly assigned to one of nine performance cue conditions ([leader performance: low vs. average vs. high] × [follower performance: low vs. average vs. high]), observed the same leader’s behavior via video, and rated the leader by completing three leadership questionnaires. The results supported the notion that these three components of information have both direct and interactive effects on leadership perceptions as measured by the questionnaires. The three components of information accounted for about 10% of the variance in the three questionnaires. The results contribute to theories of information processing by demonstrating how ILTs, attribution styles, and performance cues interact to predict leadership perceptions. Implications regarding the meaningfulness, construct validity, and utility of leadership scales are discussed.

Keywords

leadership leadership and individual differences cognitive processes

Leadership researchers have long sought valid measures of leadership. Using leadership measures, researchers have found that subordinates’ perceptions of leaders are related to followers’ performance, satisfaction, and even well-being (for a recent review, see Dinh et al., 2014). Nevertheless, there has been a growing body of research demonstrating that perceptions of leadership are biased by factors such as implicit leadership theories (ILTs), attributions, and performance information (e.g., Epitropaki, Sy, Martin, Tram-Quon, & Topakas, 2013; Lord & Maher, 1991; Martinko, Harvey, Sikora, & Douglas, 2011; Shondrick, Dinh, & Lord, 2010; Sy et al., 2010). On considering much of this research, Day (2012) recently stated, “All of the evidence suggests that it may be unrealistic to assume that we can measure leadership with a questionnaire” (p. 706). We believe that ignoring the influence of the many factors that can bias leadership questionnaires is a significant problem. Thus, the primary objective of this article is to demonstrate that traditional leadership questionnaires are biased by factors such as implicit theories, attribution styles, and performance information.

Our research assumes a limited-capacity information-processing model of leadership perception (Lord & Maher, 1991). According to this perspective, organizational members often rely on preset schemas and currently active information in order to efficiently respond to the complex social environment (Epitropaki et al., 2013). The combination of preset schema and/or currently available information that produces the follower’s perception of a leader, however, is inherently fluid and dynamic. Given these complexities, a connectionist approach to leader perception has been proposed (Brown & Lord, 1999). The connectionist approach views perception of the leader as a collective network of subunits of information that become activated based on certain stimulus inputs (Hanges, Lord, & Dickson, 2000). Importantly, the connectionist approach can explain how leadership perception can be both context-sensitive and stable over time (Foti, Knee, & Backert, 2008). Building off of a connectionist conception, the current work examined the influence of three potentially major units of cognitive information that give rise to leader perceptions in the form of responses to leadership questionnaires: ILTs, attribution styles, and performance cues.

Influences on Subordinate-Based Perceptual Measures of Leadership

Although there have been demonstrations of various influences on subordinate perceptions of leadership (e.g., Shondrick et al., 2010), we are unaware of any recent attempt to apply these insights to the leadership questionnaires that are currently being used in leadership research. Furthermore, we are unaware of any recent attempt to demonstrate more than one source of influence on subordinates’ perceptions in current leadership questionnaires, despite classic works in the 1970s and early 1980s showing that early questionnaires were biased by a variety of sources, including ILTs (Rush, Thomas, & Lord, 1977), attributions (Phillips & Lord, 1981), and performance cues (Lord, Binning, Rush, & Thomas, 1978). For example, in 1975, building on earlier work on implicit personality theories (e.g., Schneider, 1973), Eden and Leviatan introduced the concept of ILTs in the context of leadership measurement. The notion was that subordinates have preconceptions about leadership and that “questionnaire responses may be mere reflections of respondents’ prior conceptions and not veridical representations of empirical reality in the organizational environment” (Eden & Leviatan, 1975, p. 736). This notion of the influence of ILTs on subordinates’ evaluations of leadership continues to be developed (see Hansbrough, Lord, & Schyns, 2015) with, for example, recommendations for how to avoid the effects of ILTs on subordinates’ ratings (Hunter, Bedell-Avers, & Mumford, 2007; Naidoo, Kohari, Lord, & DuBois, 2010). However, the implications for leadership questionnaires currently in use have not been directly assessed. Moreover, demonstrations of other sources of influence on leadership questionnaires, besides ILTs, are underdeveloped in the literature.

Despite long-standing cautions regarding the construct validity of subordinate reports of leadership, which typically rely on retrospective recall of past leadership behaviors (e.g., Rush et al., 1977), researchers still appear to assume that assessments resulting from respondents’ questionnaire responses are valid (cf. Martinko, Harvey, & Dasborough, 2011). Building on Hansbrough et al.’s (2015) recent theorizing, we directly examine the assumptions regarding the construct validity of three current leadership questionnaires by assessing the extent to which subordinates’ cognitive preconceptions of leaders, as measured by their general ILTs, attribution styles, and their knowledge of prior performance, affect their responses to three popular leadership questionnaires. Specifically, Hansbrough et al. (2015) proposed a conceptual model of leader rating accuracy in which follower individual differences and psychological processes within followers are moderated by contextual factors to produce leadership ratings. Using this as a guide for our study, we examined an individual difference in terms of attribution styles, the psychological process of ILTs through a measure of leadership prototypes, and a contextual factor in the form of performance cues. Our logic for including these three components is included next, starting with ILTs.

Implicit Leadership Theories

Since Eden and Leviatan (1975), there have been a number of studies that have enhanced and refined our understanding of the effects of ILTs on subordinates’ perceptions of leadership (e.g., Den Hartog et al., 1999; Epitropaki & Martin, 2004; Lord, Foti, & De Vader, 1984; Lord, Foti, & Phillips, 1982). For example, Lord and Shondrick (2011) illustrate how knowledge of group performance appears to cue ILTs and affects subordinates’ perceptions of leaders of these groups. Other studies have investigated how subordinates and observers form perceptions of leaders’ behaviors. Although some of this work explicitly uses the ILT term (e.g., Epitropaki & Martin, 2004, 2005; Rush et al., 1977), other studies do not; nonetheless this latter set of studies also explored the cognitive structures that lead to subordinates’ preconceived notions of leader behavior (e.g., Johnson, Murphy, Zewdie, & Reichard, 2008; Lord et al., 1982; Phillips & Lord, 1981, 1982).

Lord and his colleagues have been the most active contributors to this body of work. Beginning in 1977, Rush, Thomas, and Lord conducted a constructive replication of the Eden and Leviatan (1975) study and found similar results. More specifically, with three different samples differing in familiarity with their leaders, they found essentially the same factor structure (i.e., consideration and initiating structure) for the Leader Behavior Description Questionnaire. In addition, they found that performance and other contextual cues significantly affected questionnaire responses. They concluded that subordinates’ leadership ratings were at least partially a function of respondents’ ILTs, which are needed because of limited information-processing capabilities. Thus, consistent with Eden and Leviatan’s (1975) findings, Rush et al. (1977) found clear evidence of threats to the construct validity of leadership questionnaires based on subordinate ratings.

In a series of studies that followed, Lord and colleagues began investigating the cognitive processes underlying the structure of observers’ leadership perceptions. Lord et al. (1982) proposed that observers develop leadership prototypes (e.g., effective vs. ineffective) that organize their observations of leader behaviors. These leadership prototypes allow observers to overcome memory limitations and guide recall of specific instances of leader behaviors. The existence of these prototypes was subsequently tested and confirmed in the research by Phillips and Lord (1981). They found that these prototypes limited observers’ abilities to recall behaviors when they were inconsistent with their leadership prototypes. Later, Lord and colleagues (Lord et al., 1982; Lord et al., 1984) demonstrated that observers’ attributions of leadership were related to the extent to which leaders matched the observers’ leadership prototypes. Again, the general notion of this research is that observers, including supervisors and subordinates, use prototypes (i.e., ILTs) that enable them to make attributions of leadership, allowing them to forego the more cognitively demanding processes described by attribution theories (Cronshaw & Lord, 1987; Naidoo et al., 2010). More recent work by Lord and colleagues has been directed toward investigation of the cognitive structures that generate these leadership prototypes and contribute to perceptions of leadership (Lord & Shondrick, 2011; Shondrick et al., 2010).

In addition to the work of Lord and his colleagues, there have been a number of other studies that have contributed to our knowledge of the factors and effects of ILTs. Specifically, Offermann, Kennedy, and Wirtz (1994) as well as Epitropaki and Martin (2004) developed or refined scales that identify the factors (or content) associated with ILTs and demonstrated that these factors are relatively stable across different groups (see Den Hartog et al., 1999). Additionally, Epitropaki and Martin (2005) demonstrated that congruence between subordinates’ ILT prototypes, as measured by their ILT scale, and their leader’s actual profile was positively related to the quality of leader–member exchange (LMX), organizational commitment, job satisfaction, and well-being. Furthermore, a study by Keller (1999) demonstrated that factors that define individuals’ ILTs are often mirror images of self and parental traits.

In general, this body of theory and research advances and supports the notion that subordinates have preconceived notions or stereotypes (i.e., ILTs/cognitive structures) that are activated when evaluating leader behavior (e.g., Lord et al., 1982; Lord et al., 1984). Nevertheless, ILTs have been found to be dynamic social constructs that are influenced by perceiver factors such as past interactions (e.g., Ritter & Lord, 2007), group membership (e.g., Hogg et al., 2005), the target’s group classification (e.g., Paris, Howell, Dorfman, & Hanges, 2009), the target’s implicit followership theories (e.g., Sy, 2010), cultural background (e.g., Menon, Sim, Fu, Chiu, & Hong, 2010), and specific contexts (e.g., Lord, Brown, Harvey, & Hall, 2001). Overall, the emphasis of these studies has been on demonstrating the nature and function of these cognitive structures, which often departs from the goals of earlier works (e.g., Eden & Leviatan, 1975; Rush et al., 1977) that questioned the construct validity of leadership questionnaires. Given the implications of ILTs for leadership measures, as well as the heavy reliance on questionnaire measures of leadership, a return to these earlier concerns regarding construct validity seems warranted (see Hansbrough et al., 2015). One contribution of our study, therefore, is that we provide a direct test of earlier concerns by demonstrating how the construct validity of three typically used leadership questionnaires is threatened by subordinates’ ILTs. As such, we hypothesize:

Hypothesis 1: ILTs account for a significant proportion of the variance in subordinates’ responses on leadership questionnaires.

Attributions and Subordinates’ Leader Perceptions

As early as the 1970s, researchers such as Pfeffer (1977) began to use attribution theory as a model of leadership perception (see also, Mitchell, Larson, & Green, 1977). In 1981, Phillips and Lord directly tested the role of causal attributions in subordinates’ perceptions of leaders. They found that although the experimental manipulations significantly affected causal ascriptions and subordinates’ leader perceptions, a large proportion of the variance in subordinates’ leader perceptions was accounted for by performance cues, which they interpreted as a categorization process.

As a follow-up to Phillips and Lord’s (1981) study, Cronshaw and Lord (1987) proposed and compared three models of subordinates’ leadership perceptions: (1) a categorization model, which proposed that subordinates’ leadership perceptions are primarily a function of categorization processes; (2) an independence model, which assumes that observers use both attributional processes and categorization processes to form leadership perceptions; and (3) an attributional model, which asserts that attributional processes are necessary to make judgments of leadership. They concluded that categorization processes, as opposed to attribution processes, were more likely to influence leadership perceptions in social situations that require quick reactions. However, they also recognized that subordinates’ attribution processes may play an important role when there is ample time to make important decisions about leader performance.

Based on their results, Phillips and Lord (1981) proposed two distinct processes in which leader perceptions may be formed: recognition- and inferential-based processes. Recognition-based processes were thought of as largely automatic categorization processes that are needed when there is limited time and high cognitive load. Inferential-based processes were conceived as controlled attributional processes that were engaged in when there is time or need for reflection. We believe that subordinates’ responses to leadership questionnaires, which provide ample time for reflection, are most likely influenced by both recognition- and inferential-based processes. As a result, we are proposing an independence model where both processes are activated to make the judgments required by responses on leadership questionnaires.

Attribution theory

Attribution theory is based on Heider’s (1958) theoretical work, which focused on individuals’ innate needs to find causal explanations for why people behave the way that they do and why certain events occur (Martinko, Harvey, & Dasborough, 2011). The most notable refinements to the theory were made by Kelley (1973) and Weiner (1985), who identified three dimensions of attributions often used by individuals: locus of causality, stability, and controllability (for a review, see Harvey, Madison, Martinko, Crook, & Crook, 2014).

Two major facets of attribution theory are the notions of causal explanations and the underlying dimensions of the explanations (Weiner, 1986; Martinko, Harvey, & Douglas, 2007). Causal explanations (i.e., attributions) are the immediate reasons people give for an outcome. Thus, for example, a subordinate who receives a poor performance appraisal may blame the supervisor. However, we cannot begin to know how this attribution affects the subordinates’ affect and behavior unless we know the underlying dimensions of the causal attributions. Locus of causality has been found to influence affect, stability influences expectancies, and controllability influences intentionality (Martinko, Harvey, et al., 2007; Martinko, Moss, Douglas, & Borkowski, 2007; Weiner, 1986). In this case, the cause is clearly external (i.e., the supervisor). If the subordinate perceives that the supervisor’s evaluations are stable and unlikely to change (as well as controllable), the subordinate is likely to experience negative affect, lowered expectancies, and blame the supervisor for the poor evaluation. On the other hand, if the subordinate sees the supervisor’s evaluations as unstable and likely to change as well as uncontrollable (perhaps due to economic conditions), the subordinate will still experience negative affect, but will have some hope that things can change.

It is also important to note that not every outcome triggers attribution processes. As explained by Weiner (1986) and emphasized by Martinko, Douglas, and Harvey (2006), attributions are most likely to occur when the outcome is important, surprising, or negative. These types of outcomes tend to result in the type of explicit processing described by Cronshaw and Lord (1987) as inferential-based processes.

Attribution styles

Attribution styles are fairly stable tendencies of attributing cause across a variety of situations (Kent & Martinko, 1995). Attribution styles can, therefore, be characterized as trait-like tendencies of individuals to make particular types of attributions (for a review, see Martinko, Harvey, et al., 2007; Martinko, Moss, et al., 2007). Another way of thinking about attribution styles is that they are descriptions of how and the degree to which a person is biased in their attribution processes (Martinko, Harvey, & Dasborough, 2011; Martinko, Harvey, Sikora, et al., 2011). Importantly, a substantial body of research demonstrates that attribution styles can be reliably measured and that they predict affect and behavior, particularly in leadership contexts (Martinko, Harvey, et al., 2007). Thus, for example, some individuals tend to be optimistic and make internal and stable attributions for success, while attributing failure to unstable and external causes. These optimistic individuals will tend to experience positive affect and display motivated behavior. In contrast, pessimistic individuals who make internal and stable attributions for failure tend to experience negative affect and perform poorly (Martinko, 2002).

Recent research has demonstrated that the attribution styles of both leaders and subordinates predict the quality of leader–member relations (Martinko, Moss, et al., 2007) and that subordinates’ attribution styles are related to their perceptions of abusive supervision (Martinko, Harvey, Sikora, et al., 2011). Considering these results and a recent meta-analysis showing that attributional processes account for a significant portion of the variance in important organizational outcomes, such as employee performance and relationships between supervisors and subordinates (Harvey et al., 2014), a reexamination of the role of attributional processes in subordinates’ perceptions of their leaders seems warranted. Furthermore, consistent with the connectionist approach to leader perception discussed by Hansbrough et al. (2015), attribution styles are likely one of the units of cognitive information that give rise to subordinates’ perceptions of their leaders. Thus, a second contribution of our study is that we directly test the proposition that subordinates’ attribution styles effect subordinates’ responses to leadership questionnaires. As such, we hypothesize:

Hypothesis 2: The dimensions of attribution styles (i.e., locus of causality, stability, and controllability) account for a significant proportion of the variance in participants’ responses on leadership questionnaires.

Although there has been considerable attention devoted to attribution styles as a topic of study in the leadership area (Martinko, Harvey, Brees, & Mackey, 2013), very few studies have examined how the numerous possible styles (i.e., the interactions of locus of causality, stability, and controllability as well as other possible dimensions) might interact to predict ratings of leadership. According to attribution theory (Martinko, Harvey, et al., 2007; Weiner, 1985, 1986) each dimension has unique effects with locus of causality affecting emotions, stability affecting expectations, and controllability affecting intentionality. Thus, although we expect that the tendency to favor any of the attributional dimensions will have independent effects, we also predict interactive effects. For example, if a subordinate attributed the failure to receive a raise to an external and stable source, such as a difficult supervisor, we expect that the subordinate’s ratings of the supervisor will be strongly negative. On the other hand, even though a subordinate may attribute the failure to receive a raise to the supervisor, if the subordinate believes that this type of decision is unstable and less likely to occur again, we expect that it would attenuate the negativity of the subordinate’s ratings.

Although there is little research regarding the interactive effects of the dimensions of attribution styles, one exception is the research on hostile attribution styles. Hostile attributions styles have been found to be related to poor leader–member relations (Martinko, Moss, et al., 2007) and abusive supervision (Brees, Martinko, & Harvey, 2016; Martinko, Harvey, Sikora, et al., 2011). For example, subordinates with hostile attribution styles (Martinko, Harvey, Sikora, et al., 2011) have tendencies to blame their failures on the internal and stable characteristics of their supervisors and perceive their supervisors as abusive. In view of the above discussion, we propose:

Hypothesis 2a: The interactions of the attribution style dimensions will account for additional variance beyond that contributed by the separate dimensions.

Hypothesis 2b: Hostile attribution style (i.e., the tendency to make external and stable attributions for failure) will be related to negative leadership ratings.

Performance Cues

Performance cues usually come in two forms when used to examine perceptions of leadership: information about the performance of the leader (e.g., Rush et al., 1977) or information about the performance of subordinates (e.g., Phillips & Lord, 1981). The reasoning behind using such manipulations is often to experimentally examine the influence of situational stimuli in the cognitive processes leading to perceptions of self and others.

Early studies typically used bogus performance feedback about subordinates (e.g., Farris & Lim, 1969; Staw, 1975). For example, Mitchell et al. (1977, Study 2) found that individuals receiving positive performance cues made more positive attributions for their leaders compared with participants who received negative performance cues. Similarly, Phillips and Lord (1981) found that participants given positive performance cues concerning a problem-solving group rated the leader as being more of a causal agent in the group, as showing more leader behaviors, and as demonstrating more overall leadership compared with participants who were given negative performance cues, despite the same leader behaviors being observed via video by all participants.

Research that manipulates performance cues has provided clear evidence that this manipulation affects perceptions of leaders in specific ways (e.g., Lord et al., 1978). For example, Rush et al. (1977) manipulated leader performance cues by stating that the person was a supervisor in charge of the best or worst departments in the company. After presentation of the manipulation, participants completed a leadership questionnaire. Results showed that participants receiving positive performance cues rated the supervisor as showing more leadership behaviors than those that received negative performance cues. Interestingly, these results were obtained even though participants never observed actual leader behaviors.

Overall, the results of research that has varied either follower or leader performance cues suggest that cognitive categorizations and attributions of leaders by followers are activated by these cues. It should be noted that although performance cues and ILTs are sometimes lumped together in the literature (e.g., Rush et al., 1977), we conceive of the two as different components in the current study insofar as our measure of ILTs taps into general prototypes of leaders, while our presentation of performance cues represents an example of currently activated information. Conceiving the performance cues as currently active information in the formation of leadership perception is consistent with more than 30 years of research in cognitive psychology that demonstrates that cognitive structures can be affected by priming stimuli (for a review, see Wittenbrink, 2007) based on principles of accessibility, applicability, and salience (Higgins, 1996). In fact, recent research has demonstrated that leadership-relevant categorizations can even be activated by cues that are nonconscious (i.e., subliminal primes; Coleman, 2004; Lord & Shondrick, 2011; cf. Gioia & Sims, 1983).

As summarized above, we have a rich empirical history documenting the effects of leadership performance cues (which sometimes come in the form of information about follower performance). In all the studies reviewed, positive cues about the leader’s performance (or follower performance) resulted in positive leadership ratings while negative cues resulted in less favorable ratings. As a result, we predict:

Hypothesis 3: Performance cues account for significant proportions of the variance in the responses to leadership questionnaires such that positive cues are related to positive ratings whereas negative cues are related to negative ratings.

Method

Participants and Procedures

Participants were employees recruited via two methods. First, participants were recruited from undergraduate classes at three southern universities located in the United States. Second, these initial participants provided the name and contact information of additional potential employee participants in exchange for extra credit according to institutional review board–approved procedures, which varied slightly because of the requirements at the respective universities. Contact information provided by students was randomly selected for general verification. No verification problems were found in the randomly selected contacts; therefore, the e-mail addresses provided in all the contact information for each group of participants were then used to send a link to an online survey. In total, 282 participants completed the study in its entirety (female = 179, M_age = 35.47 years, SD = 13.49). On average, participants worked 39.45 hours per week (SD = 13.00), had been working for 16.17 years (SD = 12.80), and were in their current job for 6.36 years (SD = 7.27).

In the e-mail participants received, they were informed that they would be responding to an anonymous online questionnaire, which included a video role-play. All participants, regardless of condition, first completed measures of attribution style and ILTs, which were presented in a randomized order. Next, an introduction and description of the video vignette was provided. Participants were told that they would be playing the role of a subordinate in a Fortune 500 pharmaceutical sales company receiving a simulated performance appraisal from the supervisor depicted in the video and were asked to vividly imagine themselves in the role of a sales representative. In the video, the supervisor faced the camera in order to simulate speaking directly to the participants and proceeded to deliver the performance review. Every participant viewed the exact same video. Thus, all participants, regardless of condition, reacted to the same leader behaviors.

The leader and follower performance manipulations were included in the introduction to the video. Participants were randomly assigned to one of nine conditions in a 3 (leader performance: low vs. average vs. high) × 3 (follower performance: low vs. average vs. high) between-subjects factorial design. Thus, the leader (i.e., the supervisor in the video) and follower (i.e., the participant) performance levels were manipulated through randomly assigned written vignettes. As an example, the high leader performance manipulation was

Your boss is a high performer who has exceeded his sales goals in each of the last ten years. He is considered one of the best of all sales managers and has had success in motivating his employees throughout his career.

As an example, the low subordinate performance manipulation was, “You have consistently been one of the poorest performing sales associates in the company and have often failed to meet your sales goals in each of the last five years. This year was also not so good.” Participants were also informed that they would be asked questions about their performance evaluation after viewing the video.

The prerecorded performance evaluation lasted approximately six minutes and is described in Brees et al. (2016). The video simulated a performance evaluation between a supervisor (a 50-year old White male with more than 35 years of administrative work experience) and his subordinate (i.e., the participant). The supervisor followed a template of talking points created by the researchers that was designed to provide participants with both supportive (i.e., prototypical) and critical (i.e., antiprototypical) communications that we believed would allow participants’ natural leadership perceptions to be activated. The performance review unfolded as follows. The supervisor began by thanking the participant for meeting via Skype and apologized for not being able to meet in person. The supervisor then shared that the purpose of the meeting was to review the participant’s sales performance over the past year and discuss how to optimize it in the future. He then told the participant that his or her performance for this past fiscal year was below company standards.

The supervisor then proceeded to discuss the five drugs that the participant was required to promote with doctors and discussed the respondent’s performance for each. The supervisor highlighted multiple examples of where the participant had opportunities for improvement and also noted areas where the subordinate met expectations in prior years. The supervisor displayed multiple prototypical leadership behaviors (e.g., understanding—acknowledging the participant’s mother’s health condition and helpful—offering to ride along on future sales calls) and antiprototypical leadership behaviors (e.g., domineering—telling the participant “you’ve got a problem” and pushy—stating beliefs about participants’ work ethic). At the end of the evaluation the supervisor reiterated that the participant had the capability to be successful in the future and suggested several activities to help develop the participant’s skills. Last, the supervisor provided the participant with a rating of “failed to meet expectations,” communicating a performance failure which we assume would trigger attributional processing.

After the video was completed, the participants were asked to complete Liden and Maslyn’s (1998) LMX measure, Tepper’s (2000) Abusive Supervision Scale, and Walumbwa, Avolio, Gardner, Wernsing, and Peterson’s (2008) Authentic Leadership Questionnaire, which were presented in random order. These questionnaires were selected because of their frequent use in the leadership literature (citation rates for 2005 to 2015 were 842, 1,320, and 880, respectively, for the three measures), they have demonstrated acceptable reliabilities, and because these measures reflected both positive (i.e., LMX, authentic leadership) and negative leadership perceptions (i.e., abusive supervision) and both older and well-established (i.e., LMX) and newer leadership measures (i.e., authentic leadership, abusive supervision). Furthermore, given the need to balance the length of the study with the need to identify established measures that would work in the context of the study, these three measures were deemed as providing a good fit. Finally, participants answered questions regarding their demographic information.

Measures

Leadership Characteristics Questionnaire (LCQ)

The LCQ (Epitropaki & Martin, 2004) was designed to be a short measure of ILTs to be used in organizations. Participants were instructed to rate how characteristic 21 traits are of a good leader on a 9-point scale. The traits included both positive-focused (e.g., understanding, sincere, helpful) and negative-focused (e.g., domineering, pushy, manipulative) characteristics. In line with prior research (e.g., Christie, Barling, & Turner, 2011; Epitropaki & Martin, 2005), we computed an average prototypical (13 items; α = .84) and antiprototypical ILT (eight items; α = .83) score for each participant.

Organizational Attribution Style Questionnaire (OASQ)

We used three scenarios from the OASQ by Kent and Martinko (1995) to assess the participants’ general attribution styles for workplace events. Although this measure includes both positive and negative outcome scenarios, we selected only negative outcome scenarios because, based on attribution theory, negative outcomes are more likely to cue attributional processing (Martinko, Harvey, et al., 2007; Martinko, Moss, et al., 2007; Weiner, 1986). Subjects were asked to imagine themselves in each scenario and to indicate the extent to which they believed that the causes of the negative outcome are due to internal or external, stable or unstable, and controllable or uncontrollable factors.

A sample scenario was “You failed to receive a promotion that you wanted for a long time” and the response stems were “To what extent is the failure to receive a promotion caused by something about you versus other people or circumstances” (i.e., internal/external), “To what extent will the things that caused the failure to receive the promotion be present in the future in similar situations” (i.e., stable/unstable), and “To what extent is the cause of your not getting a promotion due to controllable or uncontrollable things” (i.e., controllable/uncontrollable). Responses were recorded on a 7-point scale (e.g., 1 = completely due to me, 7 = completely due to other people or circumstances). Following Kent and Martinko’s (1995) instructions, three subscales were calculated by averaging scores on the three items for each subscale: Locus of Causality (higher scores reflect more external attributions; α = .66), Stability (α = .76), and Controllability (α = .62).

Leader-Member Exchange (LMX)

LMX (Liden & Maslyn, 1998) is a multidimensional measure of followers’ perceptions of their relationships with their leaders. In the current study, participants rated their anticipated or expected relationship with the supervisor in the video on a five-point Likert-type scale. The measure encompasses four dimensions: affect, loyalty, contribution, and professional respect, which are assessed with 11 questions. LMX scores were calculated by averaging across the items (α = .94).

Abusive Supervision

Abusive supervision was measured using a slightly modified version of the 15-item Tepper (2000) Scale. After experiencing the actual performance evaluation with the supervisor in the video, the subjects were asked to imagine reporting to this supervisor in real life and respond to the question, “This supervisor would . . .” Several sample items include “ridicule me,” “tell me my thoughts and feelings are stupid,” and “lie to me.” Responses were recorded on a five-point scale (1 = not likely use this behavior with me, 5 = use this behavior very often with me). Scores were calculated by averaging across the items (α = .97).

Authentic Leadership Questionnaire (ALQ)

The ALQ (Walumbwa et al., 2008) is a measure of authentic leadership comprising leader self-awareness (four items), relational transparency (five items), internalized moral perspective (four items), and balanced processing (three items) as subdimensions of the higher order construct. Participants were instructed to rate how likely the supervisor in the video would be to engage in the four behavioral components of authentic leadership using a five-point scale for the 16-item measure. As the different facets were represented by different numbers of items, in order not to overweight the contribution of each subdimension, we created an equally weighted composite of the four subdimensions to represent authentic leadership (α = .94).¹

Results

For descriptive statistics and correlations between measured variables, see Table 1. To answer our research questions, we conducted analysis of covariance (ANCOVA) using a general linear modeling approach for each of the three leadership measures separately. Leader performance cues and subordinate performance cues served as between-subject factors.² Trait prototypical and antiprototypical ILTs as well as trait locus of causality, stability, and controllability attribution style dimensions were entered as covariates.

Table 1.

Descriptive Statistics and Correlations.

	Mean	SD	1	2	3	4	5	6	7
1. Prototypical ILT	7.83^a	0.73	—
2. Antiprototypical ILT	2.74^a	1.29	−.02	—
3. Locus of causality	3.51^b	1.18	.02	.18**	—
4. Stability	3.92^b	1.36	−.07	.05	−.07	—
5. Controllability	4.07^b	1.11	.01	-.06	−.44**	.08	—
6. LMX	3.37^c	1.41	.11	.03	−.19^**	−.05	.05	—
7. Abusive supervision	2.67^c	1.09	−.05	.04	.19^**	.00	−.05	−.71^**	—
8. Authentic leadership	3.04^c	0.93	.09	.13*	−.17^*	.06	.03	.71^**	−.63^**

Note. ILT = implicit leadership theory; LMX = leader–member exchange.

Measure scored on a 9-point scale. ^bMeasure scored on a 7-point scale. ^cMeasure scored on a 5-point scale.

p < .05. **p < .01.

Direct Effects

Results of our ANCOVA analyses are reported in Table 2. For LMX, the two statistically significant predictors were locus of causality attribution style, F(1, 272) = 8.19, p = .01, and the leader performance cue, F(2, 272) = 6.40, p < .01. Participants who tended to make more external causal attributions tended to give lower LMX ratings. Follow-up t tests with Bonferroni corrections for alpha inflation (Strassburger & Bretz, 2008) adjusted for covariates (see Table 3), also show that participants in the low leader performance cue condition rated LMX as significantly lower than participants in the average and high performance cue conditions (adjusted means = 2.99 vs. 3.61 and 3.60, respectively).

Table 2.

Results of Analysis of Covariance for Leader and Subordinate Performance Cues on Leadership Ratings.

	Sum of squares	df	Mean square	F	p	Partial η²
LMX (model R² = .10)
Leader performance cue	22.80	2	11.40	6.40**	.00	.05
Subordinate performance cue	8.54	2	4.27	2.40	.09	.02
Prototypical ILT	6.07	1	6.07	3.41	.07	.01
Antiprototypical ILT	3.92	1	3.92	2.20	.14	.01
Locus of Causality	14.59	1	14.59	8.19*	.01	.03
Stability	0.39	1	0.39	0.22	.64	.00
Controllability	1.11	1	1.11	0.63	.43	.00
Error	484.90	272	1.78
Abusive supervision (model R² = .09)
Leader performance cue	11.37	2	5.69	5.23*	.01	.04
Subordinate performance cue	8.15	2	5.69	3.74*	.03	.03
Prototypical ILT	1.09	1	1.09	1.00	.32	.00
Antiprototypical ILT	0.02	1	0.02	0.02	.89	.00
Locus of causality	6.85	1	6.85	6.29*	.01	.02
Stability	.16	1	0.16	0.15	.70	.00
Controllability	.73	1	0.73	0.67	.41	.00
Error	295.95	272	1.09
Authentic leadership (model R² = .12)
Leader performance cue	8.84	2	4.42	5.73**	.00	.04
Subordinate performance cue	4.21	2	2.11	2.73	.07	.02
Prototypical ILT	3.34	1	3.34	4.33*	.04	.02
Antiprototypical ILT	7.72	1	7.72	10.00**	.00	.04
Locus of causality	7.32	1	7.32	9.48**	.00	.03
Stability	1.11	1	1.11	1.44	.23	.01
Controllability	1.15	1	1.15	1.49	.22	.01
Error	209.99	272	0.77

Note. ILT = implicit leadership theory; LMX = leader–member exchange.

p < .05. **p < .01.

Table 3.

Adjusted Marginal Means by Leader and Subordinate Performance Cue Condition.

	LMX	Abusive supervision	Authentic leadership
Leader performance cue
Low	2.99^ab	2.94^ab	2.77^ab
Average	3.61^a	2.47^a	3.14^a
High	3.60^b	2.56^b	3.17^b
Subordinate performance cue
Low	3.65	2.41	3.20
Average	3.26	2.78	2.91
High	3.28	2.78	2.97

Note. Means are adjusted for the following covariates: prototypical ILT, antiprototypical ILT, causality, stability, and controllability. Means in the same column that share a superscript are significantly different from each other (at p < .05) in post hoc pairwise comparisons using Bonferroni adjustments. ILT = implicit leadership theory; LMX = leader–member exchange.

For ratings of abusive supervision (see Table 2), the leader performance cue, F(2, 272) = 5.23, p = .01, and the subordinate performance cue, F(2, 272) = 3.74, p = .03, were statistically significant predictors. Additionally, participants who tended to make more external causal attributions tended to rate their supervisors as more abusive, F(2, 272) = 6.29, p = .01. Follow-up t tests (see Table 3) indicate that participants in the low leader performance cue condition rated the leader in the video as significantly more abusive than participants in the average or high leader performance cue conditions (adjusted means = 2.94 vs. 2.47 and 2.56, respectively). However, post hoc Bonferroni corrected t tests indicate that there were no statistically significant differences between the three subordinate performance cue conditions (adjusted means = 3.65, 3.26, and 3.28 for low, average, and high subordinate performance cues, respectively).

In predicting authentic leadership ratings (see Table 2), leader performance cues were once again a significant predictor, F(2, 272) = 5.73, p < .01. Follow-up t tests (see Table 3) again demonstrate that the differences were centered in the low leader performance cue condition relative to the average and high leader performance cue conditions (adjusted means = 2.77 vs. 3.14 and 3.17, respectively). Additionally, locus of causality attribution style, F(2, 272) = 9.48, p < .01, prototypical ILT, F(2, 272) = 4.33, p = .04, and antiprototypical ILT, F(2, 272) = 10.00, p < .01, were significant predictors of authentic leadership ratings. Participants who tended to make more external causal attributions rated the leader lower on authentic leadership. Furthermore, participants who rated ideal leaders as possessing more prototypical characteristics (e.g., dedicated, intelligent, sensitive, dynamic) rated the leader in the video as higher on authentic leadership. Somewhat surprisingly, participants who rated ideal leaders as possessing more antiprototypical characteristics (e.g., tyrannical, masculine) also rated the leader in the video as higher on authentic leadership.

Interactions Between Attribution Styles

Next, we examined Hypotheses 2 (a and b). Across the three leadership measures, we found no significant two- or three-way interactions between attribution styles in predicting leadership ratings. Planned contrasts comparing individuals with hostile attribution styles (i.e., those above the sample median on locus of causality, stability, and controllability) with those who do not revealed no significant differences in leadership ratings—LMX, t(280) = 1.35, p = .18; abusive supervision, t(280) = −0.71, p = .48; authentic leadership, t(280) = .12, p = .90. These comparisons also remained nonsignificant in ANCOVA analyses controlling for leader and follower prior performance cue and ILTs.

However, some authors have focused on just the locus of causality and stability dimensions to define hostile attribution style (e.g., Martinko, Harvey, Sikora, et al., 2011). Therefore, we repeated the planned contrasts, this time defining hostile attribution style as individuals above the sample median on locus of causality and stability dimensions. These analyses revealed significant differences in leadership ratings for LMX, t(280) = 2.58, p = .01, and abusive supervision, t(280) = −2.00, p = .046, but not for authentic leadership, t(280) = 1.60, p = .11. Individuals with a hostile attribution style generally viewed their leader more negatively than those with other attribution styles (LMX mean = 3.09 vs. 3.55; abusive supervision mean = 2.84 vs. 2.57; authentic leadership mean = 2.90 vs. 3.09). Results were similar in ANCOVA analyses controlling for leader and follower prior performance cue and ILTs, though the effect of hostile attribution style on abusive supervision, F(1, 274) = 3.79, p = .053, and authentic leadership, F(1, 274) = 3.82, p = .055, ratings became marginally significant. Thus, there is some evidence that hostile attribution style predicts leadership ratings.

Post Hoc Analyses

Sample Comparability

Since the sample for the study was recruited from three different universities we investigated whether or not there were significant differences in the three data sources. One-way ANOVAs indicated that participants from the three samples did differ on some demographic characteristics including age, F(2, 260) = 53.43, p < .01, years of work experience, F(2, 260) = 39.46, p < .01, job tenure, F(2, 260) = 18.62, p < .01, and average number of hours worked per week, F(2, 260) = 30.09, p < .01. Given concerns of nonequivalence across samples, we reran our analyses controlling for sample source. The patterns of results were similar and did not affect our conclusions.

Factor Covariate Interactions

Homogeneity of regression is an assumption underlying ANCOVA. This assumption is typically tested by examining whether there are significant factor covariate interactions (Cohen & Cohen, 1983). In our case, factors refer to our manipulations of leader and follower prior performance cues, and covariates were trait prototypical and antiprototypical ILTs as well as locus of causality, stability, and controllability attribution styles. Out of 30 omnibus F tests (i.e., 5 Covariates × 2 Manipulations × 3 Dependent variables), only one factor covariate interaction (i.e., Stability × Leader performance cue in predicting authentic leadership) was statistically significant (F[2, 252] = 3.76, p = .03). Since this reflects significance at chance levels (< 5%), we believe that our use of ANCOVA is appropriate and this significant interaction is unlikely to be replicated in other samples. Details regarding these analyses are available from the authors on request.

Interactions Between Trait ILTs and Attribution Styles

Although we did not originally hypothesize interactions between ILTs and attributions styles, we examined whether ILTs and attribution style dimensions interacted to predict leadership ratings (see Table 4). We chose to do this analysis to more fully explore how predispositions may bias leadership ratings. Results show that trait locus of causality consistently interacted with ILTs to predict leadership ratings. Specifically, the locus of causality dimension interacted independently with both prototypical ILTs, F(1, 266) = 5.50, p = .02, and antiprototypical ILTs, F(1, 266) = 10.50, p < .01, to predict LMX ratings. Additionally, there was also a significant locus of causality dimension by antiprototypical ILT interaction in predicting abusive supervision ratings, F(1, 266) = 6.73, p = .01, and a significant locus of causality dimension by prototypical ILT interaction in predicting authentic leadership ratings, F(1, 266) = 8.43, p < .01.

Table 4.

Results of Analysis of Covariance for Leader and Subordinate Performance Cues on Leadership Ratings With ILT and Attributional Style Interactions.

	Sum of squares	df	Mean square	F	p	Partial η²
LMX (model R² = .17)
Leader performance cue (LPC)	23.95	2	11.97	7.13**	.00	.05
Subordinate performance cue (SPC)	6.20	2	3.10	1.85	.16	.01
Prototypical ILT (P-ILT)	0.30	1	0.30	0.18	.67	.00
Antiprototypical ILT (AP-ILT)	5.80	1	5.80	3.45	.06	.01
Locus of causality	11.01	1	11.01	6.56*	.01	.02
Stability	2.47	1	2.47	1.47	.23	.01
Controllability	0.83	1	0.83	0.49	.48	.00
P-ILT × Locus of causality	9.24	1	9.24	5.50*	.02	.02
P-ILT × Stability	2.13	1	2.13	1.27	.26	.01
P-ILT × Controllability	0.90	1	0.90	0.53	.47	.00
AP-ILT × Locus of causality	17.63	1	17.63	10.50**	.00	.04
AP-ILT × Stability	0.22	1	0.22	0.13	.72	.00
AP-ILT × Controllability	0.00	1	0.00	0.00	.97	.00
Error	446.87	266	1.68
Abusive supervision (model R² = .16)
Leader performance cue (LPC)	11.34	2	5.67	5.49*	.01	.04
Subordinate performance cue (SPC)	6.92	2	3.46	3.35*	.04	.03
Prototypical ILT (P-ILT)	0.11	1	0.11	0.11	.74	.00
Antiprototypical ILT (AP-ILT)	3.81	1	3.81	3.69	.06	.01
Locus of causality	3.41	1	3.41	3.30	.07	.01
Stability	0.34	1	0.34	0.33	.57	.00
Controllability	1.89	1	1.89	1.83	.18	.01
P-ILT × Locus of causality	2.83	1	2.83	2.74	.10	.01
P-ILT × Stability	0.23	1	0.23	0.22	.64	.00
P-ILT × Controllability	2.13	1	2.13	2.06	.15	.01
AP-ILT × Locus of causality	6.95	1	6.95	6.73*	.01	.03
AP-ILT × Stability	0.26	1	0.26	0.25	.62	.00
AP-ILT × Controllability	0.07	1	0.07	0.07	.80	.00
Error	274.83	266	1.03
Authentic leadership (model R² = .19)
Leader performance cue (LPC)	9.69	2	4.85	6.63**	.00	.05
Subordinate performance cue (SPC)	3.40	2	1.70	2.32	.10	.02
Prototypical ILT (P-ILT)	0.69	1	0.69	0.95	.33	.00
Antiprototypical ILT (AP-ILT)	0.02	1	0.02	0.03	.87	.00
Locus of causality	5.70	1	5.70	7.81**	.01	.03
Stability	0.62	1	0.62	0.85	.36	.00
Controllability	0.48	1	0.48	0.66	.42	.00
P-ILT × Locus of causality	6.16	1	6.16	8.43**	.00	.03
P-ILT × Stability	0.53	1	0.53	0.73	.39	.00
P-ILT × Controllability	0.19	1	0.19	0.26	.61	.00
AP-ILT × Locus of causality	1.86	1	1.86	2.55	.11	.01
AP-ILT × Stability	0.77	1	0.77	1.05	.31	.00
AP-ILT × Controllability	1.47	1	1.47	2.01	.16	.01
Error	194.35	266	0.73

Note. ILT = implicit leadership theory; LMX = leader–member exchange.

p < .05. **p < .01.

These significant interactions are depicted in Figure 1. Simple slopes analyses indicate that the relationship between ILTs and leadership ratings were consistently nonsignificant among participants who have a more external causal attribution style; these participants generally rated the leader in the video more negatively regardless of their ILT scores. However, simple slopes were generally significant for the relationship between ILTs and leadership ratings among participants who have a more internal attribution style (marginal at p < .06 for the abusive supervision interaction). In all four cases, participants who had styles that biased them toward making internal causal attributions and who rated the listed leader traits as more characteristic of good leaders (for both prototypical and antiprototypical traits) rated the leader in the video most favorably (i.e., higher LMX and authentic leadership and lower abusive supervision ratings).

Figure 1.

Graphical representations of significant interactions between ILTs and attributional styles in predicting leadership ratings.

Associations Between Leadership Measures

It is also worth noting the high intercorrelations between the LMX, abusive supervision, and the authentic leadership measures documented in Table 1. These findings are consistent with Martinko, Harvey, Sikora, et al. (2011), which found that the LMX and abusive supervision measures were highly correlated and apparently confounded.

Discussion

All our hypotheses were at least partially supported. Leadership performance cues accounted for significant variance in the responses to all three leadership questionnaires. The dimensions of attribution styles, particularly the interaction of the dimensions underlying hostile attribution style, also accounted for significant variance in subordinates’ responses to the leadership questionnaires. Participants who tended toward external (and stable) attributions for failure, which likely included blaming others such as their supervisors, rated the leader more negatively on all three questionnaires. ILT direct effects accounted for significant variance in authentic leadership ratings and also interacted with the dimension of locus of causality to predict leadership ratings for all three of the leadership questionnaires.

Direct Effects

It is important to mention how ILTs influenced respondents’ ratings. Note that our measure of ILTs focuses on the content of leadership schemas. Higher scores for both prototypical and antiprototypical leader traits were associated with rating the leader as more authentic, but were not associated with either LMX or abusive supervision ratings. Although the findings regarding the positive relationship between perceptions of authenticity and antitypical leader behavior may seem contradictory, we view the finding as consistent with authentic leadership theory. A major tenet of authentic leadership is that authentic leaders act in a way that is consistent with their values and beliefs, which they share with their subordinates (Gardner, Cogliser, David, & Dickens, 2011). As a result, their behavior is transparent and followers are better able to predict and respond to their leader. Thus, leaders who display antiprototypical leader characteristics may nonetheless be viewed as revealing their true selves and are, therefore, perceived as more genuine (i.e., authentic).

The findings for attribution styles are also interesting and are supported by prior research. Specifically, as respondents became more external in their attribution styles (or possessed a hostile attribution style) they perceived lower quality LMX and that the leader was more abusive and less authentic. Similar results for LMX are reported by Martinko, Moss, et al. (2007), and similar results for abusive supervision are reported by Martinko, Harvey, Sikora, et al. (2011). Note, however, that these prior studies used cross-sectional designs with different subordinates rating different leaders and, therefore, our present results allow us to make stronger inferences regarding the cause and directionality of effects. Our findings that link follower attribution styles with ratings of authentic leadership are believed to be a new contribution to the literature.

Post Hoc Analyses

Interactions

Perhaps the most interesting set of findings involved interactions between participants’ trait ILTs and the locus of causality dimension of attribution style on leadership ratings (see Figure 1). Generally, participants who tended toward making external attributions rated the leader more unfavorably regardless of whether they were high or low for both types of leader prototypicality. This was relatively consistent for ratings on all three leadership scales and is consistent with research on external attribution styles; these individuals tend to place the blame on external sources, such as the supervisor. It also suggests that individuals who tend toward external attributions for failure disregard information that might lead them to different conclusions. Thus, external attribution style may be an important factor that predisposes subordinates to be less context-sensitive than others (Foti et al., 2008) and more rigid (or less dynamic) in their evaluations of leaders.

Before attempting to explain these interactions, it is important to note that the six-minute role-play by the actor playing the supervisor contained a fairly wide variety of what could be considered prototypical (e.g., understanding, sincere, and helpful) as well as antiprototypical (e.g., domineering and pushy) leader behaviors. As a result, evidence of both types of characteristics was available to the participants. Consistent with an information-processing perspective, we believe that the key to explaining the differences in participants’ ratings lies in understanding how ILTs and attribution styles affect the information that the participants use as the basis for their judgments of leader behavior. For respondents who tend to use internal attributions and to be high on their expectations of both prototypical and antiprototypical leader traits, they tend to perceive high-quality leader–member relations. Although we did not have an a priori hypothesis for this interaction, an explanation consistent with attribution theory is that those participants who tended toward internal attribution styles assumed some responsibility for their poor performance evaluations (i.e., made an internal attribution). As a result, they perceived the low ratings of their performance and the behavior of the supervisor as consistent with what a good leader (prototypical) would do and also see the presence of behaviors that they perceive as antiprototypical in normal circumstances (i.e., when they are performing well) to be appropriate in this circumstance because of their (i.e., the subordinate’s) poor performance. Thus, they rated their leader relatively favorably despite receiving a negative performance review.

The effects of the interactions between ILTs and the dimension of locus of causality attribution styles on ratings of authentic leadership can also be explained similarly. In this case, participants who tend toward internal attribution styles and were high in their expectations of prototypical leader behaviors tended to rate the leaders as more authentic. Assuming that the participants who favored internal attribution styles attributed their poor performance appraisal to themselves (an internal attribution), when they were also high on prototypical ILTs they viewed the leader’s feedback as more appropriate because they were cued to attend to the understanding and helpful behaviors (i.e., prototypical behaviors) that were enacted by the supervisor. As a result, they rated the leader relatively more favorably despite the negative outcome for the performance appraisal.

With regard to perceptions of abuse, we expected that the participants who tended toward internal attributions likely assumed responsibility for their poor performance. As a result, those participants who were also high on their ratings for antiprototypical ILTs probably saw the supervisor’s antiprototypical behaviors (e.g., the negative feedback as well as domineering and pushy behaviors) as more appropriate and less abusive than participants who took less responsibility for their performance ratings.

Contributions

Since all participants were exposed to the exact same supervisor behaviors, differences across conditions should not have been found if participants’ leadership ratings are high-fidelity responses to actual leader behavior. Our results demonstrate that the exact same leader behaviors are viewed differently by different respondents and that the differences in these respondents’ perceptions are accounted for, at least in part, by their ILTs and attribution styles as well as knowledge of prior leader performance. Although such a finding may seem unsurprising considering prior research and the limited information participants received concerning their supervisor, it should be noted that participants provided ratings of their ILTs and attributional styles before their exposure to the role-playing scenario, suggesting that our results cannot be simply explained by the demand characteristics of the study.

Overall, our predictors accounted for around 10% of the variance in each type of leadership rating. We view this finding as providing an important challenge to the practical utility of these measures. More specifically, 10% of the variance could easily account for the significant differences found in studies with large samples and small effect sizes, despite the arguments that small effect sizes can be very important (see Prentice & Miller, 1992). Although some of the relationships found in our study have been suggested by studies with cross-sectional designs comparing the ratings of different subordinates with different supervisors (e.g., Martinko, Harvey, et al., 2007; Martinko, Moss et al., 2007), our ability to have all of the respondents react to the same leader enacting the same behaviors in a controlled experimental design provides a much stronger case for the importance and nature of these relationships (e.g., Brown & Lord, 1999).

Although the finding that preexisting schema such as ILTs and attribution styles as well as performance information affect perceptions of leadership is not entirely new, the demonstration that these aspects of information processing bias the leadership questionnaires that are being used currently is a unique contribution, as is our discovery of consistent interactions between ILTs and attribution styles in predicting leadership ratings. In other words, although some prior research exists that shows that ILTs can serve as independent variables that influence subordinates’ leadership perceptions (e.g., Johnson et al., 2008; Whiteley, Sy, & Johnson, 2012), we believe that our study makes an incremental contribution to prior research by examining multiple influences simultaneously (i.e., attribution styles and performance cues as well as ILTs) and examining both their direct and interactive effects, the latter of which has not previously been demonstrated. As we pointed out in the introduction, although there is evidence that earlier measures were affected by these types of evaluation biases, very little has been done in the past two decades that evaluates the effects of informational biases on the construct validity of the types of leadership questionnaires that are commonly used today.

Some discussion regarding the differences in our findings compared with earlier studies of the impact of ILTs on questionnaire ratings is warranted. Eden and Leviatan (1975) and Rush et al. (1977) were focused on determining the factor structure that emerged when followers had knowledge versus no knowledge of the organization and its leaders. They conclusively demonstrated that the factor structures of groups without information versus knowledgeable groups were quite similar, thus, demonstrating ILTs. However, ILTs were an outcome in their studies. They did not measure ILTs beforehand and then assess their impact on different questionnaires or ratings. Thus, a major difference and contribution of our study is that it uses ILTs, attribution styles, and performance cues as independent variables and demonstrates their effects on existing leadership questionnaires. Although we recognize that our contribution is incremental, the demonstration that these aspects of information processing bias currently used leadership questionnaires is a unique and important contribution.

In addition to demonstrating the effects of ILTs, attribution style, and performance information on the construct validity of leadership questionnaires, our study also contributes to the dynamic nature of leader perception by describing how these different units of information contribute to leader evaluations. More specifically, it details both the direct effects of ILTs, attribution styles, and performance information as well as their interactive effects. Although neither ILTs nor attribution styles interacted with performance cues, the interaction between ILTs and attribution styles, as well as the effects of hostile attribution style, significantly predicted respondents’ leadership ratings. In general, respondents who more strongly endorsed ILTs and possessed more internal attribution styles tended to rate their leaders more favorably. It was also interesting to find that participants who tended toward external attribution styles generally rated the leaders more negatively regardless of their ILTs and performance cues. Thus, our results make a contribution by detailing some of the specific units of information that affect leadership ratings and the dynamics of these interactions.

As we indicated in our introduction, we assumed a model of information processing where both attributional and categorization processes are used concurrently to form leadership perceptions. Our results demonstrate that ILTs (categorization processes) interact with attribution styles, which suggest an interactive model. It is important to note that past support for the categorization model (e.g., Cronshaw & Lord, 1987; Phillips & Lord, 1982) was based on laboratory studies with time constraints. These conditions are in contrast with most leadership situations that unfold over significant periods of time allowing for both categorization and attributional judgment processes. Thus, we believe that an interactive model is probably more applicable to judgments of leadership that occur under normal circumstances that are not constrained by time. Thus, an interactive model may be a useful way of conceptualizing how subordinates judge their leaders when they are responding to leadership questionnaires under normal circumstances.

Limitations and Suggestions for Future Research

Although this study used random assignment, creating relatively homogenous groups and providing strong internal validity (see Mook, 1983), one limitation of our study is that the respondents were reacting to a limited sample of supervisory behaviors in a role-playing simulation. It could be argued that these same relationships might not be found in long-term leader–member dyads. We agree, but note as described above, the nature and directions of the relationships we found are consistent with theoretical and empirical work that has been conducted in actual work settings among real leader–follower dyads. In addition, we also note that it is also quite possible that the relationships we found may be stronger in long-term leader–member dyads. We are hopeful that constructive replications of this study will follow in which these same relationships are explored in other contexts. Designs with groups of subordinates who share the same supervisors are needed to have more confidence in the relationships we found. However, we note that such designs may also have difficulties disentangling leader behaviors from biases in follower perceptions as the same leader is likely to engage in different behaviors with different subordinates, as suggested by LMX theory (Henderson, Liden, Glibkowski, & Chaudhry, 2009) and found in existing empirical work (e.g., Boies & Howell, 2006; Liden, Erdogan, Wayne, & Sparrowe, 2006). Nevertheless, we are hopeful that our findings provide a basis for further exploration of the issues we have addressed.

Another limitation is that the sample was recruited from three different universities. As indicated earlier, our analyses indicated that differences between the sample sources were unlikely to have affected our results and conclusions. Nevertheless additional studies would be helpful to further explore the validity of our results and conclusions. Additionally, although ILTs were significant direct predictors of some leadership ratings, we did not find the strength of support we anticipated. Since the development of these scales is relatively recent (Epitropaki & Martin, 2004), refinements as well as other methods for assessing ILTs (i.e., indirect measures such as word fragment tasks, projective tests, and implicit association tests; see Epitropaki et al., 2013, for a review) may yield stronger results in the future.

The relatively low reliabilities for attribution styles is also a limitation of our study. During the design of the study, we were concerned about response fatigue and, as a result, elected to use only three scenarios from the OASQ. We expect that the low reliabilities are a function of the limited number of scenarios included, since prior studies using the full measure have consistently reported higher reliabilities (α = .77-.89; Martinko et al., 2006). In addition, in a recent discussion regarding the reliability of the OASQ, Harvey (2015) noted that because the OASQ queries respondents about situations that the respondents may have encountered as opposed to evaluations of leader behavior, internal consistency reliabilities may be an inadequate method for assessing the OASQ’s reliability. As a result, the reliabilities of the OASQ may be lower than typical leadership questionnaires.

Some comment regarding the purpose and utility of subordinates’ assessments of leadership is warranted. An underlying assumption of these scales appears to be that the information obtained will enable researchers to identify and understand the factors that contribute to effective leadership and desirable organizational outcomes. As we summarized in the introduction, research on subordinate-based measures of leadership has shown that these measures are related to a variety of desirable outcomes such as employee satisfaction, well-being, and performance. Nevertheless, our study indicates that these assessments are biased by factors such as attribution styles, performance information, and ILTs. As a result, the validity of subordinates’ assessments of their leaders can be questioned. Also because much of the research has been cross-sectional as opposed to longitudinal, it is difficult to know the causal relationships between subordinates’ evaluations of their leaders and outcome measures. Clearly more research is needed to understand how subordinate biases affect their evaluations and the dynamics of the relations between subordinates’ perceptions of their leaders and organizational outcomes such as employee satisfaction and productivity.

Finally, the high intercorrelations among the various measures of leadership deserves more investigation. Although the leadership behaviors depicted by the various leadership theories are quite different, the high intercorrelations suggest that these measures are confounded and essentially measuring the same variance. As a result, it is difficult to make any specific recommendations regarding how specific supervisory behaviors might affect subordinate evaluations and leadership outcomes. More research needs to be done to understand the nature of the variance associated with leadership measures.

Conclusion

An impressive body of work has been accumulated that unequivocally demonstrates that judgments of leadership are influenced by preset cognitive schemas and information about performance (e.g., Brown & Lord, 1999; Epitropaki et al., 2013; Foti et al., 2008). Our article makes two unique contributions to this literature. First, researchers have recently theorized about how the components of information interconnect to effect responses to leadership questionnaires (Hansbrough et al., 2015). Our article contributes to this literature by providing details regarding the direct effects of two preset cognitive schemas (ILTs and attribution styles) and performance cues on perceptions of leadership. More important, it details the interactions among and between these different informational components by demonstrating how ILTs and attribution styles interact to predict leadership perceptions. The finding that individuals who strongly endorsed ILTs and tended toward internal attribution styles tended to perceive their leaders more favorably is a unique contribution. It was also important to find that respondents with external attribution styles tended to rate their leaders less favorably regardless of their ILTs and performance information. This suggests that subordinates with external attribution styles were less dynamic and more static in their information-processing capacities and may typically rate their leaders unfavorably regardless of other types of information. It may be that there is little leaders can do or say to enhance subordinate leadership perceptions when subordinates have external attribution styles.

The second major contribution of our article is that we demonstrate how this body of research affects the construct validity of a sample of leadership questionnaires that are frequently used in current research. Our study demonstrates that respondents’ ILTs, attribution styles, and performance information bias peoples’ evaluations of leadership. More specifically, our study unequivocally demonstrates that respondents observing the same leader performing the exact same behaviors evaluate their leaders differently as a function of their ILTs and attribution styles as well information available concerning leader performance. As suggested long ago by Eden and Leviatan (1975), “Such scales may be reliable measures of (mis)conceptions rather than perceptions” (p. 741). As a result, we cannot conclude definitively that leaders who are rated favorably or unfavorably by their subordinates behave in a way that results in better leader–member relations than others, are more or less abusive than others, or behave in ways that are more or less authentic than others.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a research incubator grant from the Southern Management Association.

Notes

Author Biographies

Mark J. Martinko is the Anheuser-Busch Eminent Scholar of Organizational Behavior at Florida A & M University. He formerly was Professor of Management at the University of Queensland, Australia and was the Bank of America Professor of Management at Florida State University where he is a professor emeritus. His research focuses on attribution theory which he has applied to the areas of motivation, leadership, impression management, emotions, organizational deviance, abusive supervision, and entitlement. He has authored, co-authored, or edited eight books and more than 100 articles. He is an Associate Editor for the Journal Organizational Behavior and serves on the editorial boards of three other journals.

Brandon Randolph-Seng (PhD, Texas Tech University) is an associate professor of Management at Texas A&M University–Commerce. His research interests include the social and cognitive factors involved in leadership, groups, and entrepreneurship and he has published in such outlets as the Academy of Management Review, Behavior & Brain Sciences, and Leadership Quarterly. He also serves as an associate editor for Management Decision.

Winny Shen is an assistant professor of industrial/organizational psychology at the University of Waterloo. Currently, her primary research interests center on diversity and inclusion issues and leadership in the workplace. She also has secondary research interests in personality and occupational health psychology (i.e., work-family issues and worker health and well-being).

Jeremy R. Brees is an assistant professor of Management at Northern Arizona University. His research interests focus on attributions, accountability, abusive supervision, workplace perceptions, and personality. His articles have appeared in such publications as the Journal of Management, Journal of Organizational Behavior, Journal of Vocational Behavior, and Journal of Leadership and Organizational Studies.

Kevin T. Mahoney (PhD, University of Akron) is an assistant professor and I/O Psychology MS Program Coordinator, South Dakota State University. His research interests focus on emotions, decision making, and historical processes in psychology and organizational behavior. His articles have appeared in such journals as Journal of Business and Psychology, Journal of Occupational Health Psychology, Journal of Vocational Behavior, Personality and Individual Differences, and The Leadership Quarterly.

Stacey R. Kessler (PhD, University of South Florida) is an associate professor of management at Montclair State University’s School of Business. Her research interests include counterproductive work behavior (deviant behaviors in the workplace), leadership, and organizational climate/structure. She is the author of peer-reviewed journal articles in outlets such as the International Journal of Management Reviews, Journal of Management, the Journal of Organizational Behavior, and the Journal of Vocational Behavior.

References

Boies

Howell

J. M.

(2006). Leader-member exchange in teams: An examination of the interaction between relationship differentiation and mean LMX in explaining team-level outcomes. Leadership Quarterly, 17, 246-257.

Brees

J. R.

Martinko

Harvey

(2016). Abusive supervision: Subordinate personality or supervisor behavior? Journal of Managerial Psychology, 31, 405-419.

Brown

D. J.

Lord

R. G.

(1999). The utility of experimental research in the study of transformational/charismatic leadership. Leadership Quarterly, 10, 531-539.

Christie

Barling

Turner

(2011). Pseudo-transformational leadership: Model specification and outcomes. Journal of Applied Social Psychology, 41, 2943-2984.

Cohen

(1983). Applied multiple regression/correlation analysis for the behavioral sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.

Coleman

P. T.

(2004). Implicit theories of organizational power and priming effects on managerial power-sharing decisions: An experimental study. Journal of Applied Social Psychology, 34, 297-321.

Cronshaw

S. F.

Lord

R. G.

(1987). Effects of categorization, attribution and encoding processes on leadership perceptions. Journal of Applied Psychology, 72, 97-106.

Day

D. V.

(2012). Leadership. In Kozlowski

S. W. J.

(Ed.), The Oxford handbook of organizational psychology (Vol. 1, pp. 696-729). New York, NY: Oxford University Press.

Den Hartog

D. N.

House

R. J.

Hanges

P. J.

Ruiz-Quintanilla

S. A.

Dorfman

P. W.

GLOBE Associates. (1999). Culture specific and cross-culturally generalizable implicit leadership theories: Are attributes of charismatic/transformational leadership universally endorsed? Leadership Quarterly, 10, 219-256.

10.

Dinh

J. E.

Lord

R. G.

Gardner

W. L.

Meuser

J. D.

Liden

R. C.

(2014). Leadership theory and research in the new millennium: Current theoretical trends and changing perspectives. Leadership Quarterly, 25, 36-62.

11.

Eden

Leviatan

(1975). Implicit leadership theory as a determinant of the factor structure underlying supervisory behavior scales. Journal of Applied Psychology, 60, 736-741.

12.

Epitropaki

Martin

(2004). Implicit leadership theories in applied settings: Factor structure, generalizability and stability over time. Journal of Applied Psychology, 89, 293-310.

13.

Epitropaki

Martin

(2005). From ideal to real: A longitudinal study of the role of implicit leadership theories on leader-member exchanges and employee outcomes. Journal of Applied Psychology, 90, 659-676.

14.

Epitropaki

Martin

Tram-Quon

Topakas

(2013). Implicit leadership and followership theories “in the wild”: Taking stock of information-processing approaches to leadership and followership in organizational settings. Leadership Quarterly, 24, 858-881.

15.

Farris

G. F.

Lim

F. G.

(1969). Effects of performance on leadership, cohesiveness, influence, satisfaction, and subsequent performance. Journal of Applied Psychology, 53, 490-497.

16.

Foti

R. J.

Knee

R. E.

Jr. Backert

R. S. G.

(2008). Multi-level implications of framing leadership perceptions as a dynamic process. Leadership Quarterly, 19, 178-194.

17.

Gardner

W. L.

Cogliser

C. C.

Davis

K. M.

Dickens

M. P.

(2011). Authentic leadership: A review of the literature and research agenda. Leadership Quarterly, 22, 1120-1145.

18.

Gioia

D. A.

Sims

H. P.

(1983). Perceptions of managerial power as a consequence of managerial behavior and reputation. Journal of Management, 9, 7-24.

19.

Hanges

P. J.

Lord

R. G.

Dickson

M. W.

(2000). An information-processing perspective on leadership and culture: A case for connectionist framework. International Review of Applied Psychology, 49, 133-161.

20.

Hansbrough

T. K.

Lord

R. G.

Schyns

(2015). Reconsidering the accuracy of follower leadership ratings. Leadership Quarterly, 26, 220-237.

21.

Harvey

(2015). Assessing reliabilities for the OASQ. In Martinko

M. J.

Harvey

Brees

Mackey

J. D.

(2015). Measuring attributions: A panel symposium. Symposium presentation at the Southern Management Association Conference, St. Pete Beach, FL.

22.

Harvey

Madison

Martinko

M. J.

Crook

T. R.

Crook

(2014). Attribution theory in the organizational sciences: The road less traveled and the path ahead. Academy of Management Perspectives, 28, 128-146.

23.

Heider

(1958). The psychology of interpersonal relations. New York, NY: Wiley.

24.

Henderson

D. J.

Liden

R. C.

Glibkowski

B. C.

Chaudhry

(2009). LMX differentiation: A multilevel review and examination of its antecedents and outcomes. Leadership Quarterly, 20, 517-534.

25.

Higgins

E. T.

(1996). Knowledge activation: Accessibility, applicability, and salience. In Higgins

E. T.

Kruglanski

A. W.

(Eds.), Social psychology: Handbook of basic principles (pp. 133-168). New York, NY: Guilford Press.

26.

Hogg

M. A.

Martin

Epitropaki

Mankad

Svensson

Weeden

(2005). Effective leadership in salient groups: Revisiting leader-member exchange theory from the perspective of the social identity theory of leadership. Personality and Social Psychology Bulletin, 31, 991-1004.

27.

Hunter

S. T.

Bedell-Avers

K. E.

Mumford

M. D.

(2007). The typical leadership study: Assumptions, implications, and potential remedies. Leadership Quarterly, 18, 435-446.

28.

Johnson

S. K.

Murphy

S. E.

Zewdie

Reichard

R. J.

(2008). The strong, sensitive type: Effects of gender stereotypes and leadership prototypes on the evaluation of male and female leaders. Organizational Behavior and Human Decision Processes, 106, 39-60.

29.

Keller

(1999). Images of the familiar: Individual differences and implicit leadership theories. Leadership Quarterly, 10, 590-607.

30.

Kelley

H. H.

(1973). The processes of causal attribution. American Psychologist, 28, 107-128.

31.

Kent

Martinko

M. J.

(1995). The development and evaluation of a scale to measure organizational attribution style. In Martinko

M. J.

(Ed.), Attribution theory: An organizational perspective (pp. 53-75). Delray Beach, FL: St. Lucie Press.

32.

Liden

R. C.

Erdogan

Wayne

S. J.

Sparrowe

R. T.

(2006). Leader-member exchange, differentiation, and task interdependence: Implications for individual and group performance. Journal of Organizational Behavior, 27, 723-746.

33.

Liden

R. C.

Maslyn

J. M.

(1998). Multidimensionality of leader-member exchange: An empirical assessment through scale development. Journal of Management, 24, 43-72.

34.

Lord

R. G.

Binning

J. F.

Rush

M. C.

Thomas

J. C.

(1978). The effect of performance cues and leader behavior on questionnaire ratings of leadership behavior. Organizational Behavior and Human Performance, 21, 27-39.

35.

Lord

R. G.

Brown

D. J.

Harvey

J. L.

Hall

R. J.

(2001). Contextual constraints on prototype generation and their multilevel consequences for leadership perceptions. Leadership Quarterly, 12, 311-338.

36.

Lord

R. G.

Foti

R. J.

De Vader

C. L.

(1984). A test of leadership categorization theory: Internal structure, information processing, and leadership perceptions. Organizational Behavior and Human Performance, 34, 343-378.

37.

Lord

R. G.

Foti

R. J.

Phillips

J. S.

(1982). A theory of leadership categorization. In Hunt

J. G.

Sekaran

Schriesheim

(Eds.), Leadership: Beyond establishment views (pp. 104-121). Carbondale: Southern Illinois University Press.

38.

Lord

R. G.

Maher

K. J.

(1991). Leadership and information processing: Linking perceptions and performance. Boston, MA: Unwin Hyman.

39.

Lord

Shondrick

(2011). Leadership and knowledge: Symbolic, connectionist, and embodied perspectives. Leadership Quarterly, 22, 207-222.

40.

Martinko

M. J.

(2002). Thinking like a winner: A guide to high performance leadership. Tallahassee, FL: Gulf Coast.

41.

Martinko

M. J.

Douglas

S. C.

Harvey

(2006). Attribution theory in industrial and organizational psychology: A review. International Review of Industrial and Organizational Psychology, 21, 127-187.

42.

Martinko

M. J.

Harvey

Brees

J. R.

Mackey

(2013). A review of abusive supervision research. Leadership Quarterly, 34, 120-137.

43.

Martinko

M. J.

Harvey

Dasborough

(2011). Attribution theory in the organizational sciences: A case of unrealized potential. Journal of Organizational Behavior, 32, 144-149.

44.

Martinko

M. J.

Harvey

Douglas

S. C.

(2007). The role, function, and contributions of attribution theory to leadership: A review. Leadership Quarterly, 18, 561-585.

45.

Martinko

M. J.

Harvey

Sikora

Douglas

(2011). Perceptions of abusive supervision: The role of attribution style. Leadership Quarterly, 22, 751-764.

46.

Martinko

M. J.

Moss

Douglas

Borkowski

(2007). Anticipating the inevitable: When leader and member attribution styles clash. Organizational Behavior and Human Decision Processes, 104, 158-174.

47.

Menon

Sim

H. J.

Chiu

Hong

(2010). Blazing the trail versus trailing the group. Culture and perceptions of the leader’s position. Organizational Behavior and Human Decision Processes, 113, 51-61.

48.

Mitchell

T. R.

Larson

J. R.

Green

S. G.

(1977). Leader behavior, situational moderators, and group performance: An attributional analysis. Organizational Behavior and Human Performance, 18, 254-268.

49.

Mook

(1983). In defense of external invalidity. American Psychologist, 38, 379-387.

50.

Naidoo

L. J.

Kohari

N. E.

Lord

R. G.

DuBois

D. A.

(2010). “Seeing” is retrieving: Recovering emotional content in leadership ratings through visualization. Leadership Quarterly, 21, 886-900.

51.

Offermann

L. R.

Kennedy

J. K.

Jr. Wirtz

P. W.

(1994). Implicit leadership theories: Content, structure and generalizability. Leadership Quarterly, 5, 43-58.

52.

Paris

L. D.

Howell

J. P.

Dorfman

P. W.

Hanges

P. J.

(2009). Preferred leadership prototypes of male and female leaders in 27 countries. Journal of International Business Studies, 40, 1396-1406.

53.

Pfeffer

(1977). The ambiguity of leadership. Academy of Management Review, 2, 104-112.

54.

Phillips

J. S.

Lord

R. G.

(1981). Causal attributions and perceptions of leadership. Organizational Behavior and Human Decision Processes, 28, 143-163.

55.

Phillips

J. S.

Lord

R. R.

(1982). Schematic information processing and perceptions of leaderships in problem-solving groups. Journal of Applied Psychology, 67, 486-492.

56.

Prentice

D. A.

Miller

D. T.

(1992). When small effects are impressive. Psychological Bulletin, 112, 160-164.

57.

Ritter

B. A.

Lord

R. G.

(2007). The impact of previous leaders on the evaluation of new leaders: An alternative to prototype matching. Journal of Applied Psychology, 92, 1683-1695.

58.

Rush

Thomas

Lord

R. G.

(1977). Implicit leadership theory: A potential threat to the internal validity of leader behavior questionnaires. Organizational Behavior and Human Performance, 20, 93-110.

59.

Schneider

D. J.

(1973). Implicit personality theory: A review. Psychological Bulletin, 79, 294-309.

60.

Shondrick

S. J.

Dinh

J. E.

Lord

R. G.

(2010). Developments in implicit leadership theory and cognitive science: Applications to improving measurement and understanding alternatives to hierarchical leadership. Leadership Quarterly, 21, 959-978.

61.

Staw

B. M.

(1975). Attribution of the “causes” of performance: A general alternative interpretation of non-sectional research on organizations. Organizational Behavior and Human Performance, 13, 414-432.

62.

Strassburger

Bretz

(2008). Compatible simultaneous lower confidence bounds for the Holm procedure and other Bonferroni-based closed tests. Statistics in Medicine, 27, 4914-4927.

63.

(2010). What do you think of followers? Examining the content, structure, and consequences of implicit followership theories. Organizational Behavior and Human Decision Processes, 113, 73-84.

64.

Shore

L. M.

Strauss

Shore

T. H.

Tram

Whiteley

Ikeda-Muromachi

(2010). Leadership perceptions as a function of race-occupation fit: The case of Asian Americans. Journal of Applied Psychology, 95, 902-919.

65.

Tepper

B. J.

(2000). Consequences of abusive supervision. Academy of Management Journal, 43, 178-190.

66.

Walumbwa

F. O.

Avolio

B. J.

Gardner

W. L.

Wernsing

T. S.

Peterson

S. J.

(2008). Authentic leadership: Development and validation of a theory-based measure. Journal of Management, 34, 89-126.

67.

Weiner

(1985). An attributional theory of achievement motivation and emotion. Psychological Review, 92, 548-573.

68.

Weiner

(1986). An attributional theory of motivation and emotion. New York, NY: Springer.

69.

Whiteley

Johnson

S. K.

(2012). Leaders’ conceptions of followers: Implications for naturally occurring Pygmalion effects. Leadership Quarterly, 23, 822-834.

70.

Wittenbrink

(2007). Measuring attitudes through priming. In Wittenbrink

Schwarz

(Eds.), Implicit measures of attitudes (pp. 17-58). New York, NY: Guilford Press.