Personality on the Hill: Expert Evaluations of U.S. Senators’ Psychological Traits

Abstract

Long recognized for the diverse array of personalities it hosts, the U.S. Senate constitutes an institution in which individual psychological differences among its members carry significant potential consequences. Unfortunately, studying those individual differences is no easy task. This study introduces a new approach for doing so. Specifically, the study develops Big Five trait ratings for eighty-seven U.S. senators, with data drawn from assessments provided by a set of experts, U.S. Senate insiders. The paper explains the rationale for use of expert evaluations, offers evidence regarding the reliability and validity of the resulting measures, and explores possible relationships between personality and ten aspects of Senate behavior.

Keywords

elite personality Big Five U.S. Senate measurement

This study’s objective is to explore a new strategy for measuring the personality traits of U.S. Senators. With only one hundred members at a time and a reputation for seriousness and professionalism, “the world’s greatest deliberative body” is a context in which individuality can be especially consequential. With six-year terms, senators are less constrained by elections than are House members. Furthermore, individual senators serve on several committees simultaneously, a single senator can block a bill via a Senate hold, norms such as senatorial courtesy in judicial nominations empower individual senators, and very few senators toil in obscurity. Hence, one member’s stubbornness, expertise, hard work, comity, or malfeasance can shape the future course of policy. Unfortunately, despite its likely significance, systematic representation of this individuality is not easily accomplished. Past research has made headway via insightful case studies (e.g., Drew 1978) and large-scale analyses involving constructs that are easily measured, such as gender, race, age, and prior political experience. In contrast, work that has explored the implications of variation in senators’ personality traits has been less common.¹

In the past decade, attention to the political significance of personality has intensified, with particular focus on the Big Five framework, a well-established psychological model that highlights the trait dimensions of openness to experience, conscientiousness, extraversion, agreeableness, and emotional stability. Initial progress was seen in research on mass politics (e.g., Mondak 2010), but subsequent work has explored variation in the personality traits of state legislators (Dietrich et al. 2012) and members of the U.S. Congress (Ramey, Hollibaugh, and Klingler 2017). We build on the latter of these works in two manners. First, whereas most of Ramey et al.’s applications are to the U.S. House, we focus on the Senate. Second, and more centrally, we develop and implement a human coding of legislator personality, one that is an alternate to the self-report strategies employed in research on both mass politics and state legislatures and also to the machine-coding approach pioneered by Ramey et al.

This paper’s primary purpose is explication of a new procedure for the measurement of elite personality, one based on acquisition of data from expert coders. A secondary objective is to contribute additional evidence that systematic attention to elite personality can be fruitful. Hence, we not only explore the attributes, including the reliability and validity, of our Big Five measures, but we also test whether variation in personality among U.S. senators is related to multiple aspects of legislative behavior. Furthermore, to facilitate additional research in this area, we report in this study’s supplemental material our full personality data set, a database that includes Big Five values for more than one hundred U.S. senators who held office in the period 2009–2019.²

Strategies for Assessing Elite Personality

Three measurement approaches—self-reports, the automated extraction of information about personality from speech, and expert ratings provided by academics—have been used in past research on legislator personality. Our approach, which relies on experts who personally know the individuals they are evaluating, is a variant of a fourth method used more commonly in settings such as the workplace and the university. Discussion of all these applications provides context from which to gauge the relative merits of our measurement procedure.

Legislator Self-Reports

To our knowledge, research that asks U.S. legislators to assess their own personality traits has focused exclusively on state party leaders and members of state legislatures. Most of this work dates back four or more decades (Costantini and Craik 1980; Hennessy 1959; McConaughy 1950; Stone and Baril 1979), and only two studies have centered on the Big Five (Dietrich et al. 2012; Hanania 2017). Outside of the United States, however, there recently have been additional Big Five applications. Joly, Soroka, and Loewen (2019) acquired self-reported Ten Item Personality Inventory (TIPI) data from 272 Belgian parliament members, Nørgaard and Klemmensen (2019) obtained self-report data from a sixty-item Big Five battery administered to eighty-one Danish parliament members, and Scott and Medeiros (2020) received TIPI self-reports from 1,193 candidates in Canadian municipal elections.

Our primary concern about self-reports in the case of the U.S. Senate is that we suspect very few senators would be willing to provide us with personality self-ratings. In the Dietrich et al. (2012) study, the response rate among legislators in Arizona, Connecticut, and Maine averaged 21 percent. Compared with that mark, it seems likely the response rate among U.S. senators would be considerably lower. In the present study, we sought to obtain personality data on 154 senators and succeeded in obtaining at least one rating for 129 and two or more ratings for eighty-seven. In contrast with these returns, it strikes us as highly improbable that a solicitation of self-reports would have yielded more than a handful of ratings.

Even if cooperation from senators were obtained, we see additional problems with this strategy as applied to the U.S. Senate. First, based on our own experiences on Capitol Hill, many senators may delegate the task of filling out a survey to staffers. Second, tactical considerations mean that ratings could reflect how senators and/or their staff members prefer the senators to be depicted and perceived rather than accurate reflections on the senators’ personalities. Because these sources of variation would be unobservable to us, we would have no way to discern whether a given rating captured (1) how the senator truly gauged his or her own personality, (2) how the senator wished to be perceived, (3) how a staff member truly gauged the senator’s personality, (4) how a staff member thought the senator gauged his or her own personality, or (5) how a staff member thought it most useful to portray the senator. Such an array of possibilities would hopelessly complicate the effort to derive valid inferences from these data.

Speech-Based Measures

A notable breakthrough in the measurement of elite personality came with Ramey, Hollibaugh, and Klingler’s (2017) machine-coding of legislator speech. Hall (2018) used a similar method in a study of Supreme Court justices but with focus on the automated coding of text. These applications have yielded impressive data sets and have supported extensive analyses regarding the consequences of variation in elite personality. Still, skeptics may have qualms about this technique as applied to legislators, such as that the linguistic dictionaries used in these analyses were not designed as personality measures; the nature of speeches may change contextually as legislators move from rank-and-file to leadership positions, as they move in and out of majority status, and as elections near; speeches may reflect a mix of legislators’ traits and the stylistic tendencies of their speechwriters; and speeches show the public face of the legislator, which may differ from how that person behaves when working in back rooms with staff members and fellow legislators.

These issues do not constitute fatal flaws of speech-based measures, and Ramey, Hollibaugh, and Klingler (2017, 65–69) directly address several of these concerns. Still, we see the chance to reconsider this approach as it relates to our own. We take advantage of the availability of the Ramey et al. data to conduct two tests. In one, similar to Ramey et al. (Table 3.1, 68), we examine the relationship between legislator ideology and both the Ramey et al. personality measure and our own. In a second, we report the correlations between the two sets of personality measures.

Assessments by Academic Experts

Public officials are public figures, a reality that potentially makes information regarding their personalities available to a wide array of observers. Capitalizing on this, several teams of scholars have measured elite personality via ratings formed by analysts who most often have not had personal contact with the officials they evaluate but who arguably have devoted sufficient attention to those officials so as to be able to assess their personalities. The most common source of such “expert” ratings in published research has been the broader scholarly community.

Rubenzer and Faschingbauer (2005; Rubenzer, Faschingbauer, and Ones 2000) compiled Big Five data on forty-one U.S. presidents by asking more than one hundred experts—primarily presidential biographers—to provide assessments. The mean case was informed by 4.2 ratings. Nai and Maier (2018) developed Big Five profiles of the 2016 major-party U.S. presidential candidates, Donald Trump (28 ratings) and Hillary Clinton (33 ratings), by acquiring data from political scientists with expertise in political psychology and/or elections. Visser, Book, and Volk (2017) also developed measures of Trump and Clinton but with ratings provided by ten personality psychologists.

Consistent with this approach, we obtained data from academic raters as part of this study, with ratings provided by 109 political scientists from across the United States, most with expertise in legislative or state and local politics. As will be explained below, this was an ill-fated effort. We had qualms about surveying academics from the outset, but those qualms graduated into full-fledged grounds to reject the data set once we consulted recent studies that have explored the properties of academics’ ratings of political elites and especially once we examined the features of the ratings we received.

One concern with academics’ ratings pertains to the information base available to the coders. All or most of the information available to academics is part of the public record. Unless the academics happen to personally know the public figures they are rating, they must rely on observations from afar such as those supported by a subject’s speeches, debate appearances, writings, and political accomplishments. A risk that emerges is that relationships with political dependent variables may be tautological. For example, if the rater assumes extraverts like to give speeches and then uses information on speeches to inform a president’s extraversion score, it clearly would be perilous to regress speechmaking on extraversion. Likewise, if a rater surmises that only a president high in openness to experience would meet with leaders of adversary nations, then multivariate analyses again would be turned on their head.³

A second concern with academics’ ratings is that the raters’ own political orientations may color their assessments. If raters perceive certain traits to be desirable, they may disproportionately ascribe those traits to their favored presidents, legislators, and candidates. This would threaten validity in any circumstance, but the threat would be systematic if the errors were correlated with other constructs of interest. An example—one, as we argue below, that likely appears in the present case—is that if political scientists disproportionately hold ideologically liberal views and support Democratic public officials, personality ratings provided by those academics may, in the aggregate, cast elected Democrats unduly favorably. Given academics’ need to rely heavily on publicly available information, any such bias could be amplified by use of partisan media as information sources.

Bias of this form was identified by Wright and Tomlinson (2018; see also Wright 2019). Wright and Tomlinson note that Nai and Maier’s (2018) coders rated Hillary Clinton as much more conscientious, emotionally stable, and agreeable than Donald Trump and somewhat more open to experience. Upon surveying members of the public via MTurk, Wright and Tomlinson find the Nai and Maier scores are at odds with those provided by moderates but are consistent with ratings from Clinton voters. It was only on extraversion—arguably the most “public” of the Big Five traits, and also the one most rarely linked to ideology—that consensus emerged across coders irrespective of their own political orientations. This pattern suggests that academic coders may attribute socially desirable traits to the political elites they themselves favor, leading Wright and Tomlinson (2018, 24) to “caution against interpreting expert personality ratings of political candidates when the samples of experts are politically imbalanced.” Although acquiring data from apparent experts holds intuitive appeal, doing so may be fraught with risk (for a defense of academic coders, see Nai and Maier 2019).

Alternate Experts: Insiders

The strategy we employ is to obtain data from experts who work or have worked in various capacities in the U.S. Congress. Two streams of research underlie our use of insider ratings. First, a wealth of research in psychology demonstrates the utility of observer ratings of personality, including of the Big Five trait dimensions (e.g., Connolly, Kavanagh, and Viswesvaran 2007; Funder, Kolar, and Blackman 1995; Watson 1989). When ratings are provided by a person’s spouse or by peers, correlations between self and other ratings in excess of 0.50 are routinely obtained. Individuals’ personalities are on display, and the studies noted here establish that people who interact regularly with an individual experience little difficulty in identifying core personality traits. Unlike academics, who gain most of their information about senators from public sources, our Senate insiders know senators personally, having worked with them in multiple capacities and settings.

The second literature relevant is the work on candidate emergence led by Maisel and Stone (1997; for lessons derived from this for expert ratings, see Maestas, Buttice, and Stone 2014). With focus on possible U.S. House candidates, Maisel and Stone surveyed prominent individuals within congressional districts. In contrast with the academic experts called on to rate presidents and presidential candidates, Maisel and Stone’s informants personally knew many of the individuals they assessed or, at the least, knew someone who knew the targets. Indeed, given that many of the targets were not (yet) public officials, the raters’ assessments often had to be based on inside information. Although Maisel and Stone did not measure the Big Five, we see potential utility in a variant of their approach. Our optimism comes from the success of Maisel and Stone in obtaining information from their experts, coupled with the success of psychologists, as noted above, in measuring personality with data from observers such as spouses and peers.

Reflecting on these literatures, our broadest takeaway is that other-reports offer a potential path toward representation of the personality traits of political elites. That said, we see challenges in deciding who the others should be. Our concern about experts such as those surveyed by Maisel and Stone is less about data quality than about data availability. Entering this project, our concern was that Senate insiders would not be much more willing to rate the personalities of U.S. senators than would be the senators themselves. Although we might bemoan the use of ratings from academics, Senate insiders constitute an improvement as a data source only if they are actually willing to report their assessments.

Obtaining and Assessing Expert Reports

We gathered Big Five data from three sources: academics, Senate insiders, and a small group positioned between academics and insiders—individuals who had served in recent years as American Political Science Association (APSA) Congressional Fellows. In this section, we first describe the surveys and samples and then explain how data acquired via those surveys were compiled and used to construct Big Five variables. We then assess those variables in terms of reliability and validity.

Data Acquisition

We sought to obtain personality ratings on 154 U.S. senators who had served at any point beginning in 2009, and up to and including April 2018. Coders were asked to rate senators through application of the TIPI (Gosling, Rentfrow, and Swann 2003), a brief personality battery that represents the Big Five with data from two items per dimension. Raters were asked to assess all senators for whom the coders felt they had sufficient information to provide evaluations.

Senate insiders are individuals who work or who had worked in top staff positions in the U.S. Senate. Each was listed in the Congressional Staff Directory (e.g., Brownson 2008) and Congressional Yellow Book (e.g., Timmons 2015) as serving in one or more of the following senior roles: secretary of the senate (majority and minority), sergeant-at-arms, chief of staff, deputy chief of staff, staff director (for both committees and party conferences), chief counsel, and legislative director. Staffers were selected on the basis of these titles and across party lines. The same number of respondents reported they worked primarily or exclusively for Democrats as for Republicans, with a handful saying they had worked for independents or had held nonpartisan posts in which they interacted with senators of both parties. Finally, we also sought data from past directors and Senate liaisons who served in the White House Office of Legislative Affairs; these individuals entered the West Wing with extensive Capitol Hill experience compatible with the other Senate insiders. Every coder, thus, could claim many years of firsthand experience interacting with a wide array of senators. The sixteen insiders who participated in the study worked an average of 14.3 years in the Congress and 12.7 years in the Senate.⁴ The analyses reported below draw on 294 TIPI ratings⁵ submitted by the insiders in the period November 2017–April 2018.

Academics were contacted only via email. We developed a list of ten political science professors per state. In larger states, we prioritized scholars with teaching and research interests in the areas of U.S. legislative politics and in state politics and who had worked at colleges and universities in the state for multiple years. In some smaller states, we were unable to identify ten professors with interests in legislative and state politics, and thus we expanded our search on the basis of years working in the state. Emails were sent and data were compiled in March and April 2018. A total of 109 professors responded in this period, providing 573 TIPI ratings.⁶

Finally, we attempted to contact fifty-nine individuals who had served on Capitol Hill as APSA Congressional Fellows between late 2009 and early 2018. Congressional fellows typically are recent PhD recipients in political science assigned to work in House or Senate offices for a year. Nine individuals responded, providing a total of thirty-four TIPI ratings.

Constructing Big Five Measures

Our objective in calculating senator personality scores was to balance maximization of the quality of personality estimates with maximization of the quantity of available cases. Our plan was to integrate data from our three sets of raters to construct a summary measure. First, though, we opted to review the basic properties of the three sets of ratings. Following Wright and Tomlinson (2018, especially Table 1), we report descriptive statistics for each set of ratings, with the scores disaggregated by the party affiliations of the senators who are the subjects of the ratings.⁷

Table 1.

Partisan Differences in Three Sets of Personality Ratings of U.S. Senators.

	Democrats	Republicans	Difference	Cohen’s d
Insiders (N = 34 for Republicans, 44 for Democrats)
Openness	4.62 (0.87)	4.05 (1.05)	0.57	0.59
Conscientiousness	4.88 (0.87)	5.40 (0.71)	−0.52	0.65
Extraversion	4.67 (1.36)	4.15 (1.16)	0.52	0.41
Agreeableness	4.32 (0.95)	4.27 (1.22)	0.05	0.05
Emotional Stability	4.89 (1.02)	5.24 (0.94)	−0.35	0.36
Academics (N = 49 for Republicans, 45 for Democrats)
Openness	4.56 (0.78)	3.31 (0.79)	1.25	1.59
Conscientiousness	5.44 (0.68)	5.16 (0.74)	0.28	0.41
Extraversion	4.69 (1.12)	4.17 (0.97)	0.52	0.50
Agreeableness	4.56 (0.98)	3.75 (1.07)	0.81	0.79
Emotional Stability	5.14 (0.79)	4.75 (0.80)	0.39	0.49
Congressional Fellows (N = 5 for Republicans, 25 for Democrats)
Openness	4.46 (1.27)	3.40 (1.19)	1.06	0.86
Conscientiousness	5.36 (1.12)	5.70 (0.27)	−0.34	0.42
Extraversion	4.98 (1.18)	4.80 (1.44)	0.18	0.14
Agreeableness	3.97 (1.65)	3.60 (1.39)	0.37	0.24
Emotional Stability	4.48 (1.14)	4.60 (1.24)	−0.12	0.10

Previously, we alluded to our skepticism regarding data from academics. Relevant red flags appear in Table 1. A first matter to address is that there is noteworthy commonality in the insider and fellows’ ratings.⁸ For both, Democrats receive the highest average ratings on openness, extraversion, and agreeableness, with Republicans receiving the highest marks on conscientiousness and emotional stability. Furthermore, the largest effect sizes, as summarized by the Cohen’s d values, are for openness and conscientiousness. In contrast, the academics awarded Democrats the highest mean ratings on all five Big Five traits, and the Democratic advantages on openness and agreeableness were especially pronounced. The only approximate match between academics and the other two sets of raters was for extraversion.

Although we did not obtain partisan or ideological self-reports from our coders, the similarity between the data in Table 1 and that reported by Wright and Tomlinson (2018) is striking. Our academic experts produced assessments that show all signs of being skewed by, if not outright products of, partisan bias. In contrast, Senate insiders and former congressional fellows—in both instances, individuals who personally know and have worked with the senators they rated—produced evaluations that are more tempered and balanced. Moreover, these two sets of ratings are highly consistent with one another. Our skepticism regarding the academic ratings leads us to exclude those data from subsequent analyses. Thus, we proceed by merging the insider and fellows’ data sets. Moving forward, we focus on the eighty-seven senators for whom we have two or more ratings. For this group, the mean number of ratings is 3.25.⁹

Descriptive statistics for the final personality ratings are reported in Table 2. The data are similar for openness, extraversion, and agreeableness in that the means center near the scale midpoints of 4.0, with standard deviations hovering around 1.0. Higher means and lower standard deviations are observed for emotional stability and, most noticeably, conscientiousness. Substantively, this difference for emotional stability and conscientiousness presumably should be seen as good news as it implies the U.S. Senate is not populated by what observers view as erratic and irresponsible individuals.

Table 2.

Descriptive Statistics for Big Five Personality Ratings of Eighty-Seven U.S. Senators.

	M	SD	Minimum	Maximum	N
Openness to experience	4.39	0.94	2.25	6.25	87
Conscientiousness	5.12	0.81	3.13	6.75	87
Extraversion	4.51	1.20	1.75	6.58	87
Agreeableness	4.30	1.08	1.50	6.25	87
Emotional stability	4.99	0.93	2.50	6.63	87

Reliability

Because the TIPI uses only five pairs of items to represent broad trait dimensions, it is common for low inter-item correlations to be observed. Our measurement strategy partly offsets the repercussions of this due to our use of multiple coders, but reliability estimates remain modest. For example, with four ratings, Spearman-Brown reliabilities are as follows: openness to experience, .60; conscientiousness, .73; extraversion, .72; agreeableness, .71; and emotional stability, .78. The moderate reliability levels, and, indeed, our use of TIPI rather than a larger and more nuanced personality battery, means that caution should be exercised in interpretation of results in the analyses that follow.¹⁰

A second issue, one that is related to reliability, is whether evidence supports our decision to combine the insider and fellows’ data. All five correlations between the two sets of ratings are positive, although, with data from both sources for only twenty-six senators, only the correlation for extraversion (r = .65) is statistically significant. With the exception of openness (.19), the remaining correlations all are moderate in magnitude: conscientiousness, .38; agreeableness, .33; and emotional stability, .40. We would add that the fellows are themselves Senate insiders, albeit ones with fewer years on the job than our core insider group.

Validity

Evidence of three forms will be presented regarding the validity of the personality measures. First, the top and bottom three senators on each trait dimension will be reported. Although not a formal test, our expectation is that readers will be able to judge for themselves whether the categorizations are reasonable. Second, given extensive past evidence that the Big Five are correlated with ideology, including among legislators (Dietrich et al. 2012), we will regress dynamic weighted (DW)-Nominate scores, a widely used measure of legislator ideology (Poole and Rosenthal 2011), on the Big Five. Based on past research, the expected pattern includes positive relationships between ideological conservativism and both conscientiousness and emotional stability and negative relationships between conservatism and both openness to experience and agreeableness. Among these four, the strongest effects typically are observed for openness and conscientiousness. Past effects for extraversion have been varied, and, across multiple tests, centered near zero. Third, we will explore relationships between our measures and those from Ramey, Hollibaugh, and Klingler (2017) for the same senators.

The top and bottom senators on each trait are reported in Table 3. Ideally, upon perusing the list, readers will react more often with “yes, I can see that” than with “that one is a surprise.” The number in parentheses following a senator’s party and state indicates how many coders provided data for that senator. As discussed above, we expect some loss of precision when fewer coders are available. With that in mind, and recalling that most of our cases had three or four coders, the classifications in Table 3 should be seen as especially problematic if readers disagree with many of the listings for which three or more ratings are available. Our own take is that the classifications in Table 3 are encouraging. There might be room for quibbling about individual placements, but we believe there is something inherently proper about a procedure that places Ted Kennedy and Mitch McConnell at opposite sides of the extraversion scale, that codes Richard Lugar as high in conscientiousness and emotional stability, and that deems Ted Cruz to be the single most disagreeable member of the U.S. Senate.

Table 3.

High and Low Marks among Eighty-Seven U.S. Senators on Big Five Personality Traits.

	High	Low
Openness to experience	1. Mark Warner (D-VA; 2) 2. Lamar Alexander (R-TN; 4) Michael Bennet (D-CO; 2) Lindsey Graham (R-TN; 3)	1. Chris Van Hollen (D-MD; 2) 2. Jeff Sessions (R-AL; 2) 3. John Cornyn (R-TX; 2)
Conscientiousness	1. Lamar Alexander (R-TN; 4) 2. Jack Reed (D-RI; 4) 3. Jeff Bingaman (D-NM; 2) Richard Lugar (R-IN; 3)	1. Bernie Sanders (I-VT; 4) 2. Mary Landrieu (D-LA; 4) 3. Ben Nelson (D-NE; 2) Chris Van Hollen (D-MD; 2) Ron Wyden (D-OR; 2)
Extraversion	1. Ted Kennedy (D-MA; 6) 2. Cory Booker (D-NJ; 2) Sherrod Brown (D-OH; 5) Jon Tester (D-MT; 2)	1. Jeff Bingaman (D-NM; 2) 2. Mitch McConnell (R-KY; 5) 3. Herb Kohl (D-WI; 4)
Agreeableness	1. Cory Booker (D-NJ; 2) 2. Shelley Moore Capito (R-WV; 2) Joe Donnelly (D-IN; 2) Cory Gardner (R-CO; 2)	1. Ted Cruz (R-TX; 3) 2. Jim Webb (D-VA; 2) 3. John McCain (R-AZ; 6)
Emotional stability	1. Jack Reed (D-RI; 5) 2. Jeff Bingaman (D-NM; 2) Thad Cochran (R-MS; 4) Richard Lugar (R-IN; 3) Mark Pryor (D-AR; 4)	1. John McCain (R-AZ; 6) 2. Maria Cantwell (D-WA; 3) 3. Mary Landrieu (D-LA; 4)

The first person listed in the “high” column for each trait has the highest observed score among seventy-seven senators on that trait dimension, and the first person listed in the “low” column has the lowest observed score. The number in parentheses that follows each senator’s party affiliation and state represents how many coders rated that senator. Multiple entries are provided in the case of ties.

Our second validity test involves regressing DW-Nominate scores on the Big Five measures. For purposes of comparison, we also include a parallel model in which personality is represented with the Ramey, Hollibaugh, and Klingler (2017) data. Results are reported in Table 4.¹¹ Four aspects of Table 4 speak favorably with respect to validation: (1) for openness, conscientiousness, and agreeableness, all signs are in the expected direction; (2) for openness and conscientiousness, the Big Five traits most frequently linked to ideology, the relationships are statistically significant; (3) the coefficient for extraversion has the smallest absolute value among the Big Five; (4) our variables perform as well as, if not better than, those from Ramey et al. However, there is also less favorable news in Table 4 in that the effects for agreeableness and emotional stability do not reach statistical significance, and the coefficient for emotional stability has the incorrect sign.

Table 4.

Relationships between Big Five Ratings and DW-Nominate Scores for U.S. Senators.

	Insider ratings of personality	Ramey, Hollibaugh, and Klingler (2017) ratings of personality
Constant	0.11 (0.45)	0.38 (0.65)
Openness to Experience	−0.13* [−0.31] (0.06)	− 0.09 [−0.11] (0.11)
Conscientiousness	0.16* [0.33] (0.07)	0.16* [0.29] (0.08)
Extraversion	0.02 [0.05] (0.05)	−0.07 [−0.13] (0.07)
Agreeableness	−0.05 [−0.15] (0.06)	0.06 [0.05] (0.21)
Emotional Stability	−0.02 [−0.05] (0.10)	−0.15 [−0.17] (0.13)
Gender	−0.20^† [−0.20] (0.11)	−0.12 [−0.12] (0.13)
Tenure	−0.00 [−0.12] (0.00)	−0.00 [−0.08] (0.00)
R ²	.21	.16
Number of cases	86	78

Cell entries are OLS regression coefficients with standardized coefficients in brackets and standard errors in parentheses. The Ramey et al. model is restricted to include the same cases as those in the insider model. DW-Nominate = Dynamic Weighted Nominal Three-step Estimation; OLS = ordinary least squares.

†

p < .10. *p < .05.

As a final test, we calculated the correlations between our measures and those from Ramey, Hollibaugh, and Klingler (2017).¹² The result for openness is near zero (r = −.01), but the other four are all positive, and two are statistically significant: conscientiousness (r = .14), extraversion (r = .35, p < .01), agreeableness (r = .21, p < .10), and emotional stability (r = .15). Our openness measure produces the expected effect on ideological liberalism whereas the Ramey et al. measure does not, suggesting the weak correlation between the two openness ratings should not be taken as an indictment of our measure. The other four correlations strike us as reasonable. First, with very different measurement procedures, moderate rather than high correlations should be expected. Second, extraversion and agreeableness are arguably the Big Five traits most publicly on display, and thus the somewhat sharper correlations for those seem sensible.

Our own assessment is that the evidence we have reviewed collectively establishes at least a plausible case with respect to validity. That said, with a limited number of observations, we acknowledge the case leaves room for additional corroboration. As a further step in exploring the properties of our data, we turn now to examining whether variation in personality as represented here corresponds in sensible manners with patterns of legislative behavior.

Assessing the Utility of Senate Personality Measures

For efforts to measure the personalities of U.S. senators to be worthwhile, personality must be related to central aspects of Senate behavior. To test whether such relationships exist, we consider ten variables. Five of these—amendments proposed, agenda size, agenda variation, bill introductions, and introduction of bills that became law—speak to traditional legislative behavior. Two, holding a party leadership position and the Lugar Center’s Lifetime Bipartisan Index, concern partisan support and bipartisan activities. The final three—running for president, missing votes, and campaign contributions received—pertain to extra-institutional behavior.

Our decision to explore the possible effects of personality on this set of dependent variables is influenced by two sources. Bernhard and Sulkin (2018) develop a strong case for the significance of these variables, particularly in research regarding legislators’ individualistic motivations. Ramey, Hollibaugh, and Klingler (2017) also consider many of these dependent variables in their research regarding personality (as measured with a speech-based approach) on legislative behavior. Together, these two works provide a strong foundation for the present tests, a foundation anchored in research on both legislative politics and political psychology.

Consistent with our focus on personality, Bernhard and Sulkin (2018) make the case for a multifaceted approach to understanding legislative motivations. They argue legislative style consists of “everyday” (19) behaviors directly undertaken or directed by the member. This is relevant for our purposes. The components of legislative style originate from the legislators themselves and their understanding and response to the situational context. Bernhard and Sulkin focus on eight different activities—attention to the district, lawmaking, public visibility, fundraising, party loyalty in contributions, party loyalty in voting, bipartisanship, and policy focus. Most of our dependent variables either directly replicate or closely mirror Bernhard and Sulkin’s legislative style components. Because Bernhard and Sulkin’s work is applied to the House, we added select variables of greater relevance for senators (e.g., presidential bids).

Our dependent variables also overlap with those examined by Ramey, Hollibaugh, and Klingler (2017), an analysis that, like Bernhard and Sulkin’s, focuses primarily on the House. As examples, Ramey et al. include dependent variables similar to our measures of bill introductions, bipartisanship, and contributions to campaign committees.

Our expectations draw on extensive research regarding effects of the Big Five within and beyond the political realm (see Mondak 2010, chap. 2, for a review). For legislative behavior, we expect conscientiousness and agreeableness to be negatively related and openness and emotional stability positively related to legislative productivity. In the extreme, conscientiousness leads to dogmatic, inflexible behavior. Hence, highly conscientious senators may dig in their heels rather than compromise. Individuals high in agreeableness prefer to avoid contention, and thus the most agreeable may have a distaste for legislative haggling. Senators who are open to experience and emotionally stable should remain unflappable during the more trying moments of the legislative process—those high in openness because they revel in intellectual give-and-take and those high in emotional stability because they retain a cool composure in stressful situations.

We expect partisan support activities to peak among extraverts and senators high in emotional stability and to be less common among the most agreeable. Extraverts should be drawn to the team aspects of partisan support whereas emotional stability should facilitate success at the cat-herding tasks of party leadership. The most agreeable senators will be drawn toward developing as many amicable relationships as possible, making them hesitant to further the partisan divide. As to bipartisanship, we expect positive effects for openness and agreeableness and a negative effect for conscientiousness. Senators open to new experiences should be willing to cross the aisle, and those high in agreeableness also should be amenable to working with the opposition. Senators with high marks in conscientiousness again are expected to resist the compromise inherent in bipartisan collaboration.

Finally, we expect presidential bids, missed votes, and contributions received all to increase with openness and extraversion, traits more associated with seeking new adventures than fulfilling current duties. The opposite pattern is expected for conscientiousness and emotional stability, and we expect senators high in agreeableness to have a distaste for fundraising. Analyses unfold in two steps. We first describe the data sources and operational measures for the ten dependent variables. We then report and discuss coefficients for models in which these variables are regressed on our Big Five measures.¹³

Operationalizing Senate Behavior

There are five dependent variables pertaining to legislative productivity and success. The first, average annual bill introductions, draws on annual data for the years 2007–2015, with those data compiled from Congress.gov. The dependent variable is coded as the average number of introductions for all years in this period in which the senator was in office (N = 86, M = 97.77, SD = 50.73). The second dependent variable is the senator’s average number of bill introductions that became law (N = 87, M = 1.15, SD = 1.06). The third variable, average annual amendments offered, is available for the years 2001 to 2015 (N = 80, M = 43.14, SD = 28.61). The final two dependent variables related to legislative activity provide alternate measures of a senator’s legislative focus. Legislative agenda size represents each senator’s total number of agenda entries across thirty-two policy categories as defined by Congress.gov (N = 87, M = 251.53, SD = 2228.76) whereas legislative agenda variation, operationalized as the standard deviation across these same policy categories, reflects the extent to which the senator is a legislative generalist or specialist (N = 87, M = 11.82, SD = 11.76).

To measure partisan activity, we use Party leadership position, an ordinal variable coded so as to represent the highest leadership position a senator has held, with values ranging from 0 to 3 (N = 87, M = 0.92, SD = 0.97).¹⁴ Bipartisanship, which uses the Lugar Center’s Lifetime Measure, has observed values between −1.42 and 1.81 (N = 87, M = −0.05, SD = 0.63). The final three dependent variables capture behaviors that relate, directly or indirectly, to actions outside of the legislative process. The first, presidential bid, is a dichotomous measure of whether the senator has ever run for president (N = 87, twenty-one senators are coded as having run for president). Missed votes is the average number of votes missed by the senator in the period 1981–2018 as noted by GovTrack.us (N = 61, M = 3.79, SD = 6.11). Finally, contributions to campaign committee is the natural log of the total amount of money received by the senator since 1989 as recorded by opensecrets.org (N = 87, M = 17.19, SD = 0.72).

Results

Coefficient estimates for ten models—meaning a total of fifty personality coefficients—are reported in Table 5. Twelve of these reach at least the p < .10 level of statistical significance.¹⁵ Ten of these twelve are for relationships in line with our general expectations whereas two—the negative effects of extraversion and emotional stability on bipartisanship—were not expected.

Table 5.

Relationships between the Big Five Personality Traits and Ten Aspects of U.S. Senate Behavior.

	Openness to experience	Conscientiousness	Extraversion	Agreeableness	Emotional stability
Average annual bill introductions (N = 85)	1.22 [0.02] (9.27)	−17.09 [−0.26] (11.23)	5.68 [0.14] (7.43)	−19.57* [−0.41] (9.23)	19.54 [0.37] (14.82)
Average annual introduction of bills that became law (N = 86)	0.00 [0.00] (0.18)	−0.29 [−0.22] (0.22)	0.08 [0.10] (0.15)	0.19 [0.19] (0.18)	0.13 [0.12] (0.29)
Average annual amendments offered (N = 78)	−6.13 [−0.21] (4.66)	0.68 [0.02] (5.64)	3.04 [0.15] (3.74)	−9.10^† [−0.39] (5.19)	13.44^† [0.53] (7.97)
Legislative agenda size (N = 86)	−29.40 [−0.12] (39.17)	−30.47 [−0.11] (47.26)	19.34 [0.12] (31.63)	−16.31 [−0.08] (39.46)	64.85 [0.30] (62.36)
Legislative agenda variation (N = 86)	−0.78 [−0.06] (1.99)	−2.25 [−0.16] (2.40)	−0.26 [−0.03] (1.61)	−0.56 [−0.05] (2.00)	3.38 [0.30] (3.17)
Party leadership position (N = 86)	−0.05 (0.34)	−0.07 (0.42)	0.82** (0.29)	−0.79* (0.36)	1.67** (0.59)
Bipartisanship (N = 86)	0.17^† [0.27] (0.09)	0.02 [0.02] (0.11)	−0.25** [−0.54] (0.07)	0.28** [0.51] (0.09)	−0.25^† [−0.42] (0.14)
Presidential bid (N = 86)	0.55 (0.46)	−0.31 (0.55)	0.56 (0.41)	−0.10 (0.51)	−0.01 (0.79)
Missed votes (N = 60)	2.29^† [0.31] (1.30)	−1.59 [−0.17] (1.55)	−0.17 [−0.03] (1.22)	−1.62 [−0.26] (1.33)	−0.93 [−0.13] (1.97)
Contributions to campaign committee (N = 86)	0.13 [0.15] (0.12)	−0.16 [−0.16] (0.15)	0.11 [0.18] (0.10)	−0.25^† [−0.34] (0.12)	0.06 [0.07] (0.20)

Cell entries are OLS regression coefficients, with the exception of Party leadership (ordinal logistic regression) and Presidential bid (binomial logistic regression). There is one model per row. All models also include three control variables: gender, years in office, and party. Standardized coefficients are in brackets (OLS only) and standard errors are in parentheses. OLS = ordinary least squares.

†

p < .10. *p < .05. **p < .01.

Four of the ten models produced zero significant personality effects. Three of these are among the five models related to legislative activity. As expected, agreeableness is negatively related to the two dependent variables that speak most directly to actual activity—the introduction of bills and amendments. Seven of the twelve personality effects are found in just two models. Extraversion, agreeableness, and emotional stability produce the expected effects on holding a party leadership position, and four of the Big Five traits—all but conscientiousness—yield effects on bipartisanship.

In research on personality and mass political attitudes and behavior, frequent links between openness and conscientiousness and attitudinal measures are seen as are many effects for openness and extraversion on behavioral dependent variables (see, for example, the various models in Mondak 2010). With that in mind, the relatively strong showings for agreeableness and emotional stability in Table 5 are noteworthy, with those two traits accounting for eight of the twelve significant personality effects. For all of the calls for greater civility in Congress, current results suggest that some level of disagreeableness is conducive to legislative productivity. Bill introductions, amendments offered, taking on party leadership positions, and contributions to campaign committees all decline as a function of agreeableness. Only for bipartisanship was a strong positive effect of agreeableness observed. Conversely, emotional stability appears to fuel activity in the Senate, albeit activity that likely fosters partisan competition. Specifically, senators who are more calm and relaxed introduce more amendments (often an obstructionist tactic), take on party leadership positions, and are less inclined toward bipartisanship.

The broader implications of present findings are that attention to the Big Five can improve our understanding of the underpinnings of elite political behavior and that expert reports provide a useful means to obtain the requisite personality measures. With these points established, the foundation hopefully is set for further research on the psychological bases of legislative behavior.

Conclusion

With only one hundred members in office at a time and behavioral norms that privilege the power of the individual, the U.S. Senate is an institution in which differences in members’ core psychological traits may be consequential. To facilitate expanded exploration of this possibility, we have sought to develop a procedure for human coding of the personality traits of U.S. senators and to implement and test the utility of that procedure. Of 154 individuals who served in the Senate since 2009, we were able to obtain two or more personality ratings for eighty-seven, or 56 percent, and a single rating for forty-two of the remaining sixty-seven. The resulting personality variables performed satisfactorily in tests of both reliability and validity. Furthermore, variation in personality was shown to be related to multiple facets of legislative behavior. Collectively, this study’s analyses demonstrate both the utility of drawing on experts as data sources for representations of elite personality and the value of the subsequent measures for improving our understanding of elite political behavior.

Although this exercise has, as a whole, produced promising results, we would be remiss were we to neglect mention of some concerns. First, the approach we used to acquire data from Senate insiders was time-consuming, and it yielded fewer personality reports—in terms of both the number of senators for whom data are available and the average number of insider responses per senator—than we would have preferred. We encourage other analysts to replicate and extend our procedures, especially with application to officials in contexts such as the U.S. House, federal courts, and legislatures in other nations, as we see reason to expect that high-quality data will be produced. However, we must caution that the identification and solicitation of expert insiders is not a path to the rapid acquisition of large quantities of data.

A second concern pertains to the data we collected but ultimately rejected. Consistent with the warning issued by Wright and Tomlinson (2018), our exploration of the properties of data provided by academics raised serious concerns about data quality. One matter is that we are uncertain of the information base that fueled the academics’ ratings, but there are obvious reasons to suspect most were not products of lengthy in-person interactions between the raters and the senators. That situation raises the possibility of tautologies: that the raters formed inferences about personality from observations of the sorts of behaviors personality is thought to explain. Even more troubling is that the data we gathered showed clear signs of liberal, or pro-Democratic, bias. Our insiders and congressional fellows rated Republicans higher on average than Democrats on conscientiousness and emotional stability, but the academic coders scored Democrats higher on these, and, indeed, all, Big Five traits. Just as in Wright and Tomlinson (2018), the academics’ ratings appeared consonant with the other coders only on extraversion. Our findings add to the growing doubt regarding the utility of academics as data sources in circumstances in which data may be skewed by partisan or ideological bias.

The evidence seems clear that personality ratings provided by insiders are preferable to ratings from academics. But would personality self-reports from public officials such as members of the U.S. House and Senate be better still? Progress in obtaining self-reports from national officeholders has been achieved in other nations, and efforts to acquire similar data in the United States should be encouraged. That said, we see reasons to doubt such measures would be superior to reports provided by expert insiders. First, as a general matter, research in psychology has demonstrated that other-reports often outperform self-reports (e.g., Balsis, Cooper, and Oltmanns 2015). Second, particularly for busy national legislators, it would be difficult to ensure that legislators’ erstwhile self-reports truly were provided by themselves rather than their staff. Third, legislators face an incentive to cast themselves in the best light possible, a reality that raises serious concerns about validity, especially for data from career-minded national public officials.

In this study, we have attempted to produce a warts-and-all recounting of our effort to generate human-coded measures of the personality traits of U.S. senators. If, as we believe, there is utility in obtaining human-coded personality data on political elites, our experiences and analyses indicate the use of expert insiders as coders is the most promising option. Obtaining ratings from insiders is not easy, but the resulting data avoid the biases that plague assessments from academics and that likely would be seen in legislators’ self-reports.

Apart from measurement, the substantive impact of personality also warrants emphasis. Ramey, Hollibaugh, and Klingler (2017) demonstrate links between the Big Five and a wide array of measures of legislative behavior. With focus on the U.S. Senate, we have reported ten similar tests here. Other research directions emerging in the literature address personality and mass-elite linkages, including the possibility that personality functions as a basis for connections between public officials and their constituents (e.g., Caprara et al. 2003),¹⁶ and more such work is to be encouraged. What unites the studies on legislative behavior and mass-elite links is that both research agendas acknowledge that psychological differences among political elites contribute to the nature and quality of democratic governance, a bottom line we, too, endorse.

Supplemental Material

Online_Appendix – Supplemental material for Personality on the Hill: Expert Evaluations of U.S. Senators’ Psychological Traits

Supplemental material, Online_Appendix for Personality on the Hill: Expert Evaluations of U.S. Senators’ Psychological Traits by Matthew G. Rice, Megan L. Remmel and Jeffery J. Mondak in Political Research Quarterly

Footnotes

Acknowledgements

This project draws on research from the first author’s senior honors thesis at the University of Illinois, and much of the work was completed while the third author was a senior research fellow at the Center for the Study of Democratic Institutions at Vanderbilt University. We thank Tracy Sulkin for her feedback and support, three anonymous reviewers for their insightful comments, and the many Senate insiders, former American Political Science Association (APSA) Congressional Fellows, and political scientists throughout the United States who provided data for this project.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jeffery J. Mondak

Notes

Supplemental Material

Replication code and data available at: . Supplemental materials for this article are available with the manuscript on the Political Research Quarterly (PRQ) website.

References

Bakker

B. N.

Lelkes

2018. “Selling Ourselves Short? How Abbreviated Measures of Personality Change the Way We Think about Personality and Politics.” Journal of Politics 80:1311–25.

Balsis

Cooper

L. D.

Oltmanns

T. F.

2015. “Are Informant Reports of Personality More Internally Consistent than Self Reports of Personality?” Assessment 22:399–404.

Bernhard

Sulkin

2018. Legislative Style. Chicago: University of Chicago Press.

Brownson

A. L.

2008. Congressional Staff Directory. Washington, DC: Congressional Quarterly Press.

Caprara

G. V.

Barbaranelli

Consiglio

Picconi

Zimbardo

P. G.

2003. “Personalities of Politicians and Voters: Unique and Synergistic Relationships.” Journal of Personality and Social Psychology 84:849–56.

Carney

D. R.

Jost

J. T.

Gosling

S. D.

Potter

2008. “The Secret Lives of Liberals and Conservatives: Personality Profiles, Interaction Styles, and the Things They Leave Behind.” Political Psychology 29:807–40.

Charney

2008. “Genes and Ideologies.” Perspectives on Politics 6:299–319.

Charney

2015. “Liberal Bias and the Five-Factor Model.” Behavioral and Brain Sciences 38:22–23.

Connolly

J. J.

Kavanagh

E. J.

Viswesvaran

2007. “The Convergent Validity between Self and Observer Ratings of Personality: A Meta-analytic Review.” International Journal of Selection and Assessment 15:110–17.

10.

Costantini

Craik

K. H.

1980. “Personality and Politicians: California Party Leaders, 1960–76.” Journal of Personality and Social Psychology 38:641–61.

11.

Credé

Harms

Niehorster

Gaye-Valentine

2012. “An Evaluation of the Consequences of Using Short Measures of the Big Five Personality Traits.” Journal of Personality and Social Psychology 102:874–88.

12.

Dietrich

B. J.

Lasley

Mondak

J. J.

Remmel

M. L.

Turner

2012. “Personality and Legislative Politics: The Big Five Trait Dimensions among U.S. State Legislators.” Political Psychology 33:195–210.

13.

Drew

1978. Senator. New York: Simon & Schuster.

14.

Funder

D. C.

Kolar

D. C.

Blackman

M. C.

1995. “Agreement among Judges of Personality: Interpersonal Relations, Similarity, and Acquaintanceship.” Journal of Personality and Social Psychology 69:656–72.

15.

Gosling

S. D.

Rentfrow

P. J.

Swann

W. B.

Jr.

2003. “A Very Brief Measure of the Big-Five Personality Domains.” Journal of Research in Personality 37:504–28.

16.

Hall

M. E. K.

2018. What Justices Want: Goals and Personality on the U.S. Supreme Court. New York: Cambridge University Press.

17.

Hanania

2017. “The Personalities of Politicians: A Big Five Survey of American Legislators.” Personality and Individual Differences 108:164–67.

18.

Hennessy

1959. “Politicals and Apoliticals: Some Measurements of Personality Traits.” Midwest Journal of Political Science 3:336–55.

19.

Joly

Soroka

Loewen

2019. “Nice Guys Finish Last: Personality and Political Success.” Acta Politica 24:667–83.

20.

Maestas

C. D.

Buttice

M. K.

Stone

W. J.

2014. “Extracting Wisdom from Experts and Small Crowds: Strategies for Improving Informant-Based Measures of Political Concepts.” Political Analysis 22:354–73.

21.

Maisel

L. S.

Stone

W. J.

1997. “Determinants of Candidate Emergence in U.S. House Elections: An Exploratory Study.” Legislative Studies Quarterly 22:79–96.

22.

McConaughy

J. B.

1950. “Certain Personality Factors of State Legislators in South Carolina.” American Political Science Review 44:897–903.

23.

Mondak

J. J.

2010. Personality and the Foundations of Political Behavior. New York: Cambridge University Press.

24.

Mondak

J. J.

Canache

2014. “Personality and Political Culture in the American States.” Political Research Quarterly 67: 26–41.

25.

Mondak

J. J.

Hibbing

M. V.

Canache

Seligson

M. A.

Anderson

M. R.

2010. “Personality and Civic Engagement: An Integrative Framework for the Study of Trait Effects on Political Behavior.” American Political Science Review 104:85–110.

26.

Nai

Maier

2018. “Perceived Personality and Campaign Style of Hillary Clinton and Donald Trump.” Personality and Individual Differences 121:80–83.

27.

Nai

Maier

2019. “Can Anyone be Objective about Donald Trump? Assessing the Personality of Political Figures.” Journal of Elections, Public Opinion, and Parties. Published electronically June 24. doi:10.1080/17457289.2019.1632318.

28.

Nørgaard

A. S.

Klemmensen

2019. “The Personalities of Danish MPs: Trait- and Aspect-Level Differences.” Journal of Personality 87:26775.

29.

Poole

K. T.

Rosenthal

2011. Ideology and Congress. New Brunswick: Transaction Publishers.

30.

Ramey

Hollibaugh

G. E.

Jr. Klingler

J. D.

2017. More Than a Feeling: Personality, Polarization, and the Transformation of the U.S. Congress. Chicago: University of Chicago Press.

31.

Rentfrow

P. J.

Gosling

S. D.

Potter

2008. “A Theory of the Emergence, Persistence, and Expression of Geographic Variation in Psychological Characteristics.” Perspectives on Psychological Science 3:339–69.

32.

Rubenzer

S. J.

Faschingbauer

T. R.

Ones

D. S.

2000. “Assessing the U.S. Presidents Using the Revised NEO Personality Inventory.” Assessment 7:403–20.

33.

Rubenzer

S. J.

Faschingbauer

T. R.

2004. Personality, Character, and Leadership in the White House: Psychologists Assess the Presidents. Washington, DC: Potomac Books.

34.

Scott

Medeiros

2020. “Personality and Political Careers: What Personality Types Are Likely to Run for Office and Get Elected?” Personality and Individual Differences 152: 109600.

35.

Stone

W. F.

Baril

G. L.

1979. “Self-Other Orientation and Legislative Behavior.” Journal of Personality 47:162–17.

36.

Timmons

2015. Congressional Yellow Book. New York: Leadership Directories.

37.

Visser

B. A.

Book

A. S.

Volk

A. A.

2017. “Is Hillary Dishonest and Donald Narcissistic? A HEXACO Analysis of the Presidential Candidates’ Public Personas.” Personality and Individual Differences 106:281–86.

38.

Watson

1989. “Strangers’ Rating of the Five Robust Personality Factors: Evidence of a Surprising Convergence with Self-Reports.” Journal of Personality and Social Psychology 52:120–28.

39.

Wright

J. D.

2019. “Ideological Bias in Expert Personality Assessment of Political Figures: A Commentary.” https://doi.org/10.31234/osf.io/8mn6w

40.

Wright

J. D.

Tomlinson

M. F.

2018. “Personality Profiles of Hillary Clinton and Donald Trump: Fooled by Your Own Politics.” Personality and Individual Differences 128:21–24.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.41 MB