Abstract
Many labor unions assess support among prospective members to guide decision making during organizing campaigns, and to predict voting in representation elections. However, research on the actual practice of how unions make assessments is limited. We fill this void through a study that combined quantitative and qualitative analysis of the assessment activities. The quantitative portion involved a survey of eligible voters in the 2010 flight attendant representation election at Delta Air Lines. The qualitative portion involved in-depth interviews with staff involved in that campaign and organizing directors or key organizing staff in nine of the largest labor unions in the United States. We focus on the factors that influence the accuracy of assessment predictions, describe practices currently being used to predict votes in these campaigns, and discuss future research needs.
Keywords
Introduction
In an era of declining union density in the United States and growing anticollective bargaining fervor among private employers and state governments, labor unions need innovative and effective strategies for organizing workers. One such tactic is the use of benchmarks and assessments to predict outcomes during organizing campaigns and representation elections. Bronfenbrenner and Hickey (2003, 2004) describe benchmarks and assessments as written evaluations of employee support for the union at different stages of a campaign that are used by the union to set thresholds for campaign activities (such as filing a petition, or moving ahead with the campaign). 1 Assessments are typically done by staff, members, and volunteers of the organizing union.
It is unclear as to exactly when unions began the practice of formally and systematically assessing worker support during a campaign and using those data to inform subsequent organizing activities, although it appears to date back to at least the mid-1980s (Allison Porter, personal communication, August 15, 2015). The George Meany Center produced a guide on organizing in the early 1990s, which includes a brief section on assessment, including how to track the data collected (Diamond 1992). Bronfenbrenner and Hickey (2004) show that “benchmarks and assessments” can make a real difference in the outcome of representation elections. Analyzing data from 412 National Labor Relations Board (NLRB) certifications election campaigns during 1998 and 1999, they found that the use of benchmarks and assessments statistically increased the odds of a union winning a representation election by 162 percent, even when controlling for company and industry characteristics, bargaining unit demographics, and employer opposition. Of ten different organizing tactics and strategies Bronfenbrenner and Hickey (2004) analyzed, benchmarks and assessments was one of only three to be individually statistically related to win rates, highlighting the importance of this tactic for organizing outcomes. Indeed, Bronfenbrenner and Hickey (2003, 21) listed “benchmarks and assessments,” together with adequate and appropriate staff and financial resources, and an active representative committee, as the three core “building blocks of any organizing campaign upon which all the other comprehensive tactics depend.” At the same time, they report that the use of benchmarks and assessments to decide when to file for a representation petition occurred in only 24 percent of elections.
Aside from Bronfenbrenner and Hickey (2003, 2004), there is little empirical research on the practice of assessment among labor unions. Two case studies that we located (Hoerr 1997; Simmons 1994) provide some information on the use of assessment by unions in the 1980s. Simmons (1994, 69-71), for instance, discusses the use of the Blitz model of organizing by 1199 in Hartford, Connecticut in the 1980s. Assessment in this approach included organizers evaluating the behavior of worker leaders in doing things like getting others to attend initial organizing committee meetings. Hoerr (1997, 157) provides detail on the use of assessment by organizers in the drive to organize Harvard clerical and technical workers, including the initial use of a 5-point scale to evaluate the degree of support from each potential member. Sipp (2016) compares the processes of political canvassing and union organizing assessment primarily through the lens of the technology used to track those processes but does not discuss how assessments are conducted or used. She does argue that, unlike in a political election, “In a union election, every single voter must be targeted for conversation, assessment, and repeated follow up.”
Nonetheless, the literature on the use of assessment in organizing is surprisingly light. We do not know which unions use assessments, who in the union conducts the assessments, how assessments are determined, how often they are done, how they are tracked, or whether any of these or other details make a difference in the effectiveness of assessment. For example, while the organizing literature (see, for instance, Bronfenbrenner and Hickey 2003 and Hoerr 1997) suggests that unions who hire women or people of color as lead organizers have more success and that unions have been diversifying their organizing staff at all levels, we do not know to what extent assessments would improve with better demographic matches between organizers and the workers being assessed. Bronfenbrenner’s (1997) ground-breaking work on union organizing strategies does not mention the use of assessments although it does look at the numbers of worker meetings held and housecalls made, both venues where assessments might take place. Furthermore, while the political science literature includes studies on the role of canvassing in voter turnout (but not the substantive vote—see, for example, Gerber and Green 2000; Green, Gerber, and Nickerson 2003) and polling accuracy (see, for example, Healy and Malhotra 2014; Hillygus 2011; Hillygus and Shields 2008), which have some relevance, it does not appear to include studies on assessing voters in the context of political campaigns, arguably an activity very similar to assessment during organizing campaigns. 2
Given the paucity of research on this important phenomenon, we take an inductive or theory-building approach in this paper. We use mixed-methods, both quantitative and qualitative, to generate insights and a model that can be tested in future research. For our quantitative analysis, we take advantage of a unique opportunity offered by the match of worker survey data with union assessment data in the course of a single organizing campaign. We analyze these matched data, which were collected following the 2010 representation election at Delta Air Lines during which the Association of Flight Attendants–Communication Workers of America (AFA-CWA) unsuccessfully tried to organize the entire class of flight attendants following Delta’s merger with Northwest Airlines in 2008. AFA’s assessments led at least some within the union to predict with confidence that they would win the election; instead, they lost. 3 This survey was conducted primarily with the intent of explaining why flight attendants voted the way they did (or actually, the way they reported they did).
Given the perils of building theory off a single case, we supplement our quantitative analysis with a qualitative approach, conducting one-on-one interviews with fourteen union organizing directors about their unions’ use of assessment and details of their assessment system. This enabled us to both expand our understanding of the assessment process and to directly compare the AFA’s assessment practices to those of other unions—all in service to better theory-building.
Also of note is that our research study, which is grounded in the tradition of scientific inquiry and reflects our roles as academicians, has at its core a motivation to enhance both theory and future research and to provide information of use to practitioners. In a way, then, it reflects “Mode 2” management research (Gibbons et al. 1994; MacLean, MacIntosh, and Grant 2002), which has been characterized as an approach to knowledge production that is practitioner-oriented (Mitev and Venters 2009) and “carried out in the context of application” (MacLean, MacIntosh, and Grant 2002, 191).
The 2010 Certification Election at Delta Air Lines
Background
In 2008, Atlanta-based Delta Air Lines acquired a financially struggling Northwest Airlines to create what was, at the time, the world’s largest passenger airline (Eaton et al. 2014). Prior to the merger of the two groups, Delta was a largely nonunion airline (only its pilots and dispatchers were represented by labor unions), while Northwest was heavily unionized. After the merger, the AFA-CWA, which represented the over seven thousand pre-merger Northwest Airlines flight attendants, sought to represent the combined class of flight attendants at the newly formed Delta Air Lines, which included nearly thirteen thousand pre-merger Delta flight attendants who were not unionized.
Between 2008 and 2010, the AFA-CWA organized a campaign to represent the post-merger flight attendant workforce. Delta, long known for its paternal relationship with its employees (Eaton et al. 2014; Kaufman 2013), aggressively waged opposition to the union’s efforts by, among other things, highlighting the longstanding direct relationship it had been able to maintain with its flight attendants, and by claiming that representation would harm this relationship and the “Delta family” ideal. In a March 2008 company press release, Delta senior vice-president of In-Flight Service Joanne Smith noted of the election: “Delta flight attendants will make one of the most important decisions of their careers over the coming months as they choose between a direct relationship with Delta’s management team or the cost and risk of a third-party representative” (Delta Air Lines 2008). For its part, the AFA-CWA ran a multipronged campaign, which included the use of assessments and benchmarks to predict the outcome of the election. Going into the election, many within the AFA-CWA, based on data from their assessments, believed they had enough votes to win. They lost.
AFA-CWA’s Approach to Assessments during the 2010 Delta Election
During the organizing campaign, the AFA-CWA made assessments for almost nineteen thousand flight attendants, the vast majority of whom had worked for Delta or Northwest pre-merger. Their assessments were focused on predicting the voting behavior of employees. Organizers categorized flight attendants into one of four categories: (1) a “yes” vote for the AFA-CWA, (2) undecided, (3) a “no” vote for the AFA-CWA, and (4) a strong “no” vote for the AFA-CWA. The latter category included people expected to work against the union, and they were not contacted at all by organizers. Of all flight attendants assessed, 64 percent were assessed in the first category, 10 percent in the second category, 21 percent in the third category, and 5 percent in the fourth category.
Notably, while the AFA-CWA evaluated support for the union, their predictions of votes were a bit different than what is described in Bronfenbrenner and Hickey (2003, 2004). Bronfenbrenner and Hickey primarily refer to the use of assessments as a tool for deciding whether or not to file a petition and/or to move to subsequent stages of an organizing campaign, as opposed to predicting voting behavior and organizing a Get out the Vote (GOTV) around predicted “yes” voters in similar fashion to a political election campaign. In the qualitative section of this paper, we will further explore what unions actually do when they conduct benchmarking and assessments, and what processes and outcomes they tend to focus on.
Overall, there were few paid union staff assigned to the Delta flight attendant campaign: three full-time organizers, and three to four part-timers. A majority of the assessments were performed by volunteer worker organizers who had little formal training. In fact, organizing staff from CWA report that the union generally relies on members to do assessments, feeling it is important to building the union. Those conducting assessments would fax, phone, or email their reports daily into a central office, where a database would constantly be updated.
Understanding the Election Results: Where Did AFA’s Assessments Go Wrong?
On November 3, 2010, Delta announced that it had received the election results from the National Mediation Board. Of 19,887 eligible voters, 18,760 (94%) had voted—44 percent in favor of AFA-CWA representation, and 53 percent against. The union lost by 1,313 votes. AFA-CWA officials and supporters were disappointed, and many were stunned, in part because their assessments predicting voter behavior made them confident that they had a strong chance to win. 4 CWA engaged us to survey the workforce to help them understand what had happened. Essentially, we focused on variables that are central to traditional models that attempt to predict union voting behavior, such as perceptions of union instrumentality, job satisfaction, and attitudes toward the employer (e.g., Barling, Kelloway, and Bremermann 1991; Deshpande and Fiorito 1989; Eaton et al. 2014; Friedman, Abraham, and Thomas 2006), along with some additional questions about the conduct of the vote itself that the union was interested in.
Between March and April 2011, we emailed invitations to Delta flight attendants to complete an Internet-based questionnaire that asked about their attitudes, job experiences, how they voted, and demographics. The sample included pre-merger Delta and pre-merger Northwest flight attendants, as well as a very small number of flight attendants hired post-merger. Invitations to participate in our survey were emailed to 13,002 flight attendants (all those for whom we had email addresses), and 1,498 valid responses were received, for a response rate of 12 percent. 5 Actual sample size for our multiple analyses vary due to some instances of missing data; we specify the actual n in the results tables. In addition to these primary data, the union provided us with their assessments database. This included their 1 through 4 categorizations for individual flight attendants, and we combined data from the two sources by matching email addresses. 6
One question that can be asked is whether the AFA-CWA simply incorrectly assessed too many individuals, and if so, why? Of all our survey respondents, 781 flight attendants were correctly assessed as “yes” voters, 172 were correctly assessed as “no” voters, four were incorrectly assessed as likely “no” voters (but actually voted “yes” for the AFA), and 160 were incorrectly assessed as likely “yes” voters (but actually voted “no” against the AFA). Voters in this last category are the focus of our analyses in this paper; we subsequently refer to this group of voters as “misassessed.” The matching of the survey and assessment data allow us to see whether there are any systematic differences between “no” voters who were correctly and incorrectly assessed (or the 172 correctly-assessed “no” voters, and the 160 “misassessed” voters). Individuals in both of these groups were the same in that they both voted against AFA-CWA representation (i.e., they were truly “no” voters). But something about individuals in the misassessed category led organizers to erroneously predict that they were going to vote “yes” for the union.
Means comparisons
To begin to understand this, we compared the means of correctly assessed “no” voters and misassessed voters, as well as correctly-assessed “yes” voters, on a range of demographic, attitudinal and emotional, and voting variables. Given that this was an exploratory analysis, not a theory-testing exercise, we included in our analyses almost any variable from the survey that could logically be related to the accuracy of an assessment. 7 The demographic variables included age, gender, race, ethnicity, education level, years as a flight attendant, whether or not the respondent had previously been a union member, 8 and whether or not the respondent had a family member who was in a union. The attitudinal and emotion variables included job satisfaction, organizational commitment, turnover intentions, a composite variable reflecting perceptions and feelings about AFA-CWA and unions in general, and emotions toward Delta Air Lines. Items used for each of these variables are listed in Appendix A. The voting variables included the timing of respondents’ decision about how they would vote (early, midway, or near the end of the organizing campaign), and whether or not respondents voted using their “gut feelings.” Whether or not the respondent was a pre-merger Delta Air Lines flight attendant serves as a control variable.
Table 1 presents the correlation matrix for study variables, and Table 2 presents the results of these means comparisons. When specifically comparing misassessed voters to correctly assessed “no” voters across demographic characteristics, a larger proportion of misassessed voters were male, nonwhite, and had previously been a union member. The means comparisons between these two groups were statistically significant. Also, although correctly assessed “no” voters were overwhelmingly more likely to come from pre-merger Delta (92%), a much lower percentage of the misassessed had come from Delta (65%), suggesting that assessors made errors in assuming some former Northwest Airlines flight attendants would support the union that had represented them there in the past.
Correlation Matrix.
Note. Pairwise deletion of missing data. AFA = Association of Flight Attendants.
p ⩽ .05. **p ⩽ .01, two-tailed tests.
Means Comparisons between Correctly Assessed “Yes” Voters, Correctly Assessed “No” Voters, and Misassessed Voters.
Note. Comparison of means were done for each pair of voter types, and the “mean difference level of significance” columns reflect t-tests between two groups. The first of these columns reflects whether significant differences exist between correctly assessed yes voters, and misassessed voters. The middle column compares means between misassessed voters and correctly assessed no voters only. And the third column compares means between correctly assessed yes voters and correctly assessed no voters. Education is a categorical variable where 1 = high school, 2 = some college, 3 = college degree, 4 = master’s degree, and 5 = doctorate degree. All attitudinal/emotional variables were measured on a 5-point Likert-type scale where 1 indicates a lower degree and 5 indicates a higher degree of the item (e.g., greater satisfaction, higher perceived instrumentality, or more positive attitudes or emotions).
p ⩽ .10. *p ⩽ .05. **p ⩽ .01. ***p ⩽ .001.
Statistically significant differences were found between misassessed and correctly assessed “no” voters for all attitudinal and emotional characteristics we analyzed. Misassessed voters were more negative in their attitudes toward the employer than were correctly assessed “no” voters; they had lower levels of job satisfaction and organizational commitment (to Delta), higher turnover intention, and less positive emotions toward Delta. They also were more positive toward unions than the average correctly assessed “no” voter. They scored higher on perceived efficacy of the AFA-CWA, perceived instrumentality of the current Delta pilots union, and perceived instrumentality of any potential flight attendants union at Delta. They also had more positive attitudes toward labor unions in general, and more positive emotions toward the AFA-CWA.
Interestingly, mean responses for misassessed voters on attitudinal and emotional variables were generally in-between those of correctly assessed “no” voters and correctly assessed “yes” voters, and oftentimes closer to “no” voters. These attitudinal differences very likely helped lead assessors to, incorrectly as it turned out, believe they were going to vote yes. Although our focus is on comparing correctly assessed “no” voters and misassessed voters, in the layout of Table 2, we have added correctly assessed “yes” voters to illustrate this comparison between misassessed voters and both of the other correctly assessed groups.
Among the voting variables, a smaller proportion of misassessed voters had decided early on how they would vote (56%, compared with 85% for “no” voters), whereas a larger proportion of misassessed voters had decided toward the middle (29% vs. 13%) or end (14% vs. 2%) of the campaign period. These differences were statistically significant in the means tests; regressions controlling for multiple factors at once follow. 9
Regression analysis
To further explore relationships between these variables, we conducted a series of binary logistic regressions predicting misassessment. Model 1 examines the demographic variables, Model 2 examines the attitudinal and emotional variables, Model 3 examines the behavioral variables, and Model 4 is a full model that tests all predictors simultaneously. All model results are presented in Table 3.
Regression Analysis Comparing Misassessed Voters and Correctly Assessed “No” Voters on Demographic, Attitudinal and Emotional, and Behavioral Variables (Dependent Variable = Misassessed Voters).
Note. Dependent variable: “No” voters who were misassessed (i.e., predicted to be “yes” voters by the union assessment process when they were in fact “no” voters).
p ⩽ .10. *p ⩽ .05. **p ⩽ .01. ***p ⩽ .001.
When analyzing all variables together, having previously been a union member (β = 1.323, p ⩽ .01), having positive AFA and general union attitudes and emotions (β = 1.437, p ⩽ .01), and having decided at the end of the campaign (β = 2.467, p ⩽ .05) are statistically significantly related to being misassessed. Also, being a pre-merger Delta employee is negatively and statistically significantly related to being misassessed (β = −1.504, p ⩽ .05). These results are discussed further in the following.
Building Theory from the Quantitative Data Analysis
Consideration of these results led us to form some propositions concerning the systematic errors uncovered in the data analysis—both the means tests and the regressions. It also indicated to us several potential larger social science theories and research literatures that might be useful in further research on assessment practices. The qualitative portion of our study led us to flesh out several of these initial theoretical insights, but they are worth listing now.
Systematic interviewer bias
Among the demographic characteristics, having a union past made it more likely that flight attendants were misassessed by union organizers as would-be “yes” voters. It appears that the AFA-CWA assessors might have stereotyped these respondents by equating prior union membership with future support for their union. Canvassing scholarship from the political science literature describes a somewhat similar phenomenon whereby pollsters sometimes submit inaccurate interview ratings of voter respondents because of partisan bias. Healy and Malhotra (2014) present evidence that interviewers ascribe positive experiences to interactions with partisans, and they and others highlight the potential negative effect that such interviewer bias might have on polling data collection accuracy. A potential takeaway here is that during a certification election, unions ought to be cautious not to automatically equate prior union membership with current or future support for a union.
A similar process may have occurred with flight attendants who had previously worked for the union-represented Northwest, although it is even more likely that the legacy Delta workers’ longstanding pre-merger commitment to Delta was a powerful force behind their voting behavior, and that union organizers correctly recognized this force. Union organizers and those responsible for conducting benchmarks and assessments should not over-estimate prior union membership and/or under-estimate the potential influence of an employee’s prior attachment to their employer.
Demographic issues in matching assessors and assessed
Being a woman was negatively associated with being misassessed according to the mean differences; that is, they were less likely to be incorrectly assessed. Women comprised 80 percent of respondents in this analysis, and the flight attendant occupation as a whole tends to comprise mostly women (79% in the United States in 2007 according to Saenz and Evans 2009). A high percentage of assessors (member organizers) were also women. Perhaps the demographic match on gender led organizers to more accurately predict their voting behavior. The nonwhite results from the means analysis suggests that racial mismatch might have been an issue as well. We must caution that this is highly speculative given that these demographic differences did not hold up in the regression models. We explore this issue further in the qualitative section.
Predicting votes on attitudes/emotions alone
Among the attitudinal and emotion-related variables, there are indications that misassessed voters were both less positively inclined toward Delta, and more positively inclined toward unions, than correctly assessed “no” voters. (Only the second of these findings holds up in the regression models.) As with the “union past” variable earlier, when performing assessments, union organizers perhaps made the mistake of equating flight attendant beliefs that a union could positively impact their working conditions with support and a “yes” vote for the AFA-CWA (assuming that organizers tapped into and ascertained those beliefs during their assessment interactions with flight attendants). This leads us to think that assessing voters based on attitudes and emotions alone might be less accurate than assessments based on behaviors as well as attitudes and emotions, a matter that we were also able to explore in depth in the qualitative portion of the study.
Swing Voters and Ensuring Updated Assessments
The voting predictors of having decided how one would vote midway and near the end of the campaign—or in other words, being undecided until these later stages—were positively and significantly associated with being misassessed. We can think of at least two explanations for this. The first suggests that the AFA-CWA lacked some effectiveness, for whatever reason, in accurately assessing workers during the latter stages of organizing. As will be discussed later in the qualitative section of this paper, this appears to be partly due to the large size of the unit and an inability on the part of the union to update their assessment database frequently enough based on dynamic and rapidly evolving information. The second comes from the canvassing literature in political science, which notes the tendency of pollsters to overestimate support from voters who tell them they are undecided (Hillygus 2011), highlights difficulties of respondent dishonesty about voter intention (Belli et al. 1999), and points out that up to 40 percent of poll respondents change their vote intention at least once (Hillygus and Shields 2008). If any of these dynamics were at work during the AFA-CWA assessment effort, they might have contributed to the inaccuracy of the organizer’s predictions.
Qualitative Analysis
Characteristics of the Unions
Our original research design involved interviewing organizing directors (or the equivalent) of the fifteen largest unions in the United States. This design had to be modified for a variety of reasons, the primary one being difficulty in getting all these organizing directors to agree to an interview. Institutional Review Board (IRB) procedures mandated that we explain to potential interview participants what the interview would be about, and as a consequence, it seems possible that organizing directors who were not very interested in, or very experienced with, assessment procedures declined to talk to us. In other words, we may have over sampled unions with better or more extensive assessment practices.
We ended up interviewing fourteen individuals from nine unions. Most of the unions were in the service sector, many with major public sector representation. One union organizing director was in a union that was not in the top fifteen in terms of size; we included that person in order to have representation from the building trades. The fourteen respondents included two individuals from two major teachers unions (representing either different sectoral divisions or different levels—state versus national—within the union). One of the fourteen was a consultant with long experience in the field who spoke to his own experience and also that of a major manufacturing union.
With one exception, all of our informants reported that their union conducts assessments. One informant reported on a major campaign of the union where assessment was not used, interestingly because the union objected to the company doing something similar. It seems likely there was some response bias here and that the unions that did not respond to us were less likely to use assessment, but we cannot know for sure. Based on qualitative interview methodology (Rubin and Rubin 2012), we designed thirteen open-ended questions (see Appendix B Interview Protocol).
The Purpose of Assessments
All of our respondents agreed that assessment is a very important practice in organizing. 10 One respondent said “the whole campaign is built on the assessment.” Virtually all the unions used assessment in the context of new organizing campaigns, although there was some variation in how they actually used the assessments. These uses both include and go beyond what Bronfenbrenner and Hickey (2004) described, and include (1) determining if there is enough support to file for representation elections or terminate campaigns; (2) predicting the outcome of campaigns; (3) identifying individuals or groups (geographic, occupational, shift-based) to customize campaign issues and to target for GOTV efforts; (4) identifying potential volunteers, activists, and leaders; (5) building committees, community, and relations among organizers and members; (6) gauging support for negotiating the first contract; and (7) extending assessments to internal organizing and political action.
The Mechanics of Assessments
There are several variations in the specific mechanics of assessment that potentially could impact effectiveness. These include the rating scale used, the criteria used to assign a rating, and who does the assessments and the nature of their training (or lack thereof). There are also differences relating to unit size, frequency, tracking, and timing. Each of these are discussed in detail in the following.
Rating scale
Most unions use the same or a similar 4-point rating scale as illustrated in Figure 1. One union uses a slight variant with 5 points as follows: 1 = lead union member, 2 = support, 3 = undecided, 4 = anti-union, 5 = working for the company. 11 CWA normally uses a 3-point scale with the meaning of the numbers differing somewhat depending on the stage of the campaign. In general, the numbers mean the following: 1 = involved in activity on the campaign; 2 = undecided, not sure, still thinking about it; 3 = assumed or known to be a no. However, the more typical 4-point scale was used in the Delta campaign after some debate because the affiliate involved, the AFA, preferred that scale.

Excerpts from union organizing manual.
Criteria for ratings
Probably more important than the number of ratings or even the meaning of each number is the criteria and process for assigning the ranking. Figure 1 includes a brief indication of the criteria for each ranking for one union. As suggested in Figure 1, the ratings are typically behaviorally anchored. In fact, most of the unions we talked to set up “tests” or campaign actions that individuals can undertake—or not—and then keep track of who does and does not take the action. These actions typically differ over time as a campaign intensifies but are often public actions where there is some risk involved. The actions range from signing a card and wearing a button or ribbon, to attending a rally or other public event, to signing a “vote yes” flyer (which is then circulated to other workers). As one person interviewed put it, “We are looking for public demonstrations of support.” One union is moving away from the public tests because, in their opinion, they provide “too much information” to the employer about supporters. Instead, they are trying a “pledge card to yourself” where organizers ask workers to write down the reasons for supporting the union on a postcard, which is then collected by the union and mailed to workers just before the election. The card also provides fodder for the assessment.
In most cases, the behavioral tests or observations are supplemented with conversations between organizers and workers. Some unions rely more heavily on these conversations, a practice heavily criticized by some of our informants since they believe workers often tell organizers what they want to hear rather than their true feelings and intentions. As referenced earlier, some unions indicated that the criteria on which the rankings are based change over time. The riskiness and intensity of the actions required often escalate over time. One union indicated that at the early stages, during the committee building phase, assessment is largely based on conversations with workers, but that later on, actions become the focus. Some informants indicated that their unions attempt to recode all 2s and 3s (the fence sitters or ambivalent workers, depending on the scale used) to definite yes or no voters just before the election.
Reliance on conversations is particularly difficult for unions relying on member-organizers who, most of our informants (including those from AFA-CWA) report, are more likely to trust what their co-workers are telling them and less likely to give co-workers “bad grades” than are professional and experienced organizing staff. One informant had this to say about the accuracy of assessments by member-organizers: For members, accuracy is trainable. You have to destigmatize the 3 and 4. It’s not a black mark on the person’s soul. It feels [to the member-organizer] like a judgment on the person, but it’s not . . . You are not assessing the person by the action. “You can be a 4 and still be a good person” . . . I lie.
Who assesses?
Although most unions used some mix of paid staff and member-organizers, they differed on the ratio of the two. 12 One large service sector union relies primarily on paid staff while others have relatively few staff and actually prefer that workers talk to workers without paid staff being present. As earlier, CWA relies primarily on members to do assessments while recognizing that “members are always more generous [in their assessments of co-workers] than staff organizers: ‘Everyone in my work group is a supporter.’” Often when there are sufficient staff, staff will accompany member organizers at first so that they can be sure that members are assessing accurately (and presumably also to monitor the persuasion part of these conversations) or “spot check” member assessments. In CWA’s case, if doubts arise about the accuracy of a member-organizer assessments, staff will step in to work with the member. One union reported on something they call the “Triple Assessment” used toward the end of the campaign. This involves review of the pledge card to yourself described earlier, a staff organizer assessment, and the member organizing committee’s assessment, which is the least trusted but important, in the union’s view, to building community and a real organization. Another informant indicated that the organizing staff “assess organizing committee members” also to determine their effectiveness, although here, the behavioral anchor is whether the committee member is doing their assigned assessments of co-workers.
Competence, experience, motivation, and expectation of assessors
All of our respondents said that new organizers, followed by rank-and-file organizers, were less good at assessment than more experienced, professional organizers. The new organizers often are more eager to win, have less realistic expectations, and often overestimate a voter’s pro-union attitude. Doing accurate assessments takes practice. The ability and experience to ask questions, to listen, and to “read” people’s emotion and body language is critical. One safeguard includes formal training and pairing new paid staff organizers with seasoned organizers on the job (doing peer evaluation) as a type of informal training. 13
Unit size
It became apparent to us after talking to organizers that unions trying to organize very large units face a particular dilemma with regard to assessment. This was no doubt part of the problem in the Delta Air Lines flight attendant campaign. Several of the individuals we talked to spoke about increasing the number of paid staff in large campaigns so that they could maintain the ratio of paid organizers to organizing committee members, improving the reliability of assessment. But they were in unions for whom “large” meant trying to organize a hundred or a few hundred individuals—not the nearly twenty thousand in the Delta campaign. Another solution to the size problem is the mass mobilization of union staff in support of the campaign. Even with this, unions that undertake very large campaigns almost inevitably have to rely more on organizing committee members for assessment, with a likely resulting degradation in the quality of the assessment data. They can try to combat this through careful training of member organizers, for instance, but these organizers inevitably tend to be “less seasoned” than paid staff.
Training
Union practices around training assessors vary widely, although most do some sort of training. For many unions, assessment is a key aspect of the training—and evaluation—of staff organizers. As one informant put it, “Inaccurate assessments are the leading source of unemployment [for staff organizers]. It’s more important than the persuasion part of the job.” While this was an extreme view, multiple directors indicated that one way they evaluate organizing staff is whether or not their assessments are accurate. Some unions have an organizing training manual that is used for both staff and member training. Some unions do formal training with members, while almost all do some “on the job” training where staff and members are paired for home or office visits so that members can learn directly from staff. One informant explained that organizing teams come back to the union office after workplace visits and review the conversations they had and the ratings. Another venue for more informal training is the organizing committee meeting. Again, organizers will discuss the theory and practice of rankings but also go through actual rankings of unit members. One informant mentioned that training includes discussion of how to read facial expressions and body language. CWA does not differ in its approach to training from other unions we talked to, relying on both formal training and one-on-one support.
Timing, frequency, and tracking
Another potentially important variable is the frequency of assessment. All the unions that we spoke to regarded frequent assessment as desirable but practices clearly differed, based in part on the size of the unit, the ratio of paid staff involved in the campaign to prospective voters, and standard union practice. For instance, two of our respondents said “at least twice”—one said after in-person contact on a house call and then about one week before the election, while a second person said “at least once before recognition and at least once afterwards.” One respondent said at least three times. Another said that the ideal was weekly, but that did not always happen. And one said they try to assess daily as organizing committee members check in with staff, but in smaller units, it could slip to every other week. We interpret that as meaning that sometimes, unions expect organizing committee members to update staff as soon as they learn of any change in intention by an individual, but that in the heat of the campaign, this does not always happen smoothly. 14 An organizer associated with the Delta campaign reported that assessments were done “not often enough,” an observation that fits well with our finding that misassessed voters tended to decide and vote later than correctly assessed “no” voters.
Unions keep track of the ratings in various ways, and although one of our respondents was nostalgic for the older practice of using a physical chart on the wall, all are now moving to some form of electronic database. A few had simple union-constructed databases built on Microsoft Access or a similar program, but these organizers mentioned their unions were considering or definitely moving to one of the two more “organizer friendly” databases available to unions for this work. One is “the VAN,” a database that was designed for political organizing and political campaigns (see also Sipp 2016). 15 A second is a cell-phone-based program, “the LAN” or Labor Action Network—a piece of software developed by the AFL-CIO, which allows union organizers to input data using a cell phone after a house call or other conversation with a prospective member. This convenient uploading feature is critical for helping to keep the database current.
Demographic match
Finally, we asked our respondents whether, as the organizing literature seems to indicate (e.g., Hoerr 1997), it is important that the demographic characteristics of the person doing the assessing are the same in key dimensions as the individual being assessed. A couple said that age matters some—it was not a good idea to pair young college-aged assessors with older teachers, for instance. Another—responsible for organizing highly educated professionals—said that education level sometimes is an issue. One organizer said it was important to be conscious of gender and another spoke of the need in their union to have more female or Spanish-speaking organizers, but most claimed that at least as far as assessment goes (and contrary to what the literature suggests) gender, race, and ethnicity were not too important. (That does not mean that they felt there was no need for diversity among the organizing staff. On the contrary, some of them affirmed that that is important, just not from the point of view of assessment.) We also asked whether or not an assessment by a “friend” was typically better or worse than by someone who was not close to the individual being assessed. Few of those we interviewed had an opinion, although at least one union respondent said that having a positive relationship with an individual helped the accuracy of assessment.
For those unions that emphasize the use of behaviors, friendship or demographics really should not matter—a worker either does the behavior or does not; there is not supposed to be room for interpretation that could be effected by the social distance between assessor and assesse. CWA organizing staff report that there were some problems with geographic (North vs. South) cultural differences between assessors and voters in the Delta campaign. Member-organizers from the North and Midwest, who had worked for Northwest Airlines, tended to mistake Southern hospitality and politeness for a commitment to vote for the union, when it was not. Unfortunately, we did not collect geographic information on our survey and so could not examine that factor in our quantitative analysis.
Interviewee Views on the Accuracy of Assessments
Some of the organizing directors we spoke to made very strong claims about the accuracy of the assessments that they were typically able to make in an organizing drive, whereas others admitted that this is an area in which their union needs to make progress, implement more consistent standards and training, or otherwise improve. One director, whose union uses a high ratio of staff organizers systematically trained by the union, claimed that their assessments were within an accuracy of 1 percent. Another said, “within one digit.” A third interviewee stated that the assessments were accurate “if you corrected for over-confidence” of some of the people doing the assessment, a key issue that was mentioned by numerous persons we talked to. In general, the unions that used behaviorally anchored rating scales (BARS) pointed to the objective nature of that data, and stated that it helped overcome the tendency of member organizers and some staff organizers to be over-confident. Interestingly, one pointed out that they had found it important to track the behavior of “no” as well as “yes” voters, and that doing so improved accuracy—in that person’s view, one problem with the assessment used in the Delta campaign was the failure to carefully assess the true intent to vote among “nos” just as carefully as “yeses,” and as a result, the union was caught off guard when an unexpectedly large number of nos voted. Ironically, “weak nos” who ultimately decide not to vote can be a key to union representation election success; the employer’s success in this election in turning these individuals into actual voters apparently mattered for the outcome at Delta. Finally, one of our interviewees pointed out that in his organization, when assessments did not predict the election result well, they engaged in a vigorous “debriefing” process to try to understand why. This struck us as a potential best practice.
Theoretical Insights from the Combined Qualitative and Quantitative Results
The qualitative portion of the study allowed us to expand on the propositions that were initially derived from the quantitative portion of the study. It enlarged our understanding of the issues considerably, pointed to future research and practice directions that were more important, and indicated which theoretical paths were less useful.
The Link between Employee Behaviors and the Decision to Vote Union
A major contribution of our study is the highlighting of the importance of employee behavior for understanding voting intentions and decisions. Classic models of employee decision to vote for a union center around perceived instrumentality and worker attributes and attitudes (e.g., Barling et al. 1992), while more recently, analyses have also included the importance of emotions and feelings (e.g., Eaton et al. 2014; Martinez and Fiorito 2009). Despite a traditional focus on employee attributes and feelings during certification elections, the AFA-CWA procedures, along with those discussed during our interviews with union leaders, make clear that organizing directors were well aware of the difficulties of basing voting predictions on individual characteristics and attitudes alone—the most sophisticated unions in our study explored systems in which they could observe the behaviors of employees. Their goal was to get people not only to say that they supported the union but also to engage in a pro-union action.
We believe the potential use of employee behaviors as a tool for understanding voting intentions holds great promise, and encourage unions and scholars to consider ways to integrate this into practice and theory development. For example, in the employee performance management literature, there exists several tools for interpreting and leveraging employee behaviors—including BARS (Smith and Kendall 1963), behavioral observation scales (BOS; Wiersma and Latham 1986), and competency models (Campion et al. 2011). These and other approaches can likely be applied to union assessment processes, and could be tested in future empirical studies.
It should be noted that unions use “behavioral tests” in organizing not only to assess potential member voting behavior but also to change it. That is, assessment in organizing is never totally separate from persuasion—from the process of deepening commitment to the union and having potential members see the potential for building union power in order to change the workplace.
Combatting Systematic Interviewer Bias
Combatting systematic interviewer bias, especially the tendency of rank and file assessors and new organizers to over-estimate the degree of union support, was something that many unions were aware of and had adopted numerous practices to try to combat. Training of interviewers, multiple interviews and interviewers, and behavior-based assessment criteria are all being used now by unions to combat this key issue in assessment. It would be useful for researchers and union staff to explore theories and evidence-based remedies of interviewer bias in order to discern what else might be useful to combat this problem. Future research may also explore the accuracy of assessment between staff versus member organizers and experienced versus novice organizers.
Swing Voters and Updating Assessments
Organizing directors knew of the need to continually update assessments, although they were often limited in their ability to do so, particularly in large units. We learned less, however, from the interviews about the issue of swing voters. For instance, are “late deciders” in union elections typically people who ultimately “break for the employer,” that is, decide at the end of the campaign against committing to change the workplace by voting union? Or is this something that happens just in campaigns (like the Delta campaign) in which unions lose? Are late deciders often people who decide not to vote at all? These are matters for future research.
Demographic Issues in Matching Assessors and Assessed
Most of our interviewees did not think that demographic matching of assessors and the assessed was essential, although they did agree that more representation of women and minorities among organizers was desirable. This is an instance in which the qualitative portion of the study did not confirm the quantitative portion of the study and what the literature on organizing seems to suggest. However, more rigorous hypothesis-testing studies of assessment in this context would be valuable in exploring the importance, or lack of importance, of a demographic match between assessors and those they assess.
Other Theoretical and Research Implications of the Study
This study also may help fill a gap in the organizing literature concerning unit size. It is well known that unions are less successful in organizing large units than small ones (Farber 2001; Rose 1972). This study suggests that large units inherently have limitations in the quality of the assessment procedures that can reasonably be employed. Of course, we do not suggest this is the only, or even necessarily the main, reason why large units may be harder to organize. But it is apparently one reason, and that insight should be tested by others in future research on organizing success. Other implications are that unions doing very large-scale organizing need to pay particular attention to training bargaining unit members in assessment, adopt behavior-based assessment practices in which rater bias is less important, and discount positive predictions to a greater extent than other unions.
Conclusion
This research project began as an attempt to figure out why so many workers voted differently from how they were predicted to vote by the union during a particular organizing campaign. Via quantitative analyses of workers and organizers who participated in the 2010 flight attendant certification election at Delta Air Lines, we were able to identify relationships between key demographic, attitudinal and emotional, and voting variables. In particular, our analyses highlighted which worker characteristics were associated with incorrect assessments of individuals as supporters of the union and likely “yes” voters, when in fact they voted against union representation. This is important for unions since many—including the AFA-CWA during the 2010 Delta election—rely on assessment data to guide campaign decision making.
In addition, we queried fourteen organizing directors and other union leaders representing nine of the largest internationals in the United States about their assessment procedures. This portion of our project revealed key findings with practical relevance for the labor community. Among those include the importance of assessment frequency, training and development of those individuals actually performing assessments, the value of basing assessments on behaviors, and the challenges associated with assessment in very large campaigns of thousands of employees.
When assessments are used well by unions, the process becomes a critical component of the organizing process and one that, as Bronfenbrenner and Hickey (2003, 2004) note, can, when combined with benchmarks, contribute to higher win rates. This study suggests that the best assessment practices not only more accurately predict voting behavior but also actually change it by getting people to put vague pro-union sentiments into action. Organizers can use our findings to critically analyze and possibly adjust their assessments practices, and we encourage future scholarship to build upon our results and develop and test theories of assessment effectiveness. As several of our labor leader interviewees stressed, assessments represent an indispensable part of a larger strategy of building union power through mobilizing solidarity.
Footnotes
Appendix A
Appendix B
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
