How Do Legislators Assess Administrative Performance? Georgia’s Department of Transportation in the Eyes of the State’s Legislators

Abstract

State legislatures and their member legislators serve as important overseers to state administrative departments, charged to function as principals relative to departmental agents. Yet, we know relatively little about how legislators assess the performance of those departments. This research is designed to improve that knowledge through an exploratory analysis of how and why legislators in one state assess the performance of a large state government department. Using data from a survey of Georgia state legislators, the article explores legislator evaluations of the state’s Department of Transportation (GDOT) and the factors that may underlie those evaluations. The findings suggest that legislators assess administrative performance on three principal dimensions: (a) administrative service to individual legislators, (b) assistance to the legislature as a whole, and (c) performance in meeting the state’s transportation needs. Those assessments appear to be shaped by legislator perceptions of (a) personal interactions with the department and (b) the quality of specific GDOT products and services. These and earlier findings suggest that the focus of public performance measurement systems might be broadened to include measures of personal treatment by administrative agencies in addition to traditional objective service outcome measures.

Keywords

performance measurement legislative relations

State legislatures and their member legislators serve as important overseers to state administrative departments, charged to function as principals relative to their departmental agents. More than a quarter century ago, Brudney and Hebert (1987) reported that state legislatures exercise greater influence over the operations of administrative agencies than do such other stakeholders as clientele groups and professional associations. How those legislators assess the performance of those agencies could substantially affect their future, including how well they are funded and how closely they are monitored. Understanding how those assessments are made, thus, could hold significance for agency administrators as they strategize for working with legislators.

Despite an “enormous literature measuring the influence elected officials hold over the U.S. bureaucracy” (Palus & Yackee, 2013, p. 274), we know relatively little about the basis for legislative assessments of bureaucratic performance, especially at the state level. As Sarbaugh-Thompson et al. (2010) have observed, “Political scientists have written much about relationships between Congress and federal agencies, but said less about state legislatures’ relationships with state agencies” (p. 57). As a consequence, “Relationships between legislatures and state agencies are important but poorly understood.”

The available evidence suggests substantial legislative interest in administrative performance, interest that appears to have a positive impact on agency efficiency, agency reform, and the use of performance measures in the budget process (Bendor & Moe, 1985; Bourdeaux, 2006; Bourdeaux & Chikoto, 2008). However, no research appears to have examined what shapes legislator assessments of agency performance.

This research is designed as a first exploratory step in that direction. We will begin by examining the literature on legislative behavior to develop expectations for how legislators assess agency performance. Those hypotheses will be tested with data from a survey of how legislators in the state of Georgia assess the performance of the Georgia Department of Transportation (GDOT). We will conclude by considering the limitations of the research as well as what the findings may imply for legislative assessments of executive agencies and for public performance assessment more generally.

Theorizing About Legislator Assessments

An extensive literature on performance management focuses on why performance information is produced (Behn, 2003; Sanger, 2008), how performance management systems are designed and implemented (Hatry, 2014; Heinrich & Marschke, 2010; Yang & Holzer, 2006), how performance information is used (Kroll, 2014, 2015; Moynihan, 2008; Moynihan & Kroll, 2016; Moynihan & Pandey, 2010), and what effects performance information has on agency performance (Bouckaert & Peters, 2002; Gerrish, 2016; Heinrich, 2002; Moynihan, 2013; Thomas, Poister, & Ertas, 2010). This literature also tells us a great deal about executive-level performance measurement and management innovations (at the federal level, see Moynihan & Lavertu, 2012; at the local level, see Abramson & Behn, 2006). We also know that legislatures figure prominently in the assessment of the performance of administrative agencies. At the federal level, the Congressional Budget Office and the Government Accountability Office (GAO), as arms of Congress, both play substantial roles in assessing agency performance.

At the state level, although agencies are influenced by multiple external actors, including the governor, clientele groups, and professional associations (Brudney & Hebert, 1987), legislatures are critical stakeholders and may exercise substantial influence over administrative agencies (Bourdeaux & Chikoto, 2008; Sarbaugh-Thompson et al., 2010). Legislative power actually appears to have grown in recent decades as a result of “reform efforts designed to permit legislatures to respond to the problems facing state governments” (Bernick & Bernick, 2008, p. 970; see also Moncrief, Thompson, & Cassie, 1996). Some observers contend that state public administrators “view their interaction with the Legislature as more important than their relationship with the governor” (Lee, 2006, p. 1023). Yet, studies of state legislators’ assessment of agency performance are rare, leaving the issue of how legislators reach those assessments a mystery.

In pondering the issue, a first question asks what legislators mean by agency “performance.” Judging from conversations with GDOT officials and Georgia legislators (see below for more detail) and a review of the literature on legislative behavior, legislators appear to assess agency performance on three principal dimensions: (a) service to individual legislators, (b) assistance to the legislature as a whole, and (c) performance in meeting the needs of the state and its residents.

Performance Toward Mission Achievement

The last of those three elements may be what first comes to mind in thinking about an agency’s performance. This element focuses on bottom-line performance toward mission achievement: How well does a department perform at what it is supposed to achieve? In the case of a department of transportation, how effective is the agency in meeting the state’s transportation needs?

State legislatures have shown growing interest recently in this bottom-line agency performance. As Berry, Turcotte, and Latham (2002) have documented, “Legislative oversight generally and program evaluation units more specifically were shaped in the 1990s by the government accountability efforts and performance-based budgeting” (p. 75). These units conduct “performance audits and policy analyses . . ., assess the quality of performance measures and agency compliance with performance-based budgeting mandates,” and “interpret the agency performance measurement reports to the legislatures” (p. 75). As a possible consequence, according to Bourdeaux (2006), “legislative involvement may . . . be critical, perhaps even more important than gubernatorial involvement, in promoting use of performance information throughout the budgetary process” (p. 141).

Performance in Service to Individual Legislators and the Legislature

Judging from the literature and conversations with current and former Georgia legislators, legislators are also interested in two other dimensions of agency performance: (a) responsiveness to their individual requests, especially requests relayed from constituents; and (b) assistance to the legislature in formulating and reviewing legislation.

Taking the former first, constituency casework finds individual legislators seeking information and sometimes intervention from administrators on specific services or programs in their home districts. Typically, legislators forward requests from constituents, including individual citizens, local government officials, as well as interest group representatives. For a department of transportation, the requests could pertain to what projects the department will fund in a district, how to solve a problem with a project under construction, or when a project will be completed.

Constituency service holds a high priority for legislators. As Mordecai Lee (2006), himself a former state legislator, has said of constituent casework, “most legislators view it with relish, as a service that has great political payoff” (p. 1029). Behn (2002) has explained why that might be the case:

We do not really expect our individual legislator to have very much impact on the government’s long-term performance. We do, however, recognize that our legislator can definitely have a short-term impact on how many new resources are allocated to our community. (p. 12)

The evidence extends beyond the anecdotal, too, with one field experiment showing that “legislators are more responsive to service requests than to policy queries,” with that pattern “more pronounced among state legislators” than among members of Congress (Butler, Karpowitz, & Pope, 2012, p. 484). Legislators who want to be reelected seem likely to take a keen interest in constituency service.

Finally, legislators may also care about how well agencies perform in serving the legislature as a whole. Administrative service to the legislature involves providing information, as for use in reviewing administrative performance and assisting in the formulation of public policy, as by drafting policy proposals. Administrators may provide this information either at the behest of legislators or at their own initiative.

The agency role in formulating proposed public policies rests on the fact that, “Legislators need ‘product’” (Lee, 2006, pp. 1034-1035). That is, “For legislators to feel that they are performing their role requires having an inventory of bills to push,” and “Administrative agencies are a natural birthplace of ideas for legislation.” Legislative oversight responsibilities may further nurture this role because, as Palus and Yackee (2013) note, “Oversight opens channels of communication and allows agency leaders to influence the policy-making process by sharing their policy expertise” (p. 274).

As a consequence of legislative specialization, however, only a few legislators may care about this aspect of agency performance for any given agency. As Lee (2006) has explained, “It is a truism of legislative service that some legislators are more interested in specific public policy sectors than others” (p. 1032). That specialization may mean that, for any specific agency, this aspect of performance may loom large for only the handful of legislators who specialize in that agency’s functional area.

Explaining Legislator Assessments of Agency Performance

The next task is to define what might shape these legislator assessments. Explaining assessments of agency service on constituency casework could be a relatively simple matter, with those assessments likely to hinge on perceived administrative responsiveness (e.g., helpfulness, fairness, promptness). The working relationship may also encompass agency delivery of projects on a timely basis, for which legislators can then claim credit. “Credit claiming,” as Ness (2010) has observed, “requires the elected official to take personal responsibility for governmental action or other changes that are largely viewed as beneficial” (p. 37; see also Mayhew, 2004). That logic leads to the following hypothesis:

Hypothesis 1: Legislator assessments of an agency’s performance in its administrative service to individual legislators will be a function of perceptions of agency (a) responsiveness to requests and (b) timeliness in delivering projects.

The likely explanation of assessments of service to the legislature as a whole is only slightly less straightforward. Agency performance in keeping the legislature informed may be one crucial factor because being kept informed is a principal service agencies can perform for a legislature. However, assistance in formulating legislation may not be crucial for most legislators because specialization should mean that only a handful follow any particular agency’s work in that area.

Legislators’ assessments of an agency’s service to them as individual legislators, the focus of the first hypothesis, could also affect their assessments of the agency’s service to the legislature as a whole. Legislators may not see a sharp dividing line between what an agency does for them individually versus what it does for the legislature as a whole. Consistent with that speculation, the quality of personal interactions with an agency has been found to influence how some public officials, other than legislators, assess an agency’s overall performance (Thomas, Poister, & Ertas, 2010). This reasoning leads to a second hypothesis:

Hypothesis 2: Legislator assessments of an agency’s performance in meeting the needs of the legislature as a whole will be principally a function of perceptions of the agency’s performance in (a) keeping the legislature informed and (b) providing service to individual legislators.

The issue becomes more complex when we ask what factors might shape legislator assessments of bottom-line agency performance in mission achievement. Oddly, we could find no prior research on this question. Logically, we might expect those assessments to be based principally on perceptions of agency performance in providing specific products and services, especially in the legislator’s home district. With transportation, those products and services might include perceptions of, for example, the quality and safety of roads and the extent of traffic congestion in a legislator’s district.

These bottom-line assessments may also be shaped, in part, by assessments of the other two elements of agency performance, service to individual legislators and to the legislature as a whole. We already know that people in general “find it difficult to distinguish clearly between the quality of a . . . service and the process by which the service was rendered” (Fountain, 2001, p. 4), such that they often judge government’s response to a request as much by its courtesy and promptness as by whether they received the product they requested. A similar pattern appears to hold with local government officials, both elected and appointive (Thomas, Poister, and Ertas, 2010). The typical legislators’ “choice to privilege service over policy” implies that legislators’ assessments of an agency’s bottom-line performance may also be influenced by how they perceive the agency has responded to their personal requests and to the needs of the legislature as a whole (Butler et al., 2012, p. 485).

Finally, legislator assessments of mission achievement may also be influenced by perceptions of an agency’s funding and spending. If legislators perceive an agency’s funding as inadequate, they may be more inclined to perceive the agency’s performance as compromised. If they believe the agency is spending appropriately, they may be more inclined to perceive it as achieving its mission.

Among these factors, the quality of the end products (e.g., road quality, traffic congestion) may exert the strongest influence on perceived bottom-line performance because those end products appear logically more closely linked to that performance. This third hypothesis results:

Hypothesis 3: Legislator assessments of an agency’s performance in achieving its mission will be a function primarily of perceptions of (a) the quality of the agency’s core services and products and secondarily of perceptions of (b) its performance in serving individual legislators and the legislature and (c) the adequacy of its funding and the appropriateness of its spending.

Figure 1 summarizes the three predictions, also showing how the three dimensions of GDOT performance are expected to interrelate.

Figure 1.

A conceptual framework of legislator assessments of GDOT performance.

The Georgia Case: GDOT and the General Assembly

These hypotheses will be tested using data on how legislators in the state of Georgia assess the performance of the GDOT. Georgia’s legislature, known as the General Assembly, is a typical bicameral state legislature comprised of the House and the Senate. It fits the description of a “citizen legislature,” in that the pay for members is so low (in 2017, US$17,342 per year plus US$173 per diem, when in session) that most must hold other full-time jobs. Legislators are not term limited, and, despite the low pay, many hold office for multiple terms. At the time of this research, Republicans held majorities in both houses of the General Assembly as well as the governor’s office.

As a citizen legislature, the General Assembly ranks in the bottom quadrant of state legislatures across the United States in terms of professionalism, placing 39th in Squire’s (2007) most recent rankings. That ranking places Georgia in the middle of a group of Southern and Southeastern states, ranking below regional neighbors Louisiana, South Carolina, Virginia, Mississippi, and Tennessee and above West Virginia, Arkansas, and Kentucky. The low rating for professionalism reflects, in part, that most Georgia legislators lack full-time staff assistance.

The General Assembly formulates public policy on transportation matters for the State of Georgia and serves as an oversight body for GDOT. Like other state Departments of Transportation (DOTs), GDOT is one of the largest departments of the state government. In a somewhat unusual arrangement among the states, the General Assembly chooses some members of GDOT’s governing commission, the Georgia State Transportation Board, at the beginning of each legislative session, selecting members from districts that correspond to Congressional districts. Members are selected on a staggered basis for 5-year terms; many are former state legislators themselves.

However, according to the current and former Georgia legislators with whom we talked, Georgia is probably similar to other states in the strong interest legislators take in how GDOT allocates funding, presumably because projects in their districts can be cited as evidence of their legislative effectiveness. DOTs, in most states, may experience similar politics due to the legislative interest in “credit claiming” (Ness, 2010, p. 37).

Stakeholder engagement has become a priority for state DOTs for a variety of core functions and management processes in recent years (Lockwood, 2006; Schwartz, 2006). With highway planning and design, for example, many states have demonstrated a commitment to working with a variety of stakeholders, soliciting input from motorists regarding their needs and preferences for new or expanded highway facilities and developing context-sensitive designs and providing for environmental sustainability in building new projects (Bingham, 2006; Goetz, Dempsey, & Larson, 2002; Jeon & Amekudzi, 2005; Neuman, Schwartz, Clark, Benar, & Forbes, 2002).

Consistent with this pattern, GDOT has become increasingly concerned in recent years about relationships with its stakeholders. GDOT officials have voiced commitment to improving relationships with groups ranging from customer and clientele groups (e.g., professional drivers, the general public) to partners and suppliers (e.g., consulting engineers, local governments) to overseers (e.g., the General Assembly). One objective in GDOT’s strategic plan calls for improving working relationships with all its stakeholders. As GDOT’s former director of strategic development commented to the authors, “You have only a truncated view of an organization if you don’t have a view of its partnerships.” That may be especially true of GDOT because, as he noted, “the department is involved in so many partnerships.”

Toward that end, the Transportation Board commissioned a “stakeholder audit,” that is, a detailing of the full range of the department’s stakeholders plus a delineation of information the department lacked on various stakeholders (Poister, Thomas, & Berryman, 2013). As the impetus for the current research, the board subsequently sought to fill one of the information gaps by commissioning a survey of members of the General Assembly.

Method

The survey of Georgia legislators was designed to obtain feedback on their interactions with the department and perceptions of GDOT performance. Two of the authors developed the survey instrument with assistance from (a) a steering committee of the GDOT managers most knowledgeable about interactions with the General Assembly, (b) the chairs of the Transportation Committees in both houses of the General Assembly (Senator Jeff Mullis and Representative Vance Smith), and (c) three members of the State Transportation Board, including two former legislators.

The survey development process began with a meeting of the steering committee to discuss the purposes and possible content of the questionnaire. One of the authors then held brief informal meetings with each of the committee chairs and the members of the Transportation Board, using a semistructured interview format to ask about aspects of GDOT performance that should be covered in the questionnaire. The authors next revised the questionnaire for a final review by the steering committee.¹

The survey was administered in two modes, online and by hard copy via regular mail, in July and August 2007. A hard copy of the survey was initially sent to the home addresses of all 180 representatives and 56 senators in the General Assembly, accompanied by a cover letter of endorsement signed by the GDOT Commissioner, the governor-appointed head of GDOT, and the two Transportation Committees’ chairs. To maximize the response rate, the survey instrument was limited to two sides of one page. Approximately 2 weeks later, an email reminder was sent to all representatives and senators who had not responded to the initial survey, offering an online option for completing the survey. Three weeks after that, another hard copy of the original survey and cover letter was sent by regular mail to all remaining nonrespondents, with a sticker on the cover letter asking whether the individual had forgotten to respond.

As Table 1 shows, these efforts brought responses from 47% of the membership of the General Assembly, an excellent response rate for surveys of this kind, with almost identical response rates for the two houses. Approximately half, or 49.5%, of the respondents described themselves as members “of the House or Senate leadership, including a House or Senate officer, a caucus officer, and/or a committee chairperson.” Not surprisingly, more than twice as many respondents (84%) from the smaller Senate described themselves as in leader roles than from the much larger House (39.5%). Most respondents (59.1%) reported having spent more than four years in the General Assembly, with those proportions again higher for the Senate (73.1%) than the House (54.2%). Data not shown revealed that respondents came almost equally from inside and outside the Atlanta metro area, paralleling the division of the state’s population.

Table 1.

Profile of Legislative Survey Respondents.

	Senate	House	Total
Total membership (n)	56	180	236
Number of responses (n)	26	85	111
Response rate (%)	46.4	47.2	47.0
Proportion in assembly leadership positions (%)	84.0	39.5	49.5
Years as a member
>4 years (%)	73.1	54.2	59.1
2-4 years (%)	19.2	27.7	25.5
<2 years (%)	7.7	18.1	15.5

Findings

The survey contained questions that speak to each of the three crucial elements of GDOT performance.² These questions, which serve as the dependent variables, read as follows:

For service to the individual legislator: “How satisfied are you with your working relationship with GDOT?” (five options from very dissatisfied to very satisfied).

For service to the legislature: “How would you grade GDOT’s performance in serving the needs of the General Assembly?” (five options from “A”/excellent to “F”/failing).

For bottom-line mission achievement: “How would you grade GDOT’s overall performance in meeting transportation needs in Georgia?” (same five “A” to “F” options).

We will first examine the descriptive findings on these three variables.

Legislator Assessments of GDOT Performance

Judging from the data in Table 2, GDOT drew a lot of interest from legislators during this period. Asked how often they contacted various GDOT officials and offices “to request assistance or register complaints on behalf of one or more of [their] constituents, including local governments, during the past twelve months,” almost all respondents reported having initiated multiple contacts, with 58% reporting five or more contacts and another 32.1% two to four contacts. Only 9.8% reported making one or no contacts.

Table 2.

Legislator’s Contacts With GDOT.

Times of contacts in 12 months	0-1 time	2-4 times	More than 5 times	Total
Times of contacts in 12 months	9.8%	32.1%	58.0%	99.9%
Reasons for the contacts

Note. GDOT = Georgia Department of Transportation.

The principal reasons for these contacts were to obtain information on projects, presumably projects in the legislator’s district, most prominently project status updates (72.3%) and information on initiation of projects (34.8%). Next on the list came constituent complaints (59.8%) and traffic signals (46.4%), the latter itself potentially a specific kind of constituent complaint (as with requests to install traffic signals).

As shown in Table 3, most respondents (79.8%) reported being satisfied or very satisfied with their working relationship with GDOT. Consistent with that assessment, respondents also gave GDOT mostly positive ratings for its responsiveness to their contacts with the department. Substantial majorities were “satisfied” or “very satisfied” with GDOT’s “helpfulness” (88.5%), “timeliness” (87.3%), “courtesy” (94.1%), “fairness” (81%), and “overall responsiveness” (87.1%).

Table 3.

Legislator Assessments of GDOT Performance.

Satisfaction with GDOT working relationship
%	Very satisfied	Satisfied	Neutral	Dissatisfied	Very dissatisfied	Total
	41.3	38.5	15.6	3.7	0.9	100.0
Responsiveness %	Very satisfied	Satisfied	Neutral	Dissatisfied/very dissatisfied		Total
Helpfulness	47.1	41.4	7.7	3.9		100.1
Timeliness	46.1	41.2	7.8	4.9		100.0
Courtesy	53.5	40.6	5.0	1.0		100.1
Fairness	42.0	39.0	13.0	6.0		100.0
%	A/excellent	B/good	C/fair	D/poor and F/failing		Total
On-time project delivery	8.7%	24.3%	30.1%	36.9%		100.0%
Performance in serving the needs of the General Assembly
%	A/excellent	B/good	C/fair	D/poor	F/failing	Total
	18.0	41.4	26.1	6.3	6.3	98.2^a
Keeping legislature informed %	13.9	45.4	24.1	11.1	5.5	100.0
Performance in meeting Georgia’s transportation needs
%	A/excellent	B/good	C/fair	D/poor	F/failing	Total
	7.3	32.7	36.4	10.9	12.7	100.0
%	A/excellent	B/good	C/fair	D/poor and F/failing		Total
In home districts
Traffic congestion	7.3	30.3	20.2	42.2		100.0
Road safety	9.4	50.5	24.3	15.9		100.1
Road maintenance	20.4	46.3	23.2	10.2		100.1
Preserving environment	23.8	46.7	20.0	9.5		100.0
Providing variety of transport options	13.0	20.4	36.1	30.6		100.1
Spending appropriately	8.8	35.3	30.4	25.5		100.0
%	Strongly agree	Agree	Neutral	Disagree/strongly disagree
Receiving adequate funding	1.0	7.8	13.7	77.5		100.0

Note. Numbers may not add up to 100% due to rounding. GDOT = Georgia Department of Transportation.

Two respondents or 1.8% replied “don’t know.”

On-time delivery of projects to the legislator’s district represents another crucial component of this working relationship, according to our advisers. Here, as Table 3 details, the department received relatively low grades, with a 36.9% plurality of respondents giving D or F grade as compared with only 33% who gave A or B grade.

Respondents returned to a mostly positive assessment of GDOT’s performance in “serving the needs of the General Assembly,” the second principal dimension of administrative performance. As shown in the middle of Table 3, 59.4% gave the department A or B grade and only 14.4% chose D or F grade. On one specific element of this service, a 59.3% majority agreed or strongly agreed that “GDOT keeps legislators adequately informed about its programs and spending plans,” with only 16.6% disagreeing or strongly disagreeing.

As for bottom-line mission achievement, the lower part of Table 3 shows mixed assessments from respondents of the GDOT’s performance in “meeting transportation needs in Georgia.” A 40% plurality gave the department A or B grades, but a substantial 23.6% minority registered D or F grades. Those percentages equate to slightly better than a C grade, a 2.11 “grade-point average” on a 4-point grading scale, a middling assessment at best.

Respondents also offered mixed assessments of GDOT’s performance in providing various products and services, as shown in the same table.³ However, substantial majorities gave the department A or B grades for (a) “building, maintaining, and operating the state highway system” (66.7%); (b) “preserving the environment when planning and building transportation projects” (70.5%), and in their home districts; (c) “highway condition and road quality” (76.1%); and (d) road “safety” (59.8%).

However, GDOT received relatively low grades for (a) “providing Georgia residents with a variety of transportation options” (grades almost evenly divided between A/B, C, and D/F) and (b) “traffic flow and congestion” in the legislator’s home area (42.2% D or F grades). On the latter, in data not shown in the table, the lowest grades came from respondents in the notoriously congested Atlanta metropolitan area, 79.4% of whom registered D or F grades, more than three times the 24.3% proportion for respondents from outside the Atlanta area.

Finally, on spending and funding questions, respondents were divided on whether “GDOT spends its funding appropriately,” a plurality of 44.1% agreeing or strongly agreeing but 25.5% disagreeing and 30.4% neutral. By contrast, respondents mostly felt that GDOT needs more funding, with 77.5% disagreeing that “GDOT funding is adequate to meet Georgia’s transportation needs.”

Explaining Legislator Assessments

The task of explanation focuses on the three summary measures of legislator evaluation of GDOT’s performance: (a) satisfaction with the legislator’s working relationship with GDOT, (b) grades for service to the legislature as a whole, and (c) grades for performance in meeting the state’s transportation needs. Because all three variables are ordinal, the classic linear regression assumption cannot be satisfied. We chose instead to use ordered logistic regression, which most statisticians recommend as a better fit for ordinal dependent variables (e.g., Long & Freese, 2006; McKelvey & Zavoina, 1975).

Before testing hypotheses, we checked for multicollinearity in each model by using the variance inflation factor. The results indicated that none of the models suffers from a multicollinearity problem. The ordered logistic regression assumes that the relationship between any two pairs of outcome groups should be statistically the same. To test whether our data meet the proportional odds assumption, we also performed the Brant test of parallel regression assumption to check whether the logistic regression model is valid. The results indicated that the proportional odds assumption held for the first two models, but not for the last model with the dependent variable of meeting the state’s transportation needs. To address skewness in that variable, we combined responses into three categories: (a) favorable (A or B), (b) neutral (C), and (c) unfavorable (D or F). After the correction, the third model passed the Brant test on the proportional odds assumption, and, therefore, the ordered logistic model is valid.

Tables 4 to 6 present the results of the ordered logistic regressions for the three dimensions of GDOT performance. To make the results more understandable, we converted logistic coefficients to probability changes (shown in the last column), holding all other predictor variables at their means. The probability changes indicate how much a one-unit increase in the independent variable at the mean would increase the probability of legislators giving better assessments. In the first model, we focus on the probability of change from “satisfied” to “very satisfied” with their GDOT working relationship because most legislators fell in those categories (38.5% satisfied and 41.3% very satisfied). In the second and third models, we focus on the probability of changes from “neutral” to “satisfied” (“C” to “B”) or, in other words, on how to make legislators feel at least satisfied with administrative performance.

Table 4.

Determinants of Legislator Satisfaction With Georgia Department of Transportation Working Relationship.

	Coefficient	SE	Probability change % (from “satisfied” to “very satisfied”)
Responsiveness elements
Helpfulness	−0.65	0.73	−8.68
Timeliness	2.26**	0.91	30.20
Courtesy	0.19	0.78	2.60
Fairness	2.60***	0.74	34.72
Assistance from general office	1.69**	0.72	22.61
On-time project delivery	0.74**	0.32	9.96
Member >4 years	−0.86	0.78	−12.09
Legislative leader	1.56*	0.82	21.45
N = 84
LR χ²(8) = 127.31***
Pseudo R² = .658

Note. LR = Logistic Regression

p ≤ .1. **p ≤ .05. ***p ≤ .01.

Table 5.

Determinants of Performance in Meeting the Needs of the General Assembly.

	Model 1(The hypothesized variables plus both legislator characteristics)			Model 2(With on-time project delivery added and both legislator characteristics omitted)
	Coefficients	SE	Probability change %(from “C” to “A or B”)	Coefficients	SE	Probability change %(from “C” to “A or B”)
Working relationship satisfaction	1.39***	0.31	21.06	1.34***	0.33	20.67
Keeping the legislature informed	0.81***	0.24	12.27	0.45*	0.25	6.99
On-time project delivery				0.97***	0.22	14.92
Member >4 years	−0.26	0.45	−3.85
Legislative leader	0.10	0.44	1.52
N = 104				N = 101
LR χ²(4) = 61.65***				LR χ²(3) = 85.0***
Pseudo R² = .215				Pseudo R² = .303

Note. LR = Logistic Regression

p ≤ .1. **p ≤ .05. ***p ≤ .01.

Table 6.

Determinants of Performance in Meeting Georgia’s Transportation Needs.

	Model 1(Including all Georgia Department of Transportation outcome indicators and both legislator characteristics)			Model 2(With adequacy of funding and leadership role omitted)
	Coefficients	SE	Probability change %(from “C” to “A or B”)	Coefficients	SE	Probability change %(from “C” to “A or B”)
In home district
Traffic congestion	0.03	0.26	0.58	0.04	0.26	3.76
Road safety	1.01***	0.34	18.28	1.01***	0.34	17.70
Road maintenance	0.35	0.32	6.36	0.36	0.31	7.10
Providing variety of transp. options	0.80***	0.26	14.58	0.80***	0.23	15.63
Preserving the environment	−0.35	0.33	−6.36	−0.38	0.31	−7.36
Spending appropriately	0.69**	0.28	12.49	0.48**	0.24	9.31
Adequacy of funding	0.21	0.29	3.75
Performance in meeting needs of General Assembly	1.43***	0.35	25.99	1.38***	0.33	26.89
Member >4 years	0.68	0.55	12.05	0.77*	0.46	14.47
Legislative leader	0.26	0.57	4.70
N = 86				N = 92
LR χ²(10) = 100.51***				LR χ²(8) = 103.54***
Pseudo R² = .407				Pseudo R² = .395

Note. LR = Logistic Regression

p ≤ .1. **p ≤ .05. ***p ≤ .01.

Satisfaction with the GDOT working relationship

The first model examines possible determinants of legislator satisfaction with their working relationship with GDOT. Rephrasing the earlier hypothesis to fit the survey items, we hypothesize the following:

Hypothesis 1: Legislator satisfaction with their working relationship with GDOT will be a function of perceptions of (a) the helpfulness, courtesy, fairness, and timeliness of responses to requests; (b) the specific responsiveness of GDOT’s general office in Atlanta and/or area offices around the state; and (c) grades for on-time delivery of actual projects.

The results in Table 4 support the hypothesis. First, the perceived fairness and timeliness of GDOT’s responses to legislator requests both emerge as significant predictors of legislators’ satisfaction with their working relationship, with fairness appearing to exert the most influence. With other variables held at their means, if legislators’ satisfaction with “being treated fairly” increased by one unit (e.g., from poor to fair or from fair to good), the probability of being “very satisfied” rather than only “satisfied” with the working relationship would increase by 34.7 percentage points. As for timeliness, a one-unit improvement in GDOT’s timeliness would increase by 30.2 percentage points the probability of legislators being “very satisfied” as opposed to just “satisfied.” Satisfaction with the specific responsiveness of GDOT’s general office in Atlanta proved a third important factor. With other variables held at their means, a one-unit increase in the general office’s perceived responsiveness would increase by 22.6 percentage points the likelihood of being “very satisfied” instead of only “satisfied.”⁴

Timeliness plays another important role through the influence of perceived on-time project delivery on working relationship satisfaction. If GDOT improved perceived on-time project delivery speed by one grade, the probability of legislators being “very satisfied” as opposed to just “satisfied” with that relationship would increase by 10 percentage points.

Interestingly, legislator ratings of the actual helpfulness of GDOT’s response do not appear to affect working relationship satisfaction. These legislators appear to care more about being treated fairly and in a timely manner, which includes on-time project delivery, than about getting particular benefits that helpfulness implies.

Two legislator characteristics, holding a leadership role in the General Assembly and having served more than 4 years, were also included in the logistic analysis. Holding a leadership role almost achieved statistical significance as another predictor of satisfaction with the GDOT working relationship, suggesting that GDOT may be especially attentive to the needs of legislative leaders due to their potential influence over GDOT’s funding and other legislative priorities. By contrast, longer tenure in the General Assembly does not appear to affect that working relationship satisfaction.

Assistance to the General Assembly

Turning to legislator assessments of GDOT’s performance in serving the needs of the legislature, and again rephrasing the earlier hypothesis to fit the survey variables, we expect the following:

Hypothesis 2: Legislative assessments of GDOT’s performance in meeting the needs of the legislature as a whole will be a function of (a) perceptions of GDOT’s performance in keeping the legislature informed and (b) individual legislators’ satisfaction with their working relationship with GDOT.

The findings, as shown in Model 1 in Table 5, support the hypothesis as both variables emerge as highly significant predictors (p < .01) of legislator grades for GDOT service to the General Assembly.

However, recognizing the importance of the personal factor, we decided to experiment with adding on-time project delivery as another possible predictor of GDOT’s perceived performance in meeting the needs of the General Assembly. As discussed earlier, legislators likely take a keen interest in whether GDOT projects reach their districts in a timely fashion. At the same time, we dropped from the analysis the two legislator characteristics due to their lack of statistical significance in the original logistic.

As shown in Model 2, this revised logistic does a better job than Model 1 of predicting perceptions of GDOT’s performance in meeting the needs of the General Assembly. Personal factors do, in fact, emerge as the most important predictors, with working relationship satisfaction and on-time project delivery both achieving significance at the .01 level. On average, a one-unit change in working relationship satisfaction raises by 20.7 percentage points the probability of legislators giving GDOT a one-grade improvement in meeting the needs of the General Assembly. Similarly, a one-unit change in perceived on-time project delivery raises by 14.9 percentage points the probability of that same one-grade improvement. By contrast, keeping the legislature informed declines in statistical significance in Model 2. These findings add to the evidence that legislators conflate GDOT’s personal service to them (working relationship satisfaction and on-time project delivery) with its service to the legislature as a whole.

Performance in meeting Georgia’s transportation needs

As argued earlier, legislators may weigh a number of factors in assessing GDOT’s bottom-line performance in meeting the state’s transportation needs. They may think first not only about the quality of GDOT’s services and products in their home districts (e.g., highway maintenance, road safety, traffic flow, and congestion) but also about aspects of GDOT performance that extend beyond their home districts, especially—in the Georgia case anyway—providing a variety of transportation options and preserving the environment when undertaking transportation projects. Second, they may also be influenced by their perceptions of GDOT’s financial side, in particular, perceptions of the adequacy of the department’s funding and the appropriateness of its spending.

Third, legislators may again be influenced by their personal experience with GDOT, how GDOT has performed for them, and for the General Assembly of which they are a part. Here, though, feelings about the GDOT working relationship may exert an influence only indirectly through its effect on perceptions of how well GDOT meets the needs of the legislature as a whole.

Combining these several factors results in this rephrasing of the earlier hypothesis:

Hypothesis 3: Legislative assessments of GDOT’s performance in meeting Georgia’s transportation needs will be a function primarily of (a) perceptions of the quality of the agency’s performance in providing core services and products in their home districts (i.e., road maintenance, road safety, and traffic flow and congestion) and statewide (i.e., providing a variety of transportation options and preserving the environment in construction projects), and secondarily of (b) perceptions of the agency’s performance in serving the legislature as a whole, and (c) perceptions of the adequacy of the agency’s funding and the appropriateness of its spending.

Table 6 shows the results of two tests of this hypothesis, with both tests mostly supporting the hypothesis. Model 1 shows the basic test with the inclusion of all hypothesized variables, whereas Model 2 shows a modified test without two variables, perceived adequacy of funding and holding a legislative leadership role that had not approached statistical significance in Model 1. They were excluded to increase the number of cases on which the results are based.

In both models, legislator grades for how well GDOT served the General Assembly emerged as a primary factor in explaining perceptions of the department’s overall performance. On average, in the Model 2 results, a one-grade increase in that GDOT service would raise by 26.9 percentage points the probability of legislators assessing GDOT’s overall performance as A or B instead of C. Given the already documented largely personal basis for grading of GDOT service to the General Assembly, this new finding suggests a strong personal basis for grading GDOT’s overall performance too.⁵

At the same time, other aspects of GDOT’s perceived performance also appear to matter. GDOT’s grades for meeting the state’s transportation are also significantly influenced by perceptions of (a) road safety in the legislator’s home district, (b) the department’s performance in preserving the environment, and (c) the appropriateness of the department’s spending. With the first of those variables, for example, Model 2 shows that a one-grade increase in perceived road safety in the legislator’s home district would, on average, increase by 17.7 percentage points the probability of legislators assessing GDOT’s overall performance as A or B instead of C.

Finally, longer tenure in the legislature almost achieves statistical significance (p < .01) as another predictor of GDOT’s bottom-line performance. We might speculate that more time in the legislature equates to more positive interactions with GDOT, as evident in the mostly high levels of satisfaction with the GDOT working relationship, leading to more positive assessments of the department’s overall performance.

Conclusion

Legislatures are charged with overseeing the administrative branch of state governments, and, in that capacity, they have a substantial stake in administrative performance. Yet, we know little about how they assess that performance. The current research was designed as an exploratory first step toward answering that question through an examination of legislators’ assessments of the performance of the GDOT. Drawing from prior research on legislative–administrative relationships, we proposed and tested hypotheses about how legislators would assess (a) their personal working relationship with GDOT, (b) the work of GDOT in serving the Georgia legislature as an institution, and (c) GDOT’s performance in fulfilling its mission of meeting the state’s needs.

Limitations

The findings must be interpreted with caution because they come from only one administrative agency in one state. That agency, in this case, is also one that provides tangible, divisible products (e.g., roads and highways) likely to be attractive to individual legislators who wish to “claim credit” for bringing benefits to their districts (e.g., Mayhew, 2004; Ness, 2010, p. 37). The same description might not apply to many other state agencies, such as, for example, corrections or family and children’s services.

Still, given little prior research on this topic, an exploratory single-state study can be useful in paving a path for future research. As Nicholson-Crotty and Meier (2002) have observed, “For many research questions, a theoretically rigorous study of a single state is more appropriate than a less rigorous design that includes all 50 states” (p. 412).

Summary

The survey data showed Georgia’s legislators giving GDOT mostly positive assessments on all three dimensions of its performance. The department received its most positive assessments for handling requests and the related legislator satisfaction with GDOT working relationships. The assessments were only slightly less positive for GDOT’s performance in meeting the needs of the General Assembly, but more mixed when it came to the department’s bottom-line performance in meeting the state’s transportation needs.

Ordered logistic regression analysis mostly supported predictions that both personal experience with GDOT and perceptions of service quality would underlie the three assessments. Figure 2, a revised version of the earlier Figure 1, summarizes these results graphically, showing the apparent flow of influence that shapes the assessments.

Figure 2.

An empirical framework of legislator assessments of GDOT performance.

If there is a surprise in the findings, it may be the extent to which personal experience appears to influence all aspects of legislator assessments. There was no surprise, of course, in the influence of personal experience on satisfaction with the GDOT working relationship, but a similar pattern for grades for service to the General Assembly was not anticipated. Perhaps, though, that should not be wholly unexpected given the limited relationship most legislators appear to have with any particular state agency (e.g., Lee, 2006). Absent extensive interactions with an agency, legislators may fall back on their personal experience when assessing the agency’s service to the legislature.

It is not as easy to explain the apparently substantial role of personal experience in shaping legislator assessments of GDOT’s bottom-line performance in meeting the state’s transportation needs. Here, legislators might have been expected to draw primarily from their direct observation of GDOT products: Are roads well maintained? Does traffic flow well? Does the agency support a variety of transportation options? Legislators’ answers to those questions did, in fact, influence grades for GDOT’s bottom-line performance, but legislators’ assessments of their personal working experience with GDOT proved at least as influential. These Georgia legislators appeared to grade GDOT’s bottom-line performance substantially on the basis of what the agency had done for them and their districts.

Implications

We have known for some time that people in general evaluate the work of public agencies as much by how well they feel they are treated as by whether they receive the products they request (see Fountain, 2001, p. 4). Many might have doubted, though, that the same tendency would hold for public officials, especially those who are close to the development and provision of those products. By their charge as officials and their proximity to the products, these officials might be expected to assess an agency’s performance mostly on the basis of the perceived quality of the products.

That pattern does not appear to hold for Georgia’s legislators, for whom personal interactions with GDOT appeared to be more important than perceived product quality. Also, it is not the first time such a pattern has been observed with public officials. A similar pattern was earlier documented for both appointed and elected local government officials in Georgia (Thomas, Poister, & Ertas, 2010). With those local officials as with legislators here, personal experience with an agency exerted the strongest influence on all aspects of their assessment of the agency’s performance.

These findings, to the extent they can be generalized, imply a possible lesson for agency administrators. As many of them likely already know, they might be wise to accord a high priority to personal interactions with legislators if they want to remain in the good graces of those to whom they are ultimately accountable. It could be naïve to expect that an agency’s performance will be judged primarily on the basis of, as in the GDOT case, the quality and safety of roads and other tangible transportation outcomes.

The findings also provide grounds for questioning a principal or exclusive focus of public sector performance measurement systems on objective measures of service quality. Judging from the perceptions of Georgia state and local officials, those measures are important, but far from sufficient and maybe not even primary. That likelihood implies a need for performance measurement systems to include indicators of the quality of personal experiences stakeholders have with public agencies. The use of citizen satisfaction surveys may accomplish that goal for the public at large. An implication of the current research is that similar measures may be needed for other stakeholders, such as state legislators and local officials.

For the moment, these conclusions must be viewed as only suggestive because, as noted earlier, they reflect the experience of legislators in only one state and with only one administrative department. We need more research on how and why legislators in a variety of settings assess the work of the administrative departments they oversee.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors received financial support from the Georgia Department of Transportation for the research reported in this article.

Notes

Author Biographies

John Clayton Thomas is a professor of public Management and policy in the Andrew Young School of Policy Studies at Georgia State University. He has written four books and more than sixty articles in the areas of citizen-government relationships, performance measurement and program evaluation, and other aspects of public management.

Min Su is an assistant professor in the Public Administration Institute at Louisiana State University. Her research areas include public budgeting, public financial management, and local and urban governance.

Theodore H. Poister is a professor of public management in the Andrew Young School of Policy Studies at Georgia State University. He has published widely on strategic planning and performance management issues as well as stakeholder engagement in all the major public administration journals in the US.

References

Abramson

M. A.

Behn

R. D.

(2006). The varieties of CitiStat. Public Administration Review, 66, 332-340.

Behn

R. D.

(2002). The psychological barriers to performance management: Or why isn’t everyone jumping on the performance-management bandwagon? Public Performance & Management Review, 26, 5-25.

Behn

R. D.

(2003). Why measure performance? Different purposes require different measures. Public Administration Review, 63, 586-606.

Bendor

Moe

T. M.

(1985). An adaptive model of bureaucratic politics. The American Political Science Review, 79, 755-774.

Bernick

E. M.

Bernick

E. L.

(2008). Executive-legislative relations: Where you sit really does matter. Social Science Quarterly, 89, 969-986.

Berry

F. S.

Turcotte

J. W.

Latham

S. C.

(2002). Program evaluation in state legislatures: Professional services delivered in a complex, competitive environment. New Directions for Evaluation, 95, 73-88.

Bingham

L. B.

(2006). The new urban governance: Processes for engaging citizens and stakeholders. Review of Policy Research, 23, 815-826.

Bouckaert

Peters

B. G.

(2002). Performance measurement and management: The Achilles’ heel in administrative modernization. Public Performance & Management Review, 25, 359-362.

Bourdeaux

(2006). Do legislatures matter in budgetary reform? Public Budgeting & Finance, 26, 120-142.

10.

Bourdeaux

Chikoto

(2008). Legislative influences on performance management reform. Public Administration Review, 68, 253-265.

11.

Brudney

J. L.

Hebert

F. T.

(1987). State agencies and their environments: Examining the influence of important external actors. The Journal of Politics, 49, 186-206.

12.

Butler

D. M.

Karpowitz

C. F.

Pope

J. C.

(2012). A field experiment on legislators’ home styles: Service versus policy. The Journal of Politics, 74, 474-486.

13.

Fountain

J. E.

(2001). Paradoxes of public sector customer service. Governance: An International Journal of Policy, Administration, and Institutions, 14, 55-73.

14.

Gerrish

(2016). The impact of performance management on performance in public organizations: A meta-analysis. Public Administration Review, 76, 48-66.

15.

Goetz

A. R.

Dempsey

P. S.

Larson

(2002). Metropolitan planning organizations: Findings and recommendations for improving transportation planning. Publius, 32, 87-105.

16.

Hatry

H. P.

(2014). Transforming performance measurement for the 21st century. Washington, DC: The Urban Institute.

17.

Heinrich

C. J.

(2002). Outcomes-based performance management in the public sector: Implications for government accountability and effectiveness. Public Administration Review, 62, 712-725.

18.

Heinrich

C. J.

Marschke

(2010). Incentives and their dynamics in public sector performance management systems. Journal of Policy Analysis and Management, 29, 183-208.

19.

Jeon

C. M.

Amekudzi

A. A.

(2005). Addressing sustainability in transportation systems: Definitions, indicators, and metrics. Journal of Infrastructure Systems, 11, 31-50.

20.

Jeon

C. M.

Amekudzi

A. A.

Guensler

R. L.

(2013). Sustainability assessment at the transportation planning level: Performance measures and indexes. Transport Policy, 25, 10-21.

21.

Kroll

(2014). Why performance information use varies among public managers: Testing manager-related explanations. International Public Management Journal, 17, 174-201.

22.

Kroll

(2015). Drivers of performance information use: Systematic literature review and directions for future research. Public Performance & Management Review, 38, 459-486.

23.

Lee

(2006). Political-administrative relations in state government: A legislative perspective. International Journal of Public Administration, 29, 1021-1047.

24.

Lockwood

(2006, August). Changes in roles and partnerships: Towards the 21st century state DOT. Minneapolis, MN: State DOT Leadership Forum conducted at the University of Minnesota, Transportation Research Board.

25.

Long

J. S.

Freese

(2006). Regression models for categorical dependent variables using Stata. College Station, TX: Stata Press.

26.

Mayhew

D. R.

(2004). Congress: The electoral connection (2nd ed.). New Haven, CT: Yale University Press.

27.

McKelvey

R. D.

Zavoina

(1975). A statistical model for the analysis of ordinal level dependent variables. Journal of Mathematical Sociology, 4, 103-120.

28.

Moncrief

Thompson

J. A.

Cassie

(1996). Revisiting the state of U.S. state legislative research. Legislative Studies Quarterly, 21, 301-335.

29.

Moynihan

D. P.

(2008). The dynamics of performance management: Constructing information and reform. Washington, DC: Georgetown University Press.

30.

Moynihan

D. P.

(2013). Invited essay: Advancing the empirical study of performance management: What we learned from the Program Assessment Rating Tool. The American Review of Public Administration, 43, 499-517.

31.

Moynihan

D. P.

Kroll

(2016). Performance management routines that work? An early assessment of the GPRA modernization act. Public Administration Review, 76, 314-323.

32.

Moynihan

D. P.

Lavertu

(2012). Does involvement in performance management routines encourage performance information use? Evaluating GPRA and PART. Public Administration Review, 72, 592-602.

33.

Moynihan

D. P.

Pandey

S. K.

(2010). The big question for performance management: Why do managers use performance information? Journal of Public Administration Research and Theory, 20, 849-866.

34.

Ness

E. C.

(2010). The politics of determining merit aid eligibility criteria: An analysis of the policy process. The Journal of Higher Education, 81, 33-60.

35.

Neuman

T. R.

Schwartz

Clark

Benar

Forbes

(2002). A guide to best practices for achieving context sensitive solutions. Washington, DC: Transportation Research Board.

36.

Nicholson-Crotty

Meier

K. J.

(2002). Size doesn’t matter: In defense of single-state studies. State Politics & Policy Quarterly, 4, 411-422.

37.

Palus

C. K.

Yackee

S. W.

(2013). Oversight as constraint or catalyst? Explaining agency influence on state policy decision making. The American Review of Public Administration, 43, 273-291.

38.

Poister

T. H.

Thomas

J. C.

Berryman

A. F.

(2013). “Reaching out to stakeholders: The Georgia DOT 360 Degree Assessment Model.” Public Performance and Management Review, 37, 302-328.

39.

Sanger

M. B.

(2008). From measurement to management: Breaking through the barriers to state and local performance. Public Administration Review, 68(S1), S70-S85.

40.

Sarbaugh-Thompson

Strate

Leroux

Elling

Thompson

Elder

(2010). Legislators and administrators: Complex relationships complicated by term limits. Legislative Studies Quarterly, 35, 57-89.

41.

Schwartz

(2006, August). Building credibility with customers and stakeholers. Minneapolis, MN: State DOT Leadership Forum conducted at the University of Minnesota, Transportation Research Board.

42.

Squire

(2007). Measuring state legislative professionalism: The squire index revisited. State Politics & Policy Quarterly, 7, 211-227.

43.

Thomas

J. C.

Poister

T. P.

Ertas

(2010) Customer, partner, principal: Local government perspectives on state agency performance in Georgia. Journal of Public Administration Research and Theory, 20, 779-799.

44.

Yang

Holzer

(2006). The performance–trust link: Implications for performance measurement. Public Administration Review, 66, 114-126.