Apprehensions and Expectations of the Adoption of Systematically Planned,Outcome-Oriented Practice

Abstract

Objectives:

The study examined social workers’ apprehensions and expectations of the impending adoption of systematically planned, empirically based, outcome-oriented practice (SEOP).

Method:

Employing a mixed methods longitudinal design, the study used concept mapping to identify and map workers’ apprehensions and expectations and a self-reported questionnaire to measure their levels.

Results:

Apprehensions were that SEOP will increase monitoring and financial considerations in decision making and undermine professional intervention and professional responsibility. Expectations were that it will (a) empower social workers to provide better service and better cope with counterpressures and (b) systemize work procedures and improve organizational efficiency, effectiveness, and governance. Findings also showed that workers, who participated in formulating the system’s guidelines and principles, reported lower apprehensions and higher expectations than those who did not.

Conclusions:

In introducing SEOP, it is important to take workers’ apprehensions and expectations into consideration and to include workers in the development stage.

Keywords

outcome-oriented practice apprehensions expectations information system social service agency

Introduction

Recent decades have seen increasing calls for major changes in the way that social work is practiced. Scholars, questioning the effectiveness of traditional heuristic social work practice, have called for the adoption of systematically planned, empirically based, outcome-oriented practice (SEOP) by social service agencies. This is the practice in which the desired outcomes are defined at the beginning and interventions to attain them are planned in advance, on the basis of empirical evidence of their effectiveness and an explicit, clearly articulated theory of change (e.g., Gambrill, 2007; Gibbs & Gambrill, 2002; Jayaratne & Levy, 1979; Mullen, Bledsoe, & Bellamy, 2008; Rosen, 2003; Thyer, 2008). Governments and other funders have increasingly required that the social service agencies they fund systematically monitor and evaluate their practice (e.g., Burton & van den Broek, 2009; Clarkson, 2010; Leung, 2008; Shaw et al., 2009; U.S. Government Performance and Results Act 1993). The agencies, pressed by reduced funding for social programs, growing demands for accountability, and the desire to provide effective services, have increasingly complied and, to enable the change, have increasingly adopted information technology (IT).

Research consistently shows, however, that efforts to introduce SEOP and the technology that supports it into social work have been unsuccessful (Gray, Plath, & Webb, 2009; Mullen & Bacon, 2004; Rosen, 1994). As scholars have pointed out, the approach is inconsistent with practice in the field, whereby work processes are based on experience, common sense, tacit knowledge, and practice wisdom; implemented dynamically and rarely explicitly planned and articulated in advance (e.g., Hammond, Hamm, Grassia, & Pearson, 1997; Munro, 1999; Nevo & Slonim-Nevo, 2011; van de Luitgaarden, 2009). Research on the introduction of information systems, whether managerial, clinical, or integrated systems or performance measurement or decision support systems, consistently shows that they are underutilized and documents a litany of methodological problems (Ittner & Larcker, 2003; Kusek & Rist, 2004; Mayne, 2006, 2007; Perrin, 1998; Savaya, Monnickendam, & Waysman, 2000, 2006; Savaya, Spiro, Waysman, & Golan, 2004) and workers’ complaints regarding, among other matters, their unwieldiness, inflexibility, imprecision, and the lack of practical utility (Shaw et al., 2009). The outcomes are underuse or misuse of the systems that support the approach. Workers may fail to enter data or may enter partial, distorted, or inaccurate data (Bohte & Meier, 2000; Lyle & Graham, 2000; Nichols, 2005; Perrin, 1998; Smith, 1995). Those who do enter accurate data often choose not to use it in their clinical work and decision making (Monnickendam, Savaya, & Waysman, 2005; Savaya et al., 2006).

With this, the great potential inherent in systematically planned practice and the information systems that underpin it (Kusek & Rist, 2004; Mayne, 2007; Oakes, 1986; Savaya, 1998; Shavelson, McDonnell, & Oakes, 1991) mean that they are rapidly becoming imperatives in human service organizations. In consequence, efforts are constantly being made to design more relevant, user-friendly, and useful systems that workers will want to use to enhance their practice. A key way of attaining this goal, it is repeatedly claimed, is to include the workers in the development of the systems in order to make the systems more relevant to their needs and to foster a sense of ownership. The literature consistently claims that worker participation in the development of IT systems encourages more favorable attitudes toward them and greater likelihood of using them (Cavaye, 2000; Chen, 2001; Clarkson, 2010; Dennis & Garfield, 2003; Kujala, 2003; Kusek & Rist, 2004; Mayne, 2007; Perrin, 2002, 2006; Shaw et al., 2009).

This article tests the first part of the claim that worker participation in the development of systems that underpin SEOP will lead to more favorable attitudes. First, it identifies the apprehensions and expectations of workers prior to the development of a citywide managerial cum clinical information system in the Social Welfare Department (SWD) of the Tel Aviv–Yafo Municipality in Israel. Then, it compares the apprehensions and expectations of workers who did and did not participate in the development of the system.

Context

The study was conducted within the SWD of the Tel Aviv–Yafo Municipality. This is among Israel’s largest municipalities. Its SWD provides a large variety of services to close to 50,000 individuals from some 27,000 households. Several years ago, the senior management of the SWD decided to introduce planned, systematic, outcome-oriented practice, accompanied by ongoing monitoring of results. To implement the decision, it is developing a managerial cum clinical information system to be introduced into all its branches and used by all the social workers in its employ. In addition to the administrative data (e.g., number of intakes, clients, and treatment sessions) already being collected, the social workers will be asked to enter practice-oriented data relevant to clinical decision making, such as client assessment, intervention strategies, changes over time in client problems; quality and fidelity of interventions.

The plan is that the workers will assess their clients’ problems and carry out their interventions using guidelines especially developed for these purposes and enter the data into the system. The methodical assessment guidelines would enable workers to identify problems without overlooking matters and to intervene accordingly. A trial run of the guidelines for assessing the care given to elderly persons by their caregivers provides an example. A social worker who had been in weekly contact with the caregiver of a violent and abusive man suffering from dementia realized that although she had no difficulty in providing the requested information on how well the caregiver met the man’s physical needs, she could not answer the questions on her emotional situation. Her realization that she focused on instrumental matters and paid little attention to the emotional states of her elderly clients led her to plan her future interventions to include emotional issues as well as physical ones.

The timely data entry that the system requires could provide a foundation for evidence-based practice, both on the individual level and on the systemic level. On an individual level, it would enable workers to generate reports and to follow their clients’ progress, monitor their own work, and adapt their interventions in accord. In addition, because the collected data can provide information across clients, workers would be able to use their inputs to determine which interventions are most effective and in which circumstances and select their interventions in accord. On a systemic level, the aggregation of data across population groups could inform policy decisions, whether with respect to resource mobilization or other matters.

The system is being developed with an eye to avoiding the pitfalls that have undermined the use and impaired the utility of previous information systems. In particular, from the very beginning, considerable efforts were made to include both line workers and managers of all ranks in its development. One of the first activities to this end was to introduce the idea of planned systematic practice and the IT system that would be developed to facilitate it. A daylong colloquium was held with all the supervisors, branch managers, district managers, and others with managerial tasks. Among other things, the rationale, principles, and importance of the system were presented, as were the risks of misuse which have been identified in the literature. Following this, each of the nine branch mangers presented the same information to the social workers in their branches. At the end of the presentation, the workers gathered in small groups to discuss what they had heard and to express their apprehensions and expectations about adopting a system and approach that would require them to work in a planned, systematic manner. The written documentation of these discussions serves as the basis for this study.

Moreover, about a third of the workers were recruited into the development of the integrated information system. More specifically, workers from each of the branches were assigned to 12 working groups tasked with the job of developing assessment and intervention guidelines. One group developed the assessment guidelines. Eleven groups developed intervention guidelines, each for a specific population (e.g., substance abusers, children at risk, caregivers, etc.) served by the SWD. The assignments were made by the regional managers so that each group had a mixture of line workers, supervisors, and managers from the SWD’s nine branches and central administration. Participants were assigned to the teams rather arbitrarily on the basis of their availability. Each group had at least eight members and met for 3 hr at least twice a month for between 1½ and 2 years. Not all the workers in the SWD were included in the development process for logistic and other practical reasons. It was deemed important to keep the teams at a size that would enable coordination among the members, who came together from branches in different parts of the city about every 2 weeks, and enable them to work together productively and in an intimate atmosphere. Workers who were not in a working group were kept abreast of the progress of the groups and of the issues that arose in the process in formal and informal staff meetings.

To prepare the guidelines, the workers were instructed to take into consideration knowledge and information from a variety of sources: Their professional experience, the professional literature on effective social work interventions, and the laws and regulations published by the Ministry of Welfare and Social Services and by the Tel Aviv–Yafo Welfare Administration.

Method

This is a mixed method longitudinal study. It began prior to the start of the system’s development, when the workers were informed of the decision to develop it.

Data were collected at 2 times, each time in a different forum. First, the workers’ expectations and apprehensions were gathered in the discussion groups that were organized at the start of the project so as to allow all the workers in the service to voice their apprehensions and expectations. The discussions were attended by all the workers of all the branches, who were present on the days the discussions were held. The discussions were recorded so that their contents could be reviewed and used in the development and introduction of the system. Then, about 2 years later, toward the end of the development stage, the recorded statements were mapped using Trochim’s (1989) concept mapping method (explained below). The concept mapping served to identify and classify the social workers’ apprehensions and expectations. The identified statements then served as the basis for the construction of a questionnaire that was filled out both by the workers who had participated in the development process and by those who had not.

Since the study gave legitimization to the workers’ apprehensions and expectations, we did not view participation in it as carrying any risks. No pressure was exerted on workers to speak in the discussion groups. Workers who chose not to participate in the second part of the study could elect not to fill out the questionnaire. The distribution and completion of the questionnaire is described below, in the next to the last paragraph of the Method section.

Concept mapping (Trochim, 1989) is a multimethod procedure combining qualitative and quantitative techniques. It enables arranging ideas on any topic in a picture or a map and can be useful as a first stage in a variety of endeavors. It has applications in organizations, businesses, planning services, and many other areas. In the social sciences, concept mapping can be used to help articulate a theory, provide the basis for measurement, or serve as a framework for analyzing research results.

The study proceeded in two stages. The first was the creation and interpretation of a concept map. The second was the rating of the statements that went into making the map.

1. Creation, analysis, and interpretation of the concept map proceeded through the following steps.

Generation of Statements

We began by reading all the documentation to identify all the statements of apprehensions and expectations; after duplicates were removed, 53 statements were obtained.

Sorting

Fifteen social workers sorted the statements into piles that made sense to them. Fifteen is the number of sorters that Trochim (1993) found yielded a stable map.

Analysis

To analyze the data, each rater’s sorting was entered into “Concept System” computer program (Trochim, 1989). This program examines the proximity of each of the statements to one another across sorters by constructing a proximity matrix that is analyzed via nonmetric multidimensional scaling. As Trochim (1989, 1993) explains, this is a technique that takes a proximity matrix and represents it on any number of dimensions as distances between the original items in the matrix (for current purposes, the analysis is limited to two dimensions; see Davison, 1983; Kruskal & Wish, 1978). The next step is a hierarchical cluster analysis (Andeberg, 1973; Everitt, 1980)that is used to group individual statements on the map into clusters of statements, which presumably reflect similar concepts (for more details on concept mapping, see Trochim, 1989). Then, using hierarchical cluster analysis, the program groups the individual statements on the map into clusters, each representing a different concept. It then produces two visual maps: a dot map and a border map. The dot map shows the relative placement of each statement in relation to all the others. The border map presents the results of the cluster analysis. In essence, it is a dot map with a closed multisided boundary drawn around groups of dots to create clusters, in such a way that the statements within a cluster are those that were more often sorted in the same category than the statements in the other clusters. Furthermore, clusters that are closer to each other are more similar in content than clusters that are farther apart.

Interpretation

The map that emerged from the statistical procedure was interpreted by four social workers who had participated in the sorting. Each of the social workers met separately with the first author to examine the statements comprising each cluster, to identify its underlying concept, and to examine the location of each cluster on the map in relation to the others. We then considered various ways of reading the map and chose the one that made the most sense. Only four sorters were involved in interpreting the map because their interpretations were similar and no gain was expected in involving others.

Rating

The 53 statements were converted to questions and a questionnaire was constructed. (The questions will be evident from the section on Findings.) The questionnaire was distributed at staff meetings of the nine branches of the SWD. The social workers were asked to rate their agreement with each on a 5-point Likert-type scale, with 1 = strongly disagree, 2 = disagree, 3 = somewhat agree, 4 = agree, and 5= strongly agree. Reliability tests yielded Cronbach’s α from .69 to .73 for clusters with 3 or 4 items and from .76 to .90 for clusters with more than 4 items. For the 2 items in Cluster 6, we calculated the Pearson correlation (r = .27; p < .001). The reliability coefficients are presented in Table 1.

Table 1.

Cronbach’s αs or Pearson Correlation for the 10 Clusters.

Clusters	Cronbach’s α/r
Cluster 1: Increase monitoring	.73
Cluster 2: Increase role of financial considerations in decision making	.71
Cluster 3: Reduce social workers’ professional responsibility	.69
Cluster 4: Impede professional intervention	.90
Cluster 5: Empower social workers professionally	.76
Cluster 6 Promote value-based practice	r = .27; p < .001
Cluster 7: Lead to knowledge-based practice	.85
Cluster 8: Systematize work procedures	.90
Cluster 9: Increase organizational consistency and professional control	.81
Cluster 10: Increase organization’s efficiency and effectiveness	.77

Tests of validity were not performed because the ratio of number of respondents to the number of items was too small to enable performing confirmatory or exploratory factor analyses. We believe that the social workers’ sorting of the items argues for face validity.

The questionnaires were distributed in the branches. On being asked to fill out the questionnaires, the workers were once again told of the importance of their input into the development and implementation of the system. More specifically, they were told that it was important that senior management be aware of their feelings. The questionnaire was completed by 178 of the 229 social workers employed. The response rate was 77.7%. All who completed the questionnaires were on the premises when they were distributed. We cannot know how many of those who were on the premises chose not to fill them in. Of the sample, 61 (34.3%) had participated in the development of the program, 117 (65.7%) had not. This distribution is similar to that among the entire population of social workers in the nine branches (37.1% and 62.9%, respectively).

Ethical Approval

This study was carried out as part of a formative and summative evaluation that is examining the development, uses, and impacts of the system. It was approved by the Tel Aviv–Yafo municipality's Research Committee that too operates as an ethics committee.

Findings

The Concept Map

Border Map

Figure 1 presents the border map. As can be seen, the statements were grouped into 10 clusters or categories. To interpret the map that emerged from the statistical procedure, the four social workers assigned to the task first examined the statements comprising each cluster and identified their underlying concepts. Following this, they examined the location of each cluster on the map in relation to the others. What they saw was that the clusters were arrayed in an oval shape, with a single cluster in the middle, rather distant from the others, though fairly equidistant from two of the apprehension and two of the expectation clusters. The social workers then considered various ways of reading the map and chose the one that made the most sense to them.

Figure 1.

Cluster map.

This is a clockwise reading, starting with Cluster #1 going through Cluster #10 and containing Cluster #6 in the middle. The four clusters that occupy the left and top sides of the oval contain the items that express the workers’ apprehensions. The remaining six clusters, at the right, bottom, and center of the oval, contain the items that convey the workers’ expectations.

Overall, the apprehensions in the first four clusters (1, 2, 3, and 4) are that the approach will impair the social workers’ ability to do their work professionally. Clusters 1 and 2, on the left side of the map, convey apprehensions about the imposition of organizational supervision and control: Cluster 1 contains three statements referring to increased monitoring, namely, that the approach will increase the monitoring and criticism of workers and team leaders and turn team leaders into monitors. Cluster 2 contains three statements referring to financial resources that the approach will lead to reduced resources, decision making on the basis of financial rather than professional considerations, and difficulties in allocating manpower in accord with needs.

Clusters 4 and 3, at the top of the map, convey apprehensions about the impacts of the approach on the social workers’ relations with clients. Cluster 4 contains eight statements that refer to impaired professional intervention, namely, that the approach will make it more difficult to intervene with multiproblem families and with children at risk; decrease workers’ control in crises and in situations that require immediate response; will not provide a good solution to situations of individuals and families or suit changing needs; and will reduce workers’ flexibility and creativity. Cluster 3 contains four statements to the effect that the approach will undermine workers’ professional responsibility, namely, that it will turn social workers into clerks, reduce their personal responsibility, discourage them from taking responsibility, and make it difficult for them to form a therapeutic alliance with clients.

The six clusters that contain expectations of benefits (5, 6, 7, 8, 9, and 10) occupy the greater part of the map. In one way or other, they all refer to improvements that will lead to better practice and client service.

Clusters 5 and 7, at the oval’s right side, convey expectations of improved practice and client service through the impacts of the approach on the workers and (in Cluster 7) supervisors. Cluster 5 contains five statements referring to professional empowerment. These are that the approach will lead social workers to fully inform clients of the services to which they are entitled; increase social workers’ self-confidence vis-à-vis management, clients, and external bodies; and increase their professionalism and improve their professional image. Cluster 7 contains eight statements referring to knowledge-based practice, namely, that the approach will lead to planned interventions based on systematically collected knowledge and information rather than on feelings; to more structured and thorough assessment; to early identification of children at risk; and to better focused supervision.

Clusters 8, 9, and 10, at the bottom of the map, refer to expectations of improved practice and client service through improvements at the organizational level. Cluster 8 contains eight statements that refer to the benefits of systemization, namely, that the approach will result in better planning of daily work; improving documentation; encouraging evaluation of processes, implementation, outputs, and outcomes; and leading to setting priorities based on knowledge and information.

Cluster 9 contains seven statements referring to increased organizational efficiency and effectiveness, namely, that the approach will provide clear criteria for closing client files; enable more orderly and fuller transfer of client files among workers; and bolster organizational commitment to planning and monitoring, development of new community services and programs, identification of populations with unaddressed needs, and definition of the boundaries of intervention vis-à-vis other professions. Cluster 10 contains six statements referring to improved organizational governance, namely, that the approach will increase transparency, foster administrative uniformity, improve managerial oversight, bring workers’ shared problems to the surface, reduce the caseload over the long term, and bolster the ability to withstand irrelevant political pressures.

Cluster 6, in the center space of the map, contains two statements that were interpreted as referring to value-based practice, namely, that the approach will create greater equity in service delivery and foster social workers’ enthusiasm for their work.

Ratings

Table 2 presents the means (M) and standard deviations (SDs) on the 10 clusters for the entire sample. So as not to burden the reader, the table does not show the Ms and SDs on the individual statements.

Table 2.

Means and SD of the 10 Clusters (n = 178).

	Minimum	Maximum	M	SD
Apprehensions
Cluster 1: Increase monitoring	1.66	5	3.52	.74
Cluster 2: Increase role of financial considerations in decision making	1	5	2.73	.78
Cluster 3: Reduce social workers’ professional responsibility	1	4.5	2.69	.69
Cluster 4: Impede professional intervention	1	4.75	2.74	.74
Expectations
Cluster 5: Empower social workers professionally	1.20	4.80	3.38	.67
Cluster 6: Promote value-based practice	1	5	3.09	.69
Cluster 7: Lead to knowledge-based practice	1.33	4.83	3.64	.62
Cluster 8: Systematize work procedures	1	4.88	3.63	.58
Cluster 9: Increase organizational consistency and professional control	1.42	5	3.63	.57
Cluster 10: Increase organization’s efficiency and effectiveness	1.16	4.83	3.45	.60

Note. M = mean; SD = standard deviation. Scale range: 1–5.

As can be seen, almost all the Ms on the clusters that reflect apprehensions were lower than those on the clusters that reflect expectations. With the exception of Cluster 1 (increased monitoring), where the Ms was moderately high (M = 3.49, SD = .72), the levels of apprehension were rather low (from M = 2.68 to M = 2.72). Of the various items, the highest rated apprehension was that the approach would expose the workers to increased monitoring (M = 3.89, SD = .77). The lowest was that it would undermine workers’ taking responsibility (M = 2.33; SD = .89).

The highest expectations for benefits were noted for Cluster 7 (knowledge-based practice; M = 3.65, SD = .64); Cluster 8 (systematization of work procedures; M = 3.62, SD = .59); and Cluster 9 (increase organization’s efficiency and effectiveness; M = 3.59; SD = .59). In all of these, the Ms were moderately high. The highest rated statements were that the approach would provide clear criteria for closing client files (M = 4.02; SD = .74); improve managerial oversight (M = 3.96, SD = .78); and lead to decision making based on systematic collection of information (M = 3.90, SD = .76). The lowest rated statements were that the approach would create worker enthusiasm (M = 2.67, SD = .91) and enable better coping with political pressures and influences (M = 2.84, SD = 1.03).

Further analysis of the ratings was conducted in view of repeated claims in the literature that worker participation encourages more positive worker attitudes toward changes in work patterns and increases the likelihood that workers will adopt the changes. Specifically, we divided the sample into participants and nonparticipants in the development process. The division was made possible by the fact that the statements were collected from all the workers when the idea of the change first arose, but rated a year-and-a-half later, after about a third of the workers had participated in developing the guidelines for the approach, while the rest had not.

Table 3 presents the Ms, SDs, and t-test comparisons in each of the 10 clusters for the participants and nonparticipants in the development of the guidelines.

Table 3.

Means and SD of Participants and Nonparticipants in the Development Process (n = 178).

	Participation	M	SD	t	Sig.	Cohen’s d
Apprehensions
Cluster 1: Increase monitoring	Participated	3.35	.72	−2.32	.022	−0.36
Cluster 1: Increase monitoring	Did not participate	3.61	.73	−2.32	.022	−0.36
Cluster 2: Increase role of financial considerations in decision making	Participated	2.35	.70	−4.98	.000	−0.74
	Did not participate	2.94	.75	−4.98	.000	−0.74
Cluster 3: Reduce social workers’ professional responsibility	Participated	2.41	.68	−4.12	.000	−0.62
	Did not participate	2.85	.66	−4.12	.000	−0.62
Cluster 4: Impede professional intervention	Participated	2.43	.71	−4.17	.000	−0.63
Cluster 4: Impede professional intervention	Did not participate	2.90	.71	−4.17	.000	−0.63
Expectations
Cluster 5: Empower social workers professionally	Participated	3.62	.60	3.67	.000	0.56
Cluster 5: Empower social workers professionally	Did not participate	3.24	.67	3.67	.000	0.56
Cluster 6: Promote value-based practice	Participated	3.47	.60	5.71	.000	0.83
Cluster 6: Promote value-based practice	Did not participate	2.89	.67	5.71	.000	0.83
Cluster 7: Lead to knowledge-based practice	Participated	3.91	.47	4.21	.000	0.63
Cluster 7: Lead to knowledge-based practice	Did not participate	3.51	.65	4.21	.000	0.63
Cluster 8: Systematize work procedures	Participated	3.80	.45	3.07	.002	0.43
Cluster 8: Systematize work procedures	Did not participate	3.54	.62	3.07	.002	0.43
Cluster 9: Improve organizational governance	Participated	3.79	.52	2.84	.005	0.44
Cluster 9: Improve organizational governance	Did not participate	3.54	.58	2.84	.005	0.44
Cluster 10: Increase organizational efficiency and effectiveness	Participated	3.56	.55	1.64	.101	0.25
	Did not participate	3.40	.62	1.64	.101	0.25

Note. M = mean; SD = standard deviation. Scale range: 1–5.

As can be seen, in all but Cluster 10, which refers to organizational governance, there are statistically significant differences between the ratings of those who did and did not participate in the development of the approach. On average, the nonparticipants’ ratings of the statements in the four clusters that reflect apprehensions (1–4) are higher than those of the participants, while the participants’ ratings of the statements in the clusters that reflect the expectations of benefits are higher than those of the nonparticipants. In both cases, the difference is statistically significant.

Discussion

The field study reported above was a two-stage investigation. In the first stage, we conceptualized social workers’ apprehensions and expectations of the upcoming adoption of SEOP and the information system that underpins it. In the second stage, it measured the social workers’ ratings on the apprehensions and expectations that had been identified and compared those of workers who had and had not participated in the development of the assessment and intervention guidelines to be used in the new approach. As explained in the Introduction section, conceptualization, measurement, and comparison were made possible by the manner in which the new approach was developed.

As expected, the workers expressed both apprehensions and expectations of the new approach. Their apprehensions were that the approach will subject them and their supervisors to increased monitoring, augment the role of financial considerations in decision making, impede professional intervention, and undermine professional responsibility. These apprehensions are rather different from the main criticisms voiced by workers in studies carried out on their experience with IT systems that are already in place, which focus on the fragmentation, complexity, rigidity, and irrelevance of the forms they have to fill out and on the time and trouble it takes them to do so (Burton & van den Broek, 2009; Gillingham, 2009; Peckover, White, & Hall, 2008; Shaw et al., 2009). They correspond, however, to criticisms that scholars and practitioners have voiced of SEOP (e.g., Gray & McDonald, 2006; Gray et al., 2009; Harris, 1998; Webb, 2001, 2006). It is of interest to note that, unlike workers in other studies, the workers in this study did not voice concern that the introduction of the approach would increase their workload or take up inordinate amounts of time. To some extent, this difference may stem from the fact that the work environment of the SWD at the Tel Aviv–Yafo Municipality had been computerized for over a decade, so the social workers were accustomed to entering, retrieving, and making sense of the data.

The lack of concern with added burden may also reflect the orientation of the workers in this study. The apprehensions the workers expressed did not concern the impact of the new approach on themselves personally but rather how it would affect their ability to do their jobs for the benefit of their clients.

Much the same orientation is evident in the many expectations the workers expressed. In one way or another, virtually all the expectations refer to improvements that could lead to better practice and client service, whether through their impacts on the workers or on the organization. Two sets of expectations concerned the impacts of the approach on the workers. One was that it would empower them to provide better client service, better cope with counterpressures, and do a more professional job. The other was that it would lead to planned, knowledge-based practice, with more thorough and structured assessments, intervention based on systematically collected information rather than feelings, and better focused supervision. Three sets of expectations referred to the impact of the approach on the organization. The first was that it would systemize work procedures, resulting in improved planning, prioritizing, documentation, and evaluation. The second was that it would increase organizational efficiency and effectiveness, by enabling better handling of client files and bolstering organizational commitment to planning, monitoring, service development, and the identification of populations with unaddressed needs. The third was that it would improve organizational governance, manifested or resulting in greater consistency, transparency, and exposure of workers’ shared problems; and reduced caseload and better managerial oversight and ability to withstand irrelevant political pressures.

In contrast to workers’ reservations, the literature contains little or no information about workers’ expectations of the possible benefits of a systematic approach and the IT systems that underpin it. Thus, we have no way of knowing whether the expectations identified generalize beyond the study population. They are, however, consistent with the hopes that those who support systematic, evidence based, outcome-oriented practice have for this approach. The vast majority of the statements in the expectation clusters refer to benefits of the approach not to the social workers themselves, but to their practice and especially to the expectation that the approach will enhance their professionalism and client service. Moreover, although apprehensions and expectations pertaining to the practitioner and those pertaining to the organization tended to fall into different clusters and to occupy different parts of the map, there was almost no sense that the benefits to the organization would come at the expense of benefits to the workers or vice versa.

The ratings reveal generally positive attitudes. Both the participants and the nonparticipants in the development of the system rated the expectations higher than the apprehensions. All the expectations were rated moderately high (M = 3.07 to M = 3.65). All but one of the apprehensions were rated fairly low (M = 2.68 to M = 2.72; with the exception of monitoring, M = 3.49). In other words, neither those who participated in the development of the system nor those who did not were much concerned that the approach would adversely affect their practice. They did not believe that there was even a moderate chance that it would impede their professional intervention, undermine their professional responsibility, or impose financial considerations on their decision making. On the contrary, their expectations were that it would enhance their interventions, expand the services the SWD provided, and augment their professionalism and professional status. The only apprehension that was endorsed at a high level was that the approach would increase monitoring both on themselves and their superiors in the organization.

The findings also show consistent differences in the attitudes of the workers who did and did not participate in the development of the guidelines. The social workers who participated in the development of the guidelines expressed lower apprehensions and higher expectations (in all but one of the clusters) than those who did not. Since the workers’ attitudes were not measured prior to the start of the development process, we cannot claim with certainty that participation in the process was what caused the more positive attitudes. However, it should be noted that there is no reason to assume that those who participated in the development process had more positive attitudes to begin with. Those who participated in the teams did not volunteer for the job and were not permitted to opt out of it. They were chosen by the heads of the district branches, who were unfamiliar with their attitudes. In informal conversations, the branch heads said that they made their selection based on availability and so as to make sure that each team had a reasonable distribution of workers of different statuses (line workers, supervisors, and managers). For this reason, we believe that the findings lend empirical support to the repeated claims in the literature that worker participation leads to more favorable attitudes toward change.

At the same time, the ratings of the two groups were in the same direction. This is surprising because, in the light of the literature on unfavorable worker attitudes toward systematization of their work, we would expect stronger apprehensions and weaker expectations of benefits on the part of the nonparticipants. A possible explanation for the convergence of views is that all the workers in the SWD were effectively kept abreast of the project throughout the 3 years of its development. With one third of the workers in every branch directly involved in the development, those who were not involved could hardly be unaware of the ongoing activity or of the enthusiasm and absorption of their colleagues who were involved. Moreover, the radical change to an SEOP was a flagship project in the SWD and in the air in all the branches. It was discussed at staff meetings involving other matters, and several workshops attended by all the social workers were held to acquaint them with project.

Overall, the low apprehensions (except about monitoring) and moderately high expectations that the new approach will lead to improved practice bode well for its adoption by the workers and their utilization of the IT system that will enable it. Repeated findings in the IT literature show that workers who believe that a system will have practical value for their work tend to utilize it (e.g., Venkatesh, Morris, Davis, & Davis, 2003; Zhang & Gutierrez, 2007).

At the same time, the fairly high expectations can also disappoint. There are two key risks. One is that the workers will become frustrated with the hitches and drawbacks that are bound to reveal themselves in the system when it is put in place. If, as a result of this, they do not use the system fully or properly, they are unlikely to experience its benefits. The outcome may be a vicious circle in which underutilization leads to the perception that the approach is not helpful, which, in turn, leads to further underutilization (Savaya, 1998). The other risk is of monitoring. The workers were repeatedly assured that the system was being developed to help them improve their practice and not to monitor their performance. Nonetheless, the high apprehensions of monitoring even among the workers who participated in the construction of the guidelines suggest certain wariness as to whether the pact would be honored. Indeed, monitoring performance is an obvious temptation for management. If care is not taken to avoid both the actual monitoring and the suspicion of monitoring, workers may resort to gaming, that is to distorting the data so as to make themselves look good.

To avert these risks, the change will be introduced into the SWD in a 1-year pilot program. During this period, the workers will be required to apply the SOEP only to two cases and will be supervised on those cases using the principles of the new approach. Provision has been made for workers at all levels to provide feedback on the system, and they are being encouraged to enter any difficulties or issues they encounter and suggestions for improvement. Moreover, the assessment and intervention guidelines are open to modification or revision as experience with the system accumulates or regulations or conditions in the field change.

The study investigated workers’ attitudes prior to the introduction of the new approach and the accompanying technology. Its limitation is that its findings are local and contextual. To the knowledge of the authors, however, it is the first study to examine workers’ expectations as well as apprehensions. How much these will actually predict utilization remains to be seen in the follow-up studies. Further study is urged of workers’ expectations and apprehensions of the introduction of SEOP in other countries and circumstances.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Andeberg

M. R.

(1973). Cluster analysis for applications. New York, NY: Academic Press.

Bohte

Meier

K. J.

(2000). Goal displacement: Assessing the motivation for organizational cheating. Public Administration Review, 60, 173–182.

Burton

van den Broek

(2009). Accountable and countable: Information management systems and the bureaucratization of social work. British Journal of Social Work, 39, 1326–1342.

Cavaye

A. L. M.

(2000). User participation in system development revisited. Information & Management, 28, 311–323.

Chen

H. T.

(2001). Development of a national evaluation system to evaluate CDC-funded health. American Journal of Evaluation, 22, 55–70.

Clarkson

(2010). Performance measurement in adult social care: Looking backwards and forwards. British Journal of Social Work, 40, 170–187.

Davison

M. L.

(1983). Multidimensional scaling. New York, NY: Wiley.

Dennis

Garfield

(2003). The adoption and use of GSS in project teams: Toward more participative processes and outcomes. MIS quarterly, 27, 289–323.

Everitt

(1980). Cluster analysis (2nd ed.) New York, NY: Halstead Press, A Division of John Wiley & Sons.

10.

Gambrill

(2007). Transparency as the route to evidence-informed professional education. Research on Social Work Practice, 17, 553–560.

11.

Gibbs

Gambrill

(2002). Evidence-based practice: Counterarguments to objections. Research on Social Work Practice, 12, 452–476.

12.

Gillingham

(2009). The use of assessment tools in child protection: An ethnomethodological study (Doctoral dissertation), Faculty of Medicine, Dentistry and Health Sciences, The University of Melbourne, Melbourne, Australia.

13.

Gray

McDonald

(2006). Pursuing good practice? The limits of evidence-based practice. Journal of Social Work, 6, 7–20.

14.

Gray

Plath

Webb

S. A.

(2009). Evidence-based social work: A critical stance. New York, NY: Routledge.

15.

Hammond

K. R.

Hamm

R. M.

Grassia

Pearson

(1997). Direct comparison of the efficacy of intuitive and analytical cognition in expert judgment. In Goldstein

Hogarth

R. M.

(Eds), Research on Judgment and Decision Making (pp. 144–180). Cambridge, England: Cambridge University Press.

16.

Harris

(1998). Scientific management, bureau-professionalism, new managerialism: The labour process of state social work. British Journal of Social Work, 28, 839–862.

17.

Ittner

Larcker

(2003). Coming up short on nonfinancial performance Measurement. Harvard Business Review, 81, 88–95.

18.

Jayaratne

Levy

(1979). Empirical clinical practice. New York, NY: Columbia University Press.

19.

Kruskal

J. B.

Wish

(1978). Multidimensional scaling. Beverly Hills, CA: Sage.

20.

Kujala

(2003). User involvement: A review of the benefits and challenges. Behavior & Information Technology, 22, 1–16.

21.

Kusek

J. Z.

Rist

R. C.

(2004). Ten steps to a results-based monitoring and evaluation system: A handbook for development practitioners. Washington, DC: The World Bank.

22.

Lyle

C. G.

Graham

(2000). Looks can be deceiving: Using a risk assessment instrument to evaluate the outcomes of child protection services. Children and Youth Services Review, 22, 935–949.

23.

Mayne

(2006). Performance studies: The missing link? The Canadian Journal of Program Evaluation, 21, 201–208.

24.

Mayne

(2007). Challenges and lessons in implementing results-based management. Evaluation, 13, 87–109.

25.

Monnickendam

Savaya

Waysman

(2005). Thinking processes in social workers’ use of a clinical decision support system: A qualitative study. Social Work Research, 29, 21–30.

26.

Mullen

E. J.

Bacon

(2004). A survey of practitioner adoption and implementation of practice guidelines and evidence-based treatments. In Roberts

A. R.

Yeager

(Eds.), Evidence-based practice manual: Research and outcome measures in health and human services (pp. 210–218). New York, NY: Oxford University Press.

27.

Mullen

E. J.

Bledsone

S. E.

Bellamy

J. L.

(2008). Implementing evidence-based social work practice. Research on Social Work Practice, 18, 325–338.

28.

Munro

(1999). Common errors of reasoning in child protection work. Child Abuse and Neglect, 23, 745–758.

29.

Nevo

Slonim-Nevo

(2011). The myth of evidence-based practice: Towards evidence-informed practice. British Journal of Social Work, 41, 1176–1197.

30.

Nichols

S. L.

(2005). The inevitable corruption of indicators and educators through high stakes testing. Tempe, AZ: Education Policy Studies Laboratory, Arizona State University. Retrieved June 25, 2008, from http://epsl.asu.edu/epru/documents/EPSL-0503-101-EPRU.pdf

31.

Oakes

(1986). Educational indicators: A guide for policy makers. OPRE Occasional Paper Series. Office of Educational Research and Center for Policy Research in Education, Washington, DC.

32.

Peckover

White

Hall

(2008). Making and managing electronic children: e-Assessment in child welfare. Information, Communication and Society, 11, 375–394.

33.

Perrin

(1998). Effective use and misuse of performance measurement. American Journal of Evaluation, 19, 367–379.

34.

Perrin

(2002). Implementing the vision: Addressing challenges to results-focused management and budgeting. Paris, France: OECD.

35.

Perrin

(2006). Moving from outputs to outcomes: Practical advice from governments around the world. Managing for performance and results series. Washington, DC: IBM Center for the Business of Government and the World Bank.

36.

Rosen

(1994). Knowledge use in direct practice. Social Service Review, 68, 561–577.

37.

Rosen

(2003). Evidence-based social work practice: Challenges and promise. Social Work Research, 27, 197–207.

38.

Savaya

(1998). The potential and utilization of an integrated information system at a family and marriage counseling agency in Israel. Evaluation and Program Planning, 21, 11–20.

39.

Savaya

Monnickendam

Waysman

(2000). An assessment of the utilization of a computerized decision support system, for youth probation officers. Journal of Technology in Human Services, 17, 1–14.

40.

Savaya

Monnickendam

Waysman

(2006). Extent and type of worker utilization of an integrated information system in a human services agency. Evaluation and Program Planning, 29, 209–216.

41.

Savaya

Spiro

S. E.

Waysman

Golan

(2004). Issues in the development of a computerized integrated information system for a network of juvenile homes. Administration in Social Work, 28, 63–81.

42.

Shavelson

R. J.

McDonnell

Oakes

(1991). Steps in designing an indicator system. Practical Assessment, Research & Evaluation, 2. Retrieved March 15, 2009, from http://PAREonline.net/getvn.asp?v=2&n=12

43.

Shaw

Bell

Sinclair

Sloper

Mitchell

Dyson

Clayden

Rafferty

(2009). An exemplary scheme? An evaluation of the integrated children’s system. British Journal of Social Work , 39, 613–626.

44.

Smith

(1995). On the unintended consequences of publishing performance data in the public sector. International Journal of Public Administration, 18, 277–310.

45.

Thyer

B. A.

(2008). The quest for evidence-based practice? We are all positivists! Research on Social Work Practice, 18, 339–345.

46.

Trochim

W. M.

(1993). The reliability of concept mapping. Paper presented at the Annual Conference of the American Evaluation Association, Dallas, TX, November 6.

47.

Trochim

W. M.

(1989). An introduction to concept mapping for planning and evaluation. Evaluation and Program Planning, 12, 1–16.

48.

Tse Fong Leung

(2008). Accountability to welfare service users: Challenges and responses of service providers. British Journal of Social Work, 38, 531–545.

49.

U.S. Government Performance and Results Act 1993, http://www.whitehouse.gov/omb/mgmt-gpra/gplaw2m

50.

van de Luitgaarden

G. M. J.

(2009). Evidence-based practice in social work: Lessons from judgment and decision-making theory. British Journal of Social Work, 39, 243–260.

51.

Venkatesh

Morris

M. G.

Davis

G. B.

Davis

F. D.

(2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27, 425–478.

52.

Webb

S. A.

(2001). Some consideration on the validity of evidence-based practice in social work. British Journal of Social Work, 31, 57–79.

53.

Webb

S. A.

(2006). Social work in a risk society: Social and political perspectives. London, England: Palgrave Macmillan.

54.

Zhang

Gutierrez

(2007). Information technology acceptance in the social services sector context: An exploration. Social Work, 52, 221–231.