Incorporating agile practices in public sector IT management: A nudge toward adaptive governance

2.1 Agile methods and adaptive governance

Agile approaches are typically known for their specific uses in areas such as agile software development (Luna et al., 2014). They are used to overcome the challenges of traditional software development projects, such as the tendency to fail due to the emphasis on planning and the inability to respond to emerging demands (Holgeid & Thompson, 2011). The concept of agility has also received attention at the organizational level as the operational environment has become more unpredictable and the competition fiercer, leading to the adoption of the principles and values of agile software development for organizational governance processes (Luna et al., 2015). By following the principles of agile software development, organizations seek to improve their capability to respond to changing customer needs and to focus more on individual workers and the ways their knowledge can be fully utilized instead of relying on processes, tools, and contracts (Beck et al., 2001). The objective is to improve the capability of anticipating changes before they occur (Luna et al., 2014), especially in situations where the change is turbulent and abrupt (Folke et al., 2005).

Scholars have suggested that adaptive governance approaches can help the public sector cope with the current operational environment (Greve et al., 2020). The concept of adaptive governance emerged from environmental governance and was meant to help administrative systems balance change and stability (Greve et al., 2020), improving the ability to respond to rapid environmental changes (Chaffin et al., 2014; Janssen & van der Voort, 2016). While the concept was originally closely related to environmental politics (Chaffin et al., 2014; Janssen & van der Voort, 2016), its usefulness has been recognized for other administrative systems as well (Greve et al., 2020). Adaptive governance is linked to organizational resilience, especially in complex and uncertain situations (Olsson et al., 2004). In adaptive governance, top-down decisions are supported by bottom-up means, meaning that the skills and competencies of individuals can be used more effectively (Chaffin et al., 2014).

In recent literature, both agile methods and adaptive governance have been suggested as potential and similar solutions for the public sector organization’s currents struggles with the fast-changing operational environment (Greve et al., 2020; Janssen & van der Voort, 2016). This paper follows a similar approach by referring to the agile and adaptive government as the government’s ability to respond to changing public needs in an efficient manner (Mergel et al., 2020).

One of the main benefits of agile and adaptive governance is that it can enhance the organizational capability to react to the changing demands in the organizational environment (Luna et al., 2014; Richardson et al., 2014; Weingarth et al., 2018). Agile and adaptive approaches can also help public sector organizations to respond to citizens’ needs by enabling faster responses to the changes occurring in the operational environment (Mergel et al., 2020). By improving the ability to respond to changes, these approaches contribute to strengthening organizational resilience (Olsson et al., 2004). For example, in the context of the digital transformation, agility and adaptability can help organizations modify their attitude toward change, making change a norm instead of a threat (Allen & Gunderson, 2011) and helping the organizations deal with technological disruptions (Janssen & van der Voort, 2016).

Agility can also improve efficiency. By emphasizing the empowerment of bottom-up decision-making, the speed of organizational processes can be increased as the capabilities of individual employees are trusted and there is no need for management to supervise every single decision (Janssen & van der Voort, 2016). Additionally, the assumption that situations change over time encourages practitioners to revise their practices, which improves services and, in turn, efficiency (Folke et al., 2005). Agile approaches can improve the coordination between business units, improving the likelihood that the whole organization is working toward shared objectives (Luna et al., 2014).

There are also challenges associated with agile methods and adaptable governance. Hekkala et al. (2017) have shown that in traditional information system (IS) development teams, misunderstandings related to agile practices, unsuitable organizational structures for agile principles, and mismatches in values are persistent problems, which make the use of agile methods challenging. Typical issues also include insufficient capabilities, the lack of motivation, and the lack of an agile mindset (Fuchs & Hess, 2018).

When agility is scaled up, additional challenges, such as barriers to suitability, appear (Dikert et al., 2016; Fuchs & Hess, 2018). This is partly because agile methods were designed for small teams and, therefore, provide little guidance as to how they can be used in a broader context (Paasivaara et al., 2018). This knowledge gap can result in resistance to change, technical debt, lack of a common agile framework, problems with working in real-time, collaboration issues, challenges in defining the product owner, challenges in breaking down the requirements, backlog problems, and the inability to cope with constant changes (Paasivaara et al., 2018). Also, ensuring the commitment of multiple stakeholders can be challenging (Gerster et al., 2018a). These challenges should be acknowledged when agile practices are implemented.

2.2 Implementing agile practices in the public sector

When implementing agile and adaptive practices in the public sector, multiple changes in internal processes and organizational mentality – as well as in the relationship between governments and other social and political actors – are needed (Luna-Reyes & Gil-Garcia, 2014). Agile approaches were not designed for organizations that emphasize stability and pre-planning (Berger, 2007; Wisitpongphan & Khampachua, 2016), nor does the agile mentality fit well to bureaucratic, inflexible, and hierarchical government structures (Senyucel, 2007). As a result, when agile practices are implemented in the public sector, the characteristics of public sector organizations need to be considered.

When agile practices are implemented, empowering employees is often mentioned as a crucial initial step (Olsson et al., 2004). Employee empowerment is seen as a way to improve the use of internal and external capabilities as it decentralizes decision-making and enables stakeholders to self-organize and select suitable people among themselves to solve complex problems (Janssen & van der Voort, 2016). This way, the full competencies of the organization and the other stakeholders, such as citizens and consultants, can be used in difficult decision-making processes (Fowler & Higsmith, 2001; Mergel et al., 2020). However, for example, civil servants are accustomed to relying on established procedures, and giving them more freedom can result in a culture of avoiding responsibility as issues cannot be blamed on the standardized way of operation (Mergel et al., 2020). Consequently, when agile approaches are implemented in the public sector context, special attention should be given to transforming the operational culture, which requires, having a clear vision, good leadership, and trust among the different stakeholders so that individuals have a clear understanding of the objectives and feel they are supported while working toward these objectives (Olsson et al., 2004).

Second, it is important that public sector organizations shift the focus from contract negotiations to collaboration (Mergel et al., 2020), which involves an increased focus on service delivery and partnerships over traditional structures that prioritize internal efficiency, departmentalization, and hierarchical communication (Senyucel, 2007). In the public sector context, this will require careful consideration of laws and regulations because the emphasis on contracts and the existence of specific organizational structures can be dictated by laws and regulations, preventing public sector organizations from altering the existing way of operation (Savoldelli et al., 2014).

Third, when implementing agile practices, the focus should be switched from pre-planning to responding to change (Lappi & Aaltonen, 2017). This requires the capacity for monitoring and responding to environmental feedback (Olsson et al., 2004) as well as changing the traditional way of operating from comprehensive planning to an iterative approach that emphasizes continuous value delivery, end-user involvement, and constant face-to-face communication instead of following the original plan (Cao et al., 2013; Drury et al., 2012) For public sector organizations emphasizing pre-planning (Wisitpongphan & Khampachua, 2016) a shift in focus from planning to responding to change demands transformation of the core operational culture. Cultural transformation, however, takes time, which in the public sector can become a problem due to the short political cycles, as they make it difficult to establish long-term experiment-based development, whereby adjustments in practices are carried out over long periods (Allen & Gunderson, 2011).

The major shift in organizational culture also requires a process in which options are kept open, infrastructures flexible, and decision-making times short (Janssen & van der Voort, 2016). This can be seen to be in contradiction with the core values of public sector organizations, such as stability and accountability (Janssen et al., 2014). In general, the agile government does not necessarily contradict traditional public sector objectives because emphasizing stakeholder involvement corresponds with public sector organizations’ core value of citizen involvement (Mergel et al., 2020). For now, the combination of agility and stability in the public sector is poorly understood (Nograšek & Vintar, 2014), which makes the implementation of agile approaches in the public sector quite problematic (Kaczorowska, 2015) as there can be hidden tradeoffs between agility and stability (Janssen & van der Voort, 2016).

Overall, agile and adaptive governance requires significant changes in attitudes, capabilities, and processes which can result in challenges uncommon in private sector agile implementations. This does not necessarily involve a transformation of the entire governance approach. For example, Greve et al. (2020) argued that these new arrangements can be built upon existing structures – that is, rather than replacing existing reforms, agile and adaptive approaches should provide new institutional elements (Greve et al., 2020). All these actions should result in public sector organizations focusing on ensuring stability and accountability while enhancing adaptive and agile capacity (Janssen & van der Voort, 2016).

2.3 Agile methods in a municipal IT department

As public sector organizations increasingly face the pressure to provide more digital services the need for agility is also apparent at the lower organizational levels, such as the IT departments (Davison et al., 2005). For example, citizens’ demands for digital services at the municipal level are growing, meaning that IT departments need to improve their capabilities to respond to the needs of municipal business units and citizens’ demands in a more agile manner (Janssen & van der Voort, 2016). At the moment, agile methods have been used in public sector software projects but not in IT management in general (Lappi & Aaltonen, 2017; Virili & Sorrentino, 2009).

Particularly critical in the digital transformation is the way that IT departments cope with rapid changes. It is quite common for IT projects to experience delays and budget overruns (Benamati & Lederer, 2001; Shaul & Tauber, 2013). In recent decades, besides becoming more equipped to quickly respond to IT-related issues, IT departments have also had to promote digital innovation and business development (Matt et al., 2015) as well as drive the organizational digital transformation (Hansen et al., 2011; Leonhardt et al., 2017; Tanriverdi et al., 2010). This means that IT managers need to improve collaboration with businesses, with more resources being needed to manage the increasing number of ISs (Möller et al., 2011). In sum, IT departments need new practices, processes, and capabilities to effectively perform the various tasks expected from them (Leonhardt et al., 2017; Rai et al., 2012; Sambamurthy et al., 2003), all of which requires new attitudes toward IT development, a flexible IT infrastructure, and new agile working practices (Bhatt et al., 2010; Byrd & Turner, 2001; Fink & Neumann, 2009).

3.1 Case background

This qualitative interpretive case study (Walsham, 1995) was based on 44 interviews and 7 meeting observations conducted in an IT department of a large municipality in Finland. The population of the municipality has been on the rise, putting the municipality under pressure to improve its service offerings to cope with the increasing service requirements. At the time of the interviews, the municipality had approximately 210,000 inhabitants and 10,600 employees, while the IT department’s staff was fewer than 50 people, including both internal workers and consultants.

The study focused on the transformation of the centralized IT department, which served the municipality’s different business units. As the municipality had initiated a digital transformation strategy, new IT needs were emerging at an ever-increasing pace. In its state at the time, the IT department was not equipped to sustainably provide the new and expected digitalized services. In fact, the IT department was facing significant problems, especially with its IT development operations. These operations included the planning and execution of IS acquisitions, both of which were lagging, causing significant dissatisfaction throughout the municipality.

This case was selected because the municipality struggles with IT management were public knowledge despite its efforts to become a forerunner in the Finnish public sector’s digital transformation. The author also had the opportunity to gain information about the upcoming transformation efforts at the time of initial planning. Both of these factors made the case suitable for studying how agile practices are introduced in the public sector context and for analyzing the implications related to this transformation.

3.2 Data collection

The data were collected via semi-structured interviews in three rounds: the spring of 2017, the winter of 2017, and the autumn of 2018. The division into three interview rounds was made to gain a better view of the transformation during its initiation, execution, and finalization. The first round of the interviews focused on the issues that made the initial way of operating in the IT department unsustainable. The second round focused on the implemented changes and the issues that these changes had revealed. The third round again focused on evaluating the situation in the IT department after the transformation had been carried out.

The interviews were semi-structured because the interviewees had very different backgrounds and viewpoints. More structured interviews would not have resulted in in-depth discussions, while more open interviews would have made it challenging to stay on topic. Examples of themes and more specific questions can be found in Appendix A.

The data collection began by interviewing the key personnel involved in the transformation, including the chief information CIO, the head of account managers, the head of architecture, and the two consultants who were tasked with designing and leading the transformation. The other interviewees were then selected based on the suggestions of the previous interviewees, who were asked to suggest people who were involved in the transformation or people who could provide more details on the topics discussed during the interview – in short, the interviewees were selected using the snowballing tactic (Myers & Newman, 2007).

The second and third interview rounds began by contacting the key individuals identified during the previous round. Other than the account manager, all interviewees that had continued to participate in the transformation and were still working at the municipality, agreed to be interviewed again. The next interviewees were selected using the snowballing tactic (Myers & Newman, 2007). No clear differences were identified between the interviews where the interviewee was interviewed once versus multiple times.

Altogether, I interviewed 29 individuals (16 in the first, 10 in the second, and 18 in the third round) involved in the design and implementation of the digital transformation (see Table 1). The interviews lasted 47 minutes on average. While most of the interviewees were conducted on the municipality’s premises, two of the interviews were conducted via videoconference, three via phone, and one consultant interview was conducted on the consultant’s premises. All interviews were audio-recorded and transcribed.

During the first round of interviews, seven meetings related to the development of the new process were also attended. Moreover, enterprise architecture models that depicted the new processes were used as the municipality employed a publicly available modeling tool to which the researcher had access.

Data analysis began with a creation of general overview of the way that the IT development operations were conducted in the IT department. This was followed by the identification of the challenges encountered by the municipality before the transformation. The challenges were identified inductively, using a data-driven approach (Walsham, 2006). First, the quotations discussing the negative aspects of the operations were identified in the interview transcripts from the first interview round. These quotations were then iteratively grouped according to the challenge they represented. The initial categories of the challenges included, for example, the lack of shared practices, issues with collaboration, and the lack of resources. For instance, the category for the lack of shared practices was formed of quotations such as the following: “there was nothing, things just came from here and there” (Consultant A) and “most often, we made exceptions” (Account manager A). After analyzing the first round of the interviews, these challenges were presented to the key personnel from the municipal IT department to evaluate whether the findings were viable. The participants of the meeting agreed with the presented findings and were keen to reflect on how the transformation would solve the identified challenges. One reason for the fast acceptance of the findings was the timing of the meeting, which was placed at the beginning of the transformation. As the old way of operating was already deemed unsuitable, the challenges were seen as motivators for the transformation instead of problems pointed out by outsiders. The reflections on how the transformation would solve the identified challenges were used in the later analysis phases to reflect on whether the objectives of the transformation were obtained.

During and after the second and third rounds of data collection, the analysis focused on the new IT development process. First, a general description of the new IT development process was created based on the available enterprise architecture models and the interview data. Then, the analysis focused on identifying the challenges related to the new way of operating. This was initially done after the second round of the interviews and was iteratively repeated after the third round. The beginning of the analysis was similar to the identification of the initial challenges. When the categories from the first round of the interviews were applicable, they were used, and when the previous categories were not suitable, a new category was created – for example, the categories “Reluctance to change and a negative attitude toward constant change” and “Issues with following the new process” (see Table 3) were added to the existing categories.

Then, the analysis continued by identifying the advances made and the benefits gained due to the new way of operating. The identification of the advances followed a process similar to the identification of the negative aspects. However, the positive aspects were not limited to statements regarding improvements; instead, descriptions of the new process model were also compared to the negative aspects of the original process model to see whether the model had addressed these aspects. The challenges and the positive implications as well as interview quotations supporting them are presented in Tables 2 and 3 (see the Findings section).

4.1 The initial situation

The initial situation in the municipal IT department was very problematic. The IT department was struggling to respond to the growing service demands of the municipal business units. As IT development projects were proceeding slowly, the IT department also faced the issue of being unable to use its full budget. Overall, the IT department’s problem was that while the municipality wanted to increase the number of new IT related development initiatives, such as the introduction of new electronic services, the IT department was inefficient and unable to respond to the growing needs. This inefficiency included all the aspects of IT development: the identification of new IT related needs, development of the needs to solutions, prioritization of the different solutions, budget decisions, initiation of the actual development projects, and their finalization.

The difficult situation in the IT department resulted from multiple issues. First, no one in the IT department had a clear view of how the IT operations were run. Processes and responsibilities were not properly defined, and shared practices were missing. The siloed structure of the IT department prevented proper collaboration, even in cases when it was needed. There were also work-relationship problems among the employees, which made collaborating in projects difficult – for example, the relevant people were not necessarily invited to meetings, which would result in recriminations.

Both the employees of the IT department and the representatives of the municipal business units were struggling due to a poor flow of information. The people inside the IT department did not have a clear view of what the other employees were working on. The needs of the business units did not always reach the IT department, and when they did, there was no shared understanding of what the roles and the responsibilities of the business units and the IT department were. When the IT department began working on a new customer need, the actual customer was often left out of the loop as the IT department’s representative did not feel it was necessary to report on those aspects over which the business units had no influence. This situation was not clear to the business units, which made them feel that after they contacted the IT department, their projects stopped advancing. Consequently, the municipal business units were dissatisfied with the centralized IT department, making them try and carry out IT-related projects independently. These projects were often poorly planned and caused extra work for the IT department because the business units had insufficient IT knowledge.

Additionally, project portfolio management was a challenge as there was no holistic understanding of what IT projects were ongoing in the different sections of the municipality. Such lack of shared information slowed down financial decision-making and prevented the IT department from responding to the growing needs of the business units.

In general, the IT department was slow, unproductive, and unable to use its full budget, causing dissatisfaction throughout the organization. While the dissatisfaction inside and outside the IT department increased, managing the overall IT development (not to mention driving the organization’s digital transformation) seemed impossible.

4.2 The new agile approach

To solve the unstable situation in the IT department, the CIO, under pressure from the municipality’s deputy mayor, initiated a radical transformation of the IT department. The idea was that either the IT department must start to provide value to the municipality or the whole unit would be outsourced. To support the transformation efforts, the CIO invited a group of consultants to design an IT development solution that would allow the IT department to continue to provide services to the municipality. As the consultants did not have a readymade solution for the municipality, they iteratively redesigned a new IT development process for the IT department. Figure 1 is based on the enterprise architecture models of the official agile IT development process model. The solution consisted of introducing the agile approach to the management and operations of municipal IT development. After implementation, the new IT development process was iteratively developed based on the lessons learned.

OPEN IN VIEWER

The new IT development process consisted of the following three general phases: need identification, solution development, and solution implementation. Most of the efforts were focused on the first two phases. Improvements to the third phase were made through better front-end management of the IT development process.

The process began when a business unit had an IT-related need, which was then communicated to the solution office. These needs varied from extensions of existing systems to new information systems and electronic services. The IT department had a contact person appointed for each business unit who was responsible for facilitating the communication. Under the leadership of the contact person, a solution to address the need was then developed in the solution office.

The solution office was a “virtual” team of IT department employees and consultants. As the municipality’s human resource management did not allow the siloed team structures to be modified, the IT department bent the rules and regulations of the public sector organizations and created a virtual solution office on top of the existing structures. In the solution office, the IT department’s human resources were viewed as a competence pool from which suitable people could be selected to support the contact person in developing a solution for the presented need.

The virtual solution team worked in collaboration with the business unit to produce a realistic plan for approaching the presented development need. This was done by dividing the project into two-week sprints for developing the idea. Besides the solution specialists, enterprise architects were also involved to ensure that the created solution would fit within the organizational enterprise architecture. The main objective of the solution office was to implement an agile but systematic approach to managing the different development needs presented by the municipal business units – that is, the team was to take charge of the front end of IT development.

Typically, multiple solution options were presented in a prioritized manner for a business unit to choose from. Depending on the case the solutions were for example a list of potential systems to acquire or a suggestion to extend the use of an existing system already in use in other parts of the municipality. In rare cases, the solution office could also suggest a rejection of the case. This was especially the case when upcoming changes to the municipal IT infrastructure could either solve the issue or permit its development. Of course, the IT department could only suggest a rejection as the municipal business units were independent operators and IT development projects were guided by political interest as well. After the solution was presented to the business unit representatives, they could suggest its addition to the IT department’s investment list.

The IT department also introduced digitalization plans to improve the manageability of the front end of the new IT development process. Digitalization plans were four-year plans in which the main development needs of the business units were laid out along with definite development timelines. For the creation of each digitalization plan, a coordinated group selected from the competence pool as well as a group of business representatives met in a series of workshops to identify the current IT-related issues and problems and envision the future. After both the IT-department and the business-unit representatives were content, the plan was presented to the board of the respective business unit. For the digitalization plan to become official, the business unit’s board had to commit to the plan. The purpose of the digitalization plans was to help the IT department to become more proactive in IT development and to improve the view of future IT development projects.

The other day-to-day tasks in the IT department were also redesigned to follow the agile guidelines. For example, all the teams in the IT department introduced daily Scrum meetings adapted from the Scrum framework. Depending on the type of task, these meetings were held either daily or twice a week. In the meetings, each team member reported what they had done since the last meeting, what they were working on now, and if they needed any help. The agenda of the meetings was restricted to avoid prolonged meetings. Different projects and their phases were presented on a Kanban board. The tasks on the board were divided in a way that allowed them to be completed in two-week cycles, thus enabling a continuous workflow. The advancement of the projects was also constantly measured to make sure that tasks were not left on the Kanban for too long. Also, a more visible approach to the backlog of future projects was created to enable the IT department to transfer funds and resources between the current and the upcoming projects when issues preventing the advancement of ongoing projects came up. When necessary, corrective actions were taken as soon as an issue was detected. Additionally, lean approaches, such as lean enterprise architecture, were introduced as part of the operations in the IT department.

The implementation of the new agile approach in IT development resulted in multiple improvements. The various improvements are detailed in Table 2. The new agile approach enabled the IT department to gain an improved understanding of the current operational situation in the municipality. For the first time, the processes and responsibilities were clearly defined. Ongoing tasks were visualized on a Kanban board, and the backlog was updated according to real development plans. A better understanding of the overall situation also enabled a more efficient implementation of new improvements (see Table 2).

The Kanban board also helped the IT department to gain a better overview of the operational situation. This way, the new IT development process improved the visibility of the actions taken. Tasks were now visibly advancing, first on a physical board and later on the virtual Kanban. There was also a proper record of the finalized projects, which even the CIO considered a significant advancement (see Table 2). The better visibility of the overall development situation was increased by the digitalization plans as the focus was not only on the past and the current situations but extended four years into the future.

The digitalization plans improved the planning of the future. For the first time, the business units had a clear view of developments beyond the yearly budget. With the new process and the improved overview in place, the IT department was able to manage the investment list more efficiently, and its money allocation became more fluent.

As agile approaches emphasize collaboration, especially with customers, there were also efforts made to improve such collaboration. By clearly allocating the responsibilities and specifying how customers, namely the business units and their representatives, should be involved in IT development projects, collaboration was improved. The IT department employees had a better understanding of the ways that the customers were to be involved, and the representatives were more equipped to do so. Because the new virtual-team approach effectively dismantled the previous siloed team structure, the collaboration inside the IT department also improved.

These changes helped the IT department to improve its customer service. As noted in Table 2, the overview of the situation and a better understanding of the future helped the IT department to make better and also faster investment decisions, which, along with the enhanced collaboration between the IT department and the business units, led to improved customer satisfaction. Overall, the new agile approach resulted in several operational advancements.

While the attitude toward the new development was, in general, positive, it was clear that the transformation was facing challenges. For example, the employees in the IT department were becoming tired of the continuous changes, which resulted in negative attitudes toward constant changes (see Table 3). The consultants responsible for designing the new agile IT development process had a clear vision of how agile approaches could be implemented through incremental changes. However, this vision was not clear to the employees of the IT department. The new IT development process required the IT department’s employees to continuously learn new practices, something they were not accustomed to doing. There was also a reluctance to change among the municipal business units. For example, the business units were accustomed to using their budgets as they wished. Consequently, the IT department’s more proactive involvement in defining the potential IT development projects and especially the IT department’s power to suggest a rejection of projects were not positively received changes as they threatened the independence of the business units.

Additionally, insufficient time was allocated to clarifying the new IT development process to the employees. The new process was presented via complicated enterprise architecture models, which, although providing a detailed and holistic description of the changes, did not properly clarify how the different parts of the process would work in practice. As a result, there were issues with following the new process. As the process was designed by the consultants, it did not build upon the existing structures, and while the new agile mindset enabled innovative approaches, many ordinary aspects of the IT development process were not clarified. For some employees, this was an opportunity, while others returned to their old operating models. Employees were flustered that there was no higher-level decision-making body to provide clear rules on how things should be done and for example, whether previous performance evaluations were still valid. Consequently, it was safer for the internal employees to commit themselves to the original tasks. This encouraged a behavior where the employees intentionally focused on advancing IT development projects of the business unit, they were originally signed for, instead of following the new process and competence allocation.

New types of financial challenges appeared as well. As money allocation for the different projects became more flexible, project managers were given more responsibility in defining project budgets, which resulted in new challenges. For example, project managers created oversized budgets as they did not trust that the new flexible money allocation process would benefit them, tying down a lot of funds that could have been used for other development projects. This issue was emphasized because the business units no longer had specific funds allocated to their needs. Additionally, with the improved collaboration, more development needs were brought forward while the overall budget of the IT department remained the same. This increased the confrontation among the business units.

Meanwhile, as the municipality was struggling with insufficient human resources and hiring new employees in the public sector was difficult, the IT department had to rely extensively on consultants. The consultants became a critical factor in keeping the new agile process operational, which resulted in unsustainable dependence on consultants. The situation was also aggravated by the fact that many internal employees left the IT department due to the transformation, with the outgoing employees being mainly replaced with consultants.

5.1 The agile IT development and its impacts on the IT department

Through the implementation of the new agile IT development process, the IT department successfully incorporated multiple agile values into its operations. When compared to critical steps in agile implementation suggested by previous literature, the IT department managed to increase the empowerment of its employees (Olsson et al., 2004), shift the focus from contract negotiations to collaboration (Mergel et al., 2020), and focus on responding to change instead of relying on pre-planning (Lappi & Aaltonen, 2017).

The empowerment of employees: The establishment of the solution office and the introduction of the virtual team enabled the IT department’s employees and consultants to collaborate more efficiently on different solution-development projects. As the people were no longer just representatives of siloed teams focused on specific tasks but representatives of a variety of competencies, their knowledge and know-how could be utilized more effectively (Janssen & van der Voort, 2016; Mergel et al., 2020; Olsson et al., 2004).

The organizational rules and structures, however, limited the empowerment efforts. First, as the municipality’s human resource management rules did not allow the removal of existing organizational structures, the IT department had to bend the rules. Consequently, the empowerment of employees was based on an unofficial team structure, i.e. the virtual team, which did allow collaboration and self-organization but was still bound by the detailed contracts of employment. Second, as the IT department was suffering from a lack of resources and competencies, it had to heavily rely on consultants. Hiring new employees in the public sector is often slow and laborious. Therefore, the consultants seemed like a good option for the IT department to fulfill the gap in resources revealed by the new agile IT development process. Especially as the newly established virtual team enabled their fluent integration to ongoing operations. Unfortunately, the extensive reliance on consultants enforced change resistance. As the consultants were the ones who had designed the new agile way of operating, they were also the ones with a clear vision of the objectives of the transformation. This hid the fact that the vision was not shared among the IT department’s employees, which made them tired and frustrated by the continuous change.

The empowerment of employees was also hindered due to the challenges of following the new process. Scholars have typically explained these challenges by pointing to unsuitable organizational structures, mismatches in values (Hekkala et al., 2017), insufficient capabilities, and the lack of motivation and agile mindset (Fuchs & Hess, 2018). In the IT department’s case, the new process was largely run by consultants, who were not bound by the previous way of operating and were familiar with agile approaches. Consequently, the challenges can be explained by unsuitable organizational structures, which encouraged the IT departments employees to work independently to advance their own projects as this enabled them to benefit the business unit, they were structurally responsible.

A shift from contract negotiations to collaboration: The shift from contract negotiations to collaboration focused especially on the relationship with the IT department and the municipal business units. It was largely obtained through the proactive management of the front-end of the new agile IT development process, i.e. the need identification phase. For instance, the communication and collaboration with the business units were improved due to the dedicated contact person and the active involvement of the business units in solution development. Also, the close involvement of the business units in the creation of digitalization plans was critical in encouraging the business unit to see the IT department as a collaboration partner. Together, these improvements enhanced the collaboration with the customers as suggested by Fowler and Higsmith (2001). In line with the findings of Janssen et al. (2013), the focus on collaboration also enabled the IT department to successfully increase its understanding of the different stakeholders, which again improved collaboration and enhanced the IT department’s ability to provide services and drive the digital transformation.

While the creation of the digitalization plans was done in collaboration with the business units and the business units had to commit to them, these agreements were not official ones. They were not bound by the official rules of the municipality, instead, it was based on trust. Therefore, in case, where the IT department is not able to fulfill the digitalization plans the business units will most likely demand a return to more contract-based operations.

Responding to change instead of relying on plans: As the new agile process and especially the digitalization plans improved the understanding of the operational situation in the municipality, the IT department was able to move towards a more proactive approach. As suggested by the previous literature, the improved understanding of operations made the IT department more equipped to monitor and respond to environmental feedback (Olsson et al., 2004). The improvements in efficiency and planning also contributed to the IT department becoming more flexible with money allocation. Consequently, instead of following yearly budget plans, the IT department was now able to respond to changes more agilely. For example, in cases where an IT development project was temporarily halted, the funds could be moved to new projects. Additionally, in the solution office, the two-week development cycle improved flexibility, creating an opportunity to re-evaluate the situation every two weeks. This was especially beneficial when similar needs arose in other business units during the solution development. Consequently, the solution office could consider changes in the municipality during the solution development process.

However, not all the outcomes were brought about by the agile approach, with some benefits being achieved merely by clarifying the IT department’s processes, roles, and responsibilities. For example, the improved understanding of the operational situation and the improved visibility of the undertaken actions were part of the proper definition of the processes and responsibilities. This was also the case with the improved planning of the future as the systematized mapping of business units’ future needs helped the IT department plan future developments more effectively.

The introduction of agile practices in the IT department resulted in multiple positive impacts typically identified in the private sector context (Weingarth et al., 2018). The challenges on the other hand, while some closely related to general agile challenges such as reluctant to change and fatigue towards continuous change (Paasivaara et al., 2018), were more context-dependent and party related to the fact that rules and regulations constraining public sector organizations prevented straightforward adoption of agile values such as empowerment of employees. For example, the IT department had to bend the rules related to human resource management. The most obvious example of this was the creation of the virtual team, as the rules and regulations did now allow the transformation of existing organizational structures. Also, the extensive reliance on the consultants could be connected to the limitations of public sector human resource management, which prevents public sector organizations to make fast recruitment decisions. Consequently, while the rules and regulations have been identified as challenges of transformation in the context of the public sector (Savoldelli et al., 2014), the significance of human resource management should be emphasized in the future. Especially as, the case shows, the attempt to bypass rules, for example by using consultants can itself result in undesirable consequences.

5.2 The impacts of the agile IT development on the municipality

The transformation of the IT department was not limited to the IT department and required changes in other parts of the municipality as well. The creation of the solution office demanded more commitment from the business units as they could no longer conduct independent IT development projects, meaning that the business units needed to change their attitude toward IT development but also give up some of their independence. The digitalization plans required further commitment from the business units. As the prioritization of future development projects no longer belonged to the IT department’s steering group but the boards of the business units, the business units had to take responsibility for the disappointments related to prioritization.

The faster and more flexible money allocation also meant that the business units needed to change their attitudes toward each other. For example, in the case of a delay in one project, the new IT development process allowed transferring that project’s finances to another project. However, the other project did not necessarily provide value to the same business unit, pressuring the business units to change their attitudes toward each other. They could no longer see themselves as separate and independent entities unconnected with other parts of the municipality. For the new IT development process to work as intended, the business units, instead of seeing themselves as fighting for the same services, had to work collaboratively. As the new financial challenges indicated, this part of the transition was not successful. The operational culture of the municipality limited the way agile approaches could be incorporated. The lack of trust among different operators resulted in challenges in financing new IT development projects.

Consequently, the introduction of agile approaches can enhance digital transformation as indicated in the previous literature (Vial, 2019). However, while previous studies have emphasized the challenges, such as the contradiction between stability and agility (Janssen & van der Voort, 2016), the case analyzed in this paper shows that at the municipal level, issues are closely linked to a deeply rooted operational culture, whereby strict operational rules are valued more than trust and individual projects, as well as business units, fight each other for scarce resources. Consequently, while the case validates some concerns related to the use of agile in the public sector, it also shows that if there is a will to adopt agile practices in the public sector, there is also a way to do it.

5.3 Propositions for introducing agile and adaptive governance approaches in the public sector

The findings show that the introduction of agile and adaptive government to the public sector through small-scale experiments can have multiple benefits. However, the introduction of agile approaches can require bending rules such as human resource management regulations. When these experiments are extended throughout the organization also factors deeply rooted in the organizational structures such as lack of trust can make it challenging to fully benefit from the potential of agile and adaptive governance. As the implementation of agile practices is, however, often necessary, the case indicates that:

As it is often easier to implement a small-scale change than a large-scale one (Châlons & Dufft, 2016), initiating bottom-up agile transformations can create quick successes and drive the organizational-level transformation. Consequently, public sector organizations should intentionally promote these small-scale implementations. As the case study has shown, the IT departments are suitable starting points due to their potential familiarity with agile methods via previous IT projects. However, more research is needed on small-scale transformations, both inside and outside of IT departments. Additional research can provide a better understanding of the positive and negative implications of small-scale transformations and explain how such transformations influence the other parts of the organization, seeing that through collaboration, small-scale transformations can spread to the rest of the organization. This indicates that:

For agile approaches to provide holistic benefits, they need to be extended beyond individual organizational units. In the case of the studied IT department, this was done through close collaboration with the municipal business units, which enabled the agile methods to spread to other units and created positive interest. While challenges such as lack of trust emerged when the collaboration was initiated, the close collaboration can also aid in building trust and this way advancing the transition towards adaptive governance. More research about how this happens in practice is needed, especially as the current study does not consider how the agile approaches learned from the IT department were utilized in the internal processes of the municipal business units.

The implementation of agile approaches can also prompt new tensions between agility and the traditional structures of the public sector. For example, while the consultants had no issues in following the new process model, the employees struggled due to the absence of a higher-level decision-making body. Similarly, the new issues related to financing and consultant reliance were connected to the restrictive nature of public sector structures. Such challenges cannot be solved by simply increasing agility. Instead, they require changes in the organizational assumptions and attitudes as well. IT departments cannot accomplish this on their own, which indicates that:

When agile approaches spread throughout the organization, managing the emerging tensions should not be the responsibility of individual employees; instead, higher-level management is needed. Therefore, higher-level managers and politicians must pay more attention to such transformations. Future research should demonstrate the concrete benefits of large-scale agile initiatives. Also, practical examples of larger-scale implementations are needed because concrete benefits can motivate public officials to transform their traditional ways of thinking and acting.