Development method in a large-scale public sector IT programme

Making choices on the development method

How do organizations decide on which development method to use? Fitzgerald et al. (2006) describes two main approaches: The contingency factor approach and method engineering.

The contingency factor approach involves selecting a development method from a range of methods based on mapping the specific features of the development project to features of the development method. Typical challenges involved with this approach include the problem that organizations are not intimately familiar with many development methods, and it is also a considerable cost in training employees in a new method (Fitzgerald et al., 2006). Many suggestions as to which features to focus on are available. Boehm and Turner (2003) advise using several features to balance agility and discipline:

• Personnel (many with high competence favouring an agile approach)

Method engineering focuses on adaptation or ‘tailoring’ of development methods. This concept is sometimes referred to as ‘situational method engineering’ which focuses on adapting a standard method to the specific context of the development effort. A study of customization of agile methods at Intel Shannon shows how this organization selected six of twelve techniques from XP and combined these techniques with local Scrum tailoring. Scrum tailoring involved two stages in planning: (1) at the beginning of an iteration (‘Sprint’) and (2) at the beginning of the project. These steps led to benefits, such as reduction in code defects and projects delivered ahead of schedule.

Does method matter? Method in the small and the large

Does the method matter? What is the relationship between methods as described in literature or internal documents and the ‘method-in-use’ in practice? A recent study found that about 40% of the performed design activities are rooted in development method (Thummadi and Lyytinen, 2020). The remaining activities are chosen as the method does not describe the specific area, does not fit, and are chosen due to skills and habits or ‘organizational noise’. Software development ‘is continually socially constructed, and, accordingly, local abstractions are influenced by multiple, often contradictory and incomplete extrasomatic sources’ (Thummadi and Lyytinen, 2020).

In recent years, most environments that are involved in software development have been using agile approaches for such development. These methods were originally thought to fit small co-located development projects and develop software that is not life-critical (Williams and Cockburn, 2003). The most popular method is Scrum. The Scrum method recommends development of a product in iterations with a description of what is to be developed in a product backlog managed by a product owner and a team facilitated by a Scrum master. The product owner, the Scrum master and the team plan each iteration as a sprint backlog. The team demonstrate the product at the end of the iteration, and conduct a retrospective meeting to learn and adjust the development process. What is to be developed is typically described in overall epics, which are then decomposed into ‘user stories’ and then further broken down into tasks in the iteration planning. Scrum can be used for larger projects that involve several teams with joint coordination of work at Scrum-of-Scrum meetings, typically with one participant from each team.

From the mid-2000s, many organizations started using agile approaches, not only in single teams, but also for larger projects and programmes (Hoda et al., 2018), which typically consists of many development teams that work together for a long period of time. Practitioner environments have suggested several methods, such as Disciplined Agile Delivery (DAD), ⁸ Large-Scale Scrum (LeSS), ⁹ Scaled Agile Framework (SAFe) ¹⁰ and the Spotify model, ¹¹ for large-scale agile development (Dingsøyr et al., 2019). These methods prescribe agile methods at the team level combined with new roles, artefacts, and work processes to coordinate the work of many teams. LeSS, for example, introduces a joint iteration planning for all teams before teams plan their own work. LeSS comes in two versions, one for 2-8 teams and the ‘Huge’ for more than eight teams. LeSS Huge introduces additional roles, such as the ‘area product owners’ responsible for a requirements area for the product under development.

Researchers have investigated the use, efficiency, and challenges of these methods. A study found that developers often misunderstood the concepts and routines described in the methods, and it was problematic to choose between different methods (Conboy and Carrol, 2019). Some developers felt that decisions were made ad-hoc, some development teams were frustrated by frequent changes in methods, and some teams did not change their protocol after a new method was introduced. The researchers found that while agile methods at local levels allowed flexibility, the new large-scale methods came with predefines structures, routines, and tools, which were much more dominant, and that ‘small realignments can cause significant disruptions across units of an organization’ (Conboy and Carroll, 2019).

Programme organization

The programme was planned with three main releases consisting of 50,000 to 75,000 h of estimated work, which were named ‘the baseline’, ‘the settler’, and ‘the digital’. The releases were planned based on the timeframe shown in Figure 1.OPEN IN VIEWER

The baseline release was a digital application processing system that automatically processed applications for one-time benefits. The settler release expanded the application processing system to include all types of parental benefits and integration with employers’ wage systems. This release aimed to develop a complete decision-making system adapted to the requirements of calculation in the law. The aim of the last release (the digital) was to create a self-service solution integrated with an extended application processing system and to support integration with health actors. The goals for the release included creating a complete integration between a planning calendar and a dialogue about benefit applications with users and conducting a digital dialogue between the user and the application caseworker. The previous release created a minimum viable product that was to be further developed. The main difference of this last release when compared with the first two was the programme was to extend a solution that was in use and add new functionality in a domain that was less well explored.

The programme had 16 roles, such as a construction manager, controller, and customer manager, at the programme level (Table 1). The senior solution architect at programme level would work with the application architects at team level. Similarly, the quality assurance and test roles at programme level would work with the testers at team level. The work was organized into four main projects: (1) business, (2) development, (3) test, and (4) change management (Figure 2). The three releases had four phases (Figure 3): (1) analysis of needs, (2) solution description, (3) construction, and (4) test.

OPEN IN VIEWER

Table 1.

Roles at the programme level and roles on development teams during the first phase (from Dingsøyr et al., 2023).

Roles on programme level	Roles in development teams
Construction manager (1), controller (1), customer manager (1), environmental manager (1), functional architects (7), functionality responsible (1), performance test manager (1), project manager (1), project support (1), PMO (1), quality assurance (1), solution manager (1), senior solution architect (1), test automation (1), test data responsible (1), test manager (1)	Scrum master (1), application architect (1 - 2), developers (5 - 6), testers (1 - 2)

OPEN IN VIEWER

Figure 2.

Organization of the programme with four main projects (from Dingsøyr et al., 2023).

OPEN IN VIEWER

Figure 3.

Development phases (from Dingsøyr et al., 2023).

The whole programme was physically collocated in the same work area on two floors as shown at one time in Figure 4. Some participants in the programme also had experience from the large-scale Perform programme (Dingsøyr et al., 2018) and had a background in a similar development method.OPEN IN VIEWER

The business project and the development teams were, however, located in different parts of the work area. The functional architects were located with the business project but prepared solution descriptions of user stories for the development teams. These were documented in the programme wiki.

‘every product owner had a functional architect who would ensure wholeness, they had meetings across to ensure coordination’ (group interview).

Development method

The development method was an internal method that had been used for previous large-scale programmes. This method was a combination of a project management framework (based on PRINCE2; Bentley, 2010) and agile methods Scrum and XP (Baham and Hirschheim, 2021; Project Management Institute and Agile Alliance, 2017). This is often referred to as large-scale agile development methods (Dingsøyr et al., 2019; Edison et al., 2021). The contract was a target price contract that emphasizes shared responsibility and risk.

For each release, the business project was responsible for the analysis of needs phase. In this situation, requirements were first described as epics and then broken down into user stories (Cohn, 2004). In collaboration with the development project, such requirements were given a more detailed solution description before being assigned to a development team in the construction phase. According to informants, the solution descriptions were ‘very extensive’ and ‘detailed’. After development, the approval phase organized by the test project followed. The programme would then, at a particular time, be in the production phase of ‘the baseline’ release while being in the construction phase of ‘the settler’ and undertaking an analysis of needs for ‘the digital’ (Figure 3). The change management project introduced new solutions to the main user groups, namely, the end users seeking parental benefits and caseworkers at the Norwegian Labour and Welfare Administration.

As shown in the timeline in Figure 1, the programme started with one development team and gradually increased the number of participants to 10 teams, which we consider a very large programme (Dingsøyr et al., 2023). Development started in February 2017. One team was allowed to continuously add (‘deploy’) new functionality starting in October 2017, while the other teams followed Scrum iterations towards the main releases. The first release was approved in November 2017.

The development teams worked in 3-week iterations according to Scrum with the four roles on each team described in Table 1. Application architects and testers were main responsible for software architecture and software quality for their teams, but all team members would work on tasks also related to these areas. As described, teams obtained solution descriptions from the business project. Teams then underwent sprint planning, followed by daily meetings during the iteration, review and retrospective meetings at the end. A release would have several iterations in the construction phase. Sprint reviews were conducted across all teams, and sprint planning started with overall planning for all teams. Teams would use XP practices, such as pair programming, while some also used test-driven development. Testers worked in collaboration with the test project. Furthermore, teams used feature-branching in that they would make a new codebase branch when working on a user story, which was then merged with the main branch when the user story was ready. The development teams would coordinate in Scrum-of-Scrum meetings two to three times per week. Generally, dependencies between teams were initially handled by the Scrum masters but then turned over to developers who would talk directly to others, hold ad-hoc meetings, and/or use instant messaging or emails. When two teams worked on closely related user stories, they were sometimes physically moved to the same location.

Coordination challenges during ‘the settler’; task force established

The programme delivered according to plan in the first release. In focus group interviews, informants described work in ‘the settler’ as being characterized by not only time pressure but also a meeting culture in the programme when the organization had grown to seven development teams. Such a culture made it time-consuming to make decisions:

‘It was a constant pressure to deliver. We had six to seven development teams that should continuously be fed tasks for their sprints. And that is quite a number of people and quite a lot of power in consuming user stories’ (manager, development project, group interview).

An informant stated that because people tended to have full schedules, calling for a meeting often would delay decision-making by more than a week.

Interviews with key persons in the programme indicated that internal coordination in the projects was perceived to be working well:

‘Coordination internally in the business project and internally in the development project worked well’ (manager, development project, group interview).

The development project used a total of 18 different coordination mechanisms. Mainly scheduled group meetings, such as the demonstration at end of each sprint, planning, and Scrum-of-Scrum and daily meetings. The daily meetings were staggered so that meetings were open to all project participants. Ad-hoc meetings, which were easy to organize as the participants were physically close to each other in the work area, were also conducted. Furthermore, team members were rotated and pair programming and instant messaging directly between individuals and to larger groups were used. The project also used physical artefacts, such as architectural guidelines, maps of dependencies, and team whiteboards visible to all. When teams were working on interconnected tasks, they were moved closer to each other as previously mentioned.

Nine coordination mechanisms between the development and the business projects were established. A special role, namely, the ‘functional architects’, was established to handle contact between the business and development projects. They were initially asked to divide work equally between activities in the business project and be present on the development teams to answer questions and make clarifications, but in actual practice, most time was used for customer meetings. In addition, several meetings, such as the demonstrations, meetings to approve user stories, working meetings on solution descriptions, and what they called ‘user story meetings’, which were informal meetings to clarify issues in user stories, were also conducted.

Additional dependency maps, an issue tracker (Jira) for transferring user stories from the business project to the development project, and user stories themselves with a solution description attached were also incorporated.

Despite these mechanisms, all parties expressed frustration with the coordination between the business and development projects:

‘The coordination between projects was more demanding’ (manager, development project, group interview).

The timespan from approval of a user story until the initiation of the development process could be months, and when needs for clarifications arose, the people who had written the solution description for the user story were often working on new tasks and found it difficult to recall what they had meant.

Informants stated that the handover from solution description to construction was a formal transfer of tasks and was regulated in the contract. This type of transfer led to much negotiation over the status of user stories and solution descriptions, namely, ‘many steps and much formalism’ (group interview).

A retrospective evaluation in January 2018 focussing on the delivery method identified the ‘transitions between [the] phases [of] analysis of needs, solution description, and construction’ (Figure 3) as one main challenge. This transition also included a handover of tasks from the business to the development project.

Many programme participants expressed concerns that the development method might not work well when used at the ‘digital’ release at a time when the teams would be maintaining a system that was running while developing new functionality in an area that was much less explored. The programme was also expanding with plans for 10 development teams to work in parallel at the end of the ‘settler’ release. The plan with three main releases implied long delays in feedback from users. In the IT department at the Norwegian Labour and Welfare Administration, a general change to agile development methods was made.

The programme manager established a task force with broad representation from the programme that was given a specific mandate:

• ‘The task force shall evaluate if the programme can work more efficiently in the “digital” and advise the programme manager. The programme manager will then put forth a recommendation to the programme owner for final decision’.

Questions to be answered individually or discussed in a group

1. What project management practices can you identify in the programme?

Tasks to be given to a group for discussion and presentation

Given the problems of coordination towards the end of the second release, what should the programme manager do? Imagine that your group was the task force asked to advise the programme manager on development method in the last phase of the programme:

Task 1. What are the main arguments for changes, and what are the main arguments against such changes? How should the programme change development method if changing? What would you recommend?