Towards More Meaningful Evaluation of Cultural Heritage Programmes: Engaging Stakeholders Through Programme Modelling

Introduction

This article describes a technique for engaging stakeholders in a more meaningful approach to evaluating heritage management programmes. Within heritage management, the term ‘evaluation’ typically refers to an assessment of the number of ‘things’ conserved or preserved, or an analysis of the socio-economic impacts of preservation activities (Rudokas et al., 2019). Although these quantitative measures are important, we echo McGill’s (2018) reflection on how ‘we as heritage professionals assess the benefits of heritage initiatives’ (p. 1080). In other words, how might heritage managers and other stakeholders evaluate the effectiveness of ‘heritage enterprises … entities whose missions include heritage education, management and/or conservation’ (Kimball, 2017, p. 3)? Thus, we think the time is right for a more robust dialogue between the fields of evaluation and heritage management in light of recent developments in evaluation theory and practice.

This article presents a current view of evaluation and invites heritage managers to consider how this broader and more collaborative understanding of evaluation may enhance heritage initiatives, particularly at critical developmental stages in a programme’s design and implementation. Specifically, programme modelling techniques have enabled heritage programme stakeholders to develop consensus around the programmatic activities that are expected to produce intended goals (Laven et al., 2013). In many other disciplines, programme models have become prevalent in recent decades, and many government and philanthropic funders around the world now ‘encourage or require [their] use for planning, monitoring, and evaluating’ (Funnell & Rogers, 2011, pp. xix–xx).

To illustrate the role evaluation has played in heritage management, we feature two examples, one from the US National Park Service (NPS) and another from Cultural Heritage without Borders-Albania (CHwB-Albania). The NPS is charged with stewarding America’s natural and cultural heritage and telling the stories of the country’s people and places in more than 400 parks, heritage areas, historic waterways, and trails (NPS website). The NPS is the largest and most visible cultural heritage management authority in the United States. The NPS manages numerous museums and related cultural sites, while offering an extensive array of heritage-oriented programmes. The NPS example involves a small pilot programme designed to increase the diversity of the agency’s workforce. This pilot programme was created in response to long-standing concern that in order to tell the full spectrum of America’s stories, the agency’s workforce must reflect the full diversity of American society (McCown et al., 2012). This issue has been identified by many museums and heritage sites in other settings as well. The second example comes from CHwB-Albania, a nongovernmental organization that promotes ‘cultural heritage as an engine for local and regional development’ (CHwB-Albania website). The organization leverages restoration projects as opportunities to bring together workers from different religious and ethnic backgrounds in a manner that bridges historical conflicts and builds trusting relationships. This example describes how modelling techniques were applied to a signature programme that combines cultural heritage conservation with peacebuilding. These contrasting examples, drawn from two very different institutional and programmatic settings, demonstrate the value of programme modelling for heritage management.

Evaluation as a Tool for Heritage Management

Professionals in the field of heritage management have long recognized the need to design meaningful programmes that both protect heritage assets and engage a wide range of audiences (Hein, 2006; Sachatello-Sawyer et al., 2002; Soren, 2009). The field of evaluation offers an array of methodological approaches and tools that can support heritage professionals in meeting this need. A commonly used definition of ‘evaluation’ frames the field in broad terms:

Program evaluation is the systematic collection of information about the activities, characteristics, and results of programs to make judgments about the program, improve or further develop program effectiveness, inform decisions about future programming, and/or increase understanding. (Patton, 2008, p. 39)

Similarly, Dahler-Larsen (2012) describes evaluation as ‘a systematic, methodological, and thus “assisted” way of investigating and assessing an activity of public interest’ to inform decision-making about programmes (p. 9). These definitions reflect a more expansive view of evaluation than traditional, summative approaches driven by accountability concerns. They incorporate more recent formative and developmental approaches that are shaped by the interests of various stakeholders. This shift reflects the role evaluation can play in assisting with design and development throughout a programme’s entire lifecycle, as illustrated in the examples discussed below.

This article focuses on one prominent technique: programme modelling. As we reflected on evaluation efforts that we carried out in the heritage sector over the past 15 years, we found that incorporating programme modelling into evaluation processes produced similar benefits for organizations operating across widely varied scales and institutional settings. In the sections that follow, we consider recent developments in the field of evaluation and introduce key concepts that speak to the role evaluation can play in furthering the development of heritage programmes. Next, we discuss two featured programmes and describe how programme modelling techniques were used to advance both efforts. The concluding discussion explores the implications of programme modelling for the future of heritage management.

Recent Developments in the Field of Evaluation

The term ‘evaluation’ often has been associated with external accountability and quantitative methods that seek to measure and ‘objectively’ capture the impact of a particular programme or initiative. This traditional approach remains prominent in certain disciplines and government agency settings. Not surprisingly, evaluations of this sort may be viewed as inadequate for capturing the full range of programmatic benefits, and they can produce a visceral fear of being examined by an external party as part of a process that drives funding decisions. However, the field of evaluation has evolved considerably in recent years and increasingly emphasizes the importance of engaging key stakeholders, especially those responsible for managing programmes (Patton, 2008, 2011).

Evaluators use the term ‘stakeholders’ to refer to ‘the array of people with a stake in the evaluation’ (Weiss, 1998, p. 41), such as funders, programme designers, frontline staff and leaders, partner organizations and programme participants. Stakeholders are ‘the people whose lives are affected by the program under evaluation and the people whose decisions will affect the future of the program’ (Bryk, 1983, p. 1, as quoted in Mark et al., 2000). Within heritage studies, stakeholders may include funders, managers, employees and visitors to a particular site or attraction, as well as the local community (Kimball, 2017; Milan, 2017).

Next, we discuss a few other terms that are central to evaluation practice and consider their application to heritage practice. The term ‘programme’ generally refers to ‘a set of planned activities directed toward bringing about specified change(s)’ (Smith, 1989, p. 47). As Weiss (1998) noted, ‘programs vary in size, scope, duration, and clarity and specificity of goals’ (p. 335). Social, educational and health programmes typically focus on producing changes in certain individuals or groups (people). Heritage programmes often aim to bring about changes in people (i.e., learning) that promote the preservation or restoration of cultural resources. For example, museums seek to conserve a set of objects and engage visitors in the meaning of those objects. In some cases, museums (or other actors in the heritage sector) develop specific education programmes that go beyond visitation to offer deep educational experiences on complex topics. The examples presented in this article illustrate how the featured programmes were designed to achieve complex societal goals that extend beyond the preservation of tangible resources.

Working from this basic definition of ‘a programme,’ Owen (2007) notes that programmes have ‘two essential components: a documented plan and action consistent with the documentation contained in the plan’ (p. 26–27). Programme models offer a useful means of documenting ‘the plan’ for a given programme. These models come in many different forms (Funnell & Rogers, 2011; Knowlton & Phillips, 2013). While these distinctions are important within the field of evaluation, our purpose is to introduce programme modelling more broadly, thereby enabling heritage managers and scholars to consider the potential utility of this common element of evaluation practice.

Models ‘bring clarity to the planning and create consensus or a better understanding’ (Boulmetis & Dutwin, 2011, p. 120) of the programme. They provide a succinct graphic representation of ‘how a program is expected to achieve its intended outcomes … [and are] often of great benefit to program managers and other stakeholders, who … have never had the opportunity to construct a visual image of their program’ (McDavid & Hawthorn, 2006, p. 54). Evaluators frequently design and facilitate interactive workshops that engage a cross-section of stakeholders in building a programme model. Understandably, the model-building process and resulting ‘product’ may seem mysterious to the uninitiated. In our experience, most people learn by doing; the process and product become much clearer as a result of having participated in a model-building workshop or seeing the resulting model. To give readers a sense of the process and resulting model, the following discussion presents an example from the NPS.

Example One: US National Park Service Workforce Diversity Pilot Programme

The Mass Parks Student Career Intake Program (SCIP) was initiated as a small pilot programme that aimed to increase the diversity of the NPS workforce. Developed by national parks and partner organizations in the state of Massachusetts, SCIP grew to include additional national parks throughout the northeastern US. SCIP was designed as a professional development opportunity for college students from urban and minority communities. Participating students were awarded a series of summer internships, enabling them to develop professional skills and gain experience working in parks. Interns engaged in a structured progression of trainings, visited parks and partner organizations and met with staff. They received summer job placements and worked alongside permanent NPS staff that served as supervisors and mentors. SCIP prepared interns with the knowledge, skills and experience to apply for permanent jobs in the NPS. This intensive approach to professional development and staff recruitment required substantial investment and differed markedly from previous attempts to diversify the agency’s workforce, which failed to realize the level of programmatic involvement required to shift organizational hiring practices.

During SCIP’s four-year pilot phase, the National Park Service Stewardship Institute ¹ and the authors were asked to provide technical assistance to document and begin evaluating the programme. Given that SCIP represented a new programmatic approach for the agency, the evaluation team recognized the importance of developing a visual model to outline the programme’s core components. As mentioned above, such models are often constructed through facilitated workshops with a cross-section of stakeholders. In the first year of this effort, SCIP leaders convened a full-day stakeholder workshop that brought together representatives from national parks, the agency’s regional office that funded the programme and participating educational, environmental and youth organizations. In this intensive day-long workshop facilitated by the university-based evaluator, participants worked in small groups to outline key components of the programme. The small groups were organized to ensure that each group included a mix of stakeholders representing different types and levels of involvement with the project. (Detailed guidance on organizing and facilitating model-building workshops is beyond the scope of this article. General model-building resources include Frechtling, 2007; Funnell & Rogers, 2011; Knowlton & Phillips, 2013; and Weiss, 1998. See Laven et al., 2013, for a heritage-specific example.)

Unfortunately, the initial model-building workshop could not be scheduled at a time that allowed for SCIP interns to participate. Following the event, the project team created a draft model based on the materials generated during the workshop and solicited input from interns and programme staff to hone the model. In subsequent years, stakeholder meetings were scheduled during the college winter break to ensure that interns could share their experiences with other stakeholders and contribute their perspectives as the model was revised to reflect the programme’s evolution during the pilot period. Successive iterations of the model were labelled with a version number (e.g., 2.0) to emphasize its role as a living document that evolves alongside the programme. Table 1 outlines the steps involved in developing the SCIP model.

OPEN IN VIEWER

Table 1.

The Process Used to Develop the Program Model for SCIP

Step 1	Form evaluation planning team that includes program leaders and staff, along with evaluation experts.
Step 2	Exchange foundational information about the program and potential evaluation approaches to foster collaborative learning among members of the planning team.
Step 3	Convene a stakeholder workshop co-facilitated by the program representatives and evaluators that: a)  Introduces program modelling and its role in evaluation; b)  Includes small group sessions in which participants begin to outline key components of the program model.
Step 4	Create a draft program model based on materials developed during the workshop.
Step 5	Share the draft model with workshop participants and elicit feedback to hone the model.
Step 6	Disseminate the revised model to workshop participants and other program stakeholders as a ‘living document’ intended to evolve alongside the program.
Step 7	Use the model: a)  As a foundation for program evaluation to frame data collection, analysis and reporting; b)  To inform program management and communications.
Step 8	Revisit the model alongside the subsequent evaluation findings to: a)  Assess how—and how well—the program is meeting its intended goals; b)  Refine the model and/or program based on the learning that occurs.

Source: Jewiss & Laven (2021).

Figure 1 presents the SCIP model that was honed throughout the four-year pilot period. The left-hand column describes the resources necessary to carry out the programme, including staff, mentors, support from partner organizations, interns, funding and equipment. The middle column lists the activities that are intended to achieve the programme’s goals, such as offering training for interns and staff, and providing experiential learning opportunities that enable interns to visit park worksites. The right-hand column summarizes outcomes (goals) that are expected to result, over time, from the programme activities. The primary intended outcome is for SCIP graduates to be hired into permanent NPS positions. The programme aims to bring about other benefits for participating interns (including the development of professional skills and support networks) and for the national park agency (including increased cultural competency, community engagement, and diversity of its workforce). The model is designed to be read from left to right and can be loosely understood as a long ‘if-then’ sentence: If these resources are available and used to carry out these activities, then these outcomes are expected to occur.

OPEN IN VIEWER

Figure 1.

Source: Jewiss & McCown (2012).

Programme models necessarily offer a simplified representation of a more complex reality. For clarity, the SCIP model is organized in a linear fashion, although this presentation is not meant to signify that the programme follows a strictly linear sequence. For instance, as SCIP participants engaged in a series of summer internships throughout their time in the programme, they received multiple opportunities to participate in the slate of programme activities and successively deepen their knowledge, skills and support networks.

The SCIP experience, as represented in the model, was credited with making two systemic contributions to the agency’s workforce diversity efforts. Given the pressing need to diversify its workforce and the failure of prior diversification initiatives, leaders from across the agency were watching SCIP’s evolution. Outlining key components of the programme in a clear, accessible, one-page format facilitated information-sharing with other interested parties. As soon as the initial version of the model was finalized, SCIP leaders began using it as a central and highly effective tool for reporting on the programme’s progress to agency leaders responsible for making decisions about continued funding. In addition, when other leaders at a large national park began designing a new programme with a national reach, the SCIP approach outlined in the model substantially informed their thinking about what was required to recruit diverse interns, offer robust professional development opportunities and prepare interns for permanent positions in the agency. Beyond this direct contribution to a national programme, the SCIP model made transparent the often-hidden institutional infrastructure necessary to address a persistent problem. Thus, the modelling process enabled SCIP to fulfil its potential as a pilot by providing a documented, logical design that inspired and informed related initiatives across the agency.

Example Two: Cultural Heritage Without Borders-Albania’s Regional Restoration Camps

CHwB is an independent, Swedish nongovernmental organization ‘dedicated to rescuing and preserving cultural heritage affected by conflict, neglect or human and natural disaster’ (CHwB website). Founded in Bosnia and Herzegovina in 1995 as a response to systematic destruction of cultural heritage during the war following the collapse of Yugoslavia, CHwB subsequently expanded its offices to Kosovo and Albania, and conducted projects in other locations around the world, including Eastern Europe, East Africa and the Middle East (Hadžić & Eaton, 2017). Although CHwB serves as an umbrella organization, the Kosovo and Albania offices are separate entities that run their own programmes.

Along with its geographic expansion, CHwB’s approach to working with cultural heritage has evolved as well. In addition to preserving built or tangible heritage, the organization has increasingly focused on peacebuilding through programmes that use cultural heritage as a platform for cross-cultural engagement and understanding. One example is the organization’s Regional Restoration Camps (RRC), which began in 2007 with a few students working to restore a historic building in Albania. The RRC programme is led by CHwB-Albania and operated in partnership with the organization’s other offices in the Western Balkan region.

In Heritage and Peacebuilding (Walters et al., 2017), Hadžić and Eaton introduce the RRC programme within the context of the Balkan conflict. They describe the programme as ‘a simple training model’ in which participants from different countries join a two-week professional development ‘camp.’ The camps focus on restoration or preservation of significant cultural heritage assets with an eye towards cross-cultural learning and understanding. Master crafters supervise hands-on restoration work, and camp participants receive instruction on a range of topics such as conservation theory, interpretation and business planning. Recently, the RRC programme was offered in Albania, Bosnia and Herzegovina, Kosovo and Serbia. As of 31 May 2020, CHwB-Albania’s website indicated that the programme had hosted 44 RRC sessions in 10 historic locations across the Western Balkans; engaged over 1,000 participants from 28 countries; conducted 111 restoration interventions, including the conservation of nearly 100 museum objects; completed 11 business plans and 12 interpretive plans; and contributed nearly 60,000 estimated hours of hands-on work.

Despite this impressive list of achievements, the growth and success of the RRC programme raised deeper questions about the programme’s impact, sustainability and potential replication beyond the Balkan context. As Hadžić and Eaton (2017) conclude:

More research is still needed to determine the extent of the impact of the RRC. Upon returning home, do participants feel a positive change as a result of the memory of the cultural experience they had at the RRC? Do they share this with their friends, family and colleagues? How lasting are the ties that the RRC form across borders in the region? However, as an effort to engage heritage sites for peacebuilding, the RRC hold the potential for inspiring deep and lasting impact within a subset of the population of the Balkans that is young, mobile and, in many cases, likely to work closely in fields related to cultural heritage in the region. (p. 218)

These questions, coupled with the programme’s tenth anniversary, led CHwB-Albania to initiate a programme model-building process to position the programme for an in-depth evaluation. This process was similar to that used at the outset of the SCIP technical assistance effort. However, it was much shorter in duration and thus focused solely on the first six steps outlined in Table 1.

To support this process, the authors were asked to provide technical assistance to help plan and facilitate a two-day workshop to develop a programme model of the RRC. The planning team (CHwB-Albania’s executive director, programme staff and the evaluators) developed the workshop agenda and identified a diverse group of RRC stakeholders to participate in the workshop, including CHwB staff, former programme participants/volunteers, an RRC lecturer and craftsperson and representatives from supporting organizations, such as funders and universities.

An immersive, two-day workshop was held in Tirana in July of 2018 and co-facilitated by the evaluators, who guided the 15 attendees through the model-building process. As described above, a programme model can be thought of as an outline of how the programme intends to achieve its goals or a diagram that ‘tells the story’ of how a programme is designed to work. Attendees worked in small and large groups to discuss, debate and document their views on various aspects of RRC operations and goals. The workshop presented a rare opportunity for this diverse array of attendees to share their experiences with the RRC, thereby learning from each other and gaining a more holistic understanding of the programme.

The RRC model is organized into four primary categories: (a) guiding principles, (b) outcomes, (c) resources and (d) activities. Three of these categories parallel those in the SCIP model. The RRC model includes a fourth category, guiding principles, that articulates the programme’s priorities and values. A recent development in the field of evaluation is the emergence of principles-focused evaluation (Patton, 2018). This increasingly prominent approach recognizes that programmes operating in the midst of complexity and change often need to rely on a set of principles or strategies that guide decision-making and implementation. Guiding principles, much like a ship’s rudder, help programmes navigate turbulent waters of rapidly changing programmatic and funding environments. Examples of RCC guiding principles are (a) ‘create opportunities for “hand-on” experiences with heritage,’ (b) ‘transmit knowledge and experience between generations,’ (c) ‘conserve cultural heritage resources’ and (d) ‘use heritage as an enabler for human development’ (Figure 2).

OPEN IN VIEWER

Figure 2.

Source: Cultural Heritage Without Borders-Albania (http://chwb.org/).

The RRC model represents an important first step toward designing a thorough evaluation to address the aforementioned questions about the programme’s impact and potential to expand and adapt to other settings. As with SCIP, the RRC model-building process yielded additional benefits. Most important, the process served as a vital stakeholder engagement opportunity, bringing together a wide array of programme staff, funders and representatives from partner organizations—each of whom offered a distinct perspective on the programme based on their specific role. For example, former participants, instructors specializing in traditional craft skills and partner organizations that help deliver the programme all shared their respective experiences with the restoration camps, thereby enabling other workshop participants to gain a deeper understanding of how the camps function ‘on the ground’. CHwB-Albania staff described the operational activities needed to prepare for and implement each RRC session, offering workshop attendees an essential ‘behind the scenes’ view of what is required to run the programme. The model-building process also allowed the programme’s two lead funders to articulate and align their complementary interests in supporting the programme. As was the case with SCIP, the RRC model revealed a holistic picture of the programme’s many facets that all need to work in concert to achieve the intended results. In addition to the RRC model (the product) that was created, this stakeholder-focused process produced a deep, collective understanding of key components of the programme and how those components are designed to interact to generate the desired goals.

Reflections on the Contributions of These Model-building Efforts

Although these two heritage programmes are situated in very different geographic, cultural, political and institutional contexts, we observed that the SCIP and RRC model-building efforts yielded similar benefits. The RRC example highlights two important issues. First, when portrayed in model format, it becomes clear that the RRC programme is far from a ‘simple’ training programme as described by Hadžić and Eaton (2017). Second, during the RRC workshop, a fundamental discrepancy surfaced. Some of the participating stakeholders felt strongly that the programme should be judged on strict business principles and RRC’s ability to be financially self-sustaining. However, another group of stakeholders were equally passionate about the programme’s educational, heritage stewardship and peacebuilding goals. The facilitated model-building process prompted stakeholders to articulate these strongly held views and provided a forum for reconciling these critical differences.

This process fulfilled a key role of evaluation (Dahler-Larsen, 2012; Patton, 2008) by providing the technical assistance needed to systematically examine and better understand a programme to inform its continued development. By the end of the two-day facilitated workshop, RRC stakeholders reached consensus about the programme’s guiding principles, activities and intended goals. The RRC model thus serves as the ‘social contract’ for this shared understanding among the participating stakeholders. CHwB-Albania staff also recognized the model’s value for future evaluation efforts, a high priority for RRC’s funders. Lastly, RRC leaders expressed interest in using a similar model-building exercise for other CHwB-Albania programmes, which underscored the value of this approach.

Returning to the SCIP example allows us to discuss how a programme model can serve as the foundation for on-going programme evaluation. Following the initial SCIP model-building workshop, the evaluation team was charged with gathering participating interns’ perspectives on the programme to further inform continued development of this pilot programme while assessing how (and how well) the programme was achieving its goals. SCIP interns engaged in a series of individual interviews conducted annually by the lead evaluator. The use of qualitative data collection methods, which ‘rely on words and narrative rather than counts and statistical analyses’ (Weiss, 1998), has become increasingly prevalent in evaluation over the past several decades (Goodyear et al., 2014; Patton, 2015; Shaw, 1999). Interviews of SCIP interns yielded rich narrative data that helped address the agency’s primary goal of using this pilot as a learning opportunity and hone the programme accordingly. In SCIP’s case, the intentionally small size of the pilot programme limited the extent to which meaningful quantitative analyses could be conducted on the programme activities and outcomes. However, as the programme progressed, SCIP staff used readily available administrative data to document the breadth of programme activities and, most important, to track progress on the primary intended outcome: the number of SCIP graduates hired into appropriate permanent NPS jobs. As a result, the model played a central role in keeping stakeholders focused on this primary goal—the programme’s core purpose—in the midst of all the tasks required to operate a programme within a large institution.

The interviews were recorded and transcribed, and the lead evaluator analysed the transcripts according to the components of the model. Thus, the model served as a conceptual framework for data analysis and reporting. Excerpts from the interviews were used to illustrate individual components of the model, ‘bringing it to life’ by sharing a participants-eye-view of the programme. The final evaluation report (Jewiss & McCown, 2012) presented excerpts from the interviews, including the following that reflects the programme’s primary goal:

I’ve always wanted to work in the Park Service. SCIP reinforced it—definitely. I finally got my [permanent] appointment! It’s a very, very long process. It’s hard to get into the Park Service—to be a federal employee—even if the park wants you. But I finally got it! This program works—I’m proof of that. (p. 25)

In consultation with programme stakeholders during subsequent annual stakeholder workshops, findings from the interviews were used again to refine both the model and the programme delivery.

The model and accompanying qualitative evidence enabled other interested parties throughout the NPS to gain a deeper understanding of this new programmatic approach as the pilot progressed. The model played a critical role in facilitating communication with key players across the agency as they sought to address some of the ‘additional National Park Service outcomes’ outlined in the bottom right portion of the SCIP model. These broader agency-level outcomes included enhancing understanding of youth culture, cultural competency and a sense of responsibility for diversifying the agency’s workforce.

The NPS investment in the SCIP pilot and evaluation yielded a number of substantial benefits. The model-building and related evaluation activities served as a centrepiece for the programme’s annual stakeholder meetings. SCIP leaders appreciated the inclusive model-building process, which deepened engagement among the programme’s many stakeholders. Since the evaluators were ‘coming in from the outside’ with the charge of documenting and evaluating the pilot programme, they were well positioned to ask probing questions that prompted workshop participants to articulate and hone their thinking about various aspects of the programme. The SCIP model and associated discussions proved especially beneficial when new members joined the stakeholder group due to staff turnover, the addition of new partner organizations or recruitment of a new cohort of interns.

SCIP leaders also used the model as a strategic communication tool when meeting with key decision-makers in the NPS. The programme coordinator indicated that the ‘ink was barely dry’ on the first version of the model when she presented it to leaders in the regional office that funded the programme. The SCIP model played a central role in building shared understanding and agreement among stakeholders about how the programme is designed to bring about its intended outcomes. The model then served as an essential tool for explaining the programme to wider audiences and carried added weight since it represented thoughtful deliberation and consensus among the full slate of stakeholders. Although it is impossible to determine the precise degree to which the model and subsequent evaluation findings informed funding decisions made by agency leaders, it is telling that funding was allocated to sustain the evaluation throughout the programme’s four-year pilot and SCIP’s organizers were awarded funding to continue the programme well beyond the pilot period.

Implications for Heritage Management

The existing cultural heritage literature has not yet delved deeply into ways that evaluation can make meaningful contributions to heritage practice. Heritage programmes encompass a broad range of complex and sensitive topics that extend far beyond the preservation of tangible artefacts. Such programmatic complexity inevitably leads to questions about effective programme design and programme impacts, as implied by McGill’s (2018) reflection in the introduction of this article. In response, we discussed two examples and explored how a foundational evaluation technique, programme modelling, can serve the field of heritage studies in the design and delivery of such programmes.

For instance, SCIP and RRC were developed to address different challenges (workforce diversity and peacebuilding, respectively) in very specific institutional and socio-political settings. Both of these heritage programmes are situated in rapidly changing contexts and require the contributions of a wide range of stakeholders. Programme modelling offers diverse groups of stakeholders a collective opportunity to engage in reflective inquiry about the goals, principles and processes associated with the programme and its context. In this way, the SCIP and RRC models make the entire programme tangible, visible and context-sensitive. The featured examples also illustrate how programme modelling can set the stage for future programme evaluations and long-term programme development. In this sense, the SCIP and RRC models are evaluation touchstones or roadmaps. With these models, programme stakeholders can engage in informed discussion about the relationships between programme resources, activities and outcomes, and most importantly, the developmental needs of the programme. Over time, the specific concepts in each model can then be operationalized as variables, which can be assessed qualitatively and/or quantitatively. This collaborative endeavour enables evaluators to work with programme stakeholders to ask specific questions about programme effectiveness and impact, and to design appropriate studies that can shed light on these questions in meaningful and constructive ways.