Abstract
This article argues for the use of the Project Implementation Profile (PIP) tool in managing non-governmental organization (NGO) projects. It applies the PIP to a project in the non-profit NGO sector and assesses the extent to which it provides project success assurance. The findings suggest that PIP can be used in this sector. Further work to refine the tool and integrate it with the Logical Framework is suggested.
Introduction
There has been widespread adoption of the Logical Framework by the development and non-governmental organization (NGO) sectors as the primary approach to project design (Crawford and Bryce, 2003; Gasper, 2000). Within the Project Cycle Management process, the Logical Framework is an important component of the project design stage (Simpson and Gill, 2007). When used properly, the Framework provides a definition of project outcomes (Bell, 1998), and a hierarchy of goals and objectives representing the project vision and mission (Simpson and Gill, 2007). However, the use of the Framework has been extended into the project execution stage and this has proved problematic (Bell, 1998; Crawford and Bryce, 2003; Dearden and Kowalski, 2003; Simpson and Gill, 2007). The main concerns relate to the need for project monitoring information (Crawford and Bryce, 2003), a lack of detail relevant to implementation (Bell, 1998), inappropriate use (Dearden and Kowalski, 2003) and the lack of flexibility the model can engender in project managers (Simpson and Gill, 2007).
The research described here intends to demonstrate that an established project management quality assurance tool can be applied to projects in the NGO sector. The ‘Project Implementation Profile’ (PIP) is a tool developed to assess identified critical success factors in projects so that the project manager is able to provide dynamic control when managing the execution of the project work. The PIP tool assesses a set of 10 critical success factors identified in empirical research. The tool uses a structured questionnaire to gather information about the management of a specific project and provides, by means of comparison with a set of benchmark values, an assessment of the strengths or weaknesses in the project being reviewed (Pinto and Slevin, 1987, 1988a, 1988b; Slevin and Pinto, 1986). This article reports research based on the outcomes of a case where the tool was used to evaluate the management of the pilot stage of a project called the ‘Grassroots Focus Index’ (GFI). The goal of the work was to evaluate the practical application of the PIP tool in the NGO sector.
The following sections provide details about the PIP tool and then describe the GFI pilot project that the PIP was applied to. It is worthwhile noting that project success factors differ from success criteria. Project success factors are those activities performed during the execution of the project that improve the chances of a successful outcome. Success criteria are the metrics that are used at the end of a project to assess how successful it has been in achieving its goals (Andersen et al., 2006; Cooke-Davies, 2002; Diallo and Thuillier, 2004; Ika, 2009; Khang and Moe, 2008). It is also important to understand that successful management will not guarantee the product or service provided is successful in its intended purpose, but poor project management will lead to poor performance (Collins and Baccarini, 2004). In the case described, PIP was used to assess the management of the GFI pilot project.
The project implementation profile
The PIP model of critical success factors was selected as the focus of this research for the following reasons. The model was developed from research spanning a wide variety of industry sectors (Pinto and Slevin, 1987) and was subsequently tested and supported with a sample spanning several industries (Belout and Gauvreau, 2004). The focus of these researchers appears to be projects in the private sector. The research results were benchmarked against a database of over 400 projects and out of this work, a practical tool was developed (Slevin and Pinto, 1986).
The PIP model provides the project manager with the means to assess the presence of 10 identified critical success factors on a specific project (Pinto and Slevin, 2006). This is achieved by administering a short questionnaire to a few participants in the project and then assessing their perceptions of the project from the results obtained. The procedure is relatively simple to follow and can be repeated at regular intervals over the life cycle of the project. This latter point makes the tool a useful aid in monitoring and controlling a project as it progresses.
The main advantage the PIP tool has over alternative approaches to project control is the focus it places on the human relations side of project management (Pinto and Slevin, 2006). Traditional project control tools are based on quantitative information such as financial or cost data in the case of earned value management, or milestones in the case of Network and Gantt Charts. In contrast, the PIP tool addresses behavioural issues, for instance, with questions about client acceptance and top management support (Pinto and Slevin, 2006).
The 10 critical success factors examined by the PIP tool are project mission, project schedule/plans, client consultation, personnel, technical tasks, client acceptance, monitoring and feedback, communication, top management support, and troubleshooting (Pinto and Slevin, 1987, 1988a, 1988b).The questions related to each of the factors are assessed on a Likert scale ranging from 1 to 7, where 1 represents ‘strongly disagree’ and 7 represents ‘strongly agree’. The framework was developed into the PIP tool to provide a practical way to diagnose a project during and after completion (Pinto and Slevin, 1988b, 2006; Slevin and Pinto, 1986).
Since its development, the PIP tool has been widely used to retrospectively assess the management of completed projects to identify lessons that could be applied to future projects (Andersen et al., 2006; Chan and Chan, 2004; Tan et al., 2010; Turner and Muller, 2005). Examples include studies of information technology, construction, and research and development projects (Pinto and Slevin, 2006). However, very little literature exists on the use of PIP in the NGO sector. Modern thinking about project management acknowledges that the project context is important and that there is no single universal set of principles to ensure project success in all contexts (Shenhar, 2001).
About the GFI project
The project being reviewed is the GFI developed by the sponsor. The sponsor is a continental advocacy NGO that specializes in monitoring the impact of development commitments (development aid as well as resources committed by African governments through national budgets) on the marginalized communities.
The overall goal of the GFI project is to provide a basis for connecting policies and development interventions to the realities and aspirations of grassroots communities. Rather than basing the index on some arbitrary and apriority determined dimensions, the starting point in constructing the index was to ask the grassroots communities in the pilot countries of South Africa, Nigeria and Cameroon to respond to a neutral focus statement that asked them ‘What are your ideas on the grassroots importance in and influence on pro-poor development programs’. This was discussed in focus group discussions and responses were generated by the communities in the pilot countries namely South Africa, Cameroon and Nigeria. The responses were consolidated by using the concept mapping methodology and the index was constructed on the basis of the common and shared concepts/issues across the three pilot countries. The grassroots communities are defined as the poor and marginalized whose realities and aspirations are not suitably articulated and addressed by policy makers and other development agencies.
The GFI project is broken down into two phases. The first phase (which is of interest to this article) deals with the construction of the actual index and the second phase involves rolling out the GFI for use by NGO partners. The objective/outcome of phase one is to develop the index, while the objective of phase two is to influence prioritization through adoption and use by NGOs.
In implementing the first phase of the GFI research project, the sponsor contracted two consultants; one based at the University of Free State (South Africa) and the other based at Ibadan University (Nigeria). These institutions provided lead consultants who worked closely with the project manager and the sponsor in implementing the project. A Technical Advisory Group (TAG) was constituted to advise and guide the development of the index. The TAG consisted of leading economists, statisticians and development practitioners from the following institutions: the African Development Bank (AfDB), the United Nations Economic Commission for Africa (UNECA), the African Economic Research Consortium (AERC), University of KwaZulu-Natal, among others. There was also the Program Implementation Team (PIT). This consisted of the sponsor’s internal team (including the author as the project manager) as well as the consultants from the two institutions.
The key activities of phase one included: assembling the team of consultants, assembling the TAG, organizing initial engagements with grassroots groups including faith communities and community-based organizations, conducting primary research (concept mapping and focus group discussions in communities in South Africa, Cameroon and Nigeria) to generate dimensions of the index, conducting a survey in these countries following the focus group discussions, analysis, validation meetings to validate the outcome of the index, among others. Prior to engaging the consultants, several internal brainstorming sessions were conducted including meetings of stakeholders and grassroots representatives to get their views on what aspects the index should address.
Research context
This article presents results of an evaluation of PIP as a quality assurance tool in an NGO environment by retrospectively applying the PIP to a specific project called the GFI project. The GFI project intends to generate an index that can be adopted and adapted by NGOs and policy makers who need to determine and influence the prioritization of marginalized communities. In this article, the completed pilot phase of the GFI project is used to assess the usefulness of PIP in determining and assuring project success so that it can be used in the forthcoming roll-out (second) phase. For practical purposes the study used the GFI close-out report as the basis for benchmarking the PIP tool.
In the project cycle, the Logframe fits the front-end process of establishing the project goals. The PIP model assesses the management process that follows on from the front end. It provides a quality check on the process of project execution and in this respect complements the Logframe process; it does not replace it.
Within the context of social and poverty alleviation projects in the developing countries, the literature on project success criteria and factors is limited in extent (Diallo and Thuillier, 2004; Ika et al., 2012). It is generally acknowledged that project success depends on project type and the sector in which the project is being undertaken as well as the perceptions and interests of various stakeholders assessing its success (Baccarini, 1999; Diallo and Thuillier, 2004; Freeman and Beal, 1992; Lim and Mohamed, 1999; Lipovetsky et al., 1997). It is therefore reasonable to assume that the non-profit sector involved in developing countries faces a set of challenges that are unique to this context and are thus worthy of study.
It has been suggested that in the development context, the role of the NGO is one of project implementation (Charlton and May, 1995; Crawford and Bryce, 2003). The NGO sector has been singled out as different in a variety of ways from the commercial and government sectors (Najam, 1996). Notably, decision-making lies with organization leaders and members not investors or administrators, and the culture is one of consensus rather than hierarchical or market driven.
The main focus of academic project management research has been the construction sector with little work done in the agricultural and development areas (Themistocleous and Wearne, 2000). The available literature suggests that aid industry projects are distinguished by their focus on social change, are political and ‘attract a wide range of stakeholders’ (Crawford and Bryce, 2003: 364). In addition, the projects are implemented in a complex operating environment characterized by ‘wide geographical and cultural separation between project actors’ as well as challenging technical and operating conditions (Crawford and Bryce, 2003: 364).
Due to these differences, a need to modify and adapt traditional project management to the realities in the non-profits has been proposed (Khang and Moe, 2008). Whereas in commercial organizations, clients wield significant power in the sense that they can either accept or reject the project product, in the NGO sector the beneficiary is less powerful. In some cases, the beneficiaries are too poor to have a voice and are grateful for any interventions that are targeted at changing their circumstances.
In spite of the differences between projects in commercial organizations compared with non-profit organizations, the key success factors in both sectors revolve around similar common factors. These are elaborated below.
Organization-specific factors that have been highlighted as important in determining project success for international development and non-governmental projects in the literature include: extent to which project resources are available (Khang and Moe, 2008; Struyk, 2007; Youker, 1999), identification and active use of project champions, organization culture supportive to project excellence, project management structure and the extent to which organization publishes success stories (Khan et al., 2000; Vickland and Nieuwenhuijs, 2005), top managers competence on projects and the extent of bottom-up implementation (Vickland and Nieuwenhuijs, 2005) and the extent to which learning opportunities are generated and harnessed among projects (Struyk, 2007). The availability of learning opportunities has also been cited as important in determining project success (Khan et al., 2000; Struyk, 2007).
External environment-related factors that have been noted in the literature as important in determining project success for international development and non-governmental projects include: political aspects (examples include wars, unrests); legal aspects (examples include changes in laws, regulatory systems, etc.); economic aspects (examples include changes in economic regulation and conditions); environmental aspects (examples include pollution, concerns with the green economy); social aspects (examples include ethnic hostilities, religious conflicts and divisions); physical aspects (examples include natural disasters, etc.) and cultural aspects (examples include different thinking and background as well as cultural fabric) (Ika et al., 2012). These factors have also been affirmed as significant in determining project success (Crawford and Bryce, 2003; Khang and Moe, 2008; Struyk, 2007).
Client- and stakeholder-specific factors important in determining project success in development and non-governmental projects include: client consultation (Struyk, 2007); understanding of project environment, stakeholder support, strong ownership (Diallo and Thuillier, 2005; Khang and Moe, 2008) and communication and trust (Diallo and Thuillier, 2005). In addition, the geographical dispersion of the stakeholders has meant more emphasis needs to be placed on communication as well as consultation. Client consultation under the not-for-profit environment significantly differs from commercial settings in the sense that the client in the NGO environment is the beneficiary and in most cases, does not have power to influence project decisions. In contrast, commercial clients are powerful enough to reject or accept the project product through the purchase decision.
Team-specific factors that are significant in determining project success for development projects and NGOs include: flexibility of project plan and implementation (Khan et al., 2000; Khang and Moe, 2008); personnel training and use of right people from right departments (Vickland and Nieuwenhuijs, 2005); number and spread of implementing actors (Struyk, 2007); competencies of project team, commitment and adequate local capacity (Khang and Moe, 2008); trust and communication (Diallo and Thuillier, 2005) and skills and composition of project team (Khan et al., 2000; Khang and Moe, 2008; Struyk, 2007; Vickland and Nieuwenhuijs, 2005).
Project manager-specific factors that are significant in promoting project success in development projects as well as NGOs include: implementation approach and selection of the right project team (Khan et al., 2000) and strong project management (Vickland and Nieuwenhuijs, 2005).
Project-specific factors important in determining project success for development and non-governmental projects include: the extent of use of project compatible technology (Ika et al., 2012), urgency of project and whether the right project is selected (Khan et al., 2000).
Research method
A case-study approach was considered to be the most appropriate research design. This was because we were looking at a single instance of a project (Punch, 2005). We were also addressing a research question ‘how applicable is the PIP to the NGO sector’, where ‘how’ is a critical component (Yin, 2009). The approach behind the research follows the three stages of ‘contextualist’ methodology (Mjoset, 2009). The three stages are (i) select a process to be studied that relates to the research problem, (ii) define the context that is treated as the environment of the selected process and (iii) trace the links in the selected process. The selected outcome for the research is the GFI, a pilot project and the process being evaluated is the management of the pilot project. The contextual scope of the work is restricted to projects which are executed by an NGO in developing countries (Goertz and Mahoney, 2009). The unit of analysis for this research is a project, or project stage, conducted in developing countries by NGOs.
To select subjects within the case context, a non-probability purposive sampling approach was adopted. This was because the intention of the study is to get some insight into project success for the roll-out phase of GFI, generalizing the results was of secondary importance.
Standard PIP questionnaires were e-mailed to various project stakeholders in the following quantities: two questionnaires to the beneficiary group, two questionnaires to the internal project team members, one questionnaire to the project manager and two questionnaires to the two main consultants. The data obtained from the interviews were triangulated with reports from the project as well as any other relevant information.
The selected representatives from the beneficiaries represent the two broad groups that the sponsor works with and these are community-based organizations and regional NGOs. It should also be noted that most stakeholders (including beneficiary group) did not have sufficient information to answer all the questions in the PIP questionnaire. In addition, the demand side to the GFI is fully taken care of because the index’s dimensions and components were generated by the grassroots communities through focus group discussion and survey.
The current sample of seven respondents is considered reasonable within a case study framework. It compares favourably with other studies of this nature (Finch, 2003). The GFI project close-out report was used as a basis for comparison and benchmarking of the PIP tool. The relevant section of the close-out report is the one that deals with how effectively the project was executed.
The study adopted the original PIP tool without any modifications. Deviations are reported under the findings and suggestions for improving the PIP tool highlighted.
The two major contributions of this research are that (i) it seeks to use the case study approach to understanding project success within a non-profit environment and (ii) it applies the PIP tool to a broad range of stakeholders of the GFI project rather than being limited to the project manager’s view.
Study findings and discussion
Out of the seven questionnaires adminstered, six responses were obtained. The only non-response was from one of the consultants. The study used the following critical scores for the GFI evaluation: a score of between 80 and 100 per cent is excellent; a score of 70–79 per cent is good performance; 50–69 per cent is average/fair performance and 0–50 per cent is regarded as poor performance (Finch, 2003; Pinto, 1990). Detailed scores are provided in the next section. The discussion of the results is organized around the success factors and not project stakeholders. The score for each factor is obtained by averaging the scores of individual project stakeholders, namely project manager, project team, end users, and the consultants.
Each factor is discussed below; they are presented in the order of importance rather than the order they were originally listed by Pinto and Slevin, but the original numbering is retained to assist in cross referencing with other research.
Factor 1: Project mission and goal
This factor was rated the most well handled in determining GFI project success. It scored 31 out of 35 (constituting 88.6%), hence falling within the excellent category. The high score indicates that this factor was the best managed in the GFI project. The standard deviation of 2 and coefficient of variation of 0.06 (that is 2/31) for the project stakeholders rating on project mission and goal shows that this factor was scored highly with very little variation among the project stakeholders. These findings confirm similar findings in other research work (Clarke, 1999; Cooke-Davies, 2002; Pinto and Slevin, 1987; Slevin and Pinto, 1986).
The implication of the above results is that project stakeholders agree that the GFI project is aligned with the sponsor organization’s strategic direction. The organization’s overall aim is to influence development prioritization in the direction of grassroots and marginalized communities. It does this through advocacy. The GFI objective is to develop an index that assists in determining the level or extent of grassroots prioritization in development, and this is compatible with the sponsor’s overall goals. It also shows that the project stakeholders generally believed that the GFI project mission was clear.
Factor 6: Technical tasks
This factor was scored with an average of 31 out of 35 (constituting 88.57%), falling within the excellent category. It was the joint top scorer with project mission. The average score of 31 out of 35 means this factor was excellently handled and was important in contributing to the success in the GFI project. Interestingly, the GFI close-out report did not mention anything on this factor. The scores obtained here confirm the importance of this factor in determining project success (Kerzner, 1987; Pinto and Slevin, 1988a; Slevin and Pinto, 1986). The technical tasks factor had a standard deviation of 1.55 and coefficient of variation of 0.05, showing very little dispersion in responses among project stakeholders.
The high score for this factor is not surprising as the main consultants to the GFI project were very competent. Both the lead consultants are professors with extensive consultancy and index development experience spanning several decades. All of the required equipment was available and this contributed to increased project success. The project team tasks were well managed in terms of budget and capability though there were problems with the schedule and a number of activities overran. This happened due to underestimation of the duration of tasks. Because of its innovative nature, the initial project goals and plans were not well understood and clarification of these was problematic.
Factor 5: Personnel
This factor had the third highest average score of 27 out of 35 (constituting 77.1%). This shows that the factor was well handled and was important in assuring GFI success. The PIP tool confirms the GFI project close-out report’s appraisal of this factor as having been well handled and contributing to effective project execution. This factor had a standard deviation of 3.34 and a coefficient of variation of 0.12. The high scoring by the project manager indicates the importance attached to a professional team to successfully complete the GFI project.
However, the importance of the personnel factor in determining GFI project success contradicts findings published elsewhere. Other research had concluded that this variable is not important in determining project success (Belout and Gauvreau, 2004; Pinto and Prescott, 1988; Slevin and Pinto, 1987). Selection of capable consultants and signing a contract with an institution rather than individuals was key to project success. The consultants and the project manager constituted the Project Implementation Team (PIT) for the GFI project. In addition, the high rating of the personnel factor is also attributed to the role played by the TAG in the project.
Factor 8: Monitoring and feedback
This factor had an average score of 27 out of 35 (constituting 77.1%). It had a joint third highest score with personnel factor. The rating shows that this factor was well managed and was important in determining and assuring GFI project success. The PIP tool confirms the GFI project close-out report’s appraisal of this factor as having been well handled and contributing to effective project execution. The findings confirm the original research that created the PIP (Pinto and Slevin, 1987; Slevin and Pinto, 1986). This factor had a standard deviation of 4.87 and a coefficient of variation of 0.18 indicating relatively low dispersion of project stakeholder responses around the mean.
The importance of the monitoring and control factor in the GFI project can be explained by the fact that the monitoring and control system was simple, inclusive and a demand-driven system. Regular teleconferences, e-mail communication as well as face-to-face meetings were held as a way of monitoring progress and providing feedback (as noted under the communication section). Memos as well as project updates from the project leader were useful in monitoring whether the project was on course or not. This made early identification of risks that impact milestone completion dates possible. The significance of this factor to the GFI project success conforms to similar findings elsewhere (Shenhar et al., 1997; Sterman, 1989; Williams, 2005).
Note that the monitoring and feedback success factor is one factor where there were significant variances in scoring among the project stakeholders. The difference can be explained by the fact that monitoring and feedback were not done consistently and continuously.
Factor 10: Troubleshooting
This factor had an average score of 27 out of 35 (constituting 77.1%) indicating it was well managed and was important in determining GFI project success. Other studies confirm this finding (Belout and Gauvreau, 2004; Cooke-Davies, 2002; Kerzner, 1987; Pinto and Slevin, 1988a, 1988b; Slevin and Pinto, 1986). This factor had a standard deviation of 2.2 with a coefficient of variation of 0.08, indicating that there was little variability among the project stakeholders’ scoring of this factor.
Because the GFI project was executed by experienced consultants, there was little need for troubleshooting. Exceptions were team coherence and data analysis. The consultants provided project leadership. Whenever issues arose, they were communicated through teleconference meetings and solutions developed and implemented.
Factor 3: Project schedule and plan
Project schedule and plans was scored at 26 out of 35 (constituting 75%) by the project stakeholders (this score excludes clients who did not have sufficient information to complete this factor), underlining its importance in GFI project success. This factor has a standard deviation of 2.49 and a coefficient of variation of 0.10, showing low dispersion in the project stakeholder responses around the mean. These findings agree with previous research (Clarke, 1999; Pinto and Slevin, 1988b; Slevin and Pinto, 1986; Turner and Muller, 2005). This implies that this factor was well managed under the GFI project. The GFI close-out report points out that the plans and schedules were well managed. The delays in completing the project were due to initial underestimation of the task.
High average scoring of this factor shows that the planning and scheduling of the project was done accordingly and properly. However, implementation was not smooth. While all the plans and detailed budgets were in place, there was significant time overrun due to underestimation of activity durations. Unanticipated distance and team cohesion issues compromised activity implementation and impacted completion times. The PIP tool only assesses whether plans and schedules were in place. It does not ask if these plans were adhered to. It assumes that if schedules and plans are in place then they will be adhered to and this should lead to efficient project execution. This may be a conceptual gap in the PIP tool.
Factor 2: Top management support
This factor had an average score of 25 out of 35 (constituting 71.4%) showing that it was well managed. This factor had the same score as communication, albeit with a smaller standard deviation of 1 and a coefficient of variation of 0.04, indicating that there was very little variability of project stakeholder’s responses around the mean on this factor.
The importance of this factor in the GFI success can be explained by the fact that despite some initial mishaps, the sponsor’s top management were technically and administratively involved in the project to a great extent. Since the GFI is a novel undertaking which used approximately ZAR 1 million of the sponsor’s resources (constituting almost a quarter of the organization’s annual budget), it was in the best interest of the organization to support the GFI project as much as possible.
However, the relatively low scoring from the project team and project manager could be explained by the fact that as a weak ‘projectized’ structure, the sponsor’s management did not devolve sufficient authority to the project manager and the team, instead they tended to be ‘too involved’ in some instances.
Factor 9: Communication
This factor had an average score of 25 out of 35 (constituting 71.4%) indicating that this factor was well managed, falling in the good category. This finding confirms conclusions published elsewhere (Belout and Gauvreau, 2004; Clarke, 1999; Hyvari, 2006; Pinto and Slevin, 1988b; Slevin and Pinto, 1986). Communication had the same score as top management support, albeit with the second highest standard deviation of 7 and a coefficient of variation of 0.28 indicating greater dispersion of project stakeholder responses around the mean on this factor.
The relatively high average rating of this factor by project stakeholders suggests that it was well handled. The first step in communication planning for the GFI project was to determine the information needs of the project stakeholders, the TAG, consultants and the sponsor. The plan was to update the TAG regarding progress on a monthly basis. The sponsor had to be appraised of project progress including achievement of key milestones on a bi-weekly basis. The communication with the consultants had to be more frequent and on an ‘as needed basis’. Communication facilitated clarifying outputs, methodology and administrative support as well as obtaining updates. Although communication with the sponsor had been planned to be milestone specific, it ended up being process specific in some cases. This helped in ensuring that they remained supportive of the project.
In some cases, the status reports generated issues and comments as well as served the intended purpose of informing. In such cases, more information had to be provided, usually by teleconference.
Factor 4: Client consultation
This factor had an average score of 24 out of 35 (constituting 68.6%) and is within the average/fair category indicating that it was averagely handled and averagely contributed to the success of the GFI project. This factor has the highest standard deviation of 7.9, with a coefficient of variation of 0.33 indicating very high dispersion of project stakeholders’ responses around the mean. The high variability in the scoring shows that the project stakeholders scored this factor differently.
Under the GFI project, the ultimate user is the sponsor and its partners (beneficiaries). The variability in the responses of project stakeholders to this factor may be partially explained by the fact that while consultation with the sponsor was done well, consultation with the beneficiaries was inconsistently done. In addition, the low rating by the internal project team members may also be due to the fact that they were not actively involved in an initial consultation workshop that brought the beneficiary partners together to understand their views and aspirations regarding the GFI.
Factor 7: Client acceptance
This factor had an average score of 20 out of 35 (constituting 57.1%) indicating that it was poorly managed. The study findings contradict findings published elsewhere (Kerzner, 1987; Pinto and Slevin, 1988b; Slevin and Pinto, 1986) The factor had a relatively high standard deviation of 6.9 and a coefficient of variation of 0.35 showing that there is high variability in scoring by the project stakeholders.
While the primary client (sponsor) was involved throughout the project, the secondary clients (beneficiary partners) were only involved at the early stages of the project. This aligns with the observation that client involvement in early stages of projects enhances acceptance. It is generally accepted that involving the client or customer in the project front end is important; examples include new product development (Flint, 2002), construction (Kagioglou et al., 2000) and health care facility design (Tzortzopoulos et al., 2006). In addition, the fact that the sponsor was also the primary client/end user meant that little ‘selling’ to the client was required.
Analysis
The results of the survey instrument used were subjected to the Kruskal–Wallis test. This is used to test the difference in mean responses among project stakeholders. Given the small sample size and violation of normality and homoscedasticity (equal variance) that goes with it, parametric tests such as analysis of variance and regression analysis are not appropriate for analysis.
The Kruskal–Wallis test statistic obtained from the data yielded a value of 23.56. This is compared to a Chi-square table with 4-1 degrees of freedom. The calculated value of 23.56 is significant at the p < 0.001. This means that the null hypothesis of at least two equal means is rejected. The result implies that the differences in the way project stakeholders perceived and scored the project success factors are significant. This supports findings that perceptions of project success differ between stakeholders (Shenhar et al., 1997).
Management implications of results for phase two: Rolling out the GFI
To effectively roll-out the GFI in phase two, the sponsor and beneficiary partners will benefit from harnessing factors that ensured project success during phase one, and implement measures to improve and enhance those factors that were not well managed in phase two. In particular, the authors would advise that when the roll-out of GFI commences, the management should pay specific attention to the areas of client consultation and client acceptance.
They would need to ensure that they engage their partners at local level and ensure they define how they wish to use the GFI in their work. They also need to keep the partners much more involved in the project than was done in phase one of the GFI phase.
Though ranked relatively low as a success factor in phase one, it is anticipated that troubleshooting will become much more prominent in phase two, as partners seek to relate their work to the GFI. The sponsor would need to be more prepared for this phase. Initial as well as follow-up training of the end users and their partners on the importance of GFI in their work will definitely be important. Production of user guides, brochures and flyers in simple local languages will be important in influencing uptake. It is foreseen that tactical factors (consisting of monitoring and feedback, communication, troubleshooting, client consultation, client acceptance, personnel as well as technical tasks) will be more important in phase two than they have been in phase one.
Execution of GFI project has shown that project stakeholders should not be overly concerned about the metrics of time, cost and quality in determining success. Paying attention to the 10 success factors is important in ensuring project success.
In conclusion, this study has used the PIP tool to quality assure the GFI project. The GFI project close-out report was used to benchmark the PIP tool. On the basis of the above analysis, the study concludes that the PIP is a useful tool in providing assurance of project success. The highest ranked PIP success factors are project mission and technical tasks, indicating that they were well managed and hence contributed significantly to the success of the GFI project. The factors that were rated the lowest are client consultation and client acceptance, indicating that they were poorly managed during the GFI project and had the least importance in ensuring GFI project success. The other PIP factors were rated relatively highly and regarded as important and significant in determining and assuring project success.
The study also concludes (on the basis of preceding analysis) that the various project stakeholders have different perspectives on project success based on the PIP tool. This is further confirmed by results of the Kruskal–Wallis test on the differences of the means.
While the GFI close-out report provided a useful basis for evaluating project success, it was not as comprehensive in its cover as the areas addressed by the PIP tool. Against this background, it is further concluded that capturing and appraising of the PIP 10 success factors in the close-out report will strengthen its use as a benchmarking tool.
Future research
Three challenges in using the tool have been noted and further studies should be undertaken on these issues.
First, studies should reassess the conceptual link between PIP questions in each success factor and the definition of the success factor itself. Some of the questions in the PIP framework do not seem to ask and measure what the success factor is defined to capture.
Second, there is a need to include an additional question or rephrasing the current ones under each success factor in a way that captures information on whether the various actions under each factor were consistently done throughout the project or not. This will enhance conceptual alignment of the PIP tool.
Finally, while the PIP tool helps us to understand the generic factors important for project success, more studies are needed to analyze success factors among projects in the development sector to identify ‘within-group’ variability. For example, within the NGO sector studies could focus on sub-sectors such as humanitarian assistance, research and human rights. Insights from these sub-sectors could be used to develop sector- and sub-sector-specific PIP instruments.
A last important point in regard to the use of the PIP tool in the NGO sector is its relationship with the Logical Framework. The project used to evaluate the PIP tool as discussed in this article is in many ways a more conventional project than may be the case with many aid and development projects. For this reason, there is scope for additional research to see how PIP can be integrated with the Logical Framework tool and also to assess its suitability where social development goals are the main focus of the project being evaluated.