Customized project charter for computational scientific software products

1. Introduction

Quality in engineering is a complex and highly context-dependent concept and has numerous definitions and approaches as expressed in different areas of engineering. In fact, there is no single agreed-upon definition for quality [1].

The engineering process for the development of a software product – software engineering – is meant to systematically lead us from an initial conception of what the software is meant to do to an implementation that correctly meets the objectives assigned to it [2, 3]. Different domains of software applications require that the development process be customized to its specific conditions and needs, while the engineering development process entails the initiation, planning, execution, monitoring, control and finalizing of those activities necessary to meet the project’s requirements [4].

According to IEEE [5], software quality is defined by the degree to which a system, component, or process meets specified requirements as well as customer or user needs or expectations. Pressman [6] defines it as conformance to explicitly stated functional and operational requirements, conformance to explicit development standards, and conformance to professional needs. According to Gurla and Lin [7], software quality is determined by three main factors: organizational, technological, and individual. McCall et al. [8] categorize software quality factors into three sets: product operation (e.g., functionality), product revision (e.g., testability), and product adaptability (e.g., reusability). Similarly, the metric structure for the evaluation of quality varies and therefore cannot be a single acceptable measure of product quality. Essentially, there are two main concepts as to how to approach quality improvement: the “product base” approach and the “process base” approach [9]. In the “product base” approach, as it is commonly perceived, “quality software” is software where all the characteristics comply with the requirements. In practice, the characteristics that are not related to the requirements are also important. Therefore, not only the presence of certain characteristics is important; a lack of features will also affect the quality of the product [9]. On the other hand, software quality is highly influenced by the development process, as is argued by the proponents of the “process base” approach. Failures in the requirements definition process, failures in developer-customer communication, failures in the design process and failures in the quality assurance process are some of the most important factors affecting the quality of a software product [10]. Clearly, a single operational definition of the key performance indicators for the measurement of the quality of both the development process and software product quality is problematic, limited and almost impossible.

Given the absence of a single operational definition for the measurement of quality, one way of overcoming this difficulty in the development process is to focus attention on the drivers that advance quality value, i.e., QVD – quality value drivers. QVD is a managerial approach (practices, roles, ceremonies, artifacts) that delivers quality value to the organization, process, and method. The Value “V” is what we want to achieve, e.g., the improvement of the team’s knowledge; the driver “D” is the managerial approach (practices, roles, ceremonies, artifacts) that helps us to achieve the value; and the quality “Q” is the metrics and the method that help us to measure the improvement that we achieved by adopting the driver. By way of analogy, just as a scientist need not have an operational definition of truth in order to seek truth – and instead might stress the importance of rigor, methodological and critical thinking among the familiar value drivers toward truth – similarly, the quest for quality value starts with a clear understanding of those variables that actually create quality value in a significant way: i.e., the key quality value drivers. In the business world, managers seek business value drivers of different kinds, such as financial value drivers, customer value drivers, operational value drivers and so forth. The identification of the key quality value drivers needed for a given software product and which are relevant for a fuller understanding of the goals and objectives of the development process, is a complex process. It requires the prioritization of individuals and team interactions over formal processes and tools, of piecemeal development of working products over comprehensive grand project documentation, of stakeholder collaboration and involvement over contract negotiation, and of the ability to respond to change over a reliance on comprehensive plans.

One of the most complex, most important and commonest domains of software products is the scientific application. Computational science is a science that provides hardware and software solutions to scientific problems [11], with the focus on efficiency, while the narrow realm of scientific programming in computational science is concerned with the development of algorithms and codes to solve problems in science and engineering. While, scientific computing can be defined as: “…the collection of tools, techniques and theories required to solve on a computer mathematical models of problems in science and engineering…In summary scientific computing draws on mathematics and computer science to develop the best way to use computer systems to solve problems from science and engineering [12]. “ The scientific programming development process generally begins with a code development process, in which the scientist – in this case, the “computational scientist” – begins with the development of an in-house computational code [13, 14]. However, scientific systems tend to be complex, with unexpected relationships and dependencies that often make the development of a scientific program (i.e., the creation of executable sequences of instructions) an onerous and arduous task, requiring not only programing and coding, but also the collecting and collating of computer instructions (e.g., database, library, framework, OS, etc.). The complexity of the computing, as well as that of the research and coding, along with the need for high-performance computing applications [2], in many instances prompts the “computational scientist” to require the intervention of a professional who specializes in proven approaches and tools – i.e., the “software engineer” – to develop a software product that will be efficient and will be capable of dealing with a large computational component, while, at the same time, ensuring maintainability, reusability and transparency for users and developers [11].

The development of scientific software products (SSP) is the unique feature of computational science that is used for the construction of mathematical models and which involves the employment of quantitative analysis techniques in order to assist scientists in analyzing, visualizing and simulating processes and data [15]. These software products are geared toward scientific computations in which the SSP development life cycle is focused on the design, analysis and development of algorithms and software with a computational component that can model phenomena, solve problems and provide data for decision support [16] in science and engineering [17, 18]. In the adoption of the “process base” approach to quality for the modification or adjustment of a suitable engineering development process for an SSP, one must reveal and confront the unique difficulties as well as the gaps and the origin of the gaps that characterize the development process for specific domains, where the origin in the case of an SSP can arise from technical complications as well as from behavioral complications (among stakeholders). From previous studies [18, 15, 2, 19, 17, 13] and from the extensive research we have performed, we have come to the conclusion that the development process of scientific software products has unique features with complications that, in addition to the problems inherent in software development, include uncertainties and constraints stemming from various scientific processes as well as from mathematical difficulties.

Our group, the NMCRC (Negev Monte Carlo Research Center) brought together scientists from a variety of fields in order to develop Monte Carlo simulations – “…a branch of experimental mathematics in which one uses random numbers to conduct experiments [20] “ – for different scientific spheres, utilizing the requirements, rigors, and standards of software engineering. The goal of the Negev Monte Carlo Research Center is to combine research in physical, environmental, biological, chemical, and medical models with proven software capability, while ensuring software quality, software development, methodology, and advanced testing standards. Over the years, the scope of the NMCRC center expanded from Monte Carlo simulation to the applied Computational Science, and Engineering Center, which exploits the power of computation to provide scientific software products for a variety of scientific and engineering fields under proven principles, approaches, tools, and standards. In fact the suggested tool serves has been adopted as proven QVD tool for a variety of projects and ensures success. In fact, unique challenges in the development of an SSP exist at every stage of the development process. Notable constraints and uncertainties stem from varying prespectives in attitudes and expectations that can easily be identified and remedied with the use of a proper protocol. The addressing of the varying perspectives gap at the very outset of the development process will reduce unwanted dynamics throughout the development process. There is thus a need for the customized implementation of development practices and methods that will be suited to the specific conditions and needs in the development process of a given scientific software product and which will thereby ensure its quality.

2. Discussion

As part of a larger study seeking to explore the quality enhancement factors in software engineering, we conducted an online survey in January and February 2015 [21]. We used a questionnaire survey to assess the opinions and concerns of software practitioners regarding the connection between “quality” and “process approach” factors in their working environments and practices. The online survey consisted of three parts: a demographic questionnaire, professional questionnaires related to the profession of the respondents, and the study, which presented statements that reflected the attitudes of the respondents about the factors required to make a “process base” approach an appropriate perception for the ensuring of success in software development. The premise of the survey was that different domains of software applications need a customization of the development process in accordance with their specific conditions and needs; thus the survey considered only a generic “process approach”, focusing on a search for the factors that can generate success. The method used in the formulation of the statements did not disclose the subject matter of the survey, i.e., the premise and the hypothesis of the survey; therefore, the survey does not mention two terms: “quality” and “process approach factors”. The sample comprised 130 participants, 86% of whom worked at the time of the survey in the software development industry and in the academic community. The participants were from different places around the world and from a broad range of different organizations, and represented a wide variety of attributes and characteristics. Over four fifths (87%) of the respondents had a university education while three-quarters of them were academics whose subject area was computer science or software engineering.

The results of this survey revealed some noteworthy findings. The vast majority of the participants (77.09%) agreed with the statement that software engineering is a relatively young engineering field characterized by the lack of an agreed-upon theoretical and quantitative basis on which one could establish proven and binding processes. Therefore personal experiences as well as modifications to methodologies and procedures play a crucial role in the success of software development projects. Over three quarters of the respondents (78.46%) agreed with the statement that, in order to create a successful software development project, the software engineer must employ technical, engineering and professional skills, as well as social skills, for example, in interpersonal communication. Overwhelming support (76.17%) was expressed for the statement that familiarity with the ramifications of every action taken in the development process is a necessary precondition for the process’s success, while a lack of awareness of such ramifications is one of the factors that can harm the development process.

These survey results complement, support and reveal two previous significant findings. First, among the many known factors for success, three generic “process approach” factors can be identified: experience (in the specific domain), awareness (of difficulties, factors and gaps), and modification (of methodologies and procedures). Second, modification of the software development life cycle involves three elements: the “human factor”, an appropriate working environment, and the detection of factors that could positively influence the success of the project. The meaning of software quality, how we assess it, and how we generate and improve it are important elements, as noted above. There is no agreed-upon definition and there are no agreed-upon approaches for software quality, nor is there any single operational definition of the key performance indicators for measuring the overall quality of a process or product; therefore we will employ the concept “generation of quality”, which will mean the achievement of quality is by enabling a set of local modified quality generators with an inevitably relative impact to the entire quality. The modified generators are practices, artifacts roles and the like, that modified to the specific domain and cause a local significant improvement.

The scientific software product (SSP) development process can be characterized as complex, confusing, highly sensitive to parameter changes, expensive and dangerous and often exhibiting a wide gap between developer and client. Much of the literature [2, 22, 23, 19, 13] deals with factors that affect scientific software development, while many articles emphasize the presence of several wide chasms between general software development and scientific software product development [15, 13, 24] throughout the entire software development life cycle. Among the various difficulties, factors and gaps, one can quite easily identify a “conceptual gap” between developer and client. The first hindrance arises when the two parties use terms and statements that, because of their extensive knowledge in the required scientific field, are familiar to some participants, while others cannot grasp their exact meaning or the conclusions they generate. Quite often, one party is not thoroughly knowledgeable regarding the difficulties the other party confronts (both scientific and technical). Therefore, in many cases, exaggerated and unrealistic expectations may arise. In other words, the software development process itself can involve many discrepancies, differences and hindrances.

Previous works [16, 25, 26, 24, 15] have pointed to the absence of a uniform understanding of both the software development process and the objectives of the various software development phases. For example, there is a gap between developers and clients as to their understanding of the importance and objectives of software verification and testing. Most scientists have a solid grasp of the goals of software verification and comprehend its importance [16, 25]. On the other hand, it was found that many scientists do not “have [a] sufficient understanding [of] testing concepts” [16, 25]. Thus, from the scientist’s perspective, the tool most commonly used to assess software quality is testing. In other words, scientists use software testing as the main tool for the assessment of software in the light of known and trusted empirical and experimental results and in comparison with other models and with similar software products. Software developers, by contrast, treat testing as an important element of the quality assessment process, but do not necessarily see it as the most important one.

Complexity and hindrance can arise because of the place occupied by the SSP within the scope of the entire research project. From the point of view of the client (the scientist), the creation of the software is not the end goal but is simply one element in the whole process, with the actual goal being the particular scientific research study in its entirety. On the other hand, from the perspective of the software developer, the software itself is the goal. Complexity stems from the place the software development project occupies as part of the overall research project. Essentially, there are two distinct objectives in the initiation stage of a software development project: the development process and the software product.

Thus, if the development process is seen as the objective, then the motivation of the client involved in the underlying science (i.e., the scientist as the end-user) is to use the software development process as part of the entire research project and to use the software development as an operations research technique in order to explore and better understand the mechanisms involved in the corresponding real-world dynamics. Recent developments in operations research include the use of many different techniques that including also computer simulation [27]. In such cases, the client usually understands what to expect when the process will be completed, but generally only toward the end of the process; therefore the specifications for the software and most of the requirements will be discovered in the course of the project [26]. This fact creates problems with respect to the definition of the project’s boundaries, both inclusive and exclusive, and also makes it difficult, if not impossible, to ascertain whether those boundaries are respected at every point in the process [19]. This is a serious challenge to the development process, which thus requires close cooperation and flexibility in the introduction of changes.

On the other hand, if the software product is perceived as the objective, then the scientist may have some idea of what must be done and what achievements to expect, while not necessarily knowing the precise results of the actions that will be undertaken in the course of the research project; thus, the scientist will have to know in advance what results will be provided by the final software product itself. For example, there might be a situation where the developer needs to develop a software product for the comp design and simulation of a prosthetic limb for a specific individual. The scientist understands the physical situation, knows that its description will require the application of multiple partial differential equations, can assess the results, but will have trouble evaluating the interim results. Such an objective encourages a development process that emphasizes orderly, detailed requirements, design and testing.

Essentially, the success of the development project of a scientific software product will depend on the extent to which these conceptual gaps (between the developer and the client) can be bridged, while the successful completion of the development project will be measured not only in terms of the fact that the project has been completed, but also in terms of the product’s quality, i.e., the scientific value that will be provided when the product has been completed and which will stem from any improvements made in the organizational process, i.e., the quality driver. As we see it, helping the software engineer recognize – as early as possible (that is, before the commencement of the development process) – the problems that could arise from the gaps between the developer and the client with regards to both perception and knowledge is the key element to the “generation of quality” in the development process of a scientific software product. The project management planning process aims to define and refine the project’s objectives, as well as to organize, manage and plan tasks necessary for the achievement of success – namely, the project’s successful completion [4].

One of the primary tools required for the initiation and authorization of software development projects is the “project charter”. The project charter [4] is a formal statement concisely outlining what it is to be done and why, and what value the software product will provide when completed. The primary purpose of the project charter is to provide official authorization for the project by presenting the scope, objectives, vision, and responsibilities of the participants and stakeholders, and to align these factors with the project’s purpose. In practice, through the modification of the “project charter” for different domains, it can serve as the initial quality driver tool to identify and overcome constraints, uncertainties and gaps. The goal (the value) of the suggested project charter in the development of an SSP is to generate quality through the early detection and definition of possible conceptual gaps, so that a rough idea can be obtained of the scientific value that will be provided when the product has been completed and so that these conceptual gaps can be bridged through the adoption of other practices, tools or methods for the ensuring of success. We present below a framework for the modification of a project charter document for a scientific software product – the driver. Since such a framework provides the guiding principles [28, 29] needed for the initial stage, the parties will be in a better position to evaluate their ability to meet the demands of the software’s development. The proposed framework will generally outline the basic components that characterize scientific software product development in general and which must be acknowledged so that the conceptual gaps that could hinder the process can be bridged. These components can be adopted as – is, modified, ignored, or supplemented with additional elements. The purpose of the modified project charter for a scientific software product is to document the necessary information required (goals, objectives, needed knowledge, the significance of the development process as part of the overall research study, and the relationship between the stakeholders), to obtain the approval of this scientific software product for the operations research and to ensure that all the stakeholders are aware of the purpose and objectives of the SSP’s development. Although there are various approaches and documents to the development of a Project Charter, the following approach has proven itself to be specific enough to get the project on the right track and yet narrow enough so as not to be an unnecessary hurdle for SSP development.

A generic modified framework for a project charter for an SSP project will comprise five parts:

A. Rules:

•

The goal of the scientific software product

•

Declaration of concepts: Testing verification and quality

B. Overview of the scientific software product in question (assessment of the SSP’s scientific value upon its completion):

C. Scope – the need for a clear understanding of exactly what is to be done:

1. 1.

Stakeholders

2.

Objectives (choose one)

–

The development process as the objective

–

The software product as the objective

–

Other

3.

Assumptions

4.

Constraints

5.

Boundaries

D. Quality

1.

Is there another alternative (i.e., the utilization or reuse of existing software)? Yes/No. If yes, please specify:

2.

Criteria for success

3.

Verification (choose one method and add references)

–

Comparison with similar software

–

Comparison with known empirical results

–

Comparison with known experimental results

–

Other:

E. Glossary

3. Case study: Simulation software example – fiber evanescent wave spectroscopy (FEWS)

One of the most widely used forms of operation research is the development of a simulation software product, whereby researchers can develop techniques for the use of computers to numerically evaluate the operation of a variety of systems and to form a better understanding of corresponding real-world dynamics. In this paper, we demonstrate how to specifically utilize and modify a project charter in the development of a fiber optic evanescent wave spectroscopy (FEWS) simulator [30, 31, 32], an interactive product used to simulate the optimal design of a device for the spectroscopic analysis of tissue. The software product is based on FEWS technology and utilizes the Monte Carlo method as its basic computing algorithm.

Evanescent wave spectroscopy has become a widely used technique for the examination of the properties of materials, mostly within the infrared range of the electromagnetic spectrum [33, 34, 35, 36]. Its advantage derives from the phenomenon of attenuated total reflection. Fiber evanescent wave spectroscopy (FEWS) has been extensively used in biomedical skin diagnoses and chemical sensors.

A medical physics research group specializing in the design and manufacture of fiber optics turned to our research team with the request that we develop a computational simulation tool [32] that could help them design the optimal optical fibers they needed for their research. The client – in this case, the research group – required a computational tool for the assessment of the feasibility, design and development of new types of optical fibers. Specifically, the client required scientific software that would enable the exploration of new compounds, geometric structures, and spatial arrangements in the development of optical fibers – for the purpose of achieving maximum measurement accuracy. The client planned to use the simulation tool for conditions that cannot be applied today due to technological limitations and wanted to employ the simulation tool to evaluate the feasibility of the development of certain optical fibers. The client was interested in using the results from the advanced simulation tool to design fibers for medical purposes. The accurate and error-free performance of such fibers would be crucial because they could serve as an essential component in a tool used for the forming of a clinical diagnostic picture of a patient. It was therefore vital that a suitable and effective criterion for accuracy be determined and that a test procedure be formulated for it. In accordance with the client’s specifications, the end user would be a physicist from the research group, a specialist in fibers made of crystalline silver halides for use in middle-infrared (mid-IR) spectroscopy. The team also requested that the software product utilize the Monte Carlo method for simulation purposes [37]. The client emphasized the fact that its members did not possess the knowledge or ability to examine and test the computational tool and that the software tools would be tested and evaluated only after the delivery of the final results of the simulations. Furthermore, the research would be carried out against a well-known experimental benchmark – a complete three-dimensional geometry (shape and structure) of tapered optical fiber material demonstrating the absorbance of the evanescent field of an infrared signal [32] in fibers made of crystalline silver halides.

These parameters created huge challenges for us, a research group comprised of computer scientists, physicists, mathematicians, and one biologist, working in close collaboration with a development team consisting of software engineers. While the members in the research group and the development team collectively possessed rich background knowledge allowing them to cope with the development requirements, they lacked experience in, and a deep understanding of, this specific field of research. Therefore, the requirements presented many potential failure routes for the developers. For example, even if a simulation program is written in a precise and correct way, the result produced by the simulation program may not conform to the prescribed experimental benchmark. In such an instance, it would be difficult for the developer team to identify which element of the simulation failed: whether the difference in results was caused by inaccurate algorithms, coding errors, false constants or imprecision in the application of the Monte Carlo technique, or whether the incongruity arose through the use of imprecise, incorrect formulae or because of constants obtained from previous measurements. In summary, this could have happened for a variety of reasons, including choice of initial conditions (The key aspects are italicized and printed in bold for the reader’s convenience. To avoid excessive detail and for the sake of brevity and clarity, we have whittled down the content to its bare essentials).