Abstract
Quality, architecture, and process are considered the keystones of software engineering. ISO defines them in three separate standards. However, their interaction has been scarcely studied, so far. The SQuAP model (Software Quality, Architecture, Process) describes twenty-eight main factors that impact on software quality in banking systems, and each factor is described as a relation among some characteristics from the three ISO standards. Hence, SQuAP makes such relations emerge rigorously, although informally. In this paper, we present SQuAP-Ont, an OWL ontology designed by following a well-established methodology based on the re-use of Ontology Design Patterns (i.e. ODPs). SQuAP-Ont formalises the relations emerging from SQuAP to represent and reason via Linked Data about software engineering in a three-dimensional model consisting of quality, architecture, and process ISO characteristics.
Keywords
A three-dimensional view on software quality
Industrial standards are widely used in the software engineering practice: they are built on pre-existing literature and provide a common ground to scholars and practitioners to analyze, develop, and assess software systems. As far as software quality is concerned, the reference standard is the ISO/IEC 25010:2011 (ISO quality from now on), which defines the quality of software products and their usage (i.e., in-use quality). The ISO quality standard introduces eight characteristics that qualify a software product, and five characteristics that assess its quality in use. A characteristic is a parameter for measuring the quality of a software system-related aspect, e.g., reliability, usability, performance efficiency. The quantitative value associated with a characteristic is measured employing metrics that are dependent on the context of a specific software project and defined following the established literature.
The ISO quality standard only focuses on the resulting software product without explicitly accounting for the process that was followed or the implemented architecture. However, there is wide agreement [41] about the importance of the impact of three combined dimensions: software quality, software development process, and software architecture, on the successful management and evolution of information systems. In this respect, the industrial standard ISO/IEC 12207:2008 defines a structure for the software process life cycle, and outlines the tasks required for developing and maintaining software [47]. Regardless of the chosen methodology (i.e., Agile or Waterfall ones [41]), this standard identifies the relevant concepts of the life cycle and provides a useful tool for software developers to assess if they have undertaken all recommended actions or not. Each lifecycle concept can be evaluated according to its maturity level through established metrics, e.g., the Capability Maturity Model Integration (CMMI) [15]. As for the architectural dimension, the ISO/IEC 42010:2011 standard provides a glossary for the relevant objects of software architecture. Concerning software architecture evaluation, intended as a way to achieve quality attributes (i.e., maintainability and reliability in a system), some approaches have emerged, the most prominent being ATAM, proposed by the Software Engineering Institute [4,5,14,31]. Typical research in this domain is about how architectural patterns and guidelines impact software components and configurations [23]. A survey study [17] analyzes architectural patterns to identify potential risks, and to verify the quality requirements that have been addressed in the architectural design of a system.
The mutual relations among the three dimensions and their impact on the quality of software systems have been barely addressed in the literature, but a recent empirical study [44] in the domain of Software Banking Systems pointed out the importance of those relations. The study involved 13 top managers of the IT Banking sector in the first phase and 124 additional domain experts in a second validation phase. The result of such a study was a model named SQuAP that describes these relations in terms of quality factors. According to [25], the information available to guide and support the management of software quality efforts is a critical success factor for IT domains. Considering the broad coverage of its empirical provenance, a formalization of SQuAP may serve as a reference resource and practical tool for scholars and practitioners of software engineering, to assess the quality of a software system, to drive its development to meet a certain quality level, as well as to teach software engineering.
The SQuAP model builds on the concept of quality factor: a n-ary relation between software quality characteristics that cover the three dimensions of software product, process, and architecture, based on the three reference standards ISO/IEC 25010:2011, ISO/IEC 12207:2008, and ISO/IEC 42010:2011 respectively. A SQuAP quality factor can be described as a complex quality characteristic (or parameter) that provides a three-dimensional view for assessing software quality. The model identifies twenty-eight quality factors.
Our contribution consists in a resource named SQuAP-Ont, an ontology that formally represents the concept of quality factor by reusing existing ontology design patterns (e.g., Description and Situation [19,42]), instantiates all factors identified so far, and axiomatizes them in order to infer measurable factors based on the characteristics available at hand. Besides, the ontology has been annotated with OPLa1
In the rest of the paper, after discussing relevant related work (Section 2), we provide additional details about the SQuAP model by presenting two sample factors in Section 3. We describe the SQuAP-Ont ontology: its main concepts and axioms, the adopted design methodology and the reused ontology design patterns in Section 4. In Section 5 we provide examples of how to use it; and discuss the resource potential impact (Section 6) before concluding and identifying future developments, in Section 7.
The use of ontologies in the software engineering domain is very common [9,29,51]. The ISO standards referenced in Section 1 have been the subject of several ontological studies. For example, useful guidelines for their ontological representation are proposed by Henderson et al. (2014) and Gonzalez et al. (2016) [24,27].
An ontology-based approach to express software processes at the conceptual level was proposed in Liao et al. (2015) [34] and implemented in e.g., Soydan & Kokan (2006) [48] for CMMI [13]. Software quality attributes have been modeled in Kayed et al. (2009) [30], while an ontology for representing software evaluation is proposed in Castro et al. (2010) [12]. Finally, a formalisation of the ISO 42010, describing software architecture elements, is developed in Emery & Hiliard (2009) [18] and in Kumar & Prabhakar (2010) [32]; and Antunes et al. (2013) [2] argues that different architecture domains can be integrated and analyzed through the use of ontologies.
Most of the works mentioned above focus on a strict representation of standards in terms of ontologies [13,18,34,48]. Other scholars [12,30] provide only preliminary ontological solutions for modelling quality characteristics or software evaluation and, to the best of our knowledge, they overlook the reuse of ontology design patterns. In contrast, our work focuses on the relation between the different ISO standards (system quality, software development process, and software architecture) for supporting the assessment of software system quality, with the added value of following a rigorous pattern-based ontology design approach.
Also at a higher level, an ontology to harmonize different hierarchical process levels through multiple models such as CMMI, ISO 90003, ITIL, SWEBOK, COBIT was presented in Pardo et al. (2012) [39].
Ontologies referred to software quality focus primarily on quality attributes [30]. One quality evaluation, based on the ISO 25010 standard, is enhanced by taking into consideration several object-oriented metrics [37]. Similarly, Castro et al. (2010) reuse current quality standards and models to present an ontology for representing software evaluations [12].
Similarly, scholars advanced an ontology for the ISO 42010 standard regarding software architecture [18]. An ontological representation of architectural elements has also been expanded by Kumar & Prabhakar (2010) [32]. With particular reference to the architecture rationale, some visualization and comparison techniques with semantic web technologies have been proposed in literature [36]. Moreover, scholars showed that different architecture domains could be integrated and analyzed through the use of ontologies [2]
Finally, the Semantic Web community proposed also guidelines regarding the representation of ISO standards of software engineering with ontologies [24,27]. These two papers use a domain ontology, proposing the creation of a single underpinning abstract domain ontology, from existing ISO/IEC standards. According to the authors, the adoption of a single ontology will permit the re-engineering of existing International Standards as refinements from this domain ontology so that these variously focused standards may inter-operate.
Relational quality factors: The SQuAP model
The motivation for developing SQuAP is based on the understanding, in the software engineering and information systems communities, that assessing software quality for contemporary information systems requires to take into consideration the relations among different dimensional perspectives, namely: software quality, process, and architecture [41]. Accordingly, we conducted an empirical study in the banking sector [45]. The financial and banking industry is particularly useful to explore information systems quality issues for several reasons. First, it is mission-critical, i.e., the failure of even one system would lead to unpredictable consequences. Thus, the top manager showed increasing concerns about the quality and sustainability of their information systems [44]. Secondly, most institutions, such as those involved in the study, are multinational, sharing the same systems and the same concerns. Finally, this sector is highly connected and regulated centrally by banking authorities. So, many functionalities and requirements are well defined by such authorities, which means that the entire industry is using similar software products.
Therefore, to develop the SQuAP quality meta-model, we executed our research according to the following steps. In the first phase, based on the Delphi method [16], we involved 13 top managers of this sector to express their most significant software quality concerns. The result was a set of distinct 28 quality factors emerging from the elicited concerns, after a consensus-based negotiation phase (part of the Delphi method). In a second phase, we involved 124 domain experts that validated the 28 factors with a high level of agreement. Each factor has been then linked to several characteristics or elements defined in the three different standards, i.e., ISO 25010, ISO 42010, and ISO 12207, for software quality, architecture, and process respectively. We followed the theoretical coding approach by Strauss & Corbin (1997) [49] to map the factors to the ISO standards.3
Standards are de facto second-order theories, built on grounded pre-existing ones and shared among scholar’s and practitioner’s communities.
Factor 26: data analysis vs. functional analysis. This factor is defined as a relation between three quality characteristics of a software project: Functional Correctness (ISO 25010), Architectural View (ISO 45010), and Development (ISO 12207).
The selection criteria and demographics of the experts involved in both phases, the inclusion and exclusion criteria, the agreement figures as well as the industry coverage are explained in Russo et al. (2018) [45]. In the same paper, we provide further details about the methodological approach and an exhaustive description of the 28 factors. A list of them are also published on the resource website4
The 28 SQuAP factors have been rigorous, although informally defined in our previous work [45]. These definitions are the primary input for the development of SQuAP-Ont; hence, it is relevant to report here at least one them. The definition of a factor consists of a set of quotes from the experts involved in the study followed by an analysis (based on theoretical coding) on what are the main characteristics and elements from the standards that emerged as components of the factor. We choose randomly one factor (26) to explain the underlying logic of the factor mapping.
Factor 26: Data analysis vs Functional analysis. This factor explores whenever poor data analysis influences functional analysis and so, system integrity. “The “functional centric” view is quite misleading since it relegates the importance of data. Data give the “static view” of existing functionalities, which is very important for the functional analysis. One software product may have all possible functionalities required by the user but lacking fundamental data its deployment and system integration becomes impossible”, said one surveyed expert. Moreover, “data analysis skills are generally lacking and poorly used in functional analysis”, affirmed another one. This issue is present market-wide. “In my opinion, in the market, there is a lacking perception about the importance of data analysis as the preliminary phase of functional analysis”. However, other experts disagree. “Saying that poor analysis is due to poor data understanding is a quite generic (it is obvious that data processing is the main IT goal) and old issue”. Other issues are also relevant to understand the factor. “Also knowing what different data means is important”. Also, “it is not only an issue of poor technical skills”. Furthermore, “personally, I saw poorer knowledge of bank’s operation processes”. There was a shift after 2010, which give fascinating insights. “It is true for applications developed before 2010. Data governance is now more relevant, and data analysis is performed before the functional one. So I see a clear discontinuity with the past”.
Theoretical coding analysis: experts stressed the importance of data and functional analysis, impacting on the dimensions of Functional Correctness (software quality), Development (software process), and Architecture View (software architecture). They refer to the functional suitability of applications through correctness. This impacts the development process, which is supported by such an analysis. The architecture view addresses the concern of a suitable system held by the system’s stakeholders. Figure 1 shows a graphical representation of Factor 26.
In this section we first provide details about the ontology design methodology adopted (cf. Section 4.1), then we describe the SQuAP Ontology (cf. Section 4.2) and we provide its formalisation (cf. Section 4.3). Finally, we provide the implementation details of the ontology (cf. Section 4.4).
Design methodology
The XD methodology as implemented for modelling the SQuAP ontology.
The SQuAP Ontology (SQuAP-Ont) is designed by reusing ontology design patterns (ODPs) [22] according to an extension of the eXtreme Design methodology [6]. We opt for an ODP-based methodology as there are shreds of evidence [6] that the reuse of ODPs (i) speeds up the ontology design process, (ii) eases design choices, (iii) produces more effective results in terms of ontology quality, and (iv) boosts interoperability. This extension has been first described in [43] and mainly focuses on providing ontology engineers with clear strategies for ontology reuse. According to the guidelines provided by [43], we adopt the indirect re-use: ODPs are reused as templates. Hence, ODPs are specialized by specific ontology modules developed as components of the final ontologies instead of being directly reused and linked by those modules. For example, if we want to reuse the object property
The XD extension implemented in this work has been successfully adopted in several ontologies and linked open data projects so far. Examples are [11,20,35,38,40]. This extension implements an iterative and incremental approach to ontology design that involves three different actors: (i) a design team, in charge of selecting and implementing suitable ODPs as well as to perform alignments and linking; (ii) a testing team, disjoint from the design team, which takes care of testing the ontology; (iii) a customer team, who elicits the requirements that are translated by the design team and testing team into ontological commitments (i.e., competency questions and other constraints) that guide the ontology development. Figure 2 depicts the UML activity diagram that represents the XD extension described in Presutti et al. [43] as implemented in this work. The diagram is extended by explicitly identifying the actors that are associated with each action they are involved in. The first action of the methodology is the collection of requirements that involves both the customer and the design team. At this stage, the requirements are recorded as stories, which are typically used in agile software development for communicating requirements from the customer to the development team. After requirement stories are recorded, the design team starts to transform them into competency questions [26] (CQs). CQs represent the ontological commitments that drive the ontology development. Table 1 reports the CQs, identified by analysing the SQuAP model (cf. Section 3) and by discussing with domain experts (i.e., the customer team in our context).
Competency questions used for modelling SQuAP-Ont
The next action is about the design team looking for possible ODPs to reuse by analysing the resulting CQs. Those ODPs, if available, are reused for designing the modules of the ontology that addresses the specific CQs. When an ontology module is ready, the testing team performs the validation by assessing its fulfilment to the CQs. This validation is performed by (i) converting the CQs to SPARQL queries and (ii) executing those queries on a data sample, which is modelled according to the target ontology module. If the validation is successful, the design team integrates the ontology module in the final ontology. Additionally, the design team provides alignments with related external ontologies and vocabularies in the Semantic Web for boosting interoperability. Then, the testing team performs a set of integration tests aimed at (i) validating the logical consistency of the ontology and (ii) its ability to detect errors by deliberately introducing contradictory facts. Both checks can be performed by using a DL reasoner, such as HermiT,5
Figure 3 shows a diagram of SQuAP-Ont. We use the namespace https://w3id.org/squap/ . SQuAP-Ont re-uses as templates the following ontology design patterns [42]: Description and Situation (D&S)7
Core classes of SQuAP-Ont.
The D&S pattern allows representing the conceptualisation of a n-ary relation (i.e., description) and its occurrences (i.e., situations) in the same domain of discourse. For example, it is used for representing the description of a plan (D) and its actual executions (S), the model of disease (D, e.g., its symptoms) and the actual occurrence of it in a patient (S), etc. SQuAP-Ont reuses this pattern for modelling quality factors with the class
In D&S each entity that is in the setting of a
As aforementioned, SQuAP-Ont is annotated with the OPLa ontology [28] for explicitly indicating the reused patterns. We use the property
Finally, SQuAP is aligned, using an external file, to DOLCE+DnS UltraLight.9
Alignments between the classes of SQuAP-Ont and DOLCE UltraLight
Alignments between the properties of SQuAP-Ont and DOLCE UltraLight
The following is the formalisation of SQuAP-Ont described in Section 4.2. The formalisation is expressed in Description Logics. For brevity, we use the terms
Implementation details
The namespace
https://w3id.org/squap/
identifies the ontology and enables permanent identifiers to be used for referring to concepts and properties of the ontology. We define individuals’ URIs with the name of their types (e.g.,
The alignments with DOLCE+DnS UltraLight (DUL) are published in a separate OWL file,10
Flexibility is among the most relevant characteristic of this ontology. Although the higher levels of this ontology, regarding the standard and the factors mapping, are fixed, its measurement model can be adapted to the most suited scenario. In particular, the proposed way to evaluate information systems characteristics is just an illustrative example, which can be adapted to for any assessment purposes.
As a usage example of the SQuAP ontology, we show a real-world example consisting of the evaluation of a banking application employing the Goal-Question-Metric (GQM) approach [3]. GQM defines a measurement model on three levels, i.e., Conceptual level (Goal), Operational level (Question), Quantitative level (Metric). This method offers a hierarchical assessment framework, where goals are typically defined and stable in time, and metrics may be adapted according to new measurement advances. So, we stress the fact that this paper does not focus on the measurement model, preferably on the knowledge representation of SQuAP for assessment and benchmarking purposes. So, we provide the following synthetic RDF data about the assessment. The data are expressed as RDF serialised in TURTLE.14
The RDF is available at
The example describes a banking system associated with three assessments about the dimensions of software quality, architectural alignment, and process maturity. The specific measurement results are:
In this example, different standards’ items represent the Goals, which are measured with one or several (also concurrent) software quality metrics. To do so, we followed literature recommendations [10,50]. The result is the sum of different evaluators, which represent a measurement of the three standards.
Execution of a DL query on the RDF sample.
Alternatively, it is possible to define productive rules to materialise the factors that are enabled by the available measured quality characteristics. The following SPARQL CONSTRUCT is a possible productive rule for our example.
We remark that factors and quality characteristics are defined in SQuAP-Ont both as classes and individuals through OWL punning. Hence, one can decide to use DL reasoning or rules defined in any other formalism depending by the specific case, e.g., SPARQL CONSTRUCT, Shapes Constraint Language15
Another worth presenting example is a dogfooding usage scenario. Dogfooding is when an organization uses its product for demonstrating its quality. In this example, we use the SQuAP-Ont to record the metrics, the values, the factors, and the quality characteristics resulting from the measurement of its characteristics. The data of this example are expressed as RDF serialised in TURTLE.
In the dogfooding example the ontology is used for recording a measurement result (i.e.,
In the last decade, there has been a considerable effort, especially by the Management Information Systems research community, to study the phenomenon of the alignment of business and information systems [1]. What emerged is the importance of such alignment for both business’ competitiveness and technical efficiency. When it comes to integrating new solutions, modules, or interfaces, such alignment is of crucial importance. Several other scholars found similar results, suggesting the importance of standard governance defining key architecture roles, involving critical stakeholders through liaison roles and direct communication, institutionalizing monitoring processes and centralizing IT critical decisions [8]. Especially in the financial sector, architectural governance is a crucial issue for IT efficiency and flexibility [46]. Generally speaking, this finding is also primarily shared beyond the financial sector [33]. The need for people from different backgrounds (mainly business and technical ones) to align the organization is the most considerable insight into this research stream.
To tackle the issue of information systems quality from an empirical perspective, we started in 2014 to survey banking application maintenance group experts, Chief Executive Officers, Chief Information Officers, IT architects, technical sales accounts, Chief Data Officers, and maintenance managers [44]. This ongoing project is pursued with a leading consultancy firm, according to which we were able to cover with our representative sample the IT banking sector. Consequently, the need for knowledge representation of different measurement models is perceived as contingent and requested by the IT banking community in this significant project on information systems’ quality. One crucial insight that emerged from the factors is the difficulty to assess their applications, also due to the diversity and complexity of measurement models. Indeed, the three standards measure three different dimensions. Quality measures the software as a product; Process as a process; and Architecture the alignment to a taxonomy. Accordingly, metrics and predictors reflect these differences. Therefore, the development of this ontology is a direct request from practitioners.
Since this research journey started from an industry’s need, an ontology, intended as the knowledge representation of different measurement models is of pivotal importance, and a first tool to systematize the assessment of banking information systems’ quality. Thus, this ontology will be used for consultancy purposes to implement the SQuAP quality model. Moreover, it is also useful to trace changes in quality in time and suggest specific improvements. So, this ontology is the knowledge layer over which this quality model is built. Consultancy firms expressed their interest in a knowledge representation tool which can be displayed to customers in the assessment phase, to tailor their consultancy efforts. However, also the bank’s IT departments will use it for similar purposes. They can also tailor-made and modify this ontology and the underlying metrics suggested by the literature, according to their specific needs.
For this reason, we used a CC-BY-4.0 license, open to commercial use. Our industrial partners consider the use and reuse of this ontology as an excellent value for the practitioners’ community.
Conclusion and future development
In this paper, we have described SQuAP-Ont, an ontology to assess information systems of the banking sector. SQuAP-Ont a) guides its users through the (ongoing) assessment phases suggested by software engineering literature; b) helps to identify critical quality flaws within applications; and c) extends and integrates existing work on software ISO ontology terms, diagram visualizations and ontology revisions. SQuAP-Ont has been developed for commercial use, within an industrial project on quality of banking information systems. Nevertheless, like all ontologies, it is an evolving effort, and we are open to suggestions proposed by the broad researchers’ and practitioners’ communities. We have addressed several issues raised in previous studies, and according to the industry’s expectations.
Our future work aims to facilitate enrichment and refine the ontology continuously along with standards and literature recommendation changes. The enrichment is also about the introduction of specific annotations based on reference vocabularies for tracking provenance and versioning (e.g., PROV-O). Another important aspect is to validate and monitor the application of SQuAP in domains and software projects different from the banking system context. This may lead to the model’s enrichment and improvement.
Footnotes
Acknowledgements
This work was partially funded by the Consorzio Interuniversitario Nazionale per l’Informatica (CINI) and the Science Foundation Ireland grant 15/SIRG/3293 and 13/RC/2094 and co-funded under the European Regional Development Fund through the Southern & Eastern Regional Operational Programme to Lero – the Irish Software Research Centre (
).