Software test quality evaluation based on fuzzy mathematics

2.1 Fuzzy software test quality factor analysis

The development of software products is the result of a kind of intellectual labor, and software products are logical products that are intangible and invisible, which is very different from the development of most industrial products. Software quality is a relatively complex concept with many different understandings and interpretations [21, 22]. The narrow understanding is that the software has no errors, which is also the most basic meaning consistent with the requirements, because when the software contains too many functional defects, the expected functions and requirements provided cannot be met. A more general understanding is that software quality is the degree to which software conforms to clearly and implicitly defined requirements. Specifically, software quality is the degree to which the software conforms to the clearly stated function and performance requirements, the development standards clearly described in the document, and implicit features that any professionally developed software product should have.

From the perspective of software engineering, the entity to be measured usually has many attributes. For example, if we look at a software module, some attributes can be valued without executing the code. For example, the value of the scale attribute can be obtained by the calculation of the code line, the value of the complexity attribute can be obtained by the number of decision points in the code, or even by reading the code. Find the error and get the value of the defect count attribute, and others; but some attributes cannot be obtained directly, and can be found only when the code is executed, such as the ease of use of the software, the efficiency of the software in processing data, and so on. This shows that there is a big difference between the attributes of the entities to be measured, which can generally be divided into external attributes and internal attributes. The distinction and selection of the two are the basis of modeling in the field of software evaluation.

The software quality is measured from a management perspective, and the main factors affecting software quality are divided into three groups, which reflect the three different tendencies of users when using software products, namely product operation, product modification and product transfer, and are depicted in Fig. 1. The relationship between software quality factors and the above three tendencies are discussed. From the customer’s perspective, software quality refers to whether the software meets the needs of the customer, that is, whether it is “applicable". This definition takes into account the needs and expectations of users, that is, whether the software is suitable for them [23]. Because different users’ habits and workflows will be different, the software should have a variety of applications. At the same time, because software quality is the degree of consistency with requirements, requirements must be clearly identified, cannot be ambiguous, and must be measurable.

OPEN IN VIEWER

Fig. 1

The relationship between software quality factors and product activities.

Software quality is closely related to its maintainability, comprehensibility, testability, reusability, portability, reliability, software size, and program complexity. It has a strong relationship with software operation, testing, transplantation and maintenance. Among them, maintainability, understandability, testability, reusability, portability, etc. are the external quality attributes of the software, while the software scale, cohesion, and coupling is the internal attributes of the software. The measurement of software external attributes can be more clearly understood and more intuitively reflect the software quality, but the external attributes of software usually lack clear definitions and most of them cannot be directly measured objectively. Therefore, the software is usually obtained indirectly through the internal attributes of the software that are easy to measure. External attribute value, research and establishment of the relationship model between internal and external attributes of software is one of the main tasks of software measurement. The correct and reasonable software measurement can help people understand and analyze the main relevant factors in the software development process, can help us judge the changing trend of software quality, guide the re-optimization of resources in software development and the further continuous improvement of software quality, and can Make quantitative predictions and evaluations of software product quality.

The standard on which software evaluation is based is the criterion in the software quality measurement model. The criteria can be divided into various types of measurement elements. And these measurement elements can be measured to have the required evaluation indicators for software quality evaluation. Perform software quality evaluation based on these evaluation index values so that an accurate evaluation can be obtained. The determination of software quality evaluation index and software quality evaluation criterion is the most basic problem in software evaluation. In the process of evaluating software quality, to judge whether a piece of software or its intermediate product has quality, whether the quality characteristics of the software, the sub-characteristics of the software, and the metric are qualified, the evaluation criteria must be given, and there must be a criterion. According to these it can determine the weight of each characteristic and sub-characteristic index. This type of data requires accumulation of many projects and also requires expert evaluation. From software quality characteristics to software quality measurement criteria and then to software quality measurement elements, this process is a hierarchical model of software quality measurement. For software measurement, a measurement method must be established to establish a hierarchical model.

Fuzzy mathematics is a mathematical theory and method, which is mainly used to study and deal with fuzzy phenomena, and is widely used in practice. The fuzzy set can use formal language to describe the fuzzy concept, and the process of evaluation, reasoning, decision-making and control that people use the fuzzy concept can also be described by the method of fuzzy mathematics [24]. Common ones include fuzzy cluster analysis, fuzzy comprehensive evaluation, fuzzy pattern recognition, fuzzy decision-making and fuzzy prediction, fuzzy information processing, fuzzy control, etc. These methods constitute a fuzzy system theory.

When applying fuzzy evaluation to software testing quality evaluation in this paper, the evaluation process here is usually divided into three steps: first establish a complete evaluation index system, and then obtain the initial semantics of each index through expert evaluation, test analysis and other methods The evaluation value, and finally the evaluation value of each indicator is comprehensively analyzed through the evaluation synthesis algorithm, and the final evaluation result is given [25]. The so-called consistency test, specifically speaking, is to observe the logical relationship between each indicator during the entire judgment period to see if they are consistent. At present, as far as objective things are concerned, they have relatively complex variability, which directly causes people to hold different opinions on the understanding of things. Based on this situation, even if there is a nine-level indicator list, it is not enough to ensure that all judgments are consistent. Therefore, when judging the consistency of the comparison matrix, the index is used. Based on the foregoing description, this article formalizes the evaluation question was: G = ( A , B , C , D )(1)

The set of factors for software testing and evaluation can be set as: A = ( a 1 , a 2 , ⋯ , a n )(2)

The values required by the single-factor evaluation vector are the membership strength at these five levels, and each value in the comprehensive evaluation vector C reflects the membership strength at different evaluation levels. Set the comment set as: C = ( c 1 , c 2 , ⋯ , c n )(3)

Through the evaluation of each single factor, the membership value of each single factor at each comment level is given. In this way, the relationship between the evaluation factor and the comment level is established, that is, the fuzzy relationship from A to C. This relationship can be described by the aforementioned fuzzy evaluation transformation matrix B. All single factor evaluation vectors form the evaluation matrix B. B = [ b 11 b 12 b 1 n b 21 b 22 b 2 n b m 1 b m 2 b mn ](4)

The comprehensive evaluation vector B is obtained by matrix operation. H = D * B = ( d 1 , d 2 , ⋯ , d n ) [ b 11 b 12 b 1 n b 21 b 22 b 2 n b m 1 b m 2 b mn ] = ( h 1 , h 2 , ⋯ , h n )(5)

To determine the value in the single factor evaluation matrix B, it is necessary to establish a fuzzy mapping from the factor set A = (a₁, a₂ ...  a_m) to the evaluation set C = (c₁, c₂, ...  , c_n), that is, for each For each factor a_i (i = 1, 2, ...  , m), determine the correspondence b_ij between the membership degree and each evaluation c_j (j = 1, 2, ...  , n), so the single-factor evaluation vector of a₁ is: B 1 = ( f 11 k , f 12 k , ⋯ , f 1 n k )(6)

In the same way, the above can be extended to other factors. If f_ij(i = 1, 2, ...  , m, j = 1, 2, ...  , n) is the evaluation of the i-th factor a_i (i = 1, 2, ...  , m) Is the number of votes of the j-th grade c_j (j = 1, 2, ...  , n), then the single factor evaluation matrix can be obtained: B = [ f 11 k f 12 k f 1 n k f 21 k f 22 k f 2 n k f m 1 k f m 2 k f mn k ](7)

The design evaluation factor A is a very important link in the entire evaluation process. The factors that affect the evaluation object can be classified into the main factors and sub-factors in several aspects, that are, they can be divided into primary or secondary indicators. The factor set mentioned here refers to the set of indicators to be evaluated, which is represented by A =  (a₁, a₂ ...  a_m). Indicating that the evaluated object is judged and described with m factors selected. Of course, the evaluated object can have several factors and sub-factors or even multi-level factors. The ultimate goal of the fuzzy evaluation system is to comprehensively evaluate the software test quality through the feedback information of the software test quality. Therefore, it is necessary to consider the contribution of all the underlying factors to the overall goal.

As far as the determination of the degree of membership is concerned, it specifically refers to the quantitative relationship between the evaluation index and the evaluation target of the software test quality. Therefore, ensuring that it is objective and reasonable becomes the basis of evaluation. Among them, this article mainly uses fuzzy statistical methods. At the same time, through listening to the opinions of experts, in addition to consulting a large number of related documents and standards, the purpose of determining the degree of membership is finally achieved. As for the quality of software testing, the evaluation agency includes two specific results: qualified and unqualified. However, in terms of actual software testing quality, failure is prohibited.

Software product measurement focuses on understanding and controlling the quality status of current software products, and is used to predict and control product quality; software project measurement focuses on understanding and controlling the current software project situation and status; software process measurement focuses on understanding and controlling the current software development process The connotation of the situation and status of the software also includes the improvement of the software development process and the ability to predict the future process. Software project measurement is carried out for specific software projects and has tactical significance, while software process measurement is carried out within the entire organization and has strategic significance. These three forms of software measurement are closely related, and the purpose of studying them is to control and improve the quality of software products. Generally speaking, software product measurement is the foundation and the main basis for further analysis of software project status and software process performance. Figure 2 is the software quality evaluation process.

OPEN IN VIEWER

Fig. 2

Software quality evaluation process.

Based on the comprehensive evaluation process of the target, it is always based on the real situation, and finally chooses the evaluation method suitable for itself. At present, there are also many kinds of evaluation methods, including expert evaluation method, entropy method, and fuzzy comprehensive evaluation method, and the analytic hierarchy process, etc. When using different methods to evaluate the same target, the evaluation results are different. Therefore, in the final comprehensive evaluation process, reasonable methods must be adopted based on specific conditions. In addition, the methods must be fully considered. Specifically including its applicability and feasibility.

Method hiding factor is one of the two metrics used to measure data encapsulation. The method hiding factor is defined as follows: GMN = ∑ n = 1 n ∑ m = 1 m ( 1 - W ( H mn ) ∑ k = 1 k H f ( E k )(8)

Polymorphism factor is a metric used to measure polymorphism. The polymorphic factor is defined as follows: PMN = ∑ k = 1 k H f ( E k ) ∑ mn = 1 mn [ W mn ( E mn ) * DC ( E mn )(9)

Software quality evaluation is a multi-level and multi-index problem. The establishment of an evaluation index system should follow the principles of scientifically, tradition, comparability, index integration, practicability and stability. The scientific nature is mainly reflected in the scientific reflection of the evaluation purpose. Software quality characteristics, traditionalist is mainly to ensure the comprehensiveness of the evaluation; practicality is mainly to make the indicator system reasonable and maneuverable, and stability mainly means that the selected evaluation indicators should not be changed at will, and have relative stability. The external quality characteristics of software are reliable, correctness, usability, completeness, efficiency, testability, maintainability, flexibility, reusability, portability, and interoperability. The bottom layer of the software quality measurement model provides a set of basic software quality characteristics that are quite different from each other. Defined basic characteristics provide a better basis for quantitative measurement of higher-level characteristics, which can be used at higher levels. To evaluate the quality of software products, it can also guide the direction of software quality improvement at a lower level. The software quality measurement model is shown in Fig. 3.

OPEN IN VIEWER

Fig. 3

Software quality feature.

The advantage of the fuzzy comprehensive evaluation method is that it uses digital means to process the evaluation object, which is accurate. The evaluation result obtained is a vector containing a lot of information, which can provide reference information for other aspects of research; the disadvantage of the method is that the calculation is too complicated and the weight of the index. The process of determination lacks objective accuracy, and there will be hyper-ambiguity, which leads to failure of the final evaluation.

3.1 Model analysis

It is a difficult point to determine the weight of each quality index in the comprehensive evaluation of software quality. Commonly used methods include Delphi method and reverse engineering method. The software quality evaluation model based on fuzzy analytic hierarchy process uses the analytic hierarchy process to determine the weight of software quality indicators. The analytic hierarchy process is to create a judgment matrix based on the important comparison between the software quality indicators, and then process the judgment matrix to obtain the weight of the software quality indicators. This method is too subjective, the importance of indicators is artificially determined by the evaluator, and this process may only rely on one’s previous experience.

The influence of the insensitive loss parameter on the fuzzy mathematics software test quality evaluation model can use the model mean square error parameter as a reference, that is, in the process of judging its impact, by assigning the other two parameters of the fuzzy mathematics software test quality evaluation model, you can Let the penalty parameter be 10 and the kernel parameter be 25.0. The influence of the parameter value on the model is shown in Fig. 4. It can be analyzed from Fig. 4 that the mean square error of the model gradually increases with the increase of the value. That is, the performance of the model becomes less and less ideal as the parameter value increases. It can be analyzed from the image that when the value is in the range of 0.01 to 0.1, the amount of error gradually becomes smaller as the value increases, and the error takes a smaller value at 0.1; when the value is in the range of 0.1 to 0.2, the amount of error increases with the value When the value is in the range of 0.2 to 0.25, the amount of error shows a downward trend as the value increases.

OPEN IN VIEWER

Fig. 4

Influence diagram of insensitive loss parameter.

The use of fuzzy algorithms for select software test quality evaluation models is better than the cross-validation method and grid search method introduced above. First, the fuzzy algorithm overcomes the randomness and large workload of traditional evaluation model parameter selection; Second, the fuzzy algorithm is an improvement on the basis of the cross-validation method, which is better than the cross-validation method in terms of the obtained effect; third, compared to the grid search, in the case of a small number of samples, the fuzzy algorithm overcomes In addition, grid search requires a large amount of calculation, requires repeated training, is relatively simple and effective due to its high complexity and time-consuming shortcomings. The software quality evaluation studied in this paper is a small sample problem, so the method of selecting the parameters of the software testing quality evaluation model is fuzzy algorithm. Figure 5 shows the results of fuzzy algorithm selection of software test quality evaluation model parameters.

OPEN IN VIEWER

Fig. 5

Fuzzy algorithm selection results of software test quality evaluation model parameters.

Software quality evaluation is based on the product quality evaluation index system and evaluation model. The software quality evaluation sample data is shown in Fig. 6. In the sample data, the evaluation index set is B = b₁, b₂, b₃ ...  ...  b_n, and the software quality is divided into four levels: first, second, third, and poor. Its mathematical language is expressed as H = h₁, h₂, h₃ ...  ...  h_n, the evaluation grade range is 0.11, 5.75, 0.71, 0.54, using fuzzy evaluation to perform fitting analysis on sample data.

OPEN IN VIEWER

Fig. 6

Software product quality evaluation sample data.

It can be seen from Fig. 7 that the values from level 1 to level 5 are the values in the comprehensive evaluation vector B, which is the comprehensive evaluation vector of the object at this time. The semantics of the five levels of evaluation level V are set as: Very poor, poor, fair, good, excellent. Each value in B represents the strength of membership corresponding to different levels of C. In this prototype system, the quantitative value of each level in the comment set V is specified as: C1 = 25, C2 = 45, C3 = 65, C4 = 85, C5 = 100. The final evaluation result value is between 25 and 100. Usually the closer to 100, the higher the evaluation score, and the closer to 20, the lower the score. The final evaluation result score is obtained by M = BT * C = (B1 * C1 + B2 * C2 + B3 * C3 + B4 * C4 + B5 * C5). In this example, the evaluation score M = 81.24, the calculation result shows that the job applicant is at the upper level in the evaluation of the position.

OPEN IN VIEWER

Fig. 7

Evaluation results.

Compared with traditional statistical methods, the fuzzy regression method can better express the fuzzy relationship in the house of quality, and the parameters estimated by this method are closer to the real parameters of the model. Therefore, we use the linear fuzzy regression model to estimate the parameters of the relationship expression between user requirements and software characteristics. The fitting effects of software quality evaluation using radial basis and polynomial kernel function support vector machine models are expressed in mean square error (MSE) and correlation coefficient (R) as showed in Table 1.

Both the radial basis kernel function SVM model and the polynomial kernel function SVM model can fit and evaluate the software quality, and the fitted mean square error is below 0.1. Among them, the fitted mean square error of the radial basis kernel function SVM is relative to the polynomial kernel function In terms of the smaller value, it is 0.0529295; the fitting correlation coefficients of the two kernel functions are both higher, and the value is more than 90%. Among them, the fitting correlation coefficient of the radial basis kernel function is even higher. The effect is better than that of the polynomial kernel function SVM. The regression fitting results of the second-order polynomial kernel function support vector machine and the radial basis kernel function support vector machine are compared and analyzed. The fitting results are shown in Fig. 8.

OPEN IN VIEWER

Fig. 8

Fitting comparison analysis results.

Through the evaluation of the two methods, it can be found that although the traditional evaluation method has shortcomings in accuracy and error. The application of traditional evaluation for software quality evaluation is feasible and effective. This paper uses the traditional evaluation method and the fuzzy evaluation method to compare the case analysis. From the regression prediction results and the actual software quality evaluation simulation analysis, it can be seen that in the method of using the fuzzy software test quality evaluation model for software quality evaluation, although there are certain errors, the evaluation errors are basically within the controllable error range, and the errors are relatively large. Small, the accuracy is higher than that of traditional software quality evaluation methods; in terms of training time, it is faster than other evaluation methods. The comparative analysis of the evaluation errors of the two methods is shown in Fig. 9.

OPEN IN VIEWER

Fig. 9

Comparison of two methods for evaluating software product quality errors.

The error comparison chart of the two methods for evaluating software product quality shows that the error of the fuzzy model evaluation method is within the range of -0.1 to 0.1, which is a controllable range and is a very small amount of evaluation error; for the traditional method, its error The error is within the range of –0.14 to 0.21, which is also within the expected range. However, in terms of numerical value comparison, the evaluation error of the fuzzy model method is smaller than that of the traditional evaluation method. The comparison between the fitting evaluation results of the two model evaluation methods and the actual values is shown in Fig. 10. The accuracy of using fuzzy technology to build mathematical models to evaluate software quality is higher than that of traditional methods, and using fuzzy mathematical technology to build models overcomes the instability of traditional models, local optimization problems and the shortcomings of non-unique results. The fuzzy method can evaluate the quality of software products scientifically and accurately. And has certain guiding significance for software design.

OPEN IN VIEWER

Fig. 10

Software quality evaluation fitting results of two model methods.

Fuzzy thinking is well adapted to the objectively existing fuzziness problem. But in the evaluation process, there is no need to deliberately blur the original precision problem. When the software testing quality evaluation theory and fuzzy comprehensive evaluation is applied to the selection problems in the software evaluation process, the purpose is also to convert the fuzzy problems into precise values without losing information, so as to screen and sort. In the process of application, relevant teams should be set up to conduct full investigation and analysis, and grasp the needs and purpose of evaluation according to their own requirements and development needs. Reasonably set up different evaluation indicators based on the position, and set the weight and the single-factor evaluation vector for each evaluation indicator based on the discussion and analysis of the software test quality. When setting related values such as weights for software testing quality, there must be a certain degree of subjectivity, but subjectivity here is not a defect, because in the final analysis, software testing quality still needs to choose the evaluation method according to its own needs. It has a certain degree of subjectivity. Therefore, it is not easy to set values such as indicators and weight, but they can be tried and improved in practice.