RationalGRL: A framework for argumentation and goal modeling

2.1. Running example: Traffic simulator

Our examples and coding analysis are based on the data produced by a series of experiments by Tang et al. [45], who in turn base their work on the so-called Irvine experiment [46]. This experiment contains a design reasoning assignment in software engineering about a traffic simulator. In this assignment (see Appendix A), designers are provided with a problem description, requirements, and a description of the desired outcomes. The client of the project is Professor E, who teaches civil engineering courses at an American university. In order for the professor to teach students various theories concerning traffic (such as queuing theory), traffic simulator software needs to be developed in which students can create visual maps of an area, regulate traffic, and so forth. Tang et al. asked designers (groups of students) to discuss the requirements of this traffic simulator. These discussions were recorded and transcribed. We performed a coding analysis on these transcripts, which we used to develop our RationalGRL framework. Furthermore, we also use the traffic simulator case as a running example throughout this paper.

2.2. Goal-oriented requirements language (GRL)

OPEN IN VIEWER

Fig. 1.

Basic elements and relationships of GRL.

OPEN IN VIEWER

Fig. 2.

Partial GRL model of the traffic simulator example.

GRL is a modeling language for specifying intentions, business goals, and non-functional requirements of multiple stakeholders [2], based on i ∗  [51] and the NFR Framework [10]. GRL is part of the User Requirements Notation, an ITU-T standard, that combines goals and non-functional requirements with functional and operational requirements (i.e. use case maps). GRL can be used to specify alternatives that have to be considered, decisions that have been made, and rationales for making decisions. A GRL model is a connected graph of intentional elements that optionally are part of the actors. The GRL intentional elements (IEs) and relationships used in this paper are shown in Fig. 1.

Figure 2 illustrates a simplified GRL diagram from the traffic simulator design exercise. An actor represents a stakeholder of a system or the system itself (Traffic Simulator, Fig. 2). Actors are holders of intentions; they are the active entities in the system or its environment who want goals to be achieved, tasks to be performed, resources to be available, and softgoals to be satisfied. Softgoals differentiate themselves from goals in that there is no clear, objective measure of satisfaction for a softgoal whereas a goal is quantifiable. Softgoals (e.g. Realistic simulation) are often related to non-functional requirements, whereas goals (such as Generate Cars) are related to functional requirements. Tasks represent solutions to (or operationalizations of) goals and softgoals. In Fig. 2, there are two tasks Create new cars and Keep same cars: in order to achieve the goal Generate cars, the simulation can either constantly generate new ones or keep the same cars and have them reappear after they disappear off-screen. In order to be achieved or completed, softgoals, goals, and tasks may require resources to be available (e.g., Car Objects). Finally, the full version of GRL allows design rationale to be captured using beliefs. Since we capture the reasoning and rationales behind goal models using arguments, we do not include beliefs in our simplified version of GRL (but see the discussion at the end of this section).

Different links connect the elements in a GRL model. AND, IOR (Inclusive OR), and XOR (eXclusive OR) decomposition links allow an element to be decomposed into sub-elements. In Fig. 2, the goal Generate cars is XOR-decomposed to the tasks Create new cars and Keep same cars, as they are alternative ways of achieving the goal Generate cars. Contribution links indicate impacts of one element on another element, which can be positive or negative. Task Create new cars has a positive contribution to the softgoal Realistic simulation, and a negative contribution to the softgoal Simple design. Note that the full GRL specification considers different levels of positive and negative contribution values (both quantitative and qualitative), which are not directly relevant for our current purpose. Dependency links are relationships between intentional elements, which can model dependencies between actors. Here, the goal Generate cars depends on the resource Car objects.

The GRL model in Fig. 2 shows the softgoals, goals, tasks and the relationship between the different intentional elements in the model. However, the rationales and arguments behind certain intentional elements are not shown in the GRL model. Some of the questions that might be interesting to reason about are the following:

These are the types of the questions that we cannot answer by just looking at GRL models. The model in Fig. 2 does not contain information about discussions that led to the model, such as clarification steps for the naming, or alternatives that have been considered for the relationships. The idea behind the original GRL specification is that so-called belief elements can be used to capture such design rationales that make later justification and review of a model easier. In the original GRL specification, however, the exact semantics of these belief elements and the ways in which they should be used remain unclear. Furthermore, belief elements cannot be attached to links between elements, which makes answering the fourth and fifth question above impossible. We further discuss belief elements in light of our proposed argumentation-based framework in Section 8.1.

An important characteristic of human reasoning is its non-monotonic, or defeasible, nature: conclusions that are accepted at one moment may be retracted later in view of new information; on the other hand, new information may cause conclusions that were rejected earlier to be reinstated. Computational argumentation is an approach to non-monotonic reasoning that focuses on interactions between arguments and counterarguments. With applications in multi-agent systems, decision-making tools, medical and legal-reasoning, argumentation has become a major subfield of artificial intelligence [7].

One central concept in argumentation is that of argument schemes [8,50]), stereotypical patterns of non-monotonic reasoning – for example, “if an expert says something, we can usually accept his claims”. Part of argument schemes are the critical questions, which point to typical sources of doubt or implicit assumptions in arguments based on schemes – for example, “is the expert biased?” or “is the expert a recognized expert in the field that he claims?”. Critical questions can be used to make explicit the counterarguments to the original argument – e.g. “we should not accept the expert’s claim since he is biased”. Such counterarguments can be attacked in turn by their own counterarguments – “no he’s not biased since he has no stake in the case” – and thus an argumentation framework [13] of mutually attacking arguments can be constructed.

In early-phase requirements engineering, we can imagine many different argument schemes, such as for example the Appeal to Expert Opinion Scheme illustrated above. Central, however, in discussions surrounding goal models is obviously the reasoning about which goals to pursue and which actions to take, often referred to as practical reasoning. Practical reasoning has been studied extensively in computational argumentation, mainly by Atkinson and Bench-Capon [4], who have developed and formalized the Practical Reasoning Argument Scheme. A simplified version of this argument scheme, presented in syllogistic form, is as follows:

Here, G and A are variables, which can be instantiated with concrete goals and actions to provide a specific practical argument. For example, a concrete argument about the traffic simulator is as follows:

Like in all argumentation, practical reasoning is also about counterarguments. For example, we may argue that, in fact, performing action Keep same cars does not realize goal Generate cars, thus giving a counterargument to the above argument. Atkinson et al. [4] define a set of critical questions that point to typical ways in which practical reasoning arguments can be criticized. Some examples of critical questions are as follows.

Answers to these critical questions are counterarguments to the original argument. These counterarguments also follow a scheme; for example, a negative answer to CQ1 follows the scheme “Action A will not realize goal G”, which can be instantiated (e.g. “Keep same cars does not realize Generate cars”) to form a counterargument to the original argument. Another way to criticize an argument for an action is to suggest an alternative action that realizes the same goal (CQ2). For example, we can argue that performing Create new cars also realizes the goal Generate cars. Also, it is possible that performing an action has a negative side effect (CQ3). For example, while the action Create new cars realizes the goal Generate cars, it has a negative side effect, namely hurting Simple design: having the simulation constantly create new cars is fairly complex design choice. Note that argument schemes and the arguments based on them can thus be of a more syllogistic form (i.e., “G, A realizes G, Therefore A”) but also of a simpler, more propositional form (i.e., “A does not realize G”).

OPEN IN VIEWER

Fig. 3.

PRAS arguments and attacks in the traffic simulation example.

Counterarguments attack the original arguments. Given a set of arguments and attacks between these arguments, we can compute which arguments are accepted and which are rejected using different argumentation semantics [13].2

The practical reasoning scheme and its critical questions as given above could be used to argue about goal models. In fact, in some of our early work on RationalGRL [48,49] we used practical reasoning arguments such as A1 and A2 to capture reasoning about goals, combined with arguments based on other schemes such as the scheme from Expert Opinion. We then translated these arguments to goal models. However, our later coding analysis (Section 3) shows that when discussing requirements, people very often do not structure their reasoning nicely along the lines of the full argument schemes, that is, they rarely utter an argument along the lines “We have goal G, A realizes G Therefore we should perform A”. A typical discussion is much more unstructured, as is clear from the transcript excerpts in Appendix B. Stakeholders propose and debate possible goals of the system, alternative options, and so on, but the argumentation is more about making different (propositional) claims, questioning these claims and proposing counterclaims than about building inference structures from premises to conclusions. Furthermore, it turned out that the stakeholders did not use many other types of argument or argument scheme, such as Expert Opinion. Hence, we decided to develop our own set of argument schemes and critical questions by coding the transcripts of discussions about the traffic simulator, so that we would meet Requirement 1: The argumentation techniques must capture the actual discussions of the stakeholders or designers in the early requirements engineering phase. Our set of schemes, questions and our coding analysis are described in the next section.

A core aspect of the RationalGRL framework are the argument schemes – in particular the critical questions – which should be close to the actual discussions of stakeholders (Requirement 1). Recall from Section 3 that we ended up with a list of argument schemes and critical questions that were found in the transcripts (Table 1). Using this list as a basis, we further refined the set of argument schemes and critical questions for RationalGRL into the list shown in Table 2. Again, we note that the argument schemes are not of the traditional, syllogistic form, but rather represent types of claims about goal models. These schemes could be rendered in a syllogistic form: for example, we could say that the full reasoning encapsulated by scheme AS0 is something like “a is an actor in the information system Therefore a should be included as an actor in the goal model”.

It is important to note that the list we provide here is not exhaustive – it is the result of our empirical study, but new argument schemes and questions can be added depending on the problem domain. In fact, our tool (Section 6.2) contains many more critical questions, and it is fully extensible, meaning that new argument schemes and critical questions can be added easily. See Section 8.2 for more details.

AS0–AS4 and AS11–AS12 are arguments for an element of a goal model, and AS5–AS10 are related to links in a goal model. The last scheme (Att) is a scheme for a generic counterargument against any type of argument. Arguments based on the schemes in Table 2 can be used to form arguments about elements in a GRL model. Making an argument based on one of the schemes effectively adds the corresponding GRL element to the model. See, for example, Table 5 in Appendix B: the participants argue for the addition of several tasks to the goal model using argument scheme AS2.

An important part of arguing about goal models is asking the right critical questions. The critical questions presented in Table 2 are therefore related to their respective argument schemes. These questions can be answered with “yes” or “no”, and the type of answer has an effect on the original argument (INTRO, DISABLE, REPLACE). This will be further explained in Section 4.3 and in Section 5.

RationalGRL is an extension of GRL and includes all the elements shown in Fig. 1. However, there are also new elements corresponding to argumentation-related concepts. Figure 5 shows these elements.

OPEN IN VIEWER

Fig. 5.

The new elements and link of RationalGRL. Generic argument (left), attack link (middle), and disabled element (right). The disabled element in this figure is a task, but all IEs and links can be disabled in RationalGRL.

OPEN IN VIEWER

Fig. 6.

The RationalGRL metamodel. The goal model construction package (bottom) is the GRL metamodel. The practical reasoning & argumentation package is the RationalGRL extension.

The complete metamodel of the language can be found in Fig. 6. This metamodel represents the abstract grammar of the language, independently of the notation. The metamodel also formalizes the GRL concepts and constructs introduced in Section 2.2.

The metamodel consists of two packages, Practical reasoning & argumentation and Goal model construction, which correspond to the relevant activities in the RationalGRL framework (cf. Fig. 4). The goal model construction package is simply the GRL metamodel. It consists of GRLModelElements, which can be either GRLLinkableElements or ElementLinks. A GRLLinkableElement can again be specialized into an Actor or an IntentionalElement (which is either a Softgoal, Goal, Task, Resource, or a Belief). Intentional elements can be part of an actor, and GRLLinkableElements are connected through ElementLinks of different types (i.e., Contribution, Decomposition, or Dependency). Finally, a GRLmodel is composed of GRLModelElements.

The practical reasoning and argumentation package depicts the concepts we introduced in Section 4.1. An ArgumentScheme represents a scheme containing variables, and by instantiating such a scheme we obtain an Argument. Therefore, each argument is associated with at most one scheme, but a scheme can be instantiated in multiple ways. A RationalGRLmodel is composed of arguments and attack relations.

The OperationTypes in the argumentation package correspond to the operations we informally introduced in Section 4. These operations are performed by instantiating an argument scheme or answering a critical question. An INTRO operation introduces a new RationalGRL element. A DISABLE operation creates a new argument that attacks another argument or GRL element, effectively disabling it. The REPLACE operation replaces a RationalGRL element with a new element. Instantiating an argument scheme from Table 2 always leads to an INTRO operation, that is, it always introduces a new RationalGRL element. Answering a critical question can have different effects depending on the critical question and the answer. Table 2 shows these effects for the different critical questions and answers. For example, answering CQ0 with “no” disables the argument based on AS0.

There are two important links between the practical reasoning & argumentation and goal model construction packages. First, each GRLModelElement is an Argument. This means that each model element inherits the AcceptStatus as well, allowing GRL elements to be accepted or rejected. This, furthermore, means that argument schemes can be applied to all GRL elements, capturing the intuition that each GRL element can be regarded as an instantiated argument scheme. Note that besides arguments about elements of the GRL model, we also have a GenericArgument which is simply a counter-argument to an existing argument that does not relate to any of the GRL elements. Finally, the relation between GRLModelElement and Argument means that, as we already briefly indicated when discussing the framework in Fig. 4, the class of RationalGRLmodel is a superclass of GRLmodel: besides arguments about GRL elements, we can also have arguments that do not relate to any of the GRL elements.

We now discuss the interactions between the practical reasoning & argumentation (bottom left element of the framework in Fig. 4) on the one hand, and RationalGRL models (top left element of the framework in Fig. 4) on the other hand. We provide informal examples of the links between the practical reasoning found in our coding analysis transcripts and RationalGRL models – the formal grounding for this connection can be found in the RationalGRL metamodel (Section 4.2) and in the logical formalization in the next section. The connection between the RationalGRL models shown in this section and regular GRL models is further formally defined in Section 5.3.

Example 1 – introducing GRL elements with arguments (INTRO). 

OPEN IN VIEWER

Fig. 7.

RationalGRL model (right) with instantiated argument schemes (left): introducing new elements (operation INTRO).

We first show how instantiating argument schemes leads to the introduction of new RationalGRL elements in a model. Take the example in Fig. 7, which is based on the excerpt from transcript t 3 shown in Table 7 (Appendix B). On the left side, the arguments found in the transcript are shown, together with the argument scheme they are based on. The participants in the discussion argue that actor System has a goal and two tasks, and that the goal AND-decomposes into the two tasks. By putting forward these arguments, new GRL elements are introduced. These GRL elements are shown on the right side of Fig. 7; the dashed arrows indicate the links between the practical reasoning and argumentation on the left and the RationalGRL model on the right.

Example 2: Disabling GRL elements by answering critical questions (DISABLE). 

OPEN IN VIEWER

Fig. 8.

Disabling elements (operation DISABLE).

The excerpt from transcript t 1 used in this example is shown in Table 5 (Appendix B). Here, participants first sum up functionality of the traffic simulator, which can be captured as instantiations of AS2. On left side of Fig. 8 one such instantiation is shown, which leads to the addition of the task Add traffic light in the RationalGRL model on the right side of Fig. 8. However, participant P1 notes that the problem description states that all intersections have traffic lights by default, so the task Add traffic light is not necessary. This is captured using critical question CQ11. A negative answer to this question (cf. Table 2) should disable the original argument based on AS2 by attacking it. On the left side of Fig. 8 a new argument (CQ11) attacks the original argument based on argument scheme AS2. This new argument is also added to the RationalGRL model on the right-hand side of Fig. 8, where it attacks the original task Add traffic light. This attack leads to the original argument being rejected (cf. Section 2.3 and Section 5.2), indicated by being greyed out. As a result of this, the corresponding GRL task Add traffic light is also disabled.

Example 3: Changing a decomposition type by answering critical questions (REPLACE). 

OPEN IN VIEWER

Fig. 9.

Replacing an element (operation REPLACE).

The excerpt of transcript t 3 used for this example is shown in Table 7 in Appendix B. It consists of a discussion about the type of decomposition relationship for the goal Simulate Traffic (Fig. 9). Recall that in Example 1, an AND-decomposition was introduced for this goal with AS5. In the discussion of the current example, CQ5c – “Is the decomposition type correct?” – is explicitly asked. The answer is “No, it should be OR”. The original argument for AND-decomposition is now attacked by the argument for the OR-decomposition, and the new argument is linked to the OR-decomposition in the RationalGRL model.

OPEN IN VIEWER

Fig. 10.

Renaming an element name (operation REPLACE).

Example 4: Clarifying a task by answering critical questions (REPLACE). The transcript excerpt of this example is shown in Table 6 in Appendix B and comes from transcript t 1 . The discussion starts with an instantiation of argument scheme AS2: “Actor Student has task Set car influx” (Fig. 10). This argument is then challenged with critical question CQ12: “Is the task Set car influx specific enough?”. This is answered negatively, creating a new argument “Actor Student has task Set car influx per road”, which attacks the original argument for Set car influx. Note how the new task Set car influx per road also attacks (and disables) the original RationalGRL task Set car influx.

Example 5: Defending the addition of an actor (DISABLE). 

OPEN IN VIEWER

Fig. 11.

Disabling an element (operation DISABLE).

The excerpt from transcript t 3 used in this example is shown in Table 8 in Appendix B. Actor Development Team is introduced with an argument based on AS0 (Fig. 11). This is then attacked by arguing that the professor will develop the software, thus, there will not be any development team (CQ0).

OPEN IN VIEWER

Fig. 12.

Defending an element by attacking its disabling attacker (operation DISABLE).

Further in the discussion, it is then argued that the development team should be considered, since the professor does not develop the software. This is captured using a generic counterargument Att, which attacks the earlier argument based on CQ0. Figure 12 shows the situation after the counterargument has been put forward: the argument (Att) now attacks the argument (CQ0), which in turn attacks the original argument (AS0). As a result, the argument (AS0) is acceptable (cf. Section 2.3 and Section 5.2), which causes the actor in the RationalGRL model to be enabled again.

5.1. Formal specification of GRL

We formalize a GRL model based on the metamodel (Fig. 6), starting with intentional elements and actors.

Throughout this section, we adopt the convention that variables start with a lowercase letter (e.g, id , i, j, name , goal ), and sets and constants start with an uppercase letter (e.g., Type , AND , Goal ). We begin by defining general elements of our language that we use in subsequent definitions.

Next, we define an intentional element.

The definition above is sufficient to capture all intentional elements (IEs) used in GRL. Note that according to the definition above, all IEs have a decomposition type, even when the IE is not decomposed into other IEs. This is in line with the GRL metamodel. However, in the following, we sometimes leave out the decomptype if the IE is not decomposed to simplify the notation and improve readability.

Throughout this section, we use the example in Fig. 2. This example contains various IEs which can be formalized using Definition 2, for example, ( 1 , Resource , Car objects ) , ( 2 , Softgoal , Realistic simulation) and ( 5 , Goal , Generate cars, XOR).

We now define actors.

We can formalize the actor in our example (Fig. 2) as ( 0 , Actor , Traffic Simulator ) .

The relation between actors and their intentional element is formalized as follows.

Note that Actor-IE relations do not have an id : they are themselves not elements of a GRL model but rather indicate relations between elements of a GRL model (i.e., intentional elements and actors). The relation between the Traffic Simulator actor ( id = 0 ) and the Car objects resource ( id = 1 ), for example, can be formalized as ( ActIE , 0 , 1 ) .

At this point we have defined all intentional elements in GRL and a containment relation between actors and intentional elements. We now discuss the GRL links.

Similar to IEs, links have identifiers as well. Some of the links in our example (Fig. 2) are ( 8 , Dep , 4 , 1 ) , ( 9 , PosContr , 5 , 2 ) and ( 11 , Decomp , 5 , 4 ) .

We can now form GRL models as follows.

The full specification of Fig. 2 is as follows. IE = { ( 1 , Resource , Car objects ) , I E = ( 2 , Softgoal , Realistic simulation ) , I E = ( 3 , Softgoal , Simple design ) , I E = ( 4 , Softgoal , Easy to use ) , I E = ( 5 , Goal , Generate cars , XOR ) , I E = ( 6 , Task , Create new cars ) , I E = ( 7 , Task , Keep same cars ) } Act = { ( 0 , Actor , Traffic Simulator ) } R A c t I E = { ( ActIE , 0 , 1 ) , … , ( ActIE , 0 , 7 ) } Link = { ( 8 , Dep , 4 , 1 ) Link = ( 9 , PosContr , 6 , 2 ) Link = ( 10 , NegContr , 6 , 3 ) Link = ( 11 , Decomp , 6 , 5 ) Link = ( 12 , Decomp , 7 , 5 ) }

Definition 6 only sums up the elements of the model and not the constraints that make a valid GRL model. We will do so in the next definition. Note that in this definition, we use a subscript notation to refer to an element with a specific id. That is, I E i , Lin k j , and Ac t k refer respectively to the IE with id i, the link with id j, and the actor with id k.

Let us briefly verify that our previous formalization of Fig. 2 satisfies all the constraints of Definition 7:

All elements in the formalization have different ids, so this constraint is satisfied.

In order to develop a logical framework for RationalGRL, we extend the GRL logical framework of the previous section by adding two elements (see Fig. 5), namely generic arguments and attack links. We illustrate the new elements using the RationalGRL model in Fig. 13, which is an extension of Fig. 2.

OPEN IN VIEWER

Fig. 13.

Example RationalGRL model (extension of Fig. 2).

Thus, a generic argument is any element in the RationalGRL model that is not an intentional element or an actor. In the example, some of the generic arguments are ( 13 , GenArg , Redundant ) and(14, GenArg , Necessary). Note that a constraint of a GRL model (Definition 6) is that GRL links should connect IEs, which means that in generic arguments cannot be connected with GRL links.

We can now define arguments.

This definition captures the specification in the RationalGRL metamodel (Fig. 6) in which the class Argument is a superclass of GenericArgument and GRLModelElement. In sum, we define an argument as any one of the GRL IEs or links, or a generic argument. This corresponds with the argument schemes in Table 2: an argument that is a GRL model element, can be seen as a claim that this element should be part of the model.

Examples of arguments in Fig. 13 are A 5 = ( 5 , Goal , Generate cars , AND ) , A 5 ′ = ( 5 , Goal , Generate cars , XOR ) , A 4 = ( 4 , Softgoal , Easy to use ) and A 13 = ( 13 , ArgElem , Redundant ) . Note that the two arguments A 5 and A 5 ′ show an important difference between RationalGRL models and valid GRL models: While a valid GRL model disallows multiple elements with the same identifier (Definition 7, condition 1), RationalGRL models do not enforce this restriction. This is because it is possible to create multiple arguments for the same element, where each argument contains different content for the same element. However, the set of accepted elements in a RationalGRL should all have unique identifier (see Definition 15).

As an example, take the arguments A 5 for the goal ‘Generate cars’ ( AND decomposition), A 5 ′ for the goal ‘Generate cars’ ( XOR decomposition), A 4 for the softgoal ‘Easy to use’ and the generic argument A 13 (‘Redundant’) against the softgoal (‘Easy to use’). Given these arguments, there are two attack links (Fig. 13), namely ( A 5 ′ , A 5 ) and ( A 13 , A 4 ) .

We now define a RationalGRL model.

The definition of a RationalGRL model collects all the previously defined GRL and RationalGRL elements in a single definition. For completeness, we now provide the full specification of Fig. 13. Let us first enumerate all the arguments used in this example: A 0 = ( 0 , Actor , Traffic simulator ) A 1 = ( 1 , Task , Car objects ) , A 2 = ( 2 , Softgoal , Realistic simulation ) , A 3 = ( 3 , Softgoal , Simple design ) , A 4 = ( 4 , Softgoal , Easy to use ) , A 5 = ( 5 , Goal , Generate cars , AND ) , A 5 ′ = ( 5 , Goal , Generate cars , XOR ) , A 6 = ( 6 , Task , Create new cars ) , A 7 = ( 7 , Task , Wrap cars ) , A 7 ′ = ( 7 , Task , Keep same cars ) , A 8 = ( 8 , Dep , 4 , 1 ) , A 9 = ( 9 , PosContr , 6 , 2 ) , A 10 = ( 10 , NegContr , 6 , 3 ) , A 11 = ( 11 , Decomp , 6 , 5 ) , A 12 = ( 12 , Decomp , 7 , 5 ) , A 13 = ( 13 , GenArg , Redundant ) , A 14 = ( 14 , GenArg , Necessary ) , A 15 = ( 15 , GenArg , No effect ) , A 16 = ( 16 , GenArg , Changes traffic flow ) , A 17 = ( ActIE , 0 , 1 ) , … , A 23 = ( ActIE , 0 , 7 ) This model is then formalized as RatGRL = ( Arg , Att ) where: Arg = { A 0 , … , A 5 , A 5 ′ , A 6 , A 7 , A 7 ′ … , A 23 } Att = { ( A 5 ′ , A 5 ) , ( A 7 ′ , A 7 ) , ( A 13 , A 4 ) , ( A 14 , A 13 ) , Att = ( A 15 , A 9 ) , ( A 16 , A 15 ) }

In Fig. 13, it can be read from that arguments A 5 , A 7 , A 13 and A 15 are currently disabled (rejected). However, we have not yet make it clear how exactly this is computed. We will do so in the following definitions. In order to compute when an argument is accepted and when it is not we use argumentation semantics. We use the standard approach by Dung [13].

Note that the definition of a RationalGRL model (Definition 11) is exactly the same as the definition for an argumentation framework. This allows us to use the following definitions directly.

Let us explain these definitions using the example argumentation framework, in Fig. 14, which is a subset of our RationalGRL model from Fig. 13, containing only arguments A 4 , A 5 , A 5 ′ , A 13 , and A 14 . In this example, there are five admissible sets: { A 5 ′ } , { A 14 } , { A 5 ′ , A 14 } , { A 4 , A 14 } and { A 4 , A 5 ′ , A 14 } . In the admissible sets that contain both A 4 and A 14 , we say that A 14 defends A 4 against its attacker A 13 .6

OPEN IN VIEWER

Fig. 14.

Example argumentation framework, subset of the RationalGRL model from Fig. 13.

The notion of admissible sets gives rise to various possible extensions of an argumentation framework; we use the grounded extension, which is the minimal admissible set that contains all the arguments it defends all the arguments. In Fig. 14, there is one grounded extension, namely { A 4 , A 5 ′ , A 14 } . Note that there are many more semantics and types of extensions in the literature. However, for current purposes the grounded semantics are sufficient, also because the algorithms in Section 5.4 do not generate mutually attacking arguments.

The status of arguments can now be determined based on the grounded extension. Recall from the metamodel (Fig. 6) that an argument can be accepted or rejected.

In Fig. 14, we have that arguments A 4 , A 5 ′ and A 14 are accepted and arguments A 5 and A 13 are rejected. Returning to our larger example from Fig. 13, we can see that arguments A 5 , A 7 , A 13 and A 15 are rejected and all the other arguments are accepted.

5.3. Translating between RationalGRL and GRL

Now that we have formalized both GRL and RationalGRL models, we present algorithms to translate between these two models. Both of these two translation algorithms are straightforward, which is result of the fact that the two models are formalized in a very similar way.

OPEN IN VIEWER

Algorithm 1

GRL to RationalGRL translation

We start with the translation algorithm from GRL to RationalGRL, which is shown in Algorithm 1. The translation algorithm takes a GRL model GR = ( IE , Act , R A c t I E , Link ) (Definition 6) as an input. On line 2, it collects all the elements of the GRL model into a set Arg , which represents the set of arguments of the RationalGRL model. On line 3, the set of attack links is initialized with an empty set, and the new RationalGRL model ( Arg , Att ) is returned on line 4.

OPEN IN VIEWER

Algorithm 2

RationalGRL to GRL translation

The translation from a RationalGRL model to a GRL model is given in Algorithm 2. First. the arguments are each put in the corresponding GRL component sets. The procedure ComputeExtensions ( Arg , Att ) (line 2) computes the (grounded) extensions of the RationalGRL model (Definition 13). A GRL model is then generated from the grounded extension. This is done by iterating over all arguments in the extension in line 5. The switch statement on line 6 then does a case distinction on the type of the arguments – here, Arg . type refers to the type element in an argument tuple Arg  – and each case ensures the argument is put in the right GRL component. Finally, the algorithm returns a GRL model on line 16. As an example, compare the RationalGRL model in Fig. 13 to the GRL model in Fig. 2, where the latter is a translation of the former.

While we have defined a notion of a valid GRL model (Definition 7), we have not yet done the same for a RationalGRL model. We define a RationalGRL model as valid if and only if the RationalGRL to GRL translation results in a valid GRL model. Thus, we do not have to reiterate all conditions on a GRL model, but use the translation algorithm.

5.4. Algorithms for argument schemes and critical questions

We have now formalized a static representation of the RationalGRL framework. In this section, we formalize algorithms for applying argument schemes and critical questions. These algorithms are applied to RationalGRL models (Definition 11) and produce new arguments and attack relations.

As discussed in Section 4, the argument schemes and critical questions of Table 2 all lead to one of three operations: INTRO introduces a new RationalGRL element and DISABLE creates a new argument that attacks another argument. The REPLACE operation can be seen as a combination of INTRO and DISABLE: a new argument corresponding to a GRL element is added and this new argument attacks a previous version of the element or link. We will discuss all three operations in separate subsections below.

In all of the following algorithms, we assume that:

The algorithms are applied to some valid RationalGRL model RatGRL (Definition 15),

5.4.1. INTRO algorithms

The following arguments schemes and critical questions of Table 2 fall into this category:

AS0–AS12

CQ5b, CQ6b, CQ6c, CQ7b, CQ9, CQ10b

These type of algorithms are short, and consist simply of adding an argument for the element that is being added. We illustrate this with a single example algorithm for AS5.

OPEN IN VIEWER

Algorithm 3

AS5: goal with id i decomposes into task T

We have slightly reworded critical question AS5 in Algorithm 3. We assume that a goal G exists already with identifier i, and that some new task with name T is a decomposition of G. In the algorithm, on line 2 a unique identifier is created for the task, which is created on line 3. On line 4 an argument is created for the decomposition link ( mintId ( ) , Decomp , i , task _ id ) (Def. 5, going from the existing goal with identifier i to the new task with identifier task _ id . On line 5 the two arguments are added to the set of arguments Arg .

As an example, suppose we add the decomposition of goal “Generate cars”, expressed as argument A 5 = ( 5 , Goal , Generate cars , AND ) , into task “Keep same cars”, and suppose we have the following argument for the goal: . Adding the decomposition can be formalized as executing A S 5 ( 5 , Keep same cars ) , and results in two new arguments: A 7 = ( 7 , Task , Keep same cars ) and A 12 = ( 12 , Decomp , 7 , 5 ) , which are both added to the set of arguments Arg .

The critical questions of type INTRO are all very similar, with one exception: they require an answer. For instance, suppose CQ5b: “Does goal G decompose into other tasks?” is answered with: “Yes, namely into task T”. In this case, we simply obtain an instantiation of argument scheme AS5: “Goal G decomposes into task T”, which can be executed with Algorithm 3. This is the same for all the other critical questions of type INTRO. Therefore, they are not shown here as well.

5.4.2. DISABLE algorithms

As discussed before, algorithms of type DISABLE consist of adding a new argument attacking an existing argument. The argument that is added is not an argument for a GRL element or link, but it is rather a generic argument (Definition 8).

In the following algorithm, we assume the critical question is answered affirmatively, as indicated in the right-most column of Table 2. For instance, for critical question CQ0 “Is the actor relevant?”, we assume it is answered with “No”. In this case, an action is required. In contrary, if the answer to the question is “Yes”, no action is required.

OPEN IN VIEWER

Algorithm 4

CQ0: is actor with id i relevant? No

Algorithm 4 is executed when critical question CQ0 is answered with “No”. First, on lines 2 and 3, an argument is created for the critical question and added to the set of arguments Arg . Since this argument is not an argument for a GRL element or link, it is formalized as a generic counterargument ( mintId ( ) , GenArg , CQ 0 ) (Def. 8). The for loop starting at line 4 then iterates over all arguments for actors, where i is the id of the actor that is no longer relevant. The reason why there could be multiple of such actors is that the actor can be refined by an algorithm of type REPLACE. We will explain this in more detail in the example below. Then, on line 5, an attack link is created from the generic argument A to the argument for the actor A j . After executing the algorithm, all existing arguments for the actor with identifier i are attacked by a newly created argument A.

Consider for example the RationalGRL model in Figs 11, which consists of an actor and a generic counterargument. Let us reconstruct this model using the application of argument schemes and critical questions to an initially empty RationalGRL model ( Arg = ∅ and Att = ∅ ). The algorithm A S 0 ( Development team ) is executed, which results in A 1 = { ( 0 , Actor , Development team } ) . After this, critical question CQ0: “Is the actor Development team relevant?” is answered with “No, because the professor develops the software”. We can formalize this as executing algorithm C Q 0 ( 0 ) (Algorithm 4), which results in adding a generic counter argument A 2 = ( 1 , GenArg , CQ0: Professor develops software ) and an attack link ( A 2 , A 1 ) . The RationalGRL model now consists of two arguments, and the grounded extension is { A 2 } , which means that the resulting GRL model is empty, because generic arguments are not translated to GRL.

Algorithms for the remaining critical questions are omitted here since they are not very complex.

5.4.3. REPLACE algorithms

Recall that REPLACE algorithms create a new argument that attacks all arguments for an existing element.

OPEN IN VIEWER

Algorithm 5

CQ5c: is the decomposition type of element i e i correct? No, it should be X

While the original critical question CQ5c is specific to the decomposition between a goal and a task, Algorithm 5 is more generally applicable to the decomposition type of any IE, since all IEs have a decomposition type (Definition 2).

Let us go through this algorithm step by step. On line 2, a new argument A is created which is identical to original IE. On line 3 the decomposition type of the argument is changed to X – here, Arg . decomptype refers to the decomptype element in an argument tuple Arg . On line 4, the set IEArgs is assigned with all existing arguments for the input IE – here I E i is the set of all IEs with id i. The first two for loops on respectively lines 5 and 8 ensure that all attack links that existing from and to the previous versions of the IE are also carried over to the new argument A. Then, in the third for loop on line 10, we add attack links from the argument that has just been created to all existing arguments for the IE. This ensures that all previous versions of the IE are not part of the grounded extension, and as a result do not show up in the resulting GRL model. Finally, one line 14, the new argument is added to the set of arguments.

As an example, take Fig. 7. The initial RationalGRL model (before applying CQ10b), can be formalized as follows: RatGRL = ( Arg , Att ) , with Arg = { A 1 , A 2 , A 3 , A 4 , A 5 } and Att = ∅ , where:

A 1 = ( 0 , Goal , Simulate traffic , AND )

A 2 = ( 1 , Task , Dynamic simulation , AND )

A 3 = ( 2 , Task , Static simulation , AND )

A 4 = ( 3 , Decomp , 1 , 0 )

A 5 = ( 4 , Decomp , 2 , 0 )

Suppose algorithm C Q D e c o m p ( 0 , XOR ) is executed for this RationalGRL model. On lines 2 and 3 a new argument A 6 = ( 0 , Goal , Simulate traffic , O R ) is created. On line 4, IEArgs contains all arguments for element 0, which is { A 1 } (note this does not yet include argument A 6 , since it is not yet added to the RationalGRL model). The first two for loops on lines 5 and 8 do not do anything in the case of our example, since A 1 is itself not attack or does not attack any other argument. The for loop on line 10 ensures all previous versions of the element with id 0 are now attacked, which means in our case the ( A 6 , A 1 ) is added to Att . Finally, on line 14 the new argument A 6 is added to Arg .

The other REPLACE algorithms are similar to Algorithm 5, which can be used directly for CQ10c. For CQ12, we should make a small modification: instead of replacing the decomposition type of the IE, we replace its name. Since this is a very minor modification we do not show it here.

6.1. RationalGRL development process

We propose the development process shown in Fig. 15 to develop a RationalGRL model (see [20]). We assume that the initial GRL models have been created based on requirements specification documents. The rest of the steps are as follows:

OPEN IN VIEWER

Fig. 15.

The RationalGRL development process.

(1) Instantiate Argument Schemes (AS) – Here we start from the list of arguments schemes (Table 2). Whilst discussing the requirements, we select schemes from the list and instantiate them to form arguments for GRL IEs or links. In this way we build the GRL model by introducing new GRL elements (INTRO). For example, an argument scheme can be “Goal G contributes to softgoal S”. When an argument scheme is instantiated, it corresponds to an argument for or against part of a goal model.

(2) Answer Critical Questions (CQs) – After building or modifying the initial GRL model, we ask the relevant critical questions. Since each element in the GRL model corresponds to an instantiated scheme, we can look at Table 2) to see which questions are relevant given our GRL model. For example, for the argument scheme, “Goal G contributes to softgoal S”, there are two critical questions as follows: Does the goal contributes to the softgoal? and Does the goal contributes to some other softgoals?. When the analyst answers a critical question, a new argument scheme may be instantiated. This is done in step (3) when the operation INTRO is executed.

(3) Decide on Intentional Elements and their Relationships – By answering a critical question, one of the three operations (INTRO, DISABLE or REPLACE) is applied to the GRL model. Any of these operations impact the arguments and corresponding GRL intentional elements, modifying the initial GRL model into a RationalGRL model. Recall that INTRO means that a new argument scheme is created. That means, the current argument related to the critical question does not get attacked. In the case of DISABLE, the intentional element or its related links are disabled in the model. REPLACE introduces a new argument and attacks the original argument at the same time. This means that the original element of the argument scheme is replaced with a new one.

(4) Modify GRL Models – Based on the RationalGRL model of step (3), the GRL model can be modified: 1) a new intentional element or a new link is introduced; 2) an existing intentional element or an existing link gets disabled (removed) from the model; or 3) an existing intentional element or link is replaced by a new one. This results in a new, modified GRL model, which can be used as the basis for another cycle of the RationalGRL Development Process.

We can continue these four steps until there is no more intentional element or link to analyze or we reach a satisfactory model. In the next section, we will give an example of how our tool can be used together with the methodology to build a GRL model.

OPEN IN VIEWER

Fig. 16.

Overview of the RationalGRL tool.

The final requirement of our framework is that it has tool support (Requirement 5). This is important for various reasons: i) although there are various approaches attempting to combine goal modeling with argumentation, we are not aware of an existing tool support (see Section 8), ii) it allows us to conduct user studies in the next section, exploring the usability and usefulness of the RationalGRL framework compared to other goal modeling languages, and iii) tool support is an excellent demonstration to show that our formal framework and algorithms actually work. In this section, we briefly highlight some features of the tool, and we explain some of its current limitations. The tool can be accessed from: www.rationalgrl.com

Tool overview. The RationalGRL tool is an open-source web-based Javascript application, which runs on all modern browsers. It is based on our formalization in Section 5 and provides export functionality to jUCMNav, using the translation algorithm (Algorithm 2). It contains all of the GRL elements and links, except for beliefs and actors. We have argued in Section 2.2 that arguments can be seen as an extension to beliefs, which is the reason why we did not implement them. Actors are also missing, but they will be added in the future.

When the tool starts, the user is presented with a screen as in Fig. 16. This screen shows the palette of elements and links (top left pane), a canvas on which RationalGRL models can be built (top right pane), an ‘Export to GRL’ button (bottom left pane), and a details pane of the currently selected elements (bottom right pane). Elements and links can be added to the canvas by selecting them on the left and clicking on the canvas, thus capturing the INTRO operation (Section 5.4.1). This makes it possible to build and modify GRL models using the tool.

In total, there are four different types of details panes, which we now explain in turn.

IE and links details pane. Figure 16 shows the details pane for the goal ‘Generate cars’. The user can change the name of the IE by clicking ‘Rename’, which will update the naming history on the bottom of the pane. The naming history is (simplified) implementation of the REPLACE operation on IEs, since it is a very simple form of clarification. The main difference with jUCMNav is that we have a renaming history, so the user can see which names the IE had prior to the current one. If the IE has decomposition links, the user can change the decomposition type as well. Each IE and link has a set of associated critical questions, which can be answered in the details pane by click on the button next to the question.

Critical question details pane. The details pane can be used to answer the critical questions of the RationalGRL framework (Table 2): Fig. 16, for example, shows two critical questions for the goal IE “Generate cars”, namely “Can the goal be realized?” (CQ3) and “Is the goal relevant/useful?” (CQ11). Consider Fig. 17, where the softgoal “Easy to use” is questioned with the relevancy question (CQ11). It is possible to select the answer and provide an explanation for the answer.

If in the example of Fig. 17, the user selects “No” and clicks the “Answer question” button. A new argument is then automatically created that attacks the softgoal and the details pane shows the critical question as answered (Fig. 18). This is an implementation of a DISABLE algorithm (Section 5.4.2) similar to Algorithm 4: a new argument is added that attacks the existing argument.

Argument details pane. It is also possible to attack arguments by adding an “Argument” element and an “Attack” link manually. Consider, for example, Fig. 19. A new argument “Necessary”, which attacks the previously generated argument based on CQ11, has been added by the user. As the details pane shows, this new argument is not based on a CQ. It is further worth noting that it is possible to provide further a explanation in argument elements, allowing for more fine-grained rationalizations.

OPEN IN VIEWER

Fig. 17.

Critical question details pane for “is the softgoal relevant/useful?”.

OPEN IN VIEWER

Fig. 18.

Effect of answering CQ11: “is the softgoal relevant/useful?” with “no”.

Argumentation semantics. The RationalGRL tool computes the acceptability of arguments on the fly. In the example of Fig. 18, the original (argument for) softgoal “Easy to use” is rejected because its only attacker “CQ11 – Redundant” is accepted. However, if we then attack “CQ11 – Redundant” with a new argument “Necessary”, the original argument for “Easy to use” is again accepted because its only attacker is rejected (Fig. 19). Note that when computing the acceptability of arguments, the RationalGRL tool makes the assumption that there are no attack cycles in the model and that hence there is one, non-empty grounded extension (cf. Section 5.2) – if the user creates an attack cycle, an error message is shown.

OPEN IN VIEWER

Fig. 19.

Attacking an argument with a generic argument “necessary”.

Argument schemes and critical questions. The tool contains more argument schemes and critical questions than those that we initially collected in Table 2 – for example, we have added extra critical questions for contribution links from task to goal, which are not in Table 2. As mentioned before, our table is not meant to be exhaustive, and it is straightforward to add more argument schemes and critical questions if necessary.

Export to jUCMNav. GRL is implemented in the open-source tool jUCMNav [36].7

The connection to jUCMNav is maintained in the form of an export function from the RationalGRL tool to jUCMNav, essentially an implementation of the RationalGRL to GRL translation (Algorithm 2). RationalGRL models built in the RationalGRL tool can be exported to the .grl file format, which can be imported in jUCMNav. Note that in order for the GRL models to be rendered correctly, we recommend using the Graphviz dot8

Two examples of this export are provided in Appendix C. Figure 20 shows the model that was discussed earlier in this paper (Fig. 13) in the RationalGRL tool. Recall that translating this model to GRL provided us with the model in Fig. 2. This is also what follows from our export: if we export the RationalGRL model and then import the resulting GRL model in jUCMNav, we get the model in Fig. 21, which is the same as Fig. 2.

Our translation and export function uses the argument acceptability as a way of determining the GRL model. Take for example, the RationalGRL model in Fig. 22. A positive contribution is added from “Keep same cars” to “Simple design”. More importantly, an argument “not enough CPU” has been added to the model. This argument attacks the “Realistic simulation” softgoal and the “Create new cars” task, arguing that there is not enough processing power for either of these to be feasible in the traffic simulator. Now, if we export this to GRL, the GRL elements that are rejected or disabled (greyed out in Fig. 22) are not included. This can be seen in Fig. 23, where the jUCMNav GRL model that was exported from the RationalGRL tool is shown. The pair of Figs 22 and 23 also nicely shows the added value of RationalGRL: Fig. 22 shows that there can be a larger discussion and rationalization underlying even a fairly simple goal model such as the one in Fig. 23.

Limitations. For usability purposes, the REPLACE operation has been implemented differently than in our formal framework (Section 5.4.2). In the formal framework, a REPLACE introduces a new argument that replaces (and therefore attacks) all previous arguments with the same identifier. Including REPLACE like this in the tool would mean that, when the name of an element is changed, we would need to render all the previous versions of the element on the canvas of the tool. Since this would quickly become very cumbersome, we decided to implement a naming history for each element. For example, in Fig. 16, the goal “Generate cars” was previously named “Add cars”. In this way, a history is kept of how an element name was clarified without cluttering the model canvas. This feature is currently very limited and can be extended in various was, for instance by changing the IE type. Similarly, the decomposition type can be changed in the details pane (Fig. 16) without introducing new elements that attack the original ones (cf. Fig. 13, where the old decomposition type AND of “Generate traffic” is shown as an explicit argument).

7.1. Experiment design

The idea was to have our participants perform a small modeling task, and then ask them what they thought of RationalGRL. Thus, our experiment consists of three parts:

The instructions for these three parts are provided in a single document that we sent out to the participants.9

In order to allow the participants to compare their models to standard GRL models, after the modeling exercise we provide the participants with Fig. 2, which models the discussion for Part 2 (Appendix D) in standard GRL.

For Part 3, we provided the participants with a survey.10

In order to test our main hypotheses H1 and H2, we have to formulate the appropriate null hypotheses and alternative hypotheses. For H1 and H2, we hypothesize that the participants will, on average, rate the components of RationalGRL as useful (rating 4) or very useful (rating 5) for Q1, Q2 and Q3, and as making it easier (rating 4) or significantly easier (rating 5) for questions Q4, Q5 and Q6. In other words, the null hypothesis and alternative hypothesis for H1 and H2 are as follows:

H1₀ and H2₀: median rating for questions Q1–Q6 is 3 or lower.

Participants. We asked 16 participants in our network to participate in the experiment. Most of the participants were therefore either employed in industry or staff and (ex-)students from university. For those working in industry, all participants took the experiment outside of working hours to avoid conflicts of interest. Two thirds of the participants had either a PhD or Master degree, 20% a Bachelor degree, and one respondent responded with “Other”. All but one participant had a year or more experience with software development. The average experience was 6.2 years (standard deviation 6.7), with 10 participants having less than 5 years of experience, 6 participants having 8 or more years of experience and one participant having 25 years of experience. The participants also judged themselves to be quite competent in early-phase requirements engineering: on average they gave themselves a rating of 3.2 out of 5 (standard deviation 1.3) with half of the participants claiming they were at least “competent” (rating 4) or “very competent” (rating 5). However, experience with goal modeling languages was markedly less: the average rating was 1.9 (standard deviation 1.3), with 9 participants claiming never used a goal modeling language (rating 1), and only two participants displaying regular use (monthly, rating 4, or weekly, rating 5). The participants that had experience with goal modeling languages had mostly used i* (5 users), with 2 users being familiar with GRL and 2 users having used another goal modeling language.

First, the participants had to perform the modeling task. They produced RationalGRL models containing on average three arguments (in addition to goals, tasks, etc.) from the short transcript. Some of the examples of RationalGRL models, created by the users, can be found in Appendix D.1.11

After the modeling task, the survey questions were asked. We first asked the participants some questions about user experience. The users where generally positive about RationalGRL’s clear User Interface (UI), the fact that the tool gives a nice overview of the goals and opinions in a case and that it was straightforward to use and understand. With respect to weaknesses and improvements to the RationalGRL tool, the users mentioned additional UI functionalities such as the possibility to save models, having an “undo” function, and flipping the arrow after adding them. Users also suggested various language-related improvements. Some users mentioned they missed the possibility to attack links, while others mentioned that not all GRL elements are supported (for instance, it is currently not possible to add actors).

The next set of questions concerned the specific features of the RationalGRL modeling language and the difference with standard goal modeling languages. Table 3 shows the respondents’ answers to questions Q1–Q3, and Table 4 shows the answers to questions Q4–Q6. We concluded with two open questions about the comparison between RationalGRL and other goal modeling languages. When asked about the advantages of RationalGRL over standard goal modeling languages, many users agreed that making arguments explicit may force end users to have a more structured discussion: “Yes, having the possibility to add arguments seems quite useful”, ‘Generally I think its useful to explicitly document arguments of a discussion”, “Clear communication about argumentation and forcing people to think in those clear terms.”, “...you can add arguments and that you can answer questions that help you to develop arguments”, “It’s useful that discussion and explanation are close to the diagrams”, “a way to see how decisions are being shaped.”. Furthermore, one user stated that RationalGRL successfully “tries to capture the rationale behind the modeling process’ and “has a simple way to compute the status of the arguments”.

When asked about the weaknesses, users mentioned the complexity as the most important weakness: “The apparent increase in complexity might lead to negative perceptions”, “adding yet another layer of complexity scares me”. The concern that the modeling process may involve too much cognitive overhead was mentioned a few times though: “I think the overhead of inputting a (detailed) discussion in a structured manner into any system makes adoption difficult”, “the manual input is too complex and takes too much time. An automated process of parsing the conversation log would be much more helpful”, and “I still believe in the value of arguments, but there should be less confusing ways to capture them”. One user mentioned that complexity and cognitive overload is a problem with goal modeling in general: “Goal models are already complex (...) I have worked for years on the effect of context on goal models, and my conclusion is that this was very interesting academic work but with close-to-zero practical implications, unfortunately.”.

Hypothesis H1. The first analysis concerns hypothesis H1, whether the components of RationalGRL – i.e. arguments, critical questions and determination of the status of arguments – are a useful extension to standard goal modelling in GRL. Table 3 summarizes the medians, modes, and p-values for a one-tailed single-sample sign test for the relevant questions Q1–Q3. We chose the simplest non-parametric test available so as to have to take as few assumptions as possible w.r.t. the distribution of answers for each question. As it is evident from the results, the participants were generally positive about the components of RationalGRL. For Q1 and Q3 the p-values are both significantly lower than α = 0.05 , which means that for these questions we can accept H1 _a and reject H1₀. This means that the addition of arguments and the use of arguments to determine the acceptability of elements of the goal model are deemed to be useful to very useful by the participants. As can be seen in Table 3: no participant found the addition of arguments and argument status useless or very useless, and 78.6% (Q1) respectively 85.7% (Q3) found the new components useful or very useful. Furthermore, these findings fit with the answers to the open questions (see Section 7.2). For Q2, the p-value indicates that the result is not significant with α = 0.05 , which means we cannot reject the null hypothesis H1₀. This means that the participants did not think the addition of critical questions like to the RationalGRL tool was significantly useful. Thus, we can say that the addition of critical questions is perhaps not as useful as we had hypothesized. This again fits the descriptive statistics in Table 3: 21.4% of respondents found critical questions “useless”. Furthermore, in the open questions a number of participants indicated that the exact difference between some of the critical questions was unclear, and that some of the critical questions seemed unnecessary, at least for the small modeling exercise given in the experiment.

Hypothesis H2. The second hypothesis states that components of RationalGRL make it easier to express one’s opinions and beliefs about a goal model, determine the effect of one’s opinions and beliefs about a goal model and communicate one’s opinions and beliefs about a goal model to other stakeholders. Table 4 provides a summary of the statistical analysis of the relevant questions Q4–Q6. As can be seen, all the scores are significantly higher than the median of 3 for α = 0.05 , which means that we can accept H2 _a and reject H2₀ for all the separate questions Q4–Q6. In other words, the fact that the participants rate the components of RationalGRL as making it, on average, easier to reason with their beliefs and opinions about a goal model is very unlikely to be due to chance.

With respect to Q4, participants stated that arguments make it much easier to express their opinions. None of the participants found expressing their beliefs (much) more difficult, and 69.3% found it easier or much easier. This corresponds to the answers to Q1, where participants said the arguments were a useful addition. For Q5, participants agree that RationalGRL makes it easier to determine the effect of one’s beliefs on the goal model. Combining this with the results for question Q3, which asked about the usefulness of determining the status, we can say that the participants overall found this a useful feature which makes working with conflicting beliefs easier. One participant remarked that “The reasoning [in the example] seems simplistic [...] the added value of [the formal argumentation] would be justified if the reasoning is more complex”. Thus, the simplistic case in the modeling exercise possibly detracts from the results for Q5 and Q3. Finally, participants were less sure that arguments would help in communicating the goal model to others. This may be due to the fact that in the experiment, communication of opinions to others was not really tested, leaving the participants to guess what this effect would be.

7.4. Threats to validity

Internal validity is about the validity of the experiment results given our experiment setup and interpretation of findings. The main threat here is that the modeling task the participants were asked to do is not realistic. The task is quite small, and based on an existing transcript of a requirements discussion. In a realistic setting, users of RationalGRL would model much more complex requirements together with the stakeholders. The problems with complexity and cognitive overload only start to play a role for larger models, so while the participants were generally positive in the ranking questions Q1–Q6, some reservations were expressed in the open questions as to whether this approach would work in a realistic setting.

Another threat to internal validity is that many of the participants had no experience with goal modeling languages at all. Hence, the perceived usefulness of the arguments could be influenced by the fact that the participants, for the first time, were given a goal modeling language to perform structured requirements analysis. In other words, perhaps they found a structured modeling language generally quite useful. One way to better check the difference between standard goal modeling and RationalGRL would be to first provide users with a standard goal modeling tool and then have them use RationalGRL. An even more thorough way to test the usefulness of RationalGRL over, for example, standard GRL would be to compare a test and control group, where the test group uses RationalGRL and the control group uses standard GRL. We leave these types of further empirical studies for future research.

External validity concerns the generalizability of the results. As the number of participants is fairly small (16), this threatens the generalizability of our findings. However, the type of participants is realistic: software engineers having a lot of experience with requirements engineering, as opposed to, for example, engineering students with little experience. Furthermore, the results are in line with existing research on design rationale. On the one hand, these existing studies, like our study, show that structured modeling languages suffer from the common problem of high cognitive overhead [43]. One the other hand, these studies show that the inclusion of argumentation and critical questions in design and requirements discussions improve the reasoning [39,45].

7.5. Discussion

Our results point in some interesting directions for further discussion and research.

Taking the above into account, it seems natural in future work to focus more on the RationalGRL development process.12

8.1. Related work

Design rationale. Argumentation in software design has been the subject of research in so-called design rationale (DR) [34,42], which deals with an explicit documentation of the reasons behind decisions made when designing a system or software architecture. Already in 1970, the first article on Issue-based Information Systems ([29]) talked about “argumentative processes” in information system design. IBIS was further developed in the 80’s and 90’s in the influential work on issue-driven design methods such as gIBIS [11] (see also [34,42] for further work on arguments or rationales in software design).

DR looks at issues, options and arguments for and against these options in the design of, for example, a software system. Similar to the literature on goal modeling, much of the traditional DR literature provides modeling languages and diagramming tool support for building design rationales. It is in this diagramming functionality that the link with argument diagrams from philosophy, law and AI [22,40] has been made, where argument diagrams represent reasoning from premises to conclusions. One of the problems of such methods and tools is the cognitive overload that results from having to learn and use such tools at the same time as having to discuss and think about complex software designs. Furthermore, not much of the research on design reasoning has empirically investigated which techniques and how much information designers use in design reasoning, which information is missing, and how the use of design reasoning techniques affects design discussions and eventually the quality of design discourse. More recent research on DR, therefore, moves away from the idea that all decisions have to be explicitly diagrammed and focuses more on empirically investigating how critical reflection can help when designing [39,45], or which parts of the design process are best explicitly documented [16].

Two relevant articles [9,17] capture software design processes as argumentation dialogues between parties. Black et al. [9] provide a more formal rendering of the initial informal ideas by Finkelstein and Fuks [17], and propose a simple argumentation dialogue protocol. Similar to us, they base their argumentation on an analysis of transcripts of software engineers arguing in the Traffic Signal Simulator task [46]. They provide different argumentative move types, such as propose and challenge (similar to our critical questions). This preliminary work showed how techniques from computational argumentation can be applied in software design discussions.

Software design and requirements engineering (RE) are very closely related [38] and hence, the insights from the DR literature are directly applicable to RE – although for reasons unknown, the DR literature has been largely ignored by the goal-modeling community. The RationalGRL framework essentially incorporates the core ideas from DR into goal-oriented requirements engineering by explicitly including arguments for and against the various options into a goal model. In this respect, we may seem to go against the trend in DR research to focus less on diagramming and more on guidelines for design processes. There are two reasons for this: first, as opposed to DR diagramming, goal modeling is still very much an active area of research [21] with clear practical applications [37]. Secondly, we note that RationalGRL also includes a development process and critical questions that encourage reflection when thinking about the possible goals and functionalities of a system, which connects with the research ideas on DR [39,45].

Requirements engineering. There are numerous examples in the RE literature that mentions arguments, questions or conflicts in some way. Already in 1997, Smith [44] proposed a framework to develop arguments, which related a model of a proposed software system to a model of its environment, and showed that these together achieve properties described by a model of requirements. In a similar vein, Lockerbie et al. [30] describe procedures that analysts follow to use satisfaction arguments in combination with i ∗ . Horkoff and Yu [25] introduce analysis procedures for the i ∗ goal-oriented framework, allowing users to ask questions such as “what if?”, or “are certain goals achievable? how? or why not?”. Finally, Hassin and Amyot [24] analyze the correctness of GRL models by performing statistical analysis on generated questionnaires, which allows for early detection of potential conflicts between stakeholders of the system. The mentioned work, however, does not consider arguments in a similar way as RationalGRL, that is, to include the explicit opinions of stakeholders.

A number of approaches in the field of requirements engineering take arguments into account explicitly. One early example comes from Haley et al. [23], who use formal logical arguments to show that the system behavior satisfies certain security requirements, and more informal arguments to capture and validate the assumptions underlying the system behavior. This system behavior is defined by the tasks it executes and thus, arguments are given for and against system tasks, similar to how beliefs and counterarguments can be provided for tasks in RationalGRL. What Haley et al. leave implicit in their argumentation are the goals of the stakeholders on which the system tasks depend – they include the goals in their framework and mention that there will often be conflicting goals between stakeholders, but they do not explicitly model them. Furthermore, the argumentative part of their framework does not include formal semantics for resolving conflicts among arguments or determining the acceptability of arguments. Yu et al. [52] further extend the framework proposed by Haley et al., including algorithms for Dung-style argumentation semantics [13] and a database including specific ways in which to attack (or mitigate) risks. This extension can be likened to a set of critical questions for risks and security requirements (cf. Yu et al. [52] Section 3.1).

Another recent example of the use of arguments in goal-oriented requirements engineering is the work by Murukannaiah et al. [35], who propose Arg–ACH, where beliefs underlying conflicting goals can be made explicit using argumentation. Murukannaiah et al. start with the basic technique of Analysis of Competing Hypotheses (ACH), where for conflicting goals, beliefs that are consistent and inconsistent with goals are included in a matrix and counted. They, then, extend this technique into Arg–ACH: instead of just indicating whether a belief is consistent or inconsistent with a goal, each belief becomes an argument for or against a goal, which is, then, modeled using the Carneades tool [22]. Belief scores are assigned to arguments, which can be aggregated to provide one’s belief in a goal. The arguments for and against goals can be based on argument schemes, and critical questions can be used to find new arguments for or against the goals or the existing arguments. One example provided in the paper is the argument scheme from expert opinion, which allows one to draw conclusions based on expert statements and subsequently question, for example, the objectivity and veracity of the expert using critical questions. Murukannaiah et al. conducted an experiment in which two groups, one with ACH and one with Arg–ACH, performed an analysis of several conflicting goals regarding security at transport hubs. They found that, while the group that used Arg–ACH took longer, they also covered more possibilities in their belief search and the conclusions were more consistent among the group.

Another example of argumentation in RE concerns the use of argumentation in requirements elicitation. Ionita et al. [26] propose a simple argumentation dialogue game in which risk assessors try to attack each other’s arguments for certain risks attached to a system design. Dung’s semantics [13] are then used to determine the risks that are still valid, and those that have been successfully rejected. Elrakaiby et al. [14] use argumentation to explain ambiguity. They define when a statement by a client who is being interviewed about the requirements of a system presents an inconsistency or an insufficiency. An inconsistency can be either with the client’s previous statements or with the requirement engineer’s beliefs. An insufficiency occurs when an analyst needs more information from the client to accept a client’s statement.

It is clear from these literature that arguments play a core role in RE. Murukannaiah et al. [35] show that critical reflection using argument schemes and critical questions – in the same way that RationalGRL proposes – improves the quality of the reasoning in the RE process. Elrakaiby et al. [14] provide a case study similar to our research, identifying, many counterarguments specifically with respect to realizability, relevance and clarity (cf. CQ2a, CQ3, CQ11 and CQ12 in Table 2). Similar to RationalGRL, the use of Dung-style argumentation semantics to compute the acceptable claims in RE is also advocated by the literature [14,26,52].

The work on RationalGRL puts the insights from the literature in a broader framework. For example, there is a specific focus on security requirements [23,26,52] or the reasoning is about single goals or tasks instead of about the wider context as represented in a goal model [14,23,35,52]. RationalGRL provides a generic and extensible framework for arguing about goals and tasks in RE. At the moment, there is only a “generic argument” in addition to arguments about goals and tasks. However, new argument schemes and critical questions about, for example, security risks or expert opinions, can be easily added: the metamodel (Fig. 6) accommodates this and the formal specification in Section 5 is set up in such a way that extending the definition of argument and adding new algorithms for specific critical questions is easy.

Goal modeling. Argumentation has already been included implicitly in existing goal modeling languages. At its core, goal modeling is a process of argumentation: high-level goals are reasons for lower-level goals, tasks and resources. Hence, refinement and decomposition techniques used in requirements engineering [47] can be seen as explicit argumentation steps in the goal modeling process. Take, for example, CQ2 of PRAS (Section 2.3), which asks whether there are alternative ways for realizing the same goal. Providing an alternative subgoal or task in a goal model is, then, an explicit argumentative move in the discussion. In this sense, a goal model provides a justification for itself. This idea is also prevalent in our RationalGRL framework: many arguments are in fact GRL elements, and many critical questions can be answered by introducing new GRL elements. However, as was already argued in Section 1 (and also by [27]), argumentation produces different, richer and complementary information to the goal model. The goal model is the product of a process of argumentation, and does not include, for example, goals and tasks that were at some point considered but discarded. Furthermore, for goal models, it is only possible to determine the satisfiability of goals given the possible tasks and resources; what cannot be determined is the acceptability of goals, that is, whether they are acceptable (cf. Definition 14 given potentially contradictory opinions of stakeholders).

More explicit rationales for goal models are included in the original GRL specification [2] in the form of belief elements, which can be attached to other intentional elements (goals, tasks). The content of a belief is a textual description, which can explain, for instance, the rationale for including a goal in a model. In this regard, they seem like arguments or reasons for the different elements in a goal model. However, in the literature the precise usage and semantics of belief elements in GRL remain unclear: according to the metamodel [2], they can be connected to other intentional elements (including beliefs) through any type of link, but in jUCMNav tool, there is a separate belief link for connecting beliefs to goals and tasks only, and beliefs are not used in any of the algorithms to compute the satisfiability of goals and tasks. We are also not aware of many examples of the usage of belief elements in the literature. Thus, GRL belief elements can be understood as metadata that can be attached to an element.

There are further advantages of using RationalGRL over GRL with belief elements. For example, none of the GRL specifications allows to attach beliefs to other elements, thus it is not possible, for example, to provide a rationale for why some tasks positively contributes to a goal. Furthermore, the RationalGRL arguments follow clear schemes with associated critical questions explicitly tailored towards reasoning about goals and actions or tasks, while beliefs in GRL offer no further guidance as to what kinds of rationales can be given for elements in a goal model. In addition, beliefs in GRL seem not to influence a goal model in any way. Arguments in RationalGRL, in contrast, play an important role in forming the resulting goal model: if an element is defeated by an argument, it is not part of the resulting goal model.

Goal modeling and argumentation. There are several contributions to the literature that explicitly relate argumentation-based techniques with goal modeling. Bagheri and Ensan [5] and Mirbel and Villata [32] use abstract argumentation frameworks (see Definition 12) to capture individual goals and the relations between them as modeled in a goal model, so that consistent (i.e., acceptable) subsets of goals can be computed using the relevant argumentation semantics [13]. For example, if in a goal model, there is a conflict between goals G 1 and G 2 , there is an attack between the arguments representing these goals and hence G 1 and G 2 cannot be in the same extension. Or if there is a dependency link between G 1 and G 2 , then G 1 should be in every extension G 2 is in and vice versa. AND- and OR-decomposition are also modeled as such, that is, if G 3 AND-decomposes into G 1 and G 2 then G 1 and G 2 should be in every extension G 3 is in, and if G 3 OR-decomposes into G 1 and G 2 then either G 1 or G 2 should be in every extension G 3 is in.

Modeling goal models as argumentation frameworks allows the computation of consistent sets of goals and tasks in a goal model. In RationalGRL, we have opted not to provide such an argumentation-theoretic semantics to goal models, because GRL already has quite fine-grained satisfiability semantics [2], which take into account conflicts, decompositions and dependencies. RationalGRL focuses on what is not captured in the work discussed above [5,32], namely the arguments and beliefs underlying a goal model, and the way these arguments and beliefs interact with the elements of the goal model. If desired, however, it would be easy to provide Dung semantics similar to [5,32] for goal models, as the elements of a goal model are already arguments in RationalGRL (Fig. 6).

The contribution most closely related to ours is the work by Jureta et al. [27]. Jureta et al. propose a “Goal Argumentation Method (GAM)” to guide argumentation and justification of modeling choices during the construction of goal models. GAM is a high-level decision process, in which alternative solutions for an RE problem are evaluated and compared using argumentation. Jureta et al. use a well-known fictitious example of an argumentative discussion regarding a meeting scheduler, in which a goal model is being built by the stakeholders proposing tasks, goals, and alternative solutions for goals. They include clarification as an important step in their GAM process, and discuss various types of clarity problems (vagueness, ambiguity, overgenerality, synonymy) and basic techniques for dealing with them (e.g. thesaurus checks, labeling vague expressions).

The GAM process is essentially a generic, high-level process for problem solving, not specifically tailored towards goal modeling. In this sense, the RationalGRL Development Process in Fig. 15 can be seen as a more specific version of GAM explicitly meant for goal modeling. The argument schemes and critical questions in RationalGRL provide clearer handles for goal modeling (cf. requirement 4 of our framework, Section 1). Furthermore, the RationalGRL framework adds full tool support for the goal modeling and reasoning process, which GAM lacks. GAM does contain more specific ways of dealing with different types of clarity problems; in RationalGRL clarification is captured as a single critical question (CQ12, Table 2). However, as argued above, further critical questions can easily be added to RationalGRL if needed.

Jureta et al.’s framework also includes an argumentation part, where reasons for conclusions are given as formal structured arguments.13

RationalGRL allows for two-way translation between arguments about goals and standard goal models (cf. Section 5.3). In contrast, Jureta et al. only provide a mapping from goal models to structured arguments, and the step from structured arguments or argument diagrams to goal models is never formally defined. In previous work on the RationalGRL framework [48,49], we extended Jureta et al.’s work and translated argument diagrams to GRL models (an automatic translation tool is discussed in [49]). This gives us two complex diagrams, an argument diagram and a goal diagram, and a mapping between them. This was, in our opinion, ultimately an unsatisfying solution given the problems and requirements described in Section 1. One problem is that the argument diagram is at least as complex as the GRL diagram. This means that any stakeholder trying to understand the discussion has to parse two complex diagrams containing goals, alternative solutions, tasks, and so forth. RationalGRL overcomes this problem by integrating argumentation and goal modeling in a single modeling language.

RationalGRL provides a basic framework for argumentation in requirements engineering, and all aspects of this framework are intended to be fully extensible. We see many open issues to be explored in future research.

Specific domains. The argumentation schemes and critical questions presented in this paper are focused only on the core goal modeling process, that is, the discussion about the goals and functionality of an information system. In the RE process, there are many more domain specific discussions that also take place, such as discussions involving expert opinions [35], discussions about security requirements [23,26,52], architectural principles [31], legal compliance [19], and so forth. From a knowledge engineering perspective, including argument schemes and critical questions associated with these domains in the RationalGRL framework is a time-consuming task. However, from a formal perspective adding new schemes and questions is easy: as already discussed by Bex et al. [8], critical questions for argument schemes always lead to either new information being introduced, new information replacing old information, or new information attacking old information. This corresponds to the INTRO, REPLACE and DISABLE operations in the RationalGRL framework, which have been formalized in Algorithms 3–9 in Section 5.4, hence it would be suitable for other types of schemes and questions as well.

Formal argumentation. The amount of theory from computational argumentation used in this article has been relatively small. Our intention is to create a bridge between formal theories in argumentation and the practical tools in requirements engineering. Now that the initial framework has been developed, however, it is worth exploring what additional techniques computational argumentation has to offer in more detail. For instance, in our framework, we do not include the possibility of expressing preferences between mutually inconsistent arguments. Using the work by Modgil [33], it is possible to provide explicit arguments for preferences, thus allowing us to make a reasoned choice between two options. Furthermore, it would be interesting to investigate the link between RationalGRL, arguments and argumentation frameworks about goal models, and formal argumentation dialogues from the literature. Black et al. [6] have already shown that discussions can be captured as such dialogues, and it would be interesting to look which types of arguments or argument schemes correspond to their different move types in design discussions.

Belief revision. An interesting avenue for further investigation is the theoretical foundation of RationalGRL. Since it is one of our main goals to develop a framework that can be used by practitioners, we developed a formalism that is geared towards implementation using algorithms and a compact propositional logic. However, there is a long strand of research in Artificial Intelligence focusing on developing postulates that characterize how an agent can revise its knowledge base in a rational way [15]. The most famous postulates are the so-called AGM postulates [1]. It would be very interesting to study whether (a variation of) these rationality postulates can be used to characterize the algorithms we have developed in this article, and more generally, what rationality means in our setting. While there is formal work in Artificial Intelligence on the connections between argumentation and belief revision (see e.g. [15], there is very little work on applying the combination to requirements engineering. The only work we are aware of that deals with his subject is by Gervasi and Zowghi [18], who use a logical framework for belief revision to detect inconsistencies in software specifications and then revise those specifications so that the inconsistencies are removed. They do not explicitly connect their framework to goal models, however, and do not connect their framework to specific argumentation using schemes in the way RationalGRL does.

Process and tool support. The current tool is a prototype that implements the RationalGRL framework in a fairly simple way. That is, by offering the possibility to argue and ask critical questions about a goal model, and then export this goal model for further analysis in jUCMNav. Integration of RationalGRL elements into jUCMNav, or at least creating an import function to import jUCMNav goal models into the tool would allow us to tap into the large amount of research and development that is being carried out with jUCMNav. Furthermore, the tool could be expanded to include new features described above, such as new argument schemes and reasoning with preferences.

The user evaluation of the tool revealed that in order to implement the tool in realistic settings, we need to incorporate arguments in the goal modeling process in a more collaborative way. Extending the tool for collaborative online goal modeling, similar to Github is one of the possibilities: since the renaming (i.e. replacement) and deletion (i.e. disabling) of elements are all logged, it is easy for a stakeholder to continue working on a model that was made by another stakeholder. In addition, critical questions can be included for other stakeholders to answer at a later date. As a formal underpinning of this asynchronous communication between users, it would make sense to capture requirements engineering and software design processes as dialogues between parties [9,17], which are natural fit with the question-answer format employed in the RationalGRL framework.

Empirical validation. We have performed a coding analysis and found more than 200 occurrences of arguments and questions. In this study, we performed a simple experiment with 16 participants14

The process of constructing a goal model involves discussions between requirements engineers and a group of stakeholders. While it is possible to capture part of this discussion process in a goal model, for instance by specifying alternative solutions for a goal, not all of the arguments can be found back in the resulting model. This makes it not only more difficult to understand the model, but other stakeholders may end up having similar discussions throughout the design and development phase as well. Furthermore, the disconnect between goal models and their underlying beliefs and opinions may lead to a poor understanding of the problem and solution domain.

In order to solve these problems, we present RationalGRL: a framework for integrated goal modeling and argumentation. We extend the well-known goal modeling language GRL with argument schemes and critical questions that can be used to analyze and guide stakeholders’ discussions about goal models. Furthermore, we provide formal argumentation semantics for the new RationalGRL language. Our approach, thus, provides a rationalization for the elements of the goal model in terms of underlying arguments, and helps in understanding why parts of the model have been accepted while others have been rejected. In the introduction, we identified five important requirements for our framework. Below, we discuss how the RationalGRL framework meets these requirements.

Our list of argument schemes and critical questions in Table 2 was constructed by performing a coding analysis in which we analyzed 153 pages of transcripts of discussions among designers of a traffic simulator information system. In this way, we ensure that the argumentation techniques capture the actual discussions of the stakeholders or designers in the early requirements engineering phase (Requirement 1).

The metamodel of the RationalGRL framework in Fig. 6 clearly specifies the formal traceability links elements of GRL goal model and the underlying arguments. (Requirement 2). In addition to this metamodel, we provide formal semantics for RationalGRL by formalizing the GRL language in propositional logic and rendering arguments about a GRL model as a formal argumentation framework [13]. We, then, formally capture the link between argumentation and goal modeling as a set of algorithms for applying argument schemes and critical questions about goal models. These formal traceability links allow us to compute the effect of the arguments and counterarguments proposed in a discussion on a GRL model (Requirement 3). In other words, we can determine whether the elements of a GRL model are acceptable given potentially contradictory opinions of stakeholders. Thus, we add a new formal evaluation technique for goal models that allows us to assess the acceptability of elements of a goal model (in addition to their satisfiability [2]).

One of our main goals is to provide means for RE practitioners to capture the underlying arguments of goal models by using the RationalGRL framework (Requirement 4). To this end, we propose a development process, which consists of developing goal models and posing arguments based on schemes in an integrated way.

Finally, we have implemented the RationalGRL tool, a web-based prototype,15