Classification of retinal vessels using a collaborative agent-based architecture

2.1. Retinal vessels researches

The digital colour fundus photographs are an innocuous and non invasive technique to obtain the retinal vascular tree. Fundus examination is a non-invasive evaluation of the vascular damage caused by multiple factors. The damage can be observed clearly in the colour fundus images, microaneurysms, neovascularizations, hemorrhages, venous beading, hard exudates, macular oedema, etc. This paper focuses on automatic retinal vessels segmentation in 2D colour fundus image. Photographs taken by a fundus camera provide a clear image of the retina in which medical experts can discern blood vessels, their thickness, length, positioning, etc. to diagnose pathologies such as stroke [52], arterial damage [47], metabolic syndrome [50], etc.

The research field that connecting retinal blood vessels and pathologies is not a recent discovery; retinal images have been used and analyzed manually for decades [15]. The current novelty is that these studies can now use computer visualization techniques that were not previously available. Today, computer-based visual analysis is used to detect the vessel structure in retinal images. Different works, including [53] and [20], suggest the application of morphological methods like erosion or dilation [45], which are widely used when extracting different kinds of characteristics from images with a previously known shape. Algorithms based on adaptive filtering techniques to highlight the vessels and detect them easily have been used too in works such as [10]. These algorithms normally are systems with a linear filter that has a transfer function controlled by variable parameters and a means to adjust those parameters according to an optimization algorithm. Adaptive filters are used in applications in which some parameters of the desired processing operation are not known in advance or are changing. There are others methodologies such as [33] or that proposed in [16] for the detection of diabetic retinopathy, that follow a four block analysis that includes a preprocessing step, a shape estimation step, a feature extraction step and a classification step. However, existing works usually only apply one technique in every step. In this work it will be possible observe how the results can be improved by combining more than one techniques in every step.

A method using linear operators and Support vector machines (SVM) is shown in [38]. [44] presented a supervised method for retinal vessel segmentation. In [46] is described a segmentation method based on image ridges extraction.The authors acquired a fundus image database named DRIVE from a screening program, which is widely used for the testing of fundus images processing technology.

There are supervised methods for retinal vessel detection using Artificial Neural networks (ANN) such as [32]. The retinal photograph is represented by a 7-D feature vector composed of gray-level and moment invariants-based features. And in [19] is proposed a method of retinal vessel segmentation using multilayer perceptron ANN. A summary of the technologies applied by other methodologies for the analysis of retinal images can be found in [49].

Part of the medical group that collaborated in this research was previously involved in the development of another software tool (AVIndex [25]) which obtain information similar to that extracted with the software tool proposed in this work. However, the medical team realized that it was incomplete because data related to the area, length or vessel position were missing, and the analyzed area was not large enough to discover relationships with pathologies. These difficulties have been solved and refined in the investigations carried out and here the most complete version of the proposed system is shown.

2.2. Agreement technologies and multi-agent systems in e-health

The application of ICTs (Information and Communication Technologies) in healthcare is known as e-Health, electronic health, medical technology or telemedicine, among others. The new rationalization and efficiency philosophy adopted nowadays in medicine, together with the quick evolution of ICTs is mostly responsible for the e-health [17].

Every health area is susceptible to improve economically as well as when consuming resources or offering facilities to the patients. Medical equipment is probably the main area where a better scope for improving is offered. e-health is a great open environment distinguished by shared and distributed decision making and management of care requiring the communication and coordination of complex and diverse forms of information between involved organizations and people. Therefore, both MASs and health-care can have advantages from their joint: MASs have the suitable features for the modeling and performance of current and future health care applications; health-care scenarios offer the suitable requirements for experimenting at best the multi-agent technologies and, so, for giving a significant contribution to their success and evolution. The agents have characteristics such as autonomy, reactivity, reasoning, social abilities and pro-activity which make them appropriate for developing distributed systems based on highly dynamic scenarios. Software agents have been widely used for decades as a solution to solve complex problems and more recently they have been used to solve e-health problems [5,21].

Within the current tendency in development of MASs is possible to observe the use of organizational concepts in their design, which helps to focus their work in the field of e-health, since they establish a connection between a medical organizations and an organization at the software agent level with regards to structure, functionality, etc. Virtual organizations [18] are a way of understanding system models from a sociological perspective, which is closely related to the area in question. From a business perspective, a virtual organization model is based on the principles of cooperation among businesses within a shared network, and take advantage of the heterogeneous elements that provide the flexibility and quick response capability that form the strategy aimed at customer satisfaction.

Virtual Organizations of agents (VO) [41] consider social integration and performance aspects that must be taken into account when functionalities are created outside the system. Social integration is understood as the way agents organize themselves in the system in the form of a society. They can form coalitions, groups or sub-organizations with different objectives to be carried out in the system [39]. Performance aspects are understood as all the aspects that influence the performance of the system (interoperability, integration of other components, etc.). In this sense, functionalities are created outside the system is understood all modules or software components that can be implemented outside the MAS, for example, another segmentation module.

VO are aimed at the interoperability between different systems, distribution of resources, and the lack of dependency of programming languages [41]. Functionalities are linked by means of standard communication protocols that must be used by applications in order to share resources in the network. The compatibility and management of messages between functionalities is an important and complex element in any of these approaches.

However, the development of a MAS, either from the traditional point of view, or from an organizational point of view, can be an extensive process. Especially when there are very different techniques to apply by agents. In this case, the algorithms of identification and image processing require special characteristics in which the agents have to work as heterogeneous entities. During this process, it is convenient to utilize other kind of techniques, such as AT tools [3,36].

Agreement is one of the critical social concepts that helps human agents to face with their social environment and is in all human interactions. In fact, without agreement there is no cooperation and ultimately social systems cannot emerge. AT is a new approach of Distributed Artificial Intelligence for implementing large-scale open distributed computer systems. It is focused on techniques that enable software components to achieve agreements on the mutual performance of service.

AT can be successfully applied in a large number of domains [8,35] to solve complex and dynamic runtime problems. In [35] is analyzed the concept of agreement from a computing perspective. In particular, the capability of software components to dynamically forge and execute agreements at run-time, and the emerging field of agreement Technologies for the construction of large-scale open distributed software systems. This is the case of vessel identification, where agents have to determine the vessel type taking into account different parameters (defined by the retina morphology) and negotiate to reach a global agreement regarding their type. Such tools facilitate and improve the overall process, making the end result a consensus among all experts in the system, in this case, the agents of the organization.

The following subsections detail the steps followed in the methodology and the collaboration between the agents of the proposed architecture to carry them out. Step 7 of vessel classification will be the one that includes the process of negotiation between agents, in this case type VesselType and VesselManager to reach the final classification.

3.1.2. Image preprocessing

Then the system changes to a state of “Image preprocessing” where the image is processed by the Preprocessing Agent. The image, usually RGB, is combined in a single layer image (grayscale) by using different filters.

The result of this preprocessing step can be seen in the top middle of Fig. 2, where the original RGB image a) is directly converted to grayscale b) and converted by using a filtered combination c). This combination mixes red (R), green (G) and blue (B) channels in a specific mode to highlight the difference between blood vessels and the background to improve the analysis results.

OPEN IN VIEWER

Fig. 4.

Optic disc alignments.

OPEN IN VIEWER

Fig. 5.

Retina and optic disc size relationship.

3.1.4. Locating optic disk

The papilla or optic disc is another important element in the photograph that has to be situated. All the blood vessels access the eye through the optic disc. Vessels inside the optic disc are too close to detect their morphology individually, so the analysis starts from its edge. The localization step is executed by the OpticDisc Agent and it is essential for the following steps.

Fundus images can be taken with the optic disc oriented in different ways:

with the optic disc in left or right side (Fig. 4i)

with the macula centered (Fig. 4ii) or

with the optic disc centered (Fig. 4iii)

In this case, for the study, we require to obtain as much information as possible, and therefore, only the images with the optic disc in the middle are considered, because the Region Of Interest (ROI) has no hidden parts.

The ROI where the analysis is executed is defined as the area from the optic disc border and its concentric circle whose radius is three times higher. With this definition, the analysis area will be different for every retina image as the size ratio of optic disc size can vary from 1:8 the retina size to 1:6 optic to retina size. Therefore, this is the radius value that has been established for all of them. An example of this relationship is shown in Fig. 5.

The optic disc is now located by applying pixel intensity, adaptive thresholding and morphologic filters to locate the lighter blob with the optic disc morphology. Its center of mass is defined as the middle of the circle that contains the optic disc. The result of every filter can be seen in Fig. 6.

OPEN IN VIEWER

Fig. 6.

Visual analysis steps of algorithm for optic disc location.

3.1.5. Identifying eye

Once the area containing all the data to extract is determined and the location of the optic disc is known, the eye side (left or right) can be determined. This information can be interesting when analyzing the results. The EyeSide Agent executes the algorithm of this step, which is very simple and consists of locating the macula, a taint that is always located in the image side closer to the external side of the face. Images are always taken with the fundus camera opposite the face so if the macula is in the left side of the image, it indicates that the image is associated with the right eye. If the macula is in the right side of the image, it means that an image of the left eye is being analyzed. As mentioned, the algorithm that the EyeSide Agent executes to detect the macula is very simple, so the analysis time is really low. The location of the optic disc is already known and the macula must be in one of the sides and approximately at the same height. For this reason, the grayscale image is divided into two new ROIs. The width of each ROI is delimited from the x-axis center of the optic disc and the retina border and the height of both ROIs is 1:4 of the retina height centered in the y-axis center of the optic disc. An example with both ROIs ( ROI 1 and ROI 2 ) delimited is shown in Fig. 7i). Once delimited, the algorithm compares the mean color of the two ROIs. The lowest one (0 = black; 255 = white) is the darker which means that the macula is in that ROI.

OPEN IN VIEWER

Fig. 7.

Eye identification step.

OPEN IN VIEWER

Fig. 8.

Segmentation step.

3.1.6. Segmenting vessels

At this point of the methodology, the system knows all the information to segment the vasculature in the “Segmenting vessels” state, whose output will be the vascular structure separate from the background. This process requires the participation of more than one agent. More specifically, the Blur Agent executes different Gaussian filters so that blood vessels are mixed with the background. This will result in an image like the one shown in Fig. 8 a. The agent then detects the points of the new image with highest color difference from the same point in the grayscale image. It generates a binary image with those points (Fig. 8 b), which is analyzed by the Morphology Agent to evaluate the morphology of every blob. This removes noise and results in a well-defined vessel structure as shown in Fig. 8 c. The Skeleton Agent can now extract the skeleton. Skeletonization is a widely known tool to describe binary images that captures the essential topology shape information of the object in a simple form. Every point of the skeleton is now analyzed by the agent to separate every single vessel. There are four kinds of points:

final, with just one neighbor,

normal, with two neighbors

branch, with three neighbors

intersection, with four neighbors.

Every detected vessel is fragmented in a series of segments which are analyzed individually. Every segment is a set of every point between pairs of point final-final, branch-final, branch-branch, branch-cross, cross-cross and cross-final. On the other hand, full vessels (both veins and arteries) are points between a pair final-final with one or more chained segments. The followed process to get all the segments is described by the Algorithm 1.

OPEN IN VIEWER

Algorithm 1:

Extraction of skeleton vessels and segmentation

4.1. Agents coordination model

This study presents a coordination model that employs a cooperative MAS in which each agent is capable of determining results dynamically in order to achieve its objectives. Moreover, there is a global mechanism that can optimally assign a final result to the agents so that they can work in a coordinated effort. The global mechanism considers the global objective of the group of agents. VesselType agents present a deliberative CBR-BDI (Case Based Reasoning – Belief Desire Intentions) [13] agent architecture to construct the negotiation model.

A BDI agent emphasizes the reasoning process directed at an action, i.e., determining what the agent should do [6,12,51]. This process suppose two fundamental phases:

Both processes should be carried out by taking into account the limited resources of each agent. In other words, the reasoning processes should consider the restrictions imposed within the system for selecting the actions in an effective manner.

BDI architectures present interesting features that can deal with situations involving high level coordination among the agents available at a given time, and suggest the necessary extensions for this model, thus allowing the agents to engage in the correct social behavior [41].

The model proposed in this research incorporates a kind of agent capable of classifying from the perspective of an agent organization. This particular agent is a deliberative CBR-BDI agent [13]. The principal feature of this agent is the ability to integrate a CBR reasoning mechanism [1]. A CBR system launch a reasoning cycle that includes four sequential steps: retrieve, reuse, revise and retain. This cycle is incorporated within the activities of the BDI agent [13]. and identifies the phases as tasks or roles that the agent should be able to perform. This composes for one of the primary deficiencies in the BDI model, which involves the manner in which memory and past experiences are handled. Authors in [4,26] used specifically Beliefs, Desires and Intentions, to be able to obtain a representation of the case and initiate the CBR reasoning cycle. To integrate the CBR reasoning model within the structure of a deliberative BDI agent, [12,22,23] and [26] present a formula that can relate the concept of case to the fundamental concepts of BDI. The coordination of tasks that are combined to solve a common problem requires either a centralized or distributed type of classifying that can be carried out by the system agents themselves. This constitutes an open problem for which traditional classifying systems are inadequate. It is necessary to have a classifying feature that can adapt to these circumstances.

The agent-based architecture presented in this study includes a group of agents whose aim is to offer a classification vessel to the users. This group of agents is implemented by using the CBR-BDI model. The agent roles, represented in Fig. 9, necessary to carry out the methodology presented with all the proposed phases are:

OPEN IN VIEWER

Fig. 9.

Coordination problem between VesselType Agents and VesselManager Agent.

VesselType agents can produce a classification for the vessel identified in its area of performance independent of other agents, but depending on the resources and profile for each of them. The agent that assumes the role of VesselManager agent will be responsible for assigning final classification vessels according to the best individual classification. This classification is chosen taking into account the probability of the cases of the base-cases stored for this vessel in the past. In other words, there may be certain individual restrictions for each agent that must be taken into consideration by the VesselType agents in order to generate a classification vessel; however, it is the VesselManager Agent who takes into account the restrictions for the entire group of agents in order to distribute and assign the final classifications. The VesselManager agent is a coordinator that has access to the information on areas, previous classifications, tonalities, among others, stored in the system.

The CBR-BDI agent, as suggested in [12], is assigned a cycle in which it will obtain its personalized optimal classification. The information stored is a data base of cases that considers the classification performed and the resources generated by each agent. This way the VesselManager agent ensures that the vessel classification process can at least be carried out.

To supply the required characteristics, there is a decomposition of services (in this case of classification services) which agents can access to carry out their individual classification. The platform used as the basis for the creation of the MAS [54] allows to carry out this decomposition. The services are outsourced so that the agents can make use of them, depending on the role that they have assigned. Thus, an agent of type VesselType X may make use of a specific classification service, and different from another agent VesselType Y.

After that, it is necessary to address the structure of agreements which supports this decomposition, in order to make it adaptive. So, an important notion is the agreement between computational entities (agents) conceived as an architectural construct.

The interaction between the VesselType agent and the VesselManager agent is made through a negotiation protocol inspired by the standard WS-Agreement used in web services [28,37,48]. In this case, the following information is going to be considered: Inputs, Outputs, Preconditions, and Results, and a non-functional parameter: classification process execution time. When a classifying vessel process is carried out in the system, a Morphology Agent sends a request to the VesselManager Agent, the query is an OWL (Web Ontology Language) document, which contains a set of service inputs, outputs and a deadline before which the required service should be provided: query = ⟨ input , output , deadline ⟩

The VesselManager agent, in order to determine a set of appropriate classification, analyzes the existing classifications of VesselType agents considering the Morphology Agent deadline. Each individual classification of a VesselType agent consists of a set of elements: service = ⟨ serviceId , input , output , service − Duration , probability ⟩

The VesselManager Agent selects the service, taking into account the success probability of the service and its execution time. In this point is where the internal vessel classification is carried out.

Arguments exchanged among VesselType agents are tuples of the form Arg = α , β , ⟨ S ⟩ , where α is the conclusion of the argument, β is the value that the agent wants to promote, and ⟨ S ⟩ is a set of elements that justify the argument (the support set).

The support set S can consist of different elements, depending on the argument purposed. For example, if the argument justifies a potential solution for a problem, the support set is:

This type of argument is called a support argument [2,27]. If the argument attacks the argument of an opponent, the support set can also include any of the allowed attack elements of our framework. These are distinguishing premises, counter-examples, or critical questions.

The argumentation system that implements our architecture has a specific structure of domain-cases and a concrete set of argumentation-schemes, depending on the application domain where the argument has been put forward (those premises that match the problem to solve and other extra premises that do not appear in the description of this problem but that have been also considered to draw the conclusion of the argument) [28]. An example of a case used by a VesselType agent could be the following:

Argument-cases retain the data about previous arguments that an agent had proposed in a dialogue with other agents. In other words, argument cases store a problem description that has a domain context and is formed by the premises that characterize the argument. A case example for VesselType agent is shown in Table 1. In this case, it is necessary to store all information related to the classification of the vessel, that is, background color, tonality, area of performance of the VesselType agent, Type of classification result (in this case A- Artery or B- Vein), and the probability of success in the classification process.

For example, for a hypothetical simple case in which there are several classifications for the same vessel:

The VesselManager agent has to select which is the final classification. In this case a centralized decision is made. VesselManager agent is prepared to collect all the classifications made by VesselTypes agents and decide the final classification (A or B). It is based on the probabilities given by each of the VesselTypes and their classification. In this case, the strategy is simple and based on the merger of the classifiers. The merger of the classifiers assumes that all the classifiers are competitive and complementary (equally expert). For this reason, each of them issues a decision regarding each vessel classification pattern. The output of the classifiers is an H-dimensional vector containing the decisions of the H classifiers (VesselTypes agents). This vector also contains the fields of the case (Type, Tonality, Background, Area, Probability). From the vector set, VesselManager agent must make the final decision by applying one of the following principles: simple majority voting or weighted majority voting. In this case, a vote is carried out by a weighted majority, where a different weight is available for each classifier taking into account the probability of success that is stored in its case. The final decision could be made by majority, average [30], minority, median [11], or use more complex methods [29].

In this case, it will be B type, although there is an VesselType agent with a 95% probability that suppose that it is an artery. A single agent could fail although with its values of tonality, etc. consider a high percentage that the vessel is of a specific type. That is why it is important to negotiate in this step of the system. Hence the importance of the negotiation process and the collaboration between agents in the step of classification.

The success parameter is used in the argument management process of our argumentation process to represent the potentially high persuasive power of current arguments that are similar to previous arguments that were remained acceptable at the end of the agreement process.