RehabRobo-Query : Answering natural language queries about rehabilitation robotics ontology on the cloud

Abstract

We introduce a novel method to answer natural language queries about rehabilitation robotics, over the formal ontology RehabRobo-Onto. For that, (i) we design and develop a novel controlled natural language for rehabilitation robotics, called RehabRobo-CNL; (ii) we introduce translations of queries in RehabRobo-CNL into sparql queries, utilizing a novel concept of query description trees; (iii) we use an automated reasoner to find answers to sparql queries. To facilitate the use of our method by experts, we develop an intelligent, interactive query answering system, called RehabRobo-Query, using Semantic Web technologies, and make it available on the cloud via Amazon web services. RehabRobo-Query guides the users to express their queries in natural language and displays the answers to queries in a readable format, possibly with links to detailed information. Easy access to information on RehabRobo-Onto through complex queries in natural language may help engineers inspire new rehabilitation robot designs, while also guiding practitioners to make more informed decisions on technology based rehabilitation.

Keywords

Ontology systems query answering rehabilitation robotics intelligent user-interfaces controlled natural languages

1. Introduction

Neurological injuries, such as stroke, are the leading cause of permanent disability in developed countries [24]. In particular, among 17 million people that suffer from stroke, about one third are left permanently disabled each year. These disabilities place a high burden on individual welfare of patients, while negatively impacting national economies [16]. Despite the recent medical developments, the number of stroke incidents continues to increase due to the ageing population in many developed countries.

Physical rehabilitation therapy is indispensable for treating neurological disabilities. Therapies have been shown to be more effective when they are task specific [3], repetitive [5], intense [30], long term [44], and allow for active involvement of patients [35]. However, repetitive and high intensity therapies place high physical burden on the therapist, significantly increasing the cost of such treatments.

With recent advancements in electro-mechanical systems, robot-assisted rehabilitation devices have become ubiquitous, since these devices can bear the physical burden of rehabilitation exercises, while therapists are employed as decision makers. In particular, the use of robotic devices in repetitive and physically involved rehabilitation eliminates the physical burden of movement therapy for the therapists, enables safe and versatile training with increased intensity, and increases the reliability, accuracy, and effectiveness of traditional physical rehabilitation therapies. Robot-assisted rehabilitation devices can be used to support patients with all levels of impairment, can quantitatively measure patient progress, allow for easy tuning of duration and intensity of therapies, and enable realization of customized, interactive, innovative treatment protocols. Clinical trials with robot-assisted rehabilitation indicate that this form of therapy is effective for motor recovery and possesses high potential for improving functional independence of patients [29,37,40,41,47].

As more and more rehabilitation robots are deployed, the information about them also increases. However, this information is not represented as knowledge in structured forms, and usually appears as text in relevant publications. Consequently, accessing the requested knowledge and thus automatically reasoning about it has become a challenge. For instance, accessing the flexion/extension range of motion (RoM) of AssistOn-SE [49], and determining the rehabilitation robots that target shoulder movements and also have at least $210^{\circ}$ RoM for the flexion/extension movements of the shoulder are challenging tasks that require one to go through and study unstructured text from several different resources.

Furthermore, given the interdisciplinary nature of rehabilitation robotics, in many cases, the requested knowledge requires integration of further knowledge from related disciplines, such as physical medicine. For instance, determination of rehabilitation robots that can treat patients with rotator cuff lesions, requires one to know that rotator cuffs are muscle units employed to move the shoulder, and that, for patients with rotator cuff lesions, abduction and flexion movements of the shoulder should not have more than $90^{\circ}$ RoM. Relevant rehabilitation robots can only be found after one acquires this crucial information.

Additionally, given the growing number of research groups contributing to the field, different approaches display large variability and the field lacks a standardized terminology. Several efforts have been initiated for standardizing terminology, as well as assessment measures, for rehabilitation robots, e.g., by the European Network on Robotics for Neurorehabilitation.1

¹
http://www.cost.eu/COST_Actions/bmbs/TD1006

The development of such a standardization is likely a critical step for the field, helping robotic rehabilitation technology become widely understood and accepted as a useful tool.

Motivated by these challenges and efforts, we have earlier designed and developed the first formal rehabilitation robotics ontology, called RehabRobo-Onto, in OWL (Web Ontology Language) [1,25], and made it available on the cloud via Amazon Web Services [11–13] (in particular, Amazon EC2 – Elastic Compute Cloud)2

http://aws.amazon.com/ec2/

so that every rehabilitation robotics researcher can easily use it in order to publish/represent information about his/her robot, and modify this information. To facilitate such modifications of the rehabilitation robotics ontology RehabRobo-Onto, we have also developed a Web-based software (called RehabRobo-Query)3

http://hmi.sabanciuniv.edu/?page_id=781

with an intelligent user-interface. In this way, experts do not need to know the underlying logic-based representation languages of ontologies, like OWL, or Semantic Web technologies, for information entry and modification. RehabRobo-Query utilizes Amazon Web Services (Amazon EC2) for cloud computing. For further information about the design, development and maintenance of RehabRobo-Onto, and how RehabRobo-Query facilitates modification of RehabRobo-Onto, we refer the reader to our earlier article [11].

This article addresses question answering in the domain of rehabilitation robotics over RehabRobo-Onto. Note that a structured representation of information about rehabilitation robotics such as the ontology RehabRobo-Onto allows rehabilitation robotics researchers to learn various properties of the existing robots and access the related publications to further improve the state-of-the-art. Physical medicine experts also can find information about rehabilitation robots and related publications to better identify the appropriate robot for a particular therapy or patient population. Such requested information can be obtained from RehabRobo-Onto by expressing the requested information as a formal query in a query language, such as sparql [42], and then by computing answers to these queries by using a state-of-the-art automated reasoner, such as Pellet [43]. However, expressing the requested information as a formal query by means of formulas is challenging for many users, including robot designers and physical medicine experts. For that reason, we particularly focus on the process of query answering over RehabRobo-Onto, and making it easier for the users to express their queries in a natural language and to obtain answers to their queries automatically.

Towards these goals, the main contributions of this article can be summarized as follows. For expressing queries about rehabilitation robotics, we have designed and developed a novel controlled natural language for rehabilitation robots, called RehabRobo-CNL. For automatically computing answers to natural language queries, we have introduced an algorithm that transforms natural language queries in RehabRobo-CNL into formal queries in sparql. This transformation utilizes an intermediate representation: a novel tree structure, called a Query Description Tree (QDT). Once the natural language query is transformed into a formal query, Pellet is used to find relevant answers to the query. The software system RehabRobo-Query has been extended with an interactive, intelligent user interface to guide the users to enter natural language queries in RehabRobo-CNL about the existing robots, and to present the answers in an understandable form, so that the users do not have to know about the logical formalism of the ontology or the formalism to represent queries, or the use of the technologies for computing answers to their questions about rehabilitation robots.

The rest of the article is organized as follows. First, we present a brief review of RehabRobo-Onto (Section 2). We present the controlled natural language RehabRobo-CNL for expressing queries about rehabilitation robotics (Section 3), and discuss the extension of RehabRobo-Query to support querying in RehabRobo-CNL via an interactive, intelligent user-interface (Section 4). We present our transformation of a RehabRobo-CNL query into a sparql query (Section 5) whose answer can be computed using Pellet (Section 6). We evaluate the underlying methods empirically over a variety of queries, and the usefulness of RehabRobo-Query by a survey analysis with participants of different backgrounds (Section 7). After we summarize the related work about software systems that support natural language queries over ontologies (Section 8), we conclude with a summary of our contributions (Section 9).

This article significantly extends our earlier paper [14], presented at the International Conference on Knowledge Engineering and Ontology Development (KEOD 2014), (i) by providing more details about the query answering system integrated in RehabRobo-Query, the types of queries supported by RehabRobo-Query, and the complete description of the query language RehabRobo-CNL, (ii) by introducing a new algorithm for transformation of a RehabRobo-CNL query into a sparql query, and (iii) by providing evaluations of the underlying methods and the query system RehabRobo-Query.

2. A brief review of RehabRobo-Onto

RehabRobo-Onto is the first formal rehabilitation robotics ontology that represents knowledge about rehabilitation robotics in a structured form, and allows query answering about this knowledge. It has been developed in line with the efforts of the European Network on Robotics for Neurorehabilitation,4

⁴
http://www.rehabilitationrobotics.eu/

for standardizing terminology as well as assessment measures for rehabilitation robots.

Fig. 1.

RehabRobo-Onto with main classes [11, Fig. 1].

RehabRobo-Onto has been designed by considering suggestions of various rehabilitation robotics researchers and physical medicine experts, by first identifying the purpose, and then the basic concepts, their thematic classes and their relationships. The main classes and their relationships are illustrated in Fig. 1.

RehabRobo-Onto has five main concepts (or thematic classes):

RehabRobots (representing rehabilitation robots and their properties),

JointMovements (representing targeted joint movements and their properties),

Owners (representing robot designers who add and modify information in the ontology about their own robots),

References (representing publications related to rehabilitation robots),

Assessments (representing assessment measures for rehabilitation robots).

The main classes RehabRobots, JointMovements and Assessments have further subclasses. For instance, JointMovements has two main subclasses, for lower extremity joint movements and upper extremity joint movements; Fig. 3 of [11] illustrates a hierarchy of the lower extremity joint movements, while Fig. 2 shows a hierarchy of the upper extremity joint movements. A hierarchy of various types of assessment measures (e.g., movement quality assessments, effort assessments, psychomotoric assessments, muscle strength assessments, kinematic assessments) is shown in Fig. 4 of [11]. Currently there are 147 classes represented in RehabRobo-Onto.

Fig. 2.

A hierarchy of upper extremity joint movements targeted by rehabilitation robots.

The main concepts are related to each other by the following relations:

a rehabilitation robot targets joint movements,

a rehabilitation robot is ownedBy a robot designer,

a rehabilitation robot hasReferences to some publications,

a rehabilitation robot hasAssessment with respect to some evaluation measure.

RehabRobo-Onto is an OWL 2DL ontology that has been developed using the ontology editor Protégé [23]. It is open-source and available on the cloud via Amazon Web Services. Its maintenance (i.e., adding, deleting, modifying information about rehabilitation robots) can be done as part of the ontology system RehabRobo-Query.

RehabRobo-Onto has been designed to be as simple as possible for the purpose of easier maintainability and the possibility of extensions with further domain-specific properties when needed, yet as functional as needed by the experts. For instance, since the main requested information is essentially about the robot, the basic bibliography information and the domain-specific information about clinical studies are found sufficient by the experts. Therefore, References has been described as one simple class containing this information only, instead of utilizing BIBO [9] or FaBio [39] that can represent further details about publications in many classes. Similarly, for Owners, it is sufficient to introduce one class with few data properties instead of utilizing FOAF.5

⁵

http://www.foaf-project.org/

For further information about the design and development of RehabRobo-Onto, and how RehabRobo-Query facilitates maintenance of RehabRobo-Onto on the cloud, we refer the reader to our earlier article [11].

3. RehabRobo-CNL: A controlled natural language for queries about rehabilitation robotics

Queries about RehabRobo-Onto can be posed by the user in a natural language. However, natural languages may have ambiguities in their vocabularies and grammars. To overcome ambiguities of a natural language, a subset of it with a restricted vocabulary and grammar can be considered for specific domains; such languages are called Controlled Natural Languages (CNLs) [28]. Essentially, with its restricted vocabulary and grammar, a CNL is a formal language. Therefore, it is easier to convert a CNL (compared to a more general natural language) to a logic-based formalism. In that sense, a CNL facilitates the use of automated reasoners to find answers to queries expressed in a CNL. Due to these reasons, first we have identified the types of queries about rehabilitation robots, then we have designed and developed a new CNL, called RehabRobo-CNL, to express these queries in natural language.

3.1. Types of queries supported by RehabRobo-CNL

The variety of queries is important since the users may include not only rehabilitation robot designers and developers, but also physical therapists and medical doctors. Also, complex queries that not only require intelligent extraction of different types of knowledge, but their integration via conjunctions, disjunctions, negations, nested relative clauses, aggregates, and quantifications are inevitable in rehabilitation robotics, since robots, targeted movements, evaluation metrics, etc. have many features that are of interest to users. We have designed RehabRobo-CNL considering these aspects.

We have identified different types of complex queries useful from the perspectives of rehabilitation robotics and physical medicine, by gathering sample queries from a large number of experts from different disciplines (i.e., roboticists, engineers, physical therapists, physical medicine doctors, neuroscientists) whom we have been closely collaborating with and whom we have met at various meetings and workshops. We have further populated such queries (e.g., Q6 and Q19 of Table 8) considering their grammatical structures, for the purpose of testing and evaluating the CNL. Some examples of these queries are presented in the following, with respect to their topics.

Queries about mechanical properties of rehabilitation robots These are queries to extract information about the rehabilitation robots whose information is available in RehabRobo-Onto. Here are some examples:

What are the robots that target shoulder movements and that have at least $210^{\circ}$ RoM for the flexion/extension movements of the shoulder?

What are the robots that target wrist movements and that have at least 2 active degrees of freedom?

What are the shoulder robots that target flexion/extension movements and that do not target elevation/depression movements?

What are the robots with active degree of freedom ⩾ ‘3’ and that target all pelvic girdle movements with backdrivability type ‘passive’?

What are the ankle robots that target movements with electrical actuation and with cable drive transmission?

According to RehabRobo-Onto, for instance, some part of the answer to Q1 is as follows:

AssistOn-SE

Query Q1 is useful for an engineer to determine rehabilitation robots that can cover the whole range of motion of shoulder flexion/extension movements. Existing designs may not only guide new solutions for the same joint, but similar solutions may be applied to different joints of the body. For instance, AssistOn-SE [49] and AssistOn-Gait [36] rehabilitation robots rely on a similar mechanism design to target scapula-shoulder and pelvis-hip joint complexes. On the other hand, Q1 can also be used by a therapist to identify robots that can be used to target dysfunction of the scapulohumeral rhythm, as the ability to ergonomically cover large flexion/extension range of motions is critical for such treatment.

Similarly, an engineer may benefit from Q3 to determine robot designs that can align robot and human shoulder rotation axes and use these examples to design other robots that have similar alignment properties. Q3 may be used by a therapist to identify shoulder robots that cannot be used to treat shoulder pain due to shoulder subluxation, as elevation/depression movements are critical for treatment of this medical condition.

Queries about joint movements targeted by rehabilitation robots These are queries to extract information about joint movements targeted by rehabilitation robots in RehabRobo-Onto. Here are some examples:

What are the movements that are targeted by some robots with (some intervention time or with all targeted disorders)?

What are the movements that are targeted by the robot ‘AssistOn-Finger’ and with minimum range of motion ⩾ ‘20’?

What are the movements that are not targeted by any robots with kinematic type = ‘redundant’ and with mechanism type ≠ ‘parallel’?

According to RehabRobo-Onto, for instance, some part of the answer to Q7 is as follows:

Index Finger DIPFlexion/Extension

Ring Finger PIPFlexion/Extension

Pinky Finger DIPFlexion/Extension

Thumb Finger PIPFlexion/Extension

Middle Finger PIPFlexion/Extension

Query Q7 may be used by a therapist while evaluating whether AssistOn-Finger [18], which is a robot designed for tendon injuries, is also appropriate for finger range of motion exercises for stroke patients.

Query Q8 is useful for an engineer to determine joint movements that are targeted with redundant but not parallel mechanisms. Such a query may help determine similar characteristics of several joint movements and guide similar mechanism designs to be adapted for these joints.

Queries about the evaluation metrics for rehabilitation robots These are queries to extract information about metrics used to evaluate rehabilitation robots. Here are some examples:

What are the effort metrics that are evaluated by some robots with active degree of freedom ⩾ 2?

What are the movement quality metrics that are evaluated by all robots with motion capability = ‘grounded’?

What are the kinematic aspect metrics that are evaluated by some robots that target all elbow movements?

What are the muscle strength metrics that are evaluated by robots that target all wrist movements with transmission = ‘direct drive’?

What are the psychomotoric aspect metrics that are evaluated by all robots with kinematic type = ’redundant’ and that target all ankle movements with minimum range of motion ⩾ ‘30’?

According to RehabRobo-Onto, for instance, some part of the answer to Q9 is as follows:

Time To Initiate Movement

Amount Of Compensation

Biomechanical Work Energy Power

An engineer may use Q9 to determine the types of sensors to be integrated in a new multi degree of freedom rehabilitation robot, while a clinical researcher may utilize the same query to identify robots that can provide relevant effort metrics to study abnormal synergy patterns after stroke.

Similarly, Q11 is useful for an engineer who designs an elbow robot to determine the relevant kinematic aspect metrics that can be incorporated into the control/evaluation algorithms of a new design. The same query can be used by therapists to identify which kinematic aspect metrics can be measured by current robotic rehabilitation devices.

Other queries These are queries to extract information about publications and owners/institutes/laboratories of the robots. Here are some examples:

What are the publications with clinical study and that do not reference any robots with active degree of freedom $⩾ 1$ ?

What are the publications without clinical study or that reference some robots that do not evaluate any movement quality metrics?

What are the publications with place of publication ‘ICORR’ and that reference some robots that are owned/maintained by some users with institution ‘Sabanci University’?

What are the users that own/maintain some robots that target all ankle movements?

Queries Q16 and Q17 may be used by both engineers and clinical researchers to find publications of a research group or research groups that focus on certain joint movements.

Table 1
The grammar of RehabRobo-CNL

Query → WhatQuery QuestionMark

WhatQuery → What are the $Type ()$ GeneralRelation

GeneralRelation → SimpleRelation NestedRelation ^∗

SimpleRelation → (that RelativeClause)+

SimpleRelation → that InstanceRelation

SimpleRelation → WithRelation

NestedRelation → (and LP SimpleDisjunction RP)^∗

NestedRelation → (or LP SimpleConjunction RP)^∗

SimpleDisjunction → (SimpleRelation or)^∗SimpleRelation

SimpleConjunction → (SimpleRelation and)^∗SimpleRelation

RelativeClause → $Verb ()$ (some | all | the) $Type ()$

RelativeClause → Neg $Verb ()$ any $Type ()$

InstanceRelation → Neg? $Verb ()$ the $Type () Instance ()$

WithRelation → with $Noun ()$ EqCheck $Value () +$

WithRelation → with Quantifier $Noun ()$

WithRelation → (with | without) $Noun ()$

EqCheck → $= |! = | ⩽ | ⩾$

Quantifier → some | all | none

Neg → $Neg ()$

LP → (

RP → )

QuestionMark → ?

Query →	WhatQuery QuestionMark
WhatQuery →	What are the $Type ()$ GeneralRelation
GeneralRelation →	SimpleRelation NestedRelation ^∗
SimpleRelation →	(that RelativeClause)+
SimpleRelation →	that InstanceRelation
SimpleRelation →	WithRelation
NestedRelation →	(and LP SimpleDisjunction RP)^∗
NestedRelation →	(or LP SimpleConjunction RP)^∗
SimpleDisjunction →	(SimpleRelation or)^∗SimpleRelation
SimpleConjunction →	(SimpleRelation and)^∗SimpleRelation
RelativeClause →	$Verb ()$ (some \| all \| the) $Type ()$
RelativeClause →	Neg $Verb ()$ any $Type ()$
InstanceRelation →	Neg? $Verb ()$ the $Type () Instance ()$
WithRelation →	with $Noun ()$ EqCheck $Value () +$
WithRelation →	with Quantifier $Noun ()$
WithRelation →	(with \| without) $Noun ()$
EqCheck →	$= \|! = \| ⩽ \| ⩾$
Quantifier →	some \| all \| none
Neg →	$Neg ()$
LP →	(
RP →	)
QuestionMark →	?

Table 2

The ontology functions

$Type ()$	Returns the types that correspond to concept names. They are: Robots, movements, users, publications and metrics.
$Instance ()$	Returns robot names for robots and user names for users.
$Verb ()$	Returns the verbs that correspond to object properties between concepts. Returns both active and passive forms of these verbs. Active forms of these verbs are: Target, evaluate, reference, own.
$Noun ()$	Returns the nouns that correspond to data properties. ex. targeted disorder, active degree of freedom.
$Value ()$	Returns the suitable values according to a given noun. Corresponds to the pre-defined ranges of data properties.
$Neg ()$	Returns a suitable negation phrase. These phases are: do not, are not.

3.2. Grammar of RehabRobo-CNL

Once we have identified different types of complex queries about rehabilitation robotics, we have designed the controlled natural language RehabRobo-CNL for expressing them in an unambiguous way and utilizing domain-specific vocabulary. The grammar of RehabRobo-CNL, with relevant vocabulary, is presented in Tables 1–7.

Table 1 gives an overall summary of RehabRobo-CNL, providing information about what types of queries are allowed in RehabRobo-CNL. To eliminate the ambiguities in nesting of conjunctions and disjunctions, RehabRobo-CNL provides two ways of constructing a query: A query in RehabRobo-CNL should either be in Conjunctive Normal Form (CNF), or in Disjunctive Normal Form (DNF). In other words, RehabRobo-CNL supports conjunctions of simple disjunctions, and disjunctions of simple conjunctions. No further nesting of conjunctions (resp., disjunctions) in a simple disjunction (resp., conjunction) is allowed. A query can contain any number of conjunctions and disjunctions provided that they confirm with the rules above. An example of a query in CNF is as follows.

What are the robots with mechanism type = ‘hybrid’ and (with motion capability = ’grounded’ or with functionality = ‘clinic’)?

The following query is in DNF:

What are the robots with no targeted disorder or (with active degree of freedom > 1 and with control modes = ‘active’)?

Having an overall understanding of the types of queries supported by RehabRobo-CNL, let us provide some more details to relate it to rehabilitation robotics. The functions presented in italic font in Table 1 refine these queries by embedding relevant information from RehabRobo-Onto. These ontology functions are described in Table 2.

The information represented with the ontology functions are coupled by their relevance. For instance, only the verb “reference” can appear after the type Publications. By such a matching of types with verbs, it is possible to prevent semantically wrong queries like “What are the publications that target some shoulder movements?”. All of the matches between types and verbs in expressions of the form “ $Type ()$ that $Verb ()$ ” are shown in Table 3. Note that these matchings essentially come from the structure of RehabRobo-Onto, e.g., concept names and relation names.

Similarly, it is necessary to match verbs with types in expressions of the form “ $Verb ()$ (some | all | any^∗ | the) $Type ()$ ”. Table 4 lists the available types that can occur after a verb in the query (e.g., in a RelativeClause), relative to the ontology. If a quantifier such as “some” is used in a relative clause, then the types which have some subclasses are considered in queries. Here is an example query:

What are the robots that evaluate some wrist movements?

Since wrist movements class have subclasses (e.g., wrist flexion/extension, wrist radial deviation/ulnar deviation) in RehabRobo-Onto, this query will retrieve all robots that target at least one of these subclasses. If “the” keyword is used after the verb in a query, then the leaf classes are considered to select one specific type. Here is an example query:

What are the robots that target the wrist radial deviation/ulnar deviation?

Wrist radial deviation/ulnar deviation is a leaf class. It is also a subclass of wrist movements. This query will retrieve the robots that target this specific wrist movement. If there is a robot that targets some other wrist movements but wrist radial deviation/ulnar deviation, then this robot will not be included in the answer to this query.

Table 3
Verbs that can occur after the nouns

$Type ()$ that $Verb ()$

robots → target

robots → are owned/maintained

robots → evaluate

movements → are targeted by

users → own/maintain

publications → reference

effort metrics → are evaluated by

kinematic aspect metrics → are evaluated by

movement quality metrics → are evaluated by

muscle strength metrics → are evaluated by

psychomotoric aspect metrics → are evaluated by

$Type ()$	that	$Verb ()$
robots	→	target
robots	→	are owned/maintained
robots	→	evaluate
movements	→	are targeted by
users	→	own/maintain
publications	→	reference
effort metrics	→	are evaluated by
kinematic aspect metrics	→	are evaluated by
movement quality metrics	→	are evaluated by
muscle strength metrics	→	are evaluated by
psychomotoric aspect metrics	→	are evaluated by

Table 4

Types that can occur after the verbs

$Verb ()$	(some \| all \| any ∗ \| the)	$Type ()$
target	(some \| all \| any)	all movements except leaf classes
target	the	leaf classes of movements
evaluate	(some \| all \| any)	all metrics except leaf classes
evaluate	the	leaf classes of metrics
are targeted by	(some \| all \| any)	robots
are evaluated by	(some \| all \| any)	robots
reference	(some \| all \| any)	robots
own	(some \| all \| any)	robots
are owned by	(some \| all \| any)	users

^∗

any is used after negative verbs.

In RehabRobo-CNL, the instances of the concepts are represented by one of their distinctive properties. For robots, this distinctive property is its name; for users, it is the user name. To illustrate, when the users want to query about AssistOn-SE, they specify the instance using the name of the robot. For movements and metrics, there is no such distinctive property because the concept names are sufficient to specify a movement or metric. In fact, using RelativeClause is sufficient to query about them. To query about robots or users, however, InstanceRelation is used to extract the instances. Table 5 demonstrates the relevant properties of the instances that appear when a type is selected.

Table 5

Instances that can occur after the types

$Type ()$		$Instance ()$
robot	→	name of the robot
user	→	username of the user

Table 6

Nouns that can occur after the types

$Type ()$	with	$Noun ()$
robots	→	active degree of freedom
robots	→	control modes
robots	→	disorder level
robots	→	functionality
robots	→	interaction type
robots	→	intervention time
robots	→	kinematic type
robots	→	motion capability
robots	→	name
robots	→	passive degree of freedom
robots	→	targeted disorder
robots	→	targeted population
movements	→	actuation
movements	→	backdrivability
movements	→	backdrivability type
movements	→	maximum range of motion
movements	→	minimum range of motion
movements	→	range of motion type
movements	→	transmission
publications	→	authors
publications	→	clinical study
publications	→	place of publication
publications	→	title
publications	→	url
publications	→	year
users	→	institution
users	→	mail
users	→	username

Table 7

Values that can occur after the nouns

$Noun ()$	∗	$Value ()$
active DoF ∗∗	→	any integer value entered by the user
actuation	→	electrical, electro-rheological, hydraulic, pneumatic, series elastic, variable impedance, other
authors	→	one of the authors that are added to the ontology up to now
backdrivability	→	backdrivable, non-backdrivable
backdrivability type	→	active, passive
control modes	→	ADL, BCI, EMG, active, assistive, bilateral, multilateral, passive, resistive
disorder level	→	mild, moderate, severe
functionality	→	clinic, home
institution	→	one of the institutions that are added to the ontology up to now
interaction type	→	exoskeleton, mixed, suspension, end effector
intervention time	→	acute, chronic, subacute
kinematic type	→	hybrid, parallel, serial
motion capability	→	grounded, mobile
name	→	one of the robot names that are added to the ontology up to now
passive DoF	→	any integer value entered by the user
place of publication	→	one of the places of publication that are added to the ontology up to now
maximum RoM ∗∗∗	→	any float value entered by the user
minimum RoM	→	any float value entered by the user
RoM type	→	active, passive
targeted disorder	→	stroke, spine cord injury
targeted population	→	adult, pediatric
title	→	one of the publication titles that are added to the ontology up to now
transmission	→	belt drive, cable drive, capstan drive, direct drive, gear train, harmonic drive, other
url	→	one of the urls that are added to the ontology up to now
username	→	one of the usernames that are added to the ontology up to now
year	→	any year (integer value) entered by the user

^∗

(EqCheck|Quantifier)

^∗∗

degree of freedom

^∗∗∗

range of motion.

In addition to types and verbs, types are matched with the relevant nouns, as shown in Table 6. For instance, control modes are matched with robots whereas actuation is matched with movements. Note that these matchings are due to properties of concepts in RehabRobo-Onto.

Further, the values for the nouns are extracted from RehabRobo-Onto to allow suitable entries from the users. These values are listed in Table 7. The values can be considered as ranges of the nouns, that the user can choose from.

4. User-interface of RehabRobo-Query for query answering: Intelligent and interactive

The user interface of RehabRobo-Query can guide the users to add and modify information in RehabRobo-Onto. We have extended it further so that it can guide the users to ask questions in RehabRobo-CNL, and present answers to these queries in an understandable form.

Fig. 3.

Construction of a query with the guidance of the intelligent interactive user interface of RehabRobo-Query.

While the users construct questions, RehabRobo-Query’s user interface can prevent nonsensical queries. For instance, shoulder elevation/depression is not a wrist movement. The user interface takes into account such information, and guides the users intelligently to construct their queries so that queries like

What are the wrist robots that target shoulder elevation/depression?

are not possible. In that sense, the user interface is intelligent.

RehabRobo-Query’s user interface is also interactive: it shows the possible choices (for instance, the sort of movements targeted by a shoulder robot) and allows auto-completion (for instance, to obtain the full name of a robotics device). To be able to identify the choices (e.g., parameters and their values) relevant to the query, RehabRobo-Query’s user interface utilizes automated reasoning methods online. Indeed, as the user enters a query, in the background the user interface considers the part of the query constructed so far and asks RehabRobo-Query’s ontological reasoning module to compute all possible values of parameters. Once the reasoning module returns these choices, the user interface presents them, e.g., within a pull-down menu. Figure 3 shows the construction of the following query with the guidance of the user interface:

What are the robots that target some wrist movements with actuation = ‘series elastic’?

Furthermore, RehabRobo-Query provides auto-completion to help users enter values for nouns that correspond to data properties of type string. If the user should choose a concept among a hierarchy, RehabRobo-Query displays an accordion view and enables the user to click on the desired option. In addition, RehabRobo-Query allows multiple selection of values for relational properties. For functional properties, the user is able to select multiple items for inequality. The user can choose a number of options among “less than or equal”, “more than or equal”, “equal” and “not equal” while entering values for a data property of type integer or float.

Fig. 4.

An answer for the query constructed in Fig. 3, presented to the user.

Once the query is constructed, RehabRobo-Query asks the reasoning module to compute answers, and presents answers to queries concisely. It also includes links to detailed information in case the user wants further information. The answer to the query constructed in Fig. 3 is shown to the user as in Fig. 4.

Note that how the results of a query is displayed to user depends on the type of the query. For instance, if the query is about robots, then the user sees the names of the retrieved robots. If the query is about movements or metrics, then the user sees the concept names instead of the instance URIs which would make no sense to the user.

Fig. 5.

An overview of query construction in RehabRobo-Query.

5. Transforming a query in RehabRobo-CNL to a sparql query

The process of answering a query in RehabRobo-CNL, illustrated in Fig. 5, starts with a transformation of the query into a sparql query. We propose a transformation with the following steps.

We parse the query and form a query description tree.

We traverse the tree and form a sparql query.

The methodology of the first step of this transformation is domain-independent if the ontology functions are provided. Note that the ontology functions to extract relevant information from ontologies, so as to match the grammar and the vocabulary of RehabRobo-CNL, may need to be defined differently for other ontologies since the concepts/relations may be structured and named differently. The second step of the transformation, from query description trees to sparql queries, is domain-independent as well.

5.1. Query description trees (QDT)

We introduce a rooted, directed tree, called query description tree (QDT), to parse the RehabRobo-CNL query entered by the user. In this tree, there are five types of nodes:

root-node: Represents the type of the query.

that-node: Represents a relative clause beginning with “that”.

with-node: Represents a relative clause beginning with “with”.

and-node: Represents a conjunction.

or-node: Represents a disjunction.

Every root/that/with-node characterizes a phrase and a type/instance. An and/or-node cannot be a leaf. For each path from the root node to a leaf node, there do not exist three or more consecutive and/or-nodes. With-nodes are leaves only. That-node has one child only.

Every QDT is constructed online by a single pass over the input RehabRobo-CNL query guided by an interactive user-interface relative to the RehabRobo-CNL grammar. Therefore, the construction of a QDT from a RehabRobo-CNL query takes linear time in the size of the query (i.e., the number of nodes in QDT).

Fig. 6.

Tree representation of the sample query.

Consider, for instance, the QDT in Fig. 6 for the query:

What are the robots that target some shoulder movements with actuation = ‘electrical’ and (with transmission = ‘cable drive’ or with transmission = ‘direct drive’)?

This tree is constructed online while the user expresses the query by the help of the user interface. The root denotes the beginning of the query “What are the robots…”. According to the root, the answer to the query will contain robot names only. Since the query is about robots, the type contained in the root is “robot”.

The relative clause about these robots is the child of the root, and since this relative clause starts with “that”, it is a that-node. The type contained in this node is “shoulder movement”.

The query continues with a conjunction of relative clauses, including a simple disjunction. Clauses joined with a conjunction (resp., disjunction) are characterized by an and-node (resp., or-node) as their parent. Since these relative clauses start with “with”, they are with-nodes. They include values of properties instead of types.

More precisely, every root-node, and-node, and or-node n is associated with

$n . type$ : the class it describes (e.g., RehabRobots),

$n . children$ : its children nodes.

Every that-node n is associated with

$n . type$ : the class it describes, i.e., the class of its parent node (e.g., RehabRobots),

one of the following (according to RehabRobo-CNL):

$n . verb$ , $n . quantifier$ and $n . verb . type$ : object property of this class (e.g., $targets$ ) in relation with $some / all / the$ another class (e.g., $shoulderMovements$ ),

$n . verb$ , $n . verb . type$ and $n . verb . instance$ : an object property of this class (e.g., $ownedBy$ ) in relation with an instance (e.g., ‘ $OwnerX$ ’) of the relevant class (e.g., $Owners$ ),

$n . child$ : its child node,

$n . active$ : an identifier to specify that the verb describes an object property in active voice or passive voice,

$n . negation$ : an identifier to specify whether the condition specified by the that-node is negative.

Every with-node n is associated with

$n . type$ : the class it describes (e.g., RehabRobots),

one of the following (according to RehabRobo-CNL):

$n . noun$ , $n . operator$ and $n . noun . value$ : a data property of $n . type$ (e.g., hasFunctionality) specified by a value (e.g., ‘ $clinic$ ’) using a relevant operator (‘=’, ‘ $< =$ ’, ‘ $> =$ ’, ‘ $! =$ ’),

$n . noun$ , $n . noun . quantifier$ , and $n . noun . range$ : a data property of $n . type$ specified by some/all/ none of a set of values.

5.2. From QDT to a sparql query

The tree representing the query, in fact, characterizes a concept. Algorithm 1 generates this concept in sparql by a depth-first traversal of a QDT. As the algorithm traverses a QDT, according to the types of nodes, it generates parts of the concept in sparql. Since the transformation is done by a depth-first traversal of a QDT and thus every node is visited once, it takes linear time in the size of the QDT (i.e, number of nodes).

Let us explain Algorithm 1 by some examples.

Example 1 Suppose that the input is the tree in Fig. 6, the concept described by the tree is denoted by a variable $? x$ , and the prefix of RehabRobo-Onto is $pn$ .

Algorithm 1:

transform

Algorithm 2:

transformClass

Algorithm 3:

transformThatNode

Algorithm 4:

some

Algorithm 5:

objProp

Algorithm 6:

intersect

The algorithm starts with the root-node of the tree. Since the associated class is “robot”, our description in sparql starts by specifying the class $RehabRobots$ for variable $? x$ by Algorithm 2:

?x rdf:type rr:RehabRobots

Then the algorithm calls $transform$ for the child of the root node, which is a that-node. It later calls $transformThatNode$ (Algorithm 3) and passes that-node as an input, as well as the prefix name $rr$ and the variable $? x$ . Since that-node describes $? x$ “that targets some shoulder movements”, Algorithm 3 first calls $some$ (Algorithm 4) and generates the following sparql expression:

?x rr:targets ?y . ?y rdf:type rr:ShoulderMovements

This expression describes the object property “targets” in the first line above (Algorithm 5), relating rehabilitation robots with some shoulder movements in the second line above (Algorithm 2). These two lines are combined by Algorithm 6.

The shoulder movements are further refined with a conjunction of specific data properties, which is denoted by the child of that-node. Therefore, Algorithm 4 calls $transform$ with and-node as input, to obtain a sparql description for the conjunction.

Algorithm 7:

transformWithNode

Algorithm 8:

dataProp

Algorithm 9:

union

The first child of and-node describes the non-boolean data property (i.e., with actuation = ‘electrical’) expressed as an equality. By $transformWithNode$ (Algorithm 7), which further calls $dataProp$ (Algorithm 8), it is transformed into a sparql expression as follows:

?y rr:has_Actuation ‘electrical’

The second child of and-node contains a simple disjunction (i.e., with transmission = ‘cable drive’ or with transmission = ‘direct drive’). The algorithm transforms the information in the children of or-node into value restrictions and disjoins them (Algorithm 9):

{?y rr:has_Transmission ‘cable drive’.} UNION {?y rr:has_Transmission ‘direct drive’.}

Then the two children of and-node are conjoined first with each other (Algorithm 6), and then with the description of the object property above. This resulting concept is returned from Algorithm 4 (and thus from Algorithm 3), and then it is combined with the class description above as follows:

?x rdf:type rr:RehabRobots. ?x rr:targets ?y. ?y rdf:type rr:ShoulderMovements. ?y rr:has_Actuation ‘electrical’. {?y rr:has_Transmission ‘cable drive’.} UNION {?y rr:has_Transmission ‘direct drive’.}

After a sparql construct is obtained by Algorithm 1, a sparql query is formed as follows. We start with a PREFIX part and we declare the name space (the location of an ontology on the Web) of RehabRobo-Onto. Next, we continue with a SELECT clause. The instances of type $RehabRobots$ , by themselves, are not meaningful to the users. Thus, we want to display the names of the instances to the users. We specify it with an additional triple in the beginning of the WHERE clause, and continue the clause with the transformed sparql construct:

PREFIX rdf: <http://www.w3.org/...> PREFIX rr: <http://www.semanticweb.org/...> SELECT DISTINCT ?name WHERE { ?x rr:has_Name ?name. ?x rdf:type rr:RehabRobots. ?x rr:targets ?y. ?y rdf:type rr:ShoulderMovements. ?y rr:has_Actuation ‘electrical’. {?y rr:has_Transmission ‘cable drive’.} UNION {?y rr:has_Transmission ‘direct drive’.} }

Fig. 7.

Transformation of a query, Q3, from RehabRobo-CNL to a sparql.

Algorithm 10:

forall

Example 2 Let us now describe some other aspects of the transformation by another example. Consider, for instance, the transformation of Q3 presented in Fig. 7. This query is about movement quality metrics ?m and the relative clause involves universal quantification over all robots with motion capability = ‘grounded’ that has assessment ?m. Therefore, the main algorithm starts with generating the appropriate class MovementQualityAssessment for root-node:

?m rdf:type ?a. ?a rdfs:subClassOf rr:MovementQualityAssessment.

and then transforms that-node into sparql by calling Algorithm 3, which further calls Algorithm 10, as follows:

?r rdf:type rr:RehabRobots. ?r rr:hasMotionCapability ‘grounded’. ?r rr:hasAssessment ?m. FILTER NOT EXISTS { ?r2 rdf:type rr:RehabRobots. ?r2 rr:hasMotionCapability ‘grounded’. FILTER NOT EXISTS { ?r2 rr:hasAssessment ?m. } }

A universal restriction $\forall x A (x)$ corresponds to a negated existential restriction $\neg \exists x \neg A (x)$ . Each FILTER NOT EXISTS expression above (generated by Algorithm 11) corresponds to a negation. The outer FILTER NOT EXISTS describes that there exists no robot ?r2 with motion capability ‘grounded’ that has not evaluated ?m.

Algorithm 11:

complement

Algorithm 12:

filter

Fig. 8.

Transformation of a query, Q5, from RehabRobo-CNL to a sparql.

Example 3 Let us consider the transformation of Q5 presented in Fig. 8. This query is about publications ?p (described by their titles ?t) and involves a negative condition about these publications, i.e., “that do not reference any robots [with …]”. This negative condition is represented by a FILTER NOT EXISTS expression. The query also involves a data property check with an inequality: “with active degree of freedom ⩽ 1”. This check is performed using a FILTER operator (Algorithm 12).

6. Answering queries using pellet

We use the DL reasoner Pellet to find answers to queries, through the Jena framework. Consider, for instance, the query

What are the robots that target some wrist movements with actuation = ‘series elastic’?

whose sparql representation is as follows.

SELECT DISTINCT ?name WHERE { ?robot1 rr:has_Name ?name. ?robot1 rdf:type rr:RehabRobots. ?robot1 rr:targets ?movement1. ?movement1 rdf:type rr:WristMovements. ?movement1 rr:has_Actuation ‘series elastic’. }

After loading RehabRobo-Onto into Pellet, we present this sparql query to Pellet, and get the following answer:

( ?name = "AssistOn-Mobile" )

Consider, for instance, another query:

What are the publications with place of publication ‘ICORR’ and that reference some robots that are owned/maintained by some users with institution ‘Sabanci University’ ?

The sparql representation of this query is as follows.

SELECT DISTINCT ?name WHERE { ?publication1 rr:has_Title ?name. ?publication1 rdf:type rr:References. ?publication1 rr:has_PublishedAt ’ICORR’. ?robot1 rr:hasReference ?publication1. ?robot1 rdf:type rr:RehabRobots. ?robot1 rr:ownedBy ?user1. ?user1 rdf:type rr:Owners. ?user1 rr:has_Institution ‘Sabanci University’. }

We get the following answers from Pellet to this query:

( ?name = "Brain Computer Interface based Robotic Rehabilitation with Online Modification of Task Speed" )

( ?name = "Passive Velocity Field Control of a Forearm-Wrist Rehabilitation Robot" ) ( ?name = "Design of a reconfigurable ankle rehabilitation robot and its use for the estimation of the ankle impedance" )

7. Evaluations of RehabRobo-Query

We evaluated the underlying methods of the system RehabRobo-Query empirically over a variety of queries, and its user-interface by means of a survey with the help of participants of different expertise.

7.1. Experimental evaluations

We experimentally evaluated the computational performance of RehabRobo-CNL to sparql transformation and query answering. We identified 20 queries (Table 8) of different types as discussed in Section 2, paying also attention to their structures (e.g., conjunctive, disjunctive, negated, quantified, nested). Most of these queries were constructed based on the feedback of the experts, and evaluated as useful queries. We also constructed a few queries with different structures supported by RehabRobo-CNL, for the purpose of evaluating RehabRobo-Query. In particular, Q6 was constructed to illustrate the use of quantifiers, whereas Q19 was constructed to illustrate the use of nested conjunctions/disjunctions. These two queries are relevant for evaluation, but experts found Q6 not specific enough and Q19 too specific to be useful.

All computations were performed using a T1 Micro Instance of Amazon EC2. During the experiments, a T1 Micro Instance operated with up to 600 MB memory and two EC2 compute units, each unit providing an equivalent CPU capacity of a 1.0–1.2 GHz 2007 Xeon processor. For query answering, we used Pellet version 2.3.0.

Experimental results in Table 8 indicate that the transformation from RehabRobo-CNL to sparql can be completed within milliseconds, while query answering can take up to 2 seconds. Please recall that RehabRobo-CNL to sparql transformation is linear time in the size of the query and thus it is independent of the population level of RehabRobo-Onto while sparql query answering is a harder problem [38] and it is likely to be adversely affected as RehabRobo-Ontois further populated. However, RehabRobo-Query has been implemented in Amazon EC2 to ensure scalability of the system even after increasing the size of RehabRobo-Onto. In particular, larger EC2 instances with more memory and CPU units can be recruited as the ontology is further populated and increases in size.

Table 8
Experimental Evaluation of Transformation and Query Answering

	Query	RehabRobo-CNL to sparql [sec]	Query Answering [sec]
Q1	What are the robots that target shoulder movements and that have at least $210^{\circ}$ RoM for the flexion/extension movements of the shoulder?	0.0002	0.2277
Q2	What are the robots that target wrist movements and that have at least 2 active degrees of freedom?	0.0002	0.2083
Q3	What are the shoulder robots that target flexion/extension movements and that do not target elevation/depression movements?	0.0002	0.6580
Q4	What are the robots with active degree of freedom ⩾ ‘3’ and that target all pelvic girdle movements with backdrivability type = ‘passive’?	0.2658	0.3479
Q5	What are the ankle robots that target movements with electrical actuation and with cable drive transmission?	0.0003	0.1955
Q6	What are the movements that are targeted by some robots with (some intervention time or with all targeted disorders)?	0.0002	0.5368
Q7	What are the movements that are targeted by the robot ‘AssistOn-Finger’ and with minimum range of motion ⩾ ‘20’?	0.0002	1.8593
Q8	What are the movements that are not targeted by any robots with kinematic type = ‘redundant’ and with mechanism type ≠ ‘parallel’?	0.0002	0.3790
Q9	What are the effort metrics that are evaluated by some robots with active degree of freedom ⩾ 2?	0.0002	0.2765
Q10	What are the movement quality metrics that are evaluated by all robots with motion capability = ‘grounded’?	0.0003	0.2687
Q11	What are the kinematic aspect metrics that are evaluated by some robots that target all elbow movements?	0.1569	0.4238
Q12	What are the muscle strength metrics that are evaluated by robots that target all wrist movements with transmission = ‘direct drive’?	0.1570	0.3465
Q13	What are the psychomotoric aspect metrics that are evaluated by all robots with kinematic type = ‘redundant’ and that target all ankle movements with minimum range of motion ⩾ ‘30’?	0.1646	0.2351
Q14	What are the publications with clinical study and that do not reference any robots with active degree of freedom ⩾ 1?	0.0003	0.1345
Q15	What are the publications without clinical study or that reference some robots that do not evaluate any movement quality metrics?	0.0002	0.1474
Q16	What are the publications with place of publication ‘ICORR’ and that reference some robots that are owned/maintained by some users with institution ‘Sabanci University’?	0.0003	0.1495
Q17	What are the users that own/maintain some robots that target all ankle movements?	0.1392	0.1148
Q18	What are the robots that target some wrist movements with actuation = ‘series elastic’?	0.0004	0.1563
Q19	What are the robots with no targeted disorder or (with intervention time $! =$ ‘chronic’ and with motion capability = ‘grounded’) or with no disorder level?	0.0003	0.1432
Q20	What are the robots with interaction type = ‘exoskeleton’ and that target some finger movements (with actuation = ‘electrical’ or with actuation = ‘hydraulic’ or with actuation = ‘series elastic’)?	0.0003	0.1725

Table 9

Survey Questions and Summary Statistics

q1: Please rate the usefulness of the following aspects of RehabRobo-Query while querying about
	Mean	$σ^{2}$
	4.08	0.70
Mechanical properties of robots	4.07	0.80
Joint movements targeted by rehabilitation robots	4.00	0.65
Evaluation metrics for rehabilitation robots	3.93	0.70
Publications about rehabilitation robots	4.33	0.62

q2: For what purpose would you use RehabRobo-Query in your research?
	Mean	$σ^{2}$
	4.18	0.81
Identifying relevant rehabilitation robots suitable for a therapy	4.40	0.83
Identifying researchers/labs for relevant rehabilitation robots	4.00	0.65
Identifying outcomes of clinical studies for relevant rehabilitation robots	4.13	0.91

q3: Overall how would you rate the usefulness of RehabRobo-Query for the following user groups?
	Mean	$σ^{2}$
	4.33	0.66
Rehabilitation robotics engineers	4.67	0.49
Physical medicine experts	4.14	0.66
Hospitals	3.92	0.76
Students/researchers	4.53	0.52

7.2. User evaluations

We evaluated the usefulness of RehabRobo-Query with a group of real users to demonstrate how the proposed approach fits different needs of people working in the interdisciplinary domain of rehabilitation robotics. For that, we conducted a survey with 15 experts working in this domain. These experts included clinicians from the Rehabilitation Institute of Chicago and Acıbadem Hospital at İstanbul, rehabilitation robotics engineers from Northwestern University and Sabancı University, and experts from an SME specializing in commercialization of rehabilitation robots.

Each participant was given a brief overview of RehabRobo-Query (i.e., the variety of knowledge about rehabilitation robotics that is represented at RehabRobo-Onto, the types of queries supported by RehabRobo-Query to access this knowledge) and a direct access to the interface. Participants were asked to explore the system by freely formulating several queries of each type with respect to RehabRobo-CNL, about mechanical aspects of robots, joint movements, evaluation metrics, and relevant publications. After exploring RehabRobo-Query with different queries of their choice, participants were asked to fill in a questionnaire. The questions that were presented to the participants and the analysis of responses to this questionnaire are presented in Table 9. In addition to the questions in Table 9, the questionnaire included an open-ended question that asks for general evaluations and feedback of the participants.

The statistical analysis of responses revealed that occupation as a factor was not statistically significant at the 0.05 level for any of the survey questions; hence, all responses were aggregated for reporting. The Cronbach’s α values were calculated for the whole survey, and the α value was evaluated to be greater than 0.7, indicating high reliability of the survey.

The survey included 3 quantifiable questions: q1 aimed at evaluating the usefulness of several query types supported by RehabRobo-Query, q2 was for assessing the potential use of RehabRobo-Query, and q3 was for determination of target population of RehabRobo-Query. For all questions, the five-point Likert scale, ranging from “1” (e.g., least useful) to “5” (e.g., most useful) was used.

The main results of the survey can be summarized as follows:

Responses to q1 indicate that participants agreed with the usefulness of RehabRobo-Query while querying about mechanical properties of robots, joint movements targeted by rehabilitation robots, evaluation metrics for rehabilitation robots, and publications about rehabilitation robots.

From q2, we can infer that participants found RehabRobo-Query useful for identifying relevant rehabilitation robots suitable for a therapy, identifying researchers/labs for relevant rehabilitation robots, and identifying outcomes of clinical studies for relevant rehabilitation robots.

Responses to q3 indicate that participants evaluated RehabRobo-Query to be highly useful for rehabilitation robotics engineers, students/researchers and physical medicine experts, while also finding it useful for hospitals.

While the participants were testing RehabRobo-Query, we made ourselves available for any questions they might have. All participants were able to complete several examples of each query type and no participant required any clarifications about query construction during testing, even though the participants were not familiar with RehabRobo-Query before. Furthermore, we interviewed the participants after they completed their evaluations. In these interviews and the open-ended question in the questionnaire, we did not receive any negative feedback about query construction. These experiences can be viewed as a positive evidence that supports clarity of RehabRobo-CNL (e.g., participants were able to construct their queries using RehabRobo-Query) and the easy use of the interface (e.g., participants had no difficulty finding relevant tabs/menus, thanks to its interactive and intelligent design).

We received one improvement suggestion about the interface: while formulating queries one after another, it would be good to re-use some parts of the previous queries. Similarly, two of the participants noted that complex negative queries (especially the ones with double negations) can become unnatural and hard to understand. Relevant updates are planned as part of our future work.

8. Related work

The most related work involves ontology systems and tools that support natural language queries over ontologies.

Development of natural language interfaces that provide query answering over ontologies has been subject of research for many years. For this reason, many systems [2,4,6,21,26,27,31,34,45,48,50] have been developed that propose various approaches over some common challenges, such as processing of the natural language input (balancing ambiguity and expressiveness) and support for broad or narrow domains (portability).

One of the most recently developed systems is BioQuery-ASP [17], which is a software system that answers natural language queries over biomedical databases and ontologies related to drug discovery. It utilizes Answer Set Programming (ASP) [32] to query such knowledge resources. It allows the users to enter queries in a controlled natural language from its user interface, and then answers the queries by transforming the query in a controlled natural language into an ASP program. To enable interoperability over multiple biomedical ontologies and databases, it integrates ontologies via a rule layer in ASP. To answer queries, it utilizes ASP solvers such as clasp [22] and clasp-nk, and it also provides explanations to the queries. BioQuery-ASP covers sophisticated queries with nested relative clauses, aggregates, superlatives, nonmonotonic negation, etc. as necessitated by its domain of drug discovery; and to represent and answer these queries, ASP provides a better framework. RehabRobo-Query covers simpler queries due to the requirements of its domain of rehabilitation robotics, and thus sophisticated aspects of ASP are not needed. W3C recommended Semantic Web technologies provide a sufficiently good framework to represent and answer these queries. Both systems provide an interactive and intelligent user-interface to construct queries with respect to their own CNLs, avoiding ambiguities.

Ferrández et al. [20] introduce QACID, which covers a movie ontology. The idea is to train the system using many queries and keep the resulting set of clusters (mostly asked questions) in a database. Then, manually, each query type is associated with a sparql query. Finally, the queries are answered by a query engine that is implemented in QACID and proposed as a new entailment-based engine. Although there is no formal description of what types of queries are supported by QACID, all of the examples given in the paper are simple queries that do not involve disjunctions, negations, (nested) relative clauses, and quantifiers. In that sense, RehabRobo-Query supports a greater variety of queries. In terms of user interface, since QACID allows the users to enter queries as free text, there may be ambiguities. RehabRobo-Query provides an interactive user-interface to enter queries, and thus prevents ambiguities in the query.

FREyA [8] is developed by the creators of and as a development upon QuestIO [45]. In order to support natural language queries, it uses Stanford Parser [10] to generate a parse tree. Then, using GATE libraries [7], it tries to find some ontology concepts that can be mapped to the query terms. Then, it generates a sparql query and executes the query using the inference engine in BigOWLIM, that supports sparql, on top of Sesame. It relies on clarification dialogues with users in the cases of ambiguity or in the cases where the system cannot find an answer to a query. Over time, the system learns to ask the correct questions to the users by placing correct suggestions on top of similar queries. The system is tested on one dataset, and it is stated that FREyA failed to answer some questions (e.g., queries including negation) correctly. These questions could not be mapped to a sparql query in spite of the clarification dialogues and the learning mechanism. RehabRobo-Query supports queries with negation, and can guide the user to construct queries according to its CNL without any ambiguities.

Lopez et al. [33] introduce PowerAqua, which is evolved from AquaLog [34]. It provides natural language querying over multiple ontologies; thus, supports high scalability and portability. It uses GATE libraries and WordNet [19] to process natural language queries. It transforms the queries to triples and answers them with its own query engine. To limit the search space, it uses filtering and ranking heuristics. Since it does not contain any linguistic knowledge in the background, it has a limited linguistic coverage. It is good at answering simple questions yet it fails on questions that contain comparisons and quantifiers. RehabRobo-Query supports queries with quantifiers. In terms of user interface, PowerAqua queries are entered as free text, while some examples are illustrated to the user if requested. In that sense, there may be ambiguities of queries. Due to its CNL-based interactive interface, RehabRobo-Query prevents ambiguous queries.

Valencia-García et al. [46] introduce OWLPath, which gets user queries in a controlled natural language, transforms it into a sparql query and executes the query over an ontology via Jena framework and using the DL reasoner Pellet. The statements in its CNL start with “View any…” and follow English grammar. However, they are not full and valid English sentences. Although it is stated that OWLPath provides a Web interface through AJAX, it is not available online. OWLPath provides suggestions for queries, relative to the terminology of its CNL; in that sense, RehabRobo-Query and OWLPath have some similarities. For each condition in the query, OWLPath adds a FILTER statement in the sparql query. Therefore, the transformation of the query into sparql is not, in fact, a transformation to triples but a set of FILTER statements. In that sense, the underlying transformation of RehabRobo-Query is more systematic in covering other types of queries (e.g., with negation).

9. Conclusion

In this article, we have introduced methods for representing queries about rehabilitation robotics in a controlled natural language, transforming them into formal queries, and computing answers to them using automated reasoners over the first formal rehabilitation robotics ontology RehabRobo-Onto. We have also introduced an interactive and intelligent user-interface as part of the software system RehabRobo-Query, that guides the users during this whole process in such a way that the users do not have to know about the underlying logical formalism of the ontology or the formalism to represent queries; and they do not have to know about the use of the technologies for computing answers to their questions.

By means of answering sophisticated queries over RehabRobo-Onto, appropriate rehabilitation robots for a particular patient or a physical therapy can be found or designed; this further paves the way for translational physical medicine (from bench-to-bed and back) and personalized physical medicine. Also, RehabRobo-Query aids the exchange of information across rehabilitation robots researchers over the world, and therefore improves the state-of-the-art; it allows to identify “gaps” in functionality of rehabilitation robots, that can further improve research efforts.

Having a structured formal representation of knowledge about rehabilitation robotics as an ontology, allows answering complex queries that require integration with other knowledge resources (e.g., patient databases, disease ontologies). Along this research direction, integration of RehabRobo-Onto with existing anatomy, disease and patient ontologies can be achieved by providing a rule layer between these ontologies and RehabRobo-Onto, for integration of the related concepts [15]. In addition, some extensions in the grammar of RehabRobo-CNL, the algorithms and the user interface of RehabRobo-Query are needed to be able to answer complex queries about therapies, diseases and anatomy. These studies concerning interoperability of RehabRobo-Onto is part of our ongoing work. Further user studies are also planned as part of our future work.

Footnotes

Acknowledgements

We would like to thank Sibel Aksu Yildirim and Muhammed Kilinc (Hacettepe University, School of Physical Therapy and Rehabilitation) for their feedbacks on the types of queries from the perspectives of physical medicine, and Agis Papantoniou (Cognizone) for his help with the design of RehabRobo-Query. We would like to thank Ahmetcan Erdogan (Rehabilitation Institute of Chicago), Mustafa Yalcin, Murat Isik and Ata Otaran (Human-Machine Interaction Lab, Sabanci University), and Ezgi Demirel (Knowledge Representation and Reasoning Group, Sabanci University) for their helps with testing RehabRobo-Query. We would like to thank members of European Network on Robotics for NeuroRehabilitation for useful discussions. We are also very grateful for the participants of our survey about RehabRobo-Query.

We also would like to thank the reviewers Nick Bassiliades, Antonis Bikakis, and Eleni Kamateri for their insightful questions and useful suggestions on earlier drafts to improve the presentation of our work.

References

Antoniou and

F.V.

Harmelen, Web ontology language: OWL, in: Handbook on Ontologies in Information Systems, Springer, 2003, pp. 67–92. doi:10.1007/978-3-540-24750-0_4.

A.D.L.

Battista,

Villanueva-Rosales,

Palenychka and

Dumontier, SMART: A web-based, ontology-driven, semantic web query answering application, in: Semantic Web Challenge, Vol. 295, 2007. http://ceur-ws.org/Vol-295/paper17.pdf .

Bayona,

Bitensky,

Salter and

Teasell, The role of task-specific training in rehabilitation therapies, Topics in Stroke Rehabilitation12(3) (2005), 58–65. doi:10.1310/BQM5-6YGB-MVJ5-WVCR.

Bernstein and

Kaufmann, GINO – A guided input natural language ontology editor, in: Proceedings of the 5th International Semantic Web Conference (ISWC),

Cruzet al., eds, Springer, 2006, pp. 144–157. doi:10.1007/11926078_11.

Bütefisch,

Hummelsheim,

Denzler and

K.-H.

Mauritz, Repetitive training of isolated movements improves the outcome of motor rehabilitation of the centrally paretic hand, Journal of the Neurological Sciences130(1) (1995), 59–68. doi:10.1016/0022-510X(95)00003-K.

Cimiano,

Haase,

Heizmann,

Mantel and

Studer, Towards portable natural language interfaces to knowledge bases – The case of the ORAKEL system, Data and Knowledge Engineering65(2) (2008), 325–354. doi:10.1016/j.datak.2007.10.007.

Cunningham,

Maynard,

Bontcheva,

Tablan,

Aswani,

Roberts,

Gorrell,

Funk,

Roberts,

Damljanovic,

Heitz,

M.A.

Greenwood,

Saggion,

Petrak,

Li and

Peters, Text processing with GATE (Version 6), 2011. https://gate.ac.uk/releases/gate-6.1-build3913-ALL/tao.pdf.

Damljanovic,

Agatonovic and

Cunningham, FREyA: An interactive way of querying linked data using natural language, in: Proceedings of the 8th International Semantic Web Conference (ISWC),

Garcia-Castro,

Fensel and

Antoniou, eds, Springer, 2012, pp. 125–138. doi:10.1007/978-3-642-25953-1_11.

D’Arcus and

Giasson, Bibliographic ontology specification, Madrid: Biblioteca Nacional Española, 2009. http://bibliontology.com/.

10.

M.-C.

de Marneffe,

MacCartney and

C.D.

Manning, Generating typed dependency parses from phrase structure trees, in: Proceedings of the 5th International Conference on Language Resources and Evaluation, 2006. https://nlp.stanford.edu/pubs/LREC06_dependencies.pdf .

11.

Dogmus,

Erdem and

Patoglu, RehabRobo-Onto: Design, development and maintenance of a rehabilitation robotics ontology on the cloud, Robotics and Computer-Integrated Manufacturing33 (2015), 100–109. doi:10.1016/j.rcim.2014.08.010.

12.

Dogmus,

Gezici,

Patoglu and

Erdem, Developing and maintaining an ontology for rehabilitation robotics, in: Proceedings of International Conference on Knowledge Engineering and Ontology Development (KEOD), 2012, pp. 389–395. doi:10.5220/0004145303890395.

13.

Dogmus,

Papantoniou,

Kilinc,

S.A.

Yildirim,

Erdem and

Patoglu, Rehabilitation robotics ontology on the cloud, in: Proceedings of IEEE 13th International Conference on Rehabilitation Robotics (ICORR), 2013. doi:10.1109/ICORR.2013.6650415.

14.

Dogmus,

Patoglu and

Erdem, Answering natural language queries about rehabilitation robotics ontology on the cloud, in: Proceedings of International Conference on Knowledge Engineering and Ontology Development (KEOD), 2014, pp. 75–83. doi:10.5220/0005081400750083.

15.

Dogmus,

Patoglu and

Erdem, Interoperability of RehabRobo-Onto, PeerJ Preprints5 (2017), e3255v2. https://peerj.com/preprints/3255.pdf .

16.

Economic cost of stroke. http://www.who.int/cardiovascular_diseases/en/cvd_atlas_17_economics.pdf.

17.

Erdem,

Erdogan and

Oztok, BIOQUERY-ASP: Querying biomedical ontologies using answer set programming, EMBnet. journal18(B) (2011), 62–64. doi:10.14806/ej.18.B.551.

18.

I.H.

Ertas,

Hocaoglu and

Patoglu, AssistOn-Finger: An under-actuated finger exoskeleton for robot-assisted tendon therapy, Robotica32(8) (2014), 1363–1382. doi:10.1017/S0263574714001957.

19.

Fellbaum, WordNet: An Electronic Lexical Database. Language, Speech and Communication, MIT Press, 1998.

20.

Ferrández,

Izquierdo,

Ferrández and

J.L.

Vicedo, Addressing ontology-based question answering with collections of user queries, Information Processing and Management45(2) (2009), 175–188. doi:10.1016/j.ipm.2008.09.001.

21.

Frank,

H.-U.

Krieger,

Xu,

Uszkoreit,

Crysmann,

Jörg and

Schäfer, Question answering from structured knowledge sources, Journal of Applied Logic5(1) (2007), 20–48. doi:10.1016/j.jal.2005.12.006.

22.

Gebser,

Kaufmann,

Neumann and

Schaub, clasp: A conflict-driven answer set solver, in: Proceedings of International Conference on Logic Programming and Nonmonotonic Reasoning (LPNMR),

Baral,

Brewka and

Schlipf, eds, Springer, 2007, pp. 260–265. doi:10.1007/978-3-540-72200-7_23.

23.

J.H.

Gennari,

M.A.

Musen,

R.W.

Fergerson,

W.E.

Grosso,

Crubézy,

Eriksson,

N.F.

Noy and

S.W.

Tu, The evolution of Protégé: An environment for knowledge-based systems development, International Journal of Human-Computer Studies58(1) (2003), 89–123. doi:10.1016/S1071-5819(02)00127-1.

24.

Global burden of stroke. http://www.who.int/cardiovascular_diseases/en/cvd_atlas_15_burden_stroke.pdf.

25.

Horrocks,

P.F.

Patel-Schneider and

F.V.

Harmelen, From SHIQ and RDF to OWL: The making of a web ontology language, Journal of Web Semantics1 (2003), 7–26. doi:10.1016/j.websem.2003.07.001.

26.

Kaufmann,

Bernstein and

Fischer, NLP-Reduce: A naive but domain-independent natural language interface for querying ontologies, in: Proceedings of the 4th European Semantic Web Conference (ESWC), 2007.

27.

Kaufmann,

Bernstein and

Zumstein, Querix: A natural language interface to query ontologies based on clarification dialogs, in: Proceedings of the 5th International Semantic Web Conference (ISWC), Springer, 2006, pp. 980–981.

28.

Kuhn, A survey and classification of controlled natural languages, in: Computational Linguistics, Vol. 40, MIT Press, Cambridge, 2014, pp. 121–170. doi:10.1162/COLI_a_00168.

29.

Kwakkel,

B.J.

Kollen and

H.I.

Krebs, Effects of robot-assisted therapy on upper limb recovery after stroke: A systematic review, Neurorehabilitation and Neural Repair22 (2008), 111–121. doi:10.1177/1545968307305457.

30.

Kwakkel,

Wagenaar,

Twisk,

Langkhorst and

Koetsier, Intensity of leg and arm training after primary middle-cerebral artery stroke: A randomized trial, Lancet35 (1999), 191–196. doi:10.1016/S0140-6736(98)09477-X.

31.

Lei,

Uren and

Motta, SemSearch: A search engine for the semantic web, in: Proceedings of 5th International Conference on Knowledge Engineering and Knowledge Management Managing Knowledge in a World of Networks,

Staab and

Svatek, eds, Lect. Notes in Comp. Sci., Springer-Verlag, 2006, pp. 238–245. doi:10.1007/11891451_22.

32.

Lifschitz, What is answer set programming? in: Proceedings of the 23rd National Conference on Artificial Intelligence (AAAI), Vol. 3, 2008, pp. 1594–1597. http://dl.acm.org/citation.cfm?id=1620270.1620340 .

33.

Lopez,

Fernández,

Motta and

Stieler, PowerAqua: Supporting users in querying and exploring the semantic web, Semantic Web3(3) (2012), 249–265. doi:10.3233/SW-2011-0030.

34.

Lopez,

Uren,

Motta and

Pasin, AquaLog: An ontology-driven question answering system for organizational semantic intranets, Web Semantics: Science, Services and Agents on the World Wide Web5(2) (2007), 72–105. doi:10.1016/j.websem.2007.03.003.

35.

J.M.

McDowd,

D.L.

Filion,

P.S.

Pohl,

L.G.

Richards and

Stiers, Attentional abilities and functional outcomes following stroke, The Journals of Gerontology Series B: Psychological Sciences and Social Sciences58(1) (2003), 45–53. doi:10.1093/geronb/58.1.P45.

36.

Munawar,

Yalcin and

Patoglu, Redundant kinematics and workspace centering control of assiston-gait overground gait and balance trainer, in: Proceedings of 2016 IEEE International Conference on Robotics and Automation (ICRA), 2016, pp. 3704–3710. doi:10.1109/ICRA.2016.7487556.

37.

Nykanen, The effectiveness of robot-aided upper limb therapy in stroke rehabilitation: A systematic review of randomized controlled studies, Master’s thesis, Jyvaskyla University, 2010. https://jyx.jyu.fi/handle/123456789/23001.

38.

Pérez,

Arenas and

Gutierrez, Semantics and complexity of SPARQL, in: Proceedings of the 5th International Semantic Web Conference (ISWC),

Cruzet al., eds, Springer, 2006, pp. 30–43. doi:10.1007/11926078_3.

39.

Peroni and

Shotton, FaBiO and CiTO: Ontologies for describing bibliographic resources and citations, in: Web Semantics: Science, Services and Agents on the World Wide Web, Vol. 17, 2012, pp. 33–43. doi:10.1016/j.websem.2012.08.001.

40.

Platz, Evidence-based arm rehabilitation – A systematic review of the literature, Nervenartz74(10) (2003), 841–849. doi:10.1007/s00115-003-1549-7.

41.

G.B.

Prange,

M.J.

Jannink,

C.G.

Groothuis-Oudshoorn,

H.J.

Hermens and

M.J.

Ijzerman, Systematic review of the effect of robot-aided therapy on recovery of the hemiparetic arm after stroke, Journal of Rehabilitation Research and Development43 (2006), 171–184. doi:10.1682/JRRD.2005.04.0076.

42.

Prud’Hommeaux,

Seaborneet al., SPARQL query language for RDF. W3C recommendation, 15, 2008. http://www.w3.org/TR/rdf-sparql-query/.

43.

Sirin,

Parsia,

B.C.

Grau,

Kalyanpur and

Katz, Pellet: A practical OWL-DL reasoner, Web Semantics: Science, Services and Agents on the World Wide Web5(2) (2007), 51–53. doi:10.1016/j.websem.2007.03.004.

44.

Sunderland,

Tinson,

Bradley,

Fletcher,

Langton and

Wade, Enhanced physical therapy improves recovery of arm function after stroke: A randomised clinical trial, Journal of Neurology, Neurosurgery and Psychiatry55 (1992), 530–535. doi:10.1136/jnnp.55.7.530.

45.

Tablan,

Damljanovic and

Bontcheva, A natural language query interface to structured information, in: Proceedings of the 5th European Semantic Web Conference (ESWC),

Bechhofer,

Hauswirth,

Hoffmann and

Koubarakis, eds, Springer, 2008, pp. 361–375. doi:10.1007/978-3-540-68234-9_28.

46.

Valencia-García,

García-Sánchez,

Castellanos-Nieves and

Fernández-Breis, OWLPath: An OWL ontology-guided query editor, IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans41(1) (2011), 121–136. doi:10.1109/TSMCA.2010.2048029.

47.

J.H.

van der Lee,

I.A.K.

Snels,

Beckerman,

G.J.

Lankhorst,

R.J.

Wagenaar and

L.M.

Bouter, Exercise therapy for arm function in stroke patients: A systematic review of randomized controlled trials, Clinical Rehabilitation15(1) (2001), 20–31. doi:10.1191/026921501677557755.

48.

Wang,

Xiong,

Zhou and

Yu, PANTO: A portable natural language interface to ontologies, in: Proceedings of the 4th European Semantic Web Conference (ESWC),

Franconi,

Kifer and

May, eds, Springer, 2007, pp. 473–487. doi:10.1007/978-3-540-72667-8_34.

49.

Yalcin and

Patoglu, Kinematics and design of AssistOn-SE: A self-adjusting shoulder-elbow exoskeleton, in: Proceedings of 2012 4th IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), 2012, pp. 1579–1585. doi:10.1109/BioRob.2012.6290928.

50.

Zhou,

Wang,

Xiong,

Wang and

Yu, SPARK: Adapting keyword query to semantic search, in: Proceedings of the 6th International Semantic Web Conference and the 2nd Asian Semantic Web Conference (ISWC/ASWC),

Abereret al., eds, Springer, 2007, pp. 694–707. doi:10.1007/978-3-540-76298-0_50.

RehabRobo-Query : Answering natural language queries about rehabilitation robotics ontology on the cloud

Abstract

Keywords

1. Introduction

1 http://www.cost.eu/COST_Actions/bmbs/TD1006

4 http://www.rehabilitationrobotics.eu/

3.1. Types of queries supported by RehabRobo-CNL

5.1. Query description trees (QDT)

7. Evaluations of RehabRobo-Query

7.1. Experimental evaluations

Table 8 Experimental Evaluation of Transformation and Query Answering

8. Related work

9. Conclusion

Footnotes

Acknowledgements

References

¹
http://www.cost.eu/COST_Actions/bmbs/TD1006

⁴
http://www.rehabilitationrobotics.eu/

Table 8
Experimental Evaluation of Transformation and Query Answering