Abstract
Background
Informal caregivers of people with dementia often struggle with several stressors, and their mental and physical health can be negatively affected by providing care. iSupport is an online self-help programme designed by the World Health Organization to provide education, skills training, and support to those caregivers.
Purpose
This paper describes the development of an adapted version of iSupport to the Portuguese culture. We present the adaptation process, methods used and results for the country-specific version.
Methods
The adaptation of iSupport was operationalised in five steps: needs assessment; content translation by an authorised translator and technical accuracy check by health professionals; cultural adaptation; independent appraisal of contents by an expert panel; and fidelity check by programme authors.
Results
Findings indicated the adjustments needed in the original iSupport contents. Overall, 323 text excerpts were proposed to adaptation, comprising semantic and conceptual equivalence of expressions, and adjustments to cultural habits, customs, traditions, local resources, and practices. The expert panel identified the need to reformulate definitions, designations, recommendations, and case studies to fine-tune the original contents to local knowledge and practices. Most adaptations were considered to be in line with the core concepts of the original programme.
Conclusions
iSupport is an innovative tool capturing the attention of several organisations around the world interested in translating, adapting, and using the programme. This paper is the first describing the methodological approach for culturally adapting iSupport, constituting an essential resource for those aiming to replicate the process or culturally adapting evidence-based interventions. Tailoring iSupport to the Portuguese context was critical to accommodate the cultural experiences of the target group as well as the knowledge, theoretical approaches, and practices of local professionals. The European-Portuguese version of iSupport introduces the adjustments needed to implement a useful new intervention programme aimed at minimising the psychological distress of Portuguese caregivers.
Keywords
Introduction
In 2015, 47.5 million people were estimated to be living with dementia worldwide, and this number is expected to almost triple by 2050 (Alzheimer’s Disease International, 2013). Countries with higher shares of very old people typically have a greater proportion of people with dementia. In Portugal, estimations pointed to 19.9 cases of dementia per 1000 inhabitants in 2017, positioning the country above the Organisation for Economic Co-operation and Development (OECD) average and in the fourth place in what concerns dementia prevalence (OECD, 2017). Due to its high prevalence as well as the economic and social impact on families, caregivers, and communities, dementia has been established as a public health priority (World Health Organization & Alzheimer’s Disease International, 2012).
Since the syndrome affects memory, thinking, behaviour, and the ability to perform everyday activities, people with dementia often require consistent care, a responsibility frequently assumed by family members and other informal caregivers. Informal caregivers provide unpaid and ongoing assistance with basic or instrumental activities of daily living to a person with a chronic illness or disability (Roth et al., 2015). This form of assistance is most common in Southern Europe (Barbosa & Matos, 2014). Portugal has the highest rate in Europe of informal home care by caregivers with more than 50 years living with the care receiver (12.4%) (Barbosa & Matos, 2014).
Informal caregivers of people with dementia frequently struggle with several stressors, such as those related to financial, time, and care management (World Health Organization & Alzheimer’s Disease International, 2012). These caregivers are more likely to develop depression and anxiety disorders as well as hypertension, digestive, and breathing problems when compared to the general population (Ory et al., 1999; World Health Organization (WHO), 2015). The same applies when compared with informal caregivers of people with other chronic diseases (Ory et al., 1999; WHO, 2015).
Training and support programmes have been demonstrated to help reduce the psychological distress in informal caregivers of people with dementia. Several studies on the efficacy of multicomponent interventions including skills training, psychoeducation, techniques for self-care, changes in the caregiver’s setting, among others, have presented more robust results showing favourable effects of such interventions on caregiver burden, mental health, subjective well-being, skills/knowledge, and self-efficacy (Gilhooly et al., 2016; Parker et al., 2008; Paúl et al., 2019; Selwood et al., 2007; Sörensen et al., 2002). Despite evidence showing a beneficial effect of these interventions on several caregiver outcomes, the face-to-face format in which they are typically implemented have been documented to pose situational barriers to their access (Abreu et al., 2015; Egan & Pot, 2016; WHO, 2015). The same form of implementation has also been associated to high dropout rates (Abreu et al., 2015; Egan & Pot, 2016; WHO, 2015). Those barriers include difficulties in getting free from caregiving responsibilities and arranging for transportation. Regarding the offer of such programmes, barriers include geographical inequalities and lack of appropriately trained health workforce, infrastructures to scale up services, public policy initiatives, as well as funds for dementia services, research, and training (WHO, 2015). Consequently, support provision to informal dementia caregivers is either absent or depends on scattered initiatives. In Portugal, there are some good examples of such initiatives, but those usually rely on short and hardly renewable funded projects (Figueiredo et al., 2012; Paúl et al., 2019; Sousa et al., 2018).
This landscape stresses the need for implementing accessible, convenient, engaging, and affordable services targeted at informal caregivers of people with dementia (World Health Organization & Alzheimer’s Disease International, 2012). The use of Information and Communication Technology, in particular, online training and support programmes, may be instrumental in overcoming some of those barriers. Therefore, the interest in these resources has grown substantially in the last decade (Egan & Pot, 2016). Arguments supporting this form of implementation of interventions rely on their ubiquitous nature, convenient and privacy-preserving delivery, potential scalability, and presumed (cost)effectiveness (Pot et al., 2015; World Health Organization & Alzheimer’s Disease International, 2012). Favourable evidence was previously reported regarding the acceptability of internet-based training and support programmes for informal dementia caregivers (Cristancho-Lacroix et al., 2014; Marziali & Garcia, 2011; Pot et al., 2015). However, a more cautious interpretation is required with respect to the effectiveness of those programmes. Promising results have been found regarding the reduction of anxiety and depression symptoms, perceived stress and burden, and increase of caregiver well-being and self-efficacy (Boots et al., 2014; Egan et al., 2018). Nonetheless, the available supporting evidence may lack methodological quality, and the marked methodological diversity found across studies compromises a robust pooling of results (Boots et al., 2014; Egan et al., 2018).
The development of accessible, acceptable, and effective training and support interventions for informal caregivers of people with dementia is stressed as a strategic priority in the global action plan on the public health response to dementia 2017–2025 (World Health Organization (WHO), 2017). To address this priority, the WHO resorted to international experts to develop “iSupport”. iSupport is an online self-help programme designed to provide education, skills training, and social support to informal caregivers of people with dementia. The programme is based on the evidence-based guidelines and recommendations in the WHO’s Mental Health Gap Action Programme (WHO, 2016). The e-programme comprises five modules containing 23 lessons that approach well-established topics on dementia and caregiver support (cf. Figure 1). In accordance with good practices for digital engagement (Svartvatten et al., 2015), the training plan can be customised, i.e. users can adjust and choose the lessons they take according to their availability and particular needs. Each lesson includes interactive exercises that provide the user with instant feedback about given answers. The programme resorts to problem-solving and cognitive behavioural therapy techniques like psycho-education, behavioural activation, cognitive reframing, relaxation, and antecedent-behaviour-consequence analysis (Pot, n.d.). iSupport was written in English using a generic, multi-cultural approach, thus requiring translation and cultural adaptation to each local setting. Given the absence of evidence-based online training and support programmes for informal dementia caregivers in Portugal, we aim to culturally adapt iSupport to the Portuguese context. This is among the first experiences of adaptation of the generic version to a particular population and setting. The cultural adaptation of an intervention programme can be described as the process of systematically adjusting its elements to the language, culture, and context to match the cultural patterns, meanings, and values of a particular group (Bernal et al., 1995; Falicov, 2009).

Overview of the iSupport online self-help training and support programme: structure and contents. M: module; L: lesson. Adapted from Pot (n.d.).
Adapting an intervention to enhance cultural sensitivity is thought to benefit its ecological validity (Bernal et al., 1995). In the context of iSupport, ecological validity is the assumption that the programme can be generalised and useful to the target group in other cultural contexts if cultural adaptations are properly made (Pot, n.d.). Cultural adaptations of evidence-based interventions must keep the fidelity to its core aspects. Likewise, it must add cultural content to enhance the target group engagement and the impact on defined outcomes (Falicov, 2009). The extent to which the effects on defined outcomes are actually enhanced by cultural adaptation has been highly debated. However, the premise assumed in this paper is that meeting a minimal standard to deliver intervention programmes in different cultural contexts is ensuring its cultural meaningfulness and sensitivity, guaranteeing that it uses culturally fit examples.
This paper describes the development of a culturally adapted version of the WHO iSupport programme for informal caregivers of people with dementia to the Portuguese context. We describe the adaption process of the programme contents and present theoretical and interventional considerations that emerged from its analysis by a local expert panel.
Methods
While there is no universal agreement on how an intervention should be adapted to another cultural context, it is well accepted that this is a process comprising several stages (Wang et al., 2006). In this work, the adaptation process included five major steps (cf. Figure 2): (1) needs assessment; (2) content translation and technical accuracy check; (3) cultural adaptation; (4) expert panel appraisal; and (5) fidelity check. The procedures adopted were based on the adaptation guide made available by WHO to the licensees (Pot, n.d.). The guide offers standardised advice for translating and culturally adapting iSupport to fit the local target group while guaranteeing that the adapted version is equivalent to the generic one. This standardised advice includes minimal steps to be undertaken to culturally adapt the programme (e.g. translation by a certified translator, the involvement of the local Alzheimer’s organisation, fidelity check by WHO). It also includes recommendations on expressions to avoid in the translation (e.g. patients, demented persons). Per lesson, the guide also lists those contents that most probably would require cultural adaptation and provides orientations to adapt words, names, links, and resources.

Methodological approach to adapt an evidence-based online intervention for informal dementia caregivers (iSupport): a five-step procedure.AQ8
Since interventions are more likely to be accepted when cultural adaptation involves the participation of stakeholders (Gearing et al., 2013), this work was conducted in partnership with the national Alzheimer Association. Ethical approval for this study was granted by the Ethics Committee for Health of the São João University Hospital Center/Faculty of Medicine of the University of Porto (date: 27 July 2018; reference 208/18).
Needs assessment
The need for an intervention programme like iSupport and the possibility of a target group accessing it may vary from country to country. An assessment of needs was first conducted to determine the pertinence of adapting the programme to the Portuguese context. This involved: (i) a review of empirical studies on the characteristics and psychosocial needs of informal caregivers of people with dementia; (ii) an analysis of databases on dementia and caregiving statistics, and on population internet usage; and (iii) a collection of information from informal dementia caregivers on the access to training and support interventions. The discussion of the findings on needs assessment is discussed elsewhere (Teles et al., n.d.).
Content translation and technical accuracy check
The text of the iSupport generic version – roughly 60,000 words – was translated from British English into European-Portuguese. The translation was carried out by a professional translator integrated in a health research and development centre, specialised in terminology and highly experienced with health-related contents such as health communication material, intervention programmes, and scientific papers. The translation work was preceded by a debrief between the principal investigator and the translator to establish translation strategy guidelines, including the adoption of international standards on the language describing people with dementia.
The translation process was carried out following an interpretative-communicative method (Molina & Hurtado Albir, 2004; Nord, 2018), using translation techniques as needed according to the discursive and contextual characteristics of the source text unit. The units raising questions with regard to the technical meaning or context were brought to the attention of the project’s principal investigator, proposing different options of translation for discussion. The chosen translation method aimed at ensuring from a very initial stage that the end-user comprehensibility and cultural appropriateness of contents were achieved. The first version of the translated iSupport was then subjected to a technical check, where the scientific and technical accuracy of the translated text was compared to the original version. This review, focused on specialised contents, was carried by a mental health professional with the additional consultancy of other professionals (e.g. nurses, gerontologists) on specific contents (e.g. incontinence aids). Small changes were made post-review (e.g. the designation of relaxation techniques), in an iterative process with the translator.
Cultural adaptation
After content translation and technical accuracy check, the local adaptation of iSupport took place, guided by adaptation directives made available by WHO. These directives cover: the adaptation of contents per se, by providing a list of text contents which might require local adaptation together with a set of possible alternatives for some of the foreseen alterations (e.g. list of leisure activities); advice on general approaches to the adaptation work (e.g. involving local stakeholders as the local Alzheimer’s organisation); and instructions on how to report proposed adaptations, so they can be subjected to a fidelity check by WHO.
This was a two-step process. First, a preliminary adaptation of contents was carried by the research team, taking into account the orientations provided in WHO’s guide (adaptation phase 1). The elements subjected to cultural adaptation included: (1) words and expressions for reading-level adjustments to the target group; (2) words and expressions for semantic and conceptual equivalence; (3) personal names of characters used in the iSupport scenarios; (4) resources (e.g. information materials, services available, technologies and products/services); and (5) references to cultural habits, customs, and traditions (e.g. leisure activities, food, customary practices). Neither changes to core aspects of the intervention nor alterations to scenarios or answer options of the interactive exercises were proposed.
The adaptations proposed were grounded on the cultural knowledge of the research team, all Portuguese natives, based on observable characteristics, experiences, norms, values, behavioural patterns, and beliefs of the Portuguese population. The research team’s experience with informal caregivers of people with dementia was also considered. For specific contents, the adaptations were proposed resorting to national statistics and empirical studies conducted with the Portuguese population (e.g. most popular leisure activities); national registries (e.g. Institute of Registries and Notaries for registries of personal names); and resource mapping (e.g. websites and informative contents). In the second stage, the translated and adapted contents were appraised by an expert panel (cf. Expert panel appraisal under the Methods section). The purpose of requesting the panellists to analyse not only the translated but also the adapted contents was to collect feedback on the appropriateness of the adaptation work carried out by the research team. From the expert panel review, additional proposals for cultural adaptations were made and discussed in a consensus/adaptations validation round. All adaptations were registered to be reported and approved by WHO (cf. Fidelity check).
Expert panel appraisal
The translated and adapted version of iSupport was independently reviewed by a panel of four national experts on ageing, dementia and psychosocial and/or psychological support for informal caregivers of people with dementia. All experts speak both European-Portuguese and British English fluently. The panellists were recruited taking into account their heterogeneous and complementary profiles and different backgrounds, particularly in psychology, gerontology, and law. The members of the expert panel are a representative of the national Alzheimer Association; a full professor of psychology and researcher in the fields of ageing and dementia; the principal investigator of a research group on ageing topics, author and evaluator of psychoeducational programmes for informal caregivers of people with dementia; and a field professional with vast experience in designing, implementing, and assessing intervention programmes for informal caregivers of people with dementia.
Prior to the content review, the experts received a “review guide”, providing instructions on the objectives and procedures for carrying this task. The panellists were instructed to first go through the complete text of the 23 lessons, then perform a qualitative appraisal of the translation and adaptation, freely commenting and making recommendations on the documents. Panellists were also asked to assess the translation and adaptation by rating the extent to which (i) the language used was sensitive and promoted the dignity of people with dementia and informal caregivers (e.g. presence of any stigmatising expression or term); (ii) the contents were clear and comprehensible; (iii) the terms/expressions were potentially familiar to the target group (e.g. use of jargon, colloquialisms, idioms, or vernacular terms); (iv) the presentation of concepts and scenarios was theoretically, technically, and culturally precise; and (v) the contents subjected to cultural adaptation (adaptation phase 1) were adequate (e.g. names, resources, local habits).
In addition, the experts were invited to make a quantitative appraisal of the same parameters that were previously qualitatively evaluated for each lesson: each of the five parameters was assessed on a scale ranging from 1 to 4, with 1 meaning that the lesson completely missed the parameter and 4 that it fully contained the quality expressed in the parameter (e.g. language sensitivity; 1 = not sensitive, fully review; 2 = not very sensitive, needs major revision; 3 = sensitive, needs minor revision; 4 = very sensitive, no revision needed). After receiving the reviews from all experts, their comments and recommendations were extracted and systematised by the research team. Questions on translation choices were discussed again with the translator. A round of consensus and validation of adaptations was carried to discuss the contents that (i) gave place to disagreeing comments/recommendations from different experts; (ii) triggered suggestions of more in-depth changes from at least one of the experts (e.g. changing scenario elements or answer options in interactive exercises); and/or (iii) raised theoretical or interventional considerations. The remaining comments or suggestions were accepted and documented to be reviewed by WHO (cf. Fidelity check under Methods section). The same procedure was adopted for suggestions and recommendations agreed by the panel after the consensus round.
Fidelity check
In a cultural adaptation process, it is fundamental to document adaptations that have been made and the reasons for them (Bernal et al., 1995). For the Portuguese version of iSupport, all linguistic and cultural adaptations proposed were systematically documented and properly justified in structured forms provided by WHO. After receiving the adaptation forms, a fidelity check was performed by WHO, verifying to what extent the requested adaptations were in line with the original aim, core concepts, and messages of the generic version of iSupport.
Results
This section presents the results from the cultural adaptation of the iSupport contents to the Portuguese context. First, the elements subjected to cultural adaptation are described. The results have been arranged according to the categories of the adaptation that emerged. Second, outcomes from the expert panel appraisal are reported, and theoretical as well as interventional considerations raised from this assessment are discussed.
Cultural adaptation
For the 23 lessons composing iSupport (see Figure 1), a total of 323 text contents were proposed to be adapted (Module 1 = 18; Module 2 = 64; Module 3 = 46; Module 4 = 77; Module 5 = 118). These include both adaptations suggested by the research team and reviewed by the expert panel, and adaptations recommended by the experts. The number of sessions per module varies from 1 to 10 (e.g. Module 1 = 1 lesson vs. Module 5 = 10 lessons). Lesson “Personal care” (Module 4_Lesson 4) was the one receiving the largest number of adaptation proposals (n = 22), while lesson “Putting it all together” (Module 5_Lesson 10) received the lowest amount of adaptation proposals (n = 1). These numbers are a rough illustration of the adaptation work since this work is eminently qualitative, as analysed by each adapted category (cf. Methods section).
Reading-level adjustments
Portuguese informal caregivers have low to intermediate education levels which is in line with international trends (Barbosa & Matos, 2014; OECD, 2011). According to the Eurostat classification of educational activities based on the International Standard Classification of Education (UNESCO Institute for Statistics, 2011), a low education level include “Less than primary, primary and lower secondary education”, while an intermediate education level covers “Upper secondary and post-secondary non-tertiary education” (Eurostat, 2017). Barbosa and Matos (2014) found that informal Portuguese caregivers with more than 50 years of age have an average level of schooling ranging from 2.1 for co-residential caregivers to 2.8 for extra-residential caregivers, corresponding to low and intermediate education levels, respectively.
Accordingly, the translation and adaptation work required the adjustment of words and expressions to ensure the content accessibility to the target group. To be syntonic with this group, an informal and straightforward verbal style was used in the Portuguese version. Among the translation choices (e.g. synonyms), plain words were chosen whenever possible. More complex reformulations were required in some cases as no satisfactory synonyms were available to pass on the message on a plain language (e.g. “Practice listening without negative judgement” vs. the Portuguese and reading-level equivalent “Practice listening without being hard on yourself”). Moreover, the Portuguese translation from English and content adaptation expanded the text in 10.4%. Some sentences were minimally adapted from the original version to fit in two rather than one sentence in the Portuguese version, to avoid long and complicated phrases. The appropriateness of reading-level adjustments throughout the 23 lessons was globally well appraised by the expert panel (cf. Expert panel appraisal under Results section).
Conceptual and semantic equivalence
Since words and expressions often convey a different conceptual meaning or have multiple meanings across cultures, adjustments to local standards were needed to ensure that those were correctly interpreted. For instance, the word “lesson” in the source text is defined as “a period of time in which a person is taught about a subject or how to do something” (Cambridge dictionary). While having the same meaning in the target language, its use is typically reserved to formal education contexts (“Lesson: what the professor explains or the student must learn”; Priberam Portuguese dictionary). Therefore, the semantic equivalent “session” (Portuguese = “sessão”) was adopted, taking into account the intervention context. Also, the word “compassion”, extensively used in the source text to describe a strong feeling of sympathy, is commonly interpreted in the target language as being linked with religious practices, or as feeling sorry/pity for someone. If used to refer to people with dementia, the original term can be interpreted as having a depreciative tone. For that reason, the word “understanding” was used to replace it in the target text. The word “miserable” in the source text, meaning “extremely unhappy or sad”, does not have a direct translation into the target language. Some of the closest single-word equivalents like “infeliz” (unhappy) or “desafortunado” (unfortunate) do not necessarily carry the semantic traits expressing the same degree of sadness or unhappiness. In the adapted version, this word was eliminated as it appears in a context where sadness is already expressed (source text: “makes you feel angry, sad, happy, or miserable”). In another example, the expression “wash herself before breakfast” in the source text, used to describe the process of making oneself tidy and clean, would be interpreted strictly in the target language as “washing”. Thus, an equivalent word in the target language, “grooming” (Portuguese: “arranjar-se”), was used in alternative.
In addition to words or expressions conveying different conceptual meanings in the Portuguese language, conceptual equivalence was required in situations where for the source language a single word can describe a concept while the same does not occur in the target language. That was the case, for instance, of the words (incontinence) “pad”, “blender”, or the pronoun “you”. The latter is conveyed by different words in the target language depending on the formal or informal context of use (Portuguese: “tu”, “você”, “a senhora/o senhor”). The way to address someone is of great importance given that children and grandchildren of older adults living today in Portugal traditionally treat them using the formal tone since the informal could be interpreted as disrespectful. The inverse situation, i.e. the use of a single word in the target text to describe two or more words/concepts in the source text also occurred. For example, even though “taking a bath” and “showering” are two different concepts in both the source language and the target language, in practice, a single term is commonly used in the target language (Portuguese: “banho”) to refer to this action, whether the person is washing herself using a shower or a bathtub.
Additional work on conceptual equivalence relates to the institutionalised practices and services in the target culture. For instance, if considering the source text “care at home, in the community or a care facility”, we realise that “care at home” or “in the community” are different concepts in contexts where the model of life care communities is in practice. As this model is not established in Portugal, the concepts of “caring at home” or “in the community” are conceptually equivalents in the target language. On the other hand, in the Portuguese context, the concept of “care facility” might represent either a model of care delivery by the health care system (Units for Integrated Continuous Care) or by the social support system (residential solutions/nursing homes).
Personal names
Translating and adapting personal names is challenging and requires sensitive decision-making, especially in the absence of very clear translation guidelines for these contents (Pour, 2009). For the entire programme, 96 personal names of characters used in scenarios (42 male and 54 female names) were adapted to Portuguese names. As personal names are deeply rooted in the culture of the speakers of a language (Pour, 2009), the research team knowledge about cultural references and figurative use of language was a key element in this process. As this knowledge is eminently subjective, different procedures can be and were applied in the process of translating and adapting personal names. In this paper, the procedures are named according to the work of Fernandes (2006). The application of each procedure was balanced with a judgement on how common the personal name was in the Portuguese population, as framed by generational trends, i.e. if a character was an older person, how common was the name in the country at the time she/he was born. This judgement made use of national databases (e.g. Institute of Registries and Notaries).
A first approach to the translation and adaptation of personal names was the use of the copy strategy when the names in the source language exist in the target language. In those cases, the names are reproduced without any orthographic adjustment in the target text. This strategy was employed in 11 of the 96 personal names (e.g. Maria, Manuel, Marina). When the first strategy could not be implemented, a second approach – transcription – was used, i.e. the names in the source text were transliterated or adapted morphologically, phonologically, or grammatically to match the target language system. This was the case for nine names (e.g. Martha-Marta; Esther-Ester; Olivia-Olívia; George-Jorge). A third approach was to employ the strategy of phonological replacement, i.e. names in the target text were chosen to mimic phonological features of those in the source text. This strategy was used in most cases (67 names; e.g. John-João; Rosie-Rosa; Pablo-Paulo; Anne-Ana). In cases where no other strategy was possible, a formally or semantically unrelated name was used in the target text to replace another in the source text (substitution strategy). This strategy was used mostly to replace names with origin in Eastern languages (nine names; e.g. Wei-António; Xu-Luísa; Guoliang-Alberto).
Resources
In the scope of iSupport, resources can be understood as those contents referring to or redirecting the informal caregivers to: (i) additional information sources or materials; (ii) support services and specialised professionals in the local context; and (iii) technologies, products, or solutions that might be employed to address particular issues (e.g. weight loss).
The adaptation work resulted in the replacement of 14 links, either to webpages (12) or informative manuals (2). All original links are to pages and materials of Alzheimer’s Disease International. Those were relatively straightforward to replace by links to analogous pages, i.e. pages containing roughly the same information, from the national Alzheimer Association (Associação Alzheimer Portugal). However, finding alternatives for the informative manuals was challenging, as no direct equivalent was available in the Portuguese context. For example, while the e-manual “Help for caregivers” (WHO, 2006) is available in several languages, it has not yet been translated into Portuguese. Alternatively, Portuguese e-manuals targeting informal caregivers of people with dementia were mapped and their contents compared with those in the original e-manual. As a result, the e-manual “Manual do cuidador” (Alzheimer Europe & Alzheimer Portugal, 2006) was chosen as the most suitable alternative. Both pieces of information are comparable as they cover the same thematic contents, and points of comparison were detailed for the fidelity check process. In other two cases, however, several alternatives needed to be included in the Portuguese version to cover all contents approached in the original resource as no single equivalent resource was available.
Adaptations were also required concerning the reference to health or social support services available in the local context as well as the way to access them. For example, some of the recommendations in the source text are to consult a doctor to “check if the glasses prescribed to the person with dementia remain adequate” or consult the general practitioner to “identify loose teeth, ulcers or infections” due to difficulties in using the dentures. For the first case, it is more common in the Portuguese context to consult an optometrist or optician who is usually more accessible than ophthalmologists, both in terms of costs per consultation and waiting time. In the second case, consulting a dentist would be the most probable action. For each case, both alternatives were included in the target text in the interest of suitability to the local context. In addition, adjustments to text excerpts referring to the process of accessing a service were also required. For instance, in case of suspicion that a relative or friend might have dementia, the source text recommends to “call a medical doctor office and make an appointment”. In the Portuguese context, this procedure is associated with private medical practice as access to a specialised doctor through the national healthcare system, such as a neurologist, is done by referral from the general practitioner/family doctor.
Minor adaptations were proposed when the source text referred to technologies, products, or solutions that might be used to address particular problems (e.g. incontinence). While all of those mentioned in the source text are available in the Portuguese context (which might not be the case for certain countries), subtle differences were found regarding their use in care provision according to current local practices. Accommodating those differences never required the replacement of the presented solutions, but further detail was added in the target text to describe some of them. For example, while the source text refers to “commercial stir-in thickeners”, a more common recommendation in the local context would be “flavourless commercial stir-in thickeners” as a rejection of thickeners with flavour by the people with dementia is a common problem. Another example is the reference in the source text to “incontinence pads” with no further specification, while incontinence panties are currently the most recommended type of pad in the Portuguese context. Additions to the description of these solutions were mostly derived from the expert panel appraisal.
Cultural habits, customs, and traditions
A significant part of the adaptation work concerned the cultural equivalence of the Portuguese version, i.e. in adapting the original contents to capture the experience of daily life in the local culture. The need to find equivalents for cultural experiences, such as habits, customs, and traditions, was identified for several content categories including leisure activities, food, daily routines, customary practices, customary expressions, and religious practices. Those contents were adapted on the basis of cultural knowledge of the research team, and for specific topics, empirical studies carried with the Portuguese population were also used. This was the case for selecting the most popular leisure activities in the local context to replace those proposed in the source text (Instituto Nacional de Saúde Doutor Ricardo Jorge, 2017; Martins, 2010; Perista et al., 2016). For instance, activities such as “Watch birds or animals”, “Watch the clouds or explore nature”, or “Play a musical instrument” (source text) are not typical leisure activities performed by Portuguese middle-aged or older adults. Therefore, those were replaced with popular and culturally adequate activities such as “Have friends over to visit or visit friends”, “Go out to eat”, “Watch TV”, or “Shop for yourselves or others” (target text).
As gastronomy is deeply embedded in a country’s culture, references to food were adapted when needed. For instance, “having/inviting someone to have a cup of tea” from the British culture was replaced by “having/inviting someone to have a coffee” since in the Portuguese culture inviting someone to “take a coffee” is a customary expression for meeting someone to socialise. Other examples entailed the replacement of products/foods referred in the source text originating from the oriental culture, such as “noodles”, “rice cakes”, or “soy sauce”, products hardly found in a Portuguese kitchen/pantry, especially to be consumed by older generations.
The reference to cultural habits and daily routines also required, in some cases, adaptation work: for example, in the source text, the sentence “he/she may need someone to put out ingredients” (for preparing the breakfast) does make sense in a cultural context where breakfast is cooked, but it does not apply to Portuguese habits with a common breakfast including milk, coffee or tea, and bread. Other routines mentioned in the source text, such as “having dinner around 6 pm” do not fit the local context, where a common work schedule finishes at 7 pm and as a consequence dinner time is about 1–2 hours later.
Expressions that appeared in the source text referring to the mother-in-law as “mother” or saying that someone is “in his late 70 s” are not customary in the Portuguese culture, where “mother” is usually reserved to the person’s actual mother, and rather than the previous, the next age landmark is used (e.g. someone is “almost 80 years old”).
In addition, references to religious groups, practices, or habits were also adapted (e.g. resorting to a faith community for support; saying prayers before going to sleep; pray or meditate with the people with dementia). While approximately 85% of the Portuguese population reported being Roman Catholic, more than half claim not to be active participants of the church/faith community or to pray regularly (Luijkx et al., 2017). References to religious groups or practices were not eliminated in the target text but were given less relevance by using them as examples rather than highlighting them in the text, or by including disclaiming remarks such as “if this is a person’s habit and belief”.
Expert panel appraisal
Qualitative and quantitative appraisal
Most of the comments and suggestions that emerged from the qualitative appraisal carried out by the panellists were immediately accepted for proposal to WHO when concerning: (a) modifications of initial translation choices, including replacement of words by synonyms or minor sentences reformulations that did not alter in any way the meaning of the text and that were related with stylistic preferences or comprehensibility improvements and (b) additions to the adaptation work of contents similar in nature to those referred in the adaptation guide (cf. Cultural adaptation section).
In addition to the qualitative appraisal, the experts performed a quantitative evaluation of each lesson, covering five criteria (cf. Methods section). Table 1 presents the median scores per criteria and lesson, and the overall median scores per lesson.
Expert panel appraisal: quantitative appraisal for each lesson of iSupport European-Portuguese version, across five parameters.
Note: Scores are given from 1 (lesson completely miss the parameter) to 4 (lesson fully contains the parameter). M = Module; L = Lesson.
Overall, no scores of 1 indicating the absence of the analysed quality were given to any lesson, by any expert, in the five criteria. The cultural adaptation work carried out by the research team was well rated by the panel. The contents subjected to cultural adaptation (phase 1) were considered “adequate, no need for revision” (Mdn = 4) for most lessons (n = 20). Three lessons were considered to be “adequate, in need for minor revisions” by two (Mdn = 3.5) or more experts (Mdn = 3) (cf. Table 1). Minor revisions concerned the adaptation of solutions to the local nomenclature and practices (e.g. the word “pad” refers not only the Portuguese equivalent to multi-layered sheet pads but also to another kind of pads, such as panties). While scores for the cultural adequacy of the adapted contents can be directly attributed to the adaptation work, scores given to the remaining four criteria do not necessarily mirror that work, i.e. scoring a lesson with 2 or lower, for instance, on the comprehensibility of contents, can be either attributable to less adequate translation/adaptation choices or to issues encountered in the original text. For that reason, the scores were crossed with the qualitative appraisal, and clarifications were requested when needed to discern among cases. For the entire programme, only a couple of suggestions for revision derived from translation/adaptation choices which were influential in the appraisal of the clarity and comprehensibility criteria.
Specifically, the language “sensitivity” was rated well for most lessons (Mdn = 4; n = 16), and seven lessons were appraised as “adequate, in need for minor revisions” (Mdn = 3 to 3.5) (cf. Table 1). All comments identifying revision needs for improving sensitivity were related to expressions used in the source text, and suggestions were made to use hedged language, e.g. (he/she) “is unable to” (source text) to (he/she) “shows difficulty in” (target text); or “this is not correct!” (source text) to “this is not a suitable answer!” (target text).
Similarly, the “clarity and comprehensibility” of contents, as well as the “familiarity” of terms/expressions, were well rated by the panellists in most lessons. A median of four (Mdn = 4) was obtained for 14 lessons regarding the first criteria and 18 lessons regarding the second. It was considered that minor revisions would be needed in nine lessons regarding comprehensibility and in five lessons regarding familiarity (Mdn = 3 to 3.5) (cf. Table 1). Most of the revisions identified as needed to improve either comprehensibility or familiarity were related to expressions used in the source text. Recommendations included: (i) replacing non-specific words such as “thing(s)” (e.g. “do the things he is still able to do”) by words naming tasks/objects or other context-specific words; (ii) introducing a few linguistic amplifications (Molina & Hurtado Albir, 2004) by means of providing examples to clarify potentially unknown expressions (e.g. replace “change from open-ended questions to closed-ended questions” by “change from open-ended questions (What have you done today?) to closed-ended questions (Did you have tea with Maria?)”); (iii) rephrasing sentences to improve their consistency with the message to convey in the text (e.g. in the lesson “shared decision-making” replace the sentences “Prepare instructions” or “Make sure you plan well for the future” by “Help preparing instructions” and “Make sure you plan well for the future together with the person”, in order to stress the shared nature of the process); (iv) eliminating the use of jargon (e.g. “This is often referred to as “sundowning”); and (v) clarifying the explanation of concepts (e.g. replace “Unreal thoughts and seeing things that aren’t there (delusions and hallucinations)” by “when the person keeps fixed false ideas or beliefs about the reality (delusions) or sees, hears, smells, tastes or feels thing that aren’t there (hallucinations)”).
Finally, in what concerns the “precision” criteria, most lessons (n = 14) were considered to be in need of minor revisions (Mdn = 3 to 3.5), while no need for revision was seen for eight lessons (Mdn = 4) (cf. Table 1). The first lesson on module five was the only one obtaining a median rate below 3 (Mdn = 2.5) for the precision criteria, which relates to theoretical and interventional discussions presented below. Most comments suggesting the improvement of concept precision or scenario presentations according to cultural lenses stemmed from the source text rather than from translation/adaption choices. Examples include: (i) sentences containing two or more concepts, but not clearly distinguishing among them (e.g. “Delusions and hallucinations: things a person sees or hears that are not there”, where only hallucinations are defined); (ii) unstraightforward examples illustrating the concepts (e.g. “Challenging behaviour: Kyla cannot remember who came to visit”, where not remembering might or might not be framed as a challenging behaviour according to theoretical point of views); and (iii) scenarios elements that can be interpreted as culturally inappropriate (e.g. elements suggestive of gender imbalance). As some comments made to the precision quality required more comprehensive changes, those were further debated in a consensus and adaptation validation round.
Consensus and adaptation validation round
One round per module was conducted to solve disagreements expressed by divergent comments along the texts from the experts. The rounds also allowed to discuss the suggestions of more in-depth changes (for instance, in scenarios) or theoretical and interventional issues raised by one or more experts. The purpose of expanding the typical goal of disagreement resolution to consensus rounds was to provide insights on the extent to which experts who did not suggest more in-depth changes or did not raise theoretical and interventional concerns would consider culturally adapted scenarios and/or culturally framed theoretical approaches and intervention practices as relevant to achieve in the Portuguese version. All issues taken to further discussion were solved in one round.
Only a few cases were related to disagreements: examples include deciding whether the expression “person living with dementia/(someone) lives with dementia” or “person with dementia/(someone) has dementia” should be adopted; and whether using the term “condition” or “disease” was more appropriate. These expressions were used interchangeably in the source text, whereas the second options were adopted in the target text by agreement of all experts. Other topics taken to discussion included changes in scenario descriptions (e.g. reformulations to minimise suggestions of gender imbalance) as well as in answer options for iterative exercises (e.g. replace the wrong answer “They should say to Feng that (…) if he continues to behave in this way, he will be sent to a long-term care facility” (source text) by “They should say to Feng that (…) if he continues to behave in this way, their sons-in-law will be angry with him” (target text)). Recommendations for changes having theoretical and interventional implications included, for instance, the case of making decisions in the best interest of the person with dementia, where the sentence (consider) “any relevant statement written when he/she did not have dementia (source text) was suggested to be replaced by “any relevant statement written when he/she was still able to make decisions”. In spite of their complexity, a consensus was easily reached about these and other similar issues, as most were raised by two or three experts.
Theoretical and interventional discussions
As stressed by Card et al., 2011, core elements of evidence-based interventions should not be changed, unless there is convincing countervailing evidence from a stage of information gathering in the adaptation process. Nevertheless, it is not always straightforward to discern which elements of an intervention are core or discretionary (Card et al., 2011). For the Portuguese adaptation of iSupport, theoretical and interventional discussions that emerged from the expert panel appraisal were translated into specific adaptation recommendations, documented in the adaptation forms, and subjected to a fidelity check, where the issue of whether adaptations were core or discretionary is addressed. For illustrative purposes, some of those discussions are described below.
The first issue that was raised and that had a consequential influence on the presentation of the programme lessons, especially on module 5 – Dealing with challenging behaviours, was related to whether the expression “challenging behaviours” should be employed as an umbrella for a set of symptoms and common issues in dementia cases. In the generic version of the programme, this is the case for memory loss, aggression, depression or anxiety, difficulty sleeping, delusions and hallucinations, repetitive behaviour, wandering, and poor or decreased judgement. The underlying theoretical question relies on what can be framed as “challenging behaviour”. A challenging behaviour can be defined as broadly as a “behaviour of such an intensity, frequency or duration as to threaten the quality of life and/or the physical safety of the individual or others and is likely to lead to responses that are restrictive, aversive or result in exclusion” (Royal College of Psychiatrists, 2007). The same behaviour, however, may be considered challenging in certain contexts or cultures but not in others, and professionals/researchers can also disagree about whether a behaviour is challenging. For instance, the National Institute for Health and Care Excellence refers to challenging behaviours as those including aggressive, destructive, disruptive, self-injurious, and sexually harmful behaviour (National Institute for Clinical Excellence (NICE), 2015). The institute stresses, for example, that while sleep difficulties of the care receiver can be very challenging for caregivers, it is not the poor sleep that is challenging, but the behaviour occurring when the person would normally be asleep (NICE, 2015). The Portuguese expert panel showed to be more aligned with this strict understanding of the concept, which excludes, for example, difficulty sleeping or memory loss, this last understood as a cognitive symptom. In alternative, the expressions “behaviour change(s)” or “changed behaviour(s)” were proposed for the Portuguese version. The debate also focused on whether forgetting something is actually a behaviour (e.g. “Challenging behaviour: Kayla cannot remember who came to visit”; source text). A broader interpretation of behaviour as “an organism’s activities in response to external or internal stimuli, including objectively observable activities, introspectively observable activities, and nonconscious processes” as opposing to a strict interpretation as “any action or function that can be objectively observed or measured in response to controlled stimuli” (American Psychological Association, 2018) appears to be at stake.
An additional theoretical discussion raised by the panellists concerned the issue of advance directives covered in the programme (Module 2), in particular, the statement that relevant written declarations from “when he/she (the person with dementia) did not have dementia” (source text) should be taken into account. It was the panel’s understanding that first, establishing with enough certainty when the person developed the first symptoms of dementia is tricky as those are frequently confused with normal ageing; and second, it is also questionable to use the diagnosis of dementia as a landmark, especially when the mean duration from the onset of dementia-related symptoms until the diagnosis may range from one (Wilkinson et al., 2004) to three years (Speechly et al., 2009). Even though this is a controversial issue, the experts argued that it seemed more reasonable and aligned with the local approach to assume a perspective that advance directives should be completed while the person still has “legal capacity”, which is not conflicting with the disease having been already detected. For the Portuguese version of the program, the original statement was suggested to be replaced by (consider any relevant statements written) “when he/she was still able to make decisions”.
Further theoretical considerations concerned the definition of concepts like hallucinations as “things a person sees or hears that are not there” (source text); or wandering as “walking around a lot” (source text). On the first case, it was considered that in spite of being well known that auditory and visual hallucinations are the most common, the definition omits that hallucinations may affect any of the senses; while the second definition was considered little informative, being proposed to be replaced by “seemingly aimless ambulation” (target text).
Concerning interventional considerations, i.e. to what extent intervention practices described in the programme are aligned with common practices in the local context, the panel raised minor questions on specific recommendations made throughout the lessons. Adjustments proposed consisted mostly in adding disclaimers on the original recommendations. For instance, when a timetable is used to remind the people with dementia to drink throughout the day, add statements for discouraging drinking after 6 pm; when flexibility is recommended about mealtimes in case the people with dementia needs/wants to rest, add a statement on the need to guarantee that the person does not spend too many hours without eating; or when it is recommended that artificial dentures are cleaned when needed, prefer the alternative of doing it on a regular/daily basis, in line with local standards.
Finally, particular elements from scenarios were discussed. For instance, a couple of scenarios were viewed as suggesting gender imbalance, since they depict care responsibilities (and consequences) as relying on women who are relatives-in-law of the people with dementia (e.g. daughter-in-law), when a blood relative (son) or a spouse (husband) is present and available. While this is plausible to happen in the Portuguese culture, the message conveyed was considered undesirable and affecting the acceptability by the target group. In another scenario, trying to convey the message that a person with dementia should never be threatened, the illustrative menace is sending the person to a long-term care facility. Again, in spite of being a plausible situation in the local context, it may inadvertently support negative stereotypes linked to institutional care. For each theoretical and interventional discussion raised, concrete alternatives were proposed and later appreciated by the programme authors.
Fidelity check
From the fidelity check process, most adaptations proposed for the Portuguese version of iSupport were considered accurate and in line with the generic version of the programme. The original version was recommended to be kept in 11 text elements proposed to adaptation, as reformulations were considered more complex or conveyed a slightly different meaning of the original text.
Discussion and conclusions
This paper describes the first steps taken to culturally adapt the WHO iSupport programme for informal caregivers of people with dementia to the Portuguese context. Five major steps were undertaken to adapt the programme contents, and the results unveiled the complexity of translating and adapting an evidence-based intervention from its original version to a specific cultural setting. This process requires considerable time and resources allocation, keen attention to the nuances of language and culture, and the establishment of suitable partnerships and consultancy relationships.
Even though the translation step is often underestimated and seen as a surface structure change, it is critically important for cultural adaptations. When approached in a “sense to sense”, rather than in a “word to word” (literal) perspective, the translation implies an understanding of idioms, grammar and conventions in the cultural context of the source language. Accordingly, discerning which contents are the result of actual adaptation work and which are a product of a culturally aware translation is challenging. That is the case, for instance, of semantic and conceptual equivalences. The involvement and iteration of several actors were required to achieve a culturally fit translation of the programme. The process involved a specialised translator, professionals with different backgrounds and experts on the fields of ageing, dementia, and psychosocial or psychological support for caregivers of people with dementia.
Globally, the Portuguese version of iSupport remains truthful to the key elements of the original programme, but it required that the following elements were modified: words and expressions for semantic and conceptual equivalence or for reading-level adjustment to the target group; personal names of characters; resources; and references to cultural habits, customs, and traditions. Additional proposals for adaptation concerning the presentation of particular concepts and recommendations for practice resulted from theoretical and interventional considerations made by the expert panel. Resorting to this panel was instrumental as more than providing insights for refining the translated version, the panel could judge the suitability of culturally adapted contents. Moreover, the experts’ involvement resulted in a more in-depth analysis of the original contents of the programme in terms of how they adapt to the knowledge, theoretical approaches, and practices existing in the Portuguese context. Indeed, the concept of culture goes beyond national, racial, ethnic, or religious affiliation, also including beliefs and practices in health (Napier et al., 2014). Culture is key to the practices of health professions and organisations, and professions usually develop cultures and micro cultures with idiosyncratic patterns of thought and practice (Brives et al., 2016). Therefore, when consulting Portuguese stakeholders, the emergence of slight variations on the perspectives about the concepts (e.g. strict vs. broad interpretations) and practices within iSupport is not surprising, but rather an evidence of the fact that cultural adaptation is a fine-grained process. To that end, the fidelity check by the authors of the programme was crucial to decide which changes to keep or exclude in order to maintain the fidelity of the intervention post-adaptation, as a key challenge for culturally adapting interventions is precisely the balance between cultural fit and fidelity. Importantly, this process resulted to be bidirectional as the experience of adapting the programme to Portugal, as well as to other countries, contributed to improve the generic version of the programme, which was updated latter.
A limitation of the adaptation process described in this paper is the adoption, to a certain extent, of a top-down approach, where the research team proposed cultural modifications to the programme based on observable social and behavioural characteristics of the target group, on available evidence, and on the consultancy of professionals and experts in the field. In spite of the participation of key stakeholders, we consider that the involvement of informal caregivers of people with dementia, the programme’s end-users, would be required to claim for a bottom-up approach. To address this gap, the next steps for the Portuguese version of iSupport include running usability tests to assess the interface efficiency and effectiveness as well as user satisfaction. Moreover, a pilot study will be carried to determine the acceptability of iSupport’s adaptation in the Portuguese setting. Concerning usability, a few studies on ‘culturability’ have been reporting cultural differences on how usability is assessed by users (e.g. Clemmensen, 2009; Frandsen-Thorlacius et al., 2009). As usability is a multidimensional concept composed of different attributes (e.g. efficiency, effectiveness, satisfaction), it might be the case of those not being equally valued across cultures. For instance, Frandsen-Thorlacius et al. (2009), while comparing Chinese and Danish individuals with respect to usability preferences, concluded that the first preferred ease of use over effectiveness, while the former preferred the opposite. To the authors’ knowledge, there are no similar insights on how Portuguese individuals value some usability attributes over others, nor in comparison to other cultures. In this study, the concept of universal design or design for all, i.e. a design that suits all persons, was adopted in the adaptation process. For instance, reading-level adjustments of contents were made (cf. Results section) so that contents can be accessible to individuals with lower education levels. Indeed, a design-for-all philosophy was adopted, even if the authors presume that iSupport will be more used, at least in the short-term, by caregivers with higher education levels, as education is a well-known strong determinant of digital literacy and internet usage (Goldfarb & Prince, 2008). This means that accounting for the low-education levels of informal Portuguese caregivers, iSupport will most probably be used by a segment of those caregivers. However, internet usage trends have been evolving rapidly in a positive direction, and governmental efforts have been made to bridge age and education gaps at this respect. For instance, internet use in the Portuguese population has grown rapidly and substantially, doubling in less than 10 years, with 64% of all individuals using it in a daily basis according to the last available data (Instituto Nacional de Estatística, 2018). This favourable trend reinforces the approach of thinking sustainably and universally to adapt iSupport.
Future research with iSupport also includes a subsequent intervention study to determine the efficacy of the Portuguese version of iSupport in decreasing caregivers’ burden as well as symptoms of depression and anxiety (Teles et al., 2020). In what concerns the effectiveness of culturally adapted vs. non-adapted interventions on clinical outcomes, the evidence from several systematic reviews and meta-analysis is mixed (Benish et al., 2011; Cabral & Smith, 2011; Chowdhary et al., 2014; Cuijpers et al., 2008; Griner & Smith, 2006; Hall et al., 2016; Hodge et al., 2010; Huey & Polo, 2008; van Loon et al., 2013), with effect sizes ranging from almost zero (e.g. Huey & Polo, 2008) to large effect sizes favouring culturally adapted interventions (e.g. Chowdhary et al., 2014; van Loon et al., 2013). The extent to which the cultural adaptation of evidence-based interventions, such as the one described in this paper, increases the effectiveness and also the usability of those interventions is not fully known and requires testing in each particular setting. From this experience, however, the authors conclude that tailoring an intervention programme is critical to accommodate the cultural experiences of those seeking to benefit from it. This paper adds to the literature, which is scarce in accounts of cultural adaptation of intervention programmes in this field, particularly providing this level of detail. Also, lessons learned throughout the process of adapting iSupport to the Portuguese context provide value for others who might undertake similar efforts in the future. The Portuguese experience in adapting iSupport is particularly relevant because it is one of the first to materialise WHO’s aim of disseminating the programme. Since dementia is a public health priority, and support to caregivers of people with dementia is a major issue on current health care agendas, it is important to successfully replicate iSupport in other countries.
Footnotes
Acknowledgements
The authors thank WHO by conceding the licence to use, reproduce, publish, and display the iSupport programme. The Portuguese version of iSupport was copied from the online iSupport for Dementia programme
, Version 1.0, World Health Organization, Copyright (2018). Adapted and translated with permission of WHO. The authors thank the expert panel and the national Alzheimer Association (Associação Alzheimer Portugal) for the support given to the cultural adaptation. Likewise, the authors thank Professor Altamiro da Costa Pereira for enabling the use of resources from the R&D Unit CINTESIS to carry out this work.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The authors declare that they have no conflicts of interest. Katrin Seeher holds a position as Technical Officer in the Department of Mental Health and Substance Use (MSD) of World Health Organization (WHO). Neither the author nor WHO have a financial interest in the subject matter discussed in this manuscript.
Ethical approval
This study was approved by the Ethics Committee for Health of the São João University Hospital Center/Faculty of Medicine of the University of Porto (reference 208/18).
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Soraia Teles holds a grant from the Portuguese Foundation for Science and Technology (FCT) (D/BD/135496/2018; PhD Programme in Clinical and Health Services Research). Ana Ferreira was supported by FCT through the Project TagUBig – Taming Your Big Data (IF/00693/2015) from Researcher FCT Programme funded by National Funds through FCT – Fundação para a Ciência e a Tecnologia. The funding sources had no involvement in the research and the funding entity has no financial interest in the subject matter discussed in this manuscript.
).
).
