Assessing the societal value of a service-learning project in information studies during the COVID-19 pandemic

Abstract

The purpose of this work is to assess the societal value of a Service-Learning (SL) project carried out during the Covid-19 pandemic by the Faculty of Information Science of the Complutense University of Madrid (UCM) in collaboration with two Senior Centers of the City of Madrid. The aim of the project was to support elderly’s integration in the online activities carried out during the Covid-19 pandemic and to train them in the use of Information and Communication Technologies (ICT). The analysis of societal value is based on a case study and a varied range of data whose purpose is to provide multiple insights into the experience, emphasizing communicative processes. The results corroborate the educational value of experiential learning for students, although the impact on the community appears limited by the role of consumers of a service that participating elderly ended up playing. The evaluation of the project by the faculty leading the activities was corroborated by the institutional partner and provides evidence of the capacity for societal transformation of higher education institutions.

Keywords

Service-learning Library and Information Science societal value higher education institutions elderly Information and Communication Technologies (ICT)COVID-19

1. Introduction

1.1 Service-Learning in Library and Information Science

Service-Learning (SL) is a type of experiential education in which students engage with real-world problems, providing a service to the community (Riddle, 2003). As a pedagogical tool, in addition to its experiential character, it implies reciprocity, collaboration and mutual learning between students and communities, promotes civic education and the development of a sense of societal responsibility (Lim & Bloomquist, 2015), and requires reflecting on the experiences that it affords (Angel, 2016). In Library and Information Science (LIS), SL has been used as a teaching strategy in graduate and postgraduate education and as an alternative to curricular internships (Most, 2011; Roy et al., 2009; Montesi et al., 2021), and, according to Scott (2020), SL could be found in the first models of LIS education. SL initiatives have also been undertaken from academic libraries in collaboration with faculty, in order to accomplish libraries’ mission of supporting the development of university curricula (Nutefall, 2016; Scripps-Hoekstra, 2020). According to Caspe and Lopez’s survey (2018), experiential and connected learning methodologies provide students with knowledge about families and communities, develop relationship building skills and encourage a mindset of professionalism. On the other hand, some authors invite to look at experiential learning methodologies beyond SL, criticizing the unidirectional character of SL and its emphasis on the dysfunctionalities of the communities that are often deprived of their own agency and capacity of reflection (Poole, 2021). In LIS, the alternatives range from community engagement to Asset-Based Community Development, whose aim is to stress the resources and strengths of communities (Stevenson, 2020).

1.2 The different dimensions of SL impact

Much research on SL has assessed the impact of community engagement on students, leaving aside the impact on institutions, faculty and especially communities (James & Logan, 2016). The impact of SL on communities is apparently the least studied dimension (Gelmon et al., 2018; Mironesco, 2018), although little attention has also been paid to the role of institutions in articulating the guiding values of SL interventions (Chupp & Joseph, 2010). However, SL settles on civic engagement and a complex social network, which brings together universities, faculty, research, students, professionals, and communities, and thus its potential for impact goes beyond the purely educational dimension (Thompson & Hood, 2017). The difficulties of assessing the impact on the community derive in part from the lack of definition of the very concept of community (Gelmon et al., 2018). If Frank and Sieh (2016: 514) understand community as “a group of people with direct and immediate interest in a particular place, and who have been identified with that place”, James and Logan (2016) support a concept of community as a network of individuals. From the point of view of Ngui (2020), communities can be defined by geographical proximity, interaction and relationships between individuals, and identity or unity in the pursuit of a common goal. On the other hand, Gelmon et al. (2018) consider that it may be a mistake to conceive of the community as a unitary and definable notion and suggest attending instead to the processes of collaboration among community members.

The scant prior research on the impact on communities has addressed mainly the institutional partners’ willingness to participate again in SL projects, their satisfaction with the work performed by students, and the benefits for the organization. In the follow-up report of the 27 SL courses held at the Virginia Commonwealth University in 2017 (Jettner et al., 2017), partners considered that SL made its greatest contribution by enhancing organizational capacity, though it also brought social benefits, increasing social connections, and finally economic benefits. The qualitative and dialogical nature of the evaluation revealed issues that are difficult to measure, such as the importance of relationships and collaboration, as well as the intangible results derived from these initiatives, especially at the social level. The intangible nature of SL impact is also pointed out by James and Logan (2016), when they report the practical absence of references to the economic dimension of the impact in the project that they carried out with a secondary school. Mironesco (2018) defends intangibility when assessing SL impact, because civic engagement requires developing a person’s ability to understand her community and her role within it, and proposes evaluation strategies that, without neglecting indicators and quantitative measurements, seek to honestly collect the voices of the participating communities. SL returns for higher education institutions are just as intangible, and SL initiatives foster trust and credibility in local communities whilst enabling universities to act as transformative agents (Thompson & Hood, 2017). The intangible dimension of SL impact on communities highlights the limitations of Gelmon et al. (2018) triple assessment scheme for partners (organizational capacity, economic benefits, and social benefits), since it restricts the possibilities of impact to these three dimensions, without taking into account any participatory mechanism of evaluation. Finally, according to Mtawa (2019), the emphasis on instrumental and tangible results over societal and community values in the evaluation processes of SL leads to an epistemological injustice, a fact that would limit its capacity for transformation and to reduce inequalities.

1.3 SL in the framework of institutional activities

In this paper, we understand that SL assessment should be contextualized within the framework of institutional activities and should take advantage of research aimed at measuring the societal impact of institutional activities. Impact measurement is a necessary strategy to demonstrate the societal and scientific legitimacy of all actions aimed at collaboration between universities and communities, including SL initiatives (Trencher et al., 2015). Typically, the classic assessment of higher education and research institutions activities leads to university rankings that fundamentally take into account the capacity of production of scientific knowledge in journals indexed in WoS or Scopus and, additionally, in some cases, the quality of teaching or capacity for technology transfer (Fauzi et al., 2020; Fernández-Cano et al., 2018; Johnes, 2018). From the societal point of view, institutional rankings completely disregard the impact of higher education institutions in dimensions such as culture or social welfare (Daraio & Bonaccorsi, 2017). The question of the societal impact of scientific activity arises as a consequence of an enlarged system of scientific communication that admits not only purely academic actors, but also political and economic actors and citizens (Tuunainen & Kantasalmi, 2017). Even if the interest in societal impact was initially conceived as a strategy to assess the returns of government R&D investments (Miettinen et al., 2015), it also covers the societal benefits of research in social, cultural, and environmental dimensions and also sustainability (Bornmann, 2013). Smit and Hessels (2021) prefer to use the concept of societal value instead of the societal impact, in order to more easily accommodate non-tangible results. According to Trencher et al. (2013), there is still a very strong economic conception of what is understood as the societal mission of the university. Assessing impact from an economic and quantitative perspective causes such problems as causation and attribution, as it is difficult to detect cause-and-effect relationships and attribute certain results to specific actors or interventions (Temple et al., 2018). Alternatively, Temple et al. (2018) propose to move from the notion of attribution to the notion of contribution, underlining the participation of different stakeholders in innovation processes. The most popular evaluation strategy of societal impact are case studies (Tahamtan & Bornmann, 2020). According to Lauronen (2020), case studies as other impact assessment methods, cannot easily address sociopolitical aims due a problem known in anthropology as liminality, referring to the different interpretations that the academic community and political and social actors can give of the notion of impact. On the other hand, Heyeres et al.’s (2019) review of case studies as impact assessment tools reveals that little attention is usually given to the needs of target populations and to the economic value of research, claiming major transparency in the way in which resources are used. Finally, the 10 assessment methods that Smit and Hessels (2021) gather differ in the very concept of societal value, and especially in the types and roles of supporting actors in knowledge production processes and in the mechanisms of interaction that promote the transfer of knowledge. With its limitations, case studies have the advantage of adjusting to the great diversity of scenarios and social actors that can intervene in generating impact, even in the same field of knowledge (Janker & Mann, 2018). In the evaluation of the societal impact of higher education institutions, Montesi and Villaseñor Rodríguez (2018) bet on sufficiently indeterminate schemes capable of adjusting to the specificities of each university and propose a framework articulated in four the dimensions: 1) people and groups, 2) agreements and relationships, 3) events, interaction, and engagement, and 4) processes. As for the essential elements in the processes of societal evaluation, apart from the environments and the people, groups and actors involved, Wolf et al. (2013) suggest also highlighting the disciplines and specialties involved, as well as the modalities of information exchange, with emphasis on communicative processes.

1.4 Case study objective

The aim of this paper is to evaluate the results of a SL project carried out in collaboration between the Faculty of Information Science of the Complutense University of Madrid and two Senior Centers of the City of Madrid, both depending on the same management team. The inter-institutional collaboration exists since the 2017–18 academic year (Montesi et al., 2019; Cristóbal Querol et al., 2020; Montesi et al., 2021), with educational and societal purposes, and in 2020–21 it was adapted to the extraordinary circumstances of the Covid-19 pandemic. The situation of elderly after the first months of the pandemic was clearly exposed in a United Nations report published in May 2020 (United Nations, 2020), which pointed not only to the highest mortality rates for this population group, but also to the consequences of distancing measures on mental health, neglect and abuse, and the trauma of stigmas and discrimination. On the other hand, elderly presented a specific information behavior during the pandemic, preferring as information sources radio, television and personal communication and being in many cases in situations of digital exclusion (de Maio Nascimento, 2020), whilst community connections among generations have been proposed among the other actions aimed at promoting the inclusion and integration of the elderly (Pentaris et al., 2020). Madrid Senior Centers pursue active ageing and lifelong learning, supporting social relationships and the establishment of interpersonal connections (Madrid City Council, 2021). However, their face-to-face activities were interrupted in March 2020 and in June 2021 they had still not resumed. In collaboration with the management team of the two senior centers, a program of telephone calls was launched, and the students of the degree in Information Science of the UCM contacted by telephone a sample of preselected elderly whose contact details had been provided by the Senior Centers. Their purpose was to provide technical support and training with Information and Communication Technologies (ICT) and/or moral support, asking about their situation and collecting information about the impact of the pandemic on the daily life of these people.

In previous editions of the project, it had been impossible to collect data on societal outreach, among the other things, due to difficulties in engaging the elderly in the assessment process (Montesi et al., 2021). However, this edition has emphasized precisely the “societal value” of the project, as it is conceived in (Smit & Hessels, 2021), encompassing intangible results not necessarily embodied in behavioral or practical changes. According to the literature mentioned in the introduction and with the intention of emphasizing intangibility, the collaborative character and the liminality of societal value, we have designed a case study that allows to see the project from different perspectives and contributions. In this way, we hope to obtain a multidimensional vision of the project societal value, giving special attention to communicative processes, the interactions between the various participants and their different points of view.

2. Methodology

The case study was chosen as the research methodology, following McDonough et al. (2017) model and because it allows to adjust the evaluation to the specific characteristics of the project and reflect the different perspectives of all participants. An intrinsic difficulty in the evaluation of societal value concerns the way different stakeholders may be integrated in the evaluation processes. The modalities of intervention and the channels of representation of broad groups may not be clear, while there may be no common and shared conception of the key concepts under evaluation (Faure et al., 2020). Thus, we opted for the case study in order to integrate the different perspectives of all the participants in the project, emphasizing the communicative processes between them and exploiting different sources of data. Firstly, from the point of view of students, we analyze the 157 emails exchanged with them during the development of the project between October 2020 and March 2021, and 10 reflective reports that the participating students handed in to reflect on the experience, published in (Montesi et al., forthcoming). The emails allowed to reconstruct the history of the project and have been used to check the information of other sources, whereas the literal extracts reproduced in the results come from the 10 reports and are included without explicit reference to the authors, although they can be verified in the aforementioned monograph (Montesi et al., forthcoming). Nine additional reports were included as a control group, corresponding to students who participated in the project but carried out activities that did not require direct interaction with elderly. These two corpora of students’ reports were compared with Lingmotif (https://ltl.uma.es/), a sentiment analysis tool, that returns for each text analyzed a sentiment score, sentiment intensity, as well as the number of positive, negative, and neutral sentences. Lingmotif’s main features include a shifter system that accounts for valence modification by context, being in this way adequate for a varied range of texts (Moreno-Ortiz & Pérez-Hernández, 2018). Statistical differences between the two groups in terms of Sentiment Score, Sentiment Intensity and percentage of neutral sentences were calculated with IBM SPSS Statistics 25.

Secondly, the perspective of the Senior Center counted with an additional set of data, including the 88 emails exchanged between faculty and the Center itself in the period August 2020 to March 2021, which reflected and summarized much of what was discussed and agreed in the 20 face-to-face meetings and telephone interactions. The 20 meetings with the Senior Center include a final meeting whose purpose was to assess project results and that was held on June 4 ${}^{\text{th}}$ , 2021, with the attendance of two faculty members, the Senior Centers director, and a social worker, lasting about one hour. This last meeting was the only one to be recorded.

Table 1
Summary of data handled in the project

Data collected	Frequency
Emails exchanged with students	157
Reflective reports (telephonic tutoring)	10
Global word count of the reflective reports (telephonic tutoring)	17678
Control Group reports	9
Emails exchanged with the Senior Center	88
Telephone and face-to-face meetings with the Senior Center	18
Follow-up interviews with elderly	62

Finally, the point of view of the elderly was gathered through brief telephone interviews conducted by the Center’s staff according to a semi-structured set of questions agreed on with the faculty. The interviews were carried out between March and April 2021 and were immediately transcribed or summarized. Both the interviews with the elderly and the reflective reports of the students have been analyzed, highlighting the most outstanding topics, and always comparing one with another and with other sources of data. Excerpts in the text are authors’ translations and the original texts are appended in an annex to the paper.

3. Results

3.1 Global project data

According to Table 2, 89 elderlies were selected by the Senior Center to take part in the project, being the vast majority of them women (76.4%). While the Senior Center was responsible for selecting and notifying the participating elderly, the academic team of the project assigned each elderly to a student, though 8 of them could not be assigned. Twenty-two students opted for the telephone tutoring program on a voluntary basis, as alternatives were available when the activity was mandatory, representing 30.1% of all students participating in the project. This group also included students with some kind of disability or speaking Spanish as a second language.

Table 2
Summary of telephone tutorials

Participants	Frequency	%
Elderly selected by the Senior Center	89
Male elderly	21	23.6%
Female elderly	68	76.4%
Elderly assigned to students	81
Students participating in the project	73
Students participating in the telephone tutoring program	22	30.1%
Average number of elderly per student	3.7

Table 3

Tutorials data

Tutorials activity	Frequency	%
Total number of elderly selected by the Senior Center	89	100%
Elderly interviewed after the closure of the project	62	69.7%
Elderly that could not be reached after the closure of the project	27	30.3%
Contacts established by students	58	65.2%
Successful tutorials	20	22.5%
Cases of conflict between what students and elderly said	19

Data regarding the telephone tutoring program in Table 3 were obtained by comparing the information gathered from the interviews with the elderly, the emails with the students and their reflective reports. In Table 3, “established contacts” refer to those cases that counted on evidence of an attempt to get in touch, according to the emails exchanged with the students and/or the reflective reports, while, in a total of 19 cases, that are classified as “conflictive”, elderly denied having been contacted. According to communications with the Senior Center, these cases can be explained because of elderly’s cognitive problems. The cases of “successful tutoring” refer to all contacts that resulted in a positive experience for both parties (students and elderly), according once again to the interviews and the reflective reports.

3.2 Follow-up interviews with elderly

The most prominent theme emerging from the interviews with the elderly was that they conceived their participation in the project in terms of receptors of a service delivered by students, and, whilst this posture allowed some to reject the “service”, others expressed their discomfort with this role and demanded greater autonomy.

a. Refusal of students’ assistance

The follow-up interviews with the elderly confirm students’ attempts to contact, though many times the elderly reject the help offered with technology, for different reasons. Often, a relative (nephews, grandchildren, children, partner) has helped before:

“He called me, very kind. What happens is that my son had come a couple of days before and he had put it on Zoom in my tablet …” (1)

Other times, they are not interested in taking up online courses and long for face-to-face activities:

“… I don’t clear my mind up with this system. I haven’t connected because I’m not interested in online.” (2)

They can sometimes give up the tutoring because of physical, language or psychological difficulties:

“She called and I told her no, I wasn’t going to do it on the phone … for me it is complicated. I don’t have a computer, I know some French, but I have to see how people move their lips and I don’t use the Internet or the phone.” (3) “Well, she got in touch because I had signed up for English: but I told her I appreciated it very much, but I have a hearing problem and foreign languages … with the hearing aid … I don’t hear very well. So very friendly, but I didn’t need it.” (4) “Well, my daughter has been ill, and I’ve been disconnected from everything. I’m in the choir, in French, in taichi. I needed someone to help me with the internet, but I don’t remember if he called me. I am on medication …” (5)

In some cases, they may not be willing to invest in new e-devices.

“My computer is very old, I am at odds with new technologies, and I was not interested, and I am still not interested.” (6)

Also, they may ask for something more according to their personal needs, such as private lessons, that students, for time and schedule, cannot provide.

Sometimes, the rejection of tutorials sums up to a feeling of distrust, mediated by fear or the recommendations of relatives who seek to protect their elders and often are not aware of the project, and the participating elderly do not answer WhatsApp messages or hung up the phone.

“At first, I had two calls, but they were from a foreigner, and I didn’t understand it and hung up, but it was because of my fear … I got a WhatsApp and I deleted it, because I have a daughter who is a little hysterical and says: “Don’t have anyone on WhatsApp that you don’t know” … It was my fault, out of mistrust I deleted it.” (7)

In other cases, communication does not flow because of shame or embarrassment on the elderly’s part.

“They sent me a message, I read it and I forgot about it. When I read it again, I was embarrassed …” (8)

Elderly may also have forgotten the Senior Center’s communications about the project or the students’ calls.

“Well, I don’t remember … I don’t think anyone called. I can’t remember. I receive so many calls from doctors and such … If I doubt anything, I tell my daughter.” (9)

b. Service expectations and complaints

In addition to the frequent rejection of student’s attempts, another consequence of elderly conceiving of students’ calls as a service is that they feel entitled to complain about them, and they may criticize the “lack of precision of the questions” or the informality of the contact. Complaints also occur when students show certain difficulties in interpersonal communication, because questions are not well prepared, well presented, or expressed in a way appropriate enough for the target audience.

“He showed a lot of interest, but he got confused a lot. He mistook us with each other, he was a guy who didn’t get by very well talking to older people.” (10)

c. Elderly’s claims for autonomy

On the other hand, some elders claim more autonomy, make their capacities very clear and propose their participation in terms of collaboration, taking up an autonomous role different from that of the “consumer” of the service offered.

“Yes, I did an interview, she was very kind. I told her I’d take part as long as it wasn’t to research on me. She didn’t ask me very personal questions and well … I made myself available to collaborate.” (11) “I will cooperate if you explain to me who is going to read it … To talk about the pandemic and new technologies, I have friends. They’re not talking to Grandpa.” (12) “Yes, he called me, he helped me, all amazing … though I’m not very clumsy, I understand a lot of technology.” (13)

d. Students’ interpersonal skills

Complaints about students’ interpersonal skills highlight the importance that elderly attribute to these, as it is possible to appreciate when the elderly have positive feedback about students. In these cases, qualities such as empathy, affection or patience are highly valued.

“She asked me if I had cried. It was what impressed me the most. I’m not doing well but thanking God that I’m not sick.” (14) “Yes, it was very good, very nice, great. She taught me how to connect through Meet, and then she left me her phone number and then my kids helped me. The main base was her. Lovely, very nice with a lot of patience.” (15) “I enjoyed talking to him, affectionate, very polite … I can’t tell you more. It was useful to me and I thank you for it. Now I’m more focused, I take my walks …” (16)

When the tutorial takes place, elderly value what they highlighted in face-to-face workshops taught in previous editions: emotional support to get out of the problems encountered (Montesi et al., 2021).

“I was with a very kind boy: he got me out of the quagmire. I didn’t know how to get into Zoom.” (17) “He called me, amazing. I loved it. He explained to me. The problem was that my PC is from a long time ago and I had to update it … I couldn’t solve it. What I do through Meet, it’s on my phone, but I was satisfied. He left me his contact and helped me to lose my fear.” (18)

3.3 Students’ reports

The students’ reflective reports allow to see the impact of the project from the educational point of view, complementing the vision of the elderly.

a. Emotional roller coasters

A recurring theme of the reflective reports is the despondency many students feel at the beginning of the project, for the difficulties of getting in touch with unknown people, and, even more, for the rejection of many elderly after their attempts to contact. Following the first rejections, some students put the project on hold for long periods and in some cases tried to abandon it, as confirmed by the emails. Once they manage to get in touch and give their training, despondency gives way to enthusiasm and a feeling of success, producing an emotional “roller coaster” effect, as it was defined by one of the students.

“This whole experience, the truth is, it was a hard blow, but I kept hoping … I told all this to the teacher, who was also affected, since they were two “noes” in a row, although she passed me two new contacts, to see if this time I had more luck. Nothing could be further from the truth. I tried with the sixth contact that the teacher had provided me with and wrote a text explaining a new way of approaching this new tutorial. That message was also received and read, but not replied to. Again, the curse came back, all seemed lost until, at about 18:00, I received a call. After that interview, I felt better about myself, I had gotten someone who said “yes” and I had learned. The previous negative responses had been worth it with the experience I gained with the last one.” (19)

In this process, the ability to empathize becomes a tool to understand, turn the page and make sense of the rejection, inspiring appropriate communication strategies.

“After a couple of messages, I understood that that person could not or did not want to have the conversation, so I stopped insisting” (20) “With Juana Antonia, something similar happened to me as with Teresa, she was a little disoriented at the beginning of the conversation since she did not know why I had called her. To put her in place, I explained to her who I was, where I studied, how I had gotten her number, until she finally agreed to talk to me and tell me how she had passed this last year of COVID-19.” (21)

b. Supervision and autonomy

Supervision received from the project faculty team helps to overcome rejections and despondency, offering emotional support and instructions to face rejections and to engage in conversations with elderly, though students finally manage to make their own decisions autonomously.

“I tried to contact more people at the Senior Center, but there was no way to talk to them, they didn’t answer my messages. Faced with such a problem, I repeatedly asked my teacher, […], what I could do. To avoid that I would fall apart, she provided me with more people contact details, and she gave me a different mission, to talk to the elderly asking them about themselves, about how they had handled COVID-19. I thought this idea was a very good one, and I decided to keep contacting them. Seeing that on WhatsApp my messages were still unanswered, I decided to call them directly, and thus explain who I was and the reason for the call.” (22) “Our spirits regarding the project were decreasing, but, after a conversation with the teacher […], we were offered two more contacts to talk to.” (23)

The rejection of the elderly makes students question, encouraging self-criticism.

“Three, three people in a row denying my desire to teach and help people who have difficulties with technology. My family and friends were already desperate because nothing went well, since all this happened in the same week. I didn’t know if the problem was me or my way of explaining, but I was starting to despair about it.” (24)

However, other times, it is the elderly’ unconditional acceptance and lack of complaints that encourage self-criticism and the desire to improve.

“Another relevant aspect may be (or for us, it has been) the absence of complaints. At no time have we received one. We, as a group, after the sessions saw many aspects in which we could improve and we tried to do so in future meetings, but on the part of the attendees or the people who asked us for help personally, there was never any criticism, but rather the opposite, always good words, and congratulations. This not only helped us to understand our flaws, but also to be able to understand them without being told in a clear way, encouraging, and learning self-criticism in a truthful way, that self-criticism that is nothing more than a way to improve and from which, however, on many occasions we tried to escape.” (25)

Finally, the training tasks performed by students are supported by the elderly’s desire to learn and allow some students to discover themselves in new roles.

”… it has all been a matter of wanting to teach the people of the Senior Center and their desire to learn and to always know more has excited and encouraged me a lot to do this work.” (26) “The day before, I was with my groupmate going over everything that Google Meet could offer, and when it was my turn to expose that idea to my “student”, it all went smooth and I explained everything in detail, which rarely happens to me.” (27)

c. Acquisition of general and professional competencies

The experience allows participating students to acquire several skills, both general and professional or related to the discipline. In some reports, it is natural to place the project in the context of the pandemic to justify the need for the tutorials and to understand the meaning of the participation and those skills that the situation forced them to make use of. In some cases, the story begins to be told from the previous academic year, when the projects that the students had planned could not be carried out due to different measures adopted after the onset of the pandemic in March 2020, underscoring resilience as a general competency.

“It was at that moment that the center, our university, our professors and we organized to see how we could help.” (28) “It made me quite pleased to have been able to carry out this activity, as the elderly have been severely punished during the pandemic and, above all, for the distancing measures.” (29)

If, on the one hand, this situation highlights students’ resilience and capacity to adapt, as a general competence, on the other, when they contextualize the project within the pandemic, they make use of terminology from the natural and health sciences, such as Sars-CoV-2, comorbidity, COPD, immunosenescence or chronic inflammation, integrating in such a way interdisciplinary elements.

Directly or indirectly, several reports highlight communication as a competency developed in the project, even through less common means and despite the idleness brought about by the isolation of the pandemic.

“At first I didn’t feel comfortable talking to someone I couldn’t see. In the end, I’m more used to communicating by message than by phone call, it seems more personal to me.” (30)

Communicative skills are perceived to be related to empathy in the interaction with other people, and the importance of working with the human and not just the purely intellectual is highlighted.

“Being able to write a work that is not of research and that is not so much of an intellectual nature, but is of a human nature and of empathy, makes us approach it in a more relaxed way focusing more on people than on information. […] Thanks to this experience, we had to develop our empathy to perform this task, a skill that is paid little attention when it comes to training professionals for jobs that require some kind of attention to the public, but that is very important.” (31)

As for the acquisition of educational and professional skills, in addition to resilience, communicative skills and empathy, the reports present a wide range of results. Students touch with their hands issues related to the discipline, such as the ability of reading to entertain, transmit knowledge and support during the pandemic and lockdowns, a topic that emerges from numerous reports as a result of the interaction with elderly. Students also learn to value different sources of information and, in the excerpt below, the authors reflect on personal communication as a source of information different from the “objective” and distant information transmitted by the media.

“The media only shows us objective news, there have been few times when they really showed how this generation has coped with the Coronavirus.” (32)

Those who tutored in the use of technologies gained experience in the resolution of basic computer incidents.

“The problem came when I asked myself how I was going to teach using Google Meet if I didn’t know how to use it. And as they say that every problem has a solution, I set out to find it. The solution came quickly because the internet has a large assortment of video tutorials on how to use it … the tablet she used was of Apple, so it has a different operating system from the one I’m used to. This meant that we had to investigate what its use was like.” (33)

Finally, by relating to the reality of the elderly, some students develop inquisitive abilities, coming to define authentic research problems. In this sense, it is meaningful the case of three students who discovered important gender differences in the group of elderly they trained, constituted mainly by women. Faced with such disparity and the requests of many women to invite their respective husbands, they managed to encourage many men to take part and at the same time document the reasons for their hesitation, that settled fundamentally on stereotypes and prejudices, on the one hand, and on emotions such as shame, on the other.

d. The “snowball” effect: Results occurred where they were not expected

One of the most surprising results from the perspective of the students has been the ability of some initiatives to reach beyond the people who participated directly in the project, creating a domino or “snowball” effect, as reflected in the fragment below.

“After the Gran Nevada [the Heavy Snowfall], we put up a group with two people who needed help. […], after this, the link was passed via email so that 5 more people could join the group and via WhatsApp to the general group that consisted of 45 people, and finally the members of the group shared it also via SMS. The group has almost ninety people today, and the curious thing about it is that, when they have been asking for help, it has been the members of the group themselves who have solved the problem.” (34)

From the perspective of the societal value, this experience evidences the unexpected nature of certain results, that, in this specific case, occurred in people who were not part of the project, going beyond the initially defined nucleus of participants.

e. Breaking down stereotypes

Just as the interaction with elderly leads students to know themselves better, it also allows them to discover a reality that they were not aware of, that their lives and elderly’s have much in common, dismantling in this way the stereotype of opposing generational poles.

“… like ours, their classes were online, so if they did not know how to use the right technological tools, they could not take their classes.” (35) “So, we set out to help them and, without realizing it, they helped us too, because we have been able to observe from another perspective many of the problems that have arisen as a result of the Coronavirus.” (36)

The awareness that the elderly’s reality and their own have much in common ends up demolishing other prejudices and stereotypes, including some related to gender, the supposed elderly’s ineptitude at technologies, the stereotype of dependence or other generalizations about this stage of life.

“In addition, we have been able to take away from our minds that image that they have given us of the elderly, that image of a dependent person who barely understands what is happening around.” (37) “Some of these people are passionate about technologies and what their evolution will be like in the future and like to be updated at all times. However, others not so much, and they are content with the little knowledge they have.” (38) “[…] I believe that the technological capabilities of older people should not be underestimated […]” (39)

In some cases, students help elderly themselves overcome their own prejudices.

“In this sense, it was mainly women’s spouses in the group who did not want to participate in the workshop because they were ashamed. That view was very common among male participants, who mentioned stereotypes such as, “Those things only serve to gossip” and “That’s women’s stuff.” Or “How am I going to go to a Senior Center? Are we crazy?” “Yes, I’m sure I’m the only man.” And some more stereotypes that we have been collecting from the different talks that we had with them.” (40)

3.3.1 Sentiment analysis of reflective reports

As mentioned before, in the students’ reports reflection and self-criticism occurred as a consequence of both positive and negative emotional responses in the interaction with the elderly. In this section, we present the data related to sentiment analysis of the 10 reflective reports analyzed previously and of 9 reports of a control group who carried out other activities related to the project that did not require direct interaction with the elderly. The null hypothesis of equality between the two sets could be rejected only for sentiment intensity, that turned out to be significantly higher for the reports corresponding to the telephone tutorials (Student T $=$ 2,188; $p$ -value $=$ 0.042). Lingmotif also returns a score between 0 and 100%, with the values closest to zero corresponding to a negative sentiment, those closest to one hundred to a positive sentiment and the intermediate values to a neutral feeling. In this sense, the two sets of reports did not differ significantly, with the average of the tutoring group being 53.7%, and the control group 47.4%. However, it is worth noting a not significantly higher percentage of neutral sentences in the control group’s reports (54.5%) compared to the telephone tutorials reports (40.1%), partly corroborating the idea of the emotional rollercoaster that emerges from the reports.

3.4 Interactions with the Senior Center

The emails exchanged with the Senior Center attest to a difficult and often cut off communication, that at times failed for several different reasons including those related to the pandemic or to the integration of new less experienced staff, partly confirming the communication dynamics of previous editions of the project. At one point at the end of October 2020, the elderly contact details initially provided by the Center and shared with Faculty through online documents were withdrawn, because the elderly were not being tutored by students as quickly as the situation demanded. The incident was solved by the Center who appointed a single social worker, well informed about the project, to coordinate the collaboration with the university and sent reassuring messages: “… we have met and commented on what you have told us […], and, first of all, we want to convey a message of calm, because we have seniors to train all the year round … However, what do you think if we firm it up in a meeting the next Tuesday …?”. This atypical incident in the relationship between the Center and the university supports the tendency of some partner employees to conceive of students’ participation in terms of increasing organizational capacity, emphasizing the tangible versus non-tangible value of collaboration, though other intangible benefits are also valued. These two tendencies coexist in employees of the same organization that shows, on some occasions more than in others, a not homogeneous and fluid profile. The meetings held both in person and by telephone with the Center allowed to agree on all the decisions necessary for the implementation of the project, to verify elderly’s and students’ accounts, and to clarify incidents. One additional issue that needed clarification was the cases of conflict between the elderly’s and students’ accounts, as often they did not report the same experiences. At the final evaluation meeting, the Center’s representatives emphasized the peculiarity of a year in which the center’s main task had been to provide emotional support to a population deeply affected by the pandemic, with few and limited possibilities for face-to-face meetings. The cases of conflicts between elderly’s and students’ account and the progressive loss of contact with some elderly were explained as a consequence of the cognitive decline suffered by many elderly and the authentic “invasion” of telephone calls they received throughout the year from social services, city council, NGOs, and many other organizations. In a positive way, the Center highly valued the students’ initiative to reach beyond the contacts provided, stretching out to elderly not even linked to the Center, and the fact that they fostered the elderly’s capacity for mutual support and autonomy. Likewise, the results of the evaluation carried out by the Faculty team and based on the students’ reports and the interviews with elderly were confirmed.

4. Discussion and conclusions

In this case study, we have summarized different data obtained in the execution of a SL project in the Faculty of Information Science of the Complutense University of Madrid in collaboration with two Senior Centers. The experience reported from different perspectives confirms experiential learning as a powerful formative tool for students who, according to the reports provided, advanced in terms of autonomy, development of light competencies and other professional competencies, empathy and civic values, and dismantled prejudices and stereotypes. On the other hand, only 30.1% of the students participating in the project chose to interact directly with the elderly through telephone calls, while the remaining 69.9% preferred to carry out other activities linked to the project or not to participate. In general, the participation in the project has contributed to the training of students in those light competencies that, according to Saunders’ survey (2019), are highly valued by active library staff, in particular interpersonal communication, teamwork, customer service skills, interaction with diverse communities, and the ability to exercise professional practice according to reflection based on the values of diversity and inclusion. It has also trained students for the library programming functions that Norlander et al. (2020) envisage for today’s libraries as “centers for lifelong experiential learning, hubs for civic and cultural gatherings, and partners in community-wide innovation” (p. 188).

As for societal impact, only 22.5% of the participating elderly completed their participation in the project by positively evaluating the interaction with students. Even if some students managed to reach elderly beyond those formally included in the project and, in this sense, the breath of the project may have been larger, the impact achieved may appear relatively small and costly. Follow-up interviews conducted with the elderly indicate that this limited success likely depends on elderly’s expectation of service, conditioned by the SL own methodology, which emphasizes “service” and the dysfunctional aspects of the communities, and by the partner mission itself, the Senior Center. By conceiving the collaboration with UCM students in terms of “service”, many elderly felt authorized to reject, for different reasons, the students’ attempts to offer training or to complain about them, assuming a role of consumers and recipients of a service, rather than collaborators and autonomous participants in the project. McDonough et al. (2017) also detect that community members of their SL project take on a “consumer” role, worrying about students’ end products. If we adopt the suggestion of Frank and Sieh (2016) to take into account the concept of role to determine the impact of any strategy based on experiential learning, we can say that the relative failure of the project may be due to the role of recipients of a service that has been assigned to the elderly. In this sense, the interviews also hint at a demand for greater autonomy of many elderly who claim a relationship of equality and that they should be considered for their real capacities in the relationship with the students. In the specific context of the pandemic, Pentaris et al. (2020) note that the message that all people over the age of 70 are more vulnerable and in need of protection has reinforced ageist messages at the expense of recognizing their strengths and resources, denying their right to autonomous decisions, and ignoring the contribution that many of them have made to resolving the Covid-19 crisis.

The fact that the content analysis and assessment of the project carried out by the Faculty team has been corroborated by the partner reveals another dimension of the societal value of this type of actions undertaken by higher education institutions. Specifically, in this case, the process of evaluation and reasoned criticism led by the university and based on the perceptions of students and elderly has pointed out the limitations of a collaboration based on asymmetric and unidirectional relationships, conditioned by the very concept of service, and enacted through uncomfortable roles, especially for the elderly, but not only. A feeling of discomfort also emerges from the educational perspective in the reports analyzed in this case study, in the form of an emotional rollercoaster and greater emotional intensity of the reports related to telephone tutorials versus those based on classroom tasks. McDonough et al. (2017) consider that, in their project, it was a feeling of discomfort to push students beyond their comfort zone, functioning as a motivational element. On the other hand, in this project the emotional rollercoaster produces a work of self-criticism and reflection that, together with the supervision received by the direction of the project and drawing on students’ capacity to empathize, leads to autonomous decisions especially when it comes to devising appropriate communicative strategies. In this sense, emotional factors should be considered more clearly when evaluating the educational results of SL in its connection with students’ behavior because, according to Gelmon et al. (2018), they have been measured from the cognitive, affective, and psychological point of view, attending mainly to psychological aspects and treating little behavioral aspects.

References

Angel

C.M.

(2016). Collaboration among faculty members and community partners: increasing the quality of online library and information science graduate programs through academic service-learning. Journal of Library & Information Services in Distance Learning, 10(1-2), 4-14.

Bornmann

(2013). What is societal impact of research and how can it be assessed? A literature survey. Journal of the American Society of Information Science and Technology, 64(2), 217-233.

Caspe

, & Lopez

M.E.

(2018). Preparing the next generation of librarians for family and community engagement. Journal of Education for Library and Information Science, 59(4), 157-178.

Chupp

M.G.

, & Joseph

M.L.

(2010). Getting the most out of service learning: maximizing student, university and community impact. Journal of Community Practice, 18(2-3), 190-212.

Cronley

Madden

, & Davis

J.B.

(2015). Making service-learning partnerships work: listening and responding to community partners. Journal of Community Practice, 23(2), 274-289.

Cristóbal Querol

G.D.

García Moreno

Montesi

Ovalle Perandones

Portela Filgueiras

Ramírez Martín

S.M.

Ramos Simón

Rázquin Zazpe

, & Villaseñor Rodríguez

(2020). Experiencia Aprendizaje y Servicio en el Grado en Información y Documentación. Available at: https://eprints.ucm.es/id/eprint/62031/1/EXPERIENCA,%20APRENDIZAJE%20Y%20SERVICIO.pdf [Retrieved June 21

{}^{\text{st}}

, 2021].

Daraio

, & Bonaccorsi

(2017). Beyond university rankings? Generating new indicators on universities by linking data in open platforms. Journal of the Association for Information Science and Technology, 68(2), 508-529.

de Maio Nascimento

(2020). Covid-19: U3A students’ report on the impacts of social isolation on physical and mental health and access to information about the virus during the pandemic. Educational Gerontology, 46(9), 499-511.

Faure

Blundo-Canto

Devaux-Spatarakis

Le Guerroué

J.L.

Mathé

Temple

Toillier

Triomphe

, & Hainzelin

(2020). A participatory method to assess the contribution of agricultural research to societal changes in developing countries. Research Evaluation, 29(2), 158-170.

10.

Fauzi

M.A.

Tan

C.N.L.

Daud

, & Awalludin

M.M.N.

(2020). University rankings: a review of methodological flaws. Issues in Educational Research, 30(1), 79-96.

11.

Fernández-Cano

Curiel-Marin

Torralbo-Rodríguez

, & Vallejo-Ruiz

(2018). Questioning the Shanghai Ranking methodology as a tool for the evaluation of universities: an integrative review. Scientometrics, 116(3), 2069-2083.

12.

Frank

A.I.

, & Sieh

(2016). Multiversity of the twenty-first century-examining opportunities for integrating community engagement in planning curricula. Planning Practice & Research, 31(5), 513-532.

13.

Gelmon

S.B.

Holland

B.A.

, & Spring

(2018). Assessing service-learning and civic engagement: Principles and techniques. Boston, Massachussets: Campus Compact.

14.

Heyeres

Tsey

Yang

Yan

, & Jiang

(2019). The characteristics and reporting quality of research impact case studies: a systematic review. Evaluation and Program Planning, 73, 10-23.

15.

James

J.H.

, & Logan

(2016). Documenting the community impact of service-learning coursework: theoretical and practical considerations. Partnerships: A Journal of Service-Learning and Civic Engagement, 7(2), 17-36.

16.

Janker

, & Mann

(2018). Understanding the social dimension of sustainability in agriculture: a critical review of sustainability assessment tools. Environment, Development and Sustainability, 22, 1671-1691.

17.

Jettner

Pelco

, & Elliott

(2017). Service-Learning Community Partner Impact Assessment Report. Virginia Commonwealth University, Richmond, VA. Available at https://scholarscompass.vcu.edu/community_resources/75/ [Retrieved June 21

{}^{\text{st}}

, 2021].

18.

Johnes

(2018). University rankings: what do they really show? Scientometrics, 115(1), 585-606.

19.

Lauronen

J.P.

(2020). The dilemmas and uncertainties in assessing the societal impact of research. Science and Public Policy, 47(2), 207-218.

20.

Lim

, & Bloomquist

(2015). Distinguishing service learning from other types of experiential learning. Education for Information, 31(4), 195-207.

21.

Madrid City Council (2021). Carta de Servicios de los Centros Municipales de Mayores. Available at: https://www.madrid.es/portales/munimadrid/es/Inicio/Mayores/Cartas-de-servicios/Carta-de-Servicios-de-los-Centros-Municipales-de-Mayores/?vgnextfmt=default&vgnextoid=f1996630c34ea610VgnVCM1000001d4a900aRCRD&vgnextchannel=d85931d3b28fe410VgnVCM1000000b205a0aRCRD [Retrieved August 22, 2021].

22.

Marullo

, & Edwards

(2000). From charity to justice: the potential of university-community collaboration for social change. American Behavioral Scientist, 43(5), 895-912.

23.

McDonough

M.L.

Marks

, & Harris

(2017). “A truly inspiring notion:” a case-study of project-based graduate service-learning. Partnerships: A Journal of Service-Learning and Civic Engagement, 8(2), 63-78.

24.

Miettinen

Tuunainen

, & Esko

(2015). Epistemological, artefactual and interactional-institutional foundations of social impact of academic research. Minerva, 53(3), 257-277.

25.

Mironesco

(2018). Community Partner Voices: Service-Learning Perspectives from Hawai’i. eJEP: eJournal of Education Policy, Spring. Available at: https://in.nau.edu/wp-content/uploads/sites/135/2019/01/Mironesco.pdf [Retrieved June 21

{}^{\text{st}}

, 2021].

26.

Montesi

Álvarez Bornstein

Cuevas Cerveró

Fernández Bajón

M.T.

García Moreno

M.A.

Ovalle Perandones

M.A.

Ramos Simón

Sacristán Sánchez

, & Villaseñor Rodríguez

(forthcoming). Aprendizaje y Servicio en el Grado en Información y Documentación en tiempos de Covid-19. Madrid: Facultad de Ciencias de la Documentación, Universidad Complutense de Madrid.

27.

Montesi

Portela Filgueira

Ramírez Martín

, & Villaseñor Rodríguez

(2019). Aprendizaje y Servicio (ApS) en los estudios de Información y Documentación: resultados preliminares de un proyecto UCM. En: Simeão, E., Cuevas Cerveró, A., Botelho, R., & Gómez-Hernández, J.A. (eds.). Competência em Informação e Políticas para a Educação Superior. Estudos Hispano-Brasileiros, pp. 162-175. Available at: https://eprints.ucm.es/id/eprint/59952/25/volumen1.pdf [Retrieved June 21

{}^{\text{st}}

, 2021].

28.

Montesi

, & Rodríguez

I.V.

(2018). El impacto social de las instituciones de educación superior: un estudio de caso con la Universidad Complutense de Madrid. Información, Cultura y Sociedad: Revista del Instituto de Investigaciones Bibliotecológicas, 39, 37-60.

29.

Montesi

Villaseñor Rodríguez

, & García Moreno

M.A.

(2021). Aprendizaje-Servicio en los estudios de Información y Documentación: una experiencia con personas mayores. Informatio, 26(1), 280-312. doi: 10.35643/Info.26.1.14.

30.

Moreno-Ortiz

, & Pérez-Hernández

(2018, May). “Lingmotif-lex: a Wide-coverage, State-of-the-art Lexicon for Sentiment Analysis”. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), pp. 2653-2659. Available at: https://www.aclweb.org/anthology/L18-1420.pdf [Retrieved June 21

{}^{\text{st}}

, 2021].

31.

Most

L.R.

(2011). Hands on from a distance: the community-embedded learning model contextualizes online student coursework. Journal of Education for Library and Information Science, 52(4), 295-304.

32.

Mtawa

N.N.

(2019). Human Development and Community Engagement through Service-Learning. Cham, Switzerland Palgrave Macmillan.

33.

Ngui

E.M.

(2020). “How to Engage Communities in Research”. In: Weiss Roberts, L. (ed.), Roberts Academic Medicine Handbook, Cham, Switzerland: Springer Nature, pp. 347-359.

34.

Norlander

R.J.

Barchas-Lichtenstein

Fraser

Davis Fournier

Voiklis

, & Danter

(2020). Getting consensus about competencies: what’s needed for effective library programs. Journal of Education for Library and Information Science, 61(2), 188-211.

35.

Nutefall

J.E.

(ed.). (2016). Service learning, information literacy, and libraries. Santa Barbara, California: ABC-CLIO.

36.

Pentaris

Willis

Ray

Deusdad

Lonbay

Niemi

, & Donnelly

(2020). Older people in the context of COVID-19: a european perspective. Journal of Gerontological Social Work, 63(8), 736-742.

37.

Poole

A.H.

(2021). Promoting Diversity, Equity, and Inclusion in Library and Information Science through Community-Based Learning. In Diversity, Divergence, Dialogue: 16th International Conference, iConference 2021, Proceedings, Part I 16 Beijing, China, March 17–31. Springer International Publishing, pp. 529-540.

38.

Riddle

J.S.

(2003). Where’s the library in service learning?: models for engaged library instruction. The Journal of Academic Librarianship, 29(2), 71-81.

39.

Roy

Jensen

, & Meyers

A.H.

(ed.). (2009). Service learning: Linking library education and practice. Chicago: American Library Association.

40.

Samuel

G.N.

, & Derrick

G.E.

(2015). Societal impact evaluation: exploring evaluator perceptions of the characterization of impact under the REF2014. Research Evaluation, 24(3), 229-241.

41.

Saunders

(2019). Core and more: Examining foundational and specialized content in library and information science. Journal of Education for Library and Information Science, 60(1), 3-34.

42.

Scott

E.M.

(2020). Redefining the pedagogy: service-learning in libraries and archives. Pennsylvania Libraries: Research & Practice, 8(1), 37-48.

43.

Scripps-Hoekstra

(2020). “Taking the Campus into the Community: Information Literacy Instruction and Service Learning”. In: Schaub

McClure

(eds.), Engaging Students through Campus Libraries: High-Impact Learning Models, Santa Barbara, California: Libraries Unlimited, pp. 123-133.

44.

Smit

J.P.

, & Hessels

L.K.

(2021). The production of scientific and societal value in research evaluation: a review of societal impact assessment methods. ResearchEvaluation, rvab002, doi: 10.1093/reseval/rvab002.

45.

Stevenson

(2020). “Community-Led Librarianship Demands an Asset-Based Community Development Approach”. In: ALISE 2020 Proceedings, pp. 195-202. http://hdl.handle.net/2142/108812.

46.

Tahamtan

, & Bornmann

(2020). Altmetrics and societal impact measurements: match or mismatch? A literature review. El Profesional de la Información (EPI), 29(1).

47.

Temple

Barret

Blundo Canto

Dabat

M.H.

Devaux-Spatarakis

Faure

Hainzelin

Mathié

Toillier

, & Triomphe

(2018). Assessing impacts of agricultural research for development: a systemic model focusing on outcomes. Research Evaluation, 27(2), 157-170.

48.

Thompson

V.L.S.

, & Hood

S.M.

(2017). “Academic and Community Partnerships and Social Change”. In: Tate

W.F.

Staudt

Macrander

(eds.), The Crisis of Race in Higher Education: A Day of Discovery and Dialogue. Bingley: Emerald Group Publishing Limited, pp. 127-149.

49.

Trencher

Terada

, & Yarime

(2015). Student participation in the co-creation of knowledge and social experiments for advancing sustainability: experiences from the University of Tokyo. Current Opinion in Environmental Sustainability, 16, 56-63.

50.

Trencher

Yarime

McCormick

K.B.

Doll

C.N.

, & Kraines

S.B.

(2013). Beyond the third mission: exploring the emerging university function of co-creation for sustainability. Science and Public Policy, 41(2), 151-179.

51.

Tuunainen

, & Kantasalmi

(2017). Advancing Understanding about the Relationship between Science, University and Society-An Introduction [Guest Editorial]. Science & technology studies, 30(2), 2-13.

52.

United Nations (2020). Policy Brief: The Impact of COVID-19 on older persons. Available at: https://unsdg.un.org/sites/default/files/2020-05/Policy-Brief-The-Impact-of-COVID-19-on-Older-Persons.pdf [Retrieved June 21

{}^{\text{st}}

, 2021].

53.

Wolf

Lindenthal

Szerencsits

Holbrook

J.B.

, & Heß

(2013). Evaluating research beyond scientific impact. How to include criteria for productive interactions and impact on practice and society. GAIA-Ecological Perspectives for Science and Society, 22(2), 104-114.

Assessing the societal value of a service-learning project in information studies during the COVID-19 pandemic

Abstract

Keywords

1. Introduction

1.1 Service-Learning in Library and Information Science

1.2 The different dimensions of SL impact

1.3 SL in the framework of institutional activities

1.4 Case study objective

2. Methodology

Table 1 Summary of data handled in the project

3.1 Global project data

Table 2 Summary of telephone tutorials

a. Refusal of students’ assistance

b. Service expectations and complaints

c. Elderly’s claims for autonomy

d. Students’ interpersonal skills

3.3 Students’ reports

a. Emotional roller coasters

b. Supervision and autonomy

c. Acquisition of general and professional competencies

d. The “snowball” effect: Results occurred where they were not expected

e. Breaking down stereotypes

3.3.1 Sentiment analysis of reflective reports

3.4 Interactions with the Senior Center

4. Discussion and conclusions

References

Table 1
Summary of data handled in the project

Table 2
Summary of telephone tutorials