Abstract
AccesArte is an accessibility project that is part of the internship programme at Kaleidoscope, a non-profit organization founded by the authors to apply audio description (AD) research to developing accessibility programmes. The project consists of ADs of a thematic selection of visual artworks, and the development of eclectic and experience-oriented AD types. It uses online, open access videos and performs a formative and summative evaluation of the resources, with the latter being online, open and ongoing. The goal of this article is to offer a detailed description and critical analysis of the project, with an emphasis on the formative evaluation process. In this regard, the formative evaluation has allowed us to include blind and partially sighted (BPS) consultants in the process, improve resource quality, and enrich the interns’ learning experience.
Non-profits, internships, and visual art audio description
The field of museum audio description (AD) has developed considerably over the past decade in terms of the resources being created, the training opportunities that are available, and the research that is being performed in this field. Accessibility is an increasingly important goal for museums (Audio Description: Lifelong Access for the Blind, n.d.; Soler, 2012) and there has been an increase in the number of companies, organizations, and freelance professionals in the translation, accessibility, and audio-guides fields that offer museum AD services (Soler, in press). In academia, a growing number of scholars have incorporated this AD modality into their teaching and research, especially in the Translation Studies discipline (Fryer, 2016; Neves, 2012). As a result, it is fair to say that AD for museums is a growing professional practice and field of study today. Against this backdrop, an internship in museum AD was first offered in 2016 by Kaleidoscope (Luque & Soler, 2019), a non-profit association created by the authors to apply their research in AD within the Translation Studies discipline to improving accessibility to culture. To this end, we offer training, consulting services, and accessibility solutions to museums, heritage sites, and film productions, with an emphasis on museum AD for blind and partially sighted (BPS) people. One of these solutions is multisensory guided tours in art museums and exhibitions with AD and tactile, sound, and olfactory resources. So far, we have collaborated with the Sorolla Museum in Madrid, the CajaGranada Art Collection and the Alhambra monument and art exhibitions in Granada, the Iberian Museum in Jaen, and the National Archaeology Museum in Madrid, among others.
The internship programme is aimed at Translation and Interpreting Studies graduates and is offered in collaboration with an MA in Professional Translation at the University of Granada, Spain. Despite the growing number of training opportunities in this field, they are still limited. Thus, the goal of this internship programme is to foster training in this area to increase the quality and quantity of museum AD resources. The students are guided by the authors, who are professional researchers and practitioners in the AD field, to complete real accessibility projects. In previous years, interns were tasked with creating an audio descriptive guided tour of a museum or exhibition for BPS people. However, in 2020 the three interns in the programme developed an online visual art AD project titled AccesArte.
From description to action: applied research in visual art AD
According to the results of descriptive studies of AD in art museums (Soler, in press), these institutions hire freelance audio describers or companies, or commission their own art educators to create recorded ADs that BPS visitors can use during their visit. Besides, more and more of these ADs are published online by so they can also be accessed independently from a visit to a museum, thus offering valuable materials to foster access to visual art for BPS people. In all these settings, audio describers are generally individuals who are not BPS and collaborate with BPS consultants to carry out a formative evaluation of the visual art ADs, that is, an evaluation undertaken before the ADs are made available to end-users. However, to the best of our knowledge, there is a limited number of publications that deal with the creation and evaluation of these resources, and so very little is known about this process. In the AD studies field, a small but promising number of applied research projects (Eardley et al., 2016; Jankowska et al., 2017; Neves, 2016; Szarkowska et al., 2016) have been carried out in recent years that have applied various theoretical and methodological approaches to developing recorded ADs for art museums and exhibitions to tackle a real problem, namely the lack of accessibility resources for BPS visitors in the visual arts context and the lack of information about their creation and evaluation process that could inform and foster similar initiatives.
Among these applied AD studies, Eardley et al. (2016) propose a method to design a multisensory experience for two art museums following a universal design approach. Among other resources, this design involves the creation of a multimedia guide with ADs aimed at visitors with and without a visual impairment, but its creation and evaluation are not dealt with in detail in the publication. Along this line, in the Art Translates project, MA Translation students collaborated with museum experts, scholars in audio visual translation, and end-users to create an enriched descriptive guide for an art museum for people with and without a visual impairment of all ages (Neves, 2016). With regard to the ADs, ‘a short, concise and yet engaging script was achieved after multiple attempts in which audio description techniques were merged with those used in museum writing’ (p. 145). Based on the example provided in this publication, the ADs followed the approach advanced in existing guidelines and found in a majority of visual art ADs, both in their structure and the amount and type of information provided. Their evaluation by potential end-users focused on the accuracy, adequacy and impact variables, but the only detailed discussion that is offered focuses on the description of cultural references (pp. 147–149).
Szarkowska et al. (2016) and Jankowska et al. (2017) describe the Open Art Project, which focuses on the development of a mobile application for adults with and without a sensory (visual or hearing) impairment for contemporary art museums, also following the principles of universal design. The application was to be accessed both during a visit to the museum and at home. Regarding the audio descriptive content, the project focused on preferences regarding the amount of description and the inclusion of interpretations of the work, which were evaluated by potential end-users with and without a visual impairment (Szarkowska et al., 2016, p. 309). The final ADs were based on the results from these studies and followed the objectivity paradigm advanced by general AD guidelines (not specific to visual art or museums). They were 2–3 min-long, included possible interpretations of the works, and used plain language (short sentences, simple syntax, and no jargon). As stated by the authors (Jankowska et al., 2017, pp. 122–124), their creation did not take into consideration existing visual art ADs, nor published corpus-based studies and guidelines in this particular field. The ADs were positively evaluated by both BPS users and those without a visual impairment.
Similarly, AccesArte deals with recorded ADs of visual art and their linguistic features. However, in an attempt to offer more thematic and varied resources that can help improve access to visual art for BPS people, the ADs were not created in connection with a particular museum or exhibition. Besides, AccesArte does not follow the universal design approach advanced by the above-mentioned studies. Instead, it proposes the creation and evaluation of ADs following the ‘majority’ or more frequent AD approach as found in art museums and recommended by existing visual art AD guidelines, along with ‘minority’ or less frequent approaches also found among current practices in this field (Soler, 2021). The rationale behind this is twofold. First, the majority approach has not been sufficiently disseminated nor discussed among BPS people. Second, we advance a different universal design concept where a variety of ADs is offered so users with and without a visual impairment can select them and combine them based on their expectations, preferences, and needs.
This article presents the collaborative work performed by the authors as researchers and audio describers, three interns, and a group of BPS consultants. It focuses on the method followed for creating ADs of visual art; the formative evaluation carried out by these consultants; and the ways the needs and preferences of BPS people were key to review the ADs and ultimately allowed for the creation of better resources.
AccesArte: creation of audio descriptions of visual art for the web
The project consists of a series of 15 Spanish ADs of visual art belonging to different artists and movements classified under 5 thematic series that were published on the Kaleidoscope YouTube channel. 1 While these ADs are specifically aimed at BPS adults, they are also useful to audiences with other profiles. The AD resources are created in six fundamental stages: analysis of the work, investigation, drafting of the AD script, evaluation, recording, and editing. The following sections offer a detailed description of the most salient and innovative aspects of this project, which are the selection of works (thematic), the AD type (eclectic and experience-oriented), the mode (video) and medium (free-access, comment-enabled and online), and the evaluation of the resources (formative and summative, the latter being open and ongoing).
With regard to the selection of works to be audio described, a number of art museums have made their AD resources available online, but audio described works constitute only a small percentage of their collection, usually the museum’s highlights. Similarly, Art Beyond Sight, a New York company which specializes in accessibility to culture for BPS people, offers online art education materials with ADs grouped according to historical periods and movements, which is the traditional classification of contents in art history programmes and in most art museum galleries and exhibitions. Nevertheless, some scholars and practitioners of art education emphasize the benefits of designing educational materials and activities around a specific theme or concept, and this is our goal for AccesArte. At the Barnes Foundation, an art education institution founded by Albert Barnes in Philadelphia in 1922, art education programmes follow a revisited version of the visual literacy and art appreciation method proposed by Barnes in his 1925 book The Art of Painting. He was influenced by John Dewey’s work on experience-based education as director of education at the Barnes Foundation in 1923. It proposes juxtaposing works of art and other kinds of objects as a means to develop formal art analysis and interpreting skills (Burnham, 2011).
Also within the field of art museum education, Olga M. Hubard (2013) has studied the concept of ‘theme’ used by a number of museums, and classified them into four categories: the human issue that an artwork is about (theme in the literary sense); a motif or obvious topic that is reiterated and varied across artworks (theme in the musical sense); a design element or principle that is notable across a series of works (theme in the Modernist sense); and an aspect of the artists’ creative process. Hubard defines thematic museum education programmes as those that ‘present to visitors a series of works bound together by a concept that transcends chronology, artist, movement, geography, genre, and other traditional art historical categories’ (p. 91). According to her, using a theme may help to build connections between works of art and topics, explore a range of perspectives on a core idea, encourage depth of interpretation by focusing an inquiry, and delimit the act of interpreting. We decided that each intern and tutor in the AccesArte project would design a series of three ADs linked by a common theme of their choice.
With regard to the AD type, the first four series applied what we term as the ‘objective’ type or approach, recommended by current guidelines and implemented by most museums and professionals. Our proposal for an objective approach to visual art AD for this project is based on current guidelines in this field (Audio Description Coalition, n.d.; Giansante, n.d.; RNIB & VocalEyes, 2003; Salzhauer & Sobol, 2003), and its more frequent or ‘majority’ features as revealed by corpus-based descriptive studies (Luque & Soler, 2020; Perego, 2019; Soler, 2016, 2018, 2019). For this type, we created ADs that progress from a general overview of the work to a very detailed description of the visual components. They conclude with interpretive and contextual information on the artist, the movement, and technique, based on expert documentation about the work. This contextual information is also offered at the beginning when deemed necessary to understand the description of the visual components. The detailed description includes clear indications of the location of the visual components and it uses precise lexis, including terminology related to Visual Arts and Art History and the artwork’s subject matter, and their definitions when necessary. Likewise, the detailed description includes colours, often in metaphorical association with real-world elements, other types of metaphors, and repetitions to enhance textual cohesion. Excessively complicated and oversimplified syntax were avoided. For the recording, a somewhat slow speed was used, along with a tone and cadence that were appropriate to the content of the work. Finally, music and sound effects were added, creating yet another layer of intersemiotic translation of the work. The inclusion of these components was based on previous studies that advance the benefits of multisensory resources to facilitate the users’ understanding of the description and their transformative, multisensory, experience of the artwork (Eardley et al., 2017; Neves, 2012).
The highly detailed and well-organized ADs of the objective type have received some criticism favouring more subjective descriptions that focus on the impressions triggered by the artwork (Kleege, 2018, pp. 112–117). Besides, museum visitors, including BPS people, have different expectations, needs, and preferences, and so different AD types should be created and their impact on visitors’ experience assessed (Hutchinson & Eardley, 2019, p. 53). For this reason, while the objective type was used for the four first series, in the series entitled ‘Zoco Polyphony’, the second AD is objective, while the first one is what we have come to call the ‘gist’ type and the third one is of the ‘poetic’ type. These two types are based on Claire Bartoli’s ADs for the Musée d’Art Moderne de la Ville de Paris 2 and the Musée d’Art Contemporain du Val-de-Marne. 3 In line with the work of this writer and audio describer, the gist AD type reduces the detailed verbal descriptions found in objective AD to a minimum, and the location of the visual components is, for the most part, implied in the sequence used to describe the work. In addition, different vocal speeds and tones may be used to help translate the composition. The poetic AD type is inspired by Bartoli’s work as well as literary ekphrasis (Krieger, 2000). However, while Bartoli’s AD is a poetic text that includes both objective and subjective descriptions of the artwork, the poetic AD used for the AccesArte project is a subjective poetic description inspired by the visual experience of the artwork created by a poet and collaborator of the project.
Once each audio describer drafted the initial AD script, the text was subjected to three rounds of revision, which served as formative evaluations of both the interns’ work and the accessibility resources created. Subsequently, the final AD was recorded and added to a video with images of the artwork in motion. These videos focus on the different elements of the images, thus imitating eye movement and corresponding to the information flow of the AD. This approach may be beneficial for people with different types and levels of vision, although a possible drawback of this approach is that it prevents the user’s eyes from moving to sections of the image other than those being audio described, which might limit their experience of the work. The videos were edited with the Davinci Resolve 16 freeware and are inspired by the online dissemination and educational activities of artworks by museums such as the Museo del Prado, in addition to YouTube history and art channels. 4 This type of video, combining images of the artwork and the audio description, had already been used and assessed within the Open Art Project, which showed that both people with and without a visual impairment preferred this medium over text and audio (Szarkowska et al., 2016, p. 312). The finished videos were uploaded to a playlist on Kaleidoscope’s YouTube channel. This marked the beginning of the online, open, and ongoing summative evaluation of the project, an essential but often missing component in AD projects on visual art.
Evaluating AccesArte: describers and consultants in collaboration
In the AccesArte project, the formative evaluation had three stages. The first stage was a collective peer-review of the scripts carried out among the students, while in the second stage the internship tutors reviewed the students’ scripts. In the third stage, we recorded the ADs and sent them to a group of BPS consultants, along with the scripts and a questionnaire. The remainder of this section is devoted to this third stage in the formative evaluation process.
Method and participants
For this study, we chose to conduct a qualitative formative evaluation of the resources to gain a better understanding of the individual expectations, preferences, and needs of end-users. To this end, we designed a survey that focuses on five features: (1) the location of the visual components, (2) the amount of information offered, (3) the language used, (4) the description of the colours, and (5) the speed and general quality of the recording. The reason for selecting these elements is that they are among the main topics discussed in visual art AD guidelines, and descriptive studies in this field have displayed considerable variety in how audio describers approach these features. The first five sections of the survey deal with these features. They start with a title indicating the element to be evaluated and include between two and four open-ended questions with increasing levels of specificity. The location, the language, and the recording are first assessed for adequacy and subsequent questions inquire about specific elements within the category (lexis and syntax for language, and speed and quality for the recording), or the user’s comprehension in connection to that feature (for location). In addition, the location and language sections ask for users’ recommendations to improve. This is helpful because these are rather broad categories and so they could receive very different evaluations depending on the evaluators’ individual profiles. Unlike these, the amount of information section inquiries about users’ perceived level of detail of the descriptions and their usefulness to build a mental image of the work. These are followed by two other questions that are more specific and quantitative and when combined, these four questions help understand users’ preferences for this element. The colour section starts with two broad questions on the users’ experience of the colour descriptions and their perceived quality of the explanations of colour, followed by a third question on the amount of detail in the descriptions of this element. Together, these three questions help elicit evaluations of a component that may be experienced in very different ways by BPS people. One last section for comments and suggestions was added for evaluators to comment on any other aspects. The following questions were asked:
On the location of the elements of the work:
Do you think the indications on the location of the work’s components are adequate?
Were they confusing to you?
What could we improve?
On the amount of information provided:
In your opinion, are the descriptions of the works very detailed?
Have these descriptions successfully represented the works, or did they help you to imagine what the works depict?
Is there any missing information?
Would you prefer shorter descriptions?
On the language used:
Do you think the language used is adequate?
Are the words used appropriately?
Are the sentences too long, too short or of adequate length?
If not all the aspects described were clear to you, how may we improve?
On the description of the colours of the work:
How did you imagine the described colours?
Are the colours well explained?
Would you like a more detailed description?
On the recording:
Is the speed used for the voicing adequate for understanding the information?
Is the sound quality adequate?
Comments and/or suggestions:
To contact potential participants and collect data on their reception of the resources, their preferences, and needs, we used the Facebook group Ciegos de todo el mundo (Blind people from around the world), Twitter, the ASEPAU association (Spanish association of accessibility professionals), and the ONCE organization (Spanish national organization of blind people). They had approximately 2 weeks to send their evaluations. We received 15 individual responses from BPS participants, of which nine were limited to very general, concise, and positive comments. The other six evaluators offered more detailed responses. Overall, these evaluations also responded very positively. However, the criticism and suggestions for improvement that we received were extremely important to create ADs that are more adequate to users’ needs and preferences.
We completed a qualitative analysis of the responses and classified the critiques and suggestions into nine different categories grouped into the sections of the survey. These categories are location, amount of information, terminology and repetitions (language), colour, audio quality (recording), and sound effects, contextualization and interpretation (other), which are discussed in the following section.
Results and discussion
One interesting result of the analysis is that the gist and poetic ADs, which are the least common types of AD found in art museums today, received very few comments and these did not address the main differences between these types and the objective one, so it is not possible to determine how they were experienced by users. Starting with the terminology category, question number 9, ‘If not all the aspects described were clear to you, how may we improve?’, received the following comment in one case: ‘In my opinion, the language is correct, although there are some techniques that I personally did not understand, for example in the AD of Birth Power. I think that a small description or definition of the technique would help’ (our translation). The evaluator was referring to the following text: ‘Birth Power is part of this collaborative project and is a black silk in which only one embroidered figure appears and extends throughout the painting. It represents the outline of the corpulent silhouette of a full-bodied woman giving birth’. Based on the evaluator’s comment, the AD author made the following modification to the paragraph: Birth Power is part of this collaborative project and
Expanding on the terminology category, similar to the ‘embroidery’ comment, we found that several evaluators had issues with technical terms such as ‘degradé’ and ‘foreshortening effect’. We decided to explain the technical terms after introducing them. This translation technique, included in the guidelines (Salzhauer & Sobol, 2003), was actually recommended by one evaluator, who also commented that the inclusion of a technical term ‘does not represent a barrier to understand the AD’. Another evaluator proposed the creation of a small glossary of materials and techniques. While we did not implement this improvement on this occasion, we think it may be an important tool, considering that understanding of art terms and visual arts experience among BPS people may be limited in certain contexts. It could also reinforce the value of AD as a tool for art education for people with different visual capabilities. This combination of AD and glossary could be a useful tool for BPS people and the general public alike, to learn about visual art beyond a specific museum visit or educational activity, and thus become a part of their hobbies and interests.
In their comments on sound effects, several evaluators emphasized the benefits of sounds and music to complement the AD experience. We had planned to incorporate this feature and their comments reinforced our decision to implement it in the final AD versions (note that the materials sent to the consultants were the AD scripts and plain audio recordings without added sound effects). An evaluator referred to the ‘beauty’ of the experience through music, an impression that is very interesting and relevant when analysing the importance of subjective features in AD projects. The next category is the audio quality, where we addressed two issues highlighted by evaluators: the presence of noise and the emphasis on the human quality of the voices. We cleaned the audio tracks as much as we could, but a degree of interference was unavoidable given that it was a semi-professional project recorded at home. Future projects will of course improve the recording quality. Another criticism was that the voices were similar to those of a speech synthesis or text-to-speech software. Overall, evaluators who mentioned the voice aspect preferred more interpretive voices. For instance, one evaluator pointed out that one AD commenced with an interpretive voice but gradually became more neutral, which they thought was a negative change. We concluded that interpretive voices are positively received, although some BPS people may be used to non-human or synthesized voices that lack personality. However, it may sometimes be hard for an audio describer to use a more interpretive voice, given that objectivity and neutrality are key points stipulated in many guidelines.
Regarding the description of colour, most of the responses regarding this category reinforce our idea of colours as an important tool in AD. However, we would like to highlight the comments that made us rethink the use of specific hues and tones and abstraction in the description of colours in AccesArte. Adjectives such as nacarado (nacreous, pearlescent), oliváceo (a type of olive-green), and níveo (snow-like in a formal register), which could sound very specific and evocative, were considered too concrete. Some evaluators wondered whether the use of very precise shades might be counterproductive for congenitally blind people. However, other evaluators did not like the use of too basic colours, such as ‘red’ and ‘yellow’, and proposed replacing them with analogies such as ‘brick red’ and ‘vanilla yellow’. Colours appear to be a very individual and personal issue. Most evaluators, however, stated that the use of specific or general colours did not interfere with their experience, and that it may be considered an accessory provided the AD is suitable at other levels. A comment that underscores this was, The concept of colours is extremely abstract, so a person who is congenitally blind will never really understand what a colour is. But this should not pose a conflict for them, so there’s no need to dwell on it too much when describing colour; they don’t need that conflict, they just won’t pay attention to it. (our translation)
This harks back to the idea, previously mentioned in this article, that ADs must be adapted according to different user needs or profiles.
Some comments regarding the description of location referred to the confusion created by the description of the areas of a painting. For the AD of The artist painting a cow in a meadow, by Bakhuyzen, one evaluator commented: ‘It is also confusing sometimes that we move from left to right without a clear definition of what plane we are moving in’ (our translation). This led us to change how we introduced the elements. It was a very useful and important comment, since the confusion we inadvertently generate in our ADs may be more easily identified when evaluated by BPS people.
The amount of information was extremely positively received, but there were several cases where the evaluators mentioned that the information could be summarized and the descriptions shortened (e.g., in Girl with a pearl earring, by Vermeer). In the words of one evaluator: ‘I have the impression that the audio describer wanted to create an AD as long as the others’ (our translation). We did not set a specific length for the audio descriptions; rather, we thought the length and level of specificity should depend on the work, and this was the premise that we followed for the project. However, it is very useful to know that some users may have this impression when listening to thematic series of lengthy descriptions for different types of visual artwork. In this regard, the European guidelines recommend creating shorter ADs if they are to be listened to together with more descriptions in a recorded audio guide (Remael et al., 2014), but recorded ADs from art museums range from 100 to 800 words approximately (Soler, in press).
Regarding repetitions, opinions were more diverse: some evaluators preferred to have elements repeated, while others preferred a plainer and more general AD. This supports the idea of adapting the AD to different user profiles and preferences. It may be addressed either in audio descriptive tours, where discussions may help tailor the AD to the visitor, or in audio descriptive mobile guides with different layers or tracks that visitors may choose and combine for a customizable experience. For this project, we decided not to eliminate the repetitions owing to the wide range of opinions, and also because this is an accessibility technique that seeks to promote understanding by reinforcing textual cohesion (Soler & Luque, 2019, pp. 434–435).
Contextualization is an important part of the audio descriptions, according to existing guidelines and ADs in this field. In the objective type, we provide some details regarding the artist at the beginning and complement it with more details after the description of the work, at the end of the audio track. However, we received two comments highlighting the need to add further information on the artist and the location of the work (the museum and gallery where it is displayed). The final category is interpretation. Overall, evaluators were satisfied with the addition of an interpretation of the work after the verbal description and found this to be a very enlightening section. One evaluator even made an in-depth discussion of the symbolic elements in Frida Kahlo’s self-portrait: One might wonder whether or not it would better to discuss the symbolism of each element as the audio describer introduces them . . . but I think it is better that way, yes, it is better to first imagine what the painting looks like and then discuss each of its elements. (our translation)
Once all relevant modifications were made to the AD scripts, we recorded the descriptions again as necessary and added them to the videos described in the previous section. Finally, these videos were uploaded to our YouTube channel, proceeding to the summative evaluation and dissemination of the project. Although summative evaluation is not the focus of this article, it is worth mentioning that YouTube enables an open and ongoing assessment by the online community, at the same time that it enhances the dissemination of accessible content.
Conclusion
Formative evaluation is an essential aspect of any visual art AD project. In AccesArte, it allowed us to incorporate BPS consultants into the process, improve the AD quality, and enrich the learning experience of the interns. In spite of these positive aspects, however, the role of BPS people in the creation of visual art ADs is still mostly limited to consultation, mainly during the formative evaluation of the resources. When it comes to creating accessibility resources such as ADs, BPS people should not be relegated to the status of consultants only. They should also lead and participate in the research, design, and implementation of these resources as experts. This is already the case in the ‘minority’ approaches of the Tate Britain, the Grand Palais, the Musée d’Art Contemporain du Val-de-Marne, and the Musée d’Art Moderne de la Ville de Paris, where BPS audio describers are employed to create recorded audio descriptions and to guide audio descriptive tours (Soler, in press). Following these examples and the principles of Emancipatory Disability Research (Hollins, 2010), we propose that further applied research projects should be carried out in this field in which BPS people participate as researchers since the initial stage of the project.
