Abstract
This study was designed to investigate children’s audiovisual art, following intuitive listening to complete classical works from different historical periods. The study aimed to examine the effect of different musical stimuli on children’s audiovisual output. Two related questions were asked: (1) How do children visually and verbally represent music from different historical periods? (2) What musical and extra-musical elements are represented? Participants were 181 second graders (age 7.0–8.5). Three compositions were presented: a 12th-century anonymous choral from the liturgical drama “Danielis Ludus,” Chopin’s prelude op. 28 no. 10, and Bartók’s “Melody in the Mist” Vol. 4 no. 107, from “Mikrokosmos.” Each composition is based on distinct musical parameters: vocal timbre, melodic contour, and texture alternations, respectively. Data consisted of 495 audio-graphic productions and related accounts. Analysis progressed in three stages abbreviated MSC: Morphological dimensions, focusing on type of symbols employed; Structural dimensions, focusing on the overall graphical design; and Conceptual dimensions focusing on the generic meanings of the productions. Results indicated significant differences in children’s reactions to each of the three compositions, showing that MSC dimensions are influenced by the type of music presented and that differences in MSC dimensions are statistically significant. Experiencing “audiovisuology” in school is one way of promoting art integration.
Keywords
Correlations between music, colors, and shapes attracted a wide attention throughout history in philosophy, psychology, physics, literature, visual art, and music. Since the late-19th century, many artists and composers swore allegiance to hearing music as specific colors (Jewanski, 2009, 2013). Such cross-modal correspondence has been investigated in different ways in recent years, ranging from subjective reports (e.g., Holm, Aaltonen, & Siirto, 2009; Palmer, Schloss, Zoe, & Prado, 2013) to brain imaging (e.g., van Campen & Froger, 2003). The following literature review focuses on studies about children’s invented notations and “drawing music” experiments.
Children’s invented notations
Inspired by the interest in children’s drawings as an investigative tool in the field of cognitive and developmental psychology (e.g., Piaget, 1973), musicians since the late 1960s initiated studies that examined children’s attempts to intuitively notate songs, 1 rhythms, 2 pitch, and rhythm (e.g., Carroll, 2018; Lee, 2013) and their own music (e.g., Barrett, 1997; Upitis, 1990b). Other studies focused on exposing listeners to complete works from the classical repertoire. 3 Kerchner (2001), for example, asked 7- to 8- and 10- to 11-year-olds (n = 12) to listen to the first movement of Bach’s Brandenburg Concerto No. 2 and describe the music verbally, kinesthetically, and graphically; Elkoshi (2014) asked 4- to 9.5-year-olds (n = 209) to listen to a complete work by Claude Debussy—“Jimbo’s Lullaby” from the “Children’s Corner” piano suite—and describe the music graphically and verbally. Results in the latter work showed a strong dominance of associative responses and compound (associative plus sonic) responses over sonic responses at all age levels.
Several studies examined correlations between the representation of auditory fragments and complex musical compositions. Elkoshi (2002) asked second graders (n = 112) to draw a rhythm that was first produced in isolation (“fragmental task”) and later within a classical work that consists of this rhythm (“contextual task”). Findings indicated no correlation between fragmental and contextual reactions, thereby suggesting that one cannot infer from fragmental to contextual perceptions of music. A study with older 8- to 9- and 11- to 12-year-olds showed evidence of such a correlation (Verschaffel, Reybrouck, Janssens, & Van Dooren, 2010).
Various studies investigated music–color associations among schoolchildren. Simpson, Quinn, and Ausubel (1956), for example, found that elementary school children (n = 995) paired high tones with green and yellow more often than with other colors, possibly because of a cultural convention that the high pitches and colors were both considered “bright” or “happy.” Similar results had been found in a previous study with elementary school children (Omwake, 1940, in Rogers, 1987), which confirmed Marks’ (1978) doctrine of analogous sensory attributes, that both pitches and colors are arranged by convention along a “brightness” continuum. Marks (in Sidoroff-Dorso, 2009) found in children, as young as 4-year-olds, who hadn’t yet learned the terms “low” and “high” pitch, a near universal correspondence between higher pitch and greater lightness, just as he found in adults. Another study by Marks, Hammeal, and Bornstein (1987) showed that 5-year-olds reliably matched bright lights with loud and high-pitched sounds, and dim lights with soft and low sounds.
The influence of learning notation on color-hearing was examined by several researchers. Rogers (1991) researched the effect of color-coded notation on music achievement of fifth- and sixth-grade beginner wind instrument players (n = 92). An experimental group used color-coded method books in which each different pitch was highlighted with a different color. A control group used identical materials but with the notation uncolored. Results showed that colored notation helps students to learn notation and rhythms more than uncolored notation. In an 8-year longitudinal study, Elkoshi (2004) considered the effects of age and knowledge of standard notation (SN) on students’ music–color responses. Subjects were asked to represent rhythms at two developmental stages: a “Pre-literate” stage (age 7.0–8.5), with the participants (n = 17) being in their second grade, and “Post-literate” stage, with the same participants in their ninth grade, after learning SN in school while in their sixth grade. Results indicated that SN knowledge did not stifle “color-hearing” responses. On the contrary, the amount of chromic responses increased at the “Post-literate” stage.
Drawing music experiments: The “Musikalische Graphik”
As early as 1917, the Viennese art professor Oskar Rainer (1880–1941) developed a pedagogical method titled “Musikalische Graphik” (“Musical Graphics”), first implemented in Austrian schools (Rainer, 1925). As a multimedia educational factor, Rainer conducted experiments on the interrelations between complex classical works 4 and color harmonies (Haverkamp, 2013; Sündermann & Ernst, 1981). The “Musikalische Graphik” method assumed the general validity of color and form analogies to music, and cross-modal connections of the auditory with the visual modality.
Based on Rainer’s method, long-term experiments of “drawing music” followed in Kazan, Russia, at the Experimental Aesthetics Institute “Prometheus” (Galeyev, 1999; Jewanski & Düchting, 2009; Vanechkina, 1994; Vanechkina & Trofimova, 2000).
The aim of the Russian project was twofold: to test the pedagogical efficiency of Rainer’s “Musikalische Graphik” method and to study psychological regularities of color-hearing synesthesia. Participants were schoolchildren (grades 1–10) with no musical background from ordinary schools, young children from music and art schools, and college students. An experimental group included students from special schools who listened to musical works for 60 min, participated in analyzing the musical forms and contents, and then portrayed the music. The musical repertoire proceeded from short and easy pieces to complex non-programmatic compositions. A control group included general education students who listened to music for 15 min. Subjects created “visual portraits” of musical works by Mozart, Beethoven, Musorgsky, Schumann, Scriabin, Bizet, Gyorgy Sviridov, and others. 5 Music for solo instruments, such as violin, flute, clavecin, and trumpet, and various types of orchestras were employed to reveal timbre–color analogies. In one experiment, subjects watched an animated light-music film, which accompanied the music to be drawn. Follow-up studies via questionnaires were obtained after 5 years, which proved a more thorough and sound knowledge of music in the experimental classes. It was concluded that synesthesia is an ability inherent in all human beings; a manifestation of the essential forces of men. Weighty arguments were obtained against physiological, parapsychological, and mystical explanations of synesthesia (Galeyev, 2000).
Undoubtedly the “Musikalische Graphik” method proved to have great pedagogical effectiveness. However, the “learning-first” approach, namely, analyzing musical forms and contents prior to drawing the music, was supposed to influence the visual outcome. Moreover, musical films of bright light colors and shapes, which were presented, were likely to affect the audiovisual results, both graphically and perceptually. A paradigm shift to a participant-centered perspective seems appropriate for stimulating individual audiovisual elicitations. The present study displays a participant-centered methodology based on intuitive music listening and audiovisual independence.
Purpose of the study
This study was designed to investigate children’s audiovisual art following intuitive listening to complete classical works from different historical periods. The study aimed to examine the effect of different musical stimuli on children’s audiovisual output. Two related questions were asked: (1) How do children visually and verbally represent music from different historical periods? (2) What musical and extra-musical elements are represented?
The musical repertoire
The selected repertoire comprised three complete musical works; each work is based on distinct musical parameters: vocal timbre, melodic contour, and texture alternations, respectively. The following presents the salient musical features that characterize each work.
A 12th-century anonymous choral from the liturgical drama “Danielis Ludus” [“The Play of Daniel”] “Astra tenenti cunctipotenti” [“To the Almighty Holder of Heaven”]
This work is a cheerful strophic-form choral of seven melodic repetitions for each of seven verses. Sections are sung in unison, each time by a different vocal group, in the following order: soprano–alto–bass–soprano–alto–bass–alto. Medieval instruments (flute, drum, strings) accompany the repeated melody. One of the most prominent musical elements is vocal–timbre alternations in each stanza.
Figure 1 presents the first stanza of the choral.
First stanza of “Astra tenenti cunctipotenti.”
Frédéric Chopin, prelude op. 28 no. 10 in C-sharp-minor (1839)
Chopin’s piano prelude is a Molto Allegro virtuoso piece of 18-bars. A four-measure theme includes a swift leggiero steep-descending figuration, ending in a sustained low-pitched cadence. The theme is repeated four times with melodic/harmonic variations. One of the most prominent musical elements is melodic contour alternations between descending figurations and low-pitched phrases.
Figure 2 shows the opening of the prelude; circles highlight contour contrasts.
Beginning of Chopin’s Prelude op. 28 no. 10.
Bela Bartók, “Melody in the Mist” Vol. 4 no. 107, from “Mikrokosmos” (1926–1939)
Bartók’s work is a slow-tempo, Tranquillo piano composition. The theme consists of texture contrasts between dense cluster-chords which represent “mist” and a unison melody. Seven varied appearances of the theme are ending with a sustained cluster-chord cadence. One of the most prominent musical elements is texture alternations between clusters and unison. Figure 3 presents the opening of “Melody in the Mist”; circles highlight texture contrasts.
Beginning of Bartók’s “Melody in the Mist” Vol. 4 no. 107, from “Mikrokosmos.”
Method
Participants
The study took place in three elementary schools located in three large cities in Israel. Subjects were 181 Hebrew-speaking second graders (age 7.0–8.5) from seven classes, as shown in Table 1.
Participants.
Research process
Twenty-one 50-min sessions were held, three with each class. During a session, children encountered one of the three musical works. Each session consisted of four stages: (1) the classroom turned dim and children were instructed to “sit comfortably, with their eyes closed and listen to the music.” (2) The classroom lighted up while colored writing-tools were distributed. Children were told, “as you listen to the music again, look at the colored writing-tools. Try to think what colors might represent the music.” (3) Children used writing-tools of their choice to represent the music graphically. Instructions: “create an illustration that represents the music in any way that seems appropriate to you with any writing instrument you choose.” (4) During individual semi-structured interviews, each child reflected on the drawing and its relation to the music. Three questions were answered: (1) “What is in your drawing in terms of colors and shapes? (2) “What did you hear/notice in the music?” (3) “How is your drawing related to the music?”
Data collection and method of analysis
A total of 495 drawings 6 were obtained from Daniel (n = 170), Chopin (n = 165), and Bartók (n = 160). Data analysis progressed in three stages: Morphological, Structural, and Conceptual categorizations (abbreviated MSC):
Morphological categories (M) focus on the type of symbols employed. Seven Morphological sorts were obtained: (i) Color—divided into monochromes (single color), duochromes (two color), and polychromes (three colors and more); (ii) Shapes—abstract lines and/or geometrical forms; (iii) Pictures—depiction of objects, such as people and landscapes; (iv) Icons—popular images that have recognizable meanings, like “emuji,” road signs, banners; (v) Text—written language; (vi) Numeracy; (vii) SN.
Structural categories (S) focus on the overall design of a painting. Two Structural sorts were obtained: (i) Directionality—divided into horizontal, vertical, horizontal/vertical, circular, and random directions and (ii) Design—divided into two-sided bilateral symmetry and repeated shapes (patterning).
Based on MS inspections and subjects’ related accounts, Conceptual categories (C) focus on the generic meaning of the audiovisual production. Three Conceptual categories were discernible: (i) Sonic productions that represent musical features. Five sonic categories were obtained: musical form, timbre, contour, pitch, and texture; (ii) Programmatic drawings that exhibit imaginative associations; and (iii) Holistic productions that comprise sonic features and programmatic elements simultaneously.
Data examples
The following examples include Daniel (Sample A, Figure 4), Chopin (Sample B, Figure 5), and Bartók (Sample C, Figure 6) drawings, with brief MSC categorizations.
Daniel drawings.
Chopin drawings.
Bartók drawings.
Summary of the examples
The above examples include duochromes (9/10/12) and polychromes (1–8/11). Shapes (1–3/7/9/11) and pictures (5–8/10/12) are commonly used. Pictures describe objects (6/7/10/12), landscapes (5/7/8/10), persons (5/8), and actions (5/7/8). Text is used for sonic representations (4), directives (11), and captions (2/3/12). Numbers are added for summing up specific items (3/12) or numbering objects (5/8). SN (2) and icon (3) rarely appear.
Horizontal arrangements are commonly used (1 and 2/5–12), although horizontal/vertical arrangements are infrequent (3 and 4). Drawings are commonly based on repeated pictures (5–8/12), shapes (1–3/9–11) and words (4), mainly with color variations (1/2/4/6/8) and/or variations in shape (8/11) and size (1). Bilateral (two-sided) designs are occasionally used for showing sonic dualities (1–3).
Sonic components represented in the above examples include musical form, pitch, timbre, contour, variation, and texture. Musical form is often represented through patterning, when the amount of repeated graphic items correspondingly represents the amount of musical parts, whether correctly (4–8/11 and 12) or incorrectly 7 (3/9–10). Other sonic features depend on the musical stimulus. The most prominent musical components in each composition are those that are represented in the drawings: pitch (1–3) and timbre (3/4) in Daniel; contour (7–8) and variation (6–8) in Chopin; and texture representations in Bartók (10–12).
Associative ideas are embedded in Holistic drawings (5–8/10/12). Cultural-based color associations are sometimes implicitly implied; for instance, pink for female and brown for male (4). Associative ideas occasionally comply with the composer’s intentions. For example, “lampposts in the darkness” (10) and “black clouds” (12) comply with Bartók’s programmatic title “Melody in the Mist.”
Statistical analyses of the data (N = 495)
In order to examine whether differences would be found between the three kinds of music in the Morphological aspects, a chi-square analyses (χ2) was conducted. Table 2 shows the frequency of the Morphological aspects by the kind of music.
Frequency (%) of the Morphological aspects by the kind of music.
p < .001.
In order to examine whether differences would be found between the three kinds of music in the Structural aspects, a chi-square analyses (χ2) was conducted. Table 3 shows the frequency of the Structural aspects by the kind of music.
Frequency (%) of the Structural aspects by the kind of music.
p < .001.
In order to examine whether differences would be found between the three kinds of music in the Conceptual aspects, a chi-square analyses (χ2) was conducted. Table 4 shows the frequency of the structural aspects by the kind of music.
Frequency (%) of the Conceptual aspects by the kind of music.
p < .001.
In order to examine whether differences would be found between the three kinds of music in the number of forms drawn by the children, a one-way analysis of variance (ANOVA) was conducted. The independent variable was the kind of music and the dependent variable was the number of forms drawn by the children. Significant differences were found between the three kinds of music in the number of forms drawn by the children, F(2, 388) = 44.30, p < .001,
Summary of findings
Findings reveal significant differences between Daniel, Chopin, and Bartók drawings in Morphological, Structural, and Conceptual aspects.
Morphological aspects
An overwhelming majority of the drawings (over 85%) were monochromes. Daniel rendered the highest percentage of monochromes (91.8%), compared to Bartók (90%) and Chopin (85.5%). Bartók yielded a higher percentage of duochromes (6.3%) than Chopin (3.6%) and Daniel (2.4%) and Chopin yielded nearly twice as many polychromes (10.9%) as Daniel (5.9%) and Bartók (3.9%).
A significant difference was found between the amount of shapes, pictures, text, numeracy, and SN for each musical composition. Bartók yielded the highest percentage of shapes (70%), compared to Chopin (54.5%) and Daniel (40.6%); Chopin (71.5%) and Daniel (63.5%) yielded significantly more pictures than Bartók (30%); text often appeared in Daniel (51.2%), but rarely in Chopin (12.1%) and Bartók (10.6%); the use of numeracy was highest in Chopin (55.2%), compared to Daniel (33.3%) and Bartók (39.4%). SN was rare throughout; still Daniel yielded more SN (9.4%) than Chopin (3.6%) and Bartók (0.6%).
Icons were rare throughout, with insignificant difference between Daniel (9.4%), Chopin, (5.5%) and Bartók (4.4%).
Structural aspects
Horizontality was most prominent throughout the data, with a higher percentage in Chopin (69.1%) and Bartók (68.8%) drawings, compared to Daniel (46.5%). Daniel yielded significantly more horizontal/vertical arrangements (21.8%), compared to Bartók (6.9%) and Chopin (4.2%). Circular, vertical, and random directions were rare throughout, with more random designs for Bartók (11.3%), compared to Chopin (3%) and Daniel (2.4%), and more vertical arrangements for Chopin (7.3%), compared to Bartók (3.8%) and Daniel (1.2%). Circular designs appeared in Daniel drawings only (1.8%).
The highest percentage of repeated shapes (patterning) was found in Bartók (70%), compared to Chopin (54.5%) and Daniel (40.6%) drawings.
Symmetrical bilateral (two-sided) designs were generally seldom; yet, Daniel yielded significantly more bilateral arrangements (25.9%) than Chopin (11.5%) and Bartók (2.5%).
Conceptual aspects
Associative (program) responses were significantly more in Chopin (61.8%) than in Bartók (23.8%) and Daniel (14.1%). Associative responses in Chopin drawings were mostly embedded in Holistic productions (61.2%). Bartók (14.4%) and Daniel (5.3%) elicited a small amount of Holistic productions.
Conclusion
Significant Morphological differences—in color, shapes, pictures, text, numeracy, and SN—were found in the reactions to each of the three musical compositions: Daniel, Chopin, and Bartok. The “icon” reaction, however, was similarly low for the three compositions. This shows that except for “icon,” all Morphological responses were significantly influenced by the type of musical stimuli presented. It also shows that the differences in all the Morphological aspects (except for “icon”) were statistically significant (p < .001).
The statistical analysis of the Structural reaction indicates significant differences in reactions to each of the three compositions. This shows that all Structural responses—both graphical directionalities and overall designs (patterning and bilateral symmetry)—were influenced by the type of musical stimuli presented and that differences in Structural aspects were statistically significant (p < .001).
The statistical analysis of the Conceptual categories indicated significant differences in the productions of the three compositions, showing that Conceptual categories—sonic, programmatic, and holistic—were influenced by the type of musical stimuli presented, and that differences in Conceptual aspects were statistically significant (p < .001).
Discussion of results
The present study indicates that different compositions evoke significantly different Morphological, Structural, and Conceptual responses. These findings are in accord with earlier studies (e.g., Küssner, 2017; Reybrouck, Verschaffel, & Lauwerier, 2009), which showed that children may employ several approaches that seem most appropriate, given the nature of the musical stimuli.
In recording music from different historical periods, children employ a large variety of symbol systems and exploit literacy knowledge acquired by schooling (i.e., written language and numeracy). In addition, although in limited quantities, children employ rather sophisticated symbols, such as SN and icons. This findings are consistent with previous researchers (e.g., Barrett, 1997; Upitis, 1990b), who found a large variety of symbols in children’s audiovisual productions.
The dominance of monochromes in this study may be due to second-grade-school writing rules in Israel, characterized by the use of a uniform color. Duochromes occasionally showed an innate intention to express sonic and/or associative dualities. Children who produced polychromes used colors in inventive ways to represent sonic elements, including form, timbre, pitch, and texture. Thus, in agreement with Galeyev (2000), this study indicates that sound–color synesthesia is a normative phenomenon among listeners.
In former studies, notations were usually classified according to type of symbol, such as icons, discrete marks and words, and the kind of information notated, such as rhythm, pitch, or both (e.g., Barrett, 1997; Davidson & Scripp, 1988). The present study added the Structural analysis that revealed the way children transfer auditory stimuli into overall visual designs. The Structural exploration revealed the dominance of horizontality, which implies assimilation of writing conventions. Symmetrical bilateral designs showed an innate intention to convey associative contrasts (i.e., light vs. darkness) and/or sonic alternations (i.e., cluster-chords and unison).
Conceptual responses reflected the most salient musical features that stood out in each composition, while latent musical parameters went unnoticed. Thus, Chopin’s prelude, that has a clear symmetrical form, evoked the highest amount of correct form representations and the smallest amount of “unique” responses; Bartok’s “Melody in the Mist” drew attention to form due to prominent texture contrasts, yet inaccurate form representations appeared in profusion, perhaps due to its asymmetrical phrase construction. Form representation in Daniel was lowest, probably because timbre exchanges are less noticeable than texture and contour alternations, as in Bartók and Chopin, respectively.
Chopin’s and Bartok’s piano music stimulated more associations than the medieval vocal choral. One might speculate that wordless instrumental music evokes more associations than vocal music with words.
“Audiovisuology” in the curriculum
Music and visual arts are almost inseparable in today’s media-oriented society as Daniels, Naumann, and Thoben (2015) assert: The coupling of images and sounds has become as ubiquitous as it is inescapable. Through audiovisual technology, not only hearing and seeing, but also the aesthetics, technology, and economy of the visual and the auditory have become connected with one another in multifaceted ways. (p. 5)
By means of audiovisual experiments, music teachers may encourage children to experience the interconnectedness and interrelationships among the arts, thus promoting art integration, fusion, and synesthesia, which is considered an essential feature of art thinking (e.g., Bresler, 1995; Galeyev, 1999). According to Bresler (1995), art integration is the teacher’s responsibility and is left to his initiative, imagination, and resourcefulness (pp. 5–6).
“Musical Graphics” becomes an exciting activity when teachers place emphasis on the child’s individual freedom of thought; when teachers function not only as instructors but also as facilitators, and let children construct their own understanding of the music to be drawn. In this way, children understand that their creative thinking and their own interpretations are important. A student-centered environment, which allows children become accountable for their own audiovisual art, may build a foundation for subjective aesthetic development. Their output provides insights into different Morphological and Structural styles and a variety of Conceptual interpretations.
Different musical works from different historical periods have different effects on children’s audiovisual styles. Teachers may look for and expect different Morphological priorities given to different musical works. A variety of musical forms evoke a variety of Structural arrangements. Given the nature of the musical stimuli, children perceive and visualize the most prominent sonic components, and ignore or fail to notice latent sound events; Also, certain compositions raise more associations than others.
In summary, experiencing “audiovisuology” in the curriculum is one way of promoting art integration, artistic reciprocity, and synesthesia; honing individual inter-artistic creativity and aesthetic development; and mediating cross-modal experiences.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
