Abstract
Wayang kulit Kelantan, a preeminent shadow play in Malaysia is currently threatened with imminent extinction. At the time of this writing, the efforts made to preserve this traditional cultural heritage are insufficient. Therefore it is clear that with the current situation in Malaysia and the level of official support, wayang kulit Kelantan is unlikely to last long. To overcome this predicament, on the one hand, some researchers have suggested to remediate wayang kulit Kelantan into digital screen. On the other hand, there are also researchers who attested that there a dire need for Malaysians to promote this art form by using modern technology.
The first few sections of this paper present an overview of wayang kulit Kelatan, theories of remediation and the existing works related to the digital remediation of this art form. The author then proposes a methodological framework of using Computer Generated Imagery (CGI) to remediate the visual of wayang kulit Kelantan into digital screen. Computer Experiments and Interpretative Phenomenological Analysis (IPA) have been conducted to vindicate the veracity of the framework, with the results presented in the discussion and conclusion sections to confirm the feasibility and efficiency of the said framework.
Introduction
In an attempt to preserve and safeguard the unique traditional wayang kulit (shadow play), the United Nations Educational, Scientific and Cultural Organization (UNESCO) has awarded Indonesian wayang kulit purwa as a Masterpiece of Oral and Intangible Heritage of Humanity on 7th November 2003. The term wayang kulit combines two words: “wayang”, which can be translated as shadow, and “kulit”, which means leather. Thus wayang kulit indicates a form of shadow theatre performed with leather puppets [1].
Today, the preservation of Indonesian wayang kulit purwa has been executed in a very promising way but Malaysian wayang kulit Kelantan is threatened with imminent extinction [2]. This dying art form was one time fairly widespread and extremely popular in the state of Kelantan in Malaysia. In the 1960s, during its heyday, there were more than 300 puppeteers in Kelantan but now there are less than six [3].
The existing evidences indicate the origin of wayang kulit Kelantan to a proto-Javanese wayang kulit. However, after substantial adaptation and localisation,wayang kulit Kelantan has developed its idiosyncratic characteristics and styles [4]. Although both Indonesian wayang kulit purwa and Malaysian wayang kulit Kelantan use the same source of repertoire (Ramayana) with the same principal characters (with slight differences in their names); they are distinctively different in terms of their shadow colours, distortions and light source [5].
Wayang kulit Kelantan shows greater level of shadow
distortion compared to wayang kulit purwa due to the differences in the size of kelir (screen), distances and positions of light source. The puppets of traditional wayang kulit Kelantan depict near-naturalistic appearance and configurations, which is very different if compared to the disproportioned long necks, exorbitant long arm, thin and pointed noises puppets of Indonesian shadow puppets [6]. The existence of this traditional shadow play can be traced back to as early as 14th century, during the Majapahit period and it is possibly a developed art form due to the spread of a proto-Javanese wayang kulit purwa shadow play to the northern part of the Malay Peninsula [7]. The puppets of wayang kulit Kelantan cast colourful shadows on the screen whereas the puppets of wayang kulit purwa are opaque and only cast black shadows. Figure 1 depicts the visuals of wayang kulit Kelantan during a performance.
The visuals of wayang kulit Kelantan.
With the advent of modern entertainment technology and digital media such as television, cinema and Internet, wayang kulit Kelantan is facing the predicament of imminent extinction. In order to survive in the era of digitalisation and globalisation, it needs a new alternative media output, to be digitalised into cyber world and to be watched on the computer screen or cinema. Tan et al. from Universiti Sains Malaysia (USM) pointed out that there is a need for Malaysians to promote and provide greater accessibility to this dying art form by using modern technology such as digital media [8]. Che. Mohd. Nasir Yussof, a professional puppeteer who helms the wayang kulit Kelantan groups in National Arts Academy (ASWARA) affirmed that the digitalisation of wayang kulit Kelantan will be very helpful in delivering information and promoting this art form through internet or online social media [9]. Kaplin believes this digital form of puppetry will not mean the death of traditional puppetry, but will probably lead them to be “preserved” for their historic, spiritual or folkloric value; just like endangered species on a game preserve. Digital puppetry is a revolutionary idea, for it expands the realm of puppetry beyond all definitions that center upon the materiality of the puppets [10].
In the ensuing two sections, the author presents the theories of remediation and the existing works related to the digital remediation of this art form. They are followed by a section on the elucidation of the proposed methodological framework. Sections 4 to 7 of this paper comprise the processes of computer experiments and the conclusions drawn based on the analysis of results by using Interpretative Phenomenological Analysis (IPA).
When an old form of media blends or fuses with a new one; or vice versa, it is refers to as remediation. It has existed for centuries that can date back to at least the Renaissance paintings with linear perspectives. The renowned media theorist, Marshall McLuhan has mentioned: “The electric light is pure information. It is a medium without message, as it were, unless it is used to spell out some verbal advertisements or names” [11]. This fact, characteristics of all media, means that the “content” of any medium is always another medium”. In this paper, the author adopted the remediation theory of Marshall McLuhan, with his dictum “the medium is the message” to propose a methodological framework to remediate the visual on the tangible screen of wayang kulit Kelantan onto the virtual computer screen without losing its authenticity and identity. As what McLuhan believes, the introduction of any new form of media in a given culture can radically alters the way that members of that culture mediate between the real world and the given values available to them.
The logics of remediation bifurcate into immediacy and hypermediacy [12]. The first logic, namely immediacy refers to the remediation that aims to erase and eliminate all the trademarks or hints of the medium and thus making that medium “disappear” or “transparent”. The second logic of remediation refers to the remediation that do not aim to make the medium disappear/transparent, but rather to refashion the older medium in order to create a sense of multiplicity or hypermediacy. This logic of remediation is best expressed in web pages, desktop interface, multimedia applications, interactive programme, video games, etc., whereby videos, images, texts, animations, sound, music, etc. are represented and combined in ways to allow the audience to gain random access within the multiple media. The finished works of hypermediacy uses fragmentation, indeterminacy and heterogeneity to offer “random access” that cannot be achieved in the re-al world.
In both logics, the transparency is vital and important as it will enable the audience to know the remediated objects directly without being distracted or disturbed by the media. Simply put, a remediated work with high level of transparency is the one that is able to induce authentic or realistic feeling or experience from the audience, making them believe that the mediated objects or visuals presented to him are real and authentic. In the following section, the author presents the various works on the remediation of wayang kulit Kelantan and their appeal to the authenticity of the original art form.
Digital remediation of wayang kulit Kelantan
About three decades ago, Malaysian researchers start to combine the modern technology with the art of shadow play. For example, in 1996, under a short-term grant awarded by Universiti Malaysia Sarawak(UNIMAS), a research project known as “Wayang Virtual” was established as an experimental version of the traditional shadow play. The virtual version of the traditional shadow puppets were staged together with a virtual 3D animated figure which was controlled by a human puppeteer using a computer mouse while the visuals were juxtaposed and projected on a white screen [13].
In 2011, Dahlan Abdul Ghani from Universiti Kuala Lumpur (UniKL) has developed a prototype design on wayang kulit in computer-generated environment. However, the result from his survey shows that the Malaysian audiences prefer wayang kulit in traditional manner compared to his computer-generated one. The reason is may be because the virtual puppet created by him lacks the visual aesthetics of wayang kulit Kelantan even though his aim was to preserve this traditional shadow puppet play [14].
In 2012, a group of researchers in Universiti Utara Malaysia (UUM) has invented a prototype of an interactive digital puppetry called e-WayCool to help students to learn mathematics in Malaysian Primary School Standard Curriculum. One of the objectives of this research is to preserve the wayang kulit Kelantan. However, the virtual puppet used in e-WayCool apparently belongs to Indonesian wayang kulit purwa rather than Malaysian wayang kulit Kelantan [15].
Since July 2012, an award-winning character desig- ner- cum- founder of the Action Tintoy Studio in Malaysia, Tintoy Chua, has been collaborating with his friend, Take Huat to redesign and create a series of Star Wars puppets with the visual elements of wayang kulit Kelantan. They managed to invite a renowned puppeteer of wayang kulit Kelantan, Pak Dain, to perform a 25-minute preview of “Peperangan Bintang” (Malay for “Star Wars”) in October 2013 [16]. In this Wayang Fusion project, Chua and Take Huat have revamped and reinterpreted the famous characters in the popular 1977 Western science fiction blockbuster movie namely Star Wars Episode IV: A New Hope, which was written and directed by George Lucas. Based on Chua, one of the main objectives of this project is to promote the traditional wayang kulit Kelantan to the public, especially the younger generation who generally has lukewarm interest in watching the long-winded and slow-paced traditional performances. Nevertheless, this project seems to focus more on Star Wars rather than the traditional wayang kulit Kelantan, whereby the original repertoire as well as characters of the traditional art form have been completely abandoned and obliterated. The only ostensible traditional vestige in this project is the visual elements of puppets that have been used to compliment the characters of Star Wars.
With regard to the remediation of Malaysian wayang kulit by using digital media and technology, most of the efforts paid no heed to its original visual aesthetics. Most of them have depicted the visual of Indonesian wayang kulit which is very much different compared to the wayang kulit Kelantan. Although recently there is a shift of focus among Malaysian researchers from digitizing Indonesian wayang kulit to Malaysian wayang kulit Kelantan, but these cases still remain scant [17, 18].
It is very obvious that the mentioned remediated works lack the logic of transparent immediacy as mentioned in Section 2. Neither do they belong to the category of immediacy nor hypermediacy as they fail to recapitulate the vibe of wayang kulit Kelantan and therefore the audience can hardly related them to the original art form.
Therefore, this paper presents a methodological framework to emulate the identical visual of wayang kulit Kelantan by using Computer-Generated Imagery (CGI).
Methodological framework
Computer-generated imagery (CGI) is the application of the field of computer graphics or, more specifically, 3-dimensional computer graphics to create special effects in animations, video games, films, television programs, commercials, simulation and printed media [19].
The methodological framework of the experiment.
The objective of this paper is to propose a methodological framework to emulate the plausible visual of wayang kulit Kelantan. In the experiment, there are three important elements to be considered; (i) puppet design, (ii) screen and (iii) light source. Similarly to an actual wayang kulit Kelantan performance, these elements must be ready prior to the show. Moreover, an experienced puppeteer is needed to animate the puppets. For the experiment, first, we have to create the virtual puppet(s), a virtual screen and a virtual light source. Upon the completion of these elements, a computer animator has to animate these elements in ways like an actual puppeteer, by using computer input devices such as a computer mouse.
As shown in Fig. 2, the task of simulating the visual of wayang kulit Kelantan can be divided into four phases. The first phase involves the creation of virtual puppet(s), a virtual light source and a virtual screen. The tangible puppets have to be photographed, digitalized and “cleaned” before being transferred (or mapped) onto the virtual objects with identical shapes and sizes. The second phase is to animate these newly created virtual objects (puppets, light source and screen). The third phase is to create a virtual camera to capture the screen’s movement/vibration effect, puppets’ movement and light source’s movement/flicker- ing effect. The final phrase involves exportation of these visual elements. The author presents the detailed processes and methodologies of these phases in the following sections.
Two wayang kulit Kelantan puppets have been chosen to carry out the experiment; they are Pohon Beringin and Wak Long.
The Pohon Beringin is a highly elegant tree or leaf-shaped puppet shown during the opening and closing of all wayang kulit Kelantan performances. It is unanimously perceived as the most important puppet among all. There are two types of Pohon Beringin in wayang kulit Kelantan; the first type is filled with the arabesques design of vegetal patterns on its entire surface (Fig. 3). The second type is designed and crated with the motifs from natural environments such as birds, fish, crocodiles, elephants, monkeys, snakes, tree, branches, flowers, etc. on a shape of a large leaf, tree or mountain. Both sides of the Pohon Beringin are a mirror image of the other [20].
The Pohon Beringin puppet comes in one piece with the height of approximately 75 cm and there is no extra joint or parts within. In wayang kulit Kelantan performances, the Pohon Beringin puppet represents the World Tree or Cosmic Mountain which connects the earth and sky. It also symbolises the cosmos, with different levels of creation manifested in it. It encapsulates the essence of all things in the performances, reflecting all phenomena and objects in the natural world. In the performances of wayang kulit Kelantan, the shape of the puppet was distorted when the puppeteer presses it towards the screen, move or shake it swiftly.
The main puppets of wayang kulit Kelatan; from left: Pohon Beringin and Wak Long.
Wak Long is one of the two main clown characters in wayang kulit Kelantan performances, which is also an important comic characters not derived from the Hindu Ramayana epic [21]. The Wak Long puppet is always painted red and displays a comical characteristic with his broad, bulbous nose, round eye, pot-belly, large backside and sarong-clad torsos. Among all the wayang kulit Kelantan puppets, Wak Long puppet contains the most sophisticate joints with two movable articulated arms, movable mouth and eyebrow.
The setting of virtual wayang kulit Kelantan.
The computer-generated fire effects as the light source.
The virtual Pohon Beringin with attached dummy objects.
Area shadow size determines the softness of the edge of the shadow.
The photographs of both puppets were digitised, retouched and cleaned up by using computer graphic software. For the Wak Long puppet, the different joints and movable parts such as jaw and limbs need to be separated into different layers. In addition to it, the opacity map (grey-scale) files were also been created to specify the areas of opacity and transparency of the puppets. The pure white areas of the opacity map will allow the corresponding areas of the puppets to be totally visible, while pure black areas cause the corresponding areas of the puppets to be completely transparent. The level of transparency in the materials is controlled and determined by the level of darkness in the opacity map.
Both puppets were modeled in 3D computer graphic application (in this case the author use Autodesk 3D Studio Max 2016). The digital photos of both puppets were mapped into the virtual puppets accordingly with their opacity files.
Lastly in the modeling stage, a virtual screen was created with its identical setting in the real world (250 cm width
The traditional wayang kulit Kelantan uses paraffin oil lamp as its light source. The dramatic flickering of the lamp sets the mystical atmosphere over the performance. The shimmering soft light of the lamp casts a mysterious atmosphere over the performance area, and because of the flickering, the puppets were seen from the shadow side often appears as if they were alive and breathing. In the recent years the use of the electric light bulb for this purpose often resulted in a loss of this natural quality. In order to simulate the visual of the modern wayang kulit Kelantan performance, a virtual light bulb was added in the virtual setting. It was animated to create the swinging effect just like how a puppeteer moves the hanging light bulb during the actual performances (Fig. 4).
Apart from the virtual light bulb, the author also replaced it with a flickering fire effect in another experiment. This is to emulate the fire effects of a paraffin oil lamp. The intensity of the virtual light source was animated and artificial flickering flame effect has been created and superimposed digitally onto the final visual (Fig. 5).
Next, a series of dummy objects were created in order to control the different parts of the Pohon Beringin puppet. These dummy objects are invisible on the rendering/output and were linked with proper hierarchy in order to control the distortion of specific parts of the puppet (Fig. 6).
The Pohon Beringin puppet was animated with computer-generated tweening technique to perform an arc movement from right to left and touches the centre of the screen during the half-way of the movement. The term tweening in CGI software means the animator creates the keyframes (important frames of a sequence), such as the starting and ending position of the Pohon Beringin puppet, then the computer software will smoothly translate the object from the starting point to the ending point. The animator can amend the movement at any point, shifting keyframes back and forth to improve the timing and dynamics of a movement, or insert additional keyframes into the ‘in between’ to further refine and improve the movement.
When the Pohon Beringin puppet touches the screen, the top part of the puppet was distorted and this effect was achieved by moving and rotating the dummy object on the top part of the puppet. Throughout the arc movement of the puppet, different parts of it have been distorted by manipulating the relevant dummy objects.
Generally the computer-generated shadow has two important parameters to control the transparency of shadow and the softness of its edges. The area shadow function creates a soft edge that becomes more noticeable as the distance between the object and the shadow increases. The softness of the shadow’s edges was determined by the sizes of area shadow. The bigger size of the area shadow will make the softness of the shadow’s edge become more distinctive (Fig. 7).
Both of the parameters have been activated and adjusted in this experiment. In addition to it, the cast colourful shadows function in the materials of the puppets also need to be activated.
The movements of the virtual Wak Long puppet.
The Wak Long puppet consists of two moveable articulated arms. The puppeteer controls and moves the arms of the puppet by manipulating the sticks connected to the end of each articulated arm. Each arm of the Wak Long puppet consists of two movable parts. In virtual wayang kulit Kelantan environment, all of these moveable parts were modeled and texture-mapped using the methods discussed earlier. They were later been arranged, positioned and linked with proper axis points and hierarchy. These parts were later being rotated and animated to mimic the movements of the Wak Long puppet in an actual performance (Fig. 8).
The movements of eyebrow and mouth of a tangible Wak Long puppet were controlled by a string. The moveable eyebrow of Wak Long puppet is actually a bent bamboo attached to a string. By pulling the string downward and releasing it to its original position, the puppet’s eyebrow and mouth will move simultaneously.
The movements of the mouths, eyebrow and string of virtual Wak Long Puppet.
Using motion capture device to preserve the movements of puppeteer and puppet’s distortion.
To achieve the above mentioned visual, a virtual bent thin cylinder was modeled to operate as the eye brow of the puppet with a virtual string attached to it. The elasticity of the stick was manipulated and animated synchronously with the movement of its mouth and string; creating the visual of moveable mouth, eyebrow and string (Fig. 9).
The fly in the ointment of these experiments is that all the animations were not done in “real time”. “Real-time” in shadow play refers to a synchronicity between the puppeteer’s control and the puppet’s resultant movement [22]. Even though the animation process in virtual wayang kulit Kelantan eliminates the human errors, it also disconnects the interaction between the puppeteer (in this case, animator) and the audience. Therefore in the final experiment, human agent in the animation process has been removed with the use of a motion capture devise. A total of five reflective markers have been attached on the various parts of the Pohon Beringin puppet while the puppeteer wore the standard motion capture suit with reflective markers attached on its different parts. The movements of both puppeteer and puppets were captured and transformed into a virtual biped humanoid character, to achieve the absolute synchronicity in human movements and puppet’s distortion (Fig. 10).
With the use of motion capture device, there is no need to acquire skillful computer animator to animate the virtual puppet(s). It produces greater immediacy and transparency in puppet’s movements and distortions, and thus making the audience to believe that the computer-generated visual of wayang kulit Kelantan as authentic and real.
The author uses the Interpretative Phenomenological Analysis (IPA) to analysis and validate the results of experiments. It is a widely-used and popular approach to psychological research that involves a close examination of the experiences and meaning-making activities of a small number of participants who have been carefully culled via the purposive sampling method [23, 24].
Two renown and well-established researchers in the related field – Prof. Dr. Ghulam-Sarwar Yousof and Prof. Dr. Patricia Matusky as well as three professional puppeteers of wayang kulit Kelantan – Che. Mohd. Nasir Yussof, Mohd. Kamarulbahri bin Hussin and Rahim bin Hamzah have been chosen to answer a series of semi-structure questions during interviews. To facilitate the necessary reflexivity, the author has conducted formal and informal interviews with them separately from time to time since 2009. Besides, in order to build rapport with them and to ensure the collected data are rich, the author has accompanied the mentioned academic researcher(s) to attend several wayang kulit Kelantan performances performed by these puppeteers in Kuala Lumpur and Kelantan ever since this research started in 2009. These data were collected and analyzed systematically adhered to the rigorous procedures of IPA.
Based on the analyzed data, the results of the experiments have shown plausible and convincing visual of wayang kulit Kelantan. The differences between the visual of computer-generated wayang kulit Kelantan with the actual one are indistinguishable (or very subtle). These visual elements comprise distinctive shadow’s colours, distortions, softness of edges, generated abreast with the movements of the virtual puppets and light source.
In addition to the capabilities of visual simulation, from the experiment results, all computer-generated visual elements (i.e. the colour and intensity of light source, the softness of the edges of shadows and the transparency of the screen) are adjustable and can be animated. It is most likely impossible for any puppeteers to manipulate these elements in actual performances with the existing equipment and technologies.
The proposed methodological framework contributes
in term of time and cost in the production of wayang kulit Kelantan’s visual on digital screen. Previously, one have to spent few thousands Malaysian Ringgit to stage a wayang kulit Kelantan performance and have it recorded; but with the proposed methodological framework, an experienced computer animator with a Personal Computer (PC) with a functional computer graphic software is capable of producing the plausible visual of the art form. Moreover, it can be used as a guideline for the future digital remediation of this diminishing art form.
Conclusion
The objective of this paper is to propose a methodological framework of using CGI to emulate the visual of wayang kulit Kelantan. The results of the experiments have shown a high level of verisimilitude in terms of the quality of puppets’ movements, colours of shadows (translucency of puppets), movements of light source, distortions of puppets, softness of the edges of shadows, movements of eyebrow, mouth and string which was attached to the puppets. The author hopes the experiment results will boost more creation and adaptation of wayang kulit Kelantan’s visual in movies, animations, video games and films. It is also hoped that with the advent of computer technology, the simulation of virtual wayang kulit Kelantan will become more user-friendly and accessible. Last but not least, the author hopes to see more efforts in the creation of virtual wayang kulit Kelantan as both of them share the similarities in their visuals on screen and the author believes they can co-exist and benefit each other.
In summary, the work presented here focuses on the emulation of the visual of wayang kulit Kelantan with the use of CGI. On the one hand, the work by Ghani [14] considers only the 3D puppet modeling without referring to the setting, materials, techniques and visual of wayang kulit Kelantan. On the other hand, the work by Tan et al. [8] takes a different approach based on the visual simulation and interactive animation of a Javanese shadow play puppet by using OpenGL technique. While the present study is related to the diminishing wayang kulit Kelantan, it has successfully remediated the visual of wayang kulit Kelantan from tangible muslin screen to digital screen without compromising on its immediacy and transparency. This is an aspect that was not considered (or lacking) in other earlier works.
Future recommendation
Computer-generated Imagery (CGI) is a new body of knowledge with less than six decades of history [25]. It involves a broader spectrum to study and analyze, and the author simply cannot cover all topics in CGI and relate them with Malaysian wayang kulit here. Topics and experiments such as virtual/augmented reality, immersive technology, inverse kinematics, character rigging and interactive methods related to the remediation of wayang kulit are still very limited and worthy to be studied; therefore the author would recommend further research to be conducted, but not limited, on these areas.
Footnotes
Acknowledgments
This paper is an expanded version of my paper entitled “A Framework to Emulate the Visual of Malaysian Shadow Play with Computer-generated Imagery (CGI)” presented in the First EAI International Conference on Computer Science and Engineering, NOVEMBER 11–12, 2016, PENANG, MALAYSIA.
