This article has the following two overarching aims. First, it traces the development of multimodal discourse analysis and sets out its main descriptive and analytical parameters; in doing so, the article highlights the specific advantages which the multimodal approach has to offer and exemplifies its application. The article also argues that the hierarchical arrangement of different semiotics (in the way common sense construes this) should not be lost from sight. Second, and related to this last point, the article will advance a complementary perspective to that of multimodality: resemiotization. Resemiotization is meant to provide the analytical means for (1) tracing how semiotics are translated from one into the other as social processes unfold, as well as for (2) asking why these semiotics (rather than others) are mobilized to do certain things at certain times. The article draws on a variety of empirical data to exemplify these two perspectives on visual communication and analysis.
Adorno, T.
and Horkheimer, M. (1993[1944]) ‘The Culture Industry: Enlightenment as Mass Deception’, in S. During (ed.) The Cultural Studies Reader, pp. 29-43. London: Routledge.
2.
Althusser, L.
(1971) ‘Ideology and Ideological State Apparatuses: Notes Towards an Investigation’, in L. Althusser (ed.) Lenin and Other Essays, pp. 121-173. London: New Left Books.
3.
Bakhtin, M.M.
(1981) The Dialogic Imagination. Austin: University of Texas Press.
4.
Bakhtin, M.M.
(1986) Speech Genres and Other Late Essays, trans. V.W. McGee. Austin: University of Texas Press.
5.
Barthes, Roland
(1977) Image-Music-Text, trans. Stephen Heath. London: Fontana.
6.
Bateson, G.
(1973) Steps to an Ecology of Mind. New York: Granada.
7.
Bernstein, B.
(ed.) (1973) Class, Codes and Control, Vol. 2: Applied Studies Towards a Sociology of Language. London: Routledge & Kegan Paul.
8.
Bernstein, B.
(1990) The Structure of Pedagogic Discourse: Class, Codes and Control, Vol. VI. London: Routledge.
9.
Berry, M.
(1981) ‘Systemic Linguistics and Discourse Analysis: A Multilayered Approach to Exchange Structure’, in M. Coulthard and M. Montgomery (eds) Studies in Discourse Analysis, pp. 120-145. London: Routledge & Kegan Paul.
10.
Boje, D.
(2001) Narrative Methods for Organizational & Communication Research. London: Sage.
11.
Bourdieu, P.
(1990) The Logic of Practice. Cambridge: Polity.
12.
Bremmer, J.
(1991) ‘Walking, Standing and Sitting in Ancient Greek Culture’, in Jan Bremmer and Herman Roodenburg (eds) A Cultural History of Gesture, pp. 15-35. Ithaca, NY: Cornell University Press.
13.
Burnett, R.
(1995) Cultures of Vision: Images, Media and the Imaginary. Bloomington: Indiana University Press.
Butler, F.
(1995) ‘New Demotic Typography: The Search for New Indices’, Visible Language29(1): 88-111.
16.
Clanchy, M.T.
(1993) From Memory to Written Record: England 1066-1307, 2nd edn.London: Blackwell.
17.
Coulthard, M.
(1977) Introduction to Discourse Analysis. London: Longman.
18.
De Beaugrande, R.
and Dressler, W. (1981) Introduction to Text Linguistics. London: Longman.
19.
Debray, R.
(2000) Transmitting Culture. New York: Colombia University Press.
20.
Douglas, M.
(1994) ‘The Genuine Article’, in S.H. Riggins (ed.) The Socialness of Things: Essays on the Socio-Semiotics of Objects, pp. 9-22, (Approaches to Semiotics 115). Berlin: Mouton de Gruyter.
21.
Du Gay, P.
(1996) Consumption and Identity at Work. London: Sage.
22.
Eco, U.
(1976) A Theory of Semiotics. Bloomington: Indiana University Press.
23.
Eco, U.
(1984) Semiotics and the Philosophy of Language. London: Macmillan.
24.
Eco, U.
(1990) The Limits of Interpretation. Bloomington: Indiana University Press.
25.
Eggins, S.
and Iedema, R. (1997) ‘Difference without Diversity: The Semantics of Women’s Magazines’, in R. Wodak (ed.) Gender and Discourse, pp. 165-196. Thousand Oaks, CA: Sage.
26.
Fairclough, N.
(1992) Discourse and Social Change. Cambridge: Polity.
27.
Fairclough, N.
(1995) Critical Discourse Analysis: The Critical Study of Language (Language in Social Life Series). London: Longman.
28.
Fairclough, N.
and Wodak, R. (1997) ‘Critical Discourse Analysis’, in T.A. van Dijk (ed.) Introduction to Discourse Analysis. London: Sage.
29.
Firth, J.R.
(1957) Papers in Linguistics 1934-1951. London: Oxford University Press.
Foucault, M.
(1972) The Archeology of Knowledge. New York: Pantheon.
32.
Fowler, R.
, Hodge, B., Kress, G. and Trew, T. (1979) Language & Control. London: Routledge & Kegan Paul.
33.
Gumbrecht, H.U.
and Pfeiffer, K.L. (eds) (1994) Materialities of Communication. Stanford, CA: Stanford University Press.
34.
Halliday, M.A.K.
(1973) Explorations in the Functions of Language. London: Edward Arnold.
35.
Halliday, M.A.K.
(1978) Language as Social Semiotic. London: Edward Arnold.
36.
Halliday, M.A.K.
(1982) ‘The De-Automatization of the Grammar: From Priestley’s An Inspector Calls’, in J.M. Anderson (ed.) Language Form and Linguistic Variation: Papers Dedicated to Angus McIntosh, pp. 129-159. Amsterdam: Benjamins.
37.
Halliday, M.A.K.
(1984) ‘Language as Code and Language as Behaviour: A Systemic-Functional Interpretation of the Nature and Ontogenesis of Dialogue’, in R. Fawcett, M.A.K. Halliday, S.M. Lamb and A. Makkai (eds) The Semiotics of Culture and Language: Vol. 1: Language as Social Semiotic, pp. 3-35. London: Pinter.
38.
Halliday, M.A.K.
(1994[1985]) An Introduction to Functional Grammar. London: Edward Arnold.
39.
Halliday, M.A.K.
and Hasan, R. (1976) Cohesion in English. London: Longman.
40.
Harvey, D.
(1996) Justice, Nature and the Geography of Difference. Oxford: Blackwell.
41.
Heath, C.
(1984) ‘Talk and Recipiency: Sequential Organisation in Speech and Body Movement’, in J.M. Atkinson and J. Heritage (eds) Structures of Social Action: Studies in Emotion and Social Interaction, pp. 247-265. Cambridge: Cambridge University Press.
42.
Hjelmslev, L.
(1961) Prolegomena to a Theory of Language, trans. F.J. Whitfield. Madison: University of Wisconsin Press.
43.
Hodge, B.
and Kress, G. (1988) Social Semiotics. Cambridge: Polity.
44.
Hodge, B.
and Kress, G. (1993[1979]) Language as Ideology. London: Routledge.
45.
Hodge, B.
, Kress, G. and Jones, G. (1979) ‘The Ideology of Middle Management’, in R. Fowler, B. Hodge, G. Kress and T. Trew (eds) Language & Control, pp. 81-93. London: Routledge & Kegan Paul.
46.
Hymes, D.
(1972a) ‘Toward Ethnographies of Communication: The Analysis of Communicative Events’, in P. Giglioli (ed.) Language and Social Context, pp. 21-41. Harmondsworth: Penguin.
47.
Hymes, D.
(1972b) ‘Models of Interaction of Language and Social Life’, in J. Gumperz and D. Hymes (eds) Directions in Sociolinguistics: The Ethnography of Communication, pp. 35-71. New York: Holt, Rinehart & Winston.
48.
Iedema, R. (1997) ‘Interactional Dynamics and Social Change: Planning as Morphogenesis’, Department of Linguistics, unpublished PhD thesis, Sydney, University of Sydney.
49.
Iedema, R.
(1999) ‘The Formalisation of Meaning’, Discourse & Society10(1): 49-65.
50.
Iedema, R.
(2000a) ‘Bureaucratic Planning & Resemiotisation’, in E. Ventola (ed.) Discourse and Community: Doing Functional Linguistics, pp. 47-70. Tubingen: Narr Verlag.
51.
Iedema, R.
(2000b) ‘Analysing Film and Television’, in T. van Leeuwen and C. Jewitt (eds) Handbook of Visual Analysis, pp. 183-204. London: Sage.
52.
Iedema, R.
(2001) ‘Resemiotization’, Semiotica37(1/4): 23-40.
53.
Jordan, Tim
(1999) Cyberpower: The Culture and Politics of Cyberspace and the Internet. London: Routledge.
54.
Kendon, Adam
(1990) Conducting Interaction: Patterns of Behaviour in Focused Encounters (Studies in Interactional Sociolinguistics 7). Cambridge: Cambridge University Press.
55.
Kress, G.
(1976) ‘Introduction’, in G. Kress (ed.) Halliday: System and Function in Language, pp. vii-xxi. Oxford: Oxford University Press.
56.
Kress, G.
and Ogborn, Jon (1998) ‘Modes of Representation and Local Epistemologies: The Presentation of Science in Education’, Subjectivity in the School Curriculum (Working Paper 2, Institute of Education). London: University of London.
57.
Kress, G.
and Van Leeuwen, T. (1990) Reading Images. Geelong, Vic: Deakin University Press.
58.
Kress, G.
and Van Leeuwen, T. (1996) Reading Images: The Grammar of Visual Design. London: Routledge.
59.
Kress, G.
and Van Leeuwen, T. (1998) ‘Front Pages: The (Critical) Analysis of Newspaper Layout’, in A. Bell and P. Garrett (eds) Approaches to Media Discourse, pp. 186-219. London: Blackwell.
60.
Kress, G.
and Van Leeuwen, T. (eds) (2001) Multimodality. London: Sage.
61.
Labov, W.
(1970) ‘The Study of Language in its Social Context’, Studium Generale23: 66-84.
62.
Lash, S.
(1988) ‘Discourse or Figure? Postmodernism as Regime of Representation’, Theory, Culture & Society5(2-3): 311-336.
63.
Latour, B.
(1992) ‘Where are the Missing Masses? The Sociology of a Few Mundane Artifacts’, in W.E. Bijker and J. Law (eds) Shaping Technology/Building Society: Studies in Socio-technical Change, pp. 225-258. Cambridge, MA: MIT Press.
64.
Latour, B.
(1993) ‘Ethnography of a High-Tech Case’, in P. Lemonnier (ed.) Technological Choices: Transformation in Material Cultures since the Neolithic, pp. 372-398. London: Routledge.
65.
Latour, B.
(1996) ‘On Interobjectivity’, Mind, Culture and Activity3(4): 228-245.
66.
Law, J.
and Mol, A. (1995) ‘Notes on Materiality and Sociality’, The Sociological Review:274-294.
67.
Lemke, J.
(1988) ‘Towards a Social Semiotics of the Material Subject’, in Sydney Association for Studies in Society and Culture, Working Papers 2(1-2): 1-17. Sydney: University of Sydney.
68.
Lemke, J.
(1989) ‘Semantics and Social Values’, Word40(1-2): 37-50.
69.
Lemke, J.
(1990) Talking Science: Language, Learning, and Values. Norwood, NJ: Ablex.
70.
Martin, J.R.
(1992) English Text: System and Structure. Amsterdam: John Benjamins.
71.
Martinec, R.
(2001) ‘Interpersonal Resources in Action’, Semiotica135 (1/4): 117-145.
72.
Mehan, H.
(1993) ‘Beneath the Skin and between the Ears: A Case Study in the Politics of Representation’, in S. Chaiklin and J. Lave (eds) Understanding Practice: Perspectives on Activity and Context, pp. 241-268. Cambridge: Cambridge University Press.
73.
Metz, C.
(1974) Film Language: A Semiotics of Cinema, trans. Michael Taylor. New York: Oxford University Press.
74.
Metz, C.
(1977) Essays Sémiotiques. Paris: Éditions Klincksieck.
75.
Mirzoeff, N.
(1999) An Introduction to Visual Culture. London: Routledge.
76.
Morris, C.
(1946) Signs, Language and Behaviour. New York: Prentice Hall.
77.
Müller, J.-D.
(1994) ‘The Body of the Book: The Media Transition from Manuscript to Print’, in Hans Ulrich Gumbrecht and K. Ludwig Pfeiffer (eds) Materialities of Communication, trans. William Whobrey, pp. 32-44. Stanford, CA: Stanford University Press.
78.
Olson, D.R.
(1996) ‘Language and Literacy: What Writing Does to Language and Mind’, Annual Review of Applied Linguistics16: 3-13.
79.
O’Toole, M.
(1990) ‘A Systemic-Functional Semiotics of Art’, Semiotica82-3/4: 185-209.
80.
O’Toole, M.
(1991) ‘The Bronze Horseman, Looking East’, Australian Slavonic and East European Studies5(2): 67-84.
81.
O’Toole, M.
(1994) The Language of Displayed Art. London: Leicester University Press.
82.
Pêcheux, M.
(1982) Language, Semantics and Ideology. London: Macmillan.
83.
Preziosi, D.
(1984) ‘Relations between Environmental and Linguistic Structure’, in R. Fawcett, M.A.K. Halliday, S. Lamb and A. Makkai (eds) The Semiotics of Language and Culture Vol. 2: Language and Other Semiotic Systems of Culture, pp. 48-68. London: Pinter.
84.
Renkema, J.
(1993) Discourse Studies: An Introductory Textbook. Amsterdam: John Benjamins.
85.
Rogoff, I.
(1998) ‘Studying Visual Culture’, in Nicholas Mirzoeff (ed.) The Visual Culture Reader, pp. 14-26. London: Routledge.
86.
Rose, N.
(1999) Powers of Freedom: Reframing Political Thought. Cambridge: Cambridge University Press.
87.
Rossi-Landi, F.
(1992[1979]) Between Signs and Non-Signs (Critical Theory: Interdisciplinary Approaches to Language, Discourse and Ideology, Vol. 10, ed. Susan Petrilli). Amsterdam: John Benjamins.
88.
Saint-Martin, F.
(1995) ‘Visual and Verbal Semantics’, in T.A. Sebeok and J. Umiker-Sebeok (eds) Advances in Visual Semiotics: The Semiotic Web 1992-93 (Approaches to Semiotics 118), pp. 375-401. Berlin: Mouton de Gruyter.
89.
Sarangi, S.
(1998) ‘Rethinking Recontextualisation in Professional Discourse Studies: An Epilogue’, in Per Linell and Srikant Sarangi (eds) Discourse across Professional Boundaries, special issue of Text 18(2): 301-318.
90.
Schatzki, T.
(1996) Social Practices: A Wittgensteinian Approach to Human Activity and the Social. Cambridge: Cambridge University Press.
91.
Schiffrin, D.
(1994) Approaches to Discourse. Oxford: Blackwell.
92.
Sebeok, T.A.
(1994) An Introduction to Semiotics. London: Pinter.
93.
Sellers, Susan
(1995) ‘How Long Has This Been Going On? Harper’s Bazaar, Funny Face and the Construction of the Modernist Woman’, Visible Language29(1): 13-35.
94.
Sinclair, J.
and Coulthard, M. (1975) Towards an Analysis of Discourse. Oxford: Oxford University Press.
95.
Sinclair, J.
and Coulthard, M. (1992[1975]) ‘Towards an Analysis of Discourse’, in M. Coulthard (ed.) Advances in Spoken Discourse Analysis, pp. 1-34. [Reprint of central chapter of Sinclair and Coulthard, 1975.] New York: Routledge.
96.
Stenglin, M.
and Iedema, R. (2000) ‘How to Analyse Visual Images: A Guide for TESOL Teachers’, in A. Burns and C. Coffin (eds) Analysing English in a Global Context: A Reader, pp. 194-208. London: Routledge.
97.
Tannen, D.
(1984) Conversational Style: Analysing Talk among Friends. Norwood, NJ: Ablex.
98.
Thibault, P.
(1991a) Social Semiotics as Praxis: Text, Social Meaning Making, and Nabokov’s Ada. Minneapolis: University of Minnesota Press.
99.
Thibault, P.
(1991b) ‘Grammar, Technocracy, and the Noun: Technocratic Values and Cognitive Linguistics’, in E. Ventola (ed.) Functional and Systemic Linguistics: Approaches and Uses (Trends in Linguistics Vol. 55), pp. 281-305. Berlin: Mouton de Gruyter.
100.
Thibault, P.
(2000) ‘The Multimodal Transcription of a Television Advertisement: Theory and Practice’, in A. Baldry (ed.) Multimodality and Multimediality in the Distance Learning Age, pp. 311-385. Campobasso: Palladino Editore.
101.
Titcher, S.
, Wodak, R., Meyer, M. and Vetter, E. (1998) Methoden der Textanalyse. Wiesbaden: Westdeutscher Verlag.
102.
Van Dijk, T.
(1977) Text and Context: Explorations in the Semantics and Pragmatics of Discourse. London: Longman.
103.
Van Leeuwen, T.
(1984) ‘Impartial Speech: Observations on the Intonation of News Readers’, Australian Journal Cultural Studies2(1): 84-98.
104.
Van Leeuwen, T.
(1999) Speech, Sound, Music. London: Macmillan.
105.
Van Leeuwen, T.
(2000) ‘The Greenwich Time Museum: A Social Semiotic Analysis’, paper given at the Australian Museum, Sydney, September.
106.
Van Leeuwen, T.
and Jewitt, C. (eds) (2000) Handbook of Visual Analysis. London: Sage.
107.
Wilden, A.
(1980) System and Structure: Essays in Communication and Exchange, 2nd edn.London: Tavistock.
108.
Wilden, A.
(1987) The Rules Are No Game: The Strategy of Communication. London: Routledge & Kegan Paul.
109.
Williamson, J.
(1995) ‘Embodiment of Human Identity: Detecting and Interpreting Hidden Narratives in Twentieth-Century Design History’, Visible Language29(1): 36-71.
110.
Wilson, R.
(1997) Bringing Them Home: A Guide to the Findings and Recommendations of the National Inquiry into the Separation of Aboriginal and Torres Strait Islander Children from their Families. Sydney: Human Rights and Equal Opportunity Commission.
