Understanding AI-Generated Experiments in Tourism: Replications Using GPT Simulations

Abstract

The present work explores whether the generative pre-trained transformers (GPT) can complement empirical research in tourism as the GPT extends beyond commercial applications. In particular, we utilized OpenAI’s Python API to interact with the GPT-3.5-turbo. Using GPT as a special subject, we coined AI-generative study (AGS) to validate key findings of 16 scenario-based experiments published in leading journals of tourism and hospitality in two studies. This research contributes to the literature by delineating a new methodology that opens a forum for discussion on alternative means of conducting tourism research. Future studies could also utilize GPT and the ability of generative AI for tourism research in terms of pilot-/pre-testing and cross-validation. In conclusion, we recommend that GPT-generated results should serve primarily as preliminary findings and must be corroborated by data from actual human participants, thus providing converging evidence to support the corresponding research conclusions.

Keywords

GPT AI-generative study scenario-based experiments methodology tourism research

Get full access to this article

View all access options for this article.

References

Achiam

Adler

Agarwal

Ahmad

Akkaya

Aleman

F. L.

Almeida

Altenschmidt

Altman

Anadkat

Avila

Babuschkin

Balaji

Balcom

Baltescu

Bao

Bavarian

Belgum

Bello

Zoph

(2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774. https://doi.org/10.48550/arXiv.2303.08774

Ali

Abuhmed

El-Sappagh

Muhammad

Alonso-Moral

J. M.

Confalonieri

Guidotti

Del Ser

Díaz-Rodríguez

Herrera

(2023). Explainable artificial intelligence (XAI): What we know and what is left to attain trustworthy artificial intelligence. Information Fusion, 99, 101805. https://doi.org/10.1016/j.inffus.2023.101805

American Psychological Association. (2002). Ethical principles of psychologists and code of conduct. American Psychologist, 57(12), 1060–1073. https://doi.org/10.1037/0003-066X.57.12.1060

Argyle

L. P.

Busby

E. C.

Fulda

Gubler

J. R.

Rytting

Wingate

(2023). Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3), 337–351. https://doi.org/10.1017/pan.2023.2

Baktash

Huang

de la Mora Velasco

Jahromi

M. F.

Bahja

(2023). Agent-based modelling for tourism research. Current Issues in Tourism, 26(13), 2115–2127. https://doi.org/10.1080/13683500.2022.2080648

Bauer

von Zahn

Hinz

(2023). Expl (AI) ned: The impact of explainable artificial intelligence on users’ information processing. Information Systems Research, 34(4), 1321–1814. https://doi.org/10.1287/isre.2023.1199

Binz

Schulz

(2023). Using cognitive psychology to understand GPT-3. Proceedings of the National Academy of Sciences, 120(6), e2218523120. https://doi.org/10.1073/pnas.2218523120

Bonabeau

(2002). Agent-based modeling: Methods and techniques for simulating human systems. Proceedings of the National Academy of Sciences, 99(suppl_3), 7280–7287. https://doi.org/10.1073/pnas.082080899

Brand

Israeli

Ngwe

(2023). Using LLMs for market research (Harvard Business School Marketing Unit Working Paper No. 23-062). Available at SSRN. https://dx-doi-org-s.web.bisu.edu.cn/10.2139/ssrn.4395751

10.

Bubeck

Chandrasekaran

Eldan

Gehrke

Horvitz

Kamar

Lee

Y. T.

Lundberg

Nori

Palangi

Ribeiro

M. T.

Zhang

(2023). Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712. https://doi.org/10.48550/arXiv.2303.12712

11.

Carvalho

Ivanov

(2024). ChatGPT for tourism: Applications, benefits and risks. Tourism Review, 79(2), 290–303. https://doi.org/10.1007/978-3-319-01669-6_926-1

12.

Chen

Liu

T. X.

Shan

Zhong

(2023). The emergence of economic rationality of GPT. Proceedings of the National Academy of Sciences, 120(51), e2316205120. https://doi.org/10.1073/pnas.2316205120

13.

Chowdhury

M. N. U. R.

Haque

(2023, June). ChatGPT: its applications and limitations. In 2023 3rd International Conference on Intelligent Technologies (CONIT) (pp. 1–7). IEEE. https://doi.org/10.1109/CONIT59222.2023.10205621

14.

Dillion

Tandon

Gray

(2023). Can AI language models replace human participants? Trends in Cognitive Sciences, 27(7), 597–600. https://doi.org/10.1016/j.tics.2023.04.008

15.

Dogru

Line

Mody

Hanks

Abbott

Acikgoz

Assaf

Bakir

Berbekova

Bilgihan

Dalton

Erkmen

Geronasso

Gomez

Graves

Iskender

Ivanov

Kizildag

Lee

Zhang

(2023). Generative artificial intelligence in the hospitality and tourism industry: Developing a framework for future research. Journal of Hospitality & Tourism Research. Advance online publication. https://doi.org/10.1177/10963480231188663

16.

Dwivedi

Y. K.

Kshetri

Hughes

Slade

E. L.

Jeyaraj

Kar

A. K.

Koohang

Raghavan

Ahuja

Albanna

ahmad Albashrawi

Balakrishnan

Barlette

Basu

Bose

Brooks

Buhalis

Carter

Chowdhury

Wright

(2023). “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642

17.

Eloundou

Manning

Mishkin

Rock

(2023). Gpts are gpts: An early look at the labor market impact potential of large language models. arXiv preprint arXiv:2303.10130. https://doi.org/10.1126/science.adj0998

18.

Grossmann

Feinberg

Parker

D. C.

Christakis

N. A.

Tetlock

P. E.

Cunningham

W. A.

(2023). AI and the transformation of social science research. Science, 380(6650), 1108–1109. https://doi.org/10.1126/science.adi1778

19.

Hahnel

U. J.

Chatelain

Conte

Piana

Brosch

(2020). Mental accounting mechanisms in energy decision-making and behaviour. Nature Energy, 5(12), 952–958. https://doi.org/10.1038/s41560-020-00704-6

20.

Hair

J. F.

Black

W. C.

Babin

B. J.

Anderson

R. E.

Tatham

R. L.

(2006). Multivariate data analysis (6th ed.). Pearson.

21.

Hartmann

Netzer

(2023). Natural language processing in marketing. In Sudhir

Toubia

(Eds.), Artificial intelligence in marketing (vol.20, pp. 191–215). Emerald Publishing Limited.

22.

Hayes

A. F.

(2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach (pp. 34–40). Guilford Press.

23.

Salimans

Gritsenko

Chan

Norouzi

Fleet

D. J.

(2022). Video diffusion models. Advances in Neural Information Processing Systems, 35, 8633–8646.

24.

Horton

J. J.

(2023). Large language models as simulated economic agents: What can we learn from homo silicus? arXiv preprint arXiv:2301.07543. https://doi.org/10.3386/w31122

25.

Hossain

M. I.

Oppewal

Tojib

(2023). High expectations: How tourists cope With disappointing vacation experiences. Journal of Travel Research, 62(5), 989–1009. https://doi.org/10.1177/00472875221109828

26.

Islam

Ahmed

(2024, May). Gemini-the most powerful LLM: myth or truth [Conference session]. 2024 5th Information Communication Technologies Conference (ICTC) (pp. 303–308). IEEE. https://doi.org/10.1109/ICTC61510.2024.10602253

27.

Johnson

Obradovich

(2023). Evidence of behavior consistent with self-interest and altruism in an artificially intelligent agent. arXiv preprint arXiv:2301.02330. https://doi.org/10.21203/rs.3.rs-2448822/v1

28.

Kim

Jhang

Kwon

Baah

N. G.

(2023). Experimental design studies in hospitality and tourism research: constructive recommendations. International Journal of Contemporary Hospitality Management, 35(10), 3570–3591. https://doi.org/10.1108/IJCHM-06-2022-0720

29.

Koralus

Wang-Maścianica

(2023). Humans in humans out: On GPT converging toward common sense in both success and failure. arXiv preprint arXiv:2303.17276. https://doi.org/10.48550/arXiv.2303.17276

30.

Kramer

A. D.

Guillory

J. E.

Hancock

J. T.

(2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National academy of Sciences of the United States of America, 111(24), 8788–8790. https://doi.org/10.1073/pnas.1320040111

31.

Lakens’s

(2022). Sample size justification. Collabra: Psychology, 8(1), 33267. https://doi.org/10.1525/collabra.33267

32.

J. G.

Quoidbach

Gino

Chakroff

Maddux

W. W.

Galinsky

A. D.

(2017). The dark side of going abroad: How broad foreign experiences increase immoral behavior. Journal of Personality and Social Psychology, 112(1), 1–16. https://doi.org/10.1037/pspa0000068

33.

Mellon

Bailey

Scott

Breckwoldt

Miori

(2022). Does GPT-3 know what the most important issue is? Using large language models to code open-text social survey responses at scale (December 22, 2022). https://doi.org/10.2139/ssrn.4310154

34.

Miao

Yang

F. X.

(2023). Text-to-image AI tools and tourism experiences. Annals of Tourism Research, 102, 103642. https://doi.org/10.1016/j.annals.2023.103642

35.

Miotto

Rossberg

Kleinberg

(2022). Who is GPT-3? An exploration of personality, values and demographics. arXiv preprint arXiv:2209.14338. https://doi.org/10.18653/v1/2022.nlpcss-1.24

36.

OpenAI. (2022). Introducing ChatGPT. https://openai.com/blog/chatgpt

37.

Park

J. S.

O’Brien

Cai

C. J.

Morris

M. R.

Liang

Bernstein

M. S.

(2023, October). Generative agents: Interactive simulacra of human behavior [Conference session]. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (pp. 1–22). https://doi.org/10.1145/3586183.3606763

38.

Santurkar

Durmus

Ladhak

Lee

Liang

Hashimoto

(2023, July). Whose opinions do language models reflect? [Conference session]. International Conference on Machine Learning. 29971–30004. https://doi.org/10.48550/arXiv.2303.17548

39.

Shannon

C. E.

(1948). A mathematical theory of communication. The Bell System Technical Journal, 27(3), 379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x

40.

Pan

Huang

(2023). How does destination crisis event type impact tourist emotion and forgiveness? The moderating role of destination crisis history. Tourism Management, 94, 104636. https://doi.org/10.1016/j.tourman.2022.104636

41.

Tang

Nawijn

(2023). How destination social responsibility shapes resident emotional solidarity and quality of life: Moderating roles of disclosure tone and visual messaging. Journal of Travel Research, 62(1), 105–120. https://doi.org/10.1177/00472875211056683

42.

Yang

Swanson

S. R.

(2023). The influence of motive attributions for destination social responsibility on residents’ empowerment and quality of life. Journal of Travel Research, 62(8), 1737–1754. https://doi.org/10.1177/00472875221138790

43.

Sun

Law

Zhang

(2020). An updated review of tourism-related experimental design articles. Asia Pacific Journal of Tourism Research, 25(7), 710–720. https://doi.org/10.1080/10941665.2019.1699128

44.

Taecharungroj

(2023). “What Can ChatGPT Do?” analyzing early reactions to the innovative AI Chatbot on Twitter. Big Data and Cognitive Computing, 7(1), 35. https://doi.org/10.3390/bdcc7010035

45.

Wallinger

Grundner

Majic

Lampoltshammer

T. J.

(2023). Agent-based modelling for sustainable tourism [Paper presentation]. Paper presented at the Information and Communication Technologies in Tourism 2023: Proceedings of the ENTER 2023 eTourism Conference, January 18–20, 2023. https://doi.org/10.1007/978-3-031-25752-0_40

46.

Wong

I. A.

Lian

Q. L.

Sun

D. N.

(2023). Autonomous travel decision-making: An early glimpse into ChatGPT and generative AI. Journal of Hospitality and Tourism Management, 56, 253–263. https://doi.org/10.1016/j.jhtm.2023.06.022

47.

Wong

I. A.

Wan

Y. K. P.

Huang

G. I.

(2021). Green event directed pro-environmental behavior: an application of goal systems theory. Journal of Sustainable Tourism, 29(11–12), 1948–1969. https://doi.org/10.1080/09669582.2020.1770770

48.

Xiong

Wong

I. A.

Yang

F. X.

(2021). Are we behaviorally immune to COVID-19 through robots? Annals of Tourism Research, 91, 103312. https://doi.org/10.1016/j.annals.2021.103312

49.

Zhang

(2021). A commentary of GPT-3 in MIT technology review 2021. Fundamental Research, 1(6), 831–833. https://doi.org/10.1016/j.fmre.2021.11.011

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB