To support research on artificial social intelligence for successful teams (ASIST), an urban search and rescue task (USAR) was simulated within Minecraft to serve as a Synthetic Task Environment (STE). The goal for the development of the present STE was to create an environment that provides ample opportunities to allow ASI agents to demonstrate the theory of mind by making inferences and predictions of humans’ states and actions in the USAR task environment, and in the future to intervene to improve teamwork in real-time. This paper describes the STE design background, design potentials and considerations, rich data collection opportunities, and potential usage for more broad research.
AdamsJ. A. (2005). Human-Robot Interaction Design: Understanding User Needs and Requirements. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 49(3), 447–451. https://doi.org/10.1177/154193120504900349
2.
BaltrusaitisT.ZadehA.LimY. C.MorencyL. P. (2018, May). Openface 2.0: Facial behavior analysis toolkit. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) (pp. 59-66). IEEE.
3.
BartlettC. E.CookeN. J. (2015). Human-Robot Teaming in Urban Search and Rescue. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 59(1), 250–254. https://doi.org/10.1177/1541931215591051
BrehmerB.DörnerD. (1993). Experiments with computer-simulated microworlds: Escaping both the narrow straits of the laboratory and the deep blue sea of the field study. Computers in Human Behavior, 9(2-3), 171–184. https://doi.org/10.1016/07475632(93)90005-D
6.
BurkeJ.MurphyR.CooverM.RiddleD. (2004). Moonlight in Miami: A field study of human-robot interaction in the context of an urban search and rescue disaster response training exercise. Human-Computer Interaction, 19(1-2), 85–116. https://doi.org/10.1207/s15327051hci1901&2_5
7.
ChakrabortiT.KambhampatiS.ScheutzM.ZhangY. (2017). AI Challenges in Human-Robot Cognitive Teaming. ArXiv. https://arxiv.org/abs/1707.04775v2
8.
CookeN.DemirM.HuangL. (2020, July). A Framework for Human-Autonomy Team Research. In International Conference on Human-Computer Interaction (pp. 134-146). Springer, Cham. https://doi.org/10.1007/978-3-030-49183-3_11
9.
CookeN. J.ShopeS. M. (2004). Designing a synthetic task environment. In SchiflettS. G.ElliottL. R. (Eds.), Scaled worlds: Development, validation, and applications (1st ed., pp. 263–278). Aldershot: Ashgate.
10.
CookeN. J.RiveraK.ShopeS. M.CaukwellS. (1999). A Synthetic Task Environment for Team Cognition Research. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 43(3), 303–308. https://doi.org/10.1177/154193129904300337
11.
FrithC. D.FrithU. (2006). How we predict what other people are going to do. Brain Research, 1079(1), 36-46.
GardnerH. (2011). Frames of mind the theory of multiple intelligences (3rd ed.). Basic Books.
14.
GuznovS.MatthewsG.FunkeG.DukesA. (2011). Use of the RoboFlag synthetic task environment to investigate workload and stress responses in UAV operation. Behavior research methods, 43(3), 771–780. https://doi.org/10.3758/s13428-011-0085-9
15.
HampQ.ReindlL.GüthlinD. (2014). Decision-making behaviour during urban search and rescue: a case study of Germany. Disasters, 38(1), 84–107. https://doi.org/10.1111/disa.12035
16.
IrwinE.GuestR.RahmanM. (2016). Risk for First Responders Due to Cognitive Workload and Communication Loss. In Advances in Safety Management and Human Factors (pp. 297310). Springer, Cham. https://doi.org/10.1007/978-3-319-419299_28
17.
HuangL.FreemanJ.CookeN.BuchananV.WoodM.FreimanM.Colonna-RomanoJ.DemirM. (2020). Experiment 1 Study Preregistration. https://doi.org/10.17605/OSF.IO/ZWAU9
LemattaG. J.ColemanP. B.BhattiS. A.ChiouE. K.McNeeseN. J.DemirM.CookeN. J. (2019). Developing Human-Robot Team Interdependence in a Synthetic Task Environment. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 63(1), 1503–1507. https://doi.org/10.1177/1071181319631433
20.
MarklundB.BacklundP.JohannessonM. (2013). Children’s collaboration in emergent game environments. Proceedings of the International Conference on the Foundations of Digital Games, 8(1), 306–313. https://www.doi.org/10.1007/978-3-030-155698_8
21.
MarshallS. P. (2009, July). What the eyes reveal: measuring the cognitive workload of teams. In International Conference on Digital Human Modeling (pp. 265-274). Springer, Berlin, Heidelberg.
22.
MartinE.LyonD. R.SchreiberB. T. (1998). Designing Synthetic Tasks for Human Factors Research: An Application to Uninhabited Air Vehicles. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 42(1), 123–127. https://doi.org/10.1177/154193129804200128
NebelS.SchneiderS.BeegeM.KoldaF.MackiewiczV.ReyG.D. (2017). You cannot do this alone! Increasing task interdependence in cooperative educational videogames to encourage collaboration. Education Technology Research and Development65, 993–1014. https://doi.org/10.1007/s11423-0179511-8
26.
NebelS.SchneiderS.ReyG. D. (2016). From duels to classroom competition: Social competition and learning in educational videogames within different group sizes. Computers in Human Behavior, 55(A), 384–398. https://doi.org/10.1016/j.chb.2015.09.035
27.
OrlikowskiM.BongartzR.ReddersenA.ReuterJ.PfeifferT. (2013). Springen in der Virtuellen Realität: Analyse von Navigationsformen zur Überwindung von Höhenunterschieden am Beispiel von MinecraftVR. In Virtuelle und Erweiterte Realität 10. Workshop der GI-Fachgruppe VR/AR. https://pub.unibielefeld.de/record/2623948
StatheropoulosM.AgapiouA.PallisG.MikediK.KarmaS.VamvakariJ.DandoulakiM.AndritsosF.ThomasC.L. (2014). Factors that affect rescue time in urban search and rescue (USAR) operations. Natural Hazards, 75, 57-69. https://doi.org/10.1007/s11069-014-1304-3
30.
WhitmanJ.ZevallosN.TraversM.ChosetH. (2018). Snake Robot Urban Search After the 2017 Mexico City Earthquake. 2018 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), 1-6. https://doi.org/10.1109/SSRR.2018.8468633
