Orchestrating light – Gesture-based control of smart lighting

Abstract

In this paper, a gesture-based control system for smart lighting management is introduced. The system consists of multiple depth camera sensors that track the users’ positions, movements and gestures in real-time and forward the data to the main computer where the lighting control commands are generated. The implementation was built into the ‘ResponseRoom’ R&D environment, where two different lighting controls in a real-time installation were validated with the ability to dim and control colour by hand gestures. In order to study end user preference and to discover needs and guidelines for future development of the system, a user survey on gesture-based lighting control in a meeting room application was carried out. In the survey, no predetermined gesture vocabulary or real lighting control implementation were used. Instead, the respondents could freely develop their own gesture commands. Altogether 197 gained responses are analysed. The results reveal that generally intuitive control gestures are difficult to generate as there is a rather strong variability in personal preferences. The most popular gestures in the user survey were used by 25–30% of the respondents, 19% of all user responses in the test were multimodal combinations of different gestures, verbal expressions and sounds.

Get full access to this article

View all access options for this article.

References

Dong

Sanderson

. A dynamic adaptive light field sampling approach for smart lighting control. Lighting Research and Technology 2014; 46: 593–614.

Haitz

Tsao

. Solid-state lighting: ‘the case’ 10 years after and future prospects. Physica Status Solidi A 2011; 208: 17–29.

Tsao

Saunders

Creighton

Coltrin

Simmons

. Solid-state lighting: an energy-economics perspective. Journal of Physics D: Applied Physics 2010; 43: 1–17.

Juntunen

Myöhänen

Tetri

Tapaninen

Ojalehto

Heikkinen

. Rapid prototyping method of freeform optics for LED downlighter with dynamically adjustable beam. Lighting Research and Technology 2015. Published online before print June 16, 2015. doi 1477153515591390.

Lasance

CJM

Poppe

. Future LED applications. Thermal Management for LED Applications, New York: Springer Science + Business Media, 2014, pp. 532–537.

Sanaullah

Corno

Razzak

. Autonomic goal-oriented device management for Smart Environments. Journal of Ambient Intelligence and Smart Eenvironments 2015; 7: 425–448.

Escuyer

Fontoynont

. Lighting controls: a field study of office workers’ reactions. Lighting Research and Technology 2001; 33: 77–94.

Kela

Korpipää

Mäntyjärvi

Kallio

Savino

Jozzo

Di Marca

. Accelerometer-based gesture control for a design environment. Personal and Ubiquitous Computing 2006; 10: 285–299.

Wang

Bernardos

Besada

Metola

Casar

. A gesture-based method for natural interaction in smart spaces. Journal of Ambient Intelligence and Smart Environments 2015; 7: 535–562.

10.

Oviatt

. Ten myths of multimodal interaction. Communications of the ACM 1999; 42: 74–81.

11.

Turk

. Multimodal interaction: a review. Pattern Recognition Letters 2014; 36: 189–195.

12.

Gieselmann P, Denecke M. Towards multimodal interaction with an intelligent room: Proceedings of Eurospeech, Geneva, Switzerland, September 1–4: 2003: pp. 2229–2232.

13.

Richarz J, Plötz T, Fink GA. Real-time detection and interpretation of 3D deictic gestures for interaction with an intelligent environment: Proceedings of International Conference on Pattern Recognition, ICPR, Tampa, USA, December 8–11: 2008: pp. 1–4.

14.

Chang JS, Kim HS, Kim HJ. Vision-based interface for integrated home entertainment system: Proceedings of Computer Vision in Human–Computer Interaction, ICCV, Beijing, China, October 21: 2005: pp. 176–183.

15.

Do JH, Jung JW, Jung SH, Jang H, Bien Z. Advanced soft remote control system using hand gesture: Proceedings of MICAI: Advances in Artificial Intelligence, Apizaco, Mexico, November 13–17: 2006: pp. 745–755.

16.

Kivimäki T, Vuorela T, Valtonen M, Vanhala J. Gesture control system for smart environments: Proceedings of Intelligent Environments (IE), Athens, Greece, July 16–17 2013: pp. 232–235.

17.

Kühnel

Westermann

Hemmert

Kratz

Muller

Möller

. I’m home: Defining and evaluating a gesture set for smart-home control. International Journal of Human-Computer Studies 2011; 69: 693–704.

18.

Schlomer T, Poppinga B, Henze N, Boll S. Gesture recognition with a Wii controller: Proceedings of the 2nd International Conference on Tangible and Embedded Interaction, Bonn-Aachen, Germany, February 18–20: 2008: pp. 11–14.

19.

Pradhan

Prabhakaran

. Hand-gesture computing for the hearing and speech impaired. IEEE Computer Society 2008; 15: 20–27.

20.

Tsukada

Yasumura

. Ubi-finger: a simple gesture input device for mobile and ubiquitous environment. Journal of Asian Information, Science and Life 2004; 2: 111–120.

21.

Mrazovac B, Bjelica MZ, Simic D, Tikvic S, Papp I. Gesture based hardware interface for RF lighting control: Proceedings of IEEE 9th International Symposium on Intelligent Systems and Informatics, Subotica, Serbia, September 8–10: 2011: pp. 309–314.

22.

Rahman ASM, Hossain MA, Parra J, El Saddik A. Motion-path based gesture interaction with smart home services: Proceedings of the 17th ACM International Conference on Multimedia, Vancouver, Canada, October 19–24: 2009: pp. 761–764.

23.

Microsoft. Kinect for Windows sensor components and specifications. Retrieved 23 February 2016, from https://msdn.microsoft.com/en-us/library/jj131033.aspx.

24.

Microsoft. Visual Gesture Builder: overview. Retrieved 9 March 2016, from https://msdn.microsoft.com/en-us/library/dn785529.aspx.

25.

Shotton J, Fitzgibbon A, Cook M, Sharp T, Finocchio M, Moore R, Kipman A, Blake A. Real-time human pose recognition in parts from single depth images: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ‘11, Colorado Springs, USA, June 21–23: 2011: pp. 1297–1304.

26.

Zhu

Wang

Yang

Davis

Pan

. Reliability fusion of time-of-flight depth and stereo geometry for high quality depth maps. IEEE Transactions on Pattern Analysis and Machine Intelligence 2011; 33: 1400–1414.

27.

Fankhauser P, Bloesch M, Rodriguez D, Kaestner R, Hutter M, Siegwart R. Kinect v2 for mobile robot navigation: evaluation and modeling: Proceedings of International Conference on Advanced Robotics (ICAR), Istanbul, Turkey, July 27–31: 2015: pp. 388–394.

28.

DiFilippo

Jouaneh

. Characterization of different Microsoft Kinect sensor models. IEEE Sensors Journal 2015; 15: 4554–4564.

29.

Parzych M, Dabrowski A, Cetnarowicz D, Wroblewski M, Szwed H, Staszewski M. Investigation of gesture scenarios for servomotor control using Microsoft Kinect sensor: Proceedings of Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA), Poznan, Poland, September 26–28: 2013: pp. 112–117.

30.

Wakabayashi A, Motomura S, Kato S. Communicative humanoid robot control system reflecting human body movement: Proceedings of International Symposium on Micro-NanoMechatronics and Human Science (MHS), Nagoya, Japan, November 6–9: 2011: pp. 122–127.

31.

Song W, Guo X, Jiang F, Yang S, Jiang G, Shi Y. Teleoperation humanoid robot control system based on Kinect sensor: Proceedings of 4th International Conference on Intelligent Human-Machine Systems and Cybernetics, Nanchang, China, August 26–27: 2012: Vol. 2: pp. 264–267.

32.

Sandygulova

Dragone

O’Hare

GMP

. PRIveT – a portable ubiquitous robotics testbed for adaptive human-robot interaction. Journal of Ambient Intelligence and Smart Environments 2016; 8: 5–19.

33.

Vatavu D. User-defined gestures for free-hand TV control: Proceedings of the 10th European Conference on Interactive TV and Video, EuroTV12, Berlin, July 4–6: 2012: pp. 45–48.

34.

Amunts

Jancke

Mohlberg

Steinmetz

Zilles

. Interhemispheric asymmetry of the human motor cortex related to handedness and gender. Neuropsychologia 2000; 38: 304–312.