Document Type : Research Article


1 Erasmus Mundus Master in Pervasive Computing and Communications for Sustainable Development Hochschule Harz – University of Applied Sciences, Wernigerode 38855, Germany

2 Hochschule Harz – University of Applied Sciences, Wernigerode 38855, Germany


Homes and working spaces are considered significant contributors to the top percentage of energy consumption and carbon emissions worldwide. Previous studies in the field of home- and building automation have demonstrated the sustainability gain brought by smart home solutions, in terms of energy-efficiency, economic savings, and enhanced living and working conditions. A major barrier, however, to the adoption of these solutions is the complexity and poor usability of user interfaces. In addition, various modes of interactions for the control and automation of residential environments are an emerging area of study within Human-Computer Interaction. As a response to these challenges, this study investigates the use of gestures as a natural way of controlling and interacting with home automation systems. After a survey of available motion capture technologies (Microsoft Kinect and LEAP Motion) and studies related to both, a gesture dictionary will be defined as a set of meaning actions in free form in-air movements. A socio-technical study t will be conducted to measure the resulting aspects such as acceptability, ease-of-use, and culturability. Lastly, the study will present the analysis and effects of gestures control for a higher up-take of smart home solutions towards designing and maintaining buildings of the future that are both user-centric and resource efficient to reduce our overall carbon footprint. 


1.“Energy trends -Statistics Explained,” 2017. Online.. Available:
Energy_trends. Accessed: 21-May-2018.

2.“Buildings -European Commission.” Online.. Available:
buildings.Accessed: 21-May-2018.

3.GeSI, “#SMARTer2030,” 2015. Online.. Available:
Accessed: 21-May-2018.

4.Droegehorn, O.; Pittumbur, M.; Poras, J.; Front-End
Development for Home Automation Systems: A design approach
using JavaScript Frameworks. SEEDS Conf., 2017.

5.Tran. G.; GreenBe A System To Capture and Visualize User’s
Energy-Related Activities For Facilitating Greener Energy
Behavior. SEEDS Conf., 2017.

6.Hevner, A.; Scandinavian Journal of Information Systems A
Three Cycle View of Design Science Research A Three Cycle
View of Design Science Research. Scand. J. Inf. Syst., 2007.

7.David, P.; Collete N.; Magdalen, G.; Review of the current status
of research on smart homes and other domestic assistive
technologies in support of the TAHI trials Review of the current
status of research on ‘Smart Homes’ and other domestic assistive
technologies in support of TAHI trials, Loughbrgh. Univ., 2002.

8.Malcolm, J.; The Implications of Smart Home Technologies, S.
Peace, C. Holl. Incl. Hous. Ageing, 2014.

9.Martinez, C.; Remote control-based home automation usability
evaluation, SEEDS Conf., 2017.

10.Garber, L.; Gestural Technology: Moving Interfaces in a New
Direction, Computer (Long. Beach. Calif.), 2013.

11.Karam, M.; Schraefel, M.; A Taxonomy of Gestures in Human
Computer Interactions, Tech. Report, Eletronics Comput. Sci.,

12.De Carvalho Correia, A.; De Miranda, L.; Hornung, H.; Gesture-
based interaction in domotic environments: State of the art and
HCI framework inspired by the diversity, Lect. Notes Comput.
Sci., 2013.

13.Han, J.; Shao, L.; Xu,D.; Shotton, J.; Enhanced computer vision
with Microsoft Kinect sensor: A review, IEEE Trans. Cybern.,

14.Zhang, Z.; Microsoft kinect sensor and its effect, IEEE
Multimed., 2012.

15.Panger, G.; Kinect in the kitchen: testing depth camera
interactions in practical home environments, Proc. CHI 2012 Ext.
Abstr., 2012.

16.Kim, H.; Jeong, K.; Kim, S.; Han, T.; Ambient Wall: Smart
Wall Display Interface Which Can Be Controlled By Simple
Gesture for Smart Home, SIGGRAPH Asia 2011 Sketches -
SA ’11, 2011.

17.Oh, J.; Jung, Y.; Cho, Y.; Hahm, C.; Sin, H.; Lee, J.;
Hands-up: motion recognition using kinect and a ceiling to
improve the convenience of human life, Proc. CHI 2012 Ext.
Abstr., 2012.

18.You, Y.; Tang, T.; Wang, Y.; When arduino meets Kinect: An
intelligent ambient home entertainment environment, Proc. -
2014 6th Int. Conf. Intell. Human-Machine Syst. Cybern. IHMSC
2014, 2014.
19.Lin, H.; Hsueh, Y.; Lie, W.; Abnormal Event Detection Using
Microsoft Kinect in a Smart Home, Proc. -2016 Int. Comput.
Symp. ICS 2016, 2017.

20.Zhao W.; Lun,R.; A Kinect-based system for promoting healthier
living at home, 2016 IEEE Int. Conf. Syst. Man, Cybern. SMC
2016 -Conf. Proc., 2017.

21.Blumrosen, G.; Miron, Y.; Plotnik, M.; Intrator, N.; Towards a
Real-Time Kinect Signature Based Human Activity Assessment
at Home, Wearable Implant. Body Sens. Networks (BSN), 2015
IEEE 12th Int. Conf., 2015.

22.Iqbal, A.; Asrafuzzaman, S.; Arifin, M.; Hossain, S.; Smart Home
Appliance Control System for Physically Disabled People Using
Kinect and X10, 2016.

23.Piedra-Fernandez, J.; Ojeda-Castelo, J.; Bernal-Bravo, C.;
Iribarne-Martinez, L.; Sign Communication for People with
Disabilities Using Kinect Technology at Home, 2016 8th Int.
Conf. Games Virtual Worlds Serious Appl., 2016.

24.Ameur, S.; Ben Khalifa, A.; Bouhlel, M.; A comprehensive leap
motion database for hand gesture recognition, 2016 7th Int. Conf.
Sci. Electron. Technol. Inf. Telecommun. SETIT 2016, 2017.

25.Mapari R.; Kharat, G.; Real time human pose recognition using
leap motion sensor, 2015 IEEE Int. Conf. Res. Comput. Intell.
Commun. Networks, 2015.

26.Kumar, S.; Bansal, N.; Singh, S.; Smart Interaction Using Hand
Gesture Recognition, Proc. Second Int. Conf. Inf. Commun.
Technol. Compet. Strateg. -ICTCS ’16, 2016.

27.Ponraj G.; Ren, H.; Sensor Fusion of Leap Motion Controller and
Flex Sensors Using Kalman Filter for Human Finger Tracking,

28.Dharmayansa, I.; Exploration of PrayerTools in 3D Virtual
Museum Using Leap Motion For Hand Motion Sensor,” 2017
TRON Symp., 2017.

29.Lu, Y.; Bulbs control in virtual reality by using leap motion
somatosensory controlled switches, Int. Conf. Adv. Commun.
Technol. ICACT, 2017.

30.Ransalu S.; Sisil, K.; A robust vision-based hand gesture
recognition system for appliance control in smart homes, 2012
IEEE Int. Conf. Signal Process. Commun. Comput. ICSPCC,

31.Zhang, Z.; Robust Hand Gesture Recognition Based on Finger-
Earth Mover’s Distance with a Commodity Depth Camera, Hand,

32.Davis, F.; Bagozzi, R.; Warshaw, R.; User Acceptance of
Computer Technology: A Comparison of Two Theoretical
Models, Manage. Sci., 1989.

33.Nielsen, M.; Störring, M.; Moeslund, T.; Granum, E.; A
Procedure for Developing Intuitive and Ergonomic Gesture
Interfaces for HCI, 2004.

34.Porras, J.; Seffah, A.; Rondeau, E.; Andersson, K.; Alexandra,
K.; PERCCOM: A Master Program in Pervasive Computing and
COMmunications for Sustainable Development, 2016.