Development of a Python application for recognizing gestures from a video stream of RGB and RGBD cameras

Abstract

Gesture recognition systems have changed a lot recently, due to the development of modern data capture
devices (sensors) and the development of new recognition algorithms. The article presents the results of a study for
recognizing static and dynamic hand gestures from a video stream from RGB and RGBD cameras, namely from the
Logitech HD Pro Webcam C920 webcam and from the Intel RealSense D435 depth camera. Software implementation
is done using Python 3.6 tools. Open source Python libraries provide robust implementations of image processing and
segmentation algorithms. The feature extraction and gesture classification subsystem is based on the VGG-16 neural
network architecture implemented using the TensorFlow and Keras deep learning frameworks. The technical characteristics
of the cameras are given. The algorithm of the application is described. The research results aimed at comparing data
capture devices under various experimental conditions (distance and illumination) are presented. Experimental results show
that using the Intel RealSense D435 depth camera provides more accurate gesture recognition under various experimental
conditions.