Setup and Implementation
Unity Application for Data Recording
We developed a
Unity application with the help of the Dexmo Unity SDK that allows us to record synchronized data from both the Dexmo hand and the additional
accelerometer at a stable framerate of 30 Hz. The Dexmo Unity SDK provides glove data at 200 Hz by transmitting packages from the glove to a python server and further to the Unity application in a blackbox fashion. For the additional accelerometer, we use a custom application that transmits acceleration and gyroscope values as TCP packages to the Unity application, where we synchronize the datastreams to 30 Hz. We have the option to manually set markers in the data by pressing keys, which allows us to mark the beginning and ending of certain actions such as gestures. The synchronized data including potential marker can be streamed to both a csv file and a python server. Streaming to csv files is helpful for recording and later analyzing the data as in section
Recording high-dimensional data, while streaming to the python server allows us to evaluate data directly and couple in interaction tasks such as in section
Designing simple interaction.
Record and process high-dimensional data
As mentioned in section Recording high-dimensional data, we recorded 20,000 samples of data for the training set as well as five repetitions of three general and three accelerometer specific gestures for future evaluation from four users each.
The data in its original state is comprised of the following features:
-
5 dimensions: Bending state of each finger ranging from 0 (fully stretched) to 1 (fully bent)
- 5 dimensions: Bending velocity of each finger derived from the difference between consecutive bending states and the time passed
- 18 dimensions: Acceleration and gyroscope values from three accelerometer devices placed on the back of the hand, the thumb and the index finger
To smooth the data, we applied a
Savitzky–Golay filter with the window length 5 and polynomial order 2. Additionally, we scaled the data to the range from 0 to 1 using a
MinMax scaler to obtain a uniform domain for the various features.