Noah
Murad,
Master’s
candidate
David
R.
Cheriton
School
of
Computer
Science
We examine using a human hand to functionally represent an onscreen game character for direct control of simple video games. One example of this type of hand-gesture puppetry is where the index and middle fingers are a character's legs and rest of the hand representing the body. With this type of "Finger People'' input, the player can control how a character runs, jumps, and attacks. The system uses a single camera capture of the player's hand with a recognition pipeline combining optical flow and a convolutional neural network. A study shows novice gamers enjoy using this method of control over traditional keyboard game input.
We describe and demonstrate different types of hand-gesture puppetry, and argue the method is advantageous compared to other gesture recognition systems because it requires one hand to operate, it has no specialized hardware requirements, and it implements a unique paradigm for video games.