A team from Art-A-Hack™ Summer 2016

The Kinect VR team devoted their Art-A-Hack 2016 project to demonstrating creative applications of the innovative KinectVR toolkit, a body-tracking solution for virtual reality.

The toolkit allows multiple participants to interact in a virtual reality world via full-body motion tracking, using Microsoft Kinect. It is designed to be used on mobile headsets such as Samsung Gear VR, in which participants can feel more fully immersed in the VR experience because there are no wires restricting their body movement.

One of the key innovations of KinectVR is that participants see a representation of their entire body rendered below them, moving in the VR environment. Their hands and legs move as they move, without having to hold controllers, leading to a rich and immersive experience.

In addition, other players in the same physical space can see each other’s virtual bodies within their own VR experience. In this way there is a full mirroring between the activity of players both physically and virtually.

This was the largest team in Art-A-Hack 2016, with a total of six team members. Their goal was to demonstrate the capabilities and unique features of the KinectVR plugin system, and so each team member designed their own unique “virtual world”.

During the final Art-A-Hack showcase, players trying out the demo were able to navigate around the different worlds freely, and discover the properties of each.

One of the unique challenges the team faced was rendering the noisy Kinect skeleton data in VR. Multiple bodies tend to break apart when they cross each other in the Kinect’s field of view.

This causes particular interaction issues in VR, as discrepancies between a user’s virtual and physical body can lead to dissociative effects.

The playable prototype was demonstrated at the final Art-A-Hack showcase event. The team plans to continue to work on the open source software application that can be used by developers worldwide.