Open Source Hand Tracking ?

Hello Trackies !

We are back with some R&D updates 🙂

Lately we have been thinking a lot about VR interactions in VR Arcades/LBE. We have seen that many people who have never experienced virtual reality can have a hard time with the controllers. Have you tried robo recall game for example ? I takes a good 10 minutes to understand fully how to use the controllers to grab a gun, reload etc…

In a VR Arcade, where people are going to play for 20-40 minutes usually, you cannot afford 10 minutes of explanations. Therefore your experience need to be as natural as possible !

With VR Tracker you can naturally walk in the room, which is great because it avoids having to learn about the concept of “Teleportation” in VR. But when it gets to interact with the environment it can get more complicated. For now on we have mostly worked with shooter and we provided the guns, and everybody knows how to shoot a gun of course, but what if you need to grab an object in the room or open a door ?

Our first thought was : “Ok, so we should do different controllers for each experience, matching exactly how they would be in real life”. The second though was “This is going to be a mess to handle”.

In the end I remembered Steve Job’s speech on the Iphone, and that the fingers and the hand are actually the best controller in the world ! Let’s do hand tracking !

 

Hand tracking : 2 options

Camera

Systems like the Leap Motion allows to do some great hand tracking without having to wear anything. It seemed to me like a great option. I have been talking with their engineers and the Leap Motion is supposed to be integrated in many VR headset anytime now (at least that’s what they told me a year ago… still waiting). And that’s my major problem with the Leap Motion : it doesn’t seem to be moving very fast. I am still awaiting and Android compatible module for standalone headsets. There is no release date announced so I have no idea what I can expect from them… 

I’ll keep it on the side for now !

Glove

They are plenty of tracking glove out there : Manus VR, Plexus, Senso
Most of them are very expensive (except Plexus but they haven’t shipped yet). And for those I have tested, I can’t say the tracking was incredible, mostly because of very simple stuffs that could have been corrected in software but putting some joints limitations.
Once again, I haven’t seen any glove I think that works with BLE (for Android standalone). And the major issue with gloves is to keep them clean !

A few gloves start to offer force feedback like Haptx, Powerclaw, some with quite complicated and heavy systems (I haven’t seen any that would fit a VR Arcade environment), and some with simple vibrating motors that are much more convenient.

 

How to improve VR Gloves ?

From my experience, for VR Arcade we should forget any complex feedback, and stick to vibrating motors and IMUs.
Now I think the most important is to do a great software that auto calibrates and handles joints limits perfectly. There could even be some AI for IMU drift correction as we go 🙂

At VR Tracker we currently don’t want to get into such a big project, but we have all the hardware (electronic) already designed to make it work, and make it affordable ! You can see below our IMUs board that are very small and can connect to our Wifi / BLE board along with vibrating motors.

 

      

 

We decided that we could open source this project so everyone could contribute to make a great software and hand tracking system. I’ll let you watch the videos below showing the first pieces of hardware in action and check the link below if you want to discuss about it and collaborate 🙂

 

Don’t hesitate to share your ideas on the Discord channel and if you are enough people interested we can make an open source design, have some dev kit and collaborate on that project !

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>