Synchronizing cameras for more fps?

VR Tracker : 3D Positional Tracker for Virtual Reality Forums Forum HardWare Synchronizing cameras for more fps?

This topic contains 8 replies, has 3 voices, and was last updated by  Jules Thuillier 1 year, 10 months ago.

  • Author
    Posts
  • #4942

    naamapokemen
    Participant

    Currently 200+ fps object tracking cameras seem to be very expensive for people with a budget.

    50 fps is kinda slow for fast motion tracking like twitch shooting with a gun. Even with awesome smoothing and prediction algorhitms by VR Tracker.

    Would it be a formidable task to synchronize vr tracker (cmucam5) cameras, so that some of them would start a a millisecond later to provide 100 fps or 50 microseconds later for 200 fps?

     

  • #4971

    Paul Yeung
    Participant

    Hi naamapokemen,

    I believe Jules earliest prototypes for VR Tracker used a Pixy cam too… maybe there was a reason why the beta kit’s don’t have them?

    I’m all in for less latency too! 🙂 Are you hooking up the VR Tracker cameras with networking cables or going wireless? Are you using Vive or Occulus headsets? I would think these would also have a noticable effect on latency as well.

     

    Cheers,

    Paul

    VR Tracker Beta 1 Dev Kit User

  • #4972

    naamapokemen
    Participant

    I don’t have VR Tracker. At some point I was saving up cash for it, but the motions in the videos are too jerky for me with 50 fps, because of latency or both. Although, in latest videos the tracking has gotten much better.

    Still, I’m saving up more, because some other solution like if SteamVR will support more than 33×33′ and four base stations later in 2018, it will probably be cheaper and faster to setup a custom VR Arena.

    At the moment I’m looking at Vive with TPcast, 4 stations, and IKinema. Body tracking with IKinema is too pricey for me (subscription based), but maybe at some point it’ll be free. Kinect tracking is lacking. Other consumer budget(still expensive) options like VicoVR seem to be vapor.

    The few VR arenas that exist, mostly seem to use optitrack. It’d be nice if it didn’t cost hundreds of thousands if not millions for a decent size arena.

  • #4973

    Paul Yeung
    Participant

    Hi naamapokemen,

    Yeah there seems to be very little in the way of affordable tracking options at the moment. I’m guessing that will change as VR becomes more mainstream and the technology gets refined.

    I’m currently setting up my living room with 4 VR Tracker cameras with 5 tags in a 2.5 x 4m space. I’m hoping to put through VR Tracker through its paces in the next few weeks and draw some opinions from the viewpoint of an average-joe consumer.

    Looking at their newsletters it looks like they’re working on reducing latency by synchronising all the data. What this actually entails I don’t know… maybe Jules can elaborate on this a bit more on this topic. Newsletter link below:

    https://us13.campaign-archive.com/?u=79e0f9e61d76fa73cfef1cab7&id=1e518a0d79

    Cheers,
    Paul

    VR Tracker Beta 1 Dev Kit User

  • #4978

    Jules Thuillier
    Keymaster

    Hello Guys !

    First let me clarify : we are not using CMuCam5, those were for prototyping purposes only, and the resolution was only 320×240 pixels with no subpixel analysis.

    BTW, if you use more than 3 cameras, I would use at least a bigger wifi router instead of the Gateway WiFi, and if possible wire the cameras. We will also update the transmission protocol in December to divide the data over network by 3.

    Now concerning the synchronisation, Alex has started working on that about 10 days ago, it should be released by the end of the year. Also we are going to receive the new Tag prototypes within a week, and they will include a new IMU (accelerometer / gyroscope / compass). Our plan is to work in 2018 on fusing the camera and IMU datas together (use the acceleration to infer the position over short periods of time). This will allow us to improve the refresh rate, but also continu tracking for a few seconds in case of occlusion.

    Finally, we will have an R&D project that will begin before next spring to upgrade the cameras hardware to go at least up to 90FPS (hopefully more). Those new cameras will take quite some time to develop and won’t be released by the end of 2018.

    As you see we have a lots of things planned, and you guys would like to always have the latest versions, which is why we are thinking of a lease model for the hardware on a monthly basis for those creating VR Arenas. It will allow you to benefit from hardware updates.

    EDIT: Oh and I forgot, a Unity plugin update in about 10 days will also remove a 15ms in latency. Latency is the main issue we are working on right now. And honestly I think we are getting close and it should be an issue solved in 6-8 weeks.

    Cheers

    Jules

    Hardware Dev

  • #4985

    naamapokemen
    Participant

    Thanks for the info, Jules. Lease model would be great. Currently, I think I should wait for the new cameras and tags and hope you don’t get bought by some big company (good for you, bad for us :P.) Lease model, if affordable would remove this problem. Only thing that comes to mind with lease model is that it might be hard for you to keep track, if the leaser has insurance abroad. With costly parts, I think insurance is a must. You might want to consider having your own insurance which the leaser pays, when s/he rents the items.

    If the tracking comes playable at some point, I’m dreaming of doing a BIG arena, since I have some free space (3000+ m2) in an abandoned mining facility. I can rent it for very cheap, if I bring some meaningful action to the small town.

    I think you’ll have to look at Xsens suit patents with your new tags, so you don’t use their patented methods. They use some combined IMU and other sensor data.

    I found these documents interesting. They are about combining different techniques with IMUs.

    hxxp://sci-hub.bz/http://ieeexplore.ieee.org/abstract/document/7585477/?part=1

    hxxp://stanford.edu/class/ee267/Spring2016/report_zhu_shi.pdf

    Buying a flex sensor is cheap, buying a vr glove containing a few flex sensors is very expensive for some reason.

     

    Thanks,

    Ville

  • #4995

    Jules Thuillier
    Keymaster

    You are right on the insurance, it is definitely something to look into.

    300m2 would be awesome, there is so much you can do in VR in a place like that 🙂

    I’ll check into the Xsens, just in case, but I don’t think we could get into patent infringement as IMU tracking doesn’t seems to be very new, of course for VR it is a bit more. But it’s worth checking, better safe than sorry right ?

    Actually the flex sensor is quite expensive (like 15$ per sensor), so it quickly brings you to 90$ of sensors only, when an IMU costs the third of that price. The problem is that flex sensors are a patented technology, with just 1 supplier… But I have to admit tracking gloves are incredibly expensive ! We might or might not have a team of students looking into that starting in January… who knows ? I believe that hand tracking is the best way to interact in VR, but we gotta make accessible ! The electronic and software could be done quickly, but there is only 24h in day 🙁

    Hardware Dev

  • #4997

    Jules Thuillier
    Keymaster

    By the way do you have a free version of the paper (http://ieeexplore.ieee.org/abstract/document/7585477/?part=1) ? I knew the other one, but I’d like to see that one, I only were able to read the first page ^^

    Hardware Dev

  • #5041

    Jules Thuillier
    Keymaster

    Just to add a nice source of content on hand tracking with IMU, check the file attached 😉

    By the way, the boards to do those kinds of IMU tracking are in production (just the prototypes), and yesterday I have managed add the feature to handle more than one IMU on the Tag. We should be able to use at least 10 !

    I don’t have much time to work on this software, but if somebody has good skills in Arduino and I2C, you are welcome to give a hand ! XD

    Hardware Dev

You must be logged in to reply to this topic.