A man puts on his smart glasses and his world becomes a mix of virtual and real. He looks out his window and sees a weather app overlay that predicts a 30 percent chance of rain. His workout on a stationary bikes turns into a video game, as he races to collect gems projected in front of him. 

This is the future as imagined by augmented reality engine maker InfinityAR

We might be years away from that future but even at this early stage, InfinityAR, an Israel-based startup is working on closing the gap between the virtual and the physical with holographic images that you can “touch” or control with your hands. Most smart glass makers borrow smartphones gestures like pinch to zoom and swiping to control digital elements in augmented reality. InfinityAR, however, is focused on more intuitive interactions.

“Say we are playing a chess game, what is currently being done in the virtual content market is gestures,” Motti Kushnir, InfinityAR CEO and co founder, told iDigitalTimes. “So I’m doing kind of a grabbing gesture with my hand, it shows me that I picked the piece and now I can move it around. This is not the interaction we would like to create.

“We would like to mimic the way we are interacting with physical objects in real life,” he said.

So rather than grabbing nothing in mid-air, you’re placing your hands around a virtual object and it’s responding to your movements.

InfinityAR InfinityAR concept video YouTube/Screenshot

Of course, there is a crazy amount of computing that has to happen in order for you to “grab” an virtual object. To pull off this seemingly impossible feat, Infinity’s AR engine not only maps the users’ surroundings in real time but also their hands.

“What we have developed is the ability to understand the 3D model of the hand in real time,” Kushnir said. “The hand that comes into the picture has a physical quality so you can now interact with a virtual object as if it was truly a real piece in the real world.”

The technology is far from flawless, but the InfinityAR team is making strides as evidenced by the demo videos it has posted on YouTube. One video shows a hologram turtle sitting in the palms of a person’s hand. The person is able to move their hands from side to side with the turtle and they’re able to place the turtle down on the table like they would place a real-life object.

InfinityAR Pet Turtle

The video might not look too impressive but the technology is pretty astounding give the holograms we’ve seen so far in real-life like the Häagen-Dazs ice cream hologram. Those are actually like QR codes your smartphone reads rather than your camera recognizing images, let alone a 3D hand. Another video showing someone’s hand moving over a digital portrait of Mona Lisa on a table is not as successful. When the hand moves over the digital object, it obscures the view of the object as if it was real but there is a bit of distortion around the hand.

InfinityAR, though, is working on beefing up its computing power and making its algorithms more efficient. While more processing power is required for mapping the hand, Infinity’s AR platform is supposed to be more efficient as it does not require infrared 3D depth sensors to work. Its AR engine only requires two passive 2D video cameras, like the ones in your smartphone. They are mounted onto a wearable along with sensors to map the users environment and figure out where the device is positioned and oriented.

According to Infinity, smart glass developers on its platform can cut costs and energy consumption to one tenth using two normal video cameras rather than expensive and power hungry 3D depth sensors. InfinityAR’s augmented reality software platform will launch in June, coinciding with the release of a new computer headset from an unnamed leading optics maker. Kushnir said his company is currently working with three top optics manufacturers but could not disclose which ones at this time.

Even with the release of Infinity’s platform in June, Kushnir doesn’t believe augmented reality will be ready for consumers for another few years.

“In the coming two to three years those devices will mainly serve enterprise users,” he said. “Once the devices with our technology are fast enough and robust enough, then we can take them to consumers.”