In February 2017, together with World Wildlife Fund, ArtScience Museum and Google Zoo, MediaMonks launched a large-scale mixed reality experience "Into The Wild" to help people in Singapore experience the devastating effects of deforestation and learn more about some of the world’s most endangered species and their habitats. It was the world’s first Tango-enabled smartphone Lenovo Phab 2 Pro, and guided visitors through personalised digital adventures, which started with AR on the ground floor of the exhibition space, before transitioning to full VR.The end of the experience shifts back to AR, where users go up to the fourth floor for an experience that includes planting a virtual tree.Transforming over 1,000 square meters of the Singapore ArtScience Museum into a virtual, interactive rainforest, making it the largest AR experience in the world, and second ever AR museum experience developed using Google Tango.
And it wasn’t easy. From a technical perspective, we faced the massive challenge of how to accurately and smoothly map a virtual rainforest onto a physical and dynamic museum space, making sure the walls aligned with trees, corridors with the forest’s paths, and that we worked our way around the museum’s existing exhibitions and staging.
So how did we do it?
To start with, if you’re augmenting the real world with virtual objects, it’s important that the device rendering your view (such as a smartphone, monitor, CAVE or head mounted device) is exactly aware of where it is in the real world.
For this, a device needs to know its position and orientation in a three-dimensional space.
In the case of Tango, where the augmentation happens on a camera feed, the position and orientation of the rendering device needs to be in real world coordinates. Only if the position and orientation of a Tango device is reported accurately, and fast enough, proper augmented reality is possible.The fact that Google Tango does this for you is very cool because it allows developers to augment real world locations within their own virtual world which is different from Snapchat-like AR which, for example, augments bunny ears to your head.
With real world bound augmentations, you can potentially create shared AR experiences that revolve around and involve landmarks.In this case, it allowed us to transform the ArtScience Museum into a lush virtual rainforest and from the user’s perspective, exploring the rainforest becomes as natural as exploring the museum itself because every corridor or obstacle in the virtual world matches a corridor or obstacle in the real one.