“Visual positioning is not a very new technology,” says Konrad Wenzel at ESRI, a company that develops digital mapping and geographic analysis software. “But it’s clear that the more cameras we have, the better it gets.”
Niantic Spatial trained its model on 30 billion images taken in urban environments. In particular, the images are clustered around hot spots — places that served as key locations in Niantic’s games that players were encouraged to visit, such as Pokémon Battlegrounds. “We had over a million locations around the world where we could find you precisely,” McClendon says. “We know where you’re standing to within several centimeters of accuracy and, more importantly, where you’re looking.”
The result is that for each of those million locations, Niantic Spatial has thousands of images taken of more or less the same location but from different angles, at different times of day, and in different weather conditions. Each of these photos comes with detailed metadata that indicates where the phone was in space when the photo was taken, including where the phone was facing, whether it was moving, how fast and in what direction, and more.
The firm has used this data set to train a model to predict exactly where it is—even for locations other than the million hotspots, where good sources of image and location data are scarce.
In addition to GPS, Koko’s robots, equipped with four cameras, will now use this model to try to figure out where they are and where they are going. The robots’ cameras are hip-height and point in all directions at once, so their perspective is a little different than a Pokemon Go player, but the data acquisition was straightforward, Resch says.
Rival companies also use visual positioning systems. For example, Starship Technologies, a robot delivery firm founded in Estonia in 2014, says its robots use their sensors to create a 3D map of their surroundings, planning the position of building edges and street lights.
But Rash is betting that Niantic Spatial’s tech will give Coco an edge. It claims this will allow its robots to park themselves at the right pick-up locations outside restaurants, making sure they don’t get in anyone’s way, and stop just outside the customer’s door instead of a few steps away, as has happened in the past.
The Cambrian explosion in robotics
Hanke says that when Niantic Spatial started working on its visual positioning system, the idea was to apply it to augmented reality. “If you’re wearing AR glasses and you want the world to close in on where you’re looking, you need some way to do that,” he says. “But now we’re seeing a Cambrian explosion in robotics.”