Flickr has tens of thousands of images of tourist attractions like Notre Dame. New software now allows these raw images to be mapped into a 3D landscape.

Credit: Photo Tourism

Related articles

• Internet flaw could let hackers control Web
• Fly on the wall inspires vision technology
• Navigate space with Google
• New Microsoft MP3 player targets Apple's iPod
• Sydney climate change blackout a success

SYDNEY: New software can harvest thousands of individual digital images from photo-sharing web sites and stitch them into 3-D scenes.

Photo-sharing websites have taken off in a big way - hand in hand with the growing ubiquity of the digital camera. Flickr, one of the most popular, has more than 200 million images on its servers.

To take advantage of this surfeit of pictures, researchers at The University of Washington are developing software that can harvest different images of the same scene, compare them for small similarities, and stitch them together in a virtual three-dimensional space.

The effect is still crude, but the technology might one day allow users to virtually walk around and explore tourist sites such as India's Taj Mahal or Sydney's Opera House simply using holiday snaps uploaded to the internet.

Photo Tourism, as the software is called, can be used for organising the confusing jumble of images we hold on our hard drives at home too. The technology has proved so enticing that U.S. software giant Microsoft has bought a version that it is now marketing under the name of Photosynth.

If you plug Rome's 'Trevi Fountain' in as a search term on Flickr, you'll find more than 12,000 images. Browsing through thousands of pages of thumbnails can be a frustrating process, said lead developer Noah Snavely, a graduate student at the Seattle-located university.

"You might look at a photo and say 'I wonder what's just to the left of it', or 'I wonder what's just to the right of it', or 'I wish I could expand the field of view'," he said.

To solve the problem, Snavely and computer scientist Steve Seitz harnessed recent advances in computer vision research. They wrote software that analyses each image and attempts to calculate where it was taken. It does this by matching up small shared details and then stitching different images together in three dimensions. Each photo is then represented by a small square placed in the appropriate position in a 'sketch' of the scene.

Once the scene is assembled, the viewer can then select the photo they want to view, zoom-in or out on specific features, and click to the left or right to view objects on either side of the image. When a detail in the photo is selected, the software can pick out a number of different pictures as thumbnails relating to that point for the viewer to choose from.

"This makes browsing the photos more intuitive; the user can look and move around the scene in a way that simulates actually being there," said Snavely.

Check out a short video demonstration of the technique here (23 MB, hosted by the University of Washington).

The researchers first tested the software using the Trevi Fountain in Rome - somewhere Snavely has never been in person - and later experiments mapped similar 3-D scenes using thousands of Flickr derived snaps of Notre Dame Cathedral in Paris, and Half Dome - the iconic glacier-carved peak in California's Yosemite National Park.

Photosynth is part of Microsoft's new push into "Internet-based personal services," said software reviewer Pam Carol with the Australian wing of technology review site CNET.com, in Sydney. "A lot of software and mapping applications are [also] moving towards 3-D - consumers see it as a way of making the internet more life-like."

Google is already using the approach for aerial photos in its 3-D Earth application - and we will start to see it used more in other places, said Carol, suggesting that Photo Tourism is part of a wider trend.

Google and Microsoft are also starting to stitch together their own 3-D models of cities by painstakingly gathering photos and stitching them together. Although Photo Tourism isn't yet anywhere near as smooth as those - there are gaps and people sometimes pop into the photos - the fact that it can generate scenes ad-hoc from peoples' raw holiday snaps gives it great potential.

The researchers envisage that the software may one day provide a way of organising the countless million of photos that exist online to create a "visual Wikipedia."

It might also have numerous applications in places were regular digital photos are currently used.

Sports enthusiasts could recreate a game by matching photos taken from many different angles at an event. Real estate agents, hotels, museums and other attractions could use it to create a virtual tour - one in which users could zoom in to examine a painting in more detail, or read a menu.

More information:

Photosynth, Microsoft Live Labs