Those vacation photos you're uploading for your friends may be used to create virtual models of your favorite places. As this ScienCentral video explains, researchers are able to create virtual 3-D models by combining similar snapshots posted on the Internet.
Virtual Models
It's your snapshot of the Statue of Liberty. It's just like a thousand others taken that day, but this one is yours. So, you upload it to an Internet-based photo sharing service for everyone to see.
Among those who might see your photo and thousands of similar photographs are computer scientists who have created a computer program to combine tourist photos to create accurate 3-D models of such places as the Statue of Liberty or Notre Dame Cathedral.
"The images can be quite general," says Steve Seitz, Associate Professor in the University of Washington's Department of Computer Science and Engineering. He adds, "They can be day or night. They can be almost any viewpoint you want. There could be lots of people in the foreground obstructing the architecture and it will still work pretty well."
"What the algorithm (computer program) is doing is it's taking a set of photos, trying to figure out which parts of the photos overlap with each other," Seitz says.
Photographs turn the three-dimensional or 3-D world into a flat two-dimensional or 2-D representation. This program reverses the process. "To produce a reconstruction we have to figure out for every 2-D position in the image what its 3-D coordinates are," Seitz explains.
He says the program does this by "finding pixels in the different images that correspond to each other." Pixels are the individual points in a digital image that, when taken together form the picture. Seitz says two points correspond if their projection into 2-D would represent the same point in a 3-D scene.
Photographers have been able to stitch photographs together for years using many different software programs. Some are even able to create totally immersive experiences that allow the viewer to look in all directions, including up and down in a seamless photo. However, those programs require expensive special photographic equipment and manual locating of common points.
Seitz says the program starts with the easy matches first such as, "corners or distinctive feature marks on the object." The program then moves on to the hard jobs, including removing people.
Graduate student Noah Snavely is part of the team working on the project. He explains the program can remove people because, "If a person appears in one image they're not that likely to appear in the same place in different images. So, your algorithm will either not be able to find points for that person in the other images and, therefore, ignore those points or if the person moves they won't be consistent so it can ignore those as well."
Others working on the project are Brian Curless, a University of Washington associate professor of computer science and Microsoft researchers Hugues Hoppe and Richard Szeliski. Leading the 3-D mesh modeling was Michael Goesele, an assistant professor at Technical University Darmstadt in Germany.
Not only does the computer program have to match points, it has to figure out where the photographers were in relation to the building when they took the picture. It's a complex process, even for a computer or even groups of computers. Snavely explains that, "For a few hundred images, it tends to take a few days, for thousands of images it may take a few weeks."
Therefore, the engineers are still working on the program, looking for ways to make it faster and able to handle even more photographs.
Still, Seitz says the accuracy is, "pretty good… It's within about a quarter of a percent of what you would get with a laser scanner. A laser scanner is a very expensive piece of equipment to get very very accurate 3-D models in the field. This is kind of the benchmark for 3-D modeling. This is what the [special-] effects companies use."
While the 3-D modeling is their newest project and was presented for review at the 11th IEEE International Conference on Computer Vision, they have a second program called "Photo Tourism." The idea behind it is to organize online photos in a way so that they are linked visually. A viewer can start on a wide picture of someplace, and see optional views, angles and close-ups of that place, all taken by different photographers at different times and uploaded to different sites.
The program works best with exteriors of famous structures because they don't tend to change over time and people have uploaded many pictures. Seitz adds, however, "There's certainly cases where it doesn't work, and these are really the future research problems." He says those are "things like interiors with not much texture, things that are harder to match."
The researchers have some big goals for the software. Explains Seitz, "We'd like to be able to reconstruct a whole city. And, that will involve both operating on more photographs, but also generating and devising better algorithms that can process such photographs efficiently."
This research was presented for scientific review at the 11th IEEE International Conference on Computer Vision, October, 2007 in Rio de Janeiro, Brazil and was funded by The Alexander von Humboldt Foundation, the National Science Foundation, Microsoft Research and Adobe Systems Inc.
No comments:
Post a Comment