University of Pennsylvania
Matching Images: Rigidity and Co-Saliency
With the success of digital photography during the past few years we have witnessed a revolution in the way photographs and videos are captured and processed. Today, our ability to acquire images and video far outstrips our ability to make sense of that data. This is true not only in personal and commercial applications but also in the sciences, where huge amounts of image data are acquired from scanners, microscopes, telescopes, and various other instruments.
We address the problem of matching two pictures of a scene taken from two separate viewpoints, with potentially large non-overlapping parts. In the first part of the talk we only assume rigidity of the scene while in the second part we match co-salient regions. We formulate the problem as a search problem in the cartesian product of all possible correspondences.
In this space, candidate matches vote for geometry hypotheses with a vote weight depending on local image similarity. The voting process can be written as a Radon transform and we present a new scheme for computing it efficiently based on Fourier analysis on the sphere and the rotation group.
We show that the maximum of the Radon transform is a very good global similarity metric for images and apply it in the organization of unordered sets of pictures. We relax the geometry constraint and instead we ask for matching of regions which match and are salient in both images. We maximize a score function that segments jointly two images and matches their spectral embeddings. Soft segmentations from two images are aligned through rotation in the embedding space and the resulting global matching score is affected only from salient matched regions.
Thursday, September 27, 2007
JEC 3117 - 4:00 p.m. to 5:00 p.m.
Refreshments at 3:30 p.m.