How to do it?
Take a sequence of images from the same position
Rotate the camera about its optical center
Compute transformation between second image and first
Transform the second image to overlap with the first
Blend the two together to create a mosaic
If there are more images, repeat
but wait, why should this work at all?
What about the 3D geometry of the scene?
Why aren’t we using it?
Aligning images: translation
left on top
right on top
Translations are not enough to align the images
How to relate 2 images from same camera center?
how to map a pixel from PP1 to PP2
Cast a ray through each pixel in PP1
Draw the pixel where that ray intersects PP2
But don’t we need to know the geometry of the two planes in respect to the eye?
Observation: Rather than thinking of this as a 3D reprojection, think of it as a 2D image warp from one image to another
A pencil of rays contains all views
Can generate any synthetic camera view as long as it has the same center of projection!
The mosaic has a natural interpretation in 3D
The images are reprojected onto a common plane
The mosaic is formed on this plane
Mosaic is a synthetic wide-angle camera
Back to Image Warping
Which t-form is the right one for warping PP1 into PP2? e.g. translation, Euclidean, affine, projective