# The PanoramIX Authoring Tool

## Version 0.1.4.92, February 26, 1998

## 4.7 The Edit Sides Dialog

Given a set of flat side images representing overlapping views of a scene from a fixed camera position, a single panoramic image of the scene can be produced by clipping away redundant pixels where the images overlap, warping the clipped imaged onto a cylindrical surface, and concatenating the resulting warped images into a single panoramic image. Alternatively, each image can be warped onto a cylinder and the overlapping areas blended together to create a panoramic image. In either case, the ability to create a realistic and seamless panoramic image depends on how the images are aligned and trimmed. The parameters which determine how the side images are aligned, trimmed, and fitted together are controlled through the Edit Sides dialog. This dialog is opened by selecting the Sides entry in the Edit menu.

The Edit Sides dialog contains five sections:

1. Side Image Identification and Selection
2. Side Image Alignment Parameters
3. Side Image Cropping Parameters
4. Other Side Image Properties
5. Apply, Delete, and Close Buttons

### 4.7.1 Side Image Identification and Selection

The upper portion of the Edit Sides dialog provides fields for selecting and identifying one of the flat side images being used to represent a panoramic scene. Two buttons labeled Previous and Next advance the selected side to the preceding and following side images respectively in the list of images comprising the current panorama. Three fields below these buttons identify the names of the image files providing the sources of the color values, link map values (if any), and depth map values (if any).

New: A third button labelled "Synchronize" is located between the Previous button and the next button. When this button is selected (pushed in), the image shown in the Image Edit window is synchronized with the image selected for the Edit Sides dialog. In this case, selecting the Previous button or the Next button also advances the image shown in the Image Edit window as well as advancing the view shown in the main window by an azimuth angle equivalent to one image. This will also override attempts to change the selection of the image in the Image Edit window by using the tab keys or by double clicking on the main window.

If the Synchronize button is not selected (out), then the image specified in the Edit Sides dialog may be changed without affecting the image seen in the Image Edit window or the view shown in the main window. This is the default condition.

There is a button labeled Browse next to each of the image file name fields for sources of the color values, link map values, and depth map values. These buttons are intended to provide means of replacing the selected files with suitable alternative files. These functions have not yet been implemented and currently do nothing.

Back to the Edit Sides Dialog.

Back to the Edit Menu.

Back to the PanoramIX Authoring Tool.

### 4.7.2 Side Image Alignment Parameters

The Edit Sides dialog provides controls for the following image alignment parameters:

a. Clipped width
b. Clipped height
c. Horizontal shift
d. Vertical shift

These parameters allow one to select a rectangular subset of each raw image to be used in creating a panoramic image. These parameters serve several purposes. The most important objective is to establish the true center of each image. That is, which row and column of the raw image corresponds to the center of the lens used to capture the image. Ideally, the true center of each image coincides with the geometric center of the raw image so that the center column index is half of the total width and the center row index is half of the total height.

As a result of the processes used to create the digital images, the point corresponding to the center of the lens of the camera used to capture the image may not fall exactly in the centers of the raw images provided as the input to the panorama editor. For example, when a film negative is printed, the image may get cropped by small but unequal amounts on each side. When the resulting print is scanned, the print may not be perfectly aligned on the scanner or the boundaries selected for the scanned image may be too small or too large, resulting in unequal losses or gains along each edge. The resulting discrepancies between the original center point for the lens and the geometric center of a raw image is usually very small, only a few pixels, but they tend to vary from image to image and they must be handled to allow adjacent images to fit together accurately.

In principle, the horizontal shift and vertical shift represent the offsets between the geometric center of the raw image and the center point for the lens used to capture the image.

The clipped width and clipped height allow one to select a subset of the rows and columns of the raw image and discard unwanted image contents at the borders. This makes it easier to scan printed images by eliminating the need to make a "perfect" scan. That is, instead of requiring the scanned image to have exactly the right number of rows and columns, one can deliberately include some excess on all sides of an image, then digitally remove the excess in the panorama editor. This also avoids the risk of losing information by selecting an insufficient number of rows and columns when scanning an image. In this case, the burden of setting the perfect height and width for each image is being transferred from the scanner to the panorama editor.

A visual indication of the alignment parameters for the current image can be obtained by selecting the current image in the Edit Image window. In this case, the vertical lines indicate an area with the indicated clipped width offset from the left by the indicated horizontal shift value. Likewise, the upper and lower horizontal lines indicate an area with the indicated clipped height, offset from the top by the indicated vertical shift value.

The principal reason for reducing the width of a side image is removal of the overlapping areas of adjacent images. This is a critical part of the process of aligning each image with its neighbors. This information is also used to identify corresponding columns of overlapping images when creating a blended panorama. The reduction in width needed to align adjacent images is usually much greater than the amount needed to exclude excess areas at the borders. If not, there would be a gap between adjacent images making it impossible to get them aligned.

Initially, upon loading a set of side images, the clipped width of each image is set to a value based the number of images and the radius of the panorama. The radius is equivalent to the focal length of the camera measured in pixels. If there are NumSides side images in a panorama, the initial clipped width for each image is determined by ``

               width = 2*radius*tan(max_azimuth/(2*NumSides))

where max_azimuth is 360 degrees for a complete panorama, or a smaller value specified in the Maximum Azimuth dialog.

The panorama editor requires the clipped images to be symmetric with respect to the center. Because of this, the clipped width and clipped height values can only change in steps of two pixels. This represents equal changes to the width on the left and right sides, and equal changes to the height for the top half and bottom half of an image.

Because the clipped image is a subset of the original raw image, the horizontal and vertical shift values are restricted to values that keep the clipped image within the original raw image. This means that the absolute value of the horizontal shift must be no greater than half the difference between the original width of the raw image and the current clipped width. Likewise, the absolute value of the vertical shift must be no greater than half of the difference between the original height of the raw image and the current clipped height.

In most cases, the default clipped width is already sufficiently small to provide plenty of room for a wide range of horizontal shift values. An exception to this could occur if there was very little overlap between adjacent side images. In the case of the vertical clipping, however, there is no default criterion for reducing the clipped height relative to the original height. Consequently, it is the responsibility of the user to specify a reduced height for the clipped images which is sufficiently small to allow vertical adjustment of the center point, as well as excluding unwanted areas in the borders.

Warning: If the clipped height is not reduced, it will not be possible to correct vertical alignment problems.

There is a button labeled "All" next to the field for the clipped height. When this button is selected (pressed), any value of the clipped height is automatically applied to all of the side images comprising the panorama. If this is not selected (out), then the clipped height value specified here applies only to the current image. In normal situations, all side images should have the same height, so the default for this button is (pressed).

There is another button labeled "All" next to the vertical shift field. When this button is selected (pressed), any value specified in the vertical shift field is applied to all of the side images comprising the panorama. If this is not selected (out), the value in the vertical shift field is applied only to the current image. The default state of this button is (out).

There are buttons labeled "Next" located beside the fields for the clipped width and horizontal shift. If these buttons are selected (pressed), changes in the values shown in these fields are applied to the immediate neighbors, but not to the other side images. If these buttons are not selected (out) then the values in these fields are applied only to the current image. By default, these buttons are not selected (out).

If the clipped width is increased by one step (two pixels), and the horizontal shift is increased by one pixel, the net effect will be to move the right edge of the clipped image two pixels to the right while preserving the position of the left edge. Likewise, if the horizontal shift is decreased by one pixel, the net effect will be to move the left edge of the clipped area two pixels to the left while preserving the right edge. By using these combinations of width and horizontal shift changes, the left or right edge of the clipped area can be moved without moving the opposite edge. In this case, if both Next buttons have been selected, only the adjacent image on the corresponding side will be affected.

The values of the clipped width and horizontal shift can also be controlled by selecting the left and right vertical lines in the Edit Image window and dragging these lines with the mouse or pointing device. For each pixel that one of these lines is shifted, the horizontal shift value changes by one pixel, and for every two pixels by which one of these lines is shifted, the clipped width changes by one pixel. If the Next buttons are selected (pressed) for both the clipped width and horizontal shift, then the corresponding line in the adjacent image is shifted by an equal amount, preserving the horizontal alignment between these images. As these lines are dragged, the values shown in these fields will be updated to reflect the current positions of these lines.

Warning: Changing the width of any image can force changes to to the widths of all other images even when the Next buttons are not selected. Given a fixed value of the Radius (i.e., the focal length in pixels), changing the width of any image implies a change in the field of view associated with this image. The sum of field of view values for all side images, however, must add up to the maximum azimuth value (360 degrees for a complete panorama). Changing the field of view of any one image violates this sum of angles condition. This situation persists until the panorama is rebuilt, such as when the Apply button is selected. At this time, the sum of field of view angles is compared to the maximum azimuth and any discrepancy is resolved by adjusting the width and field of view angles for all images.

If the clipped height is increased by one step (two pixels), and the vertical shift is increased by one pixel, the net effect will be to move the bottom edge of the clipped image two pixels downward while preserving the position of the top edge. Likewise, if the vertical shift is shift is decreased by one pixel, the net effect will be to move the top edge of the clipped area up by two pixels while preserving the bottom edge. By using these combinations of height and vertical shift changes, the top or bottom edge of the clipped area can be moved without moving the opposite edge.

The values of the clipped height and vertical shift can also be controlled by selecting the top and bottom lines in the Edit Image window and dragging these lines with the mouse or pointing device. For each pixel that one of these lines is shifted, the vertical shift value changes by one pixel, and for every two pixels by which one of these lines is shifted, the clipped height changes by one pixel. If the All buttons are selected (pressed) for the clipped height and/or vertical shift, then the corresponding values in all other side images will be modified to match. As these lines are dragged, the values shown in these fields will be updated to reflect the current positions of these lines.

Back to the Edit Sides Dialog.

Back to the Edit Menu.

Back to the PanoramIX Authoring Tool.

### 4.7.3 Side Image Cropping Parameters

The image cropping parameters make it possible to ignore or exclude all image data outside the cropping boundaries independently of the image alignment specifications. This allows one to eliminate useless or invalid image data resulting from the way a particular image was exposed, printed, or scanned. In the case of a panorama composed without blending, this has no effect on the resulting panorama unless the cropping boundaries fall within the clipping boundaries. If the cropping boundaries fall within the clipping boundaries, the areas between the clipping boundaries and the cropping boundaries are filled with black. Such areas are considered to represent "missing data" which must be filled in later using aany appropriate image processing techniques. In the case of a blended panorama, all image data outside the cropping boundaries are excluded from the image blending process. Image data within the cropping boundaries are weighted based on the distance of each pixel from the cropping boundaries. Thus, the importance of each pixel gradually fades to zero as one approaches a cropping boundary from the interior of an image, and remains exactly zero for all pixels outside these boundaries.

The Edit Sides dialog provides controls for the following image cropping parameters:

a. Crop Left
b. Crop Right
c. Crop Top
d. Crop Bottom

A visual representation of the cropping parameters is presented in the Edit Image window where the excluded areas are shown in black.

The values of all four cropping parameters must be positive integers because these serve only to exclude areas from the blending operations. In most cases, the boundaries of the excluded areas should remain outside the boundaries defined by the clipped width, clipped height, horizontal shift, and vertical shift. Such a restriction ensures that there is no gap between adjacent images. Selection of cropping boundaries which pass inside the clipping boundaries is permitted, but the blended image will acquire black columns if there is no overlap between the cropped areas of adjacent images.

The values of the cropping parameters may be modified by typing new values directly into the corresponding fields in the Edit Sides dialog, or they may be altered incrementally by clicking on the associated spin buttons. These values may also be controlled by selecting the edges of the corresponding areas represented in the Edit Image window. Initially, these values are set to zero and the cropping boundaries coincide with the edges of the Edit Image window. In this case, it can be very difficult to select these edges with the pointing device and it may be necessary to enter a small value for each cropping boundary manually to make then easier to select.

When using a pointing device (mouse) to select and drag a cropping boundary in the Edit Image window, one is not permitted to go beyond the clipping boundaries determined by the clipped width, clipped height, horizontal shift, and vertical shift. In order to force the cropping boundaries beyond this point, one must enter the cropping boundaries directly in the corresponding fields in the Edit Sides dialog.

Each of the cropping parameters has an associated button labeled "All". If an "All" button is selected (pressed), then the specified cropping value will be copied to all side images. If the "All" button is not selected (out), then the specified cropping value applies only to the current image.

Back to the Edit Sides Dialog.

Back to the Edit Menu.

Back to the PanoramIX Authoring Tool.

### 4.7.4 Other Side image properties

The Edit Sides dialog provides controls or information for the following additional image properties:

a. Field of View
b. Radius
c. Camera Elevation
d. Image Skewing

#### 4.7.4.1 Side Image Field of View Angles

The Field of View field reports the horizontal field of view associated with the currently selected side image based on the clipped image width and the radius value, ``

                FovAngle = 2*atan(width/(2*radius)),

or, equivalently, `

                FovAngle = 2*atan(width/2, radius),

`

Unlike the other fields in this dialog, this is a read-only value which cannot be modified or specified directly. The field of view angle angle can take on values determined by the current radius and integer values of the clipped width. The field of view angles for all sides must sum to the maximum azimuth value (360 degrees for a complete panorama).

Back to the Edit Sides Dialog.

Back to the Edit Menu.

Back to the PanoramIX Authoring Tool.

#### 4.7.4.2 Side Image Radius Values

The radius value is effectively equivalent to the focal length of the lens, assuming a full frame digitized image and a focusing distance of infinity. If the lens is not focussed on infinity, then this value represents the effective focusing distance determined by ``

                 (1/r) = (1/f) - (1/d)

where "f" is the nominal focal length of the lens and "d" is the distance from the camera to a point where an object would be in best focus. In this case, "r" represents the distance from the lens to the image plane. If "d" is less than infinity, then "r" will be greater than the focal length "f" and the field of view will be less than that obtained when the camera is set for "d" equal to infinity.

The value of the radius is expressed in pixel units. This value assumes that the height in pixels and width in pixels of the each digital image represent the 24mm by 36mm image area (frame stop) in a standard 35mm camera. If a digital image provided as input to the authoring tool does not represent the full 24mm by 36mm image area of a standard 35mm camera, then the value of the radius in pixels must must be divided by the fraction of the full image area represented by the digital image. For example. if in the process of printing or scanning an image, 10% of the width or height of the original image is lost, then the value of "r" must be divided by 0.9 to compensate for the lost area. In this case, the value of "r" corresponds to the distance from the lens to a film plane in which a particular digital image corresponds to an area of 24mm by 26mm.

The initial value of the radius is determined by the width of the digital image and its estimated field of view. That is, ``

                   radius = (width/2)/tan(fov/2).

If a digital image represents less than 100% of the frame stop area, and/or the camera is focussed on a distance less than infinity, then may be necessary to increase the value of the radius. Because of uncertainties in how much of the original image may have been lost in printing or scanning, and in the focus distance setting of the camera, it may be difficult to determine the correct value for the radius in an a priori manner. In such cases, the best value for the radius may be determined through trial and error. In addition, the "Align All" function in the AutoStitch dialog will attempt to determine the best value for the radius based on the amount of overlap between the images.

There is a button labeled "All" next to the "radius" field. If this is selected (down), then the same radius value will be forced on all side images, as well as top and bottom images. If this button is not selected, then the radius values may be specified independently for each image. As long as the same camera and lens are used to capture all of the images with the same focusing distance and degree of image cropping, then the same value of the radius will be appropriate to all of the images comprising a single panorama.

The specified radius value(s) take effect when the Apply button is selected.

Back to the Edit Sides Dialog.

Back to the Edit Menu.

Back to the PanoramIX Authoring Tool.

#### 4.7.4.3 The Camera Elevation Angle

The camera elevation angle represents the angle between the camera direction and a plane perpendicular to the axis of rotation for the panorama. The camera direction is defined by the direction of a line passing through the center of the image and the center of the lens. If the axis of rotation is perfectly vertical, and the camera direction is perfectly horizontal, then the elevation angle is exactly zero. Even with a good tripod and pan-head, the camera direction vectors may differ from true horizontal by a degree or more in either direction.

The elevation angle setting allows the panorama editor to compensate for distortions to the input images caused by differences between the camera elevation angles and true horizontal. Such differences tend to be greatest near the tops and bottoms of each image.

When a non-zero elevation angle is selected, a third horizontal line will be drawn across the side image the Edit Image window. This line indicates the level of the true horizontal in the original raw image. That is, this line shows where the vertical center of the image would have been if the camera were aimed exactly horizontal (perpendicular to the axis of the panorama).

This center line can only move in one-pixel steps. Clicking on the spin control associated with the elevation angle moves this center line position up and down by one pixel and presents the corresponding elevation angle in the numerical field. Only these values of the elevation angle can be selected. Any other value entered for the elevation angle will be replaced by one of these values when a panorama is rebuilt. The size of the angular step associated with a single pixel step in the center line depends on the resolution of the image, with higher resolutions allowing smaller angular steps.

If the button labeled "All" next to the elevation angle is selected (down), then the specified elevation angle will be applied to all side images. This should be selected initially to handle any gross or systematic errors in the camera elevation angles. After correcting for such errors, this "All" button should be de-selected (up) to allow each image to be corrected individually.

Back to the Edit Sides Dialog.

Back to the Edit Menu.

Back to the PanoramIX Authoring Tool.

#### 4.7.4.4 Camera Roll Rotation Angles

In addition to random and systematic errors in the camera direction from image to image, it is also possible to have random and systematic errors in the roll angle of the camera about the direction vector. Even if the camera is mounted on a good quality pan head, and the orientation of the camera is checked with a bubble level, it is possible to have small errors in the roll angle of the camera.

In the case of a partial panorama, it may be possible to ignore small errors in the camera roll angles. In the case of a complete panorama, however, these errors accumulate in a screw-like manner which prevent the last image in a sequence from matching with the first image.

In principal, errors in the camera roll angles may be corrected by rotating the images about the center point determined by the horizontal and vertical shift values. Such image rotation functions, however, can introduce unwanted aliasing artifacts into the rotated image unless each pixel is determined by an appropriate combination of the pixels in the original image. Consequently, the panorama editor supports two means of compensating for camera rotation errors, a simple built-in method, and an external method.

The built-in method for compensating for camera roll angle errors is based on a simple skewing algorithm. In this case, each column of an image is shifted up or down by a few pixels before being clipped or blended into a new panoramic image. The external method, which is currently under development and not yet available, is based on transferring image data from the panorama editor to other image processing software such as Adobe PhotoShop where more sophisticated image rotation algorithms may be applied. The resulting rotated image is then transferred back to the panorama editor.

This portion of the Edit Sides dialog provides a data field presenting the current value of the skew parameter. This value represents the total amount of skewing to be applied from the left edge to the right edge of the original raw image. When the skew value is non-zero, then the upper and lower clipping boundaries will be represented by diagonal lines instead of horizontal lines.

The value of the skewing parameter for each image may be entered directly in this data field, or it may be changed incrementally using the associated spin control. In addition, values of the skewing parameters may be determined automatically for complete panoramas by using the "Align All" function in the AutoStitch dialog.

If the button labeled "All" next to the skew value is selected (down), then the specified skew value will be applied equally to all side images. This may be selected initially to handle any gross or systematic errors in the camera rotation angles. After correcting for such errors, this "All" button should be de-selected (up) to allow each image to be corrected individually.

Back to the Edit Sides Dialog.

Back to the Edit Menu.

Back to the PanoramIX Authoring Tool.

### 4.7.5 Apply, Delete, and Close buttons

There are three buttons at the bottom of the Edit Sides dialog labeled Apply, Delete, and Close. The Apply button updates the panorama based on the currently specified properties. The Close button closes the Edit Sides dialog, discarding any property changes entered since the most recent use of the Apply button. The Delete button removes the selected side image from the list of images comprising the current panorama and advances the side selection to the next available side. Deleting the last remaining side can have unpredictable results.

Back to the Edit Sides Dialog.

Back to the Edit Menu.

Back to the PanoramIX Authoring Tool.

Last modified February 27, 1998.
wluken@watson.ibm.com