Tips for improving PointMesh results in PhotoModeler Scanner

The following article answers some commonly asked questions about PhotoModeler Scanner and how to create good quality PointMeshes. Many of the questions have to do with modeling the human face or body, but most of the tips and methods apply to any object or scene.

The most common reason for a poor DSM result, is a poor camera orientation. That is, PhotoModeler does not solve for the correct camera positions for each photo. Poor orientations can be caused by mis-references, mis-marking, inadequate points marked per image, or poor camera calibrations.

The other common problems are a poor definition of the ‘base’ surface and/or incorrect depth range settings.

Recommended setup for good PointMesh results

  • use a digital camera with a fixed (non-zoom) lens (or keep a stable zoom)
  • use a digital camera where auto-focus can be turned off
  • ensure a very good camera calibration at a distance similar to one being used in the project (calibration project total error less than 1.0 and highest residual less than 0.5 pixel)
  • the DSM project should have a mixture of angles for the photos but use only low angle, shifted photos (stereo pairs) for DSM creation
  • the DSM project should be high quality (total error less than 1.0 and highest residual less than 1.0 pixel)
  • ensure good lighting: strong enough so there is a good depth of field and no camera blur, and lights that do not move – ie. don’t use a camera-mounted flash
  • use subpixel targets in the scene where possible to help with camera orientation and project strength
  • ensure the base surface is adequately defined (may need some manually marked and referenced points on the object within the trims)
  • pick a suitable depth range (especially important for weaker textures)

Why is orientation so important? Is it really necessary for the point mark residuals in a DSM project to be below 1.0?

Without good orientations the intricate matching done by PM Scanner will not be reliable and will usually result in noise in the point cloud. The best indicator of orientation quality is the point mark residuals.

Consider that when PM Scanner is matching one part of a photo to a part on another photo if it does not know the relationship between those photos accurately it can make wrong decisions about matches.

Strong orientations result from a) good camera calibration, b) precise marking,  and c) correct point references. Aim for at least 10 points on and/or around the object appearing in each pair of photos.. Using coded targets speeds up the process. You will notice in the tutorial projects that coded targets appear in each image and these are used mainly for orientation.

Once your photos are oriented, you can look at each point that is not a sub-pixel target dot and that has a high residual and fine-tune the 2D mark location.  Usually zooming in and using the arrow keys on the keyboard is a convenient way to fine-tune a marked point.  After re-processing and adjusting the point locations (you may need a to do few rounds of this), you can usually get the orientations to solve very well – where all residuals are below about 1.0 pixels. Note, you should NOT adjust points so that they become incorrectly marked in order to drop the residual (ie artificially lowering the residual) – you must adjust the points so that they are MORE precisely marked – this ensures strong camera orientation. Using targets for orientation is preferrable since PM Scanner can mark circular targets accurately and efficiently and this manual adjust step can be skipped.  

Adding one or more high angle photos to the project can improve overall orientation quality. These high angle photos may or may not be used for DSM generation, but by improving the overall orientation quality, the photos where DSM creation is done, will work better.

What is the best camera to use for PhotoModeler Scanner projects?

You can usually get a result using most digital cameras. However, for best results use a digital camera with a fixed (non-zoom) lens (or keep a stable zoom), and one where auto-focus can be turned off. You will also want to turn off settings like auto-rotation and high sharpening if you are using subpixel targets.

Ensure a very good camera calibration at a distance similar to one being used in the project (calibration project total error less than 1.0 and highest residual less than 0.5 pixels).

Note that a camera with a macro lens (or macro setting) can be problematic because very small changes in focus can cause large changes in camera parameters, such as focal length and lens distortion. This will mean the camera taking the photo is different than the one calibrated.

 What is a good depth range and why?

The depth range tells PM Scanner how far to search in the other image for a ‘good’ match. When you use a small depth range, PM Scanner does not need to search as widely for matching points, and this will be faster and can often lead to better results. On the other hand if the depth range is too small it won’t be able to match the correct area of the image (so might cause noise or no data).

Often the default depth range set by PM Scanner is best. If you select the ‘Depth Range’ property on the DSM Options dialog, the “Recompute default depth range” hyperlink control appears. Click that and the depth range will change to the default.

An important factor to consider when creating DSM’s is that PM Scanner uses a surface as the base for the depth range. The search depth range is above and below this surface. The surface can either be explicit (you select a surface before opeing DSM dialog), or implicit.  The 3D points in the project (and that appear on the current stereo pair) are used to form the implicit surface.

The implicit surface is formed in two stages. First a flat plane is fit to all 3d points on the photo stereo pair (ie. all 3d points referenced on both photos of pair) If you have points marked and referenced inside a DSM Trim on this pair, then these points are used to deform the planar surface so that the implicit surface is ‘built up’ and fit through these points (like a mountain range). 

If your DSM does not look right with the default depth range, even with well oriented photos, you may need to mark additional points to define a better implicit surface. The closer the implicit base surface, usually the better the DSM, due to a smaller required depth range. The more random the texture the less exacting the depth range and base surface definition need to be. Human skin on the other hand is a weak texture, so the depth range and base surface are more important. On a human face for example, to improve your base implicit surface you might mark points on the tip of the nose, in the eyes, and on the cheeks and chin.
The Help file discusses the notion of an implicit or base surface in more detail.

When marking and referencing points for the base surface they do not have to be marked at maximum precision. You’ll notice that when you reference these points they will immediately obtain a 3D position without processing (technically this is called solving by resection) if your photos are oriented. These approximate points do not need a low residual if they’re not used in processing so you won’t need to fine tune their point mark locations (you can also set them to “Do Not Use in Processing” on the properties to make sure they don’t affect your orientations). Once they have a 3D position they will be used by PM Scanner for the base/approximate surface. Do not process your project once you have defined these approximate points as processing can alter your orientations and can cause misalignment with your DSM’s.

Why shouldn’t I process my project once my cameras are oriented and I’ve created DSMs?

Once your cameras are well oriented, you DO NOT want to process the project anymore. Processing can adjust orientations, and if you create new DSM’s they may not align with those created prior to processing.
It is very important to first get your model solving very well so that you are confident in your camera orientations. The best indicator of strong orientations are your point mark residuals (should all be 1.0 or less) and your processing Total Error (should be 1.0 or less).

What are the best lighting conditions for scanning humans? Are there good settings for the camera?

The lighting requirements and camera settings will depend on the subject, but generally you want as flat light as possible so that the random texture and detail on the surface is clear in all images. You don’t want very bright/over-exposed areas, and you don’t want very dark areas, where detail is lost either way. You also want to make sure there is no glare or highly reflective areas. The lighting should not change during photography. A camera-mounted flash should not be used as it will cause the ‘look’ of the texture to change between photos.  This is especially true of skin because of its mutli-layer nature.

What are the best distances and angles between the stereo pairs?

The best distances depend on the size and shape of the object. The important value to consider is the Base-to-Height ratio. The ratio is described in the Help document. PM Scanner works best with a ratio of about 0.3, but there is some flexibility, where a somewhat smaller or bigger ratio can also work. The ‘base’ is the distance between cameras in the stereo pair, while the ‘height’ is the distance from cameras to the surface. Example, if the object is about 1m from the camera pair, the cameras in the pair should be about 0.3m (30cm) apart. Also the camera positions should be more or less parallel to each other and not converged inwards.

What settings should I use when working with faces or skin or other surfaces with a subtle ‘texture’?

When working with PM Scanner you can work with low density point clouds, and without the Meshing steps, until you find good settings. It’s a good idea to scale your project, so that settings like depth range and sample rate are relatable (rather than in an arbitrary scale).

Once you find good settings on the DSM options dialog that appear to capture the shape well, you can then increase the density (smaller sample rate number) to generate the more detailed point cloud. Lastly use the meshing steps to create a surface and clean it up. These steps minimize overall processing time since the potentially long point cloud generation and meshing steps are done less often. 

See below for more specifics on basic settings.

What sampling rate do you recommend?

See the note above about experimenting with various settings.

Low density (large sample rate value) give you smoother surfaces but not much detail.. Too high a density (small sample rate value) can sometimes result in wavy or ripple artifacts.  The actual value for the sample rate will depend on your desired results and the nature of the imagery. The default sampling rate is a good place to start (but does tend to err on the size of less detail).

For wavy and bump artifacts, you might want to experiment with the Super-sampling factor. Increasing the factor slows the subpixel stage down, but may result in a smoother surface. Also, increasing the Matching Region Radius can help.

The best way to smooth a DSM is to use a less dense point cloud. If you want more detail in a scan (past where bump and wave artifacts appear), the resolution of the images needs to increase. The wavy / rippled result is usually because there is not enough pixel data to accurately represent the surface at that sample rate. If you need more detail use higher resolution images (or move the camera closer to the subject) so that there is more pixel data for PM Scanner to work with.

What advanced settings are good for faces or skin or other surface with a subtle ‘texture’?

There is no one set of settings that works for all projects, since all shapes model differently, so some experimentation may be required. However, a good starting point for Advanced Settings for skin texture are:
Sub-pixel: On
Super-sampling factor: 2
Matching Region Radius: 20
Texture Type: 1

Do I need camera synchronization?

If you subject is static and still, you can often complete it with a single camera moved to different locations.

When modeling a human face or body or other animate object, using two or more cameras that are synchronized can help. Even subtle movements between photos can throw the DSM algorithm off (since an area in one photo will not be properly matched in another, due to movement).  The method of synchronizing cameras depends on the model of camera.  One method uses a remote control to fire multiple tripod-mounted cameras at the same time. When using a remote control to activate the camera, you should be mindful that there may be delays in some cameras. 

One test of synchronization is to listen and check that all shutters are released at the same time. You might also want to do a synchronization test by holding a stopwatch in the view of all cameras and viewing how well synchronized the shots were.

The faster the object moves, the tighter the synchronization should be. If you have an animate object but only one camera you will need to find a way to keep the subject extremely still during photography. This is difficult for most animate objects (humans, animals, etc.).

How should I position my cameras when taking photos of a body or face?

The configuration shown below generally works best for an object that has surface irregularities (eg protrusions like a nose, indentations like the space between arm and body etc):

Camera Setup

When you have a 4 camera setup like this you can use the horizontal and vertical pairs to capture detail throughout the object.  The horizontal pairs will capture the details in the shape such as the area above and below the nose or brow on a face. The vertical pairs will help capture the details in shape to the sides of the nose or body. The multiple meshes that result can be merged into one mesh and surfaced.