How to model a human face with PhotoModeler Scanner

A human face can be modeled using PhotoModeler Scanner (PM Scanner) using one or more pairs of stereo photos. Sometimes it is possible with a single camera, but due to the subtle movement of a human face (despite all efforts of your model to keep still!), you will usually get a better result with two or more synchronized cameras.

The following sequence of steps is based on the assumption that you know the basics of marking and referencing points (target and non-target points), that you are familiar with the basic steps of creating DSM’s with PM Scanner, and that you have a well calibrated camera.

For more basic instructions the PM Scanner tutorial projects and videos are a good place to start.

This article is closely related to the article here. There are links to particular sections of the article throughout this one.

 


Scene Setup and Photography

1. Place a series of targets on a wall or a rig so that targets surround the face. Ensure as many of the targets are visible in as many of the photos as possible.  Coded targets make orientation and setup phase very efficient.

2. Have your face model sitting comfortably and still.  If using just one camera, the model will need to be extremely still during photography.

The following shows a typical setup (one photo of a 4-synchronized cameras setup):

3. If using synchronized cameras, set up your cameras on tripods and test your synchronization (using a remote control or other remote triggering system).  See also this.

Arrange your cameras in a configuration similar to this (if not using synchronized cameras, move your camera to these positions as quickly as possible before your model moves!) and take the photos:

4 cameras plus 1 

The angles between these positions will be small and close to parallel. There should not be any more than a few degrees of toe-in angle.

Usually 4 photos will be enough to model the face and parts of the sides of the head. If you want to model the front, sides, and back, you’ll need to make sure your photo pairs cover the entire area of interest, and you’ll need to make sure that you have enough targets in view to adequately orient your photos.

You may want to include another photo from the side or above at a greater angle to assist in camera orientation. The camera in red above suggests this.  Adding this photo will help increase the angle of your referenced points and will help strengthen the solution.

 


Start the Project and get your photos oriented

1. Start a PhotoModeler Scanner Points-based project and import your photos. Alternately use an Automatic project if using RAD targets (note the steps below correspond to a Points-based project).

2. Mark the coded targets using the Automatic Target Marking tool in PM Scanner (Marking menu). If you are not using targets, mark and reference your points manually. Ensure your points are accurately marked and properly referenced, and process the project.

3. Check your point mark residuals (see Point Table) and make sure all points have residuals of less than 1.0, which indicates a well-solved model and strong camera orientations. See also this.

4. Save the project.

5. Idealize the photos (Project menu / Idealize Project). Reprocess the project.

6. Set your project scale. Setting a scale is not mandatory but it makes it easier to relate to the DSM settings such as sample rate and depth range when they are in real world coordinates.

7. It is usually a good idea to mark a DSM trim to outline and limit the area to be scanned. This will help speed up the DSM process as only the area of interest will be modeled. It will also help reduce noise around the area of interest. Mark a DSM trim on each photo.

You’ll see below that the coded targets are marked and the DSM trim outlines the face, neck and head:

DSM Trim

 

 

 

 


Set your DSM options

1. First, set a low-density sample-rate (ie a large value). You’ll want to make sure your settings are appropriate before doing a full detail DSM. Experimenting with a low-density DSM will help speed up the process of finding the optimal settings as you will spend less time waiting for DSM processing.  See also this.

2. Keep the default depth range. See also this.

3. When modeling a face or a material like skin typically your advanced DSM settings will be:

Sub-pixel: On
Super-sampling factor: 2
Matching Region Radius: 20
Texture Type: 1

See also this.

4. Run the DSM (without Meshing steps) and check that your model is solving reasonably well and the shape of the face can be discerned from the point cloud (ie not a lot of noise, or large gaps between points).

5. Evaluate your initial DSM. If the DSM has a lot of noise, go to the ‘Adjusting Depth Range and/or Base Surface’ section below. If it looks reasonable, move to the ‘Densify your Point Cloud’ section below.

This looks reasonably close (depth range of +/-25cm):
ok point cloud

This looks like a lot of noise (depth range of +/-5cm):
Noisey point cloud

 


Adjusting Depth Range and/or Base Surface

If your first attempt has a lot of noise, you’ll need to adjust your settings. If you’re confident with your orientations (look for small maxiumum residual as one indicator), the most likely culprit is the depth range setting. A depth range that is too small will not allow PMScanner to find appropriate matches, while one that is too big will also cause problems (esp. true with weak texture we find with human skin).  See also this.

Adjusting the depth range from +/-5cm to +/-25cm resulted in this improved initial DSM:
Point cloud fixed

Once you get a decent low density point cloud, you are ready to move on to denser point cloud and the meshing steps. See ‘Densify your Point Cloud’ section below.

However, if you find it difficult to find an appropriate depth range, and your DSM’s consistently result in a lot of noise and/or gaps etc, you may need to ‘build-up’ an implicit base surface. With a implicit base surface that better matches the true shape, your DSM will usually solve better, due to a smaller required depth range.  On a face for example, you might want to mark and reference points on the tip of the nose, around the eyes, and on the cheeks and chin in order to get a closer base/approximate surface. The Help file discusses the implicit surface in more detail.

This image shows a few points marked on the face. Once referenced on other photos, the points 3D positions will be used by PMScanner to build up an ‘internal’ base surface. You won’t be able to see this base surface, but PMScanner will use it as a guide when matching regions on the face.

point on face

You can now use a smaller depth range to generate a reasonable initial DSM.

The following shows the 3D points through which PMScanner internally will fit an ‘implicit’ surface:

Notice how the peaks and various elevations of the face have points to help define the base surface.

The following shows the low density point cloud with the points used for a base surface and a depth range of +/- 5cm:

3D point cloud

Notice how a depth range of +/-5cm works a lot better with the points marked to define an implicit base surface, as opposed to the example in #2 above where a +/-5cm depth range without the implicit base surface points resulted in a lot of noise.

You now have a reasonable low density point cloud and you are ready to move on to densifying it (see next section).


Densify your Point Cloud

Once your intial set up works well, increase the density of the point cloud (ie decrease the sample rate setting) and set up your meshing steps, then generate the new denser DSM, and run the meshing steps. A well solved DSM looks like this:

If your point cloud has noise points you may want to remove the noise points before meshing. See the ‘Clean up your Point Cloud’ section below.

If the triangulated PointMesh is rippled or wavy, you may need to regenerate your DSM using a higher sample rate (ie less dense point cloud). See also this.

 


Clean up your Point Cloud 

Once you have a reasonable point cloud at the density you need, you may have a few outlying noise points. If the DSM solved well and there are very few of these noise points, you may not need to clean up as that will be taken care of in the Meshing Steps (eg filter step, smoothing etc.).

However, if there are some very obvious noise points that you want to remove, you can use the PointMesh Edit tool, usually with the Region Select. In addition, adjusting the original trims so they cover only the shared common areas between photos can reduce the amount of edge noise (which is due to matching attempts of background imagery).

To use PointMesh Edit: Open the 3D Viewer, zoom and rotate to an area showing obvious noise points, select the PointMesh Edit Mode button on the DSM toolbar. Now select the Region Select tool (it will be enabled if the 3D Viewer is active). This allows you to draw a polygon region around the noise points in the point cloud. Once they are selected you can delete them by hitting Delete on the keyboard.

If you do not use the Region Select tool, the number of points selected in the point cloud when the mouse is clicked will depend on the zoom level and the number of points within a ‘hit test’ region. When zoomed in quite a bit, you can select one point at a time, but when zoomed out, a single click can select a large number of points in the cloud.

Be mindful of what you are selecting when editing the point cloud. You do not want to punch holes through it. It’s often a good idea to spin the model around to make sure the selected (highlighted in red) points are in fact noise only. Sometimes the angle at which you are viewing the model makes it hard to see the extents of your selection.

Repeat the process for the rest of your noise points and then move on to the Meshing steps, described in the section below.


Mesh your Point Cloud

Once your point cloud is free of obvious noise, you are ready to run the Meshing steps to triangulate it. The Meshing Options dialog and its various options and settings are described in the Help document in detail.

When you run the Meshing steps, the point cloud(s) you have selected will be processed. If you have more than one point cloud selected, and if you choose to run the Registration and Merge step, the selected point clouds will first be merged, before the following Meshing steps are undertaken. If you do not run the Registration and Merge step and if you have more than one point cloud selected, the meshing steps will only run on one of the selected point clouds.

Further when running the Meshing steps, a new point mesh will be created. This will preserve your original point clouds in case you want to revert to them and experiment with other Meshing settings.

Viewing the PointMesh table allows you to keep track of the point meshes created, and also shows the settings used to create them, which can come in handy when experimenting with settings and comparing various versions.