Skip to content

Basic_3D_Mesh_cleanup_and_merging

Jump to bottom
Sergejs edited this page Jul 26, 2022 · 1 revision

3D mesh clean-up and merging

Processing the raw 3D-scanned data into a model suitable for Mesh2HRTF simulation.

------------------------------------------------------------ Part of the the Complete Beginner’s tutorial ------------------------------------------------------------
Previous <<< 3D scanning last updated in 2022-06-23 Next >>> 3D mesh optimization

Video version of this tutorial (note, it is not necessary to watch every second of these videos - jump through the chapters to get an overview):

Mesh preparation for HRTF simulation is likely the most time consuming part of the whole process. Pre-processing of the scanned data includes fixing of holes, merging multiple perspectives, initial triangle count reduction and removing any objects and hair that should not be in the mesh.

Key ideas:

  • Consider watching the video tutorial - this tutorial is quite long and hard to follow, but there is a video instead.
  • For simulations the 3D mesh must be prepared just like for 3D printing: air-tight, manifold shell without loose, overlapping, duplicated or hidden geometry (that means there can be only one continuous outer surface - nothing can be hidden inside the "head").
  • For efficiency it is important to understand which details are relevant and which areas will get dramatically simplified in the final simulation mesh – in practice only ears and areas next to the ears need to be good, the rest of the mesh can be left quite rough and approximate. Final high-quality simulation model will have triangle edges from 0.5 mm on the ears up to 5-10 mm on the less important areas (see example).

Software to use

There is a lot of software that can be used for 3D mesh editing. In fact most tools and tutorials that explain how to prepare 3D scanned data for 3D printing apply for Mesh2HRTF usecase (as long as the model is not scaled, distorted of heavily simplified in the process). For example here is one video on YouTube to get started.

As different 3D tools have their strengths, weaknesses and availability limitations – it is nearly impossible to recommend any specific software or workflow. Therefore, this tutorial only provides examples of the basic workflow – there is no attempt to recommend any particular tool for any task. Here is some software to consider:

  • Meshlab – open-source, fast, has advanced algorithms to fix certain problems, quirky interface and it has no “Undo” feature.
  • Meshmixer – free, Windows only, nice interface, complete feature set, has sculpting tools, but it is no longer updated, uses 1-CPU core only and on some computers it can work so slow that it can be practically unusable. (Poor performance may be linked to not enough system resources for given mesh detail level or some other issues. A workaround is to use alternative software to reduce polycount before first import into Meshmixer, for example Instant Meshes or Meshlab.)
  • Blender – open-source, has sculpting tools & is a necessary part of the Mesh2HRTF workflow anyway.
  • Microsoft 3D Builder – free, Windows only, can “repair” certain kinds of mesh errors which break other programs. Also great to fix wrong units in the file (if a mesh is wrong size due to wrong units applied to the data).
  • OpenFlipper – very interesting alternative to Meshlab (both open source). Has most of the Meshlab features, nice interface, but is less popular (not as many tutorials).
  • Instant Meshes – free, may be useful for initial remeshing only – see video.
  • Proprietary software for the specific professional 3D scanner that you may have.
  • Any other suitable software.

Personal experience so far (to be updated with better approaches):

  • For initial fixes Meshmixer is easy to use but slow. Meanwhile Meshlab is faster and more powerful but without Undo it is less fun to use.
  • Merging of base and detail 3D scans can be done very nicely in Meshmixer, but also Meshlab has aligning tools that can work.
  • Sculpting to correct for hair and other defects can be done in Blender or in Meshmixer.
  • To fix some issues with Blender .ply file export it is always great to have Meshmixer available as well as Microsoft 3D Builder (but there must be other alternatives as well).
  • Remember that in the end we will dump our much-too-high-resolution mesh into a re-meshing tool that will instantly optimize the mesh for simulation.

3D mesh clean-up and merging steps

**This chapter is written primarily around Meshlab (alt1) & Meshmixer (alt2) ** based on personal experience. But the basic steps should be applicable for most other tools.

Initial assumption is that the 3D data comes from the 3D Scanning process described in the previous tutorial. All example data was scanned with iPhone and uses the principle that a rough "Base head scan" is merged together with higher quality "Detail scans" of ears.

In case the starting point is a single high quality 3D mesh then most of the described steps can be skipped.

  1. Pick the best files: 3D scans to use. If the data comes from non-professional 3D scanner (something not very robust, like iPhone) there must be multiple takes of the same scene. In practice, even professional 3D scanners may benefit from multiple takes because of mistakes while scanning, algorithm errors and subject movement.

    • After this step, only the best versions of the “Base head” and “Detail scans” for each side should remain.

    • Meshlab (alt1) is an excellent tool to quickly open and review all models. It is possible to use “Import Mesh” and select many/all 3D scans at the same time – the mesh loading performance is adequate.

    • Meshmixer (alt2) can also do this job, but it may take longer to open big STL files. Note that to speed up work in Meshmixer (if there is enough RAM) – open several instances in parallel and while you wait for one instance to load/calculate something, continue working with the other Meshmixer windows.

    • Check for obvious artifacts & quality of the reference geometry. It is easy to spot deformations and poor details in the refrence geometry as an indicator of overall quality.

    • Check for complex holes and corrupt geometry in important areas – It is very difficult to reconstruct important missing details on the ears. On the other hand the details in the hair, back of the skull, neck and even nose are not very important and will be re-meshed to quite low resolution anyway.

    • Check for deformations by comparing “Quick scans” to the “Detail scans” (if the fast reference scans are available) – the Quick scans can be imported into the scene together with the “Detail scans” or “Base head” to check for relative distortions and even to correct some problem areas.

      • Meshlab (alt1): For aligning two scans use “Point Based Gluing” technique within “Align” tool (example videos 1, 2, 3).
      • Meshmixer (alt2): For aligning two scans use “Align To Target” technique. (plus “Attract brush” under Sculpt menu of similar “Attract To Target” tool can be useful to correct distortion in the Detail meshes – to deform Detail mesh to match the reference Quick scan).

  2. Simplify the scene: the raw data from the 3D scanner can include garbage objects and can have excessively high polycounts which slow down all other processing :

    • Note – if the scanned scene contains Geometric references which were constant among all scans (were never moved), then it is best to keep these features for the best possible mesh alignment in the following step (and delete after alignment).

    • Meshlab (alt1) is reasonably fast:

      1. Delete connected unrelated geometry: use “Edit => Select Faces/Vertices inside polyline area” and click around the main mesh to roughly select the useful geometry. Press “Q” to make selection. Press “C” to clear the polyline. Then press “i” to invert selection and “Del” to remove useless geometry. (use “D” to deselect all).
      2. Delete loose garbage geometry: use “Edit => Select Connected Components in a region” and drag over the main mesh to select. Then press “i” to invert selection and “Del” to remove useless geometry.
      3. Save & “Reload all layers” after this step. Make sure that there is a backup of all 3D files that are being edited in Meshlab!!! Then use “Export Mesh…” to overwrite the opened files. (It is not recommended to use “Export mesh as” because Meshlab will continue reading the original files and will become confusing.) Finally “Reload all layers” to make sure the following steps will use the correct input.
      4. For the “Base Head” only - use “Surface Reconstruction: Screened Poisson” (more info) to generate a new mesh around the scan data. Set “Pre-Clean” to Yes and adjust the “Reconstruction Depth = 10” or try values between 9 and 12. (This feature quickly resolves a-lot of small problems in the initial mesh for the price of insignificant decrease of precision.)
      5. Reduce polycount to a more manageable amount and improve topology: Select “Simplification: Quadric Edge Collapse Decimation” (under “Filters => Remeshing…”) The default settings are OK, just adjust either the “Percentage reduction” (0.5 for 50% reduction worked well for data generated by “Heges 3D”) or “Target number of faces” to keep enough details. Be careful to not discard too much useful details. Here is an example of 50% reduction on a Detailed scan from Heges 3D app:
      6. Again Save & “Reload all layers”: use “Export Mesh…” on each model and then “Reload all layers” to make sure the following steps will use the correct data.

    • Meshmixer (alt2): almost the same as Meshlab:

      1. Delete connected unrelated geometry: Press “s” to start selecting. Drag mouse for “Screen-Space Lasso mode” to roughly select the useful geometry. Then press “i” to invert selection and “Del” to remove useless geometry.

      2. Delete loose garbage geometry: 1- Press “s” to start selecting and select any point on the good mesh. 2- Press “e” to expand the selection over all connected geometry. 3- “i” to invert and 4- “Del” to finish.

      3. Save after this step. Use native “.mix” file format for quicker re-loading.

      4. For the “Base Head” only - use “Make Solid” function (example videos 1, 2) to effectively shrink-wrap a nice new mesh around the scan data. Adjust the “Solid Accuracy Cell size” and “Mesh density Cell size” together. A value of about 2 mm can be reasonable for the Base Head mesh. (This feature quickly resolves a-lot of small problems in the initial mesh for the price of insignificant decrease of precision.)

      5. Reduce polycount to a more manageable amount and improve topology: Select geometry, then press “R”. The settings for the rough remesh are: first reset “Remesh” settings to defaults. Then use “Mode – Target Edge Length”. These settings should work fine, but you could try “Preserve Sharp Edges” option. For the “Target edge length” use:

        • 0.5 mm edge size for ear detail meshes (or original resolution if your scan has non-random, useful details at this resolution).
        • 2 mm edge size for Base Head mesh.
        • If shoulders are included – 5 mm edge size is sufficient for everything under the neck on the Base Head mesh (select relevant mesh by lasso).
        • (Remesh helps to reduce polycount for faster processing, eliminates insignificant mesh defects and (at this stage) it will even-out polygon placement to make any smoothing tools work consistently without creating extra polygons.)

  3. Align the scans. It can be good to make mesh alignment as early as possible because it is then possible to use reference geometry for alignment as well (before it is trimmed away).

    • Meshlab (alt1) : For aligning two scans use “Point Based Gluing” technique within “Align” tool (example videos 1, 2, 3). Note, that Meshlab algorithms use all geometry during alignment so it is important that obviously wrong geometry is deleted before aligning.
    • Meshmixer (alt2) : To align ear “Detail” meshes to the “Base head” mesh use the “Align To Target” feature (Link to manual). This tool allows to mark one object as “Target” in the Object Browser and then select which parts of the mesh are used for alignment. Note that is is usually necessary to use transform “T” to rotate meshes into approximate rough alignment before using the “Align To Target” feature.

  4. Cut out the ears from the Detail scan meshes.

    • Note that it important to have very smooth transition from a cheek to the ear. So it is better to keep a good chunk of the cheek from the detailed scan.

    • Meshlab (alt1) procedure:

      1. Rotate the view so that one ear is clearly visible.
      2. Select the Detail scan object for the correct ear.
      3. Cut out the high-quality ear: use “Edit =>Select Faces in a rectangular region” and draw a box around the ear that will be kept. Then press “i” to invert selection and “Del” to delete all unnecessary geometry.
      4. Select the Base mesh object.
      5. Cut out the hole for the ear: use “Edit =>Select Faces in a rectangular region”, HOLD DOWN “Alt” to only select visible faces and draw slightly larger box around the ear that will be kept. Then press “Del” to make a hole. (there will be some loose faces left - not a problem)
      6. Repeat the steps for the other side.
      7. Delete loose garbage geometry on the Base mesh: use “Edit => Select Connected Components in a region” and drag over the main mesh to select it. Then press “i” to invert selection and “Del” to remove leftovers from the ear cutouts.
      8. Again Save & “Reload all layers”: Use “Export Mesh…” on each model and then “Reload all layers” to make sure the following steps will use the correct data.

    • Meshmixer (alt2) procedure: basically the same as with Meshlab, only instead of rectangular selection it is more practical to use "Sphere Brush" of a large size.

  5. Trim away all excess geometry. Using the methods described previously review and remove all geometry that is not supposed to be there.

    • From this point it may be worth switching to Meshmixer due to the nicer interface than Meshlab. Alternatively some of the operations can be performed in Blender directly after Meshlab.
    • Meshmixer tip – selection tool is very powerful – try to be creative with it, for example “expand ring” is very useful to select every triangle in tricky places (select something nearby, then expand/contract selection with “<” and “>” until you are sure that you got everything. Then Ctrl-click to deselect excess triangles where not needed). Meshmixer offers double-click in select mode for edge selection, optimization of selection boundary with “O” and other convenient selection tricks.
    • It can be convenient to trim away excess torso geometry at this stage. If not, this trimming can also be done later in Blender.

  6. Merge the complete head.

    • Meshlab (alt1) automatic procedure: use “Surface Reconstruction: Screened Poisson” to generate a new mesh around the scan data. Set “Pre-Clean” to Yes, “Merge all visible layers” to Yes and adjust the “Reconstruction Depth = 11” or try values between 9 and 12. (This method is not the cleanest, but can produce usable results very fast. For the best possible control, consider doing the merging step in Meshmixer).

    • Meshmixer (alt2a) automatic procedure: using “Make Solid” function (example videos 1, 2) on the maximum quality settings ("Solid Type" – Accurate; max “Solid Accuracy”; max “Mesh Density”) it is possible to achieve result that is very similar to Meshlab’s “Surface Reconstruction: Screened Poisson” filter. This approach works most of the time because it builds a whole new solid mesh using the original as a reference. Therefore it may affect some finest details, but for Mesh2HRTF purposes the loss of details should not be significant.

    • Meshmixer (alt2b) high quality stitching can be done by using manual approach (this method only fills in holes, without re-meshing the existing surfaces):

      1. Use “Edit => Combine” to join all objects into one (After the previously made ear cut-outs there should be practically no overlaps).
      2. To make long seams possible to weld-shut, use Bridge “Ctrl+B” command to simplify open seams into regular holes that can be later filled-in by “F” command.
      3. In select mode, double-click the edge of an open hole and press “F” to Erase and Fill the gap. If the command fails, it is necessary to "Bridge" the hole in more places. The end result should be new air-tight head with smoothly merged in detailed ears.
      4. Optional - to get rid of the colorful look from different stitched objects it is necessary to remove face-groups. Select all triangles “Ctrl+A” and go to “Modify” ==> “Clear Face Groups”.
      5. To fix many small or insignificant holes Meshmixer has excellent “Analysis” ==> “Inspector” tool where clicking on any red marker automatically attempts to fill all defects (but does it in the simplest, flat way).

  7. Smoothing the seams and other surface corrections. When meshes have been merged, now it is a good time to fix any remaining surface defects.

    • Check the ear-canal – Even if 3D scanning is very high quality, the ear canal opening likely needs to be modified to approximate appearance of a blocked ear-canal microphone earplug. Ideal in-ear microphones do not stick out but completely block ear canal opening. See example images (Source 1 & 2):

    • Hair must be smoothed away! The final mesh must represent the skin surface, not the hair. Same applies to any other unnatural geometry caused by reference geometry, wig cap or anything else. It all can be done in:

      • Blender (alt1) sculpting and other editing tools in Blender are a good alternative. It is possible to take a rough merged mesh directly from Meshlab into Blender and make all final surface fixes in Blender. There are good introduction video tutorials for Sculpting to get started, plus practical example is provided in the [Mesh2HRTF video tutorial]. Where Blender looses out to Meshmixer is the lack of an equivalent to "Erase and Fill" tool. The closest alternative may be "Face ==> Grid Fill" tool in Edit mode that allows to fill in holes when all perimeter edges are selected. Also Blender may not have as good built-in surface smoothness analysis tools which make it harder to spot small but still significant surface defects.

      • Meshmixer (alt2) has may be even better than Blender for the final fixes:

        1. Usual Erase and Fill “F” command with some practice is often the best tool to make smooth transitions and remofe anything that sticks out. Experiment with Bulge setting and select larges areas to Erase and fill so that all bumps are replaced by the smooth transition. Also experiment with Refine setting to find appropriate level. If the seams were placed in non-critical areas, then there are no issues with detail loss.
        2. Sculpting Smoothing Brushes are also very often used. (when Erase & Fill is not enough). For sculpting it is important to experiment with different settings to get the desired result. The “Secondary brushes” are usually the most useful (Robust smooth, Shrink smooth and Bubble smooth). Effect of sculpting brushes also differs depending on the triangle density in the underlying mesh, so sometimes it is good to “Refine” or “Reduce” certain area to get desired effect. Note holding down “Ctrl” button reverses the effect of a tool (instead of pulling the tool pushes into a surface).

  8. Export the final high resolution, unoptimized mesh to Blender - Simply export your merged air-tight mesh to the “.OBJ” or “.PLY” format and proceed to the next tutorial step.

    • Remember that mesh must be air-tight! No holes are allowed at this stage.
    • Remember to keep a backup of your high-resolution 3D mesh before down-sampling it for practical use in Mesh2HRTF.

How to make sure that 3D mesh is ready

The following 4 checks will give 99+% confidence that the mesh will simulate correctly. (the ultimate test is to successfully run the simulation at high-enough maximum frequencies):

  1. Air-tight mesh check – the mesh must be suitable for 3D printing (and have no overlapping or internal geometry), so the best checks are done by the software focused on 3D-printing.

    • Meshmixer has the excellent “Analysis” ==> “Inspector” tool which always should show that there are no errors to fix. See example of 2 files - one with lots of errors and one without errors.

    • Microsoft 3D Builder is another easy to use check & fix program that can detect and automatically fix mesh problems. It is enough to just open a file and the program automatically suggest to fix it if there are any problems detected.

    • Meshmixer: Note that any mesh that was generated by "Make Solid" function is almost guaranteed to pass all checks (but this operation causes loss of small details).

  2. Scale check - while the final mesh may look perfect, if it does not match the correct size in millimeter units, the results will be garbage (of the simulation may fail). While it is impossible to detect small scaling (+/- 10%) that may be introduced by accidental use of scaling commands,

    • Measure head width - most editors have some measurement tool and the distance between ears should be between 100 and 180 mm. If the measurement is dramatically different, chances are that the file was saved or imported with wrong measurement units (dimensions are correct but only when measured in another unit).
    • For changing mesh units the easiest method is using Microsoft 3D Builder. When any 3D model is opened in Microsoft 3D Builder, the program both shows the scale chart in the background and allows to change the measurement unit. If the units are adjusted, it is best to use "Save As" and resave the file to apply the unit changes.

  3. Surface smoothness checks - these checks may not be necessary, but there are tools that help to visualize potential surface defects that may be hard to spot.

    • Meshlab (alt1) - try different Filters:

      • “Filters ==> Normals, Curvature and orientations ==> Discrete Curvatures ==> Type: Gaussian Curvature” can highlight potentially problematic areas.
      • Try out other functions under “Filters ==> Normals, Curvature and orientations ==> ...”.

    • Meshmixer (alt2) Try these tools and there is “F” "Erase and Fill" tool to directly fix any problem area:

      • “Analysis ==> Mesh Query ==> Gaussian Curvature” can highlight red areas on parts of the mesh that are intended to be smooth.
      • “Analysis ==> Mesh Query ==> Internal Angle” can highlight potential problem areas.
      • “Analysis ==> Thickness” should not find any errors.

    • "- view 3Dmesh_orig.bat" (alt3) - the PMP Library mesh viewer (mpview) that is easily accessible on Windows from Mesh2HRTF tools and it can also be compiled on other platforms. It has “Curvature ==> Gaussian Curvature” visualizer that is quick and easy to access just before or after “hrtf_mesh_grading”.

  4. “hrtf_mesh_grading” - the re-meshing/grading algorithm suggested in the next tutorial step will re-calculate all triangles and in the process is likely to fix a various small surface errors that may be left by other operations (for example "Erase and Fill" using "F" shortcut in Meshmixer is known to introduce new surface defects). Therefore it is important to not rely on final re-meshing done in for example Meshmixer, but to simulate only meshed that have passed through “hrtf_mesh_grading” or another reliable remeshing algorithm.

    • In case “hrtf_mesh_grading” crashes or outputs bad data - it usually means that there are some hidden mesh defects still present in the model (quite common after sculpting or with some 3D scanners).

  5. Last resort fixes for almost any mesh surface defect that causes problems:

    • Meshmixer use “Edit ==> Make Solid” with very high quality settings (Solid Type – Accurate; max “Solid Accuracy”; max “Mesh Density”). This approach works most of the time because it builds a whole new solid mesh (by 3D-printing standards) using the original as a reference. Therefore it may affect some finest details, but for Mesh2HRTF purposes the loss of details should not be significant.

      • - MakeSolid is still the most reliable method to fix a "broken" mesh.

    • Blender approach is less robust than Meshmixer's "Make Solid", but can be good enough assuming the mesh has only small surface defects, not major holes or hidden geometry. The approach is to go to Sculpt mode, click on "Remesh" settings and for "Voxel Size" either sample the mesh inside the ears (try sampling at least 3 times and pick the lowest value) or choose resolution manually - for example "0.16". Then just execute the remesh and export the huge file to be processed by “hrtf_mesh_grading”.

Next tutorial step >>> 3D mesh optimization

Clone this wiki locally