Building a 3D instance segmentation model for CAD designs

TL;DR:

• 🧩 Built a 3D instance segmentation model, crucial for automated cost estimation in CAD manufacturing.
• 🔧 Adapted OneFormer3D for CAD designs, using point normals instead of color information.
• 📊 The approach achieves robust segmentation with minimal training data, making it viable for industrial applications.

At D3S, we're developing cost estimation models for Computer-Aided Design (CAD) manufacturing. A crucial part of this process is automatically identifying and measuring design features that drive manufacturing costs - like holes, bends, and surface areas. For sheet metal designs in particular, accurate segmentation of one side of the sheet enables automated unfolding and nesting calculations, essential for material cost estimation.

Unlike 3D similarity tasks where pre-trained models often perform well, segmentation requires specialized approaches that understand manufacturing context. The challenge lies in precisely identifying every feature while maintaining robustness across diverse design variations.

Method

The development process involved testing multiple approaches before finding an optimal solution:

1. Semantic Segmentation Exploration:

   • Mask3D, PointSAM, and PTv3 were tested.
   • PTv3 was the most successful but lacked native instance segmentation capabilities.

2. Evolution to Instance Segmentation:

   • Implemented PointGroup with PTv3 backbone.
   • Advanced and sticked to OneFormer3D with internal CAD-specific adaptations.

Experiments

Our experimental process revealed three key insights:

• CAD-specific adaptations are crucial for effective instance segmentation. Notably, CAD 3D designs do not have color information, so we used point normals instead.
• Dataset curations are essential. Various curations techniques were implemented to ensure robustness across diverse designs. The model also needs a minimum amount of training data to generalize well (~ more than 1000 designs).
• Such 3D instance segmentation methods, initially aimed for outdoor and indoor scenes, can be effectively adapted to CAD designs part segmentation.

Conclusion

This work demonstrates the effectiveness of current SOTA 3D instance segmentation models for 3D CAD part segmentation. Our OneFormer3D adaptation, using point normals instead of colors, achieves robust results even with modest training data. This makes the solution particularly valuable for industrial applications where labeled data is scarce.

The model's success in identifying features enables automated cost estimation and manufacturing planning, providing practical value in real-world manufacturing workflows.

For technical details, feel free to contact me!