TeethDreamer: 3D Teeth Reconstruction from Five Intra-oral Photographs

Chenfan Xu, Zhentao Liu, Yuan Liu, Yulong Dou, Jiamin Wu, Jiepeng Wang, Minjiao Wang, Dinggang Shen, Zhiming Cui

International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2024

Orthodontic treatment usually requires regular face-to-face examinations to monitor dental conditions of the patients. When in person diagnosis is not feasible, an alternative is to utilize five intra-oral photographs for remote dental monitoring. (view more ) However, it lacks of 3D in formation, and how to reconstruct 3D dental models from such sparse view photographs is a challenging problem. In this study, we propose a 3D teeth reconstruction framework, named TeethDreamer, aiming to restore the shape and position of the upper and lower teeth. Given f ive intra-oral photographs, our approach first leverages a large diffu sion model’s prior knowledge to generate novel multi-view images with known poses to address sparse inputs and then reconstructs high-quality 3D teeth models by neural surface reconstruction. To ensure the 3D con sistency across generated views, we integrate a 3D-aware feature atten tion mechanism in the reverse diffusion process. Moreover, a geometry aware normal loss is incorporated into the teeth reconstruction process to enhance geometry accuracy. Extensive experiments demonstrate the superiority of our method over current state-of-the-arts, giving the po tential to monitor orthodontic treatment remotely. (view less )

[Project Page] [Paper] [Code]

3D Structure-guided Network for Tooth Alignment in 2D Photograph

Yulong Dou, Lanzhuju Mei, Dinggang Shen, Zhiming Cui

The 34th British Machine Vision Conference (BMVC) 2023

Orthodontics focuses on rectifying misaligned teeth (i.e., malocclusions), affecting both masticatory function and aesthetics. However, orthodontic treatment often involves complex, lengthy procedures. (view more ) As such, generating a 2D photograph depicting aligned teeth prior to orthodontic treatment is crucial for effective dentist-patient communication and, more importantly, for encouraging patients to accept orthodontic intervention. In this paper, we propose a 3D structure-guided tooth alignment network that takes 2D photographs as input (e.g., photos captured by smartphones) and aligns the teeth within the 2D image space to generate an orthodontic comparison photograph featuring aesthetically pleasing, aligned teeth. Notably, while the process operates within a 2D image space, our method employs 3D intra-oral scanning models collected in clinics to learn about orthodontic treatment, i.e., projecting the pre- and post-orthodontic 3D tooth structures onto 2D tooth contours, followed by a diffusion model to learn the mapping relationship. Ultimately, the aligned tooth contours are leveraged to guide the generation of a 2D photograph with aesthetically pleasing, aligned teeth and realistic textures. We evaluate our network on various facial photographs, demonstrating its exceptional performance and strong applicability within the orthodontic industry. (view less )

[Project Page] [Paper] [Code] [Poster]