Conference paper · Journal article
Prediction of Patient Demographics using 3D Craniofacial Scans and Multi-view CNNs
Department of Health Technology, Technical University of Denmark1
Digital Health, Department of Health Technology, Technical University of Denmark2
Brain Computer Interface, Digital Health, Department of Health Technology, Technical University of Denmark3
Biomedical Signal Processing & AI, Digital Health, Department of Health Technology, Technical University of Denmark4
Department of Applied Mathematics and Computer Science, Technical University of Denmark5
Visual Computing, Department of Applied Mathematics and Computer Science, Technical University of Denmark6
Technical University of Denmark7
Stanford University8
University of Copenhagen9
3D data is becoming increasingly popular and accessible for computer vision tasks. A popular format for 3D data is the mesh format, which can depict a 3D surface accurately and cost-effectively by connecting points in the (x, y, z) plane, known as vertices, into triangles that can be combined to approximate geometrical surfaces.
However, mesh objects are not suitable for standard deep learning techniques due to their non-euclidean structure. We present an algorithm which predicts the sex, age, and body mass index of a subject based on a 3D scan of their face and neck. This algorithm relies on an automatic pre-processing technique, which renders and captures the 3D scan from eight different angles around the x-axis in the form of 2D images and depth maps.
Subsequently, the generated data is used to train three convolutional neural networks, each with a ResNet18 architecture, to learn a mapping between the set of 16 images per subject (eight 2D images and eight depth maps from different angles) and their demographics. For age and body mass index, we achieved a mean absolute error of 7.77 years and 4.04 kg/m2 on the respective test sets, while Pearson correlation coefficients of 0.76 and 0.80 were obtained, respectively.
The prediction of sex yielded an accuracy of 93%. The developed framework serves as a proof of concept for prediction of more clinically relevant variables based on 3D craniofacial scans stored in mesh objects.
Language: | English |
---|---|
Publisher: | IEEE |
Year: | 2020 |
Pages: | 1950-1953 |
Proceedings: | 42<sup>nd</sup> Annual International Conference of the IEEE Engineering in Medicine & Biology Society |
Series: | Proceedings of the Annual International Conference of the Ieee Engineering in Medicine and Biology Society, Embs |
Journal subtitle: | Enabling Innovative Technologies for Global Healthcare, Embc 2020 |
ISBN: | 1728119901 , 172811991X , 172811991x , 9781728119908 and 9781728119915 |
ISSN: | 15584615 , 1094687x , 23757477 and 26940604 |
Types: | Conference paper and Journal article |
DOI: | 10.1109/EMBC44109.2020.9176333 |
ORCIDs: | Paulsen, Rasmus Reinhold , Sørensen, Helge Bjarup Dissing and 0000-0001-6986-5254 |