Conference paper
Single-Shot Analysis of Refractive Shape Using Convolutional Neural Networks
The appearance of a transparent object is determined by a combination of refraction and reflection, as governed by a complex function of its shape as well as the surrounding environment. Prior works on 3D reconstruction have largely ignored transparent objects due to this challenge, yet they occur frequently in real-world scenes.
This paper presents an approach to estimate depths and normals for transparent objects using a single image acquired under a distant but otherwise arbitrary environment map. In particular, we use a deep convolutional neural network (CNN) for this task. Unlike opaque objects, it is challenging to acquire ground truth training data for refractive objects, thus, we propose to use a large-scale synthetic dataset.
To accurately capture the image formation process, we use a physically-based renderer. We demonstrate that a CNN trained on our dataset learns to reconstruct shape and estimate segmentation boundaries for transparent objects using a single image, while also achieving generalization to real images at test time.
In experiments, we extensively study the properties of our dataset and compare to baselines demonstrating its utility.
Language: | English |
---|---|
Publisher: | IEEE |
Year: | 2019 |
Pages: | 995-1003 |
Proceedings: | 19th IEEE Winter Conference on Applications of Computer Vision |
ISBN: | 1728119758 , 1728119766 , 9781728119755 and 9781728119762 |
Types: | Conference paper |
DOI: | 10.1109/WACV.2019.00111 |
ORCIDs: | Stets, Jonathan Dyssel and Frisvad, Jeppe Revall |
Computer vision Estimation Glass Image reconstruction Rendering (computer graphics) Shape Three-dimensional displays
3D reconstruction CNN arbitrary environment map convolutional neural nets deep convolutional neural network image reconstruction image segmentation image sensors learning (artificial intelligence) object detection opaque objects physically-based renderer reflection refraction refractive objects refractive shape rendering (computer graphics) segmentation boundaries single image single-shot analysis transparent object