Machine Learning Based Perception for Inspection

Globalization raises the need for cost-effective transportation of goods and merchandise. One of the most cost-effective ways is by the sea. To ensure safe transportation, vessels are periodically inspected for faults and defects. The areas requiring inspection include confined spaces that are both difficult and dangerous to traverse for human surveyors.

Previous projects have attempted to automate the inspection process by introducing various types of robotic platforms as well as utilizing recent advances in computer vision. However, there is a gap between being able to detect a defect and being able to localize it with respect to the vessel frame and to quantify it in a human-readable manner.

Additionally, the inspection process is currently ambiguous because of corrosion residue and dirt that results in false positives. This forces surveyors to rely heavily on intuition based on experience making the inspection process highly subjective. None of the existing methods focus on capturing this ambiguity that can result in multiple inconsistent but equally valid assessments.

Therefore, this thesis presents multiple novel approaches to both address the subjective overall classification of an area under consideration during an inspection and capture the surveyor variability. The thesis aims to automate the inspection process using visual perception and to increase the objectivity of surveys by producing samples from an estimated expert distribution instead of relying on a single deterministic ground truth that is unable to capture natural variations.

During this Ph.D. project, we started with off-the-shelf deep learning architectures that have been used for object detection and localization on a wide variety of tasks. We then modified these models by expanding the number of branches to accommodate outputs that more directly align with what classification societies need.

We addressed the inherent ambiguity, induced by human subjectivity of visual inspection, by introducing a probabilistic deep learning architecture that we domain adapted and showed how a new sampling strategy can easily convergence to an expected value without loss of performance. With these findings, we have shown that the needs of the classification society can be met.

The requirements of the classification society can be met even when the system’s perception is used for optimizing the navigational behavior of the drone. Similarly, we found that the deep learning models can be constructed to support multiple ground truths and be able to estimate a distribution over them, that allows for better uncertainty estimation even across domains where domain adaptation was applied.