Synthetic Training Images For Real-World Object Detection

Deep learning image classification and object detection models have been proposed as methods to support analysts across numerous domains in identifying potentially relevant images and triaging others for additional human review. Current open source licensed and commercial deep learning algorithms for understanding image content focus on commonly known items - plants, animals, buildings, sports equipment, etc. For many applications, these common set of classes may be suitable. However, multiple efforts within the nuclear nonproliferation community currently focus on applications of these algorithms for international safeguards to support the review of surveillance images, open source images, and images within an existing internal data repository. For safeguards applications, models must be fine-tuned to recognize the specific characteristics of relevant objects or classes. Research is ongoing to try to limit the numbers of images required to fine-tune these models, but typically several thousand examples of a single class are needed. Curating a dataset of this magnitude poses several challenges for international safeguards: 1) Proliferation-relevant images may be rare due to their sensitivity or the limited availability of a technology; 2) creating relevant images through real-world staging is costly and introduces biases into the resulting model; and 3) expert-labeling is expensive, time consuming, and prone to error and dissent. To address these challenges, we are generating high-quality, three-dimensional computer graphic models which we use to render large numbers of images with high-variance lighting, background, perspective, and material properties that can be used to train deep learning models. In this paper, we will discuss our research goals, our experimental plan, and our results to-date using virtual, rendered images to train deep learning object detection models to recognize real-world images.