This paper presents an explore-and-classify framework for structured architectural reconstruction from an aerial image. Starting from a potentially imperfect building reconstruction by an existing algorithm, our approach 1) explores the space of building models by modifying the reconstruction via heuristic actions; 2) learns to classify the correctness of building models while generating classification labels based on the ground-truth; and 3) repeat. At test time, we iterate exploration and classification, seeking for a result with the best classification score. We evaluate the approach using initial reconstructions by two baselines and two state-of-the-art reconstruction algorithms. Qualitative and quantitative evaluations demonstrate that our approach consistently improves the reconstruction quality from every initial reconstruction.
Video
Overview
The geometry exploration module starts from an initial model reconstructed by another algorithm, and produces a set of offspring reconstructions by heuristic actions. The geometry classification module ranks the generated reconstructions and keep the top samples for the next iteration (graphs with green color score on the right corner are the top samples in each iteration). The system dynamically generates training data for the classification module by picking generated reconstructions and creating classification labels based on the ground-truth. The classification module consists of three local primitive classifiers (i.e., junctions, edges, and regions). The first two classifiers are neural networks, which learns to classify if each primitive in the current reconstruction is correct or incorrect. The region classifier measures the consistency with the instance segmentation masks generated by Mask R-CNN. The overall classificationscore is the sum of junction, edge, and region scores.
Related Projects
Paper
