Conv-MPN: Convolutional Message Passing Neural Network for Structured Outdoor Architecture Reconstruction

Conv-MPN: Convolutional Message Passing Neural Network for

Structured Outdoor Architecture Reconstruction

CVPR 2020

Fuyang Zhang*, Nelson Nauata*, Yasutaka Furukawa,

Paper Arxiv Code Supp.

This paper proposes a novel message passing neural (MPN) architecture Conv-MPN, which reconstructs an outdoor building as a planar graph from a single RGB image. Conv-MPN is specifically designed for cases where nodes of a graph have explicit spatial embedding. In our problem, nodes correspond to building edges in an image. Conv-MPN is different from MPN in that 1) the feature associated with a node is represented as a feature volume instead of a 1D vector; and 2) convolutions encode messages instead of fully connected layers. Conv-MPN learns to select a true subset of nodes (i.e., building edges) to reconstruct a building planar graph. Our qualitative and quantitative evaluations over 2,000 buildings show that Conv-MPN makes significant improvements over the existing fully neural solutions. We believe that the paper has a potential to open a new line of graph neural network research for structured geometry reconstruction.

Video

Related Projects

Structured Outdoor Architecture Reconstruction by Exploration and Classification

We propose a novel explore-and-classify framework for structured outdoor architecture reconstruction, seeking to improve the quality of imperfect building reconstructions. Our system learns to classify the correctness of primitives while exploring the space of reconstructions via heuristic actions.

Paper

Bibtex

@InProceedings{Zhang2020convmpn, author = {Zhang, Fuyang and Nauata, Nelson and Furukawa, Yasutaka}, title = {Conv-MPN: Convolutional Message Passing Neural Network for Structured Outdoor Architecture Reconstruction}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2020} }