Carlo Masone

Highlighted publications

1. Journal.

   Learning Sequential Descriptors for Sequence-Based Visual Place Recognition

   Mereu, Riccardo, Trivigno, Gabriele, Berton, Gabriele,  Masone, Carlo, and Caputo, Barbara

   IEEE Robotics and Automation Letters 2022

   Abs arXiv Bib Code

   In robotics, visual place recognition (VPR) is a continuous process that receives as input a video stream to produce a hypothesis of the robot’s current position within a map of known places. This work proposes a taxonomy of the architectures used to learn sequential descriptors for VPR, highlighting different mechanisms to fuse the information from the individual images. This categorization is supported by a complete benchmark of experimental results that provides evidence of the strengths and weaknesses of these different architectural choices. The analysis is not limited to existing sequential descriptors, but we extend it further to investigate the viability of Transformers instead of CNN backbones. We further propose a new ad-hoc sequence-level aggregator called SeqVLAD, which outperforms prior state of the art on different datasets. The code is available at https://github.com/vandal-vpr/vg-transformers.

   @article{MereuTrivigno-2022,
     author = {Mereu, Riccardo and Trivigno, Gabriele and Berton, Gabriele and Masone, Carlo and Caputo, Barbara},
     journal = {IEEE Robotics and Automation Letters},
     title = {Learning Sequential Descriptors for Sequence-Based Visual Place Recognition},
     year = {2022},
     volume = {7},
     number = {4},
     pages = {10383-10390},
     doi = {10.1109/LRA.2022.3194310},
     url = {https://doi.org/10.1109/LRA.2022.3194310},
   }

2. Conference Proc.

   Rethinking Visual Geo-localization for Large-Scale Applications

   Berton, G.,  Masone, C., and Caputo, B.

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022

   Abs arXiv Bib Code

   Visual Geo-localization (VG) is the task of estimating the position where a given photo was taken by comparing it with a large database of images of known locations. To investigate how existing techniques would perform on a real-world city-wide VG application, we build San Francisco eXtra Large, a new dataset covering a whole city and providing a wide range of challenging cases, with a size 30x bigger than the previous largest dataset for visual geo-localization. We find that current methods fail to scale to such large datasets, therefore we design a new highly scalable training technique, called CosPlace, which casts the training as a classification problem avoiding the expensive mining needed by the commonly used contrastive learning. We achieve state-of-the-art performance on a wide range of datasets and find that CosPlace is robust to heavy domain changes. Moreover, we show that, compared to the previous state-of-the-art, CosPlace requires roughly 80% less GPU memory at train time, and it achieves better results with 8x smaller descriptors, paving the way for city-wide real-world visual geo-localization.

   @inproceedings{Berton-2022,
     author = {Berton, G. and Masone, C. and Caputo, B.},
     title = {Rethinking Visual Geo-localization for Large-Scale Applications},
     booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
     year = {2022},
     pages = {},
   }

3. Conference Proc. Oral

   Deep Visual Geo-localization Benchmark

   Berton, G., Mereu, R., Trivigno, G.,  Masone, C., Csurka, G., Sattler, T., and Caputo, B.

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022

   Abs arXiv Bib Code

   In this paper, we propose a new open-source benchmarking framework for Visual Geo-localization (VG) that allows to build, train, and test a wide range of commonly used architectures, with the flexibility to change individual components of a geo-localization pipeline. The purpose of this framework is twofold: i) gaining insights into how different components and design choices in a VG pipeline impact the final results, both in terms of performance (recall@N metric) and system requirements (such as execution time and memory consumption); ii) establish a systematic evaluation protocol for comparing different methods. Using the proposed framework, we perform a large suite of experiments which provide criteria for choosing backbone, aggregation and negative mining depending on the use-case and requirements. We also assess the impact of engineering techniques like pre/post-processing, data augmentation and image resizing, showing that better performance can be obtained through somewhat simple procedures: for example, downscaling the images’ resolution to 80% can lead to similar results with a 36% savings in extraction time and dataset storage requirement.

   @inproceedings{Berton-2022b,
     author = {Berton, G. and Mereu, R. and Trivigno, G. and Masone, C. and Csurka, G. and Sattler, T. and Caputo, B.},
     title = {Deep Visual Geo-localization Benchmark},
     booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
     year = {2022},
     pages = {},
   }

4. Journal

   Shared Control of an Aerial Cooperative Transportation System with a Cable-suspended Payload

   Masone, C., and Stegagno, P.

   Journal of Intelligent & Robotic Systems 2021

   Abs Bib HTML PDF Video

   This paper presents a novel bilateral shared framework for a cooperative aerial transportation and manipulation system composed by a team of micro aerial vehicles with a cable-suspended payload. The human operator is in charge of steering the payload and he/she can also change online the desired shape of the formation of robots. At the same time, an obstacle avoidance algorithm is in charge of avoiding collisions with the static environment. The signals from the user and from the obstacle avoidance are blended together in the trajectory generation module, by means of a tracking controller and a filter called dynamic input boundary (DIB). The DIB filters out the directions of motions that would bring the system too close to singularities, according to a suitable metric. The loop with the user is finally closed with a force feedback that is informative of the mismatch between the operator’s commands and the trajectory of the payload. This feedback intuitively increases the user’s awareness of obstacles or configurations of the system that are close to singularities. The proposed framework is validated by means of realistic hardware-in-the-loop simulations with a person operating the system via a force-feedback haptic interface.

   @article{Masone-2021b,
     author = {Masone, C. and Stegagno, P.},
     journal = {Journal of Intelligent & Robotic Systems},
     title = {Shared Control of an Aerial Cooperative Transportation System with a Cable-suspended Payload},
     year = {2021},
     volume = {103},
     number = {40},
     pages = {},
     doi = {10.1007/s10846-021-01457-4},
   }

5. Journal

   A Survey on Deep Visual Place Recognition

   Masone, C., and Caputo, B.

   IEEE Access 2021

   Abs Bib HTML PDF

   In recent years visual place recognition (VPR), i.e., the problem of recognizing the location of images, has received considerable attention from multiple research communities, spanning from computer vision to robotics and even machine learning. This interest is fueled on one hand by the relevance that visual place recognition holds for many applications and on the other hand by the unsolved challenge of making these methods perform reliably in different conditions and environments. This paper presents a survey of the state-of-the-art of research on visual place recognition, focusing on how it has been shaped by the recent advances in deep learning. We start discussing the image representations used in this task and how they have evolved from using hand-crafted to deep-learned features. We further review how metric learning techniques are used to get more discriminative representations, as well as techniques for dealing with occlusions, distractors, and shifts in the visual domain of the images. The survey also provides an overview of the specific solutions that have been proposed for applications in robotics and with aerial imagery. Finally the survey provides a summary of datasets that are used in visual place recognition, highlighting their different characteristics.

   @article{Masone-2021,
     author = {Masone, C. and Caputo, B.},
     journal = {IEEE Access},
     title = {A Survey on Deep Visual Place Recognition},
     year = {2021},
     volume = {9},
     number = {},
     pages = {19516-19547},
     doi = {10.1109/ACCESS.2021.3054937},
   }

6. Journal

   IDDA: A Large-Scale Multi-Domain Dataset for Autonomous Driving

   Alberti, E., Tavera, A.,  Masone, C., and Caputo, B.

   IEEE Robotics and Automation Letters 2020

   Abs arXiv Bib HTML Code

   Semantic segmentation is key in autonomous driving. Using deep visual learning architectures is not trivial in this context, because of the challenges in creating suitable large scale annotated datasets. This issue has been traditionally circumvented through the use of synthetic datasets, that have become a popular resource in this field. They have been released with the need to develop semantic segmentation algorithms able to close the visual domain shift between the training and test data. Although exacerbated by the use of artificial data, the problem is extremely relevant in this field even when training on real data. Indeed, weather conditions, viewpoint changes and variations in the city appearances can vary considerably from car to car, and even at test time for a single, specific vehicle. How to deal with domain adaptation in semantic segmentation, and how to leverage effectively several different data distributions (source domains) are important research questions in this field. To support work in this direction, this letter contributes a new large scale, synthetic dataset for semantic segmentation with more than 100 different source visual domains. The dataset has been created to explicitly address the challenges of domain shift between training and test data in various weather and view point conditions, in seven different city types. Extensive benchmark experiments assess the dataset, showcasing open challenges for the current state of the art.

   @article{Alberti-2020,
     author = {Alberti, E. and Tavera, A. and Masone, C. and Caputo, B.},
     journal = {IEEE Robotics and Automation Letters},
     title = {IDDA: A Large-Scale Multi-Domain Dataset for Autonomous Driving},
     year = {2020},
     volume = {5},
     number = {4},
     pages = {5526-5533},
     doi = {10.1109/LRA.2020.3009075},
   }