Multi-Scale Improves Boundary Detection in Natural Images


Boundary detection is a fundamental problem in vision and, just like any other vision task, is multi-scale in nature. The 1980s saw a large number of studies on edge detection and many explicitly addressed the scale issue. In fact, the fabled Canny edge detector [2] was multi-scale -- i.e. feature synthesis. Scale-space theories were developed, in the context of edge detection leading to Lindeberg's scale-space edge detector [3].

However, these studies on multi-scale edge detection did not materialize. The matlab implementation of Canny is single-scale. The recent Probability-of-Boundary detector (Pb) [4], outperforming previous detectors on natural images in the Berkeley Segmentation Dataset, is also single-scale. In fact, the authors of [4] specifically claimed that they did not find any benefit going multi-scale.

I find that hard to believe -- surely natural images are multi-scale! The goal of this work is to empirically "prove" that multi-scale significantly improves boundary detection in natural images.

Multi-Scale Boundary Cues

We base our approach on Pb, which combines brightness, color and texture contrasts of two half-disks with a fixed size. The simplest multi-scale extension of Pb is to compute the Pb contrast value under multiple disk sizes. Combining these contrast values can be cast in the same classification formulation, and in our case a linear logistic classifier is used.

There are many other cues that provide boundary information at multiple scales. One cue is contrast normalization; a salient boundary has a contrast much higher than its surroundings. Another cue is localization -- the peak location of contrast for a smooth boundary does not change when we measure contrast under varying scales; while the peak of a false boundary, say in texture regions, shifts around.

Once we have a dense contrast map combining multi-scale signals, one important detail is how to do non-maximum suppression. Empirically we find out that it is best to do non-maximum suppression using only fine-scale contrast. This is consistent with intuition that fine-scale edges are better at localization.

Empirical Results on Five Datasets

This is empirical work and we have tested our algorithms on five datasets with groundtruth boundaries. Results are shown in the form of precision-recall curves (as in [4]). In all cases multi-scale improves boundary detection by a significant margin. We also compare to Canny and Scale-Space Edges.
The Berkeley Segmentation Dataset
Occlusion/Motion Boundary Dataset MSRC Object Dataset V2
PASCAL 2007 Segmentation Challenge LabelMe (Boston Houses)

Thus we have empirically proved that multi-scale improves boundary detection performance. It is our hope that soon every boundary detector will be multi-scale in nature and can work with high-resolution images from modern cameras.


  1. Multi-Scale Improves Boundary Detection in Natural Images.   [abstract] [pdf]
      Xiaofeng Ren, in ECCV '08, Marseille 2008.

  2. A Computational Approach to Edge Detection.
      J. Canny, in PAMI 8(6), 1986.

  3. Edge detection and ridge detection with automatic scale selection.
      T. Lindeberg, in IJCV 30, 1998.

  4. Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues.
      D. Martin, C. Fowlkes and J. Malik, PAMI 2004.