|
Introduction
Optical flow requires spatial integration, which essentially poses a
grouping question: what points belong to the same motion and what do not.
Classical local approaches to optical flow, such as Lucas-Kanade, use isotropic
neighborhoods and have considerable difficulty near motion boundaries.
While motion itself is a strong grouping cue, there exist many other grouping
cues such as brightness and texture similarities. In this work we show that
such single-image grouping cues can help avoid erroneous motion integration near
boundaries and greatly improve optical flow accuracy.
Soft Grouping via Intervening Contour
Local grouping cues in a single image can be well captured in a soft boundary
map such as that from the Probability-of-Boundary
detector [2]. Intervening contour [3]
defines a pairwise affinity between two points by measuring the "resistance" or
contour energy along a straight line path. This affinity function
defines the (soft) spatial neighborhood for local motion integration.
One needs to be extra careful with affinities from/to boundaries, as they
provide strong motion cues but are also potential motion discontinuities, in
which case the boundary motion is only consistent with the "figure" side.
we use an asymmetric intervening contour scheme as follows
| 
| 
|
| (a)
| (b)
| (c)
|
| Asymmetric intervening contour: (a) pairwise affinity defined by
contour energies along straight line paths; (b) A is strongly connected
to B and weakly to C and D (as C and D may belong
to the other side and should not influence A); (c) C is strongly
connected to all three to overcome the aperture problem.
|
Semi-Local optical flow using Pairwise Affinity
We proceed as in Lucas-Kanade, using the differential optical flow constraint
and local motion integration. We use the affinity values as weighting in a
robust least square estimation (please see the paper for details). Instead of a
fixed square or gaussian, the affinity-based spatial neighborhood is soft and
adaptive both in anisotropicity and scale.
In a near-uniform region, affinity values remain high on long distances,
resulting in large spatial neighborhoods. To make the motion integration
feasible, we sample the image first, at both boundaries and corners. These
sample points both retain most motion cues and are sufficient to represent
motion discontinuities.
Our results on the Middlebury Flow dataset are promising (note we do not use
color or explicitly model smoothness of flow)
|
| Dimentrodon
| Hydrangea
| RubberWhale
| Venus
| Marble
|
| AAE
| 3.34
| 2.77
| 5.32
| 3.93
| 4.88
|
|
| Army
| Mequon
| Schefflera
| Wooden
| Grove
| Urban
| Yosemite
| Teddy
|
| AAE
| 8.00
| 11.1
| 12.6
| 5.84
| 4.68
| 9.29
| 2.11
| 5.75
|
| 
| 
| 
|
| 
| 
| 
|
| 
| 
| 
|
| 
| 
| 
|
| (a)
| (b)
| (c)
| (d)
|
|
Visualization: (a) image (b) color-coded groundtruth (c) our results (d) angular error
|
References
- Local Grouping for Optical Flow.
[abstract]
[pdf]
Xiaofeng Ren, in CVPR '08, Anchorage 2008.
- Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues.
D. Martin, C. Fowlkes and J. Malik, PAMI 2004.
- Contour continuity in region based image segmentation.
T. Leung and J. Malik, ECCV 1998.
|