Neeraj Kumar

Postdoctoral Research Associate with Steve Seitz

Box 352350, Paul G. Allen Center 282

Seattle, WA 98195-2350

Research Overview

I work at the intersection of computer vision and machine learning, developing techniques that leverage large collections of real-world images for a variety of applications. I am particularly interested in the use of intermediate representations such as parts and attributes, both for improving performance on classical vision tasks such as search and recognition, as well as for creating novel applications such as automatic face replacement and exploration of image collections. Much of my work is directed toward human faces, although I am also very interested in the growing area of fine-grained visual categorization, which seeks to distinguish categories within a particular domain, such as Leafsnap -- which identifies plant species from photographs of their leaves.

Watch a recent talk I gave at UNC-Chapel Hill describing my PhD work.

News

October 14, 2012: ECCV 2012 Leafsnap presentation slides released: The slides for our ECCV 2012 presentation on Leafsnap are now available.
August 01, 2012: ECCV 2012 paper on Leafsnap released: Our ECCV 2012 paper on Leafsnap is now available. This will be an oral presentation in Florence, Italy this October.
June 20, 2012: Multi-attribute spaces project page and software: Project page for multi-attributes paper added to website, as well as link to attribute calibration software.
June 18, 2012: I'm at CVPR 2012!: I'm at CVPR 2012, and I'd love to meet you! Come to our demo on computing face attributes live and doing advanced multi-attribute similarity searches (Monday, Demo Booth #6, all day concurrent with the posters), and our poster on calibrating multiple attribute values using Multi-Attribute Spaces (Wednesday 2nd Poster session 3B, 1:30-3:30, #4).

Curriculum Vitae

CV (pdf)
Citations to my work

Teaching (see all)

CSE 590V: Computer vision seminar (Fall 2011)

Selected Projects (see all)

Leafsnap: An Electronic Field Guide

Summer 2009 - Present

Automatically identifying plant species using photos of leaves taken in the field by mobile devices

Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search

Fall 2011 - Summer 2012

Calibrating the outputs of multiple attribute classifiers into normalized multi-attribute spaces for fusion and similarity search

Localizing Parts of Faces Using a Consensus of Exemplars

Fall 2009 - Summer 2011

Robustly localizing face parts using local detectors and non-parametric, exemplar-based global models

Attribute and Simile Classifiers for Face Verification

Spring 2009 - Present

Identifying faces using describable visual attributes of the face, automatically detected using attribute and simile classifiers

FaceTracer: A Search Engine for Large Collections of Images with Faces

Spring 2008 - Present

Searching face images using automatically-trained classifiers of descriptions of face and image attributes

Face Swapping: Automatically Replacing Faces in Photographs

Fall 2007 - Fall 2008

Using large collections of images to automatically replace faces in photos

Selected Software (see all)

Attribute Calibration Software

Summer 2012 - Present

C/C++/Python code to calibrate attribute classifier outputs using the w-score formulation, based on the Extreme Value Theory. For non-commercial uses only.

Leafsnap

Spring 2011 - Present

The Leafsnap homepage, with links to download the iPhone and iPad apps, as well as browse the list of species and their images.

MugHunt Online Face Search Engine

Spring 2010 - Present

An online face search engine built using this research. This search engine is a joint collaboration with Securics, Inc..

Selected Publications (see all)

"Leafsnap: A Computer Vision System for Automatic Plant Species Identification," (oral presentation)
Neeraj Kumar, Peter N. Belhumeur, Arijit Biswas, David W. Jacobs, W. John Kress, Ida Lopez, João V. B. Soares,
Proceedings of the 12th European Conference on Computer Vision (ECCV),

October 2012.

[pdf] [bibtex] [slides (ppt)]
@InProceedings{leafsnap_eccv2012,
author = {Neeraj Kumar and Peter N. Belhumeur and Arijit Biswas and David W. Jacobs and W. John Kress and Ida Lopez and João V. B. Soares},
title = {Leafsnap: A Computer Vision System for Automatic Plant Species Identification},
booktitle = {The 12th European Conference on Computer Vision (ECCV)},
month = {October},
year = {2012}
}
"Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search,"
Walter Scheirer, Neeraj Kumar, Peter N. Belhumeur, Terrance E. Boult,
Proceedings of the 25th IEEE Conference on Computer Vision and Pattern Recognition (CVPR),

June 2012.

[pdf] [bibtex] [poster]
@InProceedings{multiattrs_cvpr2012,
author = {Walter Scheirer and Neeraj Kumar and Peter N. Belhumeur and Terrance E. Boult},
title = {Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search},
booktitle = {The 25th IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2012}
}
"Fusing with Context: a Bayesian Approach to Combining Descriptive Attributes," (oral presentation)
Walter Scheirer, Neeraj Kumar, Karl Ricanek, Terrance E. Boult, Peter N. Belhumeur,
Proceedings of the IEEE International Joint Conference on Biometrics (IJCB),

October 2011.

[pdf] [bibtex]
@InProceedings{fusion_ijcb2011,
author = {Walter Scheirer and Neeraj Kumar and Karl Ricanek and Terrance E. Boult and Peter N. Belhumeur},
title = {Fusing with Context: a Bayesian Approach to Combining Descriptive Attributes},
booktitle = {The IEEE International Joint Conference on Biometrics (IJCB)},
month = {October},
year = {2011}
}
"Two faces are better than one: Face recognition in group photographs,"
Ohil K. Manyam, Neeraj Kumar, Peter N. Belhumeur, David J. Kriegman,
Proceedings of the IEEE International Joint Conference on Biometrics (IJCB),

October 2011.

[pdf] [bibtex] [poster]
@InProceedings{twofaces_ijcb2011,
author = {Ohil K. Manyam and Neeraj Kumar and Peter N. Belhumeur and David J. Kriegman},
title = {Two faces are better than one: Face recognition in group photographs},
booktitle = {The IEEE International Joint Conference on Biometrics (IJCB)},
month = {October},
year = {2011}
}
"Describable Visual Attributes for Face Verification and Image Search,"
Neeraj Kumar, Alexander C. Berg, Peter N. Belhumeur, Shree K. Nayar,
IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI),

October 2011.

[pdf] [bibtex] [slides (ppt)] [video]
@InProceedings{facever_pami2010,
author = {Neeraj Kumar and Alexander C. Berg and Peter N. Belhumeur and Shree K. Nayar},
title = {Describable Visual Attributes for Face Verification and Image Search},
booktitle = {IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI)},
month = {October},
year = {2011}
}
"Describable Visual Attributes for Face Images," (PhD Thesis)
Neeraj Kumar,
Technical Report CUCS-035-11, Department of Computer Science, Columbia University,

August 2011.

[pdf] [bibtex] [slides (ppt)]
@InProceedings{nkthesis,
author = {Neeraj Kumar},
title = {Describable Visual Attributes for Face Images},
booktitle = {Technical Report CUCS-035-11, Department of Computer Science, Columbia University},
month = {August},
year = {2011}
}
"Localizing Parts of Faces Using a Consensus of Exemplars,"
Peter N. Belhumeur, David W. Jacobs, David J. Kriegman, Neeraj Kumar,
Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition (CVPR),

June 2011.

[pdf] [bibtex] [slides (ppt)] [poster]
@InProceedings{fiducials_cvpr2011,
author = {Peter N. Belhumeur and David W. Jacobs and David J. Kriegman and Neeraj Kumar},
title = {Localizing Parts of Faces Using a Consensus of Exemplars},
booktitle = {The 24th IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2011}
}
"Attribute and Simile Classifiers for Face Verification," (oral presentation)
Neeraj Kumar, Alexander C. Berg, Peter N. Belhumeur, Shree K. Nayar,
Proceedings of the 12th IEEE International Conference on Computer Vision (ICCV),

October 2009.

[pdf] [bibtex] [slides (ppt)]
@InProceedings{facever_iccv2009,
author = {Neeraj Kumar and Alexander C. Berg and Peter N. Belhumeur and Shree K. Nayar},
title = {Attribute and Simile Classifiers for Face Verification},
booktitle = {The 12th IEEE International Conference on Computer Vision (ICCV)},
month = {October},
year = {2009}
}
"FaceTracer: A Search Engine for Large Collections of Images with Faces,"
Neeraj Kumar, Peter N. Belhumeur, Shree K. Nayar,
Proceedings of the 10th European Conference on Computer Vision (ECCV),

pp. 340--353, October 2008.

[pdf] [bibtex] [slides (ppt)] [poster]
@InProceedings{facesearch_eccv2008,
author = {Neeraj Kumar and Peter N. Belhumeur and Shree K. Nayar},
title = {FaceTracer: A Search Engine for Large Collections of Images with Faces},
booktitle = {The 10th European Conference on Computer Vision (ECCV)},
month = {October},
year = {2008}
}
"What is a Good Nearest Neighbors Algorithm for Finding Similar Patches in Images?,"
Neeraj Kumar, Li Zhang, Shree K. Nayar,
Proceedings of the 10th European Conference on Computer Vision (ECCV),

pp. 364--378, October 2008.

[pdf] [bibtex] [poster]
@InProceedings{nn_eccv2008,
author = {Neeraj Kumar and Li Zhang and Shree K. Nayar},
title = {What is a Good Nearest Neighbors Algorithm for Finding Similar Patches in Images?},
booktitle = {The 10th European Conference on Computer Vision (ECCV)},
month = {October},
year = {2008}
}
"Face Swapping: Automatically Replacing Faces in Photographs," (oral presentation)
Dmitri Bitouk, Neeraj Kumar, Samreen Dhillon, Peter N. Belhumeur, Shree K. Nayar,
ACM Transactions on Graphics (Proceedings of SIGGRAPH),

vol. 27, no. 3, August 2008.

[pdf] [bibtex] [slides (ppt)] [slides (ppt+videos)]
@InProceedings{facereplace_siggraph2008,
author = {Dmitri Bitouk and Neeraj Kumar and Samreen Dhillon and Peter N. Belhumeur and Shree K. Nayar},
title = {Face Swapping: Automatically Replacing Faces in Photographs},
booktitle = {ACM Transactions on Graphics (Proceedings of SIGGRAPH)},
month = {August},
year = {2008}
}

Neeraj Kumar

Projects

Databases

Software

Teaching

Publications

CV