(he/him)
![]() |
I am a Postdoctoral Research Associate in Computer Science & Engineering at University of Washington. I am working with Prof. Steve Seitz, Prof. Brian Curless and Prof. Ira Kemelmacher-Shlizerman in the UW Reality Lab and GRAIL. I will be joining the University of North Carolina at Chapel Hill as an Assistant Professor of Computer Science starting Fall 2022 I completed my Ph.D. (Aug 2013 - Jan 2019) from University of Maryland, College Park (UMD), advised by Prof. David Jacobs and my undergraduate degree (2009-2013) in ECE from Jadavpur University, Kolkata, India. I also had the pleasure to work with:
|
Email: soumya91 at cs.washington.edu
Follow @SenguptRoni
Website: Google Scholar
Resume/CV: CV
Office: CSE2 233, The Bill & Melinda Gates Center
Mail: Paul G. Allen School of Computer Science & Engineering,
University of Washington, Seattle
Constraints and Priors for Inverse Rendering from Limited Observations
Soumyadip Sengupta
Doctoral Thesis, University of Maryland, January 2019
[pdf]
Real-Time Light-Weight Near-Field Photometric Stereo Near-field Photometric Stereo technique is useful for 3D imaging of large objects. We capture multiple images of an object by moving a flashlight and reconstruct the 3D mesh. Our method is significnatly faster and memory-efficient while producing better quality than SOTA methods. |
|
Robust High-Resolution Video Matting with Temporal Guidance Background Removal a.k.a Alpha matting on videos by exploiting temporal information with a recurrent architecture. Does not require capturing background image or manual annotations. |
|
A Light Stage on Every Desk We learn a personalized relighting model by capturing a person watching YouTube videos. Potential application includes relighting during a zoom call. |
|
Shape and Material Capture at Home High-quality Photometric Stereo can be achieved with a simple flashlight. Recovers hi-res geometry and reflectance by progressively refining the predictions at each scale, conditioned on the prediction at previous scale. |
|
Real-Time High Resolution Background Matting Background replacement at 30fps on 4K and 60fps on HD. Alpha matte is first extracted at low-res and then selectively refined with patches. |
|
Lifespan Age Transformation Synthesis Age transformation from 0-70. Continuous aging is modeled by assuming 10 anchor age classes with interpolation in the latent space between them. |
|
Background Matting: The World is Your Green Screen By simply capturing an additional image of the background, alpha matte can be extracted easily without requiring extensive human annotation in form of trimap. |
|
Neural Inverse Rendering of an Indoor Scene from a Single Image Self-supervision on real data is achieved with a Residual Appearnace Renderer network. It can cast shadows, add inter-reflections and near-field lighting, given the normal and albedo of the scene. |
|
SfSNet : Learning Shape, Reflectance and Illuminance of Faces in the Wild Decomposes an unconstrained human face into surface normal, albedo and spherical harmonics lighting. Learns from synthetic 3DMM followed by self-supervised finetuning on unlabelled real images. Soumyadip Sengupta, Daniel Lichy, Angjoo Kanazawa, Carlos D. Castillo, David Jacobs.IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2020. [Paper] Introduces SfSMesh that utilizes the surface normal predicted by SfSNet to reconstruct a 3D face mesh. |
|
A New Rank Constraint on Multi-view Fundamental Matrices, and its Application to Camera Location Recovery We prove that a matrix formed by stacking fundamental matrices between pairs of images has rank 6. We then introduce a non-linear optimization algorithm based on ADMM, that can better estimate the camera parameters using this rank constraint. This improves Structure-from-Motion algorithms which require initial camera estimation (bundle adjustment). |
|
Solving Uncalibrated Photometric Stereo Using Fewer Images by Jointly Optimizing Low-rank Matrix Completion and Integrability We solve uncalibrated Photometric Stereo using as few as 4-6 images as a rank-constrained non-linear optimization with ADMM. |
|
Frontal to Profile Face Verification in the Wild We introduce a dataset of frontal vs profile face verfication in the wild -- CFP. We show that SOTA face verification algorithms degrade about 10% on frontal-profile verification compared to frontal-frontal. Our dataset has been widely used to improve face verification across poses, but also for face warping and pose synthesis with GAN. |
|
A Frequency Domain Approach to Silhouette Based Gait Recognition |
“An evolutionary multiobjective sleep-scheduling scheme for differentiated coverage in wireless sensor networks”, Soumyadip Sengupta, Swagatam Das, Md Nasir, AV Vasilakos, Witold Pedrycz, IEEE Transactions Systems, Men and Cybernetics - Part C, 2012. [Paper]
“A dynamic neighborhood learning based particle swarm optimizer for global numerical optimization”, Md Nasir, Swagatam Das, Dipankar Maity, Soumyadip Sengupta, Udit Halder, PN Suganthan, Elsevier Information Sciences, 2012. [Paper]
“Evenly spaced pareto front approximations for tricriteria problems based on triangulation”, Günter Rudolph, Heike Trautmann, Soumyadip Sengupta, Oliver Schütze, Evolutionary Multi-Criterion Optimization, 2013.[Paper]
Teaching Assistant: ENEE 222 Spring 2014 : Elements of Discrete Signal Analysis
Teaching Assistant: ENEE 420 Fall 2013 : Communications Systems