Here are some of the research projects I have been working on.
Learning to Learn how to Learn: Self-Adaptive Visual Navigation using Meta-Learning
M Wortsman, K Ehsani, M Rastegari, A Farhadi and R Mottaghi (CVPR19, Oral Presentation)
There is a lot to learn about a task by actually attempting it! Learning is continuous, i.e. we learn as we perform. Traditional navigation approaches freeze the model during inference (top row in the intuition figure above). In this paper, we propose a self-addaptive agent for visual navigation that learns via self-supervised interaction with the environment (bottom row in the intuition figure above).
SAVN is a network that
Learns to adapt to new environments without any explicit supervision,
Uses meta-reinforcement learning approach where an agent learns a self-supervised
interaction loss that encourages effective navigation,
And shows major improvements in both success rate and SPL for visual navigation in novel scenes.
K Ehsani, R Mottaghi, A Farhadi (CVPR18, spotlight)
Humans have strong ability to make inferences about the appearance of the invisible and occluded parts of scenes. For example, when we look at the scene on the left we can make predictions about what is behind the coffee table, and can even complete the sofa based on the visible parts of the sofa, the coffee table, and what we know
in general about sofas and coffee tables and how they occlude each other.
SeGAN can learn to
Generate the appearance of the occluded parts of objects,
Segment the invisible parts of objects,
Although trained on synthetic photo realistic images reliably segment natural images,
By reasoning about occluder-occludee relations infer depth layering.