Shyam Gollakota

Latest News

May 2025: Check out LlamaPIE, proactive in-ear conversation assistants that is to appear at ACL

Apr 2025: Bandhav gets his Ph.D. and joins Meta Llama team, congrats! (Graduation pics #1, #2)

Apr 2025: Check out our work on spatial speech translation that appeared at CHI (MIT Tech Review)

Nov 2024: Awarded the Infosys Prize 2024 in Engineering & Computer Science

Nov 2024: Nature Electronics paper on hearable devices with sound bubbles (IEEE Spectrum)

Oct 2024: Check out our work on AI cameras for detecting clinical medication errors

Sep 2024: EMNLP paper on full-duplex dialogue agents

Jul 2024: Interspeech paper introduces the task of target conversation extraction

Jul 2024: Introducing knowledge boosting for model collaboration during low-latency inference

May 2024: Check out our work on target speech hearing that appeared at CHI (Popular science, Tech Review)

Apr 2024: Look once to hear for target speech hearing wins honorable mention award at CHI 2024

Nov 2023: Check out our exciting work on semantic hearing that appeared at UIST (Tech Review)

Sep 2023: Nature Communications paper on acoustic swarms to create speech zones

Sep 2023: Science Robotics paper on battery-free origami microfliers

July 2023: SIGCOMM paper on underwater GPS for smart devices

June 2023: Justin Chan to join CMU as an assistant professor

March 2023: Deep learning for low-power video streaming, NeuriCam

Jan 2023: Waveformer architecture to enable real-time target sound extraction

Nov 2022: Geekwire reports on Google acquisition of Sound Life Sciences

Nov 2022: TUNE project on helping create universal newborn hearing screening in Kenya (Nature Biomedical Engineering paper, Wired)

June 2022: Check out our work on AI-enabled wireless clearbuds (IEEE Spectrum, TechCrunch, Nvidia)

June 2022: Vikram Iyer awarded the ACM SIGMOBILE dissertation award

March 2022: AAAI paper on deep learning for on-device directional hearing

March 2022: Nature paper on computers and sensors that float in the wind like dandelion seeds

Feb 2022: Nature Communications paper on blood clot testing using smartphones

Students

Maruchi Kim, PhD student

Bandhav Veluri, PhD student, Meta Fellowship

Tuochao Chen, PhD student, Meta Fellowship

Malek Itani, PhD student

Guilin Hu, PhD student

Alumni

Justin Chan, PhD, assistant prof at CMU

Anran Wang, PhD, Sound Life Sciences → Google

Vikram Iyer, PhD, assistant prof at University of Washington, Marconi Young Scholar, SIGMOBILE Dissertation award

Mehrdad Hessar, PhD, OctoML

Ali Najafi, PhD, VMWare Research

Kelly Michaelsen, Mentee, assistant prof at UW Medicine

Rajalakshmi Nandakumar, PhD, assistant prof at Cornell, Marconi Young Scholar, SIGMOBILE Dissertation award

Vamsi Talla, PhD, CTO of Jeeva Wireless, SIGMOBILE Dissertation award, SIGCOMM Dissertation award

Bryce Kellogg, MS, co-founder Jeeva Wireless

Vincent Liu, assistant prof at U. Penn

Aaron Parks, co-founder Jeeva Wireless

Collin Pernu, Undergraduate, Starfish space

Hans Gaensbauer, Undergraduate, MIT PhD student

Hao Zhang, Undergraduate, Wisconsin PhD student

Kyle Johnson, Undergraduate, UW PhD student

Vicente Arroyos, Undergraduate, UW PhD student

Jose Jaime, Undergraduate, Sound Life Sciences → Google

Ryan Feng, Undergraduate, Michigan PhD student

Qifan Pu, Undergraduate, Berkeley PhD

Awards

Infosys Prize, 2024

ACM CHI Honorable mention award, 2024

Moore Inventor Fellow, 2021

ACM Grace Murray Hopper Award, 2020

SenSys Best Paper Award, 2018

IMWUT Distinguished Paper Award, 2018

SIGMOBILE RockStar Award, 2017

Forbes 30 under 30 All-Star Alumni, 2017

Popular Science Brilliant Ten, 2016

SIGCOMM Best Paper Award, 2016

Visionaries 2020, CNN, 2016

NSDI Best Paper Award, 2016

10 Tech Breakthroughs, MIT Review, 2016

World Technology Award, 2015

CoMotion Presidential Fellow, 2015

NSF Career Award, 2015

Alfred Sloan Fellowship, 2015

Forbes 30 under 30 Energy, 2015

Technology Review TR35, 2014

ACM MOBICOM Best Paper Award, 2013

ACM SIGCOMM Best Paper Award, 2013

George M. Sprowls Award for Best Dissertation, MIT, 2013

ACM Doctoral Dissertation Award, 2012

ACM SIGCOMM Dissertation Award, 2012

ACM SIGCOMM Best Paper Award, 2011

ACM SIGCOMM Best Paper Award, 2008

William A. Martin SM Thesis Award, 2008

IIT Madras Institute Award in Computer Science, 2006

Inventions from my lab

My lab has made multiple inventions including battery-free computers that float in the wind, origami microfliers, ambient backscatter, Wi-Fi sensing and imaging, acoustic swarms, battery-free phones, active sonar on smart devices and Internet of bio-inspired and biological things. We also created various computing tools for medical diagnostics like contactless breathing, cardiac arrest and irregular heart rhythm monitoring, blood clot and ear infection testing using smartphones and new-born hearing screening using earbuds.

We are currently working on augmenting the human auditory perception with artificial intelligence to create enhanced human capabilities like semantic hearing, target speech hearing, sound bubbles and, more recently, spatial speech translation that have until now been in the realm of science fiction. We have also exploring proactive in-ear conversation assistants that can anticipate human needs and move beyond currently widespread human-AI “chat” interfaces.

Latest Publications (see all)

Wireless Hearables With Programmable Speech AI Accelerators

Malek Itani, Tuochao Chen, Arun Raghavan, Gavriel Kohlberg and Shyam Gollakota
ACM MOBICOM, 2025.

webpage

code

video

Neural Speech Extraction with Human Feedback

Malek Itani, Ashton Graves, Sefik Emre Eskimez, and Shyam Gollakota
InterSpeech, 2025.

webpage

code

video

LlamaPIE: Proactive in-ear conversation assistants

Tuochao Chen, Nicholas Batchelder, Alisa Liu, Noah Smith, and Shyam Gollakota
Findings of the Annual Meeting of the Association for Computational Linguistics, 2025.

webpage

code

video