Shyam Gollakota
Inventions - Awards - Publications - Code - Group

Moore Inventor Fellow
Thomas J. Cable Endowed Professor
Paul G. Allen School of Computer Science & Engineering
University of Washington
gshyam@cs.washington.edu

Head of the Mobile Intelligence Lab

Co-leading the TUNE project

Co-founder of Hearvana AI

My inter-disciplinary group spans computer scientists, electrical, mechanical & aerospace engineers, biologists and physicians. We are driven by our curiosity to tackle hard technical and societal problems. We do not constrain ourselves with the boundaries of traditional research disciplines and instead go wherever the problems take us. We work across deep learning, signal processing and hardware to build intelligent systems that previously were not thought possible. Our work has been licensed by ResMed Inc, and is in use by millions of users. Our startup Sound Life Sciences was acquired by Google. I was also CEO of a startup where we obtained FDA 510(k) clearance for the technology developed in my lab. We also worked closely with the Washington Department of Agriculture to wirelessly track invasive "murder" hornets, which resulted in the destruction of the first nest in the United States.


Latest News

May 2025: Check out LlamaPIE, proactive in-ear conversation assistants that is to appear at ACL

Apr 2025: Bandhav gets his Ph.D. and joins Meta Llama team, congrats! (Graduation pics #1, #2)

Apr 2025: Check out our work on spatial speech translation that appeared at CHI (MIT Tech Review)

Nov 2024: Awarded the Infosys Prize 2024 in Engineering & Computer Science

Nov 2024: Nature Electronics paper on hearable devices with sound bubbles (IEEE Spectrum)

Oct 2024: Check out our work on AI cameras for detecting clinical medication errors

Sep 2024: EMNLP paper on full-duplex dialogue agents

Jul 2024: Interspeech paper introduces the task of target conversation extraction

Jul 2024: Introducing knowledge boosting for model collaboration during low-latency inference

May 2024: Check out our work on target speech hearing that appeared at CHI (Popular science, Tech Review)

Apr 2024: Look once to hear for target speech hearing wins honorable mention award at CHI 2024

Nov 2023: Check out our exciting work on semantic hearing that appeared at UIST (Tech Review)

Sep 2023: Nature Communications paper on acoustic swarms to create speech zones

Sep 2023: Science Robotics paper on battery-free origami microfliers

July 2023: SIGCOMM paper on underwater GPS for smart devices

June 2023: Justin Chan to join CMU as an assistant professor

March 2023: Deep learning for low-power video streaming, NeuriCam

Jan 2023: Waveformer architecture to enable real-time target sound extraction

Nov 2022: Geekwire reports on Google acquisition of Sound Life Sciences

Nov 2022: TUNE project on helping create universal newborn hearing screening in Kenya (Nature Biomedical Engineering paper, Wired)

June 2022: Check out our work on AI-enabled wireless clearbuds (IEEE Spectrum, TechCrunch, Nvidia)

June 2022: Vikram Iyer awarded the ACM SIGMOBILE dissertation award

March 2022: AAAI paper on deep learning for on-device directional hearing

March 2022: Nature paper on computers and sensors that float in the wind like dandelion seeds

Feb 2022: Nature Communications paper on blood clot testing using smartphones


Students

Maruchi Kim, PhD student

Bandhav Veluri, PhD student, Meta Fellowship

Tuochao Chen, PhD student, Meta Fellowship

Malek Itani, PhD student

Guilin Hu, PhD student


Alumni

Justin Chan, PhD, assistant prof at CMU

Anran Wang, PhD, Sound Life Sciences → Google

Vikram Iyer, PhD, assistant prof at University of Washington, Marconi Young Scholar, SIGMOBILE Dissertation award

Mehrdad Hessar, PhD, OctoML

Ali Najafi, PhD, VMWare Research

Kelly Michaelsen, Mentee, assistant prof at UW Medicine

Rajalakshmi Nandakumar, PhD, assistant prof at Cornell, Marconi Young Scholar, SIGMOBILE Dissertation award

Vamsi Talla, PhD, CTO of Jeeva Wireless, SIGMOBILE Dissertation award, SIGCOMM Dissertation award

Bryce Kellogg, MS, co-founder Jeeva Wireless

Vincent Liu, assistant prof at U. Penn

Aaron Parks, co-founder Jeeva Wireless

Collin Pernu, Undergraduate, Starfish space

Hans Gaensbauer, Undergraduate, MIT PhD student

Hao Zhang, Undergraduate, Wisconsin PhD student

Kyle Johnson, Undergraduate, UW PhD student

Vicente Arroyos, Undergraduate, UW PhD student

Jose Jaime, Undergraduate, Sound Life Sciences → Google

Ryan Feng, Undergraduate, Michigan PhD student

Qifan Pu, Undergraduate, Berkeley PhD


Awards

Infosys Prize, 2024

ACM CHI Honorable mention award, 2024

Moore Inventor Fellow, 2021

ACM Grace Murray Hopper Award, 2020

SenSys Best Paper Award, 2018

IMWUT Distinguished Paper Award, 2018

SIGMOBILE RockStar Award, 2017

Forbes 30 under 30 All-Star Alumni, 2017

Popular Science Brilliant Ten, 2016

SIGCOMM Best Paper Award, 2016

Visionaries 2020, CNN, 2016

NSDI Best Paper Award, 2016

10 Tech Breakthroughs, MIT Review, 2016

World Technology Award, 2015

CoMotion Presidential Fellow, 2015

NSF Career Award, 2015

Alfred Sloan Fellowship, 2015

Forbes 30 under 30 Energy, 2015

Technology Review TR35, 2014

ACM MOBICOM Best Paper Award, 2013

ACM SIGCOMM Best Paper Award, 2013

George M. Sprowls Award for Best Dissertation, MIT, 2013

ACM Doctoral Dissertation Award, 2012

ACM SIGCOMM Dissertation Award, 2012

ACM SIGCOMM Best Paper Award, 2011

ACM SIGCOMM Best Paper Award, 2008

William A. Martin SM Thesis Award, 2008

IIT Madras Institute Award in Computer Science, 2006



Inventions from my lab


My lab has made multiple inventions including battery-free computers that float in the wind, origami microfliers, ambient backscatter, Wi-Fi sensing and imaging, acoustic swarms, battery-free phones, active sonar on smart devices and Internet of bio-inspired and biological things. We also created various computing tools for medical diagnostics like contactless breathing, cardiac arrest and irregular heart rhythm monitoring, blood clot and ear infection testing using smartphones and new-born hearing screening using earbuds.

We are currently working on augmenting the human auditory perception with artificial intelligence to create enhanced human capabilities like semantic hearing, target speech hearing, sound bubbles and, more recently, spatial speech translation that have until now been in the realm of science fiction. We have also exploring proactive in-ear conversation assistants that can anticipate human needs and move beyond currently widespread human-AI “chat” interfaces.




Latest Publications (see all)

Wireless Hearables With Programmable Speech AI Accelerators


Malek Itani, Tuochao Chen, Arun Raghavan, Gavriel Kohlberg and Shyam Gollakota
ACM MOBICOM, 2025.
webpage code video

TF-MLPNet: Tiny Real-Time Neural Speech Separation


Malek Itani, Tuochao Chen, and Shyam Gollakota
Clarity Challenge, InterSpeech, 2025.
webpage code video

Neural Speech Extraction with Human Feedback


Malek Itani, Ashton Graves, Sefik Emre Eskimez, and Shyam Gollakota
InterSpeech, 2025.
webpage code video

LlamaPIE: Proactive in-ear conversation assistants


Tuochao Chen, Nicholas Batchelder, Alisa Liu, Noah Smith, and Shyam Gollakota
Findings of the Annual Meeting of the Association for Computational Linguistics, 2025.
webpage code video

Spatial speech translation: Translating across space with binaural hearables


Tuochao Chen, Qirui Wang, Runlin He and Shyam Gollakota
ACM CHI, 2025.
webpage code video

Hearable devices with sound bubbles


Tuochao Chen, Malek Itani, Emre Sefik Estimez, Takuya Yoshioka, Shyam Gollakota
Nature Electronics, 2024.
webpage code video

Detecting clinical medication errors with AI enabled wearable cameras


Justin Chan, Solomon Nsumba, Mitchell Wortsman, Achal Dave, Ludwig Schmidt, Shyam Gollakota, Kelly Michaelsen
npj Digital Medicine, 2024.

Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agents


Bandhav Veluri, Ben Peloquin, Bokai Yu, Hongyu Gong, Shyam Gollakota
EMNLP, 2024.
webpage

IRIS: Wireless ring for vision-based smart home interaction


Maruchi Kim, Antonio Glenn, Bandhav Veluri, Yunseo Lee, Eyoel Gebre, Aditya Bagaria, Shyam Gollakota
ACM UIST, 2024.
webpage code video

Knowledge boosting during low-latency inference


Vidya Srinivas, Malek Itani, Tuochao Chen, Emre Sefik Eskimez, Takuya Yoshioka, Shyam Gollakota
Interspeech, 2024.
webpage code video

Target conversation extraction: Source separation using turn-taking dynamics


Tuochao Chen, Qirui Wang, Bohan Wu, Malek Itani, Emre Sefik Eskimez, Takuya Yoshioka, Shyam Gollakota
Interspeech, 2024.
webpage code video

Look once to hear: Target speech hearing with noisy examples


Bandhav Veluri, Malek Itani, Tuochao Chen, Takuya Yoshioka, Shyam Gollakota
ACM CHI, 2024 (Honorable mention award).
webpage code video

Semantic hearing: Programming acoustic scenes with binaural hearables


Bandhav Veluri, Malek Itani, Justin Chan, Takuya Yoshioka, Shyam Gollakota
ACM UIST, 2023.
webpage code video

Solar-powered shape-changing origami microfliers


Kyle Johnson, Vicente Arroyos, Amelie Ferran, Raul Villanueva, Dennis Yin, Tilboon Elberier, Alberto Aliseda, Sawyer Fuller, Vikram Iyer, Shyam Gollakota
Science Robotics, 2023.
webpage code video

Creating speech zones with self-distributing acoustic swarms


Malek Itani, Tuochao Chen, Takuya Yoshioka, Shyam Gollakota
Nature Communications, 2023.
webpage code video

Underwater 3D positioning on smart devices


Tuochao Chen, Justin Chan, and Shyam Gollakota
SIGCOMM, 2023.
webpage code video

NeuriCam: Key-Frame Video Super-Resolution and Colorization for IoT Cameras


Bandhav Veluri, Collin Pernu, Ali Saffari, Josh Smith, Michael Taylor, Shyam Gollakota
ACM MOBICOM, 2023.
code

Wireless earbuds for low-cost hearing screening


Justin Chan, Antonio Glenn, Malek Itani, Lisa Mancl, Emily Gallagher, Randy Bly, Shwetak Patel, Shyam Gollakota
ACM MobiSys, 2023.
code

Real-time target sound extraction


Bandhav Veluri, Justin Chan, Malek Itani, Tuochao Chen, Takuya Yoshioka, Shyam Gollakota
IEEE ICASSP, 2023.
webpage code

A Low-power wearable acoustic device for accurate invasive arterial pressure monitoring


Maruchi Kim, Anran Wang, Srdjan Jelacic, Andrew Bowdle, Shyam Gollakota, Kelly Michaelsen
Communications Medicine, 2023.
code

ClearBuds: Wireless binaural earbuds for learning-based speech enhancement


Ishan Chatterjee, Maurchi Kim, Vivek Jayaram, Shyam Gollakota, Ira Kemelmacher, Shwetak Patel, Steve Seitz
ACM MobiSys, 2022.
webpage code video

Hybrid neural networks for on-device directional hearing


Anran Wang, Maruchi Kim, Hao Zhang, Shyam Gollakota
AAAI, 2022.
webpage code

Wind dispersal of battery-free wireless devices


Vikram Iyer, Hans Gaensbauer, Tom Daniel, Shyam Gollakota
Nature, 2022.
webpage code video

Underwater messaging using mobile devices


Tuochao Chen, Justin Chan, Shyam Gollakota
SIGCOMM, 2022 (ACM SIGMOBILE Highlight).
webpage code video

An off-the-shelf otoacoustic-emission probe for hearing screening via a smartphone


Justin Chan, Nada Ali, Ali Najafi, Anna Meehan, Lisa Mancl, Emily Gallagher, Randy Bly, Shyam Gollakota
Nature Biomedical Engineering, 2022.
webpage code

Performing tympanometry using smartphones


Justin Chan, Ali Najafi, Mallory Baker, Julie Kinsman, Lisa Mancl, Susan Norton, Randall Bly, Shyam Gollakota
Communications Medicine, 2022.
webpage code

Micro-mechanical blood clot testing using smartphones


Justin Chan, Kelly Michaelsen, Joanne Estergreen, Daniel Sabath, Shyam Gollakota
Nature Communications, 2022.
webpage code video


This site built using Amy Zhang's github code.