Ruoqi Shen

Ph.D. Student Paul G. Allen School of Computer Science & Engineering University of Washington Email: shenr3 [at] cs [dot] washington [dot] edu Google Scholar

About

I am a PhD student in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, advised by Yin Tat Lee. I received my B.S. in Computer Science from California Institute of Technology in 2018.

I am a consulting researcher and interned during Summer 2021 and 2022 with Sébastien Bubeck and Suriya Gunasekar in the Machine Learning Foundations group at MSR Redmond.

At UW, I work on sampling algorithms for high-dimensional and ill-conditioned distributions. At MSR, I work on language models, where I currently focus on using transformers to do arithmetic operations.

Selected Publications

Sampling

Non-Euclidean Sampling

• Algorithmic Aspects of the Log-Laplace Transform and a Non-Euclidean Proximal Sampler.
  (α-β) Sivakanth Gopi, Yin Tat Lee, Daogao Liu, Ruoqi Shen, Kevin Tian. COLT 2023.
  

• Condition-number-independent Convergence Rate of Riemannian Hamiltonian Monte Carlo with Numerical Integrators.
  (α-β) Yunbum Kook, Yin Tat Lee, Ruoqi Shen, Santosh S. Vempala. COLT 2023.
  

Proximal Sampling

• Structured Logconcave Sampling with a Restricted Gaussian Oracle.
  (α-β) Yin Tat Lee, Ruoqi Shen, Kevin Tian. COLT 2021.
  A proximal sampler. A non-Eculidean extension is shown in [GLLST23].

Complexity of Sampling

• Analysis of Langevin Monte Carlo from Poincaré to Log-Sobolev.
  (α-β) Sinho Chewi, Murat A. Erdogdu, Mufan Bill Li, Ruoqi Shen, Matthew Zhang. COLT 2022.
  

• Lower Bounds on Metropolized Sampling Methods for Well-Conditioned Distributions.
  (α-β) Yin Tat Lee, Ruoqi Shen, Kevin Tian. NeurIPS 2021 (oral presentation).
  A lower bound matching the upper bound in [LST20].

• Logsmooth Gradient Concentration and Tighter Runtimes for Metropolized Hamiltonian Monte Carlo.
  (α-β) Yin Tat Lee, Ruoqi Shen, Kevin Tian. COLT 2020.
  A tight analysis of Metropolis-adjusted Langevin algorithm (MALA) and one-step MHMC from a feasible start.

• The Randomized Midpoint Method for Log-Concave Sampling.
  Ruoqi Shen, Yin Tat Lee. NeurIPS 2019 (spotlight presentation).
  A sampling algorithm with the state-of-the-art guarantee on dimension dependence for strongly log-concave distributions. [CLW21] shows the randomized midpoint method is an optimal discretization method.

Applications

• Private Convex Optimization in General Norms.
  (α-β) Sivakanth Gopi, Yin Tat Lee, Daogao Liu, Ruoqi Shen, Kevin Tian. SODA 2023.
  

• Sampling with Riemannian Hamiltonian Monte Carlo in a Constrained Space.
  (α-β) Yunbum Kook, Yin Tat Lee, Ruoqi Shen, Santosh S. Vempala. NeurIPS 2022.
  A package to sample from ill-conditioned, non-smooth, constrained distributions in high dimensions. A 1,000-fold speed-up for sampling from the benchmark human metabolic network (Recon3D).

Deep Learning

• How to Fine-Tune Vision Models with SGD.
  Ananya Kumar, Ruoqi Shen, Sébastien Bubeck, Suriya Gunasekar. Preprint.
  

• Data Augmentation as Feature Manipulation.
  Ruoqi Shen, Sébastien Bubeck, Suriya Gunasekar. ICML 2022.
  

(α-β stands for alphabetical author order. Full publication list.)