Publications Google Scholar

In an overparameterized Bayesian linear model with a general prior \(\pi\), we characterize when good generalization requires near-interpolation and when fitting the training data too closely is harmful. The transition is governed by noise thresholds determined by the prior's Fisher information and variance, giving explicit conditions under which memorization is either necessary or detrimental.

We develop Hoeffding- and Bernstein-type concentration inequalities for structured weighted sums under exchangeability, including tensor inner products and sequential matrix sums. The bounds recover optimal constants in classical scalar and random-matrix settings, sharpen results for combinatorial matrix sums, and support applications to multi-factor response models and fixed-design sketching in federated averaging.

We show that learning from aggregated noisy labels can provide robustness unavailable to procedures trained on raw labels. In particular, label aggregation yields stronger risk-consistency guarantees under surrogate losses and remains reliable under mild model misspecification, clarifying how data collection, cleaning, model fitting, and prediction interact to produce robust classifiers.

We establish state-evolution predictions for high-dimensional iterative algorithms that may interleave first-order and saddle-point updates and need not be coordinate-wise separable. The analysis also gives finite-sample deviation guarantees, using Hilbert-space lifting, Bolthausen conditioning, and a sequential form of Gordon's Gaussian comparison inequality.

We study how constraint and gradient geometry determine the computational and statistical limits of stochastic optimization. Diagonally preconditioned stochastic gradient methods are minimax optimal for quadratically convex constraints, while departures from quadratic convexity quantify when nonlinear updates such as mirror descent are needed; the results include \(\ell_p\)-balls with \(p<2\).

We identify minimax limits for prediction sets and calibration error in predictive inference. For witness classes of VC dimension \(d\), weighted conformal coverage cannot converge faster than order \(d^{1/2}n^{-1/2}\), while weighted weak calibration error has a lower-bound rate of order \(d^{1/4}n^{-1/2}\); existing procedures attain the corresponding rates in key settings.

We study collaborative least-squares learning for \(m\) agents that observe different, structured subsets of the available features. The distributed semi-supervised algorithm Collab combines local training, aggregation, and redistribution without communicating labeled data, yet is nearly asymptotically local-minimax optimal even relative to methods that can share those labels.

We derive nonasymptotic, dimension-free approximations for the bias and variance of ridge regression, allowing feature vectors to be high- or infinite-dimensional. The risk is compared multiplicatively with an equivalent sequence model, extending random-matrix predictions beyond proportional asymptotics and yielding sharp results for Hilbert-space covariates and benign overfitting near interpolation.

We examine the common practice of aggregating multiple annotations into a single gold-standard label. A theoretical model shows when retaining the full, non-aggregated label information improves calibration and learning rates, and when aggregation instead offers slower but more robust behavior; experiments on real and synthetic data support these predictions.

In overparameterized linear regression \(y=X\theta+w\) with \(d/n\to\gamma>1\), we characterize the relationship between training error and prediction error. As the noise variance \(\sigma^2\) vanishes, estimators with training error of order at least \(\sigma^4\) are necessarily suboptimal, showing that optimal prediction may require fitting far more accurately than the nominal noise floor.

We characterize bounded-time, high-dimensional limits of deterministic flows driven by random data matrices. Their behavior is described by a finite-dimensional nonlinear stochastic process determined through a fixed-point condition, providing a rigorous dynamical mean field theory via approximate message passing and establishing universality for several gradient-flow models.

We give nonasymptotic convergence guarantees for entropy-regularized natural policy gradient methods with softmax policies in discounted Markov decision processes. With exact policy evaluation, the method converges globally at a linear rate and locally at a quadratic rate; the analysis also quantifies stability under inexact evaluation and accommodates a broad range of learning rates.

We develop eigenvector estimation and inference methods for low-rank symmetric signals observed with independent, asymmetric, heteroscedastic noise. Eigen-decomposition of the asymmetric data yields distribution-free, noise-adaptive, and Gaussian-minimax-optimal procedures for eigenvectors, linear functionals, and eigenvalue confidence intervals, even when the relevant eigen-gaps are much smaller than classical perturbation theory permits.

We show that asymmetry in noisy observations can improve spectral estimation of a symmetric low-rank matrix. Eigen-decomposition of an asymmetric data matrix can estimate the leading eigenvalue up to roughly \(\sqrt{n}\) more accurately than an unadjusted singular value, while also controlling linear functionals of the leading eigenvector and adapting automatically to heteroscedastic noise.