ECE 8201: Advanced Topics in Signal Processing (Offered at OSU)

This course will discuss advances in sampling and analysis of high-dimensional data under the umbrella of subspace methods, including low-dimensional data models such as sparse and low-rank models, convex and non-convex approaches for solving linear inverse problems, dimensionality reduction, clustering, and streaming data processing, with applications in signal processing, machine learning, imaging and network inference. The goal of the course is to develop expertise at the intersection of optimization, signal processing, statistics and computer science to address emerging challenges of data science.

Course Syllabus

Lecture Notes

Reading List (Compiled by Y. Chi, last update Sep. 1, 2015):

  • Reference textbook:
    • S. Foucart and H. Rauhut. A Mathematical Introduction to Compressive Sensing.
  • Sparse recovery with \ell-1 optimization:
    • D. Donoho and M. Elad. Optimally Sparse Representation in General (non-Orthogonal) Dictionaries via \ell-1 Minimization. PDF
    • E. J. Candes. The restricted isometry property and its implications for compressed sensing. PDF
    • E. J. Candes and Y. Plan. A Probabilistic and RIPless Theory of Compressed Sensing. PDF
    • D. Donoho. Compressed Sensing. PDF
    • R. Baraniuk, M. Davenport, R. DeVore and M. Wakin. A Simple Proof of the Restricted Isometry Property for Random Matrices. PDF
    • E. J. Candes and T. Tao. The Dantzig Selector: Statistical Estimation When p Is Much Larger than n. PDF
    • E. J. Candes and Y. Plan. Near-ideal model selection by \ell-1 minimization. PDF
  • Sparse recovery with greedy pursuit:
    • J. A. Tropp. Greed is Good: Algorithmic Results for Sparse Approximation. PDF
    • J. A. Tropp and A. C. Gilbert. Signal Recovery from Random Measurements via Orthogonal Matching Pursuit. PDF
    • D. Needell and J. A. Tropp. CoSaMP: Iterative signal recovery from incomplete and inaccurate samples. PDF
    • M. A. Davenport and M. B. Wakin. Analysis of Orthogonal Matching Pursuit Using the Restricted Isometry Property. PDF
  • Advanced topics on sparse recovery:
    • D. L. Donoho, A. Maleki, and A. Montanari. Message Passing Algorithms for Compressed Sensing. PDF
    • R. Calderbank, S. Howard and S. Jafarpour. Construction of a Large Class of Deterministic Sensing Matrices that Satisfy a Statistical Isometry Property. PDF
    • Y. Plan and R. Vershynin. One-bit compressed sensing by linear programming. PDF
  • Low-rank matrix completion:
    • B. Recht. A Simpler Approach to Matrix Completion. PDF
    • B. Recht, M. Fazel, and P. A. Parrilo. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. PDF
    • R. Mazumder, T. Hastie, and R. Tibshirani. Spectral regularization algorithms for learning large incomplete matrices. PDF
    • S. Negahban and M. J. Wainwright. Restricted strong convexity and weighted matrix completion: Optimal bounds with noise. PDF
  • High-dimensional structural inference:
    • E. D. Kolaczyk and R. D. Nowak. Multiscale likelihood analysis and complexity penalized estimation. PDF
    • S. N. Negahban, P. Ravikumar, M. J. Wainwright and B. Yu. A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers. PDF
    • V. Chandrasekaran, B. Recht, P. A. Parrilo, and A. S. Willsky. The Convex Geometry of Linear Inverse Problems. PDF
  • Sparse spectrum analysis and time-frequency analysis:
    • Y. Chi, L. L. Scharf, A. Pezeshki, and A. R. Calderbank. Sensitivity to basis mismatch in compressed sensing. PDF
    • E. J. Candes and C. Fernandez-Granda. Towards a mathematical theory of super-resolution. PDF
  • Streaming models and online learning:
    • S. Muthukrishnan. Data Streams: Algorithms and Applications. PDF
    • P. Indyk. Sketching, streaming, and sub-linear space algorithms. PDF