Description for each assignment

A1

Image denoising with L2 norm regularization using closed form solution

A2

Image denoising with L2 norm regularization using gradient descent with Lipschitz constant as the step size

gradient descent with line-search

gradient descent with Armijo line-search

A3

Image denoising with L1 norm regularization using gradient descent with Armijo line-search. Use the pseudo-Huber function to smooth the problem

Gradient descent with simple line-search

Accelerated gradient descent with Lipschitz constant

Accelerated gradient descent with Armijo line-search

A4

use Hing-loss and L2-regularized logistic regression as objective function to model classifier. And apply the classifier to three different datasets. The last two datasets are big. cannot upload to github

apply following methods to train:

1) Stochastic sub-gradient

2) Stochastic gradient

3) Mini-batch (sub-)gradient

4) Stochastic average sub-gradient (SAG)

5) Stochastic average gradient (SAG)

6) Gradient descent with Armijo line-search

7) Acceleratd gradient with Armijo line-search

A5

solve l1-regularized logistic regression problem using

Proximal gradient descent

Accelerated proximal gradient descent Proximal coordinate descent Accelerated proximal coordinate descent

label propagation using coordinate descent

local graph clustering using page ranking

Reccomender system using proximal gradient descent

Nonnegative Matrix Factorization: facial feature extraction

A6

Use ADMM to separate a background image form foreground interference

Implement AM-RR (alternating minimization for robust regresion) on the same dataset as Q1. Form a matrix X whose columns are the first 70 bird images. Form a vector y that is the 71st image. Then try to fit y = Xw using the Robust Linear Regression problem.