390043 UK VGSCO Course (2021S)

In this course, we focus on some simple iterative optimization approaches
that, thanks to the advent of the "Big Data era", have re-gained popularity in the last few years.
We first review a bunch of classic methods in the context of modern real-world applications. Then, we discuss
both theoretical and computational aspects of some variants of those classic methods. Finally, we examine current challenges and future research perspectives. Our presentation, strongly influenced by Nesterov’s seminal book, includes the analysis of first-order methods, stochastic optimization methods, randomized and distributed methods, projection-free methods. The theoretical tools considered in the analysis, together with the broad applicability of those methods, makes the course quite interdisciplinary and might be useful for PhD students in different
areas (like, e.g., Analysis, Numerical Analysis, Operations Research, Probability and Mathematical Statistics).

1. Methods for Unconstrained Optimization:
1.1 Gradient and accelerated gradient methods
1.3 Block-Coordinate approaches
1.4 Stochastic Gradient and its variants
1.5 Real-world Problems

2. Methods for Constrained Optimization, Projection-based and Projection-free Approaches:
2.1 Projected Gradient
2.2 Frank-Wolfe Method and its Variants
2.3 Real-world Problems

3. Challenges and Future Research