Topics in Applied Optimization
Course Outcomes
- Learn additional theory needed from calculus and linear algebra for optimization.
- Learn to model various applications from data science as an optimization problem.
- Learn to prove convergence estimates and complexity of the algorithms.
- Learn to code optimization solvers efficiently using Python.
- Demonstrate expertise in applying optimization methods in research problems.
Syllabus
- Unit 1: Convex Sets, Convex Functions, Duality, Convex Optimization Problems (9 hours)
- Unit 2: Steepest Descent, Newton methods, Quasi-Newton Methods, Interior Point Methods, Stochastic Optimization algorithms (SGD, RMSprop, ADAM, SVRG, etc), Basic Game Theory: two player games, Min-Max Problems in Generative models (GANs), Convergence Estimates (6 hours)
- Unit 3: Applications of optimization: Recommender Systems, Support Vector Machines, Neural networks (Optimization for RNN, CNN, Transformers), Extreme Classification, Image and Video Processing, 3D Reconstruction, Differential privacy. (9 hours)
References
- Stephen Boyd and Lieven Vandenberghe, Convex Optimization, Cambridge University Press, 2004.
- Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016.
Weightages
- Assignments in theory: 15 marks, Mid Semester Examination: 25 marks, End Semester Examination: 30 marks, Assessment of four projects: 30 marks
Course Material
- For course material and notes please login to Moodle: www.courses.iiit.ac.in