Robust and Scalable Optimization Methods for Deep Learning

In this topic, we develop robust solvers for various types of neural network architectures such as ANN, RNN, LSTM, Transformers, GAN, etc. Some of our recent ongoing research is on second order gradient methods for GANs, and CNN. As we can see from the figure below, it is not easy to find minima of such a difficult loss landscape. This project started in 2020, more on this very soon!