A road through Leslie N. Smith’s works, to understand the intuition and application of the One-Cycle policy, the Hyper-parameters settings in a Deep Neural Network and catch Super Convergence.
This post provides a guide and an intuition into choosing the appropriate values for training a Deep Neural Network. In particular, it shows a detailed overview of the phenomenon called Super-Convergence where a Deep Neural Network can be trained in order of magnitude faster compared to conventional training methods. The key elements follow the One-Cycle policy and Leslie N. Smith’s teachings and Hyper-Parameter settings.
Data Scientist | Read & experiment with enthusiasm