How to achieve Super-Convergence and exploit One-Cycle policy: a simple guide

A road through Leslie N. Smith’s works, to understand the intuition and application of the One-Cycle policy, the Hyper-parameters settings in a Deep Neural Network and catch Super Convergence.

This post provides a guide and an intuition into choosing the appropriate values for training a Deep Neural Network. In particular, it shows a detailed overview of the phenomenon called Super-Convergence where a Deep Neural Network can be trained in order of magnitude faster compared to conventional training methods. The key elements follow the One-Cycle policy and Leslie N. Smith’s teachings and Hyper-Parameter settings.

In this session, the papers that most…