A road through Leslie N. Smith’s works, to understand the intuition and application of the One-Cycle policy, the Hyper-parameters settings in a Deep Neural Network and catch Super Convergence.

This post provides a guide and an intuition into choosing the appropriate values for training a Deep Neural Network. In particular, it shows a detailed overview of the phenomenon called Super-Convergence where a Deep Neural Network can be trained in order of magnitude faster compared to conventional training methods. The key elements follow the One-Cycle policy and Leslie N. Smith’s teachings and Hyper-Parameter settings.

In this session, the papers that most…

Michela Sessi

Data Scientist | Read & experiment with enthusiasm

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store