How Lightning Helps You With Long Sequences
Lightning is a lightweight PyTorch wrapper for high-performance AI research that reduces the boilerplate without limiting flexibility. In this series, we are covering all the tricks Lightning offers to supercharge your machine learning training.
Too many recursive steps can lead to unstable gradients. Truncating the sum of gradients of only the last N steps can eliminate this issue. In this video we explain how you can automatically truncate a time sequence with Lightning’s implementation of Truncated Back-propogation Through Time using the truncated_bptt_steps flag.
Follow along with this notebook: https://bit.ly/33YzC1P