A way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs

A way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs

LSTMs have become a basic building block for many deep NLP models. In recent years, many improvements and variations have been proposed for deep sequence models in general, and LSTMs in particular.

Share article

LSTMs have become a basic building block for many deep NLP models. In recent years, many improvements and variations have been proposed for deep sequence models in general, and LSTMs in particular. We propose and analyze a series of augmentations and modifications to LSTM networks resulting in improved performance for text classification datasets. We observe compounding improvements on traditional LSTMs using Monte Carlo test-time model averaging, average pooling, and residual connections, along with four other suggested modifications. Our analysis provides a simple, reliable, and high quality baseline model.

Citation credit

Shayne Longpre, Sabeek Pradhan, Caiming Xiong, Richard Socher
A Way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs

Share article

Just For You

Explore related content by topic

• AI
• AI Research

Salesforce Research provides data-driven insights to help businesses transform how they drive customer success. Browse all reports at salesforce.com/research.