Want to create an interactive transcript for this episode?
Podcast: Machine Learning Street Talk (MLST)
Episode: #044 - Data-efficient Image Transformers (Hugo Touvron)
Description: Today we are going to talk about the *Data-efficient image Transformers paper or (DeiT) which Hugo is the primary author of. One of the recipes of success for vision models since the DL revolution began has been the availability of large training sets. CNNs have been optimized for almost a decade now, including through extensive architecture search which is prone to overfitting. Motivated by the success of transformers-based models
in Natural Language Processing there has been increasing attention in applying these approaches to vision models. Hugo and his collaborators used a different training strategy and a new...