The success of transformers in processing sequential data for language and vision tasks (such as ChatGPT and DALL·E) has sparked strong interest in applying these models to forecasting problems. This has led to a proliferation of research proposals that attract significant attention, while also raising debate and scepticism about the true capabilities of these inherently black-box models. This project aims to address these concerns by adopting a mechanistic interpretability approach—an approach that has proven valuable in understanding language and vision models. In doing so, we seek to provide deeper insight into the internal workings of time-series transformers.
