Go to AAAI 2025 Workshop AICT homepageIntegrating Transfer Entropy into Transformer for Time Series Forecasting
Keywords: Time series forecasting, Transformer, Transfer Entropy, Causal Analysis
Abstract: Time series forecasting is an important task in a variety of domains. Recently, transformers have shown promise by effectively modeling long-range dependencies through self-attention mechanisms. However, they inherently assume symmetric relationships and lack the ability to explicitly model the directionality of information flow—a critical aspect in time series data where causality plays a significant role. This paper addresses this research gap by proposing a novel transformer architecture that integrates transfer entropy into the attention mechanism to explicitly model directional dependencies and causal relationships in time series forecasting. Empirical validation on the M4 benchmark dataset, a comprehensive collection of time series data for forecasting, shows that our model outperforms state-of-the-art transformer-based models, achieving superior forecasting accuracy. Our method represents a remarkable advancement in time series forecasting by uniquely combining the benefits of transformers with the ability to model causal relationships without requiring additional causal graphs or prior knowledge about the data.
Submission Number: 14
Loading