Assembling the complete forecasting pipeline: log returns, stationarity test, ARIMA for mean, GARCH for variance, Transformer for long-horizon. All three compared head-to-head with the Python code to run them.
Architecture overview
The full system is a modular pipeline. Each stage can be swapped independently. ARIMA and GARCH are classical statistical models fit in seconds; the Transformer requires GPU training but generalises to multi-step horizons.
Complete forecasting pipeline
Model comparison
Typical results on SPY daily log-returns (5 years of data, 252-bar walk-forward evaluation). Each model has its strength: ARIMA for 1-step mean, GARCH for volatility, Transformer for multi-step mean at long horizons.
ARIMA 1-step MAE
0.0078
Dir. accuracy 52.1%
GARCH QLIKE loss
2.14
Volatility forecast R² 0.31
Transformer 5-step MAE
0.0091
Dir. accuracy 54.8%
Walk-forward accuracy comparison across 252-bar test window
Directional accuracy matters more than MAE in trading. A model that always predicts 0 achieves excellent MAE but 50% directional accuracy — useless. Even 54% directional accuracy (above chance) can produce alpha if combined with position sizing and risk management.
Python code
Four files cover the full system. Click a tab to view the code.
Model selection in practice. Start with ARIMA(1,0,1) for mean, GARCH(1,1) for variance. Test Ljung-Box on residuals and ARCH-LM on squared residuals. Only add complexity if diagnostics fail. For most daily-return forecasting tasks, simple models win.
Transformer hyperparameters to tune. Lookback L (try 96, 192, 336), patch_len (L/8 to L/4), d_model (64, 128, 256), dropout (0.1–0.3). Use early stopping on validation MAE. Batch size 32–128. AdamW with lr 1e-3 and cosine decay.
Extensions. Add macroeconomic features (VIX, yield curve slope, sector ETFs) to the transformer input channels. Ensemble ARIMA + GARCH + Transformer forecasts (equal weights or learned stacking). Replace the linear head with NormalNLLLoss for probabilistic forecasts with calibrated uncertainty intervals.