Time Series Benchmark

Multivariate time series forecasting benchmark suite with GPU-accelerated evaluation, statistical profiling, and difficulty calibration. Supports LSTM, DLinear, and FITS architectures across five heterogeneous datasets with standardized preprocessing and fair comparison protocols.

Datasets

Due to file size limits, raw datasets are not included in this repository. Download them from:

• Google Drive

Extract data.zip to the project root so that the directory structure is ./data/*.csv.

Quick Start

# 1. Clone the repository
git clone https://github.com/wangdad007/Time-Series-Benchmark.git
cd Time-Series-Benchmark

# 2. Download and place datasets in ./data/ (see above)

# 3. Install dependencies
pip install -r requirements.txt

# 4. Run the full benchmark
python batch_experiment.py

Scripts

Script	Purpose
batch_experiment.py	Main benchmark runner — trains LSTM, DLinear, FITS on 5 datasets x 4 horizons, outputs benchmark_results_gpu.csv
ablation_experiment.py	Hardware I/O ablation — quantifies pinned memory and non-blocking transfer speedup
data_characteristics.py	Statistical profiling — ADF stationarity test, STL trend/seasonality decomposition, correlation heatmaps
extended_statistical_analysis.py	Advanced profiling — ACF decay, power spectral density, Hurst exponent (R/S analysis). Outputs figures to figures/
difficulty_calibration.py	4-dimension difficulty scoring (non-stationarity, horizon, noise, dimension) with Spearman validation. Outputs figures/difficulty_vs_mse.pdf
plot_results.py	Visualizes benchmark results — MSE vs prediction horizon curves for all model/dataset combinations

Results

After running batch_experiment.py, results are stored in benchmark_results_gpu.csv with columns:

• Model — LSTM_Baseline, DLinear, or FITS
• Dataset — Electricity, ETTh1, Exchange, Traffic, or Weather
• Pred_Len — 96, 192, 336, or 720
• MSE — Mean squared error on test set
• MAE — Mean absolute error on test set

Benchmark Protocol

• Preprocessing: Missing value interpolation + Z-Score standardization
• Train/test split: 7:3 temporal ordering (no shuffle, no look-ahead leakage)
• Look-back window: 96 time steps
• Prediction horizons: 96, 192, 336, 720
• Training: 20 epochs, batch size 32, Adam optimizer (lr=0.001), MSE loss
• Hardware: Auto-detects CUDA > MPS > CPU

License

MIT