Model Performance
Validation results on 2024 data — never seen during training. Evaluated using F1 Macro and Balanced Accuracy.
Validation Scorecard (2024)
| Model | F1 Macro | Bal Accuracy | F1 Weighted |
|---|---|---|---|
XGBoost✅ Best | 0.3739 | 0.3769 | 0.4149 |
LightGBM | 0.3674 | 0.3713 | 0.4099 |
LSTM | 0.3573 | 0.3832 | 0.3987 |
Ensemble | 0.3545 | 0.3807 | 0.4066 |
Key Findings
• XGBoost is the best performing model with F1 Macro of 0.3739 on 2024 validation data.
• LSTM achieves the highest Balanced Accuracy (0.3832), showing it handles class imbalance better.
• Ensemble underperforms the best base model — the meta-model over-relies on LSTM Hold signal (coefficient 1.80).
• All models struggle with Sell and Buy classes — Hold is predicted most reliably across all models.
• Random baseline F1 Macro = 0.33. All models beat the baseline.
Training Configuration
Training Period
2019 – 2023
Validation Period
2024
Total Stocks
20 US Stocks
Total Rows
29,100
Features
17 Technical Indicators
Target
5-day Forward Return
Classes
Buy / Hold / Sell
CV Strategy
5-fold TimeSeriesSplit