Author(s)

Haoyu Wu ¹, Jingwen Zhang ²

Affiliation(s)

¹ International Business College, Dongbei University of Finance and Economics, Dalian, China
² School of Accounting, Dongbei University of Finance and Economics, Dalian, China

Corresponding Author

Haoyu Wu

ABSTRACT

In this paper, we investigate the application of Transformer-based deep learning models to structured tabular data, with a focus on credit default prediction. Traditional machine learning methods often struggle to capture complex feature interactions and require extensive feature engineering when dealing with heterogeneous categorical and numerical variables. To address this challenge, we adopt the TabTransformer architecture, which combines column embeddings and self-attention mechanisms to enable end-to-end representation learning on mixed-type economic data. Extensive experiments on a benchmark credit dataset demonstrate that TabTransformer outperforms baseline models—including logistic regression, random forests, and multilayer perceptrons—in terms of classification performance. In addition to predictive accuracy, we integrate SHAP-based interpretability, categorical embedding visualization, and attention heatmaps to provide transparent insights into the model's decision-making process. Our findings confirm the efficacy of deep Transformer models in structured data modeling and reinforce their potential for deployment in real-world financial risk assessment systems.

KEYWORDS

TabTransformer; Structured Data Modeling; Deep Learning; Explainable AI; Credit Risk Prediction

CITE THIS PAPER

Haoyu Wu, Jingwen Zhang, TabTransformer-Based Credit Default Prediction for Structured Economic Data. Information Systems and Economics (2025) Vol. 6: 71-80. DOI: http://dx.doi.org/10.23977/infse.2025.060210.

REFERENCES

[1] M. Xia, X. Mo, Y. Zhang, and X. Hu, "A knowledge graph construction and causal structure mining approach for non-stationary manufacturing systems," Rob. Comput. Integr. Manuf., vol.95, p.103013, Oct. 2025, doi: 10.1016/j.rcim.2025.103013.
[2] M. Chakraborty, N. Naoal, S. Momen, and N. Mohammed, "ANALYZE-AD: a comparative analysis of novel AI approaches for early alzheimer's detection," Array, vol. 22, p. 100352, Jul. 2024, doi: 10.1016/j.array.2024.100352.
[3] J. Kriebel and L. Stitz, "Credit default prediction from user-generated text in peer-to-peer lending using deep learning," Eur. J. Oper. Res., vol. 302, no. 1, pp. 309–323, Oct. 2022, doi: 10.1016/j.ejor.2021.12.024.
[4] I. Aruleba and Y. Sun, "Enhanced credit risk prediction using deep learning and SMOTE-ENN resampling," Mach. Learn. Appl., vol. 21, p. 100692, Sep. 2025, doi: 10.1016/j.mlwa.2025.100692.
[5] T. Campisi, E. Kuşkapan, M. Y. Çodur, and D. Dissanayake, "Exploring the influence of socio-economic aspects on the use of electric scooters using machine learning applications: a case study in the city of palermo," Res. Transp. Bus. Manag., vol. 56, p. 101172, Oct. 2024, doi: 10.1016/j.rtbm.2024.101172.
[6] Y. Fang, Y. Liu, Y. Yang, B. Lucey, and M. Z. Abedin, "How do Chinese urban investment bonds affect its economic resilience? Evidence from double machine learning," Research in International Business and Finance, vol. 74, p. 102728, Feb. 2025, doi: 10.1016/j.ribaf.2024.102728.
[7] Y. Lu, Z. Zhao, Y. Tian, and M. Zhan, "How does the economic structure break change the forecast effect of money and credit on output? Evidence based on machine learning algorithms," Pac.-basin Finance J., vol. 84, p. 102325, Apr. 2024, doi: 10.1016/j.pacfin.2024.102325.
[8] C. Li, W. He, and E. Cao, "Impact of green data center pilots on the digital economy development: an empirical study based on dual machine learning methods," Comput. Ind. Eng., vol. 201, p. 110914, Mar. 2025, doi: 10.1016/j.cie.2025.110914.
[9] M. I. Al-Karkhi and G. Rza̧dkowski, "Innovative machine learning approaches for complexity in economic forecasting and SME growth: a comprehensive review," J. Econ. Technol., vol. 3, pp. 109–122, Nov. 2025, doi: 10.1016/j.ject.2025.01.001.
[10] K. Sadeghi, S. H. Ghazaie, E. Sokolova, V. Sergeev, N. Ksenia, and L. Yang, "Machine learning-based correlation for economic evaluation of HTSE-nuclear cogeneration plant,” Int. J. Hydrogen Energy, vol. 114, pp. 337–351, Mar. 2025, doi: 10.1016/j.ijhydene.2025.02.423.
[11] A. Caplin, D. Martin, and P. Marx, "Modeling machine learning: a cognitive economic approach," J. Econom. Theory, vol. 224, p. 105970, Mar. 2025, doi: 10.1016/j.jet.2025.105970.
[12] H. Li and A. Kayae, “Predicting energy consumption in Mexico: integrating environmental, economic, and energy data with machine learning techniques for sustainable development,” Energy, vol. 324, p. 135992, Jun. 2025, doi: 10.1016/j.energy.2025.135992.
[13] D. Yang, B. Gao, S. Wang, and H. Xiang, "Robustness test for fouling state identification of homogeneous pressure electrodes based on confidence ellipsoids," IEICE Electron. Express, 2025, doi: 10.1587/elex.22.20240283.
[14] C.-H. Lin, T. Liu, and K. Vincent, "Should economic theories guide the machine learning model in forecasting exchange rate?," Econ. Model., vol. 151, p. 107224, Oct. 2025, doi: 10.1016/j.econmod.2025.107224.
[15] A. Cai, "Uncovering the multiple socio-economic driving factors of carbon emissions in nine urban agglomerations of China based on machine learning," Energy, vol. 319, p. 134859, Mar. 2025, doi: 10.1016/j.energy.2025.134859.

Sponsors, Associates, and Links

---

• Accounting, Auditing and Finance
  
  
• Industrial Engineering and Innovation Management
  
  
• Tourism Management and Technology Economy
  
  
• Journal of Computational and Financial Econometrics
  
  
• Financial Engineering and Risk Management
  
  
• Accounting and Corporate Management
  
  
• Social Security and Administration Management
  
  
• Population, Resources & Environmental Economics
  
  
• Statistics & Quantitative Economics
  
  
• Agricultural & Forestry Economics and Management
  
  
• Social Medicine and Health Management
  
  
• Land Resource Management
  
  
• Information, Library and Archival Science
  
  
• Journal of Human Resource Development
  
  
• Manufacturing and Service Operations Management
  
  
• Operational Research and Cybernetics