Journal of Computer Technology and Software

Vol. 4 No. 10 (2025)
Articles

Multimodal Representation Learning for Robust Clinical Risk Prediction from Heterogeneous Electronic Health Records

  • Arjun Venkatesh

Published 2025-10-30

How to Cite

Venkatesh, A. (2025). Multimodal Representation Learning for Robust Clinical Risk Prediction from Heterogeneous Electronic Health Records. Journal of Computer Technology and Software, 4(10). Retrieved from https://www.ashpress.org/index.php/jcts/article/view/385
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Accurate clinical risk prediction from electronic health records (EHRs) remains challenging due to heterogeneous data modalities, irregular observations, missing values, and distribution shifts across patient populations. This study proposes a multimodal representation learning framework for robust clinical risk prediction by jointly modeling structured clinical variables, longitudinal laboratory measurements, and unstructured clinical notes. A modality-specific encoding architecture first extracts complementary representations from heterogeneous data sources, followed by a cross-modal attention module that dynamically captures interactions among clinical modalities. To improve robustness under incomplete observations, a modality-aware masking strategy is introduced during training, enabling the model to maintain predictive performance when individual data sources are unavailable. In addition, a distribution alignment objective is incorporated to reduce representation shifts across demographic and temporal subgroups. Experiments on real-world EHR data demonstrate that the proposed framework improves prediction performance and calibration compared with unimodal and conventional multimodal baselines. The results suggest that robust multimodal representation learning can provide a practical foundation for AI-assisted clinical risk assessment.

References

  1. X. Yan, W. Wang, M. Xiao, Y. Li, and M. Gao, “Survival prediction across diverse cancer types using neural networks,” Proceedings of the 2024 7th International Conference on Machine Vision and Applications, pp. 134–138, 2024.
  2. J. N. Acosta, G. J. Falcone, P. Rajpurkar, and E. J. Topol, “Multimodal biomedical AI,” Nature Medicine, vol. 28, pp. 1773–1784, 2022.
  3. J. Wei, Y. Liu, X. Huang, X. Zhang, W. Liu, and X. Yan, “Self-Supervised Graph Neural Networks for Enhanced Feature Extraction in Heterogeneous Information Networks,” 2024 5th International Conference on Machine Learning and Computer Application (ICMLCA), pp. 272–276, 2024.
  4. E. Choi et al., “RETAIN: An interpretable predictive model for healthcare using reverse time attention mechanism,” Advances in Neural Information Processing Systems, 2016.
  5. H. Zheng, Y. Ma, Y. Wang, G. Liu, Z. Qi, and X. Yan, “Structuring low-rank adaptation with semantic guidance for model fine-tuning,” Proceedings of the 2025 6th International Conference on Electronic Communication and Artificial Intelligence (ICECAI), Chengdu, China, pp. 731–735, 2025.
  6. Y. Li, W. Zhao, B. Dang, X. Yan, M. Gao, W. Wang, and M. Xiao, “Research on adverse drug reaction prediction model combining knowledge graph embedding and deep learning,” Proceedings of the 2024 4th International Conference on Machine Learning and Intelligent Systems Engineering (MLISE), pp. 322–329, 2024.
  7. A. Rajkomar et al., “Scalable and accurate deep learning with electronic health records,” npj Digital Medicine, vol. 1, Art. no. 18, 2018.
  8. M. Xiao, Y. Li, X. Yan, M. Gao, and W. Wang, “Convolutional neural network classification of cancer cytopathology images: Taking breast cancer as an example,” Proceedings of the 2024 7th International Conference on Machine Vision and Applications, pp. 145–149, 2024.
  9. X. Yan, Y. Jiang, W. Liu, D. Yi, and J. Wei, “Transforming Multidimensional Time Series into Interpretable Event Sequences for Advanced Data Mining,” 2024 5th International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI), pp. 126–130, 2024.
  10. K. Singhal et al., “Large language models encode clinical knowledge,” Nature, vol. 620, pp. 172–180, 2023.
  11. X. Yan, J. Du, X. Li, X. Wang, X. Sun, P. Li, and H. Zheng, “A Hierarchical Feature Fusion and Dynamic Collaboration Framework for Robust Small Target Detection,” IEEE Access, vol. 13, pp. 92953–92964, 2025, doi: 10.1109/ACCESS.2025.3570669.
  12. R. Miotto, L. Li, B. A. Kidd, and J. T. Dudley, “Deep Patient: An unsupervised representation to predict the future of patients from the electronic health records,” Scientific Reports, vol. 6, Art. no. 26094, 2016.
  13. Y. Li, X. Yan, M. Xiao, W. Wang, and F. Zhang, “Investigation of Creating Accessibility Linked Data Based on Publicly Available Accessibility Datasets,” Proceedings of the 2023 13th International Conference on Communication and Network Security, pp. 77–81, 2024.
  14. X. Yan, J. Du, L. Wang, Y. Liang, J. Hu, and B. Wang, “The Synergistic Role of Deep Learning and Neural Architecture Search in Advancing Artificial Intelligence,” Proceedings of the 2024 International Conference on Electronics and Devices, Computational Science (ICEDCS), pp. 452–456, 2024.
  15. B. Shickel, P. J. Tighe, A. Bihorac, and P. Rashidi, “Deep EHR: A survey of recent advances in deep learning techniques for electronic health record analysis,” IEEE Journal of Biomedical and Health Informatics, vol. 22, no. 5, pp. 1589–1604, 2018.
  16. H. Zheng, L. Zhu, W. Cui, R. Pan, X. Yan, and Y. Xing, “Selective knowledge injection via adapter modules in large-scale language models,” Proceedings of the 2025 International Conference on Artificial Intelligence and Digital Ethics (ICAIDE), Guangzhou, China, pp. 373–377, 2025.
  17. H. Harutyunyan, H. Khachatrian, D. C. Kale, G. Ver Steeg, and A. Galstyan, “Multitask learning and benchmarking with clinical time series data,” Scientific Data, vol. 6, Art. no. 96, 2019.
  18. W. Wang, Y. Li, X. Yan, M. Xiao, and M. Gao, “Breast cancer image classification method based on deep transfer learning,” Proceedings of the International Conference on Image Processing, Machine Learning and Pattern Recognition, pp. 190–197, 2024.