10th International Congress on Information and Communication Technology

Authors - Hayat Bihri, Soukaina Sraidi, Haggouni Jamal, Salma Azzouzi, My El Hassan Charaf
Abstract - Predictive analytics and artificial intelligence (AI) offer significant potential to improve healthcare, yet challenges in achieving interoperability across diverse settings, such as long-term care and public health, remain. Enhancing Electronic Health Records (EHRs) with multimodal data provides a more comprehensive view of patient health, leading to better decision-making and patient outcomes. This study proposes a novel framework for real-time cardiovascular disease (CVD) risk prediction and monitoring by integrating medical imaging, clinical variables, and patient narratives from social media. Unlike traditional models that rely solely on structured clinical data, this approach incorporates unstructured insights, improving prediction accuracy and enabling continuous monitoring. The methodology includes modality specific preprocessing: sentiment analysis and Named Entity Recognition (NER) for patient narratives, Convolutional Neural Networks (CNNs)for imaging, and Min-Max scaling with k-Nearest Neighbors (k-NN) imputation for clinical variables. A unique patient identifier ensures precise data fusion through multimodal transformers, with attention mechanisms prioritizing key features. Real-time monitoring leverages streaming natural language processing (NLP) to detect health trends from social media, triggering alerts for healthcare providers. The model undergoes rigorous validation using metrics like AUC-ROC, AUC-PR, Brier score, SHAP values, expert re-views, and clinical indicators, ensuring robustness and relevance.