A Multimodal Deep Learning Framework for Pancreatic Disease Severity Estimation using 3D CT and Clinical Data

Pancreatic diseases require advanced diagnostic evaluation, which extends beyond classical binary-type diagnostics. This work introduces a novel multimodal deep learning framework for estimating continuous pancreatic disease severity by linking 3D computed tomography (CT) imaging with structured clinical metadata. In summary, we propose an architecture that harmoniously integrates a 3D convolutional neural network for volumetric CT feature extraction with a multilayer perceptron for clinical feature encoding, which is followed by late fusion and regression-based severity prediction. Evaluation on the publicly available CPTAC-PDA cohort highlight that the proposed multimodal framework shows excellent, mean absolute error 6.34% and Pearson correlation coefficient of 0.467. The proposed model successfully learns trends of severity progression that are missed by unimodal imaging-only and clinical-only models. This model shows excellent prediction accuracy for mild, moderate and severe cases; extensive qualitative analysis across the spectrum of severity confirms its clinical relevance. This study provides a solid basis for AI-assisted long-term pancreatic disease monitoring, raising the prospect for precision medicine applications.