In this proposal, one of the two focuses is to leverage recent advances in large language models (e.g., BERT, T5, GPT-2) in Natural Language Processing (NLP) to extract valuable information from unstructured documents, which include clinical documents and user-generated content. The second is on machine learning using medical imaging data. Our aim is to develop robust privacy-preserving diagnostic and prognostic models that are explainable. Finally, the third aim is to build visio-lingual models using text and imaging data to facilitate a variety of downstream multi-modal tasks, such as question answering.