Thesis Defence: From Architecture Selection to Efficient Adaptation

Dina El-Kholy, supervised by Dr. Mohamed Shehata, will defend their thesis titled “From Architecture Selection to Efficient Adaptation: Deep Learning for Medical Image Analysis” in partial fulfillment of the requirements for the degree of Master of Science in Computer Science.

An abstract for Dina El-Kholy’s thesis is included below.

Defences are open to all members of the campus community as well as the general public. Registration is not required for in-person defences.

Abstract

Deep learning has significantly advanced medical image analysis, yet selecting appropriate architectures, pretraining strategies, and adaptation methods remains challenging. This thesis addresses these design questions through three interconnected studies spanning classification and segmentation in chest X-ray and dermatology imaging.

The first study presents a controlled comparison of convolutional neural networks (CNNs), Vision Transformers (ViTs), and hybrid architectures across multiple medical image classification benchmarks. Models are matched by parameter count and trained under identical protocols, with additional evaluation of medical foundation models under linear probing. Results show that self-supervised CNNs achieve the highest average accuracy/AUC across six MedMNIST+ task, that architectural inductive biases often matter more than model scale, and that foundation models can underperform smaller fine-tuned models when domain alignment is limited.

The second study investigates parameter-efficient fine-tuning (PEFT) of the MedMAE foundation model using Low-Rank Adaptation (LoRA) and Infused Adapter by Inhibiting and Amplifying Inner Activations (IA3). Across classification and segmentation tasks, LoRA achieves performance comparable to full fine-tuning while updating only a small fraction of model parameters, and matches full fine-tuning for segmentation. IA3 provides greater compression at modest performance cost.

The third study synthesizes these findings into a hybrid architecture that prepends a trainable CNN stem to a LoRA-adapted transformer backbone. Systematic evaluation demonstrates that this design achieves the strongest segmentation results and competitive classification performance while substantially reducing the number of trainable parameters.

Together, these studies demonstrate that effective medical image analysis requires alignment between architecture, pretraining objective, and adaptation strategy. The thesis provides a systematic benchmark, empirical validation of PEFT for medical foundation models, and a principled hybrid design that balances performance and efficiency.