Skin Cancer Detection under Extreme Class Imbalance (ISIC 2024)
Overview
Course group project on skin cancer detection with ISIC 2024. The core challenge was extreme class imbalance (roughly 1000:1 benign to malignant), and the goal was to study model behavior and failure modes rather than optimize leaderboard rank.
Data and Approach
We modeled both metadata and images.
Metadata models: logistic regression (PyTorch), random forest, XGBoost.
Image models: simple CNN and ResNet-18 variants.
Evaluation focused on malignant-class precision/recall, not accuracy.
Key Challenges and Findings
Class imbalance pushed many models toward mostly benign predictions. We tested weighted loss and weighted sampling; sampling made training more stable but did not reliably improve results. Across metadata and image models, loss often plateaued early, suggesting limited usable signal. High recall was achievable, but usually with very low precision.
Interpretation
Standard supervised pipelines struggled in this setting. Better performance likely needs stronger inductive bias (for example pretrained representations, specialized losses, or multi-task setups) plus better-targeted data. The main value of this project was diagnosing failure modes and using the right metrics for an imbalanced clinical task.
Tools
Python, PyTorch, scikit-learn, XGBoost, CNNs, class-imbalance methods