This repository contains a research project focused on Segmentation of Intracranial Arteries using SegFormer (SIAS). The project implements a state-of-the-art transformer-based approach for medical image segmentation, specifically targeting intracranial arterial structures from medical imaging data.
Intracranial artery segmentation is a critical task in medical image analysis, essential for diagnosis, treatment planning, and monitoring of cerebrovascular diseases. This work leverages the SegFormer architecture, a hierarchical vision transformer designed for semantic segmentation tasks, and adapts it for the specialized domain of intracranial vascular structures.
The implementation utilizes the SegFormer-B0 (mit-b0) architecture from Hugging Face Transformers:
- Encoder: Hierarchical transformer encoder for multi-scale feature extraction
- Decoder: Lightweight MLP decoder for efficient segmentation
- Binary Classification: Background (0) vs. Foreground (1) arterial structures
- Pre-trained SegFormer model fine-tuned on intracranial artery data
- Binary segmentation approach for artery identification
- Efficient transformer-based architecture optimized for medical imaging
- Custom data pipeline for medical image preprocessing
- Source: Private Dataset from Hugging Face
- Structure:
- Training set: 151 examples
- Validation set: 55 examples
- Image format: RGB images (converted from original format)
- Mask format: Binary masks (artery=1, background=0)
-
Image Processing:
- Automatic RGB conversion for grayscale images
- SegFormer-specific image preprocessing
- Normalization and resizing to model requirements
-
Mask Processing:
- Binary mask creation with artery structures as foreground
- Consistent mask-image pair alignment
- Data augmentation through SegFormer processor
model = SegformerForSemanticSegmentation.from_pretrained(
"nvidia/mit-b0",
num_labels=2,
id2label={0: 'background', 1: 'foreground'},
label2id={'background': 0, 'foreground': 1}
)- Learning Rate: 6e-5 (optimized for fine-tuning)
- Optimizer: AdamW
- Batch Size: 8 (training and validation)
- Epochs: 10
- Metric: Mean Intersection over Union (mIoU)
The model utilizes the built-in SegFormer loss function, which combines:
- Cross-entropy loss for pixel-wise classification
- Auxiliary losses from intermediate decoder outputs
- Data Loading: Custom PyTorch Dataset class for medical image-mask pairs
- Model Initialization: Pre-trained SegFormer with custom classification head
- Training Loop:
- Forward pass with pixel-wise classification
- Loss computation and backpropagation
- Metric evaluation (mIoU, accuracy)
- Progress tracking with tqdm
- Mean Intersection over Union (mIoU): Primary segmentation metric
- Pixel-wise Accuracy: Secondary evaluation metric
- Visual Inspection: Qualitative assessment through prediction visualization
The training encountered CUDA memory limitations, requiring:
- Batch size optimization
- Gradient accumulation strategies
- Memory-efficient data loading
Adapting SegFormer from natural images to medical imaging required:
- Careful preprocessing pipeline design
- Binary classification head configuration
- Domain-specific evaluation metrics
The training process demonstrated:
- Successful model initialization with pre-trained weights
- Proper loss computation and gradient flow
- Effective fine-tuning on medical imaging data
The implementation includes:
- Custom color palette for segmentation visualization
- Overlay techniques for result interpretation
- Qualitative assessment tools
DIAS/
├── DIAS_SegFormer_Custom_FineTuning.ipynb # Main implementation notebook
├── train/ # Training data directory
│ ├── images/ # Input medical images
│ └── masks/ # Binary segmentation masks
├── val/ # Validation data directory
│ ├── images/ # Validation images
│ └── masks/ # Validation masks
├── README.md # This documentation
└── LICENSE # License file
pip install transformers datasets evaluate torch torchvision
pip install numpy matplotlib PIL tqdm- Data Preparation: Ensure DIAS dataset is properly downloaded and structured
- Model Training: Execute the Jupyter notebook cells sequentially
- Inference: Use trained model for new image segmentation
# Load trained model
model = SegformerForSemanticSegmentation.from_pretrained("./trained_model")
# Process new image
inputs = image_processor(image, return_tensors="pt")
outputs = model(**inputs)
# Generate prediction
prediction = outputs.logits.argmax(dim=1).squeeze().cpu().numpy()- Domain Application: Successful adaptation of SegFormer to intracranial artery segmentation
- Medical AI: Contributing to automated medical image analysis
- Transformer Segmentation: Demonstrating transformer effectiveness in specialized medical domains
- Open Research: Providing reproducible implementation for research community
- Framework: PyTorch, Hugging Face Transformers
- Model: SegFormer-B0 (mit-b0)
- Input Resolution: Variable (processor-dependent)
- Output: Binary segmentation masks
- Hardware: CUDA-compatible GPU recommended
- Memory constraints during training
- Limited dataset size
- Binary segmentation scope
- Multi-class arterial segmentation
- Larger dataset integration
- Performance optimization for clinical deployment
- Integration with medical imaging standards (DICOM)
transformers>=4.21.0
torch>=1.12.0
torchvision>=0.13.0
datasets>=2.4.0
evaluate>=0.2.0
numpy>=1.21.0
matplotlib>=3.5.0
Pillow>=8.3.0
tqdm>=4.64.0
- Hugging Face Transformers team for the SegFormer implementation
- DIAS dataset creators for providing the medical imaging data
- NVIDIA for the pre-trained SegFormer models
- Research community for foundational work in medical image segmentation
For questions, issues, or collaborations regarding this research project, please contact imkaleemahmad@gmail.com.
Note: This is a research project intended for academic and research purposes. Medical applications require proper validation and regulatory approval.