8000 GitHub - imkaleemahmad/SIAS · GitHub
[go: up one dir, main page]

Skip to content
imkaleemahmadPublic

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

5 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 

Repository files navigation

SIAS: SegFormer-based Intracranial Artery Segmentation

Overview

This repository contains a research project focused on Segmentation of Intracranial Arteries using SegFormer (SIAS). The project implements a state-of-the-art transformer-based approach for medical image segmentation, specifically targeting intracranial arterial structures from medical imaging data.

Abstract

Intracranial artery segmentation is a critical task in medical image analysis, essential for diagnosis, treatment planning, and monitoring of cerebrovascular diseases. This work leverages the SegFormer architecture, a hierarchical vision transformer designed for semantic segmentation tasks, and adapts it for the specialized domain of intracranial vascular structures.

Architecture

SegFormer Model

The implementation utilizes the SegFormer-B0 (mit-b0) architecture from Hugging Face Transformers:

  • Encoder: Hierarchical transformer encoder for multi-scale feature extraction
  • Decoder: Lightweight MLP decoder for efficient segmentation
  • Binary Classification: Background (0) vs. Foreground (1) arterial structures

Key Features

  • Pre-trained SegFormer model fine-tuned on intracranial artery data
  • Binary segmentation approach for artery identification
  • Efficient transformer-based architecture optimized for medical imaging
  • Custom data pipeline for medical image preprocessing

Dataset

DIAS Dataset

  • Source: Private Dataset from Hugging Face
  • Structure:
    • Training set: 151 examples
    • Validation set: 55 examples
    • Image format: RGB images (converted from original format)
    • Mask format: Binary masks (artery=1, background=0)

Data Preprocessing

  1. Image Processing:

    • Automatic RGB conversion for grayscale images
    • SegFormer-specific image preprocessing
    • Normalization and resizing to model requirements
  2. Mask Processing:

    • Binary mask creation with artery structures as foreground
    • Consistent mask-image pair alignment
    • Data augmentation through SegFormer processor

Methodology

Model Configuration

model = SegformerForSemanticSegmentation.from_pretrained(
    "nvidia/mit-b0",
    num_labels=2,
    id2label={0: 'background', 1: 'foreground'},
    label2id={'background': 0, 'foreground': 1}
)

Training Parameters

  • Learning Rate: 6e-5 (optimized for fine-tuning)
  • Optimizer: AdamW
  • Batch Size: 8 (training and validation)
  • Epochs: 10
  • Metric: Mean Intersection over Union (mIoU)

Loss Function

The model utilizes the built-in SegFormer loss function, which combines:

  • Cross-entropy loss for pixel-wise classification
  • Auxiliary losses from intermediate decoder outputs

Implementation Details

Training Pipeline

  1. Data Loading: Custom PyTorch Dataset class for medical image-mask pairs
  2. Model Initialization: Pre-trained SegFormer with custom classification head
  3. Training Loop:
    • Forward pass with pixel-wise classification
    • Loss computation and backpropagation
    • Metric evaluation (mIoU, accuracy)
    • Progress tracking with tqdm

Evaluation Metrics

  • Mean Intersection over Union (mIoU): Primary segmentation metric
  • Pixel-wise Accuracy: Secondary evaluation metric
  • Visual Inspection: Qualitative assessment through prediction visualization

Technical Challenges

Memory Management

The training encountered CUDA memory limitations, requiring:

  • Batch size optimization
  • Gradient accumulation strategies
  • Memory-efficient data loading

Domain Adaptation

Adapting SegFormer from natural images to medical imaging required:

  • Careful preprocessing pipeline design
  • Binary classification head configuration
  • Domain-specific evaluation metrics

Results and Analysis

Model Performance

The training process demonstrated:

  • Successful model initialization with pre-trained weights
  • Proper loss computation and gradient flow
  • Effective fine-tuning on medical imaging data

Visualization

The implementation includes:

  • Custom color palette for segmentation visualization
  • Overlay techniques for result interpretation
  • Qualitative assessment tools

File Structure

DIAS/
├── DIAS_SegFormer_Custom_FineTuning.ipynb  # Main implementation notebook
├── train/                                   # Training data directory
│   ├── images/                             # Input medical images
│   └── masks/                              # Binary segmentation masks
├── val/                                    # Validation data directory
│   ├── images/                             # Validation images
│   └── masks/                              # Validation masks
├── README.md                               # This documentation
└── LICENSE                                 # License file

Usage

Environment Setup

pip install transformers datasets evaluate torch torchvision
pip install numpy matplotlib PIL tqdm

Running the Model

  1. Data Preparation: Ensure DIAS dataset is properly downloaded and structured
  2. Model Training: Execute the Jupyter notebook cells sequentially
  3. Inference: Use trained model for new image segmentation

Inference Example

# Load trained model
model = SegformerForSemanticSegmentation.from_pretrained("./trained_model")

# Process new image
inputs = image_processor(image, return_tensors="pt")
outputs = model(**inputs)

# Generate prediction
prediction = outputs.logits.argmax(dim=1).squeeze().cpu().numpy()

Research Contributions

  1. Domain Application: Successful adaptation of SegFormer to intracranial artery segmentation
  2. Medical AI: Contributing to automated medical image analysis
  3. Transformer Segmentation: Demonstrating transformer effectiveness in specialized medical domains
  4. Open Research: Providing reproducible implementation for research community

Technical Specifications

  • Framework: PyTorch, Hugging Face Transformers
  • Model: SegFormer-B0 (mit-b0)
  • Input Resolution: Variable (processor-dependent)
  • Output: Binary segmentation masks
  • Hardware: CUDA-compatible GPU recommended

Limitations and Future Work

Current Limitations

  • Memory constraints during training
  • Limited dataset size
  • Binary segmentation scope

Future Directions

  • Multi-class arterial segmentation
  • Larger dataset integration
  • Performance optimization for clinical deployment
  • Integration with medical imaging standards (DICOM)

Dependencies

transformers>=4.21.0
torch>=1.12.0
torchvision>=0.13.0
datasets>=2.4.0
evaluate>=0.2.0
numpy>=1.21.0
matplotlib>=3.5.0
Pillow>=8.3.0
tqdm>=4.64.0

Acknowledgments

  • Hugging Face Transformers team for the SegFormer implementation
  • DIAS dataset creators for providing the medical imaging data
  • NVIDIA for the pre-trained SegFormer models
  • Research community for foundational work in medical image segmentation

Contact

For questions, issues, or collaborations regarding this research project, please contact imkaleemahmad@gmail.com.


Note: This is a research project intended for academic and research purposes. Medical applications require proper validation and regulatory approval.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

0