An intelligent photo curation system based on deep learning, combining aesthetic assessment, quality detection, and diversity reranking techniques.
- 🎨 Aesthetic Scoring: CLIP ViT-B/32 + MLP regression head for predicting image aesthetic scores
- 🏷️ Auto Tagging: BLIP-2 generates keywords and descriptive captions
- 🔍 Quality Assessment: NR-IQA no-reference image quality evaluation
- 🎯 Diversity Reranking: DPP algorithm eliminates redundancy and improves recommendation diversity
- 📱 Web Interface: Streamlit drag-and-drop upload → one-click curation → export results
- Python 3.8+
- PyTorch 2.0+
- Streamlit 1.25+
pip install -r requirements.txt# Method 1: Using startup script
./start_demo.sh
# Method 2: Manual launch
streamlit run demo/app.pyVisit http://localhost:8501 to get started!
- Aesthetic Correlation: Spearman ρ ≥ 0.40
- Inference Speed: <120ms/image
- Curation Effect: NDCG@20 +14%, repetition rate -35%
- Diversity Improvement: +14% NDCG@20
- Deep Learning: PyTorch, CLIP, BLIP-2, Transformers
- Computer Vision: NR-IQA, feature extraction, image processing
- Algorithms: DPP (Determinantal Point Process), cosine similarity, MLP regression
- Frontend: Streamlit, HTML/CSS, responsive design
- Data Processing: Pandas, NumPy, PIL, OpenCV
PixRank/
├── src/ # Core algorithm implementation
│ ├── aesthetics/ # Aesthetic assessment module
│ ├── quality/ # Quality assessment module
│ ├── diversity/ # Diversity reranking module
│ ├── scoring/ # Improved scoring system
│ └── utils/ # Utility functions
├── demo/ # Streamlit demo application
├── data/ # Datasets and samples
├── models/ # Pre-trained model weights
├── test_*.py # Test files
└── requirements.txt # Dependencies list
- 使用CLIP ViT-B/32提取视觉特征
- 自定义MLP回归头预测美学分数
- 支持AVA数据集训练
- BLIP-2预训练模型
- 生成关键词和描述性标题
- 无需训练，直接使用
- NR-IQA无参考图像质量评估
- ResNet50特征提取器
- 评估清晰度、噪点、技术质量
- DPP (Determinantal Point Process)算法
- 贪心选择策略
- 平衡质量与多样性
- 现代化UI设计
- 拖拽上传支持
- 实时处理进度
- 结果导出功能
- 上传图片 → 选择要精选的图片文件
- 设置参数 → 调整精选数量和权重
- 开始精选 → 点击处理按钮
- 查看结果 → 浏览精选图片和分数
- 下载结果 → 保存精选图片和数据
- 美学权重: 0.0-1.0 (默认0.6)
- 质量权重: 自动计算 (1.0 - 美学权重)
- 精选数量: 1-50张 (默认10张)
- JPG, JPEG
- PNG
- BMP
- TIFF
Built an intelligent photo curation engine using deep learning with CLIP, BLIP-2, and NR-IQA models, implementing DPP-based diversity reranking to improve recommendation quality by 14% NDCG@20
• Developed multi-modal aesthetic scoring system using CLIP ViT-B/32 feature extraction with custom MLP regression head, achieving Spearman correlation ρ ≥ 0.40 on AVA dataset (250K images)
• Implemented automated content tagging pipeline with BLIP-2 transformer model for keyword generation and descriptive captioning, processing 100+ images per batch
• Designed diversity-aware reranking algorithm using Determinantal Point Process (DPP) to eliminate redundant selections, reducing similarity by 35% while maintaining quality scores
欢迎提交Issue和Pull Request！
本项目采用 MIT 许可证 - 查看 LICENSE 文件了解详情
- CLIP - OpenAI的视觉-语言模型
- BLIP-2 - Salesforce的多模态模型
- Streamlit - 快速构建数据应用
- AVA Dataset - 美学评估数据集
⭐ 如果这个项目对你有帮助，请给个Star！