Scrapy project

scrapy Public
Scrapy, a fast high-level web crawling & scraping framework for Python.

scrapy/scrapy’s past year of commit activity

Python 53,172 BSD-3-Clause 10,565 447 (21 issues need help) 221 Updated Nov 15, 2024
protego Public
A pure-Python robots.txt parser with support for modern conventions.

scrapy/protego’s past year of commit activity

DIGITAL Command Language 55 BSD-3-Clause 28 5 (1 issue needs help) 1 Updated Nov 15, 2024
scrapyd Public
A service daemon to run Scrapy spiders

scrapy/scrapyd’s past year of commit activity

Python 2,969 BSD-3-Clause 569 7 0 Updated Nov 11, 2024
scrapy.org Public
The scrapy.org website

scrapy/scrapy.org’s past year of commit activity

HTML 60 139 1 1 Updated Oct 31, 2024
parsel Public
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

scrapy/parsel’s past year of commit activity

Python 1,148 BSD-3-Clause 146 29 (1 issue needs help) 12 Updated Oct 16, 2024
w3lib Public
Python library of web-related functions

scrapy/w3lib’s past year of commit activity

Python 392 BSD-3-Clause 104 11 (1 issue needs help) 4 Updated Oct 16, 2024
itemloaders Public
Library to populate items using XPath and CSS with a convenient API

scrapy/itemloaders’s past year of commit activity

Python 45 BSD-3-Clause 16 17 4 Updated Oct 16, 2024
cssselect Public
CSS Selectors for Python

scrapy/cssselect’s past year of commit activity

Python 291 61 17 4 Updated Oct 16, 2024
queuelib Public
Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python

scrapy/queuelib’s past year of commit activity

Python 269 BSD-3-Clause 54 3 2 Updated Oct 16, 2024
itemadapter Public
Common interface for data container classes

scrapy/itemadapter’s past year of commit activity

Python 62 BSD-3-Clause 13 6 4 Updated Oct 16, 2024

View all repositories

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scrapy project

Pinned Loading

Repositories

People

Top languages

Most used topics