Scrapy Django Dashboard - Documentation¶
Scrapy Django Dashboard is a fork of Django Dynamic Scraper (DDS) by Holger Drewes. It is a web app allowing users to create and manage Scrapy spiders through Django Admin dashboard.
Note
Latest new features added :
- Python
3.7.7 - Django
3.0.6 - Scrapy
2.1.0 - Django Grappelli (Grid-based Django admin dashboard extension)
2.14.2 - Javascript rendering
- Scraping
JSONcontent - More flexible ID and detail page URL(s) concept
- Custom
HTTP Header/Body,Cookies,GET/POSTrequests - Scrapy Meta attributes
- Scraper/Checker Monitoring
Features¶
- Create and manage Scrapy spiders via Django admin dashboard
- Support Scrapy regular expressions, processors, and pipelines (see Scrapy Docs)
- Support image/screenshot scraping
- Schedule spiders via
django-celery - Check existing items
User Manual¶
- Introduction
- Installation
- Getting Started
- Advanced topics
- Basic services
- Reference
- Development