multi-threaded webs crawler
- Python
Run trigger.py, before run this script please edit this script set HOME_PAGE and PROJECT_NAME parameter value PROJECT_NAME value is name of directory where all data will store, HOME_PAGE is base url of web site which you want crawler e.x www.w2school.com
It will collect all links from that web site and store in crawled.txt file