Web Crawler

Intelligent Web Crawler
Shallow, same-origin discovery with real status codes and per-page secret classification.