- Proper handling of null response from
robots.txt&sitemap.xml - Python2 compatibility
- Proper handling of non-ascii chars
- Added ability to specify custom regex pattern
- Display total time taken and average time per request
- Disabled colors on windows and mac
- Cross platform file handling
First stable release.