sitecheck v1.5 - Modular web site spider for web developers.

Much more than just a link checker, sitecheck is a website spider (also known as a crawler) which can assist with SEO by testing an entire site plus both inbound links from search engines and outbound links to other sites for the following issues: looping redirects (HTTP 301/302), broken links (HTTP 404), server errors (HTTP 500), spelling mistakes, low readability scores (using the Flesch Reading Ease test), missing/empty/duplicate meta tags, duplicate content, slow page speed, W3C validation errors and accessibility errors.

Sitecheck can also spot some common causes of PCI compliance failure such as insecure content on secure pages, SQL injection/cross-site scripting (XSS) vulnerabilities, insecure encryption ciphers and open mail relays. Sources of information leakage such as email addresses and IP addresses in the headers or the page will be logged. Includes a separate module called domaincheck which checks the domain expiry date, SSL certificate expiry date and SPF records.

Features

• Authentication by HTTP form submission
• Suspend and resume
• Download and save a site
• Highly configurable with support for multiple profiles
• Modular processing of responses
• HTML or plain text logging
• Tests inbound links from search engines and outbound links
• Markup validation (W3C)
• Accessibility (WCAG)
• SQL injection and cross-site scripting (XSS) detection
• Spelling check with custom dictionary and readability score
• Checks for duplicate content (for SEO purposes)
• Checks domain expiry date
• Checks SSL certificate expiry date
• Checks for insecure content referenced from secure pages

updated version Download sitecheck-1.5.zip (39.4 kB)

Download other versions 

Source-

http://sourceforge.net/projects/sitecheck/