200 OK          The request has succeeded.
403 Forbidden   The server understood the request, but is refusing to fulfill it.
404 Not Found   The server hasn't found anything matching the Request-URI.
302 Found       The requested resource resides temporarily under a different URI.
...

$ python3 parsero.py -h

usage: parsero.py [-h] [-u URL] [-o] [-sb]

optional arguments:
-h, --help  show this help message and exit
-u URL      Type the URL which will be analyzed
-o          Show only the "HTTP 200" status code
-sb         Search in Bing indexed Disallows

• See the big picture and think out of the box
• More efficiently find, verify and combine vulnerabilities
• Have time to investigate complex vulnerabilities like business logic/architectural flaws or virtual hosting sessions
• Perform more tactical/targeted fuzzing on seemingly risky areas
• Demonstrate true impact despite the short timeframes we are typically given to test.

Features

• OWASP Testing Guide-oriented: owtf will try to classify the findings as closely as possible to the OWASP Testing Guide
• Report updated on the fly: As soon as each plugin finishes or sometimes before (i.e. after each vulnerability scanner finishes)
• "Scumbag spidering": Instead of implementing yet another spider (a hard job), owtf will scrub the output of all tools/plugins run to gather as many URLs as possible. This is somewhat "cheating" but tremendously effective since it combines the results of different tools, including several tools that perform brute forcing of files and directories.
• Resilience: If one tool crashes owtf will move on to the next tool/test, saving the partial output of the tool until it crashed
• Easy to configure: config files are easy to read and modify
• Easy to run: No strange parameters, DB setup requirements, libraries, complex dependencies, etc
• Full control of what tests to run, interactivity and hopefully easy to follow examples and help :)
• Easy to review transaction logs and plain text files with URLs, simple for scripting
• Basic Google Hacking without (annoying) API Key requirements via "blanket searches", trying a bunch of operators at once, you can then narrow the search down if you find something interesting.
• Easy to extract data from the database to parse or pass to other tools: They are all text files

Requirements

• Linux (any Ubuntu derivative should work just fine) and python 2.6.5 or greater
• Latest Kali version not required but helpful (almost 0 setup time)
• You do NOT have to have all tools installed: owtf will move on with an error for the missing tools

• Spawn and Hook to Application – Enables you to spawn an application, and hook into it
• Hook to a specific running process – Allows you to hook to a running (active) process
• Static Malware Analysis – Scans PE/Windows executables to identify potential malware traces
• Application crash analysis – Allows you to analyse memory content when an application crashes
• Exe extractor – This module essentially extracts executables from running process/s

• You need to provide range of ip's (e.g. 127.0.0.1 - 127.0.0.10); program will perform operations on each address separately, basing on selected options, then it will print out the response.
• By default, program will only check open ports, print http response headers, test for HTTP methods and check FTP for anonymous login.
• If 'SSH/IMAP/DB's' are checked, program will try to bruteforce SSH,IMAP,MySQL,PostgreSQL & MsSQL using array of passwords and users defined on the beggining of script. Notice, that it will try to login as user with grand permissions (e.g. root, postgres), although you're able to edit it.
• If 'FTP User' is set, program will also try to bruteforce FTP with specific user using mentioned passwords array. By default it's not, and it will only test for anonymous login.
• If 'Deep Scan' is set, program will perform all aforementioned operations. Further, it will use nmap with specific parameters you're able to edit, also, the traceroute scan will be performed and displayed - just like nmap. Deep Scan also gather some useful informations about a website (if it's running), such as interesting files/folders and www title.
• ?url=website.com for quick IP address of specific website.
  ...so - what it does it do? Nmap, traceroute, port scan, ftp anonymous login, ftp/ssh/imap/mysql/pgsql/mssql bruteforce, http (website) info gathering, 
  Crawler accepts as a parameter array of files and folders that you can manually edit, just like others options.