Celerystalk - An Asynchronous Enumeration In Addition To Vulnerability Scanner

celerystalk helps you lot automate your network scanning/enumeration procedure amongst asynchronous jobs (aka tasks) spell retaining total command of which tools you lot desire to run.

• Configurable - Some mutual tools are inwards the default config, simply you lot tin add together whatever tool you lot want
• Service Aware - Uses nmap/nessus service names rather than port numbers to determine which tools to run
• Scalable - Designed for scanning multiple hosts, simply industrial plant good for scanning i host at a time
• VirtualHosts - Supports subdomain recon too virtualhost scanning
• Job Control - Supports canceling, pausing, too resuming of tasks, inspired past times Burp scanner
• Screenshots Automatically takes screenshots of every url identified via brute force (gobuster) too spidering (Photon)

Install/Setup

• Supported Operating Systems: Kali
• Supported Python Version: 2.x

You must install too run celerystalk every bit root

# git clone https://github.com/sethsec/celerystalk.git # cd celerystalk/setup # ./install.sh # cd .. # ./celerystalk -h

You must install too run celerystalk every bit root

Using celerystalk - The basics
[CTF/HackTheBox mode] - How to scan a host past times IP

# nmap 10.10.10.10 -Pn -p- -sV -oX tenten.xml                       # Run nmap # ./celerystalk workspace create -o /htb                            # Create default workspace too laid output dir # ./celerystalk import -f tenten.xml                                # Import scan  # ./celerystalk db services                                         # If you lot desire to run into what services were loaded # ./celerystalk scan                                                # Run all enabled commands # ./celerystalk question spotter (then Ctrl+c)                           # Watch scans every bit movement from pending > running > consummate # ./celerystalk written report                                              # Generate written report # firefox /htb/celerystalkReports/Workspace-Report[Default.html] &  # View written report 

[Vulnerability Assessment Mode] - How to scan a listing of in-scope hosts/networks too whatever subdomains that resolve to whatever of the in-scope IPs

# nmap -iL client-inscope-list.txt -Pn -p- -sV -oX client.xml       # Run nmap # ./celerystalk workspace create -o /assessments/client             # Create default workspace too laid output dir # ./celerystalk import -f client.xml -S scope.txt                   # Import scan too reach files # ./celerystalk subdomains -d client.com,client.net            # Find subdomains too determine if inwards reach # ./celerystalk scan                                                # Run all enabled commands # ./celerystalk question spotter (then Ctrl+c)                           # Wait for scans to destination # ./celerystalk written report                                              # Generate written report # firefox /celerystalkReports/Workspace-Report[Default].html &# View written report 

[URL Mode] - How to scan a a URL (Use this manner to scan sub-directories flora during showtime moving ridge of scans).

# ./celerystalk workspace create -o /assessments/client             # Create default workspace too laid output dir # ./celerystalk scan -u http://10.10.10.10/secret_folder/           # Run all enabled commands # ./celerystalk question spotter (then Ctrl+c)                           # Wait for scans to destination # ./celerystalk written report                                              # Generate written report # firefox /celerystalkReports/Workspace-Report[Default].html & View written report 

Using celerystalk - Some to a greater extent than detail

1. Configure which tools you'd similar celerystalk to execute: The install script drops a config.ini file inwards the celerystalk folder. The config.ini script is broken upwards into 3 sections:
   Service Mapping - The showtime department normalizes Nmap & Nessus service names for celerystalk (this persuasion was created past times @codingo_ inwards Reconnoitre AFAIK).
   

   [nmap-service-names] http = http,http-alt,http-proxy,www,http? https = ssl/http,https,ssl/http-alt,ssl/http? ftp = ftp,ftp? mysql = mysql dns = dns,domain,domain

   Domain Recon Tools - The minute department defines the tools you'd similar to role for subdomain discovery (an optional feature):
   

   [domain-recon] amass               : /opt/amass/amass -d [DOMAIN] sublist3r           : python /opt/Sublist3r/sublist3r.py -d [DOMAIN]

   Service Configuration - The remainder of the confi.ini sections define which commands you lot desire celerystalk to run for each identified service (i.e., http, https, ssh).
   
   • Disable whatever command past times commenting it out amongst a ; or a #.
   • Add your ain commands using [TARGET],[PORT], too [OUTPUT] placeholders.
   Here is an example:
   

   [http] whatweb             : whatweb http://[TARGET]:[PORT] -a3 --colour=never > [OUTPUT].txt cewl                : cewl http://[TARGET]:[PORT]/ -m vi -w [OUTPUT].txt curl_robots         : ringlet http://[TARGET]:[PORT]/robots.txt --user-agent 'Googlebot/2.1 (+http://www.google.com/bot.html)' --connect-timeout xxx --max-time 180  > [OUTPUT].txt nmap_http_vuln      : nmap -sC -sV -Pn -v -p [PORT] --script=http-vuln* [TARGET] -d -oN [OUTPUT].txt -oX [OUTPUT].xml --host-timeout 120m --script-timeout 20m nikto               : nikto -h http://[TARGET] -p [PORT] &> [OUTPUT].txt gobuster-common     : gobuster -u http://[TARGET]:[PORT]/ -k -w /usr/share/seclists/Discovery/Web-Content/common.txt -s '200,204,301,302,307,403,500' -e -n -q > [OUTPUT].txt photon              : python /opt/Photon/photon.py -u http://[TARGET]:[PORT] -o [OUTPUT] ;gobuster_2.3-medium : gobuster -u http://[TARGET]:[PORT]/ -k -w /usr/share/wordlists/dirbuster/directory-list-lowercase-2.3-medium.txt -s '200,204,301,307,403,500' -e -n -q > [OUTPUT].txt

2. Run Nmap or Nessus:
   
   • Nmap: Run nmap against your target(s). Required: enable version detection (-sV) too output to XML (-oX filename.xml). All other nmap options are upwards to you. Here are roughly examples:

      nmap target(s) -Pn -p- -sV -oX filename.xml   nmap -iL target_list.txt -Pn -sV -oX filename.xml

   • Nessus: Run nessus against your target(s) too export results every bit a .nessus file
3. Create worksapce:
   

   Option	Description
   no options	Prints electrical flow workspace
   create	Creates novel workspace
   -w	Define novel workspace name
   -o	Define output directory assigned to workspace

     Create default workspace    ./celerystalk workspace create -o /assessments/client   Create named workspace      ./celerystalk workspace create -o /assessments/client -w customer   Switch to roughly other worksapce ./celerystalk workspace client

4. Import Data: Import information into celerystalk
   

   Option	Description
   -f scan.xml	Nmap/Nessus xml Adds all IP addresses from this file to hosts tabular array too marks them all inwards reach to travel scanned. Adds all ports too service types to services table.
   -S scope.txt	Scope file Show file differences that haven't been staged
   -D subdomains.txt	(sub)Domains file celerystalk determines whether each subdomain is inwards reach past times resolving the IP too looking for IP inwards the DB. If at that spot is a match, the domain is marked every bit inwards reach too volition travel scanned.

   Import Nmap XML file:       ./celerystalk import -f /assessments/nmap.xml  Import Nessus file:         ./celerystalk import -f /assessments/scan.nessus  Import listing of Domains:     ./celerystalk import -D  Import listing of IPs/Ranges:  ./celerystalk import -S  Specify workspace:          ./celerystalk import -f       Import multiple files:      ./celerystalk import -f nmap.xml -S scope.txt -D domains.txt

5. Find Subdomains (Optional): celerystalk volition perform subdomain recon using the tools specified inwards the config.ini.
   

   Option	Description
   -d domain1,domain2,etc	Run Amass, Sublist3r, etc. too shop domains inwards DB After running your subdomain recon tools celerystalk determines whether each subdomain is inwards reach past times resolving the IP too looking for IP inwards the DB. If at that spot is a match, the domain is marked every bit inwards reach too volition travel scanned.

   Find subdomains:       celerystalk subdomains -d domain1.com,domain2.com

6. Launch Scan: I recommend using the import command showtime too running scan amongst no options, soundless you lot create create got the pick to create it all at i time (import too scan) past times using the flags below. celerystalk volition submit tasks to celery which asynchronously executes them too logs output to your output directory.
   

   Option	Description
   no options	Scan all inwards reach hosts Reads DB too scans every inwards reach IP too subdomain. Launches all enabled tools for IPs, simply alone http/http specific tools against virtualhosts
   -t ip,vhost,cidr	Scan specific target(s) from DB or scan file Scan a subset of the inwards reach IPs and/or subdomains.
   -s	Simulation Sends all of the tasks to celery, simply all commands are executed amongst a # earlier them rendering them inert.
   Use these alone if you lot desire to skip the import stage too import/scan all at once
   -f scan.xml	Import too procedure Nmap/Nessus xml earlier scan Adds all IP addresses from this file to hosts tabular array too marks them all inwards reach to travel scanned. Adds all ports too service types to services table.
   -S scope.txt	Import too procedure reach file earlier scan Show file differences that haven't been staged.
   -D subdomains.txt	Import too procedure (sub)domains file earlier scan celerystalk determines whether each subdomain is inwards reach past times resolving the IP too looking for IP inwards the DB. If at that spot is a match, the domain is marked every bit inwards reach too volition travel scanned.
   -d domain1,domain2,etc	Find Subdomains too scan inwards reach hosts After running your subdomain recon tools celerystalk determines whether each subdomain is inwards reach past times resolving the IP too looking for IP inwards the DB. If at that spot is a match, the domain is marked every bit inwards reach too volition travel scanned.

   Scan imported hosts/subdomains
   

   Scan all inwards reach hosts:    ./celerystalk scan     Scan subset of DB hosts:    ./celerystalk scan -t 10.0.0.1,10.0.0.3                             ./celerystalk scan -t 10.0.0.100-200                             ./celerystalk scan -t 10.0.0.0/24                             ./celerystalk scan -t sub.domain.com Simulation mode:            ./celerystalk scan -s

   Import too Scan
   

   Start from Nmap XML file:   ./celerystalk scan -f /pentest/nmap.xml -o /pentest Start from Nessus file:     ./celerystalk scan -f /pentest/scan.nessus -o /pentest Scan all inwards reach vhosts:   ./celerystalk scan -f  -o /pentest -d domain1.com,domain2.com Scan subset hosts inwards XML:   ./celerystalk scan -f  -o /pentest -t 10.0.0.1,10.0.0.3                             ./celerystalk scan -f  -o /pentest -t 10.0.0.100-200                             ./celerystalk scan -f  -o /pentest -t 10.0.0.0/24 Simulation mode:            ./celerystalk scan -f  -o /pentest -s

7. Rescan: Use this command to rescan an already scanned host.
   

   Option	Description
   no option	For each inwards reach host inwards the DB, celerystalk volition enquire if if you lot desire to rescan it
   -t ip,vhost,cidr	Scan a subset of the inwards reach IPs and/or subdomains.

   Rescan all hosts:           ./celerystalk rescan Rescan roughly hosts           ./celerystalk rescan-t 1.2.3.4,sub.domain.com   Simulation mode:            ./celerystalk rescan -s   

8. Query Status: Asynchronously depository fiscal establishment check the condition of the tasks queue every bit often every bit you lot like. The spotter manner truly executes the linux spotter command hence you lot don't fill upwards up your entire finally buffer.
   

   Option	Description
   no options	Shows all tasks inwards the defualt workspace
   watch	Sends command to the unix spotter command which volition allow you lot teach an updated condition every 2 seconds
   brief	Limit of five results per condition (pending/running/completed/cancelled/paused)
   summary	Shows alone a banner amongst numbers too non the tasks themselves

   Query Tasks:                ./celerystalk question                              ./celerystalk question spotter                                                            ./celerystalk question brief                             ./celerystalk question summary                                                             ./celerystalk question summary watch

9. Cancel/Pause/Resume Tasks: Cancel/Pause/Resume whatever task(s) that are currently running or inwards the queue.
   

   Option	Description
   cancel	Canceling a running task volition mail a kill -TERM Canceling a queued task* volition brand celery ignore it (uses celery's revoke). Canceling all tasks* volition kill running tasks too revoke all queued tasks.
   pause	Pausing a unmarried task uses kill -STOP to suspend the process. Pausing all tasks* attemps to kill -STOP all running tasks, simply it is a piffling wonky too you lot remove heed demand to run it a few times. It is possible a task completed earlier it was able to travel paused, which agency you lot volition create got a worker that is soundless accepting novel jobs.
   resume	Resuming tasks* sends a kill -CONT which allows the procedure to start upwards i time again where it left off.

   Cancel/Pause/Resume Tasks:  ./celerystalk  5,6,10-20          #Cancel/Pause/Resume tasks 5, 6, too 10-20 from electrical flow workspace                             ./celerystalk  all                #Cancel/Pause/Resume all tasks from electrical flow workspaces

10. Run Report: Run a written report which combines all of the tool output into an html file too a txt file. Run this every bit often every bit you lot like. Each fourth dimension you lot run the written report it overwrites the previous report.
    

    Create Report:              ./celerystalk written report                    #Create a written report for all scanneed hosts inwards electrical flow workspace 

    Screenshot:
    
    
11. Access the DB: List the workspaces, hosts, services, or paths stored inwards the celerystalk database
    

    Option	Description
    workspaces	Show all known workspaces too the output directory associated amongst each workspace
    services	Show all known opened upwards ports too service types past times IP
    hosts	Show all hosts (IP addresses too subdomains/vhosts) too whether they are inwards reach too whether they create got been submitted for scanning
    paths	Show all paths that create got been identified past times vhost
    -w workspace	Specify a non-default workspace

    Show workspaces:            ./celeryststalk db workspaces Show services:              ./celeryststalk db services     Show hosts:                 ./celeryststalk db hosts Show paths:                 ./celeryststalk db paths

12. Export DB: Export each tabular array of the DB to a csv file
    

    Option	Description
    no options	Export the services, hosts, too paths tabular array from the default database
    -w workspace	Specify a non-default workspace

    Export electrical flow DB:        ./celerystalk db export Export roughly other DB:        ./celerystalk db export -w test

Usage

Usage:     celerystalk workspace create -o  [-w workspace_name]     celerystalk workspace []     celerystalk import [-f ] [-S scope_file] [-D subdomains_file] [-u ]     celerystalk subdomains -d  [-s]     celerystalk scan [-f ] [-t ] [-d ] [-S scope_file] [-D subdomains_file] [-s]     celerystalk scan -u  [-s]     celerystalk rescan [-t ] [-s]     celerystalk question ([full] | [summary] | [brief]) [watch]     celerystalk question [watch] ([full] | [summary] | [brief])     celerystalk written report     celerystalk cancel ([all]|[])     celerystalk interruption  ([all]|[])     celerystalk resume ([all]|[])     celerystalk db ([workspaces] | [services] | [hosts] | [vhosts] | [paths])     celerystalk db export     celerystalk shutdown     celerystalk interactive     celerystalk (help | -h | --help)  Options:     -h --help           Show this concealment     -v --version        Show version     -f       Nmap xml import file     -o      Output directory     -S       Scope import file     -D  Subdomains import file     -t         Target(s): IP, IP Range, CIDR     -u             URL to parse too scan amongst all configured tools     -w       Workspace     -d --domains        Domains to scan for vhosts     -s --simulation     Simulation mode.  Submit tasks comment out all commands  Examples:    Workspace     Create default workspace    celerystalk workspace create -o /assessments/client     Create named workspace      celerystalk workspace create -o /assessments/client -w customer     Switch to roughly other worksapce celerystalk workspace client2    Import     Import Nmap XML file:       celerystalk import -f /assessments/nmap.xml     Import Nessus file:         celerystalk import -f /assessments/scan.nessus     Import listing of Domains:     celerystalk import -D      Import listing of IPs/Ranges:  celerystalk import -S      Import multiple files:      celerystalk import -f nmap.xml -S scope.txt -D domains.txt    Subdomain Recon     Find subdomains:            celerystalk subdomains -d domain1.com,domain2.com    Scan     Scan all inwards reach hosts:    celerystalk scan     Scan subset of DB hosts:    celerystalk scan -t 10.0.0.1,10.0.0.3                                 celerystalk scan -t 10.0.0.100-200                                 celerystalk scan -t 10.0.0.0/24                                 celerystalk scan -t sub.domain.com     Simulation mode:            celerystalk scan -s    Import too Scan     Start from Nmap XML file:   celerystalk scan -f /pentest/nmap.xml     Start from Nessus file:     celerystalk scan -f /pentest/scan.nessus     Scan subset hosts inwards XML:   celerystalk scan -f  -t 10.0.0.1,10.0.0.3                                 celerystalk scan -f  -t 10.0.0.100-200                                 celerystalk scan -f  -t 10.0.0.0/24                                 celerystalk scan -f  -t sub.domain.com     Simulation mode:            celerystalk scan -f  -s    Rescan    Rescan all hosts:            celerystalk rescan    Rescan roughly hosts            celerystalk rescan-t 1.2.3.4,sub.domain.com    Simulation mode:             celerystalk rescan -s    Query Mode     All tasks:                  celerystalk question     Update condition every 2s:     celerystalk question spotter     Show alone five tasks per mode: celerystalk question brief     Show stats alone             celerystalk question summary     Show stats every 2s:        celerystalk question summary spotter   Job Control (cancel/pause/resume)     Specific tasks:             celerystalk cancel 5,6,10-20                                 celerystalk interruption 5,6,10-20                                 celerystalk resume 5,6,10-20      All tasks electrical flow worspace: celerystalk cancel all                                 celerystalk interruption all                                 celerystalk resume all    Access the DB     Show workspaces:            celeryststalk db workspaces     Show services:              celeryststalk db services     Show hosts:                 celeryststalk db hosts     Show vhosts alone            celeryststalk db vhosts     Show paths:                 celeryststalk db paths    Export DB     Export electrical flow DB:          celerystalk db export 

Credit
This projection was inspired past times many nifty tools:

1. https://github.com/codingo/Reconnoitre past times @codingo_
2. https://github.com/frizb/Vanquish past times @frizb
3. https://github.com/leebaird/discover past times @discoverscripts
4. https://github.com/1N3/Sn1per
5. https://github.com/SrFlipFlop/Network-Security-Analysis past times @SrFlipFlop

Thanks to @offensivesecurity too @hackthebox_eu for their lab networks
Also, thank you lot to:

1. @decidedlygray for pointing me towards celery, helping me solve python problems that were over my head, too for the extensive beta testing
2. @kerpanic for inspiring me to dust off an one-time projection too plough it into celerystalk
3. My TUV OpenSky squad too my IthacaSec hackers for testing this out too submitting bugs too features

Download Celerystalk