Security Onion - Linux Distro For IDS, NSM, And Log Management

Security Onion is a free and open source Linux distribution for intrusion detection, enterprise security monitoring, and log management. It includes Elasticsearch, Logstash, Kibana, Snort, Suricata, Bro, OSSEC, Sguil, Squert, NetworkMiner, and many other security tools. The easy-to-use Setup wizard allows you to build an army of distributed sensors for your enterprise in minutes!

Security-onion project

This repo contains the ISO image, Wiki, and Roadmap for Security Onion.

Looking for documentation?

Please proceed to the Wiki.

Screenshots

Download Security-Onion

Analyze The Security Of Any Domain By Finding All the Information Possible - Domain Analyzer

Domain analyzer is a security analysis tool which automatically discovers and reports information about the given domain. Its main purpose is to analyze domains in an unattended way.

How

Domain analyzer takes a domain name and finds information about it, such as DNS servers, mail servers, IP addresses, mails on Google, SPF information, etc. After all the information is stored and organized it scans the ports of every IP found using nmap and perform several other security checks. After the ports are found, it uses the tool crawler.py from @verovaleros, to spider the complete web page of all the web ports found. This tool has the option to download files and find open folders.

Current version is 0.8 and the main features are:

• It creates a directory with all the information, including nmap output files.
• It uses colors to remark important information on the console.
• It detects some security problems like host name problems, unusual port numbers and zone transfers.
• It is heavily tested and it is very robust against DNS configuration problems.
• It uses nmap for active host detection, port scanning and version information (including nmap scripts).
• It searches for SPF records information to find new hostnames or IP addresses.
• It searches for reverse DNS names and compare them to the hostname.
• It prints out the country of every IP address.
• It creates a PDF file with results.
• It automatically detects and analyze sub-domains!
• It searches for domains emails.
• It checks the 192 most common hostnames in the DNS servers.
• It checks for Zone Transfer on every DNS server.
• It finds the reverse names of the /24 network range of every IP address.
• It finds active host using nmap complete set of techniques.
• It scan ports using nmap (remember that for the SYN scan you need to need root).
• It searches for host and port information using nmap.
• It automatically detects web servers used.
• It crawls every web server page using our crawler.py tool. See the description below.
• It filters out hostnames based on their name.
• It pseudo-randomly searches N domains in Google and automatically analyze them!
• Uses CTRL-C to stop current analysis stage and continue working.
• It can read an external file with domain names and try to find them on the domain.

Bonus features
@verovaleros developed a separate python web crawler called "crawler.py". Its main features are:

• Crawl http and https web sites.
• Crawl http and https web sites not using common ports.
• Uses regular expressions to find 'href' and 'src' html tag. Also content links.
• Identifies relative links.
• Identifies domain related emails.
• Identifies directory indexing.
• Detects references to URLs like 'file:', 'feed=', 'mailto:', 'javascript:' and others.
• Uses CTRL-C to stop current crawler stages and continue working.
• Identifies file extensions (zip, swf, sql, rar, etc.)
• Download files to a directory:
  • Download every important file (images, documents, compressed files).
  • Or download specified files types.
  • Or download a predefined set of files (like 'document' files: .doc, .xls, .pdf, .odt, .gnumeric, etc.).
• Maximum amount of links to crawl. A default value of 5000 URLs is set.
• Follows redirections using HTML and JavaScript Location tag and HTTP response codes.

This extended edition has more features!

• World-domination: You can automatically analyze the whole world! (if you have time)
• Robin-hood: Although it is still in development, it will let you send automatically an email to the mails found during scan with the analysis information.
• Robtex DNS: With this incredible function, every time you found a DNS servers with Zone Transfer, it will retrieve from the Robtex site other domains using that DNS server! It will automatically analyze them too! This can be a never ending test! Every vulnerable DNS server can be used by hundreds of domains, which in turn can be using other vulnerable DNS servers. BEWARE! Domains retrieved can be unrelated to the first one.

Examples

• Find 10 random domains in the .gov domain and analyze them fully (including web crawling). If it finds some Zone Transfer, retrieve more domains using them from Robtex!!

  domain_analyzer.py -d .gov -k 10 -b

• (Very Quick and dirty) Find everything related with .edu.cn domain, store everything in directories. Do not search for active host, do not nmap scan them, do not reverse-dns the netblock, do not search for emails.

  domain_analyzer.py -d edu.cn -b -o -g -a -n

• Analyze the 386.edu.ru domain fully

  domain_analyzer.py -d 386.edu.ru -b -o

• (Pen tester mode). Analyze a domain fully. Do not find other domains. Print everything in a pdf file. Store everything on disk. When finished open Zenmap and show me the topology every host found at the same time!

  domain_analyzer.py -d amigos.net -o -e

• (Quick with web crawl only). Ignore everything with 'google' on it.

  domain_analyzer.py -d mil.cn -b -o -g -a -n -v google -x '-O --reason --webxml --traceroute -sS -sV -sC -PN -n -v -p 80,4443'

• (Everything) Crawl up to 100 URLs of this site including subdomains. Store output into a file and download every INTERESTING file found to disk.

  crawler.py -u www.386.edu.ru -w -s -m 100 -f

• (Quick and dirty) Crawl the site very quick. Do not download files. Store the output to a file.

  crawler.py -u www.386.edu.ru -w -m 20

• (If you want to analyze metadata later with lafoca). Verbose prints which extensions are being downloaded. Download only the set of archives corresponding to Documents (.doc, .docx, .ppt, .xls, .odt. etc.)

  crawler.py -u ieeeexplore.ieee.org/otherfiles/ -d -v

Most of these features can be deactivated.

Screenshots

1. Example domain_analyzer.py -d .gov -k 10 -b

Installation
Just untar the .tar.gz file and copy the python files to the /usr/bin/ directory. Domain_analyzer needs to be run as root. The crawler can be run as a non-privileged user. If you want all the features (web crawler, pdf and colors), which is nice, also copy these files to /usr/bin or /usr/local/bin

• ansistrm.py
• crawler.py
• pyText2pdf.py

If you have any issues with the GeoIP database, please download it from its original source here. And install it in where your system needs it, usually at /opt/local/share/GeoIP/GeoIP.dat

Download Domain Analyzer

Advance Android Malware Analysis Framework - Droidefense

Droidefense (originally named atom: analysis through observation machine)* is the codename for android apps/malware analysis/reversing tool. It was built focused on security issues and tricks that malware researcher have on they every day work. For those situations on where the malware has anti-analysis routines, Droidefense attemps to bypass them in order to get to the code and 'bad boy' routine. Sometimes those techniques can be virtual machine detection, emulator detection, self certificate checking, pipes detection. tracer pid check, and so on.

Droidefense uses an innovative idea in where the code is not decompiled rather than viewed. This allow us to get the global view of the execution workflow of the code with a 100% accuracy on gathered information. With this situation, Droidefense generates a fancy html report with the results for an easy understanding.

Usage

TL;DR

java -jar droidefense-cli-1.0-SNAPSHOT.jar -i /path/to/your/sample.apk

Detailed usage

java -jar droidefense-cli-1.0-SNAPSHOT.jar

________               .__    .___      _____                            
\______ \_______  ____ |__| __| _/_____/ ____\____   ____   ______ ____  
 |    |  \_  __ \/  _ \|  |/ __ |/ __ \   __\/ __ \ /    \ /  ___// __ \ 
 |    `   \  | \(  <_> )  / /_/ \  ___/|  | \  ___/|   |  \\___ \\  ___/ 
/_______  /__|   \____/|__\____ |\___  >__|  \___  >___|  /____  >\___  >
        \/                     \/    \/          \/     \/     \/     \/ 


 * Current build:    2017_12_05__12_07_01
 * Check out on Github:    https://github.com/droidefense/
 * Report your issue:    https://github.com/droidefense/engine/issues
 * Lead developer:    @zerjioang

usage: droidefense
 -d,--debug                 print debugging information
 -h,--help                  print this message
 -i,--input            input .apk to be analyzed
 -o,--output        select prefered output:
                            json
                            json.min
                            html
 -p,--profile               Wait for JVM profiler
 -s,--show                  show generated report after scan
 -u,--unpacker    select prefered unpacker:
                            zip
                            memapktool
 -v,--verbose               be verbose
 -V,--version               show current version information
 

Useful info

• Checkout how to compile new version at:
  • https://github.com/droidefense/engine/wiki/Compilation
• Checkout report example at:
  • https://github.com/droidefense/engine/wiki/Pornoplayer-report
• Checkout execution logs at:
  • https://github.com/droidefense/engine/wiki/Execution-logs

Download Droidefense

Tool to simulate fake processes of analysis sandbox/VM software - Fake Sandbox Processes (FSP)

This small script will simulate fake processes of analysis, sandbox and/or VM software that some malware will try to avoid. You can download the original script (made by @x0rz ) in the orig directory.

You can also download my slightly optimized script in the main directory. The file is named fsp.ps1.

Script-Features

• Some (good) spyware will stop spying on you as long as the processes run, which are created by this script.
• Requirements: Powershell (preinstalled on Win 7 and newer)
• Runs on every Windows since Vista
• Tiny size
• No CPU load
• Easy to use
• No network connection required

Installer-Features

• Automatically install the script to your autostart directory
• Extremely easy to install
• Uninstaller to purge all files
• NO requirements
• Tiny size
• Offline package
• Automatic updater included - only if you want!

Usage:
Open the command line and paste this command (don't forget to adjust the path!):

Powershell -executionpolicy remotesigned -File "Your\Path\fsp.ps1"

After pressing ENTER you will be asked to either start or stop all processes.

Autostart
If you execute the powershell script with the above command, you will have to rerun it after every login or startup. In order to autostart the script I made an easy-to-use installer.

• Download the fsp-installer.bat file from the release section.
• Double-click it.
• Now you will see this cmd window:

• Choose "i" to start the installer.
• Now enter "y" to start installing or "n" to abort.
• If you entered "y" this image will now pop up:

• Enter "y" to install the auto-updater or "n" to not install it (it'll work anyways, but it is recommended to install the updater).

You can now close the window or press any key to close it. The (un)installation is complete.

Uninstall
If you no longer want this program on your computer, you'll need the fsp-installer.bat file again. Run it and chose "u" to start the uninstallation process. Then enter "y" if you are ready to uninstall it.

If the process was successful, you'll see a confirmation screen.

Successfully tested on the following Windows versions:

• Win 10 Professional
• Win 8.1 Home
• Win 7 Professional

Download Fake Sandbox Processes

Mail Header Analyzer - MHA

Mail header analyzer is a tool written in flask for parsing email headers and converting them to a human readable format and it also can:

• Identify hop delays.
• Identify the source of the email.
• Identify hop country.

MHA is an alternative for the following:

Name	Dev	Issues
MessageHeader	Google	Not showing all the hops.
EmailHeaders	Mxtoolbox	Not accurate and slow.
Message Header Analyzer	Microsoft	Broken UI.

Installation
Install system dependencies:

sudo apt-get update
sudo apt-get install python-pip
sudo pip install virtualenv

Create a Python virtual environment and activate it:

virtualenv virt
source virt/bin/activate

Clone the GitHub repo:

git clone https://github.com/lnxg33k/MHA.git

Install Python dependencies:

cd MHA
pip install -r requirements.txt

Run the development server:
python server.py -d
You can change the bind address or port by specifying the appropriate options: python server.py -b 0.0.0.0 -p 8080
Everything should go well, now visit http://localhost:8080.

Docker
A Dockerfile is provided if you wish to build a docker image.

docker build -t mha:latest .

You can then run a container with:

docker run -d -p 8080:8080 mha:latest

Download MHA

Tools to analyze MS OLE2 files and MS Office documents, for malware analysis, forensics and debugging - oletools

oletools is a package of python tools to analyze Microsoft OLE2 files (also called Structured Storage, Compound File Binary Format or Compound Document File Format), such as Microsoft Office documents or Outlook messages, mainly for malware analysis, forensics and debugging. It is based on the olefile parser. See http://www.decalage.info/python/oletools for more info.

News

• 2016-11-01 v0.50: all oletools now support python 2 and 3.
  • olevba: several bugfixes and improvements.
  • mraptor: improved detection, added mraptor_milter for Sendmail/Postfix integration.
  • rtfobj: brand new RTF parser, obfuscation-aware, improved display, detect executable files in OLE Package objects.
  • setup: now creates handy command-line scripts to run oletools from any directory.
• 2016-06-10 v0.47: olevba added PPT97 macros support, improved handling of malformed/incomplete documents, improved error handling and JSON output, now returns an exit code based on analysis results, new --relaxed option. rtfobj: improved parsing to handle obfuscated RTF documents, added -d option to set output dir. Moved repository and documentation to GitHub.
• 2016-04-19 v0.46: olevba does not deobfuscate VBA expressions by default (much faster), new option --deobf to enable it. Fixed color display bug on Windows for several tools.
• 2016-04-12 v0.45: improved rtfobj to handle several anti-analysis tricks, improved olevba to export results in JSON format.

See the full changelog for more information.

Tools:

• olebrowse: A simple GUI to browse OLE files (e.g. MS Word, Excel, Powerpoint documents), to view and extract individual data streams.
• oleid: to analyze OLE files to detect specific characteristics usually found in malicious files.
• olemeta: to extract all standard properties (metadata) from OLE files.
• oletimes: to extract creation and modification timestamps of all streams and storages.
• oledir: to display all the directory entries of an OLE file, including free and orphaned entries.
• olemap: to display a map of all the sectors in an OLE file.
• olevba: to extract and analyze VBA Macro source code from MS Office documents (OLE and OpenXML).
• MacroRaptor: to detect malicious VBA Macros
• pyxswf: to detect, extract and analyze Flash objects (SWF) that may be embedded in files such as MS Office documents (e.g. Word, Excel) and RTF, which is especially useful for malware analysis.
• oleobj: to extract embedded objects from OLE files.
• rtfobj: to extract embedded objects from RTF files.
• and a few others (coming soon)

Projects using oletools:

oletools are used by a number of projects and online malware analysis services, including Viper, REMnux, FAME, Hybrid-analysis.com, Joe Sandbox, Deepviz, Laika BOSS, Cuckoo Sandbox, Anlyz.io, ViperMonkey, pcodedmp, dridex.malwareconfig.com, and probably VirusTotal. (Please contact me if you have or know a project using oletools)

Download and Install:
The recommended way to download and install/update the latest stable release of oletools is to use pip:

• On Linux/Mac:  Sudo -H pip install -U oletools
• On Windows:  Pip install -U oletools 

This should automatically create command-line scripts to run each tool from any directory: olevba, mraptor, rtfobj, etc.
To get the latest development version instead:

• On Linux/Mac: sudo -H pip install -U https://github.com/decalage2/oletools/archive/master.zip
• On Windows: pip install -U https://github.com/decalage2/oletools/archive/master.zip

See the documentation for other installation options.

Documentation:
The latest version of the documentation can be found online, otherwise a copy is provided in the doc subfolder of the package.

Download oletools

A DNS Reconnaissance Tool for Locating Non-Contiguous IP Space - Fierce

First, credit where credit is due, fierce was originally written by RSnake along with others at http://ha.ckers.org/ . This is simply a conversion to Python 3 to simplify and modernize the codebase.

The original description was very apt, so I'll include it here:

Fierce is a semi-lightweight scanner that helps locate non-contiguous IP space and hostnames against specified domains. It's really meant as a pre-cursor to nmap, unicornscan, nessus, nikto, etc, since all of those require that you already know what IP space you are looking for. This does not perform exploitation and does not scan the whole internet indiscriminately. It is meant specifically to locate likely targets both inside and outside a corporate network. Because it uses DNS primarily you will often find mis-configured networks that leak internal address space. That's especially useful in targeted malware.

Installing

$ pip3 install fierce
$ fierce -h

OR

$ git clone https://github.com/mschwager/fierce.git
$ cd fierce
$ pip3 install -r requirements.txt
$ python3 fierce.py -h

Using
Let's start with something basic:

$ fierce --domain google.com --subdomains accounts admin ads

Traverse IPs near discovered domains to search for contiguous blocks with the --traverse flag:

$ fierce --domain facebook.com --subdomains admin --traverse 10

Limit nearby IP traversal to certain domains with the --search flag:

$ fierce --domain facebook.com --subdomains admin --search fb.com fb.net

Attempt an HTTP connection on domains discovered with the --connect flag:

$ fierce --domain stackoverflow.com --subdomains mail --connect

Exchange speed for breadth with the --wide flag, which looks for nearby domains on all IPs of the /24 of a discovered domain:

$ fierce --domain facebook.com --wide

Zone transfers are rare these days, but they give us the keys to the DNS castle. zonetransfer.me is a very useful service for testing for and learning about zone transfers:

$ fierce --domain zonetransfer.me

To save the results to a file for later use we can simply redirect output:

$ fierce --domain zonetransfer.me > output.txt

Internal networks will often have large blocks of contiguous IP space assigned. We can scan those as well:

$ fierce --dns-servers 10.0.0.1 --range 10.0.0.0/24

Check out --help for further information:

$ fierce --help

Download Fierce

Multi-Architecture GDB Enhanced Features for Exploiters & Reverse-Engineers - GEF

GEF is aimed to be used mostly by exploiters and reverse-engineers. It provides additional features to GDB using the Python API to assist during the process of dynamic analysis or exploit development.

GEF fully relies on GDB API and other Linux specific source of information (such as /proc/pid ). As a consequence, some of the features might not work on custom or harden systems such as GrSec. It has fully support for Python2 and Python3 indifferently (as more and more distro start pushing gdb compiled with Python3 support).

Quick start

Simply make sure you're having a GDB 7.x+ .

 $ wget -q -O- https://github.com/hugsy/gef/raw/master/gef.sh | sh

Then just start playing (for local files):

$ gdb -q /path/to/my/bin
gef> gef help

Or (for remote debugging)

remote:~ $ gdbserver 0.0.0.0:1234 /path/to/file 

And

local:~ $ gdb -q
gef> gef-remote your.ip.address:1234

Show me

x86

ARM

PowerPC

MIPS

Dependencies

There are none: GEF works out of the box! However, to enjoy all the coolest features, it is recommended to install:

• capstone highly recommended
• ROPgadget highly recommended

Note : if you are using GDB with Python3 support, you cannot use ROPgadget as Python3 support has not implemented yet. Capstone and radare2-python will work just fine.
Another note : Capstone is packaged for Python 2 and 3 with pip . So a quick install is

$ pip2 install capstone    # for Python2.x
$ pip3 install capstone    # for Python3.x

And same goes for ropgadget



$ pip[23] install ropgadget

The assemble command relies on the binary rasm2 provided by radare2 .

Download GEF

Script for searching the extracted firmware file system for goodies! - Firmwalker

A simple bash script for searching the extracted or mounted firmware file system.

It will search through the extracted or mounted firmware file system for things of interest such as:

• etc/shadow and etc/passwd
• list out the etc/ssl directory
• search for SSL related files such as .pem, .crt, etc.
• search for configuration files
• look for script files
• search for other .bin files
• look for keywords such as admin, password, remote, etc.
• search for common web servers used on IoT devices
• search for common binaries such as ssh, tftp, dropbear, etc.
• search for URLs, email addresses and IP addresses
• NOTE: Some of the data written to the file may be quite verbose. It that case, the data can be reviewed and then deleted if desired from the file.

Usage

• If you wish to use the static code analysis portion of the script, please install eslint: npm i -g eslint
• ./firmwalker {path to root file system} {path for firmwalker.txt}
• Example: ./firmwalker linksys/fmk/rootfs ../firmwalker.txt
• A file firmwalker.txt will be created in the same directory as the script file unless you specify a different filename as the second argument
• Do not put the firmwalker.sh file inside the directory to be searched, this will cause the script to search itself and the file it is creating
• chmod 0700 firmwalker.sh

How to extend

• Have a look under 'data' where the checks live or add eslint rules - http://eslint.org/docs/rules/ to eslintrc.json

Script created by Craig Smith and expanded by:

• Athanasios Kostopoulos
• misterch0c

Links

• https://craigsmith.net
• https://woktime.wordpress.com
• https://www.owasp.org/index.php/OWASP_Internet_of_Things_Project#tab=Firmware_Analysis

Download Firmwalker