Gitgot - Semi-Automated, Feedback-Driven Tool To Chop-Chop Search Through Troves Of Populace Information On Github For Sensitive Secrets

GitGot is a semi-automated, feedback-driven tool to empower users to apace search through troves of populace information on GitHub for sensitive secrets.

How it Works
During search sessions, users volition render feedback to GitGot close search results to ignore, as well as GitGot prunes the educate of results. Users tin dismiss blacklist files past times filename, repository name, username, or a fuzzy jibe of the file contents.
Blacklists generated from previous sessions tin dismiss hold upward saved as well as reused against like queries (e.g., example.com v.s. subdomain.example.com v.s. Example Org). Sessions tin dismiss also hold upward paused as well as resumed at whatsoever time.
Read to a greater extent than close the semi-automated, human-in-the-loop pattern here: https://know.bishopfox.com/blog/going-semi-automated-in-an-automated-world-using-human-in-the-loop-workflows-to-improve-our-security-tools

Install Instructions
[1] Install the ssdeep dependency for fuzzy hashing.
Ubuntu/Debian (or equivalent for your distro):

apt-get install libfuzzy-dev ssdeep

or, for Mac OSX:

brew install ssdeep

For Windows or *nix distributions without the ssdeep package, delight encounter the ssdeep installation instructions.
[2] After installing ssdeep, install the Python dependencies using pip:

pip3 install -r requirements.txt

Usage
GitHub requires a token for rate-limiting purposes. Create a GitHub API token amongst no permissions/no scope. This volition hold upward equivalent to populace GitHub access, simply it volition permit access to utilization the GitHub Search API. Set this token at the come about of gitgot.py every bit shown below:

ACCESS_TOKEN = ""

After adding the token, you lot are cook to go:

# Query for the string "example.com" using the default RegEx listing as well as logfile place (/logs/.log) ./gitgot.py -q example.com  # Using GitHub advanced search syntax ./gitgot.py -q "org:github cats"  # Custom RegEx List as well as custom log files place ./gitgot.py -q example.com -f checks/default.list -o example1.log  # Recovery from existing session ./gitgot.py -q example.com -r example.com.state  # Using an existing session (w/blacklists) for a novel enquiry ./gitgot.py -q "Example Org" -r example.com.state

Query Syntax
GitGot queries are fed conduct into the GitHub code search API, as well as hence banking concern check out GitHub's documentation for to a greater extent than advanced enquiry syntax.

UI Commands

• Ignore like [c]ontent: Blacklists a fuzzy hash of the file contents to ignore hereafter results that are like to the selected file
• Ignore [r]epo/[u]ser/[f]ilename: Ignores hereafter results past times blacklisting selected strings
• Search [/(mykeyword)]: Provides a custom regex facial expression amongst a capture grouping to searches on-the-fly (e.g., /(secretToken))
• [a]dd to Log: Add RegEx matches to log file, including all on-the-fly search results from search command
• Next[], [b]ack: Advances through search results, or returns to previous results
• [s]ave state: Saves the blacklists as well as progress inwards the search results from the session
• [q]uit: Quit

Download GitGot