Top 10 Popular Open Source Intelligence (OSINT) Tools
%20Tools.jpg)
Top 10 Popular Open Source Intelligence (OSINT) Tools
Introduction
In this article, we will be discussing various OSINT tools that are available in the market. When we search the internet there are multiple pages of results that are presented. We just have a look at the first page and research and if we do not get what we are expecting, we stop right? But have you ever wondered what lies in those hundreds of pages of result? "Information”! Let's get this free information using various tools. Tools are important but not knowing the usage of a tool will leave the user helpless. Before digging into the tools let's have a fair idea of what OSINT is and what can be achieved out of it.
What is Open Source Intelligence?
OSINT stands for open source intelligence. The Internet is an ocean of data which is an advantage as well as a disadvantage.
Pros are that the internet is free and accessible to everyone unless restricted by an organization or law. The Internet has all the information readily available for anyone to access. Cons are that the information is available that can be misused by someone with a malicious intent. Collection and correlation of information using these tools are referred to as open source intelligence. Information can be in various forms like audio, video, image, text, file etc. Below is the bird's eye view of the data categories available on the internet:
- Social media websites like Twitter, Facebook etc. hold a lot of user data.
- Public facing web servers: Websites that hold information about various users and organizations.
- Newsletters and articles.
- Code repositories: Software and code repositories like Codechef, Github hold a lot of information but we only see what we are searching for.
Why do we need tools?
Getting to know that the information is available is one thing. Collection of the information is second and making an analysis or intelligence out of them is the third. The information can be gathered manually as well but that will take the time that can instead be used in the later stages. Tools can help us gather the data from hundreds of sites in minutes and thus easing the collection phase. Let us say that the task is to identify whether a username is present and if so, on which all social media websites. One way is to log in to all the social media websites (I bet you don't know all of them!) and then testing the username in that. Another way is to use an open source tool that is connected to various websites more than what we can remember and checks the usernames presence on all the websites at once. This is done just in seconds. Run multiple tools to gather all target related information that can be correlated and used later.
You may also like: Fundamentals of Website Security for Online Retailers
OSINT Tools
1. Maltego
Maltego is developed by Paterva and is used by security professionals and forensic investigators for collecting and analyzing open source intelligence. It can easily collect Information from various sources and use various transforms to generate graphical results. The transforms are inbuilt and can also be customized based on the requirement. Maltego is written in Java and comes pre-packaged in Kali Linux. To use Maltego, user registration is required, the registration is free. Once registered users can use this tool to create the digital footprint of the target on the internet.
2. Shodan
Google is the search engine for all but shodan is the search engine for hackers. Instead of presenting the result like other search engines it will show the result that will make more sense to a security professional. As a certified information security professional one of the important entity is digital asset and network. Shodan provides you a lot of information about the assets that have been connected to the network. The devices can vary from computers, laptops, webcams, traffic signals, and various IOT devices. This can help security analysts to identify the target and test it for various vulnerabilities, default settings or passwords, available ports, banners, and services etc.
You may also like: Brute Force Attacks: Prominent Tools to Tackle Such Attacks
3. Google Dorks
Google is one of the most commonly used search engine when it comes to finding stuff on the internet. For a single search, the results can be of various hundred pages sorted in order of relevance. The results vary from ads, websites, social media posts, images etc. Google Dorks can help a user to target the search or index the results in a better and more efficient way. Let us say that the user wants to search for the word usernames but only requires the results with PDF files and not websites. This is done as below:
Some of the other indexing options are:
- Inurl: search for a string in URL of the page.
- Intitle: To search the title for a keyword.
- Ext: To search for a particular extension.
- Intext: Search for a particular text in a page.
Sometimes it is also referred to as Google hacking.
4. The Harvester
A harvester is an excellent tool for getting email and domain related information. This one is pre-bundled in Kali and can be very useful in fetching information. Below is an example of the output when we try to search for emails for Microsoft in PGP server. You can explore more as per requirement.
E.g the harvester –d Microsoft.com –b pgp
5. Metagoofil
Metagoofil is written by Christian Martorella and is a command line tool that is used to gather metadata of public documents. The tool is pre-bundled in Kali Linux and has a lot of features searching for the document type on the target, local download, extraction of metadata and reporting the results. For example: Users can scan for a particular kind of documents on a particular domain. Metagoofil –d nmap.org –t pdf.
6. Recon-ng
Recon-ng is a great tool for target information collection. This is also pre-bundled in Kali. The power of this tool lies in the modular approach. For those who have used Metasploit will know the power of modular tools. Different modules can be used on the target to extract information as per need. Just add the domains in the workspace and use the modules. For starters, here is a sample of the tool helping you.
You may also like: Top 15 Prominent Wireless Hacking Tools to watch out for in 2018
7. Check Usernames
Social networking websites hold a lot of information but it will be really boring and time taking task if you need to check whether a particular username is present on any social media website. To get such information there is a website www.checkusernames.com. It will search for the presence of a particular username on more than 150 websites. The users can check for the presence of a target on a particular website so as to make the attack more targeted.
A more advanced version of the website is https://knowem.com which has a more wide database of more than 500 websites along with a few more services?
8. TinEye
Tineye is used to perform an image related search on the web. It has various products like tineye alert system, color search API, mobile engine etc. You can search if an image has been available online and where that image has appeared. Tineye uses neural networks, machine learning, and pattern recognition to get the results. It uses image matching, watermark identification, signature matching and various other parameters to match the image rather than keyword matching. The website offers API extensions and browser extensions as well. You can simply visit the image and right click on it to select search on tineye. Link: https://www.tineye.com
9. Searchcode
Searching for text is easy as compared to searching for a code snippet. Try searching for a code sample on google and you will be prompted with no results or irrelevant results. Search code offers you a feature to search for a line of code which could have been present in various code sharing websites like Github etc. Users can search for functions or methods, variables, operations, security flaws and anything that can constitute a code segment. Users can search for strings as simple as "a++" too complex methods. The search results can be further filtered basis a particular repository or language. Do consider a few things before you hit search.
10. Recorded Future
Recorded Future is an AI-based solution to trend prediction and big data analysis. It uses various AI algorithms and both structured and unstructured data to predict the future. The users can get past trends and future trends basis OSINT data.
The utility of tools within the organization
Using the information that is freely available is an important task for the organization (maybe not for all but mostly). Let's take a few examples where these tools and manual means come to our rescue and generate something that makes sense to the organization.
Another interesting read: Penetration Testing: Step-by-Step Guide, Stages, Methods and Application
CASE 1: Hiring an employee
Company X wants to hire some employees for a team that is handling the data which is sensitive to the organization. Usually, organizations do a background check for the employees before hiring them. A background check will include referring to various kinds of information to check the integrity of the information. An employee might say that he has passed from B University presenting the certificates as well. How do we ensure that the university is there or not? Various universities offer search systems online that can be checked for certificate validation. Police verification for criminal records and searching various job portals to get the information is also a form of information. Organisations share candidate feedback on the job portals as well just In case they have to blacklist someone. Will you be interested in hiring a candidate that is being blacklisted? Maybe yes, but this needs to be thought twice.
CASE 2: We are being attacked
Cybersecurity is booming because of increasing cyber threats. Let's consider a scenario where an organization is getting large traffic from unexpected countries. This raises a concern that this is a DOS attack. Referring to various databases that are freely available on the internet can help you to get a clear idea of the incident. Is the IP blacklisted? Is this traffic expected? Various details about the source can be gathered from the internet using OSINT techniques. A lot of free network-based tools are available on the internet that has been connected to various databases for this kind of information.
CASE 3: We are the ORIGINALS
Organisations that deal with content can refer the Open intelligence to identify the genuineness of the content.
- Universities can check for copied submissions of assignments and projects.
- Industries can check for copied codes.
- Track usage of copyright images and content.
- Identify plagiarised articles and literature.
Conclusion
In a nutshell, the article has tried to cover 2 aspects of OSINT. One is the technology bit of how OSINT can help. Second is the power of OSINT that can help us in our day to day tasks. With all that information freely available multiple actors can accomplish various tasks. A security professional can use the information for data protection, security testing, incident handling, threat detection etc. A threat actor, on the other hand, can gain information to perform phishing attacks, targeted information gathering, DDOS attacks and much more.
Here we have discussed only a few aspects of the OSINT model but there is more to it like Usernames, Emails, IP addresses, Domains, Media/videos, news, articles, Social networks, people search, telephone numbers, transport, maps, archives, metadata, search engines, dark web, digital currency, tools, malicious content, documentation etc. The list can go on and on, it's just the requirement that can help in selecting the right tools and techniques. Since this is all free, users can make their own list as well where they can find information. A few curated lists of such tools are already available on the internet, thanks to OSINT. Check them out on Github or other reports. Good luck, Happy hunting!!!
About The Author
Harpreet Passi is an Information Security enthusiast with a great experience in different areas of Information Security. Harpreet holds CEH v9 and many other online certifications in the cybersecurity domain. Standard Chartered Bank acknowledged him for outstanding performance and a leading payment solution firm rewarded him for finding vulnerabilities in their online and local services. He loves to write, meet new people and is always up for extempore, training sessions and pep talks.