OSINT: Practical Exploration of Public Data Gathering Tools

Aryan SharmaAryan Sharma
5 min read

Introduction

In today’s digital age, knowing how to gather and analyze publicly available information has become a critical skill — not just for cybersecurity professionals but for anyone looking to understand the digital footprint of individuals, organizations, and even trending online phenomena. Recently, I immersed myself in the world of OSINT (Open Source Intelligence), employing powerful tools and methods to conduct passive reconnaissance. In this post, I’ll share my active learning experience using some popular OSINT tools: OSINT Framework, WhatsMyName, SpiderFoot, and Recon-ng.

What is OSINT?

OSINT refers to the process of collecting information from sources available to the public. This includes everything from social media profiles to public records and beyond. The idea is simple: by collecting data that is openly available, you can build a comprehensive view of your target — be it for cybersecurity assessments, competitive intelligence, or even academic research.

OSINT Tools: How I Use Them

1. OSINT Framework

Overview:
The OSINT Framework is a web-based directory that organizes numerous tools into clear, intuitive categories. While many tools within the framework help dive into specific aspects of OSINT, one area I found particularly useful was tracking usernames across various platforms.

How I Use It:

  • Step 1: Visit osintframework.com and navigate to the “Username” category.

  • Step 2: Identify and select tools like WhatsMyName for deep dive searches.

    Fig. 2: OSINT Framework Interface

    This resource acts as a launchpad for pinpointing which tools to use for different recon needs.

2. WhatsMyName

Overview:
WhatsMyName is an excellent tool designed to check where a username appears across the internet. It’s especially valuable when trying to build a digital profile for an individual or organization.

How I Use It:

  • Search Process: I enter the username (for example, “ishowspeed”) into the search box.

  • Review: The tool returns a list of platforms where the username exists, helping reveal an online pattern or digital identity.

  • Documentation: I then export the data into a CSV or PDF, which assists in creating detailed recon reports.

Fig. 3: WhatsMyName Web Interface showing result for the search "ishowspeed".

3. SpiderFoot

Overview:

SpiderFoot is one of the most comprehensive automated OSINT scanners available. Running on Kali Linux, it taps into more than a thousand different public information sources, providing rich data such as domain details, email addresses, IP addresses, and much more.

How I Use It:

Getting Started:

Launch SpiderFoot:

  • Open a terminal in Kali Linux.

  • Run the command: spiderfoot -l 127.0.0.1:5001

Access the GUI:

  • Open your web browser and navigate to http://127.0.0.1:5001 to interact with SpiderFoot’s graphical interface.

Fig. 4: Interface of SpiderFoot

Running a Scan:

  • Choose a Target: You can initiate a scan by entering a domain name (e.g., “techhive.com”) or an email address.

  • Select a Mode: SpiderFoot offers several scan profiles:
    All: Retrieves every available piece of data (this might take longer).
    Footprint: Focuses on mapping the target’s network perimeter and associated identities.
    Investigate/Passive: Ideal for when you want to remain discreet.

  • Explore Modules: In the SpiderFoot interface, navigate to the Settings tab to review available modules. Each module (prefixed with sfp_) is tuned to gather specific types of information. For example:

  • sfp_accounts gathers data on possible user accounts.

  • sfp_emailcrawlr pulls public email addresses related to a target.

    Fig. 5: Scan result on an target

Real-World Example:
During one scan, by entering a domain as my target, SpiderFoot uncovered additional IP addresses and subdomains that I had not initially known. This kind of comprehensive data gathering is invaluable for a robust recon report.

Fig. 6: Detailed view can be viewed by clicking on the bar graph.

4. Recon-ng

Overview:
Recon-ng is a powerful, modular, command-line OSINT tool that allows you to integrate multiple types of data collection into one cohesive framework. Unlike SpiderFoot — which focuses on automation and a GUI-based approach — Recon-ng provides flexibility through its extensive set of modules.

How I Use It:

  • Starting Recon-ng: recon-ng
    Prompt changes to [recon-ng][default]. Use help to see available commands.

  • Creating and Managing Workspaces:

workspaces create test
workspaces list          # To display available workspaces
workspaces remove test   # To delete a workspace
workspaces select default
  • Exiting a Workspace: Just type: back

  • Exploring Modules:
    Viewing installed modules: modules search
    Searching the marketplace: marketplace search shodan
    Look for the D (Dependencies) and K (API Keys) columns

    Fig. 7: Creating workspaces in Recon-ng

5. Practical Steps and Learning Outcome

Step-by-Step Process Summary:

  1. Start with OSINT Framework:
    Identify useful tools, beginning with a focus on username tracking.

  2. Explore WhatsMyName:
    Input various usernames to see the breadth of their online presence. Save and document the findings.

  3. Deploy SpiderFoot:
    Launch SpiderFoot, run a target scan, and dive into the data provided. Analyze settings and module outputs.

  4. Leverage Recon-ng:
    Utilize this tool for tailored recon tasks and data correlation to further enrich your investigations.

What I Learned:

  • Tool Diversity: Each tool has its strengths — from broad automated scanning with SpiderFoot to precise, command-line-based investigations with Recon-ng.

  • Passive Recon: The importance of collecting data without alerting the target, thereby preserving stealth and integrity during an investigation.

  • Data Correlation: How combining information from different sources can reveal a more comprehensive digital profile.

Conclusion

My journey through OSINT tools has been a deep dive into digital detective work. By experimenting with platforms like WhatsMyName, SpiderFoot, and Recon-ng, I’ve witnessed firsthand how publicly available data can be transformed into actionable intelligence. The skills acquired here are not only critical for ethical hacking and cybersecurity assessments but also empower anyone with the curiosity to explore the vast online world in a responsible and informed way.

Feel free to reach out if you have any questions or tips on OSINT tools. Stay curious, and happy recon!

0
Subscribe to my newsletter

Read articles from Aryan Sharma directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Aryan Sharma
Aryan Sharma