Network Traffic Monitor: My Journey in Real-Time Traffic Monitoring

Introduction: The Problem

As a cybersecurity enthusiast and network administrator, I often found myself wondering what was happening on my network in real-time. I wanted to see which devices were consuming the most bandwidth, identify unusual traffic patterns, and have a way to monitor my network's health without going deep into cumbersome tools like Wireshark for every detail. Although Wireshark is powerful, it can be overkill when you just need a high-level view of your network traffic.

The problem was clear: I needed a lightweight, real-time network traffic monitor that could give me insights into my network's data usage without bogging me down in complex setups. The goal was to provide a user-friendly, web-based interface that anyone with a little technical knowledge could easily run and monitor their network.

The Solution: Building a Network Traffic Monitor

Key Technologies I Chose

To create this tool, I decided to rely on a few key technologies that I was already familiar with, but that I knew would give me the flexibility and power I needed:

1. Flask: A lightweight Python web framework to build a user-friendly interface.

2. Pyshark: A Python wrapper for Wireshark, allowing me to capture live network traffic and analyze it in real-time.

3. Matplotlib & Seaborn: Python libraries for graphing and visualizing data, which I used to create dynamic visual insights into network traffic.

4. Threading: For real-time data capture, I needed a way to process the network traffic data in the background while serving the web interface without delays.

With these tools in hand, I set out to create the Network Traffic Monitor.

---

Key Features of My Solution

1. Live Traffic Monitoring

The first requirement was clear: I needed to capture live traffic from the network interface. I used Pyshark to capture packets on the selected network interface. Pyshark is incredibly powerful, as it builds on Wireshark's packet analysis engine, and yet provides a Pythonic interface.

For live monitoring, I created a method that runs continuously and captures network traffic in the background. The captured data includes information such as:

• Source IP

• Destination IP

• MAC addresses

• Packet sizes

This live traffic data was stored in a dictionary structure (data_usage) where each source IP was the key, and associated data like the total sent bytes and destination IPs were recorded.

Here’s how I captured live traffic:

def capture_live_traffic(interface):
    capture = pyshark.LiveCapture(interface=interface)
    for packet in capture.sniff_continuously():
        # Process packet details (source IP, destination IP, MAC address, etc.)

2. Top 10 Data Consumers Graph

Once the traffic data was collected, I needed a visual way to represent the top 10 data consumers on the network. This would allow me to quickly see which devices were hogging the most bandwidth. I used Matplotlib and Seaborn to create a bar chart that dynamically updates every few seconds.

I chose a calming color palette, dubbed Northern Lights, to make the graph easy on the eyes during long monitoring sessions.

Here’s how I generated the graph:

def generate_top_10_graph():
    fig, ax = plt.subplots(figsize=(12, 6))  # Create a large figure
    top_10_ips = sorted(data_usage.items(), key=lambda x: x[1]['total_sent'], reverse=True)[:10]
    palette = sns.color_palette(["#164773", "#0B2B40", "#1E5959", "#3B8C6E", "#89D99D"] * 2)
    sns.barplot(x=data, y=ips, ax=ax, palette=palette, legend=False)

The graph is converted to a base64-encoded image and displayed dynamically in the web interface, refreshing every 5 seconds.

3. Real-Time Traffic Data Table

While the graph provided a high-level overview, I needed a way to dig deeper into the traffic data. I created a real-time table that shows traffic details for each device on the network, including source IP, MAC address, destination IPs, and the total data sent.

For easier navigation, I implemented pagination, displaying 10 rows per page. Users can click "Next" or "Previous" to browse through all the traffic data being captured.

Here’s the Flask route that fetches the paginated data:

@app.route('/fetch_data')
def fetch_data():
    page = int(request.args.get('page', 1))
    paginated_data = paginate_data(page)
    table_data = [{
        'ip': ip,
        'mac': info['mac'],
        'dest_ip': list(info['destinations'].keys()),
        'total_sent': f"{info['total_sent'] / (1024 * 1024):.2f} MB"
    } for ip, info in paginated_data]
    return jsonify(table_data)

4. Log Rotation

Another crucial part of this tool was logging network activity. I wanted to log all captured traffic data in separate log files, but at the same time, I didn’t want these logs to consume all my disk space. To solve this, I implemented log rotation using Python’s RotatingFileHandler.

Logs are saved in 5MB files, and up to 5 backups are kept. Once the log reaches its limit, the old logs are rotated out, keeping disk usage under control.

handler = RotatingFileHandler(log_file, maxBytes=5 * 1024 * 1024, backupCount=5)

---

How to Set Up and Run the Tool

1. Prerequisites

Before you begin, ensure that the following are installed on your system:

• Python 3.10+

• Tshark (a requirement for Pyshark, the network traffic capture tool)

• Python libraries listed in requirements.txt

Install Tshark on Linux using:

sudo apt-get install tshark

2. Cloning and Installing Dependencies

Clone the repository and install the required dependencies:

git clone https://github.com/aaditya-cpu/NetworkLeech.git
cd NetworkLeech
pip install -r requirements.txt

3. Running the Application

Since network traffic capturing requires root access, run the Flask application with elevated permissions:

sudo python app.py

Once the server starts, access the web interface via:

http://127.0.0.1:5000

If you're running this on a network server and want to access it from another device, replace 127.0.0.1 with your server’s IP address.

---

Project Structure

Here’s the structure of my project:

.
├── app.py                   # Main Flask application
├── requirements.txt          # Python dependencies
├── static/
│   ├── css/
│   │   └── styles.css        # Custom CSS for styling the web interface
│   └── js/
│       └── main.js           # JavaScript for handling dynamic data updates
├── templates/
│   └── index.html            # HTML template for the web interface
├── logs/
│   └── network_traffic.log    # Traffic logs (created during runtime)

---

Challenges and Enhancements

Challenges I Faced

1. Real-Time Traffic Capture: Capturing real-time traffic and processing it efficiently without performance bottlenecks was a challenge. Using Python's threading helped me solve this by running the traffic capture process in the background while still serving the web interface.

2. Dynamic Graphs: Ensuring the graphs updated smoothly every 5 seconds without reloading the entire page required careful implementation of AJAX in the frontend.

3. Disk Space Usage: I needed to be cautious about logging, as network traffic can generate significant amounts of data. Using rotating logs mitigated this issue.

Future Enhancements

1. Multi-Interface Monitoring: Currently, the user can monitor one network interface at a time. Adding support for monitoring multiple interfaces simultaneously would be a great next step.

2. Security Features: Adding basic authentication to protect the web interface would ensure only authorized users can view the network traffic data.

3. Alerts and Notifications: A possible enhancement would be to send alerts (e.g., email or SMS) when certain devices exceed a predefined data usage threshold.

4. Detailed Data Export: Allow users to export the captured data in CSV, JSON, or other formats for offline analysis.

---

Conclusion

In creating this Network Traffic Monitor, I’ve solved my own problem of needing a lightweight, easy-to-use tool for monitoring network activity in real-time. By combining Python’s Flask, Pyshark, and Matplotlib, I’ve built a solution that’s not only functional but also visually intuitive.

This tool provides a solid foundation for network monitoring and can be easily extended or enhanced to fit more advanced use cases. Whether you’re managing a small home network or monitoring a larger corporate environment, this solution will give you the insights you need to keep your network running smoothly.

Check out my code @

https://github.com/aaditya-cpu/NetworkLeech

About me:
I am a tech enthusiast with a strong focus on cybersecurity, Python development, and real-time data analytics. I enjoy tackling complex problems and finding innovative solutions, especially in network security. My work revolves around building efficient, user-centric systems that are both practical and forward-thinking, pushing the boundaries of technology.