SIEM stands for Security Information and Event Management, which as the name implies collects security information and manages those event. The main purpose of a SIEM is to centralize logs which are the records of events, and make sense of the collected data by relating the data with one another and presenting them.

Why do we even need to centralize the logs?

To understand the importance of logs, let's first define what they are. Logs are records of events within a system, capturing everything from system errors and user activities to application successes, failures, and process changes. Logs serve as invaluable records of past events, providing essential insights to address these questions. By preserving an archive of historical activities, we can bolster our security posture and protect our digital assets more effectively.

Every system generates a massive amount of logs due to its numerous applications and activities. This is true not only for operating systems like Windows and Linux but also for network devices such as routers, switches, and firewalls. Given the volume of data, it can be challenging for security analysts, engineers, or administrators to grasp what’s happening across the entire organizational infrastructure. Centralizing and organizing logs is critical for effectively investigating security breaches, as it provides a comprehensive view of the attack and its impact.

Another key reason for centralizing logs is to automate data collection and ensure timely, secure transmission. Managing logs from multiple sources can be overwhelming. Centralizing them streamlines administration, maintains consistency, and simplifies overall management across the organization.

Why is it Called SIEM?

SIEM stands for Security Information and Event Management, and it’s a perfect name for what it does in cybersecurity. Let’s break down why each part of the name makes sense.

Security: At its heart, SIEM is all about safeguarding your IT infrastructure. It’s designed to protect data, networks, and systems from cyber threats and unauthorized access.
Information: SIEM systems handle vast amounts of data, including logs and events from various sources. The "information" part underscores their role in collecting and managing diverse security-related data.
Event: Events refer to any observable occurrences within your IT environment, such as login attempts or file modifications. SIEMs monitor these events to detect potential threats and vulnerabilities.
Management: Beyond just collecting data, SIEMs manage it by analyzing, correlating, and organizing it. This management capability helps in generating alerts, producing reports, and coordinating incident response.

In summary, SIEM reflects a system designed to offer comprehensive oversight and protection by focusing on security, managing information, monitoring events, and overseeing the entire process.

How does a SIEM work?

A SIEM system works through these processes:

Log Collection: The Foundation of SIEM

The journey begins with log collection, the first step in any SIEM system. Logs are essentially records of events and activities from various sources like servers, network devices, and security appliances. There are two main methods for gathering these logs:
- Agent Based:
  - In order to implement this method, a log agent software is required. Agents often have parsing, log rotation, buffering, log integrity, encryption, conversion features. In other words, this agent software can take action on the logs it collects before forwarding them to the target.
  - The agent may be proprietary to the SIEM product like Splunk: universal forwarder and ArcSight: ArcSight Connectors or open source agents like Beats and NXlog.
  - The main disadvantage of this method is that As the additional features are activated, resource consumption increases. That requires the system's resources such as CPU, RAM to be increased, so the cost increases.
- Agentless:
  - Here, logs are gathered using network protocols or API connections, without the need for additional software. Agentless log sending process is sometimes preferred as there is no installation and update cost.
  - Easier to prepare and manage than the agent method. However, it has limited capabilities and potential security risks due to reliance on network protocols.
  - Common Agentless Log Collection Method:
    - Syslog
    - WMI
    - Windows Event Forwarding (WEF)
    - NFS
    - SSH
    - Cloud Integration
    - API based

This doesn't mean that the log can be collected only through agents or agentless method, it can also be manually collected, however it can be quite troublesome and repetitive to do so if we are handling multiple servers and nodes.

Data Aggregation: Collecting and Preparing Logs

Once collected, logs are sent to a log aggregator. Think of the aggregator as a central hub where all logs converge. We can edit the logs coming here before sending them to the destination and as such we can filter, modify and enrich them.
Data Processing and Enrichment: Making Data Useful

We can edit the logs coming in the log aggregator before sending them to the destination and as such we can filter, modify and enrich them.
- Data Processing:
  - Parsing: Extracting structured information from raw log data. Example: A raw firewall log might be parsed to identify fields like timestamp, source IP, destination IP, and action (allow/deny).
  - Modification: Adjusting or refining the parsed data to fit standardized formats. Example: IP addresses might be standardized into a uniform format, and timestamps might be converted to a consistent time zone.
  - Normalization: Converting data into a consistent format across different log sources to reduce redundancy or duplication of data and for faster data access. Example: Different devices might use different formats for logging failed login attempts, but normalization converts them into a uniform format.
- Data Enrichment:
  - Contextual Enrichment: Adding additional context to the processed data, such as threat intelligence or geographic information, to enhance the ability to detect and understand security threats. Example: An IP address from a log is enriched with information about whether it is associated with known malicious activity from threat intelligence feeds.

This step is critical because it ensures that the raw data collected from various sources is converted into a usable format and enhanced with relevant contextual information. This processed and enriched data then feeds into the SIEM's correlation engine, enabling more accurate detection of potential security threats and more effective response.

Data Storage: Keeping Data Accessible

With logs aggregated, they need to be stored efficiently. SIEM systems use various storage solutions, such as databases or cloud storage.

One of the common mistakes made in SIEM structures is to focus on storage size. High-sized storage is important, as well as the speed of accessing this data. For example, let’s say we collect all the logs such as WAF, Firewall, Proxy, etc. and imagine that it takes 15 minutes to make a search in these logs. In a situation where it is so difficult to access data, the studies will not be very productive. For this reason, the speed of data access should also be considered in storage.

SIEM tools store log data in a centralized repository for extended periods which helps with forensic investigations, historical analysis, and compliance requirements.
Correlation and Analysis: Spotting Threats

The correlation engine analyzes normalized data, applying rules and correlation algorithms to identify patterns or anomalies that indicate potential security threats. Rules are simply a predefined criteira as to how the event is identified or handled, they are used in firewalls, IDS,IPS,and other security tool to handle such datas.
Alert Generation: Notifying the team

Based on the analysis, alerts are generated and prioritized according to their severity and potential impact.
Incident Management: Taking Aciton

Alerts are reviewed by security analysts, who investigate and determine whether they represent genuine security incidents. Appropriate response actions are taken, which may include automated responses, manual intervention, or further investigation.

Enhancing SIEM with Key Integrations and Tools

To boost the effectiveness of your Security Information and Event Management (SIEM) system, consider integrating these essential tools:

Case Management Systems: These tools help organize and manage security incidents by streamlining ticket creation, tracking, and documentation. They ensure systematic handling and efficient resolution of security events.
Threat Intelligence Platforms: They provide external data on threats and vulnerabilities, enriching SIEM alerts with contextual information. This improves threat understanding and helps in making informed responses.
Endpoint Detection and Response (EDR): EDR tools offer visibility into endpoint activities, detecting suspicious behavior and supporting forensic analysis. This enhances the SIEM’s ability to monitor and respond to threats targeting devices.
Network Security Tools: These tools analyze network traffic to detect anomalies and potential intrusions. Integrating them provides better visibility into network activities and helps prevent network-based threats.
Automation and Orchestration (SOAR): SOAR platforms automate security tasks and coordinate responses, reducing manual effort and speeding up incident management through streamlined workflows.

Others

Additional integrations can further enhance SIEM capabilities, including:

Vulnerability Management Tools: Identify and prioritize security weaknesses.
File Integrity Monitoring (FIM): Track file and directory changes to detect unauthorized modifications.
User and Entity Behavior Analytics (UEBA): Detect anomalies based on user behavior.
Cloud Security Tools: Monitor and manage cloud-based assets.
Data Loss Prevention (DLP): Protect sensitive data from unauthorized access.
Business Intelligence (BI) Tools: Offer detailed reporting and data analysis.

Integrating these tools with your SIEM system enhances visibility, improves response capabilities, and streamlines security operations.

Most Common SIEM Products

There are two main types of SIEM products available, and they are categorized based on these:

Open Source SIEM:
- ELK Stack (Elastic Search, Logstash, Kibana): Not a SIEM by default but can be widely used for log management and analysis, often extended with Beats for data collection.
- OSSEC: An open-source host-based intrusion detection system (HIDS) that can be used as a component in a SIEM setup.
- Wazuh: An open-source fork of OSSEC, providing enhanced security monitoring and log analysis.
- Graylog: Provides powerful log management and analysis with an intuitive interface and real-time search capabilities.
- Security Onion: A comprehensive Linux distribution that includes various tools for intrusion detection, network monitoring, and log management.
Propiertary SIEM:
- LogRhythm: Provides a unified platform with advanced analytics and automation features for threat detection and response.
- LogPoint: Used for its comprehensive and user-friendly SIEM capabilities that provide advanced threat detection, analytics, and compliance management across diverse IT environments.
- Splunk Enterprise Security: Known for its powerful analytics, flexibility, and ease of integration with various data sources.
- IBM QRadar: Offers advanced threat detection and security intelligence with robust integration capabilities.
- Arcsight: A long-standing player in the SIEM market, known for its scalability and comprehensive feature set.
- Microsoft Sentinel: A cloud-native SIEM solution that integrates seamlessly with Microsoft Azure and other cloud services.

Companies pick between open-source and proprietary SIEMs based on what they need. Open-source SIEMs are generally cheaper and more customizable but may need more management. Proprietary SIEMs often have advanced features, are easier to use, and come with strong support, making them simpler to integrate.

Sometimes, companies use different SIEMs for different clients to meet specific needs and compliance requirements. This ensures each client gets a solution that suits their particular challenges and budget.

Conclusion

SIEM (Security Information and Event Management) systems play a crucial role in safeguarding IT infrastructure by centralizing and analyzing security logs from various sources. These tools streamline data management through collection, processing, normalization, and enrichment of logs, which are then stored for in-depth analysis. SIEM solutions generate alerts for potential threats and assist in incident management, making them essential for proactive security. Whether opting for open-source or proprietary solutions, each type offers unique benefits. Ultimately, centralizing logs is vital for achieving comprehensive security oversight and effective threat detection.

References

https://tryhackme.com/r/room/introtologs

https://logrhythm.com/blog/what-is-siem/

https://www.splunk.com/en_us/blog/learn/siem-security-information-event-management.html

My Understanding of How a SIEM Works

Table of contents