03: Security Monitoring and Analysis¶
In this session we will cover:
- What is a Security Information and Event Management (SIEM) tool?
- What data sources may we use with a SIEM?
- Hands-on with Splunk and the "Boss of the SOC"
- Cyber Ranges
Security Information and Event Management¶
SIEM Tools¶
Security Information and Event Management (SIEM) is software that improves security awareness of an IT environment by combining security information management (SIM) and security event management (SEM). SIEM solutions enhance threat detection, compliance, and security incident management through the gathering and analysis of real-time and historical security event data and sources. SIEM main capabilities provide a broad range of log event collection and management, increasing the ability to analyze log events and other data across dissimilar sources, and operational capabilities including incident management, dashboards, and reporting. SIEM also offers data aggregation across the enterprise network and normalization of that data for further analysis. Additionally, SIEM helps enable security monitoring, user activity monitoring, and compliance.
Splunk¶
Splunk is a widely-used SIEM platform. It essentially provides the mechanism to ingest and search data at scale, meaning that it is well-suited for machine data such as machine-generated logging. The use case of Splunk actually extends beyond cyber security to various other big data analysis domains. Users can search data using the Splunk Query Language, a similar concept to that of Structured Query Language used for common databases.
Alternative SIEMs¶
Formerly known as ELK (elasticsearch, logstash, kibana) elastic is another popular SIEM platform. Others include greylog, LogPoint, and SecurityOnion.
... and even Jupyter SIEM integration!¶
Microsoft Threat Intelligence Center (MSTIC) - a prime example of where Jupyter is being used for security investigations - thanks to their MSTICpy and MSTICnb libraries. Notebooks are a fantastic way to extend capability of a SIEM and ensuring repeatability of playbooks.
Yet More Data Sources...¶
Suricata¶
Suricata is an open-source Intrusion Detection System (IDS). 'Suricata is the leading independent open source threat detection engine. By combining intrusion detection (IDS), intrusion prevention (IPS), network security monitoring (NSM) and PCAP processing, Suricata can quickly identify, stop, and assess the most sophisticated attacks'.
- Logging is captured in the
/var/log/suricata/eve.jsonfile (event file). - Useful guide for setting up Suricata
import pandas as pd
df = pd.read_json('./data/eve.json', lines=True)
print (df['event_type'].unique())
df[df['event_type']=='http'].head(5)