Why Log Monitoring and Management is Critical to IT Security

Why Log Monitoring and Management is Critical to IT Security

In our last blog post titled “The Fundamentals of Network Security Analysis and Monitoring” we discussed active versus passive security devices, flow analysis versus packet analysis, and several of the security technologies used to analyze or stop traffic. Prior to that blog post, we wrote another post titled “Network Security Design is Critical to Eliminating Security Gaps and Reducing Costs” – “The 5 Pieces to the Cybersecurity Puzzle,” where we discussed the five core elements of Secure Network Design

In this blog post, we would like to take a step further and dive into what logs are, how they can be analyzed for security purposes and how log management programs are typically handled.

 

What is a Log and How is it Used to Improve IT Security

First, very simply, a log is a record of the events occurring within an organization’s computer systems, servers, and networks. Logs are composed of log entries – each entry contains detailed information related to a specific event that has occurred within an application, system or network such as an error, log-in details, IP addresses accessing the system, system resources used and much more depending on the type of log. Many logs within an organization contain information or records specifically for IT security, though I’m sure you knew that.

Second, these computer security logs are generated by many sources, including security software, such as antivirus software (AV), firewalls (FW or NGFW), and intrusion detection (IDS) and prevention systems (IPS); operating systems on servers, workstations, and networking equipment like routers; and applications. In order to detect security issues automatically, security analysts set up monitors on the generated logs. The log monitors scan the log files and search for known text patterns and rules that indicate significant security events such as blacklisted application use, IP addresses from suspect geographies, or unusual data flow.

Several Examples of Software Security Log Entries

 

 

Once an event is detected by the monitoring software or system, the monitoring system will send an alert, either to a person or to another software/hardware system. Monitoring logs help to identify security events that occurred or might occur.

Third, almost countless logs are generated by a variety of devices and technologies which is what makes log monitoring and management so complex and difficult. However, with SIEM’s or Security Information and Event Management systems gigabytes of logs can be ingested, normalized and analyzed. SIEM’s apply correlation rules to find anomalies or uncover suspicious behavior through logs.

Devices that Generate Logs and those that Don’t

 

 

 

The 3 Elements to a Log Management Infrastructure

While most organizations cannot have identical log infrastructures throughout the entire IT infrastructure, the following 3 elements are typical in most organizations:

• Log Generation – Some hosts run logging client applications or services that make their log data available through networks to log servers in the Log Analysis and Storage stage. Other hosts make their logs available through allowing the servers to authenticate to them and retrieve copies of the log files.
• Log Analysis and Storage – This second stage is composed of one or more log servers that receive log data or copies of log data from the hosts. The data is transferred to the servers – servers that receive log data from multiple log generators are sometimes called collectors or aggregators.
• Log Monitoring – These are consoles that may be used to monitor and review log data and the results of automated analysis. Log monitoring consoles can also be used to generate reports and much more…these are SIEM’s in many cases.

 

How Log Management & Leveraging SIEM Technology Help Identify Attacks

Let’s start by clarifying several terms that we often hear in the security industry.

• Log – raw data stored by a computer system.
• Event – something that occurs somewhere on a network or computer system, can be extracted from log data.
• Incidents – events that are identified as possible security breaches. This can include unauthorized access to data or IT systems, violation of security policies, etc.

Log monitoring is the act of reviewing collected logs as they are recorded. This typically involves the assistance of log management software. Log management software can be configured to listen for specific application-related events and alert the proper people within a development organization when such an event occurs, among other benefits.

Effective log management is essential to an organization’s security. Monitoring, documenting and analyzing system events is a crucial component of IT security. Log management software or SIEM’s automate many of the processes involved. A SIEM as we suggested earlier a combination of:

• SIM – Security information management – Long-term storage as well as analysis and reporting of log data.
• SEM – Security event manager – Real-time monitoring, correlation of events, notifications and console views.

Thus, a SIEM combines SIM and SEM and provides real-time analysis of security alerts generated by network hardware and applications. They employ or handle the following:

• Data aggregation: SIEM’s aggregate data from many sources, including network, security, servers, databases, applications. They provide the ability to consolidate monitored data so that analysts can monitor and analyze data in a “single pane of glass.”
• Correlation: SIEM’s search for common attributes, and link security events together in order to make sense of the information. The technology provides the ability to perform a variety of correlation techniques to integrate different sources.
• Alerting: SIEM’s are configured to alert analysts when events such as intrusion detection, access control or compliance controls are triggered. SIEM’s are far from perfect and are often blamed for being inaccurate and causing “alert fatigue” from analysts.
• Dashboards: SIEM’s provide dashboards so that data can be charted, and patterns of the data more easily identified.
• Retention: SIEM’s can employ the storage of historical data to facilitate the correlation of data over time, and to provide the retention necessary for compliance requirements.
• Forensic analysis: Because SIEM’s can retain data, and because breaches are not normally detected in real-time, the archived logs are critical to forensic investigations to conduct the post-mortem of a breach and understand its root cause.

 

Conclusion

Log monitoring and management is a deep subject so limiting this post to the capabilities of an SIEM probably makes sense. Because of the number of logs that are generated today and the number of reasons to analyze those logs including security, compliance, application issues and so much more we will likely address SIEM’s specifically in a future blog post.

As a final note, from a security standpoint, log monitoring and management are critical because security teams can identify failed log-ins, escalation of privileges, traffic anomalies, lateral movement and much more. We hope we helped clarify what a log is, the information it contains, how this information is valuable to the security team and how analyzing and managing logs can provide insight for IT security team to defend the organization more effectively.

As always, we at SecureOps are here when you need us. We provide SIEM-as-a-Service or Managed SIEM solutions so we are pretty familiar with Log Rhythm, Splunk, Alien Vault and many other SIEM providers. You can reach us at 1 (888) 982-0678 for any questions.

 

For More Information about our Managed SIEM or SIEM-as-a-Service Information

https://secureops.com/device-management-portfolio/siem-as-a-service/