Using LogRhythm to Streamline Operations
While the LogRhythm platform is typically used as a security solution, it is a flexible and powerful tool that can be used for operational use cases as well.
LogRhythm’s ability to process a wide variety of log sources (flat files, syslog, UDLA, Windows Event Logs, etc.), combined with our powerful structured and unstructured search capabilities, make it easy to ingest log data from a multitude of log sources and make that log data highly searchable. These features, alongside our AI Engine and SmartResponse remediation capabilities, create a powerful tool that can be used for more than security intelligence.
LogRhythm in Action
Let’s say that you are looking to empower your development team to quickly and easily search across multiple log files to find common errors within a custom Java application running on a Linux server.
In this use case, the particular application utilizes Log4j for logging, and writes its log output to the /usr/home/log4j directory. Below you’ll find a sample log message. In the image, you’ll notice that I’ve highlighted the bit of data that our developers would like to be able to search for. (In this example, the developer is looking for a particular user ID.)
Figure 1. Raw Log Message
The Set Up
The first step is to get LogRhythm collecting the application log data. Utilizing LogRhythm’s flat file collection methodology, it’s straight forward to configure LogRhythm to collect all log4j log files in the /usr/home/log4j directory. Additionally, you have the ability to tell LogRhythm a bit about how to interpret date data in the log message.
Figure 2. Log Message Source Properties
Once you have the flat file configuration set up, LogRhythm will begin to collect logs and index them into our Elasticsearch persistence layer. This will allow for fast and efficient searching of that log data without first having to create parsing rules to bring the data in and make use of it.
Searching and Results
Now that the log data has been collected and indexed, your developer can run a quick search from LogRhythm’s WebUI against all log4j Application logs looking for the “UserID:12345678” string of data.
Figure 3. LogRhythm Search
The search results indicate multiple occurrences of this UserID across three separate log messages. Don’t worry—we can view the full log message right from the WebUI.
Figure 4. Viewing Logs in the LogRhythm WebUI
Going Farther
To further normalize metadata from this log source, you could easily create a custom message processing policy (utilizing regular expressions). For our example, it may be helpful to parse out the event type, the remote IP address, the user agent string, the action, the user name, and the org ID so that we could run further logic against those fields.
Once you’ve further normalized your metadata, your possibilities are endless when it comes to optimizing your operations. Here are a couple more cool things you could do with your enhanced visibility:
- You could write alarming rules based on your metadata—for example, alerting on a known error ID—that could alert development staff to a code issue.
- You could correlate this custom application log data with other system log data and alert on it. For example, was there a configuration change on a server followed by a 2x increase in errors from our custom application log?
- Utilizing our SmartResponse scripting capabilities, you could even take action on known issues. For example, if we saw a configuration change followed by a 2x increase in errors, we could run a script that would (with the proper approval) return the configuration back to a known good state.
This use case is just one example showcasing LogRhythm’s ability to quickly and easily ingest log data from a custom log source. You could repeat the steps above for any custom log utilizing any of our log collection methodologies just as easily.
LogRhythm’s ability to quickly gather log data, combined with our powerful search capabilities, really make centralized logging of your custom application log data easy to implement.