LogRhythm Precision Search: An Unstructured Journey

According to Wikipedia, unstructured data (or unstructured information) refers to information that either does not have a pre-defined data model or is not organized in a pre-defined manner.

Unstructured information is typically text-heavy, but it may also contain data such as dates, numbers and facts. This results in irregularities and ambiguities that make it difficult to understand using traditional programs, as compared to data stored in field form in databases or annotated (semantically tagged) in documents.

Finding this data that contains so much useful (and sometimes vital) metadata is now becoming the norm. Working for a large enterprise or a SMB can have very similar challenges when it comes to data mining.

Most days, a security team will be cleaning out the ticket queue and responding to incidents. Some days are lighter than others, but for the most part, an analyst will need to prioritize the tickets and respond to only the most urgent incidents.

This is where my experience in the network operations center (NOC) comes in handy—as I was the guy who had to divide the workload as evenly as I could with my team of four. Back in 2008, I did not have a sweet tool like LogRhythm to help with managing the day-to-day activities in the NOC (the acronym SOC—security operations center—was not popular back then). My team was pushing for security and network operations team, but the acronym “SNOT” did not stick!

In 2008, we were doing networking and security all from one place. Our single pane of glass was a 32-inch TV with manual shifting between applications as we looked for needles in the haystack. We still used Cisco PIX and monitored with Nagios, Cacti and other software I can’t recall.

Fast forward to 2015 and LogRhythm 7. Knowing I work for LogRhythm and I love it here, I had to write this from the perspective of an engineer who has been thrust into a new tool. That clarified, let’s discuss how precision search will quicken your investigation and make your SIEM more useful.

Suppose you need to find suspicious behavior and all you have is a name or Mac address. Someone may just be streaming content. But it can be a headache in SIEM 1.0 to find the information you may need. In the figure below, you will see how easy and useful LogRhythm 7 is.

Back to our scenario: You are searching for network traffic on a Mac address—and you know only that this individual is streaming on some service.

Figure 1. Search for Network Traffic on a Mac Address

Figure 2. Precision Search Results from Mac Address and Network Traffic

The results are quick. You receive a ton of data that will point you in the direction you need to get started—focused on the Mac address and network traffic-enabled precision search to gather any data on the raw logs that had the parameters you set to begin with.

Along with the information you needed, you now have an IP address, URL and a multitude of other data that will allow you to create a case on something for which you had no direction on to begin with.

In the next scenario, suppose you search for “Boulder” in any of the fields in your raw log. Most any unstructured entry can be used to bring back results to point you on a path that will lead to better adoption and day-to-day functionality that the market is missing. You don’t have to be a coding wizard or a scripting savant—you just have to have a small idea of what you need and then search on it.

Figure 3. Search on Boulder

LogRhythm 7 can take your unstructured nightmare and put some real actionable data behind your “hunch.”

Figure 4. Unstructured Search Results

Figure 5. Unstructured Search Details

There you have it: A quick tutorial on using LogRhythm 7 and our precision search functionality with an unstructured, keyword-only example.