Using Data Visualizations to Prove the Return on Investment of Your Security Program

Senior Security Engineer Rob Sweeney is a guest blogger from Penn Medicine and a valued LogRhythm contributor. Rob’s presentation at LogRhythm’s third annual user conference, RhythmWorld, was so well received by our users that we asked him if he would share his insights with our blog readers — he graciously agreed.

One of the most difficult-to-answer questions management often asks related to security tools boils down to “What’s my return on investment?” or “What value has this tool added to our program?” Security is a unique industry that measures the effectiveness of your team and tools by the periods of time when nothing significantly bad happens.

But, how do security teams prove that money spent on security tools and staff helps secure their organization’s environment or adequately supports department or organizational objectives? And how can managers and directors justify the need for additional headcount or advanced tools if “nothing bad is happening?”

The answer to both challenges is metrics and the ability to present definitive trends in day-to-day operations over time. The LogRhythm NextGen SIEM Platform provides an excellent foundation for basic metrics in its Case Metrics dashboards and stock case reports, but if a security team is more mature, it will likely want to track and monitor more specific trends around alarms, cases, and analyst activity.

At Penn Medicine, we use Microsoft’s Power BI — a data visualization tool — to pull information from LogRhythm’s SQL database and create customized visualizations. Here are a few ways my security team uses visualizations to highlight focus areas for our internal teams, including security engineering, security operations, and our management.

Metrics to Test New Alarms

Our team members reached a point in its maturation where engineering could no longer create and enable alarms or reports handed off to them on an ad-hoc basis. We had to create a standardized and documented procedure to vet new alarms before they were put into “production.”

To streamline this process, we implemented a practice where we put new alarms into a “beta” status, and create draft procedures before production. The operations team reviews and triages the beta alarms and provides recommendations to adjust alarm criterion or for procedural changes. Next, the alarms are closed with specific reasons; generally, “Closed: False Alarm” or “Closed: Monitor” classifications. We use the closure reasons to generate a bar graph using a custom calendar feature in Power BI combined with the “_AlarmDetails”, “AIERule”, and “Alarm” tables from within the events management database (EMDB), a repository of all the events. See Figure 1 for a snapshot of this graph.

We include all alarms with “beta” from AIERule[Name], the alarm closure status from _AlarmDetails[Status], enter the last 30 days in the calendar, and a count of Alarm[AlarmID].

The graph shows that we had several alarms ready for “production” status. For us, this means that the alarm is ready to automatically create cases when it triggers and playbooks should be built, so the standardized and vetted procedures will automatically load into the case once it’s created.

Beta alarm for flase positive ratesFigure 1: Beta alarm false positive rates for the past 30 days

Visualization to Assess the Effectiveness of Your SOC

To show the effectiveness of a security solution — the LogRhythm NextGen SIEM feature set in particular — we visualize the time savings we get from automation. For example, we’ve been able to see how much time our team has saved when we leveraged the LogRhythm NextGen SIEM in conjunction with our Palo Alto firewalls to implement an auto-blocking capability.

Periodically, we undergo various “attacks of the month” where some adversary is continuously attempting to exploit some flavor of vulnerability for days or even weeks. Once we identify an ongoing pattern, we use LogRhythm’s SmartResponse™ Automation feature and our firewall’s API to simply block the attacker’s IP address anytime we detect the attack.

I was curious to see how effective this was, so I created another bar graph to show the number of hits we had for different types of vulnerabilities over 30 days. The bar graph in Figure 2 shows that our SmartResponse triggered over 1,000 times in 30 days. To build this visualization, we pull AIERule[Name] where those rules contained “autoblock”, select the last 30 days from our custom calendar, and Counts of Alarm[AlarmID].

There is a caveat to this; our firewalls are already configured to block this type of activity when it occurs. Our intention is not simply to stop that specific attack; we are looking to stop the attacker from being able to send us any traffic at all after they’ve demonstrated their intentions. Assuming it takes approximately two to five minutes for someone to manually update the firewall with the IP addresses, and our team sees approximately 1,000 of these events per month, we’re effectively saving 2,000–5,000 minutes or 33.3–83.3 hours per month. Turning that into dollars, assuming a salary between $55,000–$75,000 per year, that translates to approximately $27.5–$37.5 per hour. The savings from this single initiative ends up being somewhere between $900–$3,100 per month, or $10,80–$37,200 per year. Of course, this is all assuming that a team is taking the steps necessary to blacklist IP addresses attempting to actively compromise a network.

Dashboard of 30-day summary of SmartResponse AutomationFigure 2: 30-day summary of SmartResponse Automation and block activity

Visualization of the LogRhythm Deployment

The last metrics we’ve put into visualizations thus far provide an overview of our LogRhythm deployment. These visualizations are somewhat complex, so I won’t touch on exactly where this data is coming from within SQL, but I will explain the value of each.

The LogRhythm Deployment Overview visualization shown in Figure 3, provides a quick and easy way for our team to see the current status of the deployment. This includes new log sources and assets being onboarded, and the number of logs, events, alarms, cases, and incidents over a period selected in the date slider. There is also a small section on the right that shows trends for each of these items over the period selected.

LogRhythm Deployment Dashoboard in Power BIFigure 3: Overview of LogRhythm deployment in Power BI

Visualizing Alarm Metrics

For operational purposes, we also track overall alarm metrics that our management teams can review for desired periods using the calendar slider. We also use dynamic updates to quickly gauge if alarms are triggering with more frequency than we expect or if there are spikes in the amount of alarms being created, which is an indicator of potentially suspicious or malicious activity. This dashboard can also be used to show the number of alarms per day that are not triaged vs. the total number of alarms in a given period — a metric that managers could use to argue their case for additional headcount or other needed resources.

LogRhythm alarm metrics dashboard in Power BIFigure 4: LogRhythm alarm metrics visualization in Power BI

Dashboard for Case Management Metrics

Lastly, we use the below visualization for Case Management metrics. This displays the total number of cases, how many cases were escalated into incidents, and the average mean time to close cases within a chosen period. The primary graph in this visualization helps us see the number of cases created daily within a selected period, and it highlights cases that have remained open for several days. We also use this report to show open cases by priority, by analyst — this allows our management team to quickly evaluate the workload assigned to each analyst to determine if work needs to be redistributed.

LogRhythm Case Management metrics dashboard in Power BIFigure 5: LogRhythm Case Management metrics dashboard in Power BI

Start Using Visualization Tools to Empower Your SOC

There’s a tremendous amount of potential value within the LogRhythm SQL database that can be used to support your security program initiatives. Using visualization tools like Power BI or Tableau can empower operations and management — helping them achieve their goals and strategies, show ROI, and demonstrate the need for investment in a security program.

Full disclosure, Penn Medicine did not create all these visualizations. Credit for the visualizations in Figure 3, Figure 4, and Figure 5 goes to Nicholas Fanelli, Manager I/S Security & Risk Management at Lehigh Valley Health (thanks, Nick). Our team greatly appreciates Nick’s work and willingness to share. Stay tuned for the dashboard templates Nick has provided for LogRhythm users to download from Community.

Rob is a valued LogRhythm contributor and has earned 37 Community badges. LogRhythm Community badges