I always enjoy going to our SplunkLive! events and most recently I was representing Splunk at SplunkLive in Chicago. The number of attendees was clearly more than expected at nearly 100 folks and chairs had to be found for another 20 people. Three customers presented their ‘Splunk stories’ and the sessions were very interactive.
The first customer presentation shared some of the advantage of using Splunk to monitor security events:
- Dramatically higher productivity and faster response time:
- Definitive answers for requests in 15 minutes vs. previously, would take days or even weeks for definitive answer
- Identifying serious security attacks: bots “calling home” to known malware domains
- Correlating events with malware domain look-ups
- Send matching events to ticketing system for further investigation
Before Splunk, there was no systematic way to do this. They were also kind enough to share a few of their security searches.
This search looks through web proxy logs for a strain of malware: [search index=proxy_logs"mc.php?id=" | regexmisc="(php\?id=(0[1-9]|1[0-9]|2[0-9]|3[0-2])(0[1-9]|1[0-2])20\d\d_)” | fields Internal_IP] index=active_directory” OR index=”*” NOTindex=proxy_logsusername=”*” Machine_Name=”*” | top username, Machine_Namelimit=”100″
This search looks through proxy logs for HTTP POSTs over 10 MB: index=proxy_logscs_method=POST cs_bytes>10000000
This search looks for users who log on from more than one computer over the network: index=active_directoryevent_id=”675″ AND NOT username=”NT AUTHORITY\\SYSTEM” | top username, Internal_IP
The second customer discussed liking Splunk’s approach to handling data, compared to ArcSight. ArcSight forces data normalization, reducing agility and flexibility. Splunk applies context to raw data and allowed them to easily add new data sources as needed, on their own our terms. Moving to Splunk allowed them to utilize a combination of Tenable Security Center and Splunk to replace ArcSight resulting in significant cost saving.
The final customer presentation gave insight into using Splunk to move from a reactive to a proactive approach monitoring applications that “don’t log in a standard way.” This allows them to avoid downtime before it happens. And, best of all no grep. This Splunk customer has about 100+ users logging into specific dashboards monitoring specific statistic represented in dashboards and reports in real time. These specific dashboards include ones for their NOC, Operations Center, Customer Service team.
My favorite part of the day is always listening to customers in the hallways during breaks and at lunch share ideas. Customers in the windy city traded tips on how to mask data, talk to databases, monitor airline ticket kiosks, and use Splunk during M and A activity.