At Moogsoft, we are constantly extending technical alliances with key vendor partners we integrate with, making it easier to exchange data streams between Incident.MOOG and leading partners’ tools or infrastructure.
We are therefore pleased to announce that the Incident.MOOG LAM for Splunk© Enterprise and Splunk Cloud™ is now available on Splunkbase, Splunk’s online marketplace for complementary solutions. In addition, both companies are promoting each other as technical partners, see Splunk’s technical partner list here. By integrating with Splunk, Moogsoft adds real-time machine learning analytics and situation-based contextualization to Splunk data, helping IT Ops and DevOps teams to further reduce their Mean-Time-To-Detect (MTTD) and Mean-Time-To-Restore (MTTR) for production incidents.
Understanding Log Analysis Using Splunk
Splunk provides the leading software platform for real-time Operational Intelligence. Its log analysis software and cloud services allow organizations to capture, index and correlate real-time data in a searchable repository, where Splunk users can then generate graphs, reports, alerts, dashboards and visualizations.
In essence, Splunk is a powerful tool that makes it easy to index and search for information from the voluminous data that reside in logs. It’s particularly awesome for forensic analysis. Similar to Google search, if you know what you’re looking for, you simply query for it and the information results come back in real-time.
But sometimes you don’t know what you looking for, like when a service-affecting situation starts to unfold in a production environment. With applications and associated services capable of generating millions of log entries daily, where to start looking is often a challenge. In this case, you first need real-time situational awareness before you can start drilling down with a powerful search tool like Splunk.
Likewise, you can sometimes get overwhelmed with the results from log queries because the “signal” you’re looking for can’t be separated from all the “noise” in between. For example, if you look at the contents of a log file, you’ll notice that a lot of duplicated data and entries go far back in time. No surprise here, because after all, a log is essentially a raw audit trail. But things can start to slow down when, say, in a collection of indexed log files there are lots of non-critical events that go far back in time, and they keep showing up in your data queries, making it hard to find what you’re looking for. You’re likely to run multiple and frequent queries into ALL of the log files that could be potentially relevant to an outage. But when an outage is unfolding, how long does it take before you get notified to start responding? Then, how long to determine which are the relevant logs and what to query for? Then, how long to interpret the results?
Adding Situational Awareness with Moogsoft
Moogsoft adds real-time machine learning analytics and situation-based contextualization to Splunk to accelerate incident detection and remediation. By integrating seamlessly with Splunk data, Moogsoft can ingest raw events, apply machine learning to separate the signal from the noise, and contextualize the related events and alerts into clusters for each service-affecting situation. In this way, Moogsoft adds real-time situational awareness to Splunk so you can detect and resolve service-affecting incidents faster. Moogsoft can ingest data streams from the rest of the IT environment that may not yet reside in Splunk’s data repository, correlate it with the data from Splunk, and then feed resulting situation data back into Splunk.
Our mutual customers (and there are many) have discovered the top three benefits for using Moogsoft and Splunk together:
1. Moogsoft as a real-time alerting mechanism to mitigate scheduled ad-hoc queries
In Splunk, live event feeds go into an indexer and are then fed into a data repository. To make use of this data, you must query it through the UI. If you want to create an alarm, you need to set up this query as a repeating job. Let’s say a query will be executed every 10 minutes – the results will run through some threshold or decision criteria and then an alert will be sent when appropriate. The best scenario is that you find out about an alert every 10 minutes (a potential 10 minute delay). Additionally, these continuous polled queries can consume computational resources, particularly in a large and multi-faceted IT environment.
Instead of polling intensive queries every 10 minutes and delaying awareness of occurring incidents, why not feed event data into Moogsoft to automatically identify incidents in real-time? As live events enter the Incident.MOOG LAM for Splunk, Moogsoft immediately starts pre-processing them with machine learning algorithms making the resulting data easier to index and query. The result is real-time alerting notification for faster response and significant resource savings from reducing scheduled ad-hoc queries.
2. Moogsoft as an analytics processor to add context to Splunk data
Moogsoft takes a situation-based approach to incident management. That means that instead of forcing operators to look at massive volumes of individual alerts, Moogsoft uses machine learning algorithms to reduce the noise and create clusters of related alerts (Situations) that identify anomalous incidents as they occur in real time.
With this product integration, you can now take Moogsoft situations and index them into Splunk. This means that you can now store contextualized data in the context of all your other data and use Splunk as a powerful and consolidated reporting engine.
3. Moogsoft as an aggregator of all events, then feeding it all to Splunk
There may be cases where a customer doesn’t want to plug Splunk into every device in their environment and store every single log file. Instead, they can now use Moogsoft to collect all event data, contextualize it, and feed it into Splunk for storage. The result is a more polished and organized data set within Splunk for querying and reporting.
Realize the Benefits of Moogsoft and Splunk – Next Steps
The Splunk Adaptor to Incident.MOOG helps users of Splunk uncover contextualized situations of related events and alerts in their Splunk data repository, as generated by the IT environment. To download the Splunk Adaptor for Incident.MOOG, go to: https://splunkbase.splunk.com/.
About the author Sahil Khanna
Sahil Khanna is a Sr. Product Marketing Manager at Moogsoft, where he focuses on the emergence of Algorithmic IT Operations. In his free time, Sahil enjoys banging on drums and participating in high-stakes bets.