On any given day, a large enterprise might see something in the range of one or two Priority 1 incidents, five or six Priority 2 incidents and 100 Priority 3 incidents. But does everyone really need to be aware of each and every one? If an enterprise’s entire support staff gets notified about every single incident, there’s a lot of attention being distracted away from core responsibilities, which are quickly becoming more and more specialized. In reality, this happens on a daily basis at every enterprise. Just think about the time wasted around the world from incidents being juggled back and forth between support teams who may not have any designated ownership.
Large enterprise support teams can no longer cope with the volume of incidents occurring on a daily basis with their current approach. Traditional ITOA tools display a comprehensive view of ALL incidents occurring across an organization, which is useful for certain administrators and key decision makers, but each support team has a unique responsibility and really only needs to be looking at certain types of incidents in order to do their job. All of the additional information is actually overwhelming and even distracting.
One approach is for individuals to each set event display filters. Unfortunately, this approach is tedious and simply doesn’t scale for large organizations that incur changes on a fairly regular basis. There has to be a better way…
Moogsoft Automates Situation Workflow with Teams
In response to demand from leading enterprises, Moogsoft now automatically delegates the right Situations (aka incidents) to the right teams at the right time. The days of “tell everyone” will be a thing of the past. We enable administrators to properly define their teams, team members, and areas of focus so that the communication of Situations and notifications can be highly relevant and automated.
With Incident.MOOG, teams can now be created and modified with a click of a button. Members can be added either manually (by an administrator) or automatically (based on specified expertise and areas of interest). Team administrators have the ability to define the Situations that are of interest to their particular team and as Situations are generated, any team that satisfies the team definition criteria is automatically associated with those Situations. So when a user logs in to Incident.MOOG, they are automatically presented with all of the Situations that are associated with their team.
Let’s look at an example of how this can improve operational efficiency:
A global enterprise may have somewhere from 10 to 20 different support teams. These teams might be categorized around areas like web, compute, database, storage, cloud, sysadmin, etc. Let’s say that an outage occurs and something like 10,000 events get fired over the period.
Traditionally: Everyone hops on a bridge call, triages their domain with tools and presents/argues their findings.
Approximate Man Hours Taken: 20
With Moogsoft: A Situation is created. 10,000 events get analyzed, of which 25 events are related to the outage, but all events are related to compute and web tiers. Moogsoft only notifies the Compute and Web Teams – all other teams are not notified.
As a member of the Compute Team and an Interested Party member of Web related events, I get notified of this Situation. When I log in, the new Situation appears right above the 6 other relevant Situations that I should be focused on.
Total Man Hours Taken: 2
Let’s assume over a year there are 200 P1 incidents. If Moogsoft provides 18 hour time savings per incident, that means 3,600 hours per year of saved team productivity! The average employee works 1,900 hours per year (38 hours x 50 weeks/year), therefore the productivity savings can be significant for a typical enterprise.
What are some other ways in which Moogsoft teams can accelerate productivity?
Incident Escalation and Team Performance Management
By clicking on any of the Situations in the above screenshot, I’m taken to a Situation Room that provides a full narrative including all discussions, resolutive actions taken, knowledge from prior similar Situations and so on, so that each team can collaborate to resolve incidents as much as possible and prevent unnecessary escalation.
At times when escalation is required, modern IT support makes it challenging to decide who to pass the torch over to because of organizational complexity and a lack of clearly defined roles. Often times, senior support staff end up receiving the bulk of unresolved incidents and quickly become overwhelmed with escalation triage. Incident.MOOG now indicates all ‘Interested Parties’ for each Situation. This means that each Situation displays the team responsible, interested parties, each individual’s levels of expertise and their current availability. Operators can then decide how incidents should be passed along to be resolved most efficiently.
Delegating the right incidents to the right teams is a huge step towards making your organization operate more efficiently. However, once incidents are in the right hands there’s likely still significant room to improve the way in which those incidents are handled. How much time was spent on each incident? What were the steps taken from beginning to end? Were there any noteworthy insights and root causes? Were they properly documented? By tracking these types of metrics, Moogsoft provides administrators with a constant view of their team’s productivity to facilitate continuous improvement.
Moogsoft has addressed enterprise and service provider requirements by introducing new teaming capabilities that are dramatically improving support effectiveness in several ways. These include:
- Streamlined Onboarding: Enhance the onboarding experience for new operators. Once logged in, they are automatically presented with Situations for their domain. There is no need to select or remember any Situation filters.
- Increased Productivity: Eliminate the unnecessary data obstructing supports teams’ view. Ensure that the right information gets to the right people at the right time and track team performance to continuously improve productivity.
- Situational Awareness: All members of a team see their team’s Situations. This ensures a consistent situational view that all operators are working from. Additionally, by providing visibility into the set of teams that are impacted or interested in certain situations, teams now have a clear view of all parties that are potential owners or stakeholders in a Situation so escalation can be conducted effectively.
- Change Tolerance: In response to changing IT infrastructures and incident types/frequencies, the support organization is constantly changing. Moogsoft allows Support to keep up with the change by easily defining and modifying teams, members and their roles.
A company’s greatest asset is its people and if those people aren’t organized pragmatically, there’s a high cost to be paid. Today, enterprises are suffering from the cost of inefficient IT support but Moogsoft is quickly changing that. By clearly defining teams and their designated responsibilities to automate the delegation of incidents, enterprises can now optimize their support to resolve incidents faster. Ready, set, team!
About the author Sahil Khanna
Sahil Khanna is a Sr. Product Marketing Manager at Moogsoft, where he focuses on the emergence of Algorithmic IT Operations. In his free time, Sahil enjoys banging on drums and participating in high-stakes bets.