This is the second in a series of three blog posts exploring the relationship between observability and a set of SDLC practices.
This is the second in a three-post series themed around Ops-led DevOps, where I explore the relationship between observability and a set of software delivery lifecycle practices that support the adoption of DevOps principles and the transition from project to product centric ways of working. I started with Site Reliability Engineering, here I consider Value Stream Management (VSM) and I will finish with Continuous Delivery.
Forrester defines VSM as:
“A combination of people, process, and technology that maps, optimizes, visualizes, measures, and governs business value flow (including epics, stories, and work items) through heterogeneous enterprise software delivery pipelines. VSM tools are the technology enabling the practices of VSM.” 1
VSM demands teams think like value streams. Define ‘value stream’ as anything that delivers a product or a service and the start point is when an idea is had. The finish is when that idea is received by a customer and becomes an outcome. A value stream team also needs to maintain and support their product or service, and they aim to avoid this activity interrupting the flow of innovation. The key goal is to continuously reduce the time to value.
Culture and Humans
VSM is primarily concerned with optimizing the flow of work and making it visible. Transparency builds trust, but in technology we are particularly challenged by the nature of our work; that it’s invisible, unique and unknowable. Value Stream Mapping has long been recognized as an essential tool to start a DevOps journey; through identifying a value stream we can clearly identify the end-to-end steps to enhance an existing product or service, and the exercise provides an unparalleled opportunity for a multi-functional team to visually collaborate on their work. It also is a platform for building empathy between team members for the challenges each face in the daily effort to get their job done. But it has its limitations; a key one being that it’s human driven, not data driven. And another, that it tends to be a one-off exercise, not an ongoing, continuous inspection of progress. VSM, alongside observability, can solve some of these challenges for us.
The three pillars of the most popular agile framework, Scrum, are transparency, inspection and adaptation. Cultures that embrace these attributes find themselves receiving fast feedback which allows them to pivot in the face of changing circumstances and continuously deliver value to their customers. VSM platforms hook into every tool in the DevOps toolchain, including observability tools, and make the data available (transparent) for inspection during sprint events such as planning, review, retrospectives and daily stand ups, so that the team can adapt their work as needed. Essentially, it provides a real-time, data-driven value stream map that teams can continuously improve, focusing on the cycle time from idea to value realization.
Value Streams and Processes
A value stream is anything that delivers a product or a service and is a set of interconnected steps, activities or processes that start when an idea for a feature is had, and finishes when that idea is in the hands of the customer and delivering value. How long it takes end-to-end is the cycle time, and this is the key metric for improvement. But metrics are interconnected.
Whilst observability is mainly concerned with the product in production, so post cycle time, it’s worth considering how the maintenance and operation of a product can impact a team’s ability to enhance a product. Whether the team is working to a “we build it, we own it” model or whether there are separate teams to manage production support, someone will be impacted by poor performance and outages in live. The unplanned work and rework that result from problems and incidents deduct time from planned work, whether that work is new features for the product or platform improvements for the product. That means less innovation and a potential slowing of cycle time (certainly if the team owns their product end-to-end). VSM tooling can show us when and where the team is slowing down, or experiencing constraints.
VSM makes the value stream visible. Observability isn’t just about fixing problems quicker, it’s also about understanding customer experience better. Where VSM can tell us how fast our customers are receiving value, observability can tell us what that value really is.
Automation and Tooling
As organizations adopt DevOps ways of working, they’ll start by practicing continuous integration (CI), where every developer merges code at least daily into trunk and unit, integration and user acceptance tests are automated, which puts them in a place where software is always in a releasable state and they can practice continuous delivery (CD) – releasing enhancements on demand into the live environment. Building a CI/CD pipeline takes time, and having a fully integrated end-to-end DevOps toolchain takes even longer. The CI/CD pipeline itself must be observable so that data and insights are available as to what tests might be failing and causing flow turbulence.
Observability isn’t just about incident management in production services – it’s about observing everything; having telemetry everywhere. That means our pre-production environments too, our DevOps toolchain and our end-to-end value stream.
Our observability and AIOps platforms don’t just tell us what’s working, why and how to fix but can also be leveraged for product performance and value-based analytics and insights. Teams using DevOps principles and value stream thinking will estimate the value outcome of their experiments (enhancements) and can use observability and AIOps to display and augment behaviors using metrics such as page conversion and bounce rates. The goal of seeing flow from ideation to value realization is achieved, and this data is inspected and drives adaptation conversations to define the next set of experiments.
How Observability Helps Organizations Adopt VSM Practices
- Shortening MTTR means more time for innovation and more time to spend on adapting ways of working to optimize flow and more value outcomes delivered to customers
- Reducing the risk and costs associated with outages directly improving a product teams Profit and Loss (P&L) or cost:value ratio
- Improving customer delight which can be understood by metrics such as Net Promoter Score (NPS) and referrals
- Using observability throughout the value stream, so in pre-production and across the DevOps toolchain, means less defects and more predictability in production
- Making the value stream visible and making value measureable
What to Do Next
- Sign up for a Moogsoft Express trial
- Watch the on-demand webinar by Helen Beal and Adam Frank, ‘Telemetry Everywhere: Observability and AIOps in the DevOps Cosmos’
- Download the free eBook ‘Observability with AIOps for Dummies’ by Adam Frank
About the author
Helen Beal is a DevOps and Ways of Working coach, Chief Ambassador at DevOps Institute and an Ambassador for the Continuous Delivery Foundation. She provides strategic advisory services to DevOps industry leaders and is an analyst at Accelerated Strategies Group. She hosts the Day-to-Day DevOps webinar series for BrightTalk, speaks regularly on DevOps topics, is a DevOps editor for InfoQ and also writes for a number of other online platforms. Outside of DevOps she is an ecologist and novelist.