Rethink Your Understanding

Transforming Software Delivery

Delivering Value

Navigating the Digital Product Workflow Metrics Landscape: From DORA to Comprehensive Value Stream Management Platform Solutions

by

16 min read

This article is designed to help senior leaders navigate the landscape of digital product delivery metrics platforms and tools. It offers insights into the various options available and the rationale for selecting one solution over another when determining the most suitable metrics platform.

In the rapidly evolving landscape of software engineering and digital product delivery, leaders are increasingly reevaluating their approaches to team performance, productivity, and metrics. These leaders are turning to quantitative and qualitative data to make delivery teams’ performance and efficiency improvement efforts more transparent. Teams seek data-driven insights to identify bottlenecks, uncover root causes, and evaluate potential enhancements. They aim to eliminate or refine these obstacles by experimenting with various changes to achieve greater efficiency.

Embracing new metrics for today’s work environment is not immune to Goodhart’s law and the potential pitfalls of gamification. Therefore, leaders and delivery teams must embrace these metrics differently and separate business discussions and team conversations when analyzing and utilizing them.

However, having data insights and optimizing the performance of your delivery team is not just a necessity but also a gateway to capturing performance insights, maintaining competitiveness, and enhancing processes, practices, automation, and team design to deliver value effectively. The potential benefits of these platforms are vast, and with the right approach, you can significantly improve your digital product delivery.

As a senior leader with extensive experience guiding technology teams through digital transformation, I have witnessed firsthand the vital role of selecting the right metrics platform. During 2020 and 2021, I faced a similar challenge while assessing and implementing a metrics platform to gain insights into our digital transformation efforts, team structure, and automation investments. Leveraging my background, I quickly gained a deeper understanding of modern metrics.

At that time, a senior leader in our division struggled to grasp the various metrics options and their relevance to our current delivery capabilities, often fixating on vendor pricing—a legitimate concern given our tight budget. This misunderstanding led to friction as we aimed to identify the best solution. While I acknowledged the advantages of different platforms, the leader’s emphasis on cost over functionality complicated our decision-making process.

After more than eight years of investing in our architecture, team design, automation, and delivery processes, we arrived at a critical decision point: Should we invest in a Software Engineering Intelligence (SEI) platform tailored for delivery, or should we pursue a comprehensive Value Stream Management (VSM) solution that encompasses the entire lifecycle from ideation to production? It’s important to note that the concept of an SEI platform didn’t even exist three years ago when we began our evaluation. At that time, we only had DORA and various vendors offering metrics platforms that needed more coverage across all stages, focused solely on parts of the delivery process, or had notable gaps in delivery insights, even among holistic solutions. My goal was to compare all vendor options, assess their coverage and focus, identify gaps across each workflow phase, and select the best fit.

We initially assessed ten vendors and ultimately narrowed our options to two: our preferred SEI solution and our top choice for the VSM platform. If budget constraints had not been an issue, I would have acquired both solutions to bridge the gaps between them seamlessly. I even lightheartedly suggested a potential merger for both vendors! We chose a VSMP solution based on where we assessed our bottlenecks and stage. This experience also inspired one of my early articles on the subject – Finally, Metrics That Help. Contact me if you’d like to explore my experiences, methodologies, discussions, and presentations related to this evolution in metrics.

Fast forward to 2024, and we discover the inspiration behind this article. I learned about a senior technology leader tasked with transforming and enhancing the large technology team he inherited upon joining an enterprise company. His primary objective is to turn the department around and to improve digital product delivery. He aims to reintegrate QA and product delivery responsibilities into teams—responsibilities removed during previous cost-cutting measures implemented by earlier stakeholders before he arrived in the organization. Based on my understanding, part of his strategy includes adopting delivery performance metrics to establish team metrics that will create a baseline for current performance, offer valuable insights, and initiate a pathway to enhancing overall performance and efficiency. He has focused explicitly on or selected DORA metrics to better understand product delivery roles and processes. Although I didn’t have the opportunity to discuss this with the leader before his decision, I am intrigued by how he arrived at the choice of DORA metrics. Given that this enterprise technology team has been delivering work effectively in an agile environment for years, I wonder if his decision truly addresses the more significant challenges faced by his division in improving the delivery process. Are these the appropriate metrics and feedback mechanisms to resolve workflow bottlenecks?

Stages of the Product Delivery Workflow

Once you know the workflow, you can analyze and pinpoint its bottlenecks. This awareness enables you to implement metrics that offer valuable feedback and insights on these bottlenecks, helping you identify areas for improvement.

The stages of the digital product workflow, often discussed in frameworks such as Value Stream Management (VSM) and Flow Engineering advocated by Steve Pereria and Andrew Davis and outlined in Mik works, like Project to Product, typically follow the journey from an initial idea to the moment the product delivers value to the customer. Understanding these phases is essential for grasping how work flows within an organization and identifying where value is created.

The key stages or phases of the digital product workflow are:

  • Ideation (or Discovery)
  • Delivery
  • Operations
  • Support

Ideation (or Discovery): This phase focuses on generating, prioritizing, and refining ideas. It involves understanding what should be developed based on customer needs, market demands, and strategic objectives. This stage lays the groundwork for all future efforts and encompasses market research, gathering user feedback, and creating early design prototypes.

Delivery: Once an idea is validated, it transitions into the delivery phase, where actual development occurs. This includes coding, testing, and deploying the product. This phase emphasizes transforming ideas into tangible products ready for customer delivery. It also encompasses the CI/CD pipeline, where DORA metrics are often utilized.

Operations: Once the product is delivered, it transitions into the operations phase. This stage focuses on maintaining and managing the product within a live environment. Essential activities during this phase include performance monitoring, infrastructure management, and ensuring the product operates smoothly and efficiently.

Support: The final stage is support, encompassing customer assistance, incident management, and ongoing enhancements based on user feedback. This phase ensures that customers derive continued value from the product while effectively addressing any issues arising after deployment.

Value Stream Management resources, Product Flow, and Flow Engineering often highlight these stages. They offer a structured framework for understanding the workflow, guiding it from the initial idea to continuously delivering value to customers.
These stages enable organizations to grasp how value is generated and delivered, facilitating more effective management of the entire product lifecycle.

Understanding the Types of Metrics Platforms

Before exploring specific tools, it’s essential to grasp the three main categories of qualitative metrics platforms focused on delivery performance insights as I understand them: DORA metrics, Software Engineering Intelligence (SEI) platforms, and Value Stream Management (VSM), along with flow metrics.

DORA Metrics

DORA (DevOps Research and Assessment) metrics are designed to measure the effectiveness and efficiency of your software delivery process from code commit to deployment. These metrics include:

  • Change lead time
  • Deployment frequency
  • Change fail percentage (previously change failure rate)
  • Failed deployment recovery time (previously mean time to recovery or MTTR)

The information regarding throughput, stability, and metric definitions is sourced from the dora.dev website.1

Throughput

Throughput measures the velocity of changes that are being made. DORA assesses throughput using the following metrics:

  • Change lead time – This metric measures the time it takes for a code commit or change to be successfully deployed to production. It reflects the efficiency of your delivery pipeline.
  • Deployment frequency – This metric measures how often application changes are deployed to production. Higher deployment frequency indicates a more agile and responsive delivery process.

Stability

Stability measures the quality of the changes delivered and the team’s ability to repair failures. DORA assesses throughput using the following metrics:

  • Change fail percentage – This metric measures the percentage of deployments that cause failures in production, requiring hotfixes or rollbacks. A lower change failure rate indicates a more reliable delivery process.
  • Failed deployment recovery time – This metric measures the time it takes to recover from a failed deployment. A lower recovery time indicates a more resilient and responsive system.

Why Choose DORA?

The bottleneck is deployment.

  • Focused on Delivery Efficiency: DORA metrics are ideal for organizations that need to optimize their CI/CD pipeline and improve deployment frequency and stability.
  • Narrow Scope, High Impact: DORA metrics can provide precise insights to drive improvements if your primary bottlenecks relate to deployment.

Software Engineering Intelligence (SEI) Platforms

In recent years, Software Engineering Intelligence (SEI) platforms have evolved into a distinct category, broadening the spectrum of metrics they address beyond those defined by DORA. While DORA metrics primarily assess the efficiency of the CI/CD pipeline from code commit to deployment, SEI platforms take a more comprehensive approach by focusing on the entire delivery process. This includes everything from when a work item enters the product backlog to its eventual deployment in production.

SEI platforms offer a more comprehensive array of metrics encompassing the traditional DORA metrics and additional measures such as a broader definition of cycle time, throughput, work-in-progress (WIP), and quality indicators like pull request size and rework rates. These platforms prioritize integrating diverse engineering tools—including Git, CI/CD, and project management—to provide a cohesive view of the delivery process.

One of the primary distinctions between DORA and SEI metrics lies in SEI’s emphasis on actionable insights and benchmarking. SEI platforms frequently utilize data science to create benchmarks that enable teams to assess their performance against industry standards, a crucial factor for establishing informed improvement goals. Furthermore, SEI platforms are designed to enhance visibility into business value metrics, aligning engineering initiatives with overarching business objectives—an aspect that traditional DORA metrics often overlook.

A key feature of SEI platforms (and VSM platforms) is their focus on enhancing developer experience (DX) and promoting organizational efficiency. Many of these platforms assess developer productivity, which often sparks debate and varies by context—particularly when distinguishing between team productivity and individual performance. They also track health and sentiment through metrics such as work-in-progress (WIP) and burnout alerts. These metrics are vital for sustaining a healthy and efficient engineering team.

When comparing DORA and SEI platforms, it becomes evident that DORA offers valuable insights into the deployment phase. In contrast, SEI platforms provide a more comprehensive perspective on the broader delivery process, making them better suited for organizations aiming to optimize their operations beyond just CI/CD.

Considering these distinctions, your organization’s decision between DORA and SEI should be guided by its current bottlenecks and the specific challenges it aims to tackle. DORA metrics may be adequate if your primary obstacles lie within the deployment phase. However, an SEI platform would be the most suitable if your goal is to optimize the entire delivery pipeline and better align engineering efforts with business objectives.

Gartner has recently published a market guide on SEI platforms for those interested in exploring this further.

Here are some key types of metrics commonly found in some of the more known SEI platforms :

  • Cycle Time: Measures the time it takes for a work item to move from the backlog to deployment. This metric helps identify inefficiencies in the development process and highlights areas where delays occur.
  • Throughput: This metric tracks the number of work items completed over a specific period. It provides insight into team productivity and helps assess whether teams meet their delivery goals.
  • Work in Progress (WIP): Monitors the number of items being worked on or in progress. High WIP can indicate bottlenecks or overloading of the team, which can slow overall delivery.
  • Pull Request (PR) Size and Rework Rate: Evaluate the size of pull requests and the frequency of rework required. Smaller PRs with lower rework rates tend to be merged faster and with fewer issues, contributing to smoother deployments.
  • Planning Accuracy: This compares planned versus actual delivery timelines. It helps teams understand how well they estimate their work and whether they consistently meet their commitments.
  • Developer Experience (DX): SEI platforms often include metrics that assess developer satisfaction and efficiency, such as time spent in meetings versus coding and the overall impact of tools and processes on developer productivity.

Why Choose SEI?

Your bottleneck remains in the workflow’s delivery phase, but you have the budget for an SEI solution and need insights beyond code commit to delivery.y.

  • Optimizing Development and Delivery: SEI platforms are perfect for teams looking to streamline the development and delivery process from backlog to production, balancing speed with quality and introducing team health insights.
  • Broader Than DORA, Narrower Than VSM: SEI platforms offer a middle ground, providing insights into the development process without tracking the entire lifecycle.

Value Stream Management (VSM) Platforms and Flow Metrics

VSM offers an end-to-end perspective, focusing on the entire digital product lifecycle—from ideation through delivery to operations and support. The core idea behind VSM is to visualize, measure, and optimize the flow of value across all stages of this lifecycle. VSM platforms are designed to provide deep insights into how value is delivered across complex and interconnected systems, identifying inefficiencies and bottlenecks not just in the development process but across the entire organization. These platforms are ideal for companies looking to align their technical and business objectives, ensuring that every part of the process contributes to delivering value to the customer.

Organizations ready to adopt VSM typically have a mature or highly performant delivery practice. They have likely already optimized their CI/CD pipelines, backlog refinement (grooming), planning and technical analysis and design, and task breakdown practices, and seek to improve the overall flow of work across different teams and departments. These organizations often deal with complex value streams that require integration across multiple tools and departments. VSM helps them visualize and manage these streams to optimize the delivery of value to customers.

Organizations that are prepared to adopt Value Stream Management (VSM) typically possess a mature and efficient delivery practice (delivery is not or is no longer the bottleneck). They have likely optimized their CI/CD pipelines, backlog refinement, planning, technical analysis and design, and task breakdown, code development, and pre-code commit processes and seek to improve the overall flow of work across different stages. Usually, discovery is next, but it could be operation or support. These organizations often navigate complex value streams requiring integration across multiple tools and departments. VSM enables them to visualize and manage these streams effectively, optimizing the entire customer value delivery workflow.

Value Stream Management (VSM) platforms are designed to provide comprehensive insights into the technical aspects of software delivery and their alignment with broader business objectives. Leading VSM platforms offer a range of business metrics that empower organizations to assess the return on investment (ROI) from their software development efforts, ensuring that their initiatives effectively contribute to achieving desired business outcomes.

Here are some key business metrics commonly found in top VSM platforms today:

  • Flow Velocity: This metric tracks the number of flow items—such as features, defects, risks, and debts—completed within a defined time frame. It provides valuable insight into the speed at which value is being delivered.
  • Flow Efficiency: The ratio of active time to the total time a flow item spends in the value stream serves as a key metric for assessing the efficiency of work processing.
  • Flow Time: The total duration for a flow item to progress from ideation to completion is a vital metric for assessing time-to-market.
  • Flow Load: Tracks the number of flow items currently in progress. A high volume may signal potential bottlenecks or team overloads.
  • Flow Distribution: Tracks the balance among various types of work—features, defects, risks, and technical debt. This metric ensures that all efforts align with strategic priorities.
  • Cost of Delay: This approach assesses the financial consequences of postponing the delivery of features or projects, enabling organizations to prioritize tasks based on the potential business value lost due to these delays. By quantifying the cost of delay, companies can make informed decisions about which features to prioritize, ultimately maximizing their return on investment (ROI).
  • Value Stream ROI: This metric evaluates the return on investment for various value streams by comparing development costs with the business value generated from delivered features. It enables businesses to assess the financial effectiveness of their development processes, allowing them to adjust resources or priorities to optimize returns.

Like various SEI platform solutions, many top VSM platforms provide valuable insights into team health and metrics associated with developer and team experiences.

Why Choose VSM?

  • Holistic View of the Value Stream: VSM tools offer comprehensive visibility, insights, and feedback across the entire digital product workflow. They assist in optimizing each stage, from idea generation to production.
  • Alignment with Strategic Goals: These platforms help ensure that all phases of your software lifecycle are aligned with your business objectives.

Choosing a Platform

I think selecting the appropriate metrics platform requires careful consideration of several factors, such as your budget, the locations of bottlenecks in your product workflow, the specific challenges you seek to resolve, and the maturity or efficiencies of your current processes.

Identify Your Bottlenecks

  • Deployment Challenges: DORA metrics might be the ideal solution if you are on a tight budget and experience delays mainly after commits, characterized by lengthy lead times or frequent deployment failures. DORA-based tools and surveys can provide the insights necessary to enhance your CI/CD pipeline.
  • Development and Delivery Challenges: If inefficiencies arise during the development phase—such as extended cycle times or excessive work-in-progress—SEI platforms can help you identify and effectively address these challenges. Additionally, if you encounter deployment bottlenecks, including issues from code commit to deployment, while also having a favorable budget, leveraging an SEI platform can offer significant advantages over DORA.
  • Systemic Issues Across the Lifecycle: For organizations facing extensive inefficiencies from ideation to production, VSM platforms offer a comprehensive solution. These platforms adeptly identify and eliminate bottlenecks throughout the value stream, although they may also represent a significant budget investment.

Why Not Start with a Broad Scope VSM Solution?

While starting with a comprehensive Value Stream Mapping (VSM) solution for complete visibility across your value stream may seem reasonable, there is a strategic advantage in first targeting specific issues: bottlenecks. The reasoning is simple: optimizing downstream of a bottleneck often results in insufficient work while improving upstream processes without addressing the bottleneck can lead to a backlog at that critical choke point. Furthermore, it is crucial to consider several factors, including the organization’s overall understanding of digital product delivery practices — not just within product and technology teams — along with its size and scale, as well as the costs linked to different solutions.

You can start by identifying and optimizing the bottleneck in your process. DORA metrics and SEI platforms can be considered for diagnosing CI/CD pipeline issues and improvements. After resolving these challenges, you can broaden your focus using SEI or VSM platforms to optimize the broader scope of your value stream or product workflow. This comprehensive approach guarantees that your improvements yield significant and sustainable enhancements in overall efficiency.

When to Consider a Broader Scope Platform

Organizations that have invested in and optimized their delivery processes—by establishing highly automated CI/CD pipelines, minimizing waste, and reducing wait times between stages—are ideally positioned to embrace broader-scope platforms, provided they have the budget to support these tools. Here’s why:

  • Emphasize the Early and Late Stages: After optimizing delivery, the subsequent focus should be on refining the ideation, operational, and support phases. Comprehensive platforms ensure that only the most valuable ideas progress, operations run efficiently, and support remains proactive and effective.
  • Comprehensive Optimization: Platforms with a broader scope, such as VSM tools, offer the visibility required to optimize the entire lifecycle. This ensures that every stage—from ideation to support—remains aligned and efficient.
  • Maximizing ROI: These platforms enhance your return on investment by tackling inefficiencies upstream and downstream of delivery, optimizing your existing investments in CI/CD and delivery processes.
  • Summary and The Importance of Team Health Metrics
  • Selecting the right metrics platform for your software delivery process is a crucial decision that hinges on your organization’s size, specific needs, and level of maturity. Whether your focus is refining your CI/CD pipeline with DORA metrics, enhancing the development process using SEI platforms, or achieving comprehensive visibility through VSM tools, aligning your choice with your strategic objectives is vital.
  • Based on my experience in guiding technology teams through digital transformation and my recent deep dive into evaluating the market and adopting a platform, I’ve learned that while the allure of a more expansive platform can be tempting, it’s crucial to ensure that your leadership and organizational mindset are ready for the journey. Without this alignment, even the most sophisticated metrics platform may fail to achieve its full potential.

Team Sentiment: It is essential to consider team health and quality metrics, such as Developer Experience (DX), which emphasizes team sentiment and well-being. The team performance insights and optimization qualitative measures are critical to improvement. Still, team health metrics complement the quantitative data from delivery metrics, creating a comprehensive view of software efficiency alongside team health. This combination ensures that you optimize processes and nurture a motivated and effective team. While this topic could warrant a follow-up article, or you can read some of my related articles on adopting metrics, it’s essential to integrate qualitative and quantitative feedback into your overall strategy.

By carefully choosing an appropriate metrics platform and complementing it with team health indicators, you can foster continuous improvement, boost efficiency, and provide greater value to your customers and business.

Side Note: In August 2024, I participated in a digital software product delivery workshop. The session delved into the DORA, SPACE, and DevX frameworks Led by industry experts on these topics. The workshop brought together practitioners from delivery teams and C-level executives, enhancing our understanding of these areas related to digital software delivery, optimization, and team health.

References

  1. DORA, https://dora.dev/guides/dora-metrics-four-keys/

Related Posts

Poking Holes

I invite your perspective on my posts. What are your thoughts?.

Let’s talk: [email protected]

Evolving the Agile Leadership Role: Integrating Value Stream Management into Agile Leadership

by

9 min read

This post is part of my ongoing series showcasing the competitive edge provided by maintaining Scrum Master or Agile Leader roles within teams. If you’d like to read more, please review the related posts section at the end of this article.

Value Stream Management

Value Stream Management doesn’t require you to transform your software delivery practices beforehand or adopt Agile, Lean, or DevOps practices first. It provides visibility into how you ideate, plan, create, and deliver digital products or services. It optimizes the time spent on these tasks by automating and standardizing the workflow from ideation to production and operation. Based on the insights, you can discuss, decide, and experiment with frameworks and practices to improve your system and workflow. I discovered Value Stream Management in late 2020 and introduced it to my department in 2021, after more than six years into our digital evolution. VSM is a natural complement to our Agile, Lean, and DevOps initiatives.

Like Agile, DevOps, and Lean practices, Value Stream Management (VSM) demands organizational understanding, alignment, and expertise. Successful adoption of VSM requires everyone to understand its purpose, benefits, and implementation. Subject matter experts play a crucial role in this process. Introducing dedicated VSM roles—such as Value Stream Managers, Value Stream Architects, and Flow Advisors—is essential for ensuring continuous value delivery and operational efficiency.

Expertise can be sourced externally, identified within the organization, or cultivated through strategic investment. Unlike the early attempts to transition teams from waterfall to agile by merely rebranding Project Managers or Business Analysts as Product Managers or Product Owners, these new roles are distinct and require dedicated training and development. Investing in this development is essential for success.

These new roles can be added to the organization, departments, or current members of cross-functional delivery teams. This article supports the cross-functional team option, recommending that Agile Leaders in cross-functional teams are a great option and can be trained as their teams’ VSM and Flow experts. We are testing the idea of developing VSM skills and expanding the responsibilities of Agile Leaders or Scrum Masters within our cross-functional delivery teams.

The Conversation

A VSM colleague, Patrice Corbard, shared an insightful observation based on published results about Scrum Master maturity in a community Slack message. While I am still determining the research methods used or the accuracy of the research findings representation of the industry, the discussion it sparked was quite interesting. Patrice highlighted that only 8% of Scrum Masters can optimize product value streams. He posed three questions:

  1. Are you surprised by this very low proportion?
  2. What are the benefits of the value streams approach to get scrum masters interested in the subject, particularly in these difficult times when agile coaching roles are being called into question?
  3. What do you suggest to change the game and motivate scrum masters and their leaders/managers?

This inspired me to share how, I have been enhancing the Scrum Master/Agile Leader role by integrating VSM knowledge, experience, and responsibilities.

From the ScrumMatch website: 

Scrum maturity reflects the capability to leverage Scrum to deliver increased business value more efficiently. ScrumMatch evaluates the Scrum maturity of both Scrum Masters and organizations, providing transparency so each can understand how well they align with each other’s needs and expectations. A Scrum Master’s maturity level is displayed on their profile, rated on a 7-point scale from low to high maturity. 1, 2

Here is my summary of their findings. The top three points on the current state of Scrum Master maturity are:

  • Substantial Proportion Lack True Scrum Mastery: 38% of candidates fail to exhibit the essential qualities of a proficient Scrum Master.
  • Most Candidates Are Still Developing Foundational and Applied Skills: 37% of candidates are at the intermediate level, with 10% possessing foundational knowledge, 13% understanding and applying Scrum principles, and 14% starting to enhance their Scrum practices.
  • Limited Advancement to High Organizational Impact: A small percentage achieve higher maturity levels. Specifically, 14% utilize Scrum to enhance product development, 8% focus on optimizing the product value stream, and only 3% can optimize the entire organization.

First Question

“Are you surprised by this very low proportion?”

These findings don’t surprise me; they align with my experiences. Similar to the Product Owner role, The Scrum Master or Agile Leader team role is crucial. Enhancing its value and bringing it to maturity requires continuous investment in learning and hands-on experience. This dedication is essential for adding value to the team.

I attribute the questioning of the value and maturity level of Scrum Masters to several key factors: the commercialization of certifications (ease of access and proliferation of certification mills), the expertise and mindset of senior leadership or executive sponsors (entrenched or outdated perspectives), investment in continuous training, and hands-on practical experience.

Like other team roles, new Scrum Masters and Agile Leaders need time to develop their skills and gain experience. This should be complemented by continuous investment in learning and growth.

Second Question

“What are the benefits of the value streams approach to get scrum masters interested in the subject, particularly in these difficult times when agile coaching roles are being called into question?”

Starting with the second part of Patrice’s second question, which points to the industry debate on Agile and the value of the Scrum Master/Agile Leadership role, this topic commands a separate discussion beyond the scope of this article, and I will be brief. 

Many agile or digital transformation struggles and failures can be traced to leadership, prevailing mindsets, and experiences. However, I agree that the role of the Scrum Master has been diluted by the proliferation of accelerated certifications and the commercial aspects surrounding them. This trend has undermined the significance of the Scrum Master’s responsibilities and their potential to make a meaningful impact on cross-functional software delivery teams. 

Do we need the Scrum Master/Agile Leader role?

Your organization and team design will decide.

First-time product owners need time, trust, and support to grow into their new role. – Roman Pichler

This quote applies to Scrum Masters, Agile Leaders, and Product Owners or Product Managers. These roles add value as they gain knowledge and experience and improve their skills and trust in their teams.

My responsibilities include team design, systems thinking, and delivering high-quality technology quickly. Although I have extensive experience in agile delivery, I am not a Scrum Master and have never held that role. However, I have led and mentored our agile leadership team and am a strong advocate for its value within cross-functional delivery teams.

Conversely, as we progress through decades of digital transformation and move away from traditional waterfall methods, the benefits of Agile practices and the role of Scrum Masters are increasingly debated. Many organizations still struggle with Agile transformations, often concluding that Agile falls short, leading them to cut Scrum Master positions and similar roles from their budgets.

Industry experts, such as the highly esteemed Marty Cagan, argue in his latest book “Transformed” and in related interviews that these roles, as currently defined, are either redundant or lack substantial value. As I interpreted it, Cagan asserts that the responsibilities typically assigned to Scrum Masters should fall to other team members, particularly the Product Manager. I can only partially agree with this suggestion when teams have evolved to mature self-managed levels.

Third Question

“What do you suggest to change the game and motivate scrum masters and their leaders/managers?”

I will combine my response to the first part of question two, “”What are the benefits of the value streams approach to get scrum masters interested in the subject” and question three.

Evolving the Scrum Master/Agile Leader role with VSM

I appreciate that the role of the Scrum Master/Agile Leader remains impartial and uninfluenced by any particular type of the team’s work — be it feature development, technical debt, defect resolution, or security risk. This neutrality allows for a balanced and objective focus on the team’s success. While other team roles have specific areas of focus, the Scrum Master is uniquely positioned to ensure the overall health and performance of the team, utilizing tools like Value Stream Management and Flow Metrics.

Motivation stems from the Scrum Master’s or Agile Leader’s commitment to the team’s overall performance. The primary responsibility of a Scrum Master or Agile Leader on a cross-functional delivery team is to enhance team effectiveness and support greater efficiency. This involves managing rituals, team health, performance metrics, retrospectives, and collaboration. By leveraging Agile and Lean methodologies and incorporating Value Stream Management (VSM), as I propose, these leaders can achieve these goals more effectively. With executive sponsorship and support, the Scrum Master or Agile Leader is ideally positioned to lead VSM initiatives, focusing on the team’s overall success rather than specific product features or technical solutions.

Delivery optimization and oversite responsibilities are natural progressions of responsibility for Agile Leaders. One way to achieve this is through Value Stream Management (VSM), which analyzes the product creation and delivery process. Each team member should understand VSM principles, and having a subject matter expert on the team enhances delivery efficiency and customer satisfaction. Integrating VSM oversight into the role of Scrum Master or Agile Leader helps optimize team performance and align efforts with business objectives, ultimately improving the delivery of business value.

Benefits:

  1. Holistic Team Focus:
    By integrating VSM, the Scrum Master can oversee and enhance the entire workflow, ensuring all processes contribute effectively to delivering value. This comprehensive perspective complements the Scrum Master’s focus on team health and performance.
  2. Enhanced Process Efficiency:
    VSM helps identify bottlenecks and inefficiencies in the delivery process. A Scrum Master with VSM responsibilities can facilitate improvements across the value stream, leading to faster and more reliable delivery of products.
  3. Balanced Work Prioritization:
    The Scrum Master can leverage VSM to encourage teams to balance work priorities more effectively. This ensures that feature development, technical debt, defect resolution, and security risks are all addressed appropriately. This can provide a more even distribution of effort and attention across different types of work.
  4. Improved Metrics and Insights:
    Integrating VSM allows the Scrum Master to track flow metrics and other key performance indicators that provide deeper insights into the team’s productivity and areas for improvement. This data-driven approach fosters continuous improvement initiatives and enables teams to identify actionable steps for enhancement.
  5. Enhanced Collaboration and Communication:
    With a focus on the entire value stream, the Scrum Master can better facilitate collaboration and communication between different team members and stakeholders, ensuring everyone is aligned on goals and processes.
  6. Strategic Alignment:
    By managing the value stream, the Scrum Master can ensure that the team’s work aligns with the organization’s strategic objectives, enhancing the overall value delivered to customers and stakeholders.

Challenges 

Integrating Value Stream Management (VSM) into Agile Leadership tackles the growing complexity of digital product environments. This approach requires a comprehensive perspective on product delivery and team efficiency beyond traditional task execution. However, VSM is still in its early adoption phase in the industry. Despite its strategic importance, expertise in optimizing value streams is scarce in the job market. Organizations like the one I led take proactive steps by recommending and providing VSM training to their Agile Leaders. VSM adoption necessitates investing in training, experimentation, and collaboration with industry experts.

Organizational Impact and Future Prospects 

Adding Value Stream Management capabilities elevates how software product delivery teams and their leaders perceive their contributions to organizational goals. By emphasizing continuous improvement and strategic value creation, Scrum Masters and Agile Leaders can be better equipped through VSM to influence and elevate cross-functional team performance and organizational outcomes. This transformation tackles some of the immediate skills gaps identified by ScrumMatch. It establishes a new benchmark for the capabilities of Agile practitioners’ responsibilities and overall team value, clarity, and flow.

Integrating Value Stream Management into the Agile Leadership role marks a significant evolution in Agile team practices. It bridges a crucial gap, enabling Agile practitioners to link their team’s daily activities with strategic business outcomes. Organizations embracing this approach will likely experience increased agility, better alignment with business goals, and superior performance, positioning them to tackle future challenges and seize opportunities more effectively.

References

  1. ScrumMatch, https://scrumatch.com/, Understanding Scrum Maturity, https://scrummatch.com/en/support/understanding-scrum-master-maturity
  2. ScrumMatch LinkedIn post, April 2024, https://www.linkedin.com/posts/scrummatch_hr-recruiting-scrummaster-activity-7183423788401762304-psjP?utm_source=share&utm_medium=member_desktop
  3. Scrum Masters play a role, LinkedIn Post, https://www.linkedin.com/posts/patricecorbard_hr-recruiting-scrummaster-activity-7185600586954788864-0FmS?utm_source=share&utm_medium=member_desktop, Patrice Corbard, April 2024.

Resources and Recommendations

Related Posts

Poking Holes

I invite your perspective on my posts. What are your thoughts?.

Let’s talk: [email protected]

Beyond Facilitation: The Agile Leader’s Place in Cross-Functional Team Dynamics

by

9 min read

“Alignment is a force multiplier.”

― Gereon Hermkes

I was invited recently to a meeting with Agile Leaders, Product Owners, and various cross-functional roles. The aim is to establish a regular meeting among team members to align on the reasoning behind our team structures, identify who leads the delivery, and clarify team roles. Following a word cloud exercise, it became evident that team members were not on the same page regarding why our teams are structured as they are, the essence of agility, diverse perspectives on metrics, and, notably, a product owner questioning the value of the Agile Leadership role on the team or not understanding the role responsibilities. Besides administrative tasks and facilitation, what are the responsibilities of a Scrum Master or Agile Leader?

What problem am I trying to address? Team alignment: should we have agile leaders within teams? If yes, what additional responsibilities do they carry beyond team facilitating rituals?

Questioning the Agile Leader role is not uncommon. More recently, certain organizations have dismissed these team members to cut costs, failing to recognize the significance of their role. They have redistributed some responsibilities among the remaining agile team members.

Cross-functional delivery teams striving for enhanced alignment on Agile and Lean practices, team purpose, responsibilities, and the Agile leader’s role may encounter hurdles. Conversations within my extensive network in the past two years and themes in various podcasts have shed light on the dynamic between Product Managers/Owners and Agile Leaders/Scrum Masters. In misaligned teams, Product Owners often scrutinize the roles of Scrum Masters or Agile Leaders, the team itself, its practices, and the accountability for the team’s deliverables.

Two books, among many that shaped my perspective

In Agile software development, balancing various responsibilities and understanding the unique team roles is crucial to fostering an environment that encourages efficiency, innovation, and adaptability. This article synthesizes insights from Agile methodologies, “Team Topologies” by Matthew Skelton and Manuel Pais, and “Project to Product” by Mik Kersten, offering a comprehensive view on enhancing team dynamics for optimal performance. Here, we delve into the essence of Agile leadership, team composition, challenges linked to dominant product management, and the vital importance of harmonizing the four work types: features, defects, technical debt, and risk.

 “Team Topologies” emphasizes the importance of intentionally designing team structures and interactions to optimize workflows and enhance the software delivery process. This approach is fundamentally aligned with the principles of Agile methodology, where the focus is on creating teams that can operate with high autonomy, clear purpose, and minimal necessary interaction to reduce cognitive load and foster innovation.

The book introduces four fundamental team types: Stream-aligned teams, Enabling teams, Complicated Subsystem teams, and Platform teams. Each serves a unique purpose within the organization, from directly delivering value to customers to providing specialized knowledge or infrastructure that supports other teams. This structured approach to organizing teams underlines the necessity of having an Agile leader who is not only versed in facilitating collaboration and alignment but also skilled in navigating the team through the complexities of modern software development environments.

Reducing the unnecessary cognitive burden on teams, a fundamental concept elucidated in “Team Topologies,” clarifies why a product manager, already juggling multiple responsibilities, cannot effectively assume the role of an Agile leader. Product managers are crucial in shaping the product vision, aligning it with market demands, and focusing on customer-centric features and growth. On the other hand, Agile leaders foster team dialogues, ensuring a comprehensive understanding of the organization’s current and future states and that of the customer and system. This ensures conversations encompassing feature development, technical debt management, quality enhancement, and risk mitigation. By adopting this approach, the team’s efforts are channeled toward sustainable and high-quality software delivery.

“Project to Product” by Mik Kersten, enriches our understanding of Agile team dynamics and the importance of Agile leadership, especially in balancing different types of work. Kersten’s book emphasizes modern businesses need to adopt a continuous product-focused value stream to avoid becoming obsolete, much like Nokia or Blockbuster did. This approach aligns with Agile principles by focusing on delivering end-to-end value to customers, underscoring the critical role of Agile leaders in facilitating this process.

Kersten introduces the Flow Framework, which models the entire organization as a value stream, highlighting four types of work: features, defects, technical debt, and risk. This framework complements our discussion by providing a structured method for balancing these work types, ensuring that teams stay focused on one at the expense of others. A dominant product manager might skew the team’s efforts towards feature development, neglecting technical debt and risk mitigation. However, with their unbiased perspective and dedication to the team’s overall health, Agile leaders are crucial for ensuring that all types of work are addressed appropriately, aligning with organizational goals and the system’s performance.

“Project to Product” also challenges outdated business and technology ideas, advocating a shift from project-oriented management to a product-centric approach. This shift is essential for fostering Agile teams that are adaptable, collaborative, and capable of delivering continuous value. The book’s emphasis on avoiding local optimization and focusing on the end-to-end value stream resonates with the Agile leadership role in facilitating a balanced approach to work prioritization.

Incorporating Kersten’s insights, it becomes evident that Agile leadership is not just about facilitating daily stand-ups or managing sprints; it’s about embracing a broader vision that aligns team structures, software architecture, and value stream architecture. This alignment is essential for creating high-performing teams that can navigate the complexities of modern software development, delivering value rapidly and effectively while maintaining the health and performance of the entire system.

The conflation of product management with Agile leadership is a common pitfall 

An overbearing product manager, assuming control over the team and dictating priorities based on a singular focus on growth, can stifle the very essence of Agile methodology. Such an approach undermines the team’s autonomy and creativity, jeopardizing the product’s long-term sustainability and quality. It creates a culture of mistrust and resentment, diminishes morale, and disrupts the collaborative dynamics essential for Agile success.

The balance and interplay among Agile leadership, product management, and technical leadership are where the magic of Agile truly happens. Agile leaders ensure that all types of work — features, technical debt, quality assurance, and risk mitigation — are balanced and prioritized according to the team’s collective understanding of organizational goals. This balanced approach, guided by Agile leadership, empowers teams to deliver high-quality products that meet customer needs while fostering an environment of continuous improvement and adaptation.

Agile leadership, like product ownership or the technical leader, is not about hierarchical control but about enabling a culture of collaboration, empowerment, and excellence. It’s about guiding teams through the complexities of software development, ensuring that every member feels valued, heard, and motivated to contribute their best. When product owners or managers question their role, Agile leaders have a compelling narrative to share — one that underscores their critical contribution to nurturing high-performing teams that are agile, resilient, and aligned with the organization’s strategic goals.

The strength of Agile software development lies in its emphasis on people, interactions, and customer-centric outcomes. By affirming the unique value of Agile leadership, organizations can ensure that their teams are efficient and productive, engaged, innovative, and aligned with a shared vision for success. This holistic approach to team dynamics and delivery sets Agile apart, making it an indispensable methodology for achieving excellence in today’s fast-paced and ever-evolving software development landscape.

Understanding our Agile team structure

Agile teams are intentionally cross-functional, comprising diverse skill sets and expertise to ensure that the team is self-sufficient in delivering value. This structure is designed to provide all the skills needed to deliver value in a team, foster collaboration, encourage shared accountability, and facilitate rapid responses to change while focusing on delivering customer value. 

Every team member contributes unique value, whether developers, quality assurance specialists, or product managers. While we require various specific roles, no single role outweighs the importance of others. Instead, Agile thrives on peer-to-peer interaction, underscoring the importance of every team member being seen as equals. This egalitarian approach eliminates silos, encourages open communication, and leverages the collective wisdom of the team to overcome challenges and seize opportunities.

The ecosystem of roles

In a high-performing department, Agile teams, renowned for their collaborative approach, typically consist of developers, quality assurance specialists, product managers, and Agile leaders like Scrum Masters. This diverse structure promotes autonomy and swift adaptation to change, highlighting peer-to-peer engagement and transparent communication. The power of Agile lies in breaking down barriers and harnessing shared expertise.

The Agile Leadership role

At the core of the Agile framework lies the Agile leader. This role can be a topic of debate, especially when compared to dominant product management. The Agile leader is not another cog in the wheel of task execution but a pivotal force that empowers the team to reach its full potential. Unlike product managers, who focus on market demands and product features, Agile leaders bring unique skills to nurture team health, streamline processes, and uphold Agile values. The Agile leader operates from a vantage point of neutrality, emphasizing team health, process efficiency, and adherence to Agile principles. They act as the team’s compass in navigating work complexities, stressing collaboration, team coherence in four types of work, problem-solving, and continuous improvement. 

  1. Delivery efficiency and optimization: Measuring flow (delivery efficiency) and realization (value impact) ensures you’re building the right things and building them efficiently. Operational efficiency improvements must focus on both. As Value Stream Management subject matter experts, agile leaders help the team improve delivery efficiency and flow. They can help the teams identify bottlenecks and waste and improve flow. They analyze the data and bring the conversations to the team to understand bottlenecks and experiment with improvements. They challenge product managers or owners to define and track anticipated outcomes to avoid misalignment with broader goals, championing the importance of feedback loops in verifying value realization and “closing the loop.”
  2. Facilitation Skills: Agile leaders are trained in facilitation techniques to conduct effective meetings, workshops, and Agile ceremonies. This ensures that team interactions are productive, engaging, and aligned with Agile principles.
  3. Coaching and Mentoring: A significant part of Agile leadership training involves coaching and mentoring skills. Agile leaders learn how to coach team members, foster a growth mindset, and facilitate personal and professional development within the team.
  4. Conflict Resolution: Agile leaders are equipped with conflict resolution techniques to navigate and mitigate interpersonal conflicts within the team. This training is crucial for maintaining a collaborative team environment.
  5. Change Management: Agile transformation often involves significant changes in team dynamics and organizational culture. Agile leaders are trained in change management strategies to smoothly transition teams to Agile ways of working, ensuring buy-in and minimizing resistance.
  6. Neutrality: The Agile leader is the team’s compass, steering the group through the complexities of project delivery without bias towards any specific type of work. They aim to ensure the team can perform at its best, irrespective of the work type—new features, technical debt, bug fixes, or risk mitigation.
  7. Removing Impediments: Agile leaders are skilled in identifying and removing obstacles that hinder the team’s progress. Their sole incentive is the team’s success, which allows them to act impartially and decisively in addressing challenges that impede efficiency and productivity.
  8. Promoting Agile Values and Practices: Through coaching and mentoring, Agile leaders instill and reinforce Agile values within the team, promoting practices that enhance collaboration, continuous improvement, and customer satisfaction. They are the custodians of Agile ceremonies, ensuring that these practices are meaningful and contribute to the team’s growth and success.
  9. Ensuring Collaboration and Consensus: In fostering a collaborative team environment, Agile leaders ensure that decisions are made collectively, with input from all team members. This consensus-building approach ensures alignment with organizational goals and harnesses the team’s collective expertise.

On the flip side, the role of the product manager or owner is distinct yet crucial. They act as the link between customer needs, business goals, and the development team. Their main aim is the product’s success — establishing the vision, deciding on features, and ensuring deliverables meet market demands. While focusing on product and growth is key, it can sometimes narrow the perspective by prioritizing feature development over critical aspects like addressing technical debt, ensuring product quality, and managing risks.

Conclusion

Your choice is how you organize your teams, considering that context varies and there is no one-size-fits-all approach. I acknowledge the importance of the Agile Leadership role in propelling our team to success. The alignment among Agile leadership, product management, and technical proficiency is crucial for top-tier Agile teams. Agile leaders foster team engagement and advancement by leveraging data insights, assessing efficiency metrics, handling team dynamics, promoting team coherence on objectives, and facilitating necessary discussions for teams to align with current requirements and establish work priorities. Collaborating, embracing shared responsibility, and managing diverse work types are vital for team efficiency and focusing on delivering sustainable, high-caliber software solutions. Making decisions on features, defects, technical debt, and risks in line with organizational objectives is essential for the overall health and performance of systems and solutions.

Related Posts

Poking Holes

I invite your perspective to analyze this post further – whether by invalidating specific points or affirming others. What are your thoughts?.

Let’s talk: phil.clark@rethinkyourunderstanding.

Unpacking The State of Value Stream Management Report 2023

by

3 min read

VSM Adoption

In December 2020, I introduced Value Stream Management (VSM) and Flow Metrics to our teams. Our journey of adoption has been a continuous learning experience, drawing insights from the Value Stream Management Consortium and other industry leaders as we exchange experiences. Over the past three years since our initial adoption, VSM has made significant strides in the industry.

VSMC State of Value Stream Management Report

Since 2021, The VSM Consortium1 annual State of Value Stream Management Report2 offers a comprehensive overview of the recent evolution in this field. This year’s report covers valuable insights that deserve careful analysis and distillation. I’d like to highlight three aspects that have captured my attention regarding measuring and prioritizing value and the accompanying advice provided.

Measuring Value:

  1. Value Reporting Lines Matter (slide 19): Teams reporting to CEOs indicate higher Performance and a closer connection between software engineering teams and the rest of the organization.
  2. Value Metrics (slide 20): Organizations prioritize business over customer metrics. Typical metrics include sales/revenue, active users, feature usage, NPS, and profit margin.
  3. Key Practices (slide 21): The report highlights the importance of measuring the value achieved, taking into account observability and customizing metrics based on the nature of the work.

Leading with Value:

  1. Visualizing the Entire Organization as a Value Stream Network is Key (slide 25): According to the data, one-third of the respondents have visually documented their organization as a network of interconnected value streams. The findings indicate that higher-performing organizations and larger companies are inclined to perceive their organization as a value stream network.
  2. VSM Implementation Roadmap (slides 23-24): VSM practices have evolved, and organizations can use an implementation roadmap to track progress. The “Connect” step has experienced significant growth. Respondents who complete the ‘Connect’ steps are 5-10% likelier to achieve a lead time of less than one week. Conversely, those who haven’t completed any roadmap steps are over three times as likely to have the longest lead time, exceeding three months, compared to those who have completed any step—and four times more likely than those who completed the Connect steps. Next is the inclusion of the Assess stage in the roadmap this year. Establishing a baseline of the current state when starting a journey and measuring progress as an organization progresses is important. This new step provides a comprehensive understanding of the journey and enables effective progress monitoring.
  3. Organizational Performance (slide 26): VSM adoption drives organizational Performance by aligning teams, making work visible, and using insights for continuous improvement.

Advice Section (slide 27):

  1. For Change Agents: Emphasizes the importance of visualizing work, taking data-driven actions, focusing on value realization, and understanding that flow and value realization are equally important.
  2. For CXOs: Advocates for aligning teams, visualizing value stream networks, connecting product management and software engineering, and adopting a data and insight-driven approach.
  3. For Product Creators and Software Engineering: Encouraging collaboration between product creators and software engineers, adopting VSM as a framework, and having a dedicated value stream lead (something we will consider for 2024).

Final Thought

This report sheds light on the evolving world of Value Stream Management, its impact on organizational dynamics, and its importance in the broader business landscape.

The adoption of Value Stream Management (VSM) is gaining momentum, and organizations must stay current with this trend. The VSM Consortium is a fantastic hub for VSM knowledge, education, and assistance. It’s an invaluable resource for anyone looking to dive into the world of VSM!

References:

  1. Value Stream Management Consortium, https://vsmconsortium.thinkific.com/
  2. The State of Value Stream Management Report 2023, https://www.vsmconsortium.org/state-of-value-stream-management-reports

Poking Holes

I invite your perspective on my posts. What are your thoughts?.

Let’s talk: [email protected]

Developer Experience: The Power of Sentiment Metrics in Building a TeamX Culture

by

6 min read

“If you are in a good culture, you will feel and know it, and it’s sometimes hard to put words on those things.” ~ Wayne Crosby

What is the objective of our focus on Developer Experience? We aim to address various aspects, such as enhancing efficiency, encouraging collaboration, boosting job satisfaction, improving output quality, and fostering innovation and creativity.

The Buzz Around Developer Experience

There have been so many publications on this topic lately. Google “developer experience,” and it will return a list of links to DevX definitions, examples of DevX teams, and frameworks.

DevX is a new spin on prioritizing the investment in people and ways of working. I recall every presentation emphasized a people-first culture four years ago. Still, lately, there has been a surge in the number of articles published about developer experience (a.k.a. DX, DevX, and DevEx).

Why is developer experience becoming more prevalent?

During the previous waterfall and project-based software delivery practices, some have argued that developers were treated like a resource from a factory line. They were often referred to as “resources” by the business, measured by their code output and utilization. It’s great to be recognized as a human being and feel engaged and valued. But do the attributes of DevX apply only to developers or possibly many others on a delivery team? In many cases today, cross-functional delivery teams are delivering value.

I have spent much of my career as a software developer and manager of software development teams, my contributions and output have measured me, and I have measured others similarly. I have worked with previous cultures, tools, and practices, and today’s tools, architectures, and ways of working. More than anyone else, I can appreciate the message and focus on the developer experience.

Attributes of Developer Experience

The term “developer experience” refers to the experience of developers as they do their everyday work, including any difficulties they may encounter.

The attributes of developer experience (DevEx, DX) are as follows:

  1. Perception of the development infrastructure: How developers perceive the technical infrastructure (e.g., development tools, issue trackers, programming languages, cloud platforms, libraries) and ways of working (e.g., working agreements, processes, and methods)​.1
  2. Feelings about work (happiness and engagement): How developers feel about their work, including whether they feel respected, care about it, and feel like they belong in their team.1
  3. Value of work (purpose and success): How developers value their work, including whether they feel they’re making an impact and whether their values and goals align with the company​.1

In addition, a fourth attribute, Onboarding and investment in upskilling: Is how developers value an organization or department that prioritizes the onboarding process for new members and invests in their ongoing skills development.

Here are a few of the initiatives that are driving the developer experience:1

  1. Reduce developer wait times and interruptions
  2. Invest in maintaining a healthy codebase
  3. Make deployments safe and fast
  4. Empower teams
  5. Optimize for high work engagement

Developers with high work engagement exhibit persistence, dedication, and a commitment to delivering quality software. They proactively support the organization and consistently produce excellent work when they have the tools, autonomy, mastery, purpose, and a sense of success.

Success Comes From The Team and Team Experience

As of 2023, many organizations have significantly invested in transforming their ways of working through culture, Agile, Lean, DevOps, and cloud technologies. They invested in DevOps and Platform teams that build the capabilities for teams to improve software delivery and the developer experience. It still takes a team to deliver software today. What is so different about developer experience versus quality assurance experience, agile leadership experience, or product owner experience? We should expand the message to Team Experience (TeamX).

I recognize and respect software developers’ specific type of work; it is knowledge work, so developer experience must be acknowledged. However, we need to expand the focus to the delivery team experience, which includes developer experience.

  • What if the Quality Assurance Engineer could spin up an ephemeral test environment to test changes and have innovative tools and ways to run performance, exploratory, and chaos testing?
  • What if product owners could press the “delivery” trigger in an evolved, highly confident continuous delivery pipeline to deliver features to production or review features in an ephemeral environment?
  • Why would we ignore the agile leaders’ need for tools to facilitate team building, retrospectives, sentiment analysis, cycle management, and more?

Most of the “developer experience” aspects relate to the other team roles on a cross-functional team and the team’s overall experience. Therefore prefer to focus on team experience and promote that “teams and team members with high work engagement exhibit persistence, dedication, and a commitment to delivering quality software. They proactively support the organization and consistently produce excellent work when they have the tools, autonomy, mastery, purpose, and a sense of success”.

Treating all workers with respect is important, but for creative work to thrive, a supportive environment must also be provided. I will continue to advocate for team experience (TeamX, TX) over developer experience (DevX, DX), and that developer experience is part of team experience.

Unlocking the Potential of Metrics

As a follow-up to my first post on modern-day metrics, “Finally, Metrics That Help: Boosting Productivity Through Improved Team Experience, Flow, and Bottlenecks,” this post highlights the exciting combination of modern-day insights available today. These insights come from both your delivery systems and the team’s sentiment.

Measuring team experience requires both delivery efficiency (system metrics) and team feedback (sentiment metrics).

System metrics: I have become an evangelist and promoter of today’s system metrics and data insights based on value stream management, the Theory of Constraints, and a mix of flow metrics and DORA metrics as a holistic workflow and measurement to accelerate efficiencies and product and portfolio delivery.

Sentiment metrics: Since 2022, I have increased my focus on sentiment frameworks like the SPACE framework2 and, more recently, the DevEx framework created by Abi Noda, Margaret-Anne Storey (author of SPACE), Nicole Forsgren (creator of DORA), and Michaela Greiler (previously Microsoft Research).3

I have learned that it is not uncommon for organizations to start with system metrics and then realize they can benefit from targeted frequent sentiment metrics.

One unique thing about my experience at my current organization is that in addition to a semi-annual organization-wide employee net promoter score type survey (eNPS), we have been collecting simple team sentiment over many years using a Google Sheet, wherein each team member’s sentiment is recorded at the end of their daily standup comments: How are you feeling today? Positive, Negative, or Neutral.

Wanting to expand on our sentiment feedback, we are looking into creating short, consistent, and frequently delivered surveys in-house using existing tools that provide us with this capability or investing in services with significant experience in this area and the types of questions that bring the best results. As we still learn to master value streams and system flow metrics, we must expand and invest in our sentiment metrics.

Final thoughts

Creating and delivering digital products is currently an exciting field. Modern delivery practices, methodologies, and innovative measurement techniques bring positive changes. Two types of data analysis are necessary to evaluate team effectiveness and happiness: delivery system metrics (such as Flow and DORA) and sentiment metrics (measured through surveys).

To remain competitive and succeed in today’s business environment, software delivery organizations must update their delivery practices and adopt modern system metrics and sentiment measurements.

Poking Holes

I invite your perspective on my posts. What are your thoughts?

Let’s talk: phil.clark@rethinkyourunderstanding.

References

  1. Ari-Pekka Koponen (28 February 2023), The ultimate guide to developer experience, swarmia.com, short URL: bit.ly/468g0q2
  2. Nicole Forsgren, Margaret-Anne Storey, Chandra Maddila, Thomas Zimmermann, Brian Houck, and Jenna Butler (06 March 2021), The SPACE of Developer Productivity: There’s more to it than you think, queue.acm.org, https://queue.acm.org/detail.cfm?id=3454124
  3. Abi Noda, Margaret-Anne Storey, Nicole Forsgren, and Michaela Greiler (03 May 2023), DevEx: What Actually Drives Productivity: The developer-centric approach to measuring and improving productivity, queue.acm.org, https://queue.acm.org/detail.cfm?id=3595878

Outcome Metrics and the Difficulty of Reporting on Value

by

4 min read

What does it mean to “deliver value”? Defining value deserves its own focus. This article picks up at the point of the delivery backlog, assuming that your product leadership has identified the customers’ or organization’s needs, prioritized, defined, and outlined the value for the business and its customers, created a business case for the investment (including impact mapping and cost analysis) and defined the expected outcomes from changes or improvements to their digital product.

What problem are we trying to solve?

The outcomes are not kept from the teams, ensuring we are closing the loop.

This article dives into the crucial topic of measuring the outcomes following the release of enhancements or changes and informing the team(s) that delivered the work. Did the change or new feature deliver the expected value? Are we delivering the right things? Knowing the outcome or level of success motivates team members and bolsters their purpose. Teams can use the results to glean valuable insights even when they do not meet expectations.

Fast and agile delivery is not the end goal; value is the end goal

“Making the wrong thing faster only makes us wronger.” 1

In software delivery, it is essential to remember that delivery is not the end goal; value is. It is easy to fall into the trap of delivering software quickly and efficiently. Still, it is all for nothing if it does not provide value to the customer or organization. Delivering unwanted features can be a sad waste of productivity and a misuse of talent.1 These are just a couple of reasons why it is crucial to understand what value means in software delivery and how to measure it.

Organizations need to understand the real-world impact of their digital product changes, so measuring its outcome value, determining the return on the investment, and learning from outcomes are critical. Unfortunately, accurately tracking and reporting outcomes and value returned can be complex due to several challenges.

The challenges of measuring the final outcome of digital changes

What are the meaningful outcome metrics? Are such metrics communicated down to the delivery team level? Do companies practicing OKRs report on the final outcome of those OKRs?

First, many organizations need more tools to help them measure the value of outcomes from software delivery. The lack of tools and data insights can make it challenging to track and report on the success of the delivered changes.

Secondly, measuring the actual ROI requires significant time and effort. It is essential to determine the impact of digital product changes on the business or customer; this can be a complex process. This work may require additional resources, like data analysts or business intelligence tools.

Third, the impact of the software changes may take time to become apparent. It might take months or even years to see the actual effect of the changes delivered on the business or customer. Time duration can make it challenging to accurately track and report the real degree of success or value delivered within the allotted time to influence teams.

Fourth, accounting for the success of an outcome and the value it returns may require additional resources and a shift in the organization’s mindset to prioritize measuring this work.

Finally, there could be pushback when inquiring about the value of the product or platform changes teams delivered. To ensure that the value outcome is consistently tracked and reported, organizations must determine who is best suited for monitoring and reporting the value outcomes of what the teams deliver.

Teamwork and transparency at the team level

For those using Scrum or Kanban or similar lifecycle practices and tools, consider adding elements to the delivery team’s Epics, Features, and possibly User Stories.

Why: Why are we working on this?

Value: Short description of the expected outcome for the organization or customers.

These can align with OKRs for those using them.

Benefits:

  • Shared understanding, alignment, purpose, driven development, and delivery.
  • Documenting the why and value enables team alignment and autonomy and increases team member engagement.
  • A more precise understanding of priority reasoning.
  • Learn from the outcomes (gain insights).

Challenges:

  • Tools to help measure the outcome.
  • Measuring the outcome requires significant time and effort.
  • The impact of the change(s) delivered may take time to become apparent, ranging from weeks to months or even longer.

Closing the loop with the delivery teams:

  • Schedule outcome retrospectives with teams.
  • Document the outcome(s) details to Jira, Azure DevOps, Rally, or whatever tool your teams use.

Final thoughts

In many organizations, technology leadership must measure and report on the performance of the software delivery teams.2 Do your delivery teams receive feedback on their work’s value to the organization or customer? Are they aware of the impact and success of their efforts after they deliver on a change? If not, it’s time to reevaluate your approach.

By providing your teams with regular feedback and, more importantly, the overall results or outcomes from their work, you can increase their motivation and a sense of purpose, leading to a more engaged and productive workforce.

If you aren’t doing so already, start tracking, measuring, and reporting on your team’s outcomes to align your business objectives and change investments with their performance, avoid costly and wasteful overproduction, learn from the changes made to your delivered product, and achieve greater success. Are your teams “delivering value”?

Related articles:

  1. Value part 1: Maximizing Technology Team Effectiveness: Insights from a CEO Conversation
  2. Measuring delivery teams: Finally, Metrics That Help: Boosting Productivity Through Improved Team Experience, Flow, and Bottlenecks.

References:

  1. Smart, Jonathan [@jonsmart]. “From Faster to Sooner” Twitter, 26 June 2021

Poking Holes

I invite your perspective to analyze this post further – whether by invalidating specific points or affirming others. What are your thoughts?.

Let’s talk: phil.clark@rethinkyourunderstanding.