Rethink Your Understanding

Transforming Software Delivery

Engineering

Pressures of Strategic Talent Cost Rebalancing in Agile Teams: Optimizing Global Geographical Costs for Cohesion and Efficiency

by

10 min read

“If you choose not to leverage global talent and economic benefits while your competitors do, you’re essentially steering your business towards obsolescence.”

adaptation of Lee Kuan Yew’s quote regarding outsourcing

Updated May 15, 2024:

I published this article in February 2024. Recently, a senior leader in my network sought help addressing concerns about his request for a “second shift”—working hours aligned with US time zones for talent located 7 to 9 hours ahead to shift talent in two of his department’s cross-functional delivery teams.

His broader organization informed this leader that finding talent in this region willing to work off-hour shifts to align with US hours is challenging. This leader’s goal is eventually to relocate the entire cross-functional teams to a more cost-effective region, which is also my recommendation. However, given current conditions, he shifted only specific roles from existing teams to the new region (specifically software developers). These team members will need to work off-hours that match US working hours. I do not recommend this approach and believe entire teams should be in locations that foster collaboration. Still, for his specific case, I revisited my article to create a response to his organization’s concerns. 

Here is the message he sent me for help with his response. I changed his name and location to keep him anonymous.

Hi Mark,

Questions are coming up about why we need complete overlap between County/Region X and the United States instead of just a few hours. I’d like to explain why our development process differs from Organization Y (our parent organization). Please send us a justification to help get off-hours approved. Please get it to us by Monday.

What do you think about having a 2-hour overlap instead of the full 8 hours? Also, if we move one or two managers over there, can they operate independently on that country’s/region’s time? Please include this in the write-up as well.

I sent him the following response summarized from this article:

For teams not in the same location or close time zones, without delving deep into organizational design, we have two key structures impacting software delivery: functional teams, organized by specialized skills, and cross-functional teams. Our approach leverages the benefits of the cross-functional team structure.

These small, cross-functional units comprise the minimum roles required to design and deliver software with maximum autonomy, minimizing reliance on external teams or resources and reducing dependencies.

For a cross-functional team to succeed, team members must communicate well, collaborate effectively, and be available. Even though members work independently, teams in different locations should have at least three or four hours of overlap to create a successful working environment.

Team rituals that rely on real-time collaboration may suffer when impacted by time differences. Question and answer sessions during planning, grooming, design reviews, code assessments, or pair programming are crucial for maintaining high code standards and fostering knowledge exchange within agile teams. Substantial time variations to facilitate additional asynchronous collaboration can pose unique challenges:

  • Delayed Code Review
  • Slower Issue Resolution
  • Planning meetings, Design Meetings, Reviews, and Retrospectives
  • Mentoring, Pair Programming, and Swarming

Delays in collaboration and resolving issues can extend delivery times and lower team efficiency, especially for US-based team members collaborating across different time zones. Overlapping working hours can prevent longer delivery schedules and downtime for team members waiting on feedback or approvals, slowing the team’s response to customer needs and market changes.

Original article

This article assumes that your organization has transformed team delivery and design practices. Instead of functional teams based on skills, it focuses on small, cross-functional, and potentially long-lived teams. There are various variations of these team topologies.

Experience

During my 24-year career in software and technology, I’ve seen organizations try to achieve cost synergies by outsourcing and near-shoring. However, due to delays and project management issues, they often bring the talent back in-house, only to reconsider outsourcing in the future due to the higher costs associated with US-based software engineers.

Recently, I discussed dealing with similar pressures with someone in my network. The organization is pressured to achieve specific cost savings by replacing a set number of higher-cost team members with more cost-effective team members from different, more cost-effective locations.

In the changing global business world, larger entities’ acquisition of US-based software companies has made operational cost optimization a critical strategic planning focus. This emphasis on cost synergy actions and efficiency has prompted considerations of balancing global talent, aiming to take advantage of the financial benefits of staffing in more economically favorable locations. At first, the senior leadership suggested a basic concept: if teams in the US face attrition, they would replace the team members with more cost-effective members globally.
This article delves into the challenges of a strategy where your team structure shifts to small cross-functional teams. It suggests a nuanced approach focusing on relocating entire teams to uphold agile principles. The discussion extends to impacts on code review, collaboration, and delivery efficiency.

Reevaluating the Initial Strategy

The senior leadership’s proposal to fill vacancies left by US team members with individuals from cost-effective locations across significantly different time zones emphasizes the potential risks to team dynamics, collaboration, and delivery efficiencies. While cost-effectiveness is crucial, this approach neglects the importance of cohesive, immediate interaction and a shared sense of purpose that fuels agile teams. You risk losing the agility to adapt swiftly to changes in the ecosystem, priorities, or market dynamics, depriving yourself of the ability to be responsive and noble. This inherent delay contributes to increased costs and time constraints.

The Challenge of Integrating Global Talent

“The question is really about the bandwidth of human interactions.”

Brian Graham

The following applies to organizations with entities in more cost-favorable countries or organizations with an outsourcing model. More and more outsourcing service providers are offering entire agile teams.

The main concern in global talent rebalancing lies in something other than remote work, as both companies already have remote solid cultures. Instead, the potential disruption to team bonding, daily synchronous collaboration, and delivery efficiencies are paramount. Introducing team members from different time zones with an 8- to 12-hour difference can significantly impact these critical aspects of software development, especially for agile teams known for their cohesive collaboration and quick turnarounds.

Small cross-functional delivery team model examples:

Impact on Team Rituals and Collaboration

Team rituals that rely on real-time collaboration may suffer when impacted by time differences. Question and answer sessions during planning, grooming, design reviews, code assessments, or pair programming are crucial for maintaining high code standards and fostering knowledge exchange within agile teams. Nevertheless, overcoming the hurdles posed by substantial time variations to facilitate additional asynchronous collaboration can pose unique challenges:

  • Delayed Code Reviews: Conducting reviews is crucial to uphold coding standards and ensure code quality. However, the asynchronous nature of these reviews across different time zones can result in longer lead times for pull requests, ultimately slowing down the development process.
  • Slower Issue Resolution: The ability to quickly address and rectify discovered issues is compromised, extending the feedback loop and potentially allowing defects to persist longer than necessary.
  • Planning meetings, Design Meetings, Reviews, and Retrospectives: Team members need to have overlapping working hours to ensure they get all the benefits of real-time planning discussions, design meetings, and retrospectives. While some of these can be handled asynchronously using tools like Slack or a wiki page, the collaboration could be more effective, and there may be delays in responses.
  • Mentoring, Pair Programming, and Swarming: The option to engage in pair programming or swarming to address intricate problems or explore new tasks may no longer be available.

Implications for Delivery Time and Team Efficiency

Delays in collaboration and issue resolution can lead to extended delivery timelines and reduced team efficiency, especially for United States-based team members working with counterparts in different time zones. Misaligned working hours may result in prolonged delivery schedules and downtime for team members awaiting feedback or approvals, hampering the team’s responsiveness to customer needs and market dynamics.

Considering the Argument for Lower Labor Costs

Cost synergies are highly prized in an offshore model for several reasons. The underlying belief is that they have the potential to result in equivalent delivery performance and reduced perceived costs compared to anticipated costs. When considering employees versus outsourcing to a service provider, managing balance sheets using contractors is often simpler due to the perception of them as variable costs. Contractors may appear more advantageous when examining the fully burdened cost associated with employees. However, a common mistake is expecting contractors or service providers to work as fast and effectively as employees, which is usually different based on my experience and several others in my network. Furthermore, costs can come up due to time-sliced individuals, such as team leads serving as a go-between between you and the delivery teams.

Emotional issues and cultural nuances can have a profound impact on team unity. At DHL, I gained invaluable insights into the power of globally diverse teams working in similar time zones, specifically the Integration and EDI teams operating during US hours.

I shared this article draft for feedback with Bob Langan, a former senior leader at DHL between 2003 and 2014, now retired, and a trusted mentor who offered feedback on the challenges posed by the “follow the sun” model and subsequent offshore/onshore approaches. Despite the higher costs of US employees’ salaries, we often worked far more than the standard 40-hour workweek. We also had fewer national/state holidays and fewer annual vacation days. In contrast, European and Asian colleagues tended to work fewer hours weekly as a general practice, taking longer to achieve the same results as the US teams. DHL Delivery teams charged the commercial business units for their time on a daily rate defined by resource (annual salary / potential person-days). Through analysis, Bob discovered that US teams were frequently as, if not more, cost-effective in terms of cost of delivery due to fewer person-days being consumed and many more potential person-days to work. While this perspective may be different from your current considerations, it was crucial in advocating for maintaining a US team in the initial phases.

The discussion of reduced expenses is beyond the scope of this article. Times have changed. Please feel free to do your research on the current data trends.1,2

Emphasizing a Cohesive Relocation Strategy

Key point: Make a strategic shift to maintain team unity and increase flexibility. Instead of spreading out team members across different time zones, moving entire teams to more cost-effective locations can be a better solution. This approach emphasizes anticipated financial savings while maintaining important team dynamics for agile success. 

I’ve seen the benefits of having teams in different time zones at DHL and my current company. In one team, a member adjusts to a 12-hour time difference, following US hours for years. Dedication is vital when working opposite hours from your usual life. Another agile team has developers in Eastern Europe with overlapping work hours of 3 to 4 US hours. Providing support and proper management for these teams is crucial. Organizations may consider team relocations to align with services or products globally.

Strategic Recommendations

The following strategic recommendations address this nuanced understanding of talent rebalancing, with a renewed focus on:

  1. Prioritizing Team Unity Over Individual Replacement: The strategy now focuses on relocating entire teams instead of individually back-filling positions. This approach aligns with cost-saving goals and the need to maintain effective teamwork within agile frameworks.
  2. Maintaining Agile Integrity Through Cohesion: Effective work schedules, improved collaboration (both asynchronous and synchronous), and strengthened team integration efforts are essential to uphold the core values of agile methodologies and efficient value delivery even in geographical changes.

Conclusion: Reframing the Approach to Global Talent Rebalancing

My primary focus is understanding how distributing team members across different time zones affects team dynamics and effectiveness. This article discussed the cost-saving debate in labor across various global geographic locations, where labor costs and talent frequently shape global redistribution strategies.

The initial idea proposed by senior leadership to replace departing US team members through attrition with more cost-effective global talent highlights the delicate balance between financial efficiency and preserving essential team dynamics for agile success.

As a technology leader in the US and a former software engineer, I prefer something other than outsourcing these positions to other countries. However, in my role and a globally competitive market, it’s important to endorse this approach. It’s a challenge I’ve faced, managed, and supported at times for over two decades.

If you or your organization is facing these demands, I suggest a comprehensive approach to talent rebalancing. This approach focuses on relocating and rebalancing agile cross-functional delivery teams based on geography and overlapping time zones rather than individuals. By prioritizing team cohesion and geographical consolidation, organizations can meet the financial demands of global business while sustaining innovation, efficiency, and competitive advantage in software development. This balanced approach ensures operational cost optimizations without compromising the dynamic interplay and shared commitment that drives high-performing agile teams, preserving their collaborative essence and productivity.

References:

  1. Pete Grieve; Americans Work Hundreds of Hours More a Year Than Europeans: Report, https://money.com/americans-work-hours-vs-europe-china/, Money, January 06, 2023.
  2. Charlie Giattino, Esteban Ortiz-Ospina, and Max Roser; Working Hours, https://ourworldindata.org/working-hours, Our Wold in Data, revision December 2020.

Poking Holes

I invite your perspective on my posts. What are your thoughts?.

Let’s talk: phil.clark@rethinkyourunderstanding.com

Shift Left Security, Security Unit Tests, OWASP Top 10, and AI: Key Practices for Secure Development

by

6 min read

What are we trying to improve? The adoption of practices to find security vulnerabilities early in the development lifecycle.

What outcome do we hope to achieve? Additional security coverage, where applicable, earlier in the software development lifecycle.

Let’s Shift Left

Are you familiar with the term “shift left”? It is a popular concept in the tech industry for good reasons. I define shift left as enabling the earliest feedback. It’s about determining if your code modification is functioning as intended and detecting any potential damage to pre-existing code as soon as possible. 

Why should we postpone identifying an issue until the last minute? The cost of detecting an issue increases as it is detected later in the delivery flow (cost is a whole other conversation). For us, we started shifting left for quality, running tests at all levels, and moving from depending on extensive, long-running tests in staging or production environments to reducing the number of these tests and replacing them with tests earlier in the flow (shifting the tests left). Several libraries support unit test implementation in most languages. Our journey started several years back with Martin Fowler’s article on the practical test pyramid1 and adopting the practice of test-driven development (TDD).

Security Shift Left Mindset

Security has become indispensable to our work in the evolving technology and software development landscape. It’s no longer just about developing features but ensuring they are secure and reliable for our users. Significant improvements have been made at the platform and systems levels. Here’s the kicker: what if we did the same “shift-left” approach for security? Developers can access helpful tools such as profilers, static analysis, and dynamic analysis scanners. The objective is to identify security issues quickly alongside quality and defects. Why not make security-based unit tests a core practice of your team?

OWASP Top 10: Key Practices for Secure Development in the coding stage

We want to ensure our code is secure and get feedback early. One way to do this is to follow the OWASP Top Ten2 security risks while writing and compiling our code. We can use unit tests to help prevent these risks from happening.

1. Injection (OWASP A1): Create input validation tests to mitigate injection flaws.

// Java
@Test 
public void testSqlInjectionVulnerability() { 
  String maliciousInput = "1'; DROP TABLE users; --";  
  assertFalse(isSqlInjectionSafe(maliciousInput)); 

}

2. Broken Authentication (OWASP A2): Develop tests to verify session management and authentication.

// Java
@Test 
public void testSessionExpiration() { 
  User testUser = new User("Test User"); 
  Session testSession = new Session(testUser); 

  Thread.sleep(MAX_SESSION_TIME + 1); 
  assertFalse(testSession.isValid()); 
}

3. Sensitive Data Exposure (OWASP A3): Formulate tests to prevent inadvertent data leaks.

// Java
@Test 
public void testDataLeak() { 
  User testUser = new User("Test User", "password"); 
  Logger testLogger = new Logger(); 

  testLogger.log(testUser);
  assertFalse(testLogger.containsSensitiveData()); 
}

4. XML External Entity (XXE) (OWASP A4): Test XML parsers for correct configuration.

// Java
@Test 
public void testXXE() { 
  String maliciousXML = "..."; // some malicious XML   
  assertThrows(XXEException.class, () -> parseXML(maliciousXML)); 
}

5. Broken Access Control (OWASP A5): Assert appropriate access levels for different user roles.

// Java
@Test 
public void testAdminOnlyAccess() { 
  User testUser = new User("Test User", Role.USER); 
  Resource restrictedResource = new Resource("Restricted Resource"); 
  assertThrows(AccessDeniedException.class, () -> restrictedResource.access(testUser)); 
}

6. Cross-Site Scripting (XSS) (OWASP A7): Implement tests to check how the application handles untrusted data.

// Java
@Test 
public void testXSSVulnerability() { 
  String maliciousInput = "<script>alert('XSS');</script>"; 
  assertFalse(isXssSafe(maliciousInput)); 
}

Other examples for the user interface (JavaScript)

1. Cross-Site Scripting (XSS) Protection: To prevent XSS attacks, you should test that your rendering function properly escapes user input.

describe('XSS Protection', () => {
  it('should escape potential script tags in user input', () => {
    const userInput = '<script>alert("xss")</script>';
    const escapedInput = escapeUserInput(userInput);
expect(escapedInput).toEqual('&lt;script&gt;alert("xss")&lt;/script&gt;');
  });
});

2. Injection and Input Validation: Confirm your software correctly validates the user input and prevents SQL injection.

describe('Input Validation', () => { 
  it('should invalidate input containing SQL Injection attempt', () => { 
    const userInput = "'; DROP TABLE users; --";
    expect(isInputValid(userInput)).toBe(false); 
  });
});

3. Authorization/Access Control: Ensure certain UI elements are accessible only to authenticated or authorized users.

describe('Authorization', () => { 
  it('should not show the admin button for non-admin users', () => { 
    const user = { isAdmin: false };
    render(<Dashboard user={user} />);
    expect(screen.queryByText('Admin Panel')).not.toBeInTheDocument();
  });
});

4. Token Handling: Verify that authentication tokens are stored and handled securely.

describe('Token Handling', () => { 
  it('should not store tokens in localStorage', () => {
    setAuthToken('exampleToken');
    expect(window.localStorage.getItem('authToken')).toBeNull();
  });
});

Check out this YouTube video: DevSecOps wins with Security Unit Tests.3

What about AI?

As we focus on modern security practices, it’s worth touching on these efforts’ using artificial intelligence (AI) advantages.

While you can find posts and articles from earlier this year (2023) concerning the security vulnerabilities that code assistance tools like GitHub copilot can create. However, you can also find posts and articles detailing how quickly the security features of these tools are improving.4

There is no perfect solution. Still, tools like copilot can learn from past incidents, analyze patterns, and predict vulnerabilities. They can generate test cases based on software behavior and suggest edge cases to enhance our unit testing process. These tools can enhance security unit tests. Machine learning models can be trained on numerous secure and insecure code examples, predicting whether a new piece of code might contain security vulnerabilities based on patterns they’ve learned. These tools can flag potential security issues as developers write code, providing immediate feedback and opportunities for learning.

AI can assist in static and dynamic security testing. One example is that AI can help with the time-consuming task of sorting through false positives in static code analysis results. Additionally, AI can identify patterns in code that humans may overlook and point out areas that require further examination. In dynamic analysis, AI can help mimic the actions of real users by interacting with the software as humans would, finding vulnerabilities that manual testing might not uncover. The continuous learning process of AI models also ensures that our testing procedures will evolve alongside new threat patterns.

Wrapping this up

While the OWASP Top Ten may not encapsulate all possible security issues, it provides a robust foundation for our security practices. Incorporating these tests into our daily workflow is a strategic move that will substantially enhance our products’ security. Such tests should be designed to verify that the user interface effectively implements various security measures. For teams yet to adopt this “shift left” practice, it is imperative to start integrating security testing earlier in the development process and uphold high-security standards.

To stay ahead of evolving threats and reinforce our software’s security, we should also consider the integration of AI into our security strategy. AI can enable us to identify and tackle potential vulnerabilities proactively. However, it is essential to remember that these powerful tools are intended to supplement, not substitute, our existing security practices and intuition.

Let’s focus on enhancing our security practices as we move forward. We should adopt emerging technologies like AI while keeping our main goal in mind – creating secure and reliable software for our users. By implementing strategies such as “shift left” security and utilizing the tools available to test the security of our code, we can stay ahead of evolving security threats and maintain the trust of our users.

Poking Holes

I invite your perspective to analyze this post further – whether by invalidating specific points or affirming others. What are your thoughts?.

Let’s talk: phil.clark@rethinkyourunderstanding.

References

  1. Martin Fowler (26 February 2018), The Practical Test Pyramid, martinFowler.com, https://martinfowler.com/articles/practical-test-pyramid.html
  2. Aimee, Nikki, and featuring Abhay Bhargav (26 September 2021), DevSecOps wins with Security Unit Tests, youtube.com, https://www.youtube.com/watch?v=i34Ihbuslgw
  3. OWASP Top Ten, owasp.org, https://owasp.org/www-project-top-ten/
  4. Shuyin Zhao (14 February 2023), better AI model and new capabilities, github.blog, https://github.blog/2023-02-14-github-copilot-now-has-a-better-ai-model-and-new-capabilities/

Outcome Metrics and the Difficulty of Reporting on Value

by

4 min read

What does it mean to “deliver value”? Defining value deserves its own focus. This article picks up at the point of the delivery backlog, assuming that your product leadership has identified the customers’ or organization’s needs, prioritized, defined, and outlined the value for the business and its customers, created a business case for the investment (including impact mapping and cost analysis) and defined the expected outcomes from changes or improvements to their digital product.

What problem are we trying to solve?

The outcomes are not kept from the teams, ensuring we are closing the loop.

This article dives into the crucial topic of measuring the outcomes following the release of enhancements or changes and informing the team(s) that delivered the work. Did the change or new feature deliver the expected value? Are we delivering the right things? Knowing the outcome or level of success motivates team members and bolsters their purpose. Teams can use the results to glean valuable insights even when they do not meet expectations.

Fast and agile delivery is not the end goal; value is the end goal

“Making the wrong thing faster only makes us wronger.” 1

In software delivery, it is essential to remember that delivery is not the end goal; value is. It is easy to fall into the trap of delivering software quickly and efficiently. Still, it is all for nothing if it does not provide value to the customer or organization. Delivering unwanted features can be a sad waste of productivity and a misuse of talent.1 These are just a couple of reasons why it is crucial to understand what value means in software delivery and how to measure it.

Organizations need to understand the real-world impact of their digital product changes, so measuring its outcome value, determining the return on the investment, and learning from outcomes are critical. Unfortunately, accurately tracking and reporting outcomes and value returned can be complex due to several challenges.

The challenges of measuring the final outcome of digital changes

What are the meaningful outcome metrics? Are such metrics communicated down to the delivery team level? Do companies practicing OKRs report on the final outcome of those OKRs?

First, many organizations need more tools to help them measure the value of outcomes from software delivery. The lack of tools and data insights can make it challenging to track and report on the success of the delivered changes.

Secondly, measuring the actual ROI requires significant time and effort. It is essential to determine the impact of digital product changes on the business or customer; this can be a complex process. This work may require additional resources, like data analysts or business intelligence tools.

Third, the impact of the software changes may take time to become apparent. It might take months or even years to see the actual effect of the changes delivered on the business or customer. Time duration can make it challenging to accurately track and report the real degree of success or value delivered within the allotted time to influence teams.

Fourth, accounting for the success of an outcome and the value it returns may require additional resources and a shift in the organization’s mindset to prioritize measuring this work.

Finally, there could be pushback when inquiring about the value of the product or platform changes teams delivered. To ensure that the value outcome is consistently tracked and reported, organizations must determine who is best suited for monitoring and reporting the value outcomes of what the teams deliver.

Teamwork and transparency at the team level

For those using Scrum or Kanban or similar lifecycle practices and tools, consider adding elements to the delivery team’s Epics, Features, and possibly User Stories.

Why: Why are we working on this?

Value: Short description of the expected outcome for the organization or customers.

These can align with OKRs for those using them.

Benefits:

  • Shared understanding, alignment, purpose driven development, and delivery.
  • Documenting the why and value enables team alignment and autonomy and increases team member engagement.
  • A more precise understanding of priority reasoning.
  • Learn from the outcomes (gain insights).

Challenges:

  • Tools to help measure the outcome.
  • Measuring the outcome requires significant time and effort.
  • The impact of the change(s) delivered may take time to become apparent, ranging from weeks to months or even longer.

Closing the loop with the delivery teams:

  • Schedule outcome retrospectives with teams.
  • Document the outcome(s) details to Jira, Azure DevOps, Rally, or whatever tool your teams use.

Final thoughts

In many organizations, technology leadership must measure and report on the performance of the software delivery teams.2 Do your delivery teams receive feedback on their work’s value to the organization or customer? Are they aware of the impact and success of their efforts after they deliver on a change? If not, it’s time to reevaluate your approach.

By providing your teams with regular feedback and, more importantly, the overall results or outcomes from their work, you can increase their motivation and a sense of purpose, leading to a more engaged and productive workforce.

If you aren’t doing so already, start tracking, measuring, and reporting on your team’s outcomes to align your business objectives and change investments with their performance, avoid costly and wasteful overproduction, learn from the changes made to your delivered product, and achieve greater success. Are your teams “delivering value”?

Related articles:

  1. Value part 1: Maximizing Technology Team Effectiveness: Insights from a CEO Conversation
  2. Measuring delivery teams: Finally, Metrics That Help: Boosting Productivity Through Improved Team Experience, Flow, and Bottlenecks.

References:

  1. Smart, Jonathan [@jonsmart]. “From Faster to Sooner” Twitter, 26 June 2021

Poking Holes

I invite your perspective to analyze this post further – whether by invalidating specific points or affirming others. What are your thoughts?.

Let’s talk: phil.clark@rethinkyourunderstanding.

Maximizing Technology Team Performance: Insights from a CEO Conversation

by

6 min read

In this article, I share an enhanced version of a conversation with a CEO in August of 2022 regarding the effectiveness of technology teams, measuring improvement, building the right things, and how understanding the purpose and value of their work can impact team effectiveness.

The Importance of Purpose and Value

Achieving success requires understanding the purpose and value of your work. If disconnected from a shared vision, it is vital to pause and reconsider. When clear on how you are adding value, teams can collaborate optimally, prioritize tasks effectively, and comprehend the importance of their efforts. Effective teams solve problems with a sense of mission while being driven by what they bring to these solutions in terms of purpose.

The Measurement Challenge

I recently talked with a CEO from my network about the challenges of measuring the productivity and performance of software engineers and teams and determining if we are making progress. Measuring developer productivity has been a persistent challenge for engineering leaders for decades.

In this discussion, the CEO, whom I’ll refer to as Steve, insisted on his VP of Engineering report on the organization’s engineering productivity. Despite being familiar with outdated metrics centered on individual output, Steve was open to new ideas. As I shared my experience working on this issue in my organization, he was interested in learning how we measure and report the return on investment of the value delivered.

Measuring Progress and Building the Right Things

To assess improvement, it’s crucial to measure both delivery performance and the outcomes of our efforts. Steve mentioned that technology is responsible for measuring delivery performance, but I emphasized the importance of the business quantifying the impact of change and communicating it to delivery teams. Product managers should assess and measure the return on investment of value delivered to stakeholders. However, measuring the ROI of work delivered and business outcomes can be challenging even for experienced leaders. During the conversation, we discussed delivery team performance, but I also wanted to explore the impact of having a clear purpose and value on delivery performance. I asked Steve about his experience measuring the impact or ROI of value delivered by teams.

I suggested: “What if a team could improve from delivering five widgets per week to eight, with the same number of team members and the same number of hours?” I asked Steve, “Did we get better?” His answer was “Yes,” but I disagreed. “How will we know that we are building the right things?” and “How will we know that we have not overproduced and wasted time?” I stated, “the additional three widgets may not provide the expected value, and we could be wasting time and effort without realizing it. Are we overproducing?” As Jonathan Smart said, “Making the wrong thing faster only makes us wronger.” This marked a turning point in the tone of the conversation.

Building the Right Things with Purpose

“People work better when they know what the goal is and why. It’s important that people look forward to coming to work.” – Elon Musk.

The conversation shifted to understanding and communicating value and outcomes rather than just focusing on output. Steve commented that it wasn’t the job of a Vice President of Engineering to question the value; this responsibility for questioning and communicating feature work value lies with the Product Manager, or more likely, the senior product leader — not the head of engineering. Not surprisingly, I took his comments personally. Changing tactics, my intention now was to tie employee engagement back to knowing the value of the work. I asked Steve about employee retention and whether it was the responsibility of the Vice President of Engineering; he agreed and added that it was the responsibility of the Engineering Senior Leader to build a great employee experience and culture.

I brought up the concept of purpose-driven development, where people are motivated by a sense of meaning and fulfillment in their work. I asked Steve about his understanding of extrinsic and intrinsic motivation and if he believed people could be driven by autonomy, mastery, and purpose beyond salary. He agreed. I took this opportunity to tie in the value of engineering teams needing to know the value. Without a shared understanding of value, teams can become demotivated and feel like they are just following orders. To avoid this, it is crucial for everyone from the executive team down to the delivery team members to understand the value of the work they are producing. This leads to more efficient work and prevents wasted energy from overproducing or creating unwanted items. The goal is to ensure that work occurs efficiently and that teams feel a sense of purpose and connection to the outcomes they deliver.

We circled back to the origin of the conversation about measuring performance. The conversation ended with the question, “How do we know that engineering delivery teams are getting better?” As a technology leader, we will measure engineering delivery performance to identify ways to continue improving (get better). My takeaway highlights the importance of understanding the value of what we are producing and how it impacts customer experience and the organization’s goals. Only then can we genuinely answer if we are making progress and delivering the right things.

Measuring the actual outcome of delivery

As colleagues serving our actual customers, the external ones, it is our responsibility as IT and Product to ensure value is delivered to the customer and organization. This helps to reduce the risk of waste through the overproduction of incorrect items and allows us to motivate and retain our team members through purpose-driven development.

Delivery teams must focus on the efficiency and effectiveness of what is being delivered. Unless an experiment is being run to determine the usefulness of a product to a customer group, delivering items of work that have no value to the customers or targeted customer population is a waste of time and energy.

Beyond senior leadership, delivery teams should engage in value-driven discussions, considering the customer’s and organization’s goals, expected outcomes, and the value that will be provided to people. This helps to ensure that the team has a clear understanding of the purpose behind their work.

The product owner is responsible for explaining the value behind each feature and how it connects to the organization’s goals. If the product owner cannot do so, they should re-evaluate their approach. Teams should not be reduced to mere “feature factories” without a clear understanding of the purpose behind their work.

Leaders must measure productivity and performance improvements and communicate the value and outcomes of what their teams are delivering. This creates a shared understanding throughout the organization and ensures that valuable work time is spent on creating truly needed things.

Team members should know how their work contributes to the organization’s performance and customer engagement, fostering a sense of purpose and engagement. This helps to avoid wasting resources in terms of money, employee engagement, and other aspects.

Unfortunately, teams often do not see the outcomes of their delivery. Measuring and reporting on these outcomes is important to determine if the right things were built and if the organization is improving.

Final thoughts

Just as IT is held accountable for measuring and improving productivity, the product team and the organization should be held accountable for communicating value and for measuring and communicating the outcomes of the delivered value. To answer the question, “Are we improving as we get bigger?” one must consider delivery performance and know the outcomes of what has been delivered (are we building the right things?). Defining value and communicating outcomes are essential to ensure everyone is aligned and working towards the same goal.

Link to the next article, value part two: Unlocking the Value of Software Delivery: Difficulty of Reporting on Value

Poking Holes

I invite your perspective to analyze this post further – whether by invalidating specific points or affirming others.

Let’s talk: phil.clark@rethinkyourunderstanding.com.

Finally, Metrics That Help: Boosting Productivity Through Improved Team Experience, Flow, and Bottlenecks.

by

13 min read

Note: This article has two parts: First, a view on today’s data insights and metrics. Second, our experience researching the tools in the market.

Data is key to improvement, but it is not just about collecting and reporting it. We must use the right data for the right purpose to improve flow and efficiency, and team health. The way that leadership approaches the metrics ultimately still has the largest impact on success. We can unleash our full potential and reach our goals by adopting a metrics-driven improvement culture and methods that provide insights to encourage and motivate teams.” ~ Phil Clark

What is great software delivery?

To be excellent at delivering software, deliver the right changes efficiently, frequently, with quality, and get customer feedback. Move quickly with the confidence and knowledge that you are building the right things with quality and security. Innovate, Adapt, learn, and solve real customer problems faster than your competition.

What problem are we trying to solve?

Better ways of measuring software delivery performance.
How can we make the most optimal impact? We can assess software delivery team performance with useful metrics that both the organization and team members desire and those that encourage and enable teams to experiment and drive their own improvements. With a continuous improvement mindset, no matter how good we are today, we will get just a little better tomorrow.

Digital Product Workflow

Digital products flow across three primary stages: Product Discovery, Product Delivery, and Operations.

If you are performing value stream management, you create an end-to-end (concept to cash) cross-organizational flow of work for the value stream throughout the organization, understanding both upstream and downstream work and eliminating dependencies, waste, and friction.

End-to-end, Product Discovery, Product Delivery, and Product Operation: All stages of digital product management from idea generation to production or customer use, a.k.a concept to cash.

Product Discovery: The conceptual vetting stage of a digital product, from idea to backlog. Ideation, discussion, evaluation, and consideration for which features or products to prioritize and build to set the stage for increasing organizational performance.

Product Delivery: The delivery stage of a digital product, backlog to production.

Product Operation: The operation and support stage of a digital product.

Product discovery uncovers what creates value, while product delivery produces what creates value.

Measuring engineering performance

How are we doing? What are we doing well? What areas can we improve? What are we getting from our investment?

Measuring engineering performance has historically been a low point of my 20+ year technology career. During the first decade in technology, I was assessed solely as a practitioner by the metrics of that time. I vividly recall the metrics constantly reinforcing the notion of being perceived as a mere cog in a machine and the disheartening label of being a cost center.

As a manager, I was responsible for evaluating the performance of software engineers using the same criteria that I was assessed on as a practitioner. Based on these metrics, I would make decisions regarding individual team members, including stacked ranking. I have experienced the negative impacts on culture, engagement, and adverse side effects stemming from legacy metrics that treated people like machines (individual productivity measures, lines of code, and tracking hours).

Knowledge work is distinct from the repetitive workings of a mere machine.

Avoid measuring individual development activities in your business as it harms team dynamics and productivity; doing so will not help you reach your business goals, making it an ineffective use of time and resources. It is not just about the performance of individuals but all about successful teams. A shift to focusing on team outcomes and capabilities and measuring performance for continuously improving team flow is inspiring. Today’s delivery teams are seeking ways to maximize efficiency and streamline workflows. They can now access meaningful metrics to verify their efforts are paying off. By utilizing these revolutionary team metrics, teams can gain visible insights and signals to enhance their product development and delivery process further.

Whether Flow- or DORA-based, today’s metrics should combine system/process data (quantitative) and self-reported data (qualitative).

We are not the teams of yesterday

Significant influences have shifted how we can instrument and measure engineering performance (IT, digital product delivery) from the past. 

The team’s design has changed: shifting from large, skill-based functional silos that hand off work to the next functional team to small, cross-functional teams that can deliver solutions with the least dependency on other teams.

Capabilities have changed: architecture (microservices, DDD, event-driven, lambda, serverless computing, and more), culture, team design, continuous integration, continuous delivery, cloud strategies, automation, quality practices, SDLC (agile, lean), DevOps, platform engineering, and more.

Systems measures have changed (quantitative): It provides visibility to change the system rather than focusing on the individual people.

Dora Metrics (Product Delivery: Development to Production)

Flow Metrics (Product Discovery and Product Delivery: Idea to Production)

  • Project to Product by Mik Kersten
  • Lean practices and Value Stream Management (VSM) and Flow metrics
  • Platforms: There are several options in the VSM and Flow space, such as Planview Tasktop Viz

Self-reported data around team health and sentiment metrics have improved (it can be both qualitative and quantitative).

People aren’t happy because they’re successful; they’re successful because they’re happy. Happiness is a predictive measure. 1

  • Developer Experience (DX); I recommend and focus on Team Experience (TX)
  • SPACE, a New Framework to Understand and Measure Developer Productivity
  • Platforms: DX (getdx) plus one that focuses on team experience (TX) like TeamRetro or agilityhealth. Most likely, several more in the market should be mentioned here. 

Note: I have no equitable affiliation or endorsement benefits from any of my recommendations

Separate the business from the team conversation

Separating business pressure to drive metrics from a team’s motivation to use metrics for improvement is crucial. Teams that focus on continuous improvement and have visibility into metrics can identify and solve bottlenecks, leading to better outcomes and aligned results with business expectations. Metrics can be imposed by the business, risking gamification, or embraced as tools for team improvement.

Teams need motivating metrics. When teams manage improvement insights, they view the metrics as feedback to provide visibility into areas where they are performing well or where they can improve. They mine for bottlenecks and improve overall flow and outcomes. Despite their imperfections, metrics like these help to evaluate the outcomes that teams produce rather than just focusing on the numbers or outputs of individual contributors. Through monthly reviews, the teams can collaborate and develop ways or experiments to improve the bottlenecks.

How might metrics be viewed differently between the business and the delivery teams?

Business interest (focus in on the numbers): 

  • Are team metrics, the numbers, improving?
  • How can we predict the delivery estimates of a team?
  • Are department metrics improving (aggregate of all teams or portfolios)?
  • What actions are teams taking to improve?
  • Capacity, development cost per product, and return on investment.

Benefits:

  • Team throughput and delivery insights.
  • Time to market.
  • Improved predictability.
  • Delivery performance is an artifact of organization valuation. Over time, this creates a higher-quality system with fewer problems.

Risks: 

  • Pressure for individual metrics. Developers are more than just coders. Overemphasizing individual developer focus can be detrimental to team culture and overall performance. It is the collective effort of teams that delivers software, not the work of individual developers.
  • Organization leaders with a heavy hand push on the numbers. Teams use gamification to drive numbers, please the business for what their company wants to see, and risk limited or no real improvements to throughput.
    • If you still fear team gamification and misuse of metrics that defeat the value of modern ways to measure and motivate efficiency improvement, consider improving your leadership before blaming the tools or teams.
  • Comparing teams, pitting them against one another and the behaviors that arise from this, and knowledge hoarding. Each team’s work is most likely different than other teams’ work, making it difficult to compare.
  • Retention.

Teams view (focus on bottlenecks and conversations):

  • How are we doing? Where and how can we move the needle to improve throughput flow?
  • What areas are we doing well? Why might this be?
  • What are our bottlenecks to flow (what steps in our workflow take the longest time)?
  • What are the current obstacles we face regarding lead time, cycle time, PR time, and failure rate?
  • Where can we reduce waste? Can automation help?
  • What capabilities or practices can we change or add to help us?
  • What types of work need to receive the proper attention?
  • More experienced teams: How do we compare to similar cohorts or cohort clusters in the industry (if this data is available)?

Benefits:

  • Teams have the data insights to highlight underlying bottlenecks.
  • Improved time spent in active states versus wait states.
  • Collaboration and knowledge sharing across teams.
  • Autonomy, teams have meaningful metrics to allow them to solve efficiency problems (address the bottlenecks).
  • Teams use qualitative feedback (sentiment metrics) to encourage conversation, improve collaboration, and reduce toil and friction.
  • Team member engagement and employee retention.

Risks:

  • Focusing on individual team member performance. It has been my experience that small teams will not tolerate underperforming team members over time; if the team can’t solve the concern, they can surface the issue to the individual’s manager. I like using the analogy of a music band. The other band members will recognize an underperforming band member.

Leaders must avoid the outdated mindset of individual metrics, measuring individual productivity and lines of code and having teams compete against one another (comparing teams by metrics). Instead, leaders should focus on metrics that improve team flow and delivery efficiency. These metrics should combine system/process data (quantitative) and self-reported data (qualitative).

“When teams recognize the value of today’s metrics and utilize them to enhance their delivery flow, focusing on the bottlenecks and friction points, the business numbers will naturally follow suit. By embracing this approach, improvements become inherent, allowing the numbers to care for themselves.”

Qualitative Metrics: Developer Experience (DX) versus Team Experience (TX)

Prioritizing workplace happiness is crucial for success as it can lead to improved productivity, innovation, and revenue. It surpasses traditional metrics and should not be considered a perk but a key driver of a more profitable workplace.

Developer experience (DX) has been a hot topic in the technology industry lately. Most organizations build software as teams; I speak for cross-functional teams that consist of developers and other roles necessary to deliver software. Developers construct the code, but the whole team must provide the solution.  I recommend ongoing survey questions for self-reported qualitative metrics centered on developer and team feedback (TX). Earlier in this post, I shared the names of a couple of solution providers.

Team experience (TX) includes day-to-day work delivering work on teams. It encompasses all the different points of friction team members encounter in their work, including the organization’s tools, processes, and culture. Improving TX is critical to the business’s bottom line, as research shows it is a top predictor of engagement, productivity, and satisfaction. Companies with top-quartile team experience outperform their competition regarding productivity and their ability to innovate faster.

Summary on metrics

Effective teams want to improve. By offering teams a clear strategy, including an illustrative map (like a value stream or flow map), and specific expectations of outcomes, they can be afforded autonomy. They will be free to make their own decisions. Additionally, by providing them with team-based metrics that spotlight any bottlenecks in their process, the team members can confidently decide how best to positively improve the overall efficiency and effectiveness of their role in the workflow.

The business still needs to see results. Assuming team autonomy and motivating team metrics, as a business leader, you can step away from the teams and leave them alone to do their best work. You identify leaders to support the team to focus on outcomes and performance. You can focus on aggregating the signals of the metrics that matter to you. There is no “one” metric to gauge efficiency. Combining the feedback from multiple signals before raising concerns would be best. This approach means you don’t have to interrupt people or teams to get them to justify their numbers unless you have concerns. Suppose the data over a more extended period raises concerns. In that case, you can gain an understanding from the team of what might be happening and provide any guidance or action that may help them get back on track, if necessary.

A fresh perspective on performance measurement is emerging in the industry. These metrics focus on optimizing flow across the organization and should provide system and process level data (quantitative data) combined with self-reported health metrics data (qualitative data) to remove negative impacts on team culture. This improves engagement and puts improvement decisions at a team level. With this approach, the entire organization should see gains in optimizing the flow of work and improving delivery outcomes.

Removing the pressure of the business’s view on the team’s metrics is essential to foster confidence and trust that the team can review and act on insights, allowing them to identify areas where they can improve how they deliver software. To better insights and increase team engagement, using system-measurable metrics and regular, consistent surveys capturing teams’ experiences (sentiment) is essential. These visible metrics empower the teams to review and experiment with improvements.

Researching the market

The challenge

As a department leader, I am responsible for equipping our product development and delivery teams with performance feedback and metrics that will provide them with insights they can use to continue improving, even if they are already performing at a high level.

My team and I recently evaluated Value Stream Management (VSM), Flow, and DORA metrics platforms to assess the depth of data visibility into the product discovery and delivery phases and workflows through dashboards. The current market at the time of my writing provides many solution providers boasting different features, focuses, and prices, so it’s essential to do your due diligence before choosing one that fits your organization’s needs best.

Most solution providers focusing on VSM and Flow Metrics serve mid-market and enterprise organizations. In contrast, the solution providers focusing on DORA metrics serve small and medium businesses (SMB) and mid-market organizations. This evaluation experience was rewarding; it generated great conversations with solution providers and within my team and inspired this post.

Selecting a provider and partner

Selecting a metrics tool provider requires thoroughly understanding your organization’s needs, goals, and budget. Although we recommend a solution that covers all stages of the value stream, these options are targeted at enterprise-level budgets. They may not be affordable for SMB and mid-market organizations.

After evaluating the market, we identified two primary categories of metrics tool providers. The first category provides a comprehensive view of the entire value stream and can ingest data from various sources. In contrast, the second focuses on development and delivery, with optics around or extensions of the DORA metrics. We found that VSM metrics platforms tend to target enterprise budgets, while DORA metrics platforms are more affordable for SMBs and mid-market organizations.

We prefer a think big, start small, scale-out approach for most change efforts, as we did with our agile and DevOps transformation and our decision to select a VSM metrics solution provider. If development and delivery are your most significant bottlenecks, start with DORA and local metrics. Otherwise, you can expand to other parts of the delivery planning or discovery optics from value stream management solutions to gain operational efficiencies.

In my selection research, I found that some vendors measure performance and efficiency across the entire flow, while others focus only on one stage, measuring product delivery. However, I did not find one solution that measured both stages well. Additionally, I needed help finding a solution that provided strong team health metrics and qualitative data around human dynamics and team operations, in addition to system measures.

Another critical difference I discovered between service providers was the degree of data integrations or data read access available. While all platforms offer visualizations to identify bottlenecks, some providers do not integrate with systems of record that provide deeper insights into stages and the tasks causing the bottlenecks. When evaluating a solution, consider whether additional metrics are necessary to narrow the bottleneck or which components need to change to improve the bottleneck.

Visualizing the instrumentation options

I created the following diagram to help me convey these concepts when having this conversation with others:

Final thoughts

I am encouraged by the digital product delivery metrics available today that focus on teams’ usage to make decisions to improve performance and efficiency. Two measurements are necessary for teams to have the information they need to succeed: system process performance data and human dynamics, sentiment, and how teams operate.

While some solution providers have been in the market for some time, it was apparent that this market is still relatively young and proliferating with new vendors and improved features from competitors. Your approach to metrics and the degree to which you can visualize data in one solution can be the difference for your team. There are drastic differences in prices and capabilities among these solutions. I suggest you examine how you present flow and optimization across the organization. Select a solution that best fits the context of your teams and organization.

Another exciting aspect: Linking data to business value outcomes and impact (closing the loop). We can quantify the impact of our investments in agile, lean, and DevOps software delivery best practices.

Poking Holes

I invite your perspective to analyze this post further – whether by invalidating specific points or affirming others. What are your thoughts?.

Let’s talk: phil.clark@rethinkyourunderstanding.

Related Posts

References

  1. Sutherland, Jeff; Sutherland, J.J.. Scrum (p. 148). Crown. Kindle Edition