Trust but Verify – Measuring Value with Agile
In my last article, I talked about how trust is the secret ingredient to high-performing Agile teams.
Product management and project management are each disciplines to channel effort towards a goal. Businesses generally choose between a project or product mindset in software development, and attitudes toward both have shifted over the years. So, perhaps it's time for your organization to reselect.
In early software development, a project-centric approach prevailed, because it complemented the Waterfall methodology software development teams commonly implemented to run large software projects. This reality has changed with the growth of iterative Agile and DevOps so that those teams now prioritize a product management mindset. The product mindset focuses attention on creating value for the business and customer, rather than an arbitrary milestone.
Project management ensures that widgets adhere to a set of specs. Product management aims to release widgets without any defects that would hurt business. As companies shift from a project mindset to a product mindset, software QA experts must correlate code changes to customer experience and business results.
While the two approaches often complement each other, the major difference between project and product mindsets today is where companies place their focus, and how they organize teams.
"In a project mindset, the assumption is that work is predictable and repeatable enough that it can be broken down into a series of steps with milestones, each allocated to resources and tracked with a timeline and budget," said Mik Kersten, author of Project to Product, and CEO of Tasktop, a value stream analysis software provider.
The project-centric approach, Kersten said, works well for construction and data center builds, but it falls short in measuring the value software products deliver to customers. "It fails completely for software innovation, which is overly complex, dynamic and changing," he said. With a product mindset, organizations create stable and customer-centric value streams and measure outcomes instead of activities.
The modern discipline of project management traces back to the mid-1800s. Henri Fayol, an engineer in an iron and coal company, codified five functions of management and 14 principles of project management.
Then, in the early 1900s, Henry Gantt formulated the Gantt chart. This diagram of work helps teams manage dependencies that need to be completed in sequence. The Gantt chart works well for large, infrequently released software projects, as it enforces an order of steps.
Product management harkens back to 1931. Neil McElroy, then the president of Procter & Gamble, was evaluating how to differentiate his company's Camay soap from the competition. He determined that brand men should hold absolute responsibility for all aspects of their brand; they should track sales, product development and marketing. Consequently, Procter & Gamble restructured by brands.
Over the years, the brand man role evolved into product manager. In the early 1980s, Scott Cook, who worked at Procter & Gamble, brought its product-focused concepts to Intuit, in the company's software development efforts. The arrival of Agile development in the early 2000s and DevOps several years later improved -- and emphasized -- the ability to track how changes to software and services affect both customers and the bottom line.
Enterprises have started to recognize the danger of a project mindset, namely, that everyone focuses less on the product.
"A perfect project management system can complete every task ... in a vacuum, with amazing results -- and still fail when it comes time to go to market," said Alexander M. Kehoe, operations director at Caveni Digital Solutions, a web design consultancy.
Apple has applied both project and product mindsets. Apple's iPhone innovation enabled it to grow into one of the largest companies in the world. However, critics accuse Apple of releasing a nearly carbon-copy iPhone each year. According to these critics, product quality for these phones has stagnated, as Apple finishes projects with little or no consideration on the product side. Because of this reliance on project-oriented thinking, Kehoe said, the next major mobile phone innovation might not come from Apple. If another company takes the lead in mobile phone innovation, Apple might see its market dominance evaporate overnight, he said.
For traditional enterprises, moving to product-minded development is a matter of survival.
Mark OrttungCEO, Nexient
Unlike consumer-goods companies, most software development firms can release products with no physical item to manufacture. One of the main aims of DevOps is to release software faster. So, DevOps adopters must prioritize the product, while listening for shakeups in the market. If you take too long on a project, the product it created might already be obsolete by the time the software is available to the consumer. With a product mindset, the business stays adaptable. The team constantly takes direct feedback from the target user and adjusts.
As digital-first companies like Uber and Airbnb disrupt long-standing industries, more companies recognize the value of a product mindset.
"For traditional enterprises, moving to product-minded development is a matter of survival," said Mark Orttung, CEO of Nexient, an Agile software development services company. Project management involves a top-down, command-and-control approach that can put enterprises at a disadvantage. Many larger companies will get crushed if they don't adapt to the innovative approaches taken by smaller disrupters, he said.
A product mindset shifts the focus to customer and value, rather than proxy metrics and activities, Kersten said. Product management and delivery is really DevOps -- in particular, the three ways of DevOps established by Gene Kim:
Organizations can't adopt a faster release cadence through CI/CD alone, Kersten said. Instead, they must concentrate on how software development affects the business and the customer. "[It's a] way of scaling the principles of DevOps to the business," he said.
The shift to a product mindset resets how IT relates to what the business produces. Entrust Datacard, a security and identity management software provider, went with the product approach to move from traditional software products to SaaS offerings. Its customers include banks, governments and enterprises.
"An organization's technology and IT strategy is not just a business enabler, it is inherently part of the product or service being delivered," said Greg Wetmore, vice president of product and software development at Entrust Datacard.
The product mindset hopes to directly tie work or investment to positive business outcomes.
Greg WetmoreVP products and software development, Entrust Datacard
In the transition to building cloud-based SaaS, Wetmore created product teams staffed by people across the business. Development and product management personnel work on product teams with representatives from customer success, IT, information security and operations.
Those diverse teams use a product mindset to deliver secure, reliable and valuable software. To support the product, they build a long-term strategy, deliver incremental value quickly and apply innovation and creativity.
"The product mindset hopes to directly tie work or investment to positive business outcomes," Wetmore said. These product teams measure success based on business metrics like revenue, cost savings, ROI and customer satisfaction.
DevOps and the product mindset have naturally aligned philosophies. When DevOps and product management occur in tandem, organizations can improve ROI and feedback loops.
"The ability to reliably and quickly release incremental new capability to customers allows the product-focused team to deliver value in short cycles, measure success, optimize and improve, then repeat," Wetmore said.
Product creation and management expands the focus of DevOps beyond just developers and operations teams. The practices get the business side involved too.
The project mindset tends to get too entrenched in the plan and minimize deviations from the plan.
Travis James FellProduct manager, Hypori
The product mindset orients the software delivery team toward several DevOps hallmarks:
Conversely, the project mindset tends to orient the delivery team toward:
"The project mindset tends to get too entrenched in the plan and minimizes deviations from the plan," said Travis James Fell, product manager at Hypori by Intelligent Waves, a virtual smartphone product.
Catalyze Systems, which develops medical software in India, has seen enormous benefits in expanding the scope of DevOps, said Manish Raje, founder and CEO. Specialized teams like design, engineering, QA and marketing are all tightly coupled. If the support team finds a reoccurring issue, it can give that feedback to engineering quickly, so they address the problem in future versions, for example. If the sales team can't get customer buy-in, that adoption feedback turns into a priority for the product team: Remove the hurdle for sales.
The product design and development lifecycle encompasses more than a standard project lifecycle. The types and timing of activities differ between a project and product focus. Each product lifecycle contains multiple project lifecycles.
"A significant difference in these lifecycles is the way the product's assets [such as code, tests and documentation] are organized and preserved," said Nancy Kastl, executive director at SPR, an IT consultancy. For example, a project-focused QA team might organize test cases within a project folder in a test repository. In this setup, there's no guarantee that tests are easy to find or reusable, especially if the project scope grows unwieldy over time. With a product mindset, QA organizes tests by the product's modules, pages or functionality. This setup enables QA to easily test product changes in the future.
The responsibilities of a product manager vs. a project manager differ as well. Product managers own the concept, design, engineering, QA, marketing, sales and support, as well as ongoing innovation to keep the product relevant in a changing market. On Agile product teams, someone in engineering or QA is always aware of what they are building, and for whom.
In a project team, these roles all contribute to a product, but they maintain individual responsibility for each piece of it. "Someone else is responsible to think about how your piece fits in the overall scheme of things," said Catalyze Systems' Raje.
In the project lifecycle, the team's commitment only lasts for the duration of the project, or until the end of the service contract. Project teams are typically short-lived, and the enterprise shifts team members to different projects. Unless organizations determine a way to retain knowledge across projects, it's difficult to deliver repeatable results with a project-based approach, Raje said.
Another big difference between projects and products in software development is the orientation of the teams.
Recruiting requirements also differ between these mindsets. Project-based software development organizations hire people with narrow specialist skills to work in functional silos. A product-minded approach calls for staff who are "T-shaped," meaning they are highly skilled in their areas of expertise, but conversant in many others. "That allows them to collaborate easily with experts in different fields and recognize when it's time to call [on others for] help," Nexient's Orttung said.
Companies implement a product mindset via two new roles: a product manager and a product owner. Some organizations combine these roles, but there are distinct differences between a product manager and product owner, Kastl said.
Product managers operate at a strategic level. They focus on long-term product strategy that the company's objectives, the product's vision, market trends and competition all determine. For commercial software, the product manager's responsibilities include marketing, sales support, budgeting, forecasting, customer care and delivery team support.
Product owners are tactical. Part of the Scrum framework for Agile software development teams, the product owner provides direction on what developers should build, based on the product's vision. The product owner creates a shared understanding between the business side and the development team. They describe and prioritize backlog items and determine satisfactory delivery.
Project-based skills have a place on Agile product teams, said Hypori's Fell. Project managers can become Scrum masters; business analysts can become product owners. But the shift to a product-oriented role involves more than a title change. Product-based roles require rapid and iterative delivery models and, thus, a dedicated, cross-functional team. For example, software engineers and testers often work in the same product team to ensure quick QA and feedback.
Entrust Datacard pairs a product leader with a development leader, then designates them the "CEOs of their product." Together, they define a product strategy and roadmap, prioritize work and define release timelines, build an investment model and take accountability for product success.
"This approach provides a team clear ownership over a product and ensures that both product management and product development are completely aligned on a plan," Wetmore said.
It takes commitment to see the long-term benefits of product versus project teams, management and lifecycles. This is especially true if the organization must adapt to Agile or DevOps as part of the transition. Here are six practical ways an organization can prioritize product over project.
1. Set clear responsibilities. Give team members clear responsibilities that align with a product mindset. Senior leaders should create teams that understand their responsibility to own the product over the long term, Wetmore said. Product leaders should seek regular updates on strategy, as well as measure and report on the business metrics that define product success.
These teams must evaluate whether the product delivers the desired business outcome, from its initial ideation phases to an eventual decision on product end-of-life. Clear delineation of work is essential to avoid short- and long-term confusion.
2. Design with the customer in mind. Establish channels for constant communication with end users. The goal is always to gather customer insights via techniques like UX monitoring, canary releases and beta testing. Product-minded teams devote nearly 80% of their time to understand, validate and even challenge every user insight. Agile or DevOps organizations adjust their strategies according to the feedback.
"Positive customer experience is a competitive advantage, and [QA] engineers take a vast share of responsibility for this," said Dmitriy Nortenko, CEO and founder of QA Madness, a software testing services provider. QA specialists need to balance technical skills with a user focus. QA should also consult the development team on how to mimic the UX for true-to-life tests and learn what kind of data to gather about how changes affect it.
Enterprises must invest in qualitative and quantitative feedback to empower developers and QA teams to adjust and make decisions on the fly. Qualitative data might include satisfaction score, feature requests, bug reports and focus groups -- things that measure how a user feels about an app. Quantitative data provides objective feedback, such as load time and usage metrics. This feedback can help the organization decide whether to iterate on a feature or shelve it.
3. See the bigger picture. Each development decision affects the product. Consider the ramifications of each choice over the long term. "When teams think about being part of a product, they tend to associate with the bigger picture," Raje said.
Agile or DevOps organizations should develop and release products that are easy to test, deploy and support. If the team doesn't think about ongoing support during the build, for example, support costs will skyrocket when the product faces issues in production.
Instill a sense of product ownership in all team members. Each person should feel like they have ownership of the product.
One way to help promote big-picture thinking is to have QA engineers work with software developers to build a testable product. "If the product cannot be tested easily by the in-house team, it is surely going to land in trouble when in production," Raje said.
4. Invest in organizational change. The hardest aspect of a product vs. project overhaul is getting everyone to buy in. The transition to a product mindset is a culturally challenging effort.
It requires many people to change their habits, especially people in positions of authority, Hypori's Fell said. Start at the top. Executives should evangelize the change, lay out a roadmap, allocate funding for training and new staff, and periodically update the organization on progress.
Next, identify the product leaders. Find progressive-thinking, product-minded people and recruit them to help lead the change. Also, invest in training. "[Onboarding] a critical mass of employees in Agile thinking will be critical for the product mindset to take root and flourish," Fell said.
Don't let your organization lapse back into old, project-thinking ways; that will only leave it vulnerable to more Agile, product-oriented competitors. Fell's previous employer attempted a product transformation to stay competitive in its industry. However, many processes remained mired in the project mindset. The organization had a two-year lag time to add innovative new companies to its approved vendor list, for example.
"We used to joke that, in two years, we'd be owned by Amazon," Fell said.
5. Build the right thing for today. There's a big difference between project and product managers' approach to failure. Project managers try not to fail. Product managers find ways to fail fast, learn from the experience and move on. "The project mindset assumes failure is expensive, so value is all about reducing cost and risk," Orttung said.
When it comes to building a product "the right way," product-minded development teams assume the definition of "right" will change over time. The business, customers and market always evolve. Development teams build the right thing for today, with the flexibility to adapt to emerging needs.
A project release attempts to cram a lot into one big burst, which carries risk. Each element in that single release raises the stakes of failure for the entire collection. Product-minded development uses shorter and iterative cycles, that constant feedback guides. The product approach not only increases a team's release cadence and overall speed, it enables testing at an earlier, less expensive stage.
Orttung recommends organizations adopt the simple product mantra: "Fail fast, learn fast."
6. Fix the pig. Rather than fix their problems, many organizations simply dress them up. As the idiom goes, it's like putting lipstick on a pig. A product mindset requires a shift throughout the organization -- a holistic focus on results, not on processes.
"Convince the C-suite that fixing the pig is cheaper and faster in the long run [than cosmetic changes], even if it requires investment up front," Orttung said.
Expect pushback. Change doesn't happen overnight.
Metrics are the most important components in this product revolution. Measure success according to business goals, not IT service-level agreements. Don't settle for tracking mandated uptime. Measure metrics like deployments per year or call center volume -- any statistics that determine product value.
Business leaders don't know -- or care -- much about specs, servers and software-defined networks. They care about business wins. So, too, must the product team.
"Make an effort to speak their language, and you might be surprised how quickly the C-suite gets on board," Orttung said.