How to develop products from a data-driven perspective?
August 20, 2020
There has been a massive shift in the way we do product management in the past decade. It started with writing user stories that were backed by extensive customer research, however there was very little control over the outcomes. Product success involved a considerable dose of wishful thinking and/or hoping for positive business outcomes.
By Kirthi Vani and Steve Roberts, Accenture
What began as hoping for positive outcomes in the past has now evolved into driving better results through data and experimentation.
As many product managers have found, however, designing with the customer in mind, while necessary, is not enough to ensure a positive outcome. To increase the odds of a successful product, product managers need data.
A recent survey by Splunk showed that organizations that place a strategic emphasis on data and have an advanced strategy to extract business value have added 83 percent more revenue to their topline and 66 percent more profit to their bottom line in the past 12 months. In addition, 93 percent of these organizations feel they tend to make better, faster decisions than competitors. And 91 percent believe that their organization is in a strong position to compete and succeed in its markets over the next few years.
More precisely, they need measurable signals—signals that can only be obtained by developing an advanced strategy to collect, organize, and analyze data in the product definition stage.
We believe that companies can leverage data-driven product strategies that lead to differentiation and competitive advantage. To do this organizations must be able to pivot quickly based on their data experiments:
If data-driven product management had a mantra, it would be “test early and pivot quickly.“ Failure is an essential part of learning and growth. Product managers should not have to come up with a perfect idea on the first try. Instead, they should be encouraged to treat everything as an experiment—iterating quickly in short build/measure/learn cycles.
more revenue added organization who place a strategic emphasis on data and have an advanced strategy to extract business value.
Amazon CEO Jeff Bezos considers Amazon‘s experimental culture to be a significant strategic advantage and a major reason why the company reached $100 billion in sales faster than any other company.
An experimental culture relies upon data democratization, meaning that everybody has access to the data they need to make decisions. Everyone in the organization should understand and value data.
Data governance is essential to data democratization. Leadership oversees how data is collected, annotated, and accessed. Leading companies often do this by setting up a center of excellence for data management or by appointing a Chief Data Officer, Chief Digital Officer, or Chief Data Product Leader.
Similarly, the organization’s data strategy should go beyond product development. Successful data-led product managers measure and instrument product usage to determine best offerings, licensing, and product strategies. This data can also help customer support, supply chain and manufacturing improve their performance and metrics.
Data quality is another vital consideration in democratization. Many product managers are somewhat focused on data quality, but often, this isn’t enough. Data quality needs to evolve continuously, through constant, iterative improvement. Data quality metrics should be shared across the organization to ensure transparency and a high level of confidence.
Data democratization provides access to multiple sources of data. Most organizations have a lot of dark data generated by system logs—usually unstructured, untagged, and untapped.
This should be leveraged and used in combination with product data to determine product strategy. Tools like Splunk can help mine, model, and analyze unstructured log data.
In data democratization, self-service analytics is critical. The right set of tools can connect siloed data and make it accessible. These tools should be customized with filters and analytics based on the people accessing the data.
Finally, in a truly data-driven organization, everyone should be trained in the basic concepts of data, analytics, and the tools required to access this information.
The critical element in data collection is value in exchange for privacy. For example, collecting location information for advertising purposes only is not ideal. However, if that information is used to help the user geo-tag a tweet, get the most relevant tweets for the customer in that location, or use that location information to auto-tag photos taken from a camera, then there is value created for the end-user.
With data democratization, the product manager can apply data to every question and decision for product strategy and development. However, product managers should be very careful in separating correlation from causation. In statistics, correlation tells us how strongly a pair of variables are related and how they change together. Causation takes this further and states that any change in the value of one variable will cause a change in the value of another variable.
For example, a correlation might be the increase in usage of new functionality “x” with an increase in user retention. The product manager can predict a causation and create a primary hypothesis—“Improving user engagement with feature ‘x‘ will directly impact conversion”—but she or he must then test it.
To minimize the confusion, managers should verify that the null hypothesis—in this case, the statement that “there is no relationship between feature ‘x‘ and conversion“—can be disproved with statistical significance before testing the primary hypothesis.
The product manager of the future will rely on artificial intelligence (AI) as well as data. The AI-led product manager knows what data is being collected and plans a foundation layer along with a core infrastructure to create an AI “flywheel.” The flywheel runs on the momentum generated by structured and unstructured data from internal and external sources, combined with multiple machine learning algorithms. The whole is exponentially greater than the sum of its parts. At most companies, AI initiatives are not integrated, but sharing AI and machine learning models across teams can help create the flywheel effect. Of course, this depends not only on product managers’ efforts but also on those of data engineers, data scientists, and DevOps engineers.
An efficient operating model and governance are also critical because there is a cost to maintaining artificial intelligence. So, managing the AI flywheel prudently means combining use cases wherever feasible and making smart choices about data; for example, re-using existing cases as often as possible and thinking creatively about data from “unexpected sources." In short: Always build models that are fit for purpose and the problem at hand.
Product managers are typically responsible for driving business results from a specific product or portfolio of products over time. However, data democratization and AI flywheel efforts also require driving and measuring short-term growth metrics across the organization.
This has led to the emergence of the Growth Product Manager (sometimes also called “Outbound Product Manager“). These experts typically focus on growing traffic, users, engagement and other elements to drive short-term results. They often own the growth strategy and plans to optimize revenues to support multiple products in the organization.
The Growth Product Manager defines the key growth metrics for the product—such as user acquisition, renewals, conversion, or reduction in churn—and regularly reviews growth trends through insights published on tools such as Adobe Analytics or Google Analytics. Other product managers then use these insights to improve the product user experience.
The Growth Product Manager should support the product manager to ensure that short term metrics are trending in the right direction. The focus should be on understanding the customer and on solving the right problems within the product area.
The Authors of the article thank the following for providing valuable insights: