Accenture Digital Video

Test And Learn


Jon Stribley, Accenture Digital Video, North America

We’ve all been raised to avoid failure. We train for success, and consider its opposite the enemy. But what if a different attitude to failure could unlock innovation in our video, advertising and content?

That’s the idea behind Test and Learn. Being open to failing fast, learning from it and moving on, is the key to quickly iterating and measuring the impact you’re having on the business. Integrating that mentality into a structured innovation framework is intrinsic to driving change that will bring fresh thinking to your products, services and features.

While many would agree with the concept, far too many companies are struggling to use data-led thinking to drive an iterative methodology across their business units. They’ve deployed enabling processes like Agile to drive cross-functional thinking.

They use Big Data extensively to capture and report. Successfully tying these two concepts together to become a truly data driven organization across business, operational and product priorities, however, is a bigger stretch. That takes a different way of thinking.

As companies move forward with digital innovation, it’s important to banish legacy thinking along with legacy systems. The modern approach: Proactively design hypotheses, then have the data provide a clear answer. Proceed iteratively and let the answers guide direction. Failure is as important in these tests as success, as long as you learn from everything. Companies ahead of this curve are organizing their business units around data, using it appropriately and aren’t afraid to fail fast. To do this well, they focus on the concept of Test and Learn.


Test and Learn is a powerful and useful data-driven methodology. Organizations use it to quickly experiment on hypotheses that they prioritize across their whole business, or individual units, quickly measuring and understanding the KPIs around those hypotheses to drive their future decision making.

The experiment-led approach of Test and Learn is a proven one that gets structured data into your organization. It helps determine the factors that drive results, supporting fast, validated progress with innovation.

"Test and Learn is proven to get structured data into your organization"


Test and Learn provides an organizational framework, with supporting processes and instrumentation, to approach innovation as a series of challenges. You frame each challenge as a succession of possible solutions (hypotheses). Link each of these hypotheses to your value tree to ensure you are working on KPIs that matter and addressing the right problems.

You can apply Test and Learn to virtually any aspect of video, advertising and content decisioning, including:

  • Customer Service: to reduce calls to Service Centers by prioritizing drivers of calls

  • Product Development: to prioritize feature roadmaps, addressing cost / revenue drivers

  • Marketing: to generate better campaign returns by selecting which promotional offers trigger the biggest uplift in sales

  • User Experience: to select which program guide UI design will drive viewer engagement


Test and Learn is based on the assumption that you will be more likely to succeed if you:
Know what specific, measurable questions or problems your changes (experiments) are solving,

Know what specific, measurable questions or problems your changes (experiments) are solving,

Recognize whether those are the biggest problems to solve for your business and customers, and

Recognize whether those are the biggest problems to solve for your business and customers, and

Have confidence in your data to indicate whether your solutions actually worked (versus a control).

Have confidence in your data to indicate whether your solutions actually worked (versus a control).

Use each innovation hypothesis to direct your efforts to create a targeted experiment. That can then be tested, via a series of real world scenarios, with data from each test being used to drive decisions.

A common pitfall with these types of experiments is a narrow focus on just one element of product or process. For example, developers may be pushing to make their app faster, but performance speeds may have no positive impact on the business’ most pressing problems. To truly reflect the business, hypotheses should be developed across cross functional “pods” that focus on specific problems, but have input from different entities spanning the organization.

The real value comes with a team at the intersection of business, product and operations. For a company working on a video product, this means bringing together product management, marketing, customer service and development. When they create a hypothesis, they’ll have a holistic view of the problem, avoiding a narrow focus.


You can then make your investments on the back of the findings from your experiments, in accordance with the data they’ve revealed. This sets your basis for being a data driven organization. You can then repeat the process and improved upon it, continuously, as the organization gets more data. You’re creating a feedback loop that continues to build upon itself.

In today’s market, with extremely agile disruptors able to pivot on new opportunities or experiences very quickly, traditional operators and broadcasters are going to need to pick up the pace. Shifting to a more data-driven approach, structured with Test and Learn methodologies, will make their current businesses more efficient and help them pivot to new opportunities. While creating this shift in an established organization can be a difficult task, the momentum you can build behind the change will ultimately lead to data-led decision making and the ability to eliminate, or abandon, practices that are not providing an expected return.



Jon Stribley

Jon Stribley
Marketing & Data Monetization Services, North America
Accenture Digital Video

Connect with Jon Stribley's Profile on LinkedIn. This opens a new window.
James Weeks

James Weeks
Marketing & Data Monetization Services Lead, Europe
Accenture Digital Video

Connect with James Weeks' Profile on LinkedIn. This opens a new window.