The path from data to knowledge

Data requirements are not just growing in volume, but also in the diversity of types and sources of data used in models. Data is no longer limited to structured, transactional, internally generated types. Today, when looking to enable greater use of artificial intelligence, organizations should incorporate machine-generated, unstructured, semi-structured, streaming, external and probabilistic data.

The limits of traditional MDM

As data proliferates and becomes more heterogeneous, business requirements for it are changing. For example:

• Companies most time need to respond to events and opportunities faster, and act proactively to meet customer needs; this can demand continuous, real-time access to comprehensive, current customer data.
• Social network, mobile device, web activity, location and other external context data should be reconciled with internal customer data to help drive marketing activities and customer experience.
• Some time there is a need to maintain dynamically changing, unified data that accounts for the interactions and relationships that exist within data entities across all touch points across the enterprise.
• Increasing regulatory compliance requirements demand greater data management and governance capabilities.
• Companies’ master data systems should include probabilistic data and fuzzy matching algorithms alongside more certain data.

When it comes to integrating these data types, “point-to-point” or “match and merge” approaches of traditional MDM would be too complex and slow; new techniques are required to adequately capture, link and curate these newer types of data.

Expanding and evolving MDM

Instead, a flexible “Digital Master” model is emerging that can be tailormade to deliver new business value from vast amounts of diverse data.

The tasks performed by traditional MDM, focused around identifying missing or erroneous entries in highly structured data sets and eliminating duplicate records, remain vital to modern business requirements. But MDM should be expanded to meet the business requirements described above. The solution is the “Digital Master”: an evolution of traditional MDM, designed to bring data to the heart of the business and organize data at scale, combining data from a wide variety of sources.

The Digital Master helps companies to move beyond having a limited and static "golden record" (of a customer, for example) to having a "golden profile" that not only contains more types of data and insights but can be updated and expanded in real time. (More on Golden Profiles in the full report on page 12.)

Compared with the use of traditional MDM, the Digital Master offers several advantages:

• It can support deeper insights into customer behavior, more specific and accurate predictions, improved customer experience and real-time responses to events and opportunities.
• It can be more business-outcome-focused, insight-driven, adaptable, comprehensive, agile and scalable across the enterprise.
• It can support enhanced business reporting but also drive operations directly by automating many real-time decisions.
• It can put AI and analytics at the center of every business process and any effort to transform customer insights and relationships.
• It can allow organizations to get more from AI and become more data-driven.

For example, we helped a large hospitality company build a digital data platform that allowed it to capture, curate, process and store emerging data types. The company’s objectives included a radical improvement in customer experience and personalization, to help retain loyal customers and increase market share. Now, with its Digital Master in place, the company has a 360-degree view of existing and potential customers—they are able to recognize individuals at any point of interaction across its channels—and it can therefore enrich offers with intelligent insight and get ever closer to its key objectives.

Making the transition

Organizations tend to transition to the new Digital Master in one of two ways:

• Extend. A two-step process that involves maintaining the existing MDM in an integration hub and integrating it with a data lake, which is concurrently expanded to incorporate new data sources and capabilities
• Replace. A greenfield approach in which a robust Digital Master is built entirely around a new data lake.

In both cases, aggregating the organization’s existing data into a data lake is the starting point. This involves bringing together data from past transactions, customer records, activity on the corporate website (including from cookies) and social media channels. To make sense of this basic data, we need to add more meaning to it in the form of contextual insights. For this we can use a knowledge graph. This is flexible enough to work like a bridge between various data types and formats, while at the same time adding descriptions and associations that help machines to discern what the data is all about.

This process helps us to create rich, “golden” profiles of dimensions such as customers, businesses and products. It can also provide us with powerful insights into customer attitudes and behaviors (for example, by identifying correlations between different buyers of the same product).

Knowledge graphs have been around for a couple of decades. Their ability to generate insights have been transformed in recent years, however, thanks to the proliferation of (mostly unstructured) data and enormous increase in computing power at organizations’ disposal. Knowledge graphs are what enable Apple’s, Amazon’s and Google’s virtual assistants to provide extremely relevant answers to users’ questions. Within a business, a knowledge graph that is constantly expanded with contextual and other data (such graphs can scale relatively easily) can identify and expand an increasing number of use cases for products, services or any number of business missions. And applications or dashboards can be created for some use cases, making it easy for users to quickly glean insights from the data.

A vehicle for embedding AI

With all the elements of the Digital Master in place, enterprises can harness artificial intelligence and machine learning across their operations, supporting powerful breakthroughs in insights, data-driven decision-making and predictive capabilities.

The knowledge graph bridges the gap between traditional data stores and AI capabilities. This is because AI models can directly leverage the graph, so they can learn from massive amounts of meaningful data, bringing predictive power to new domains and increasing accuracy. Unsupervised learning algorithms can explore the knowledge graph and discover patterns that humans could never see.

eBay offers an example of how AI can leverage knowledge graphs to expand its capabilities. The company’s popular “ShopBot”—an AI-driven shopping assistant—is powered by knowledge graphs, according to its creators. eBay’s knowledge graph incorporates data from roughly 16 million active buyers. The bot has powerful algorithms, but when a user voices a query through the app about a certain product, its desired features and intended uses, it is the data provided by the knowledge graph that enables the bot to provide intelligent and relevant responses to the user.

Not many companies today can leverage such capabilities at scale. In addition to eBay, online giants such as Amazon, Google, Microsoft, Facebook, Alibaba, Baidu and others are using AI in conjunction with knowledge graphs to drive their core operations and services. There is no reason, however, that other types of organizations cannot leverage these capabilities to drive positive outcomes across their business.

Digital Master starting points

In this article we have explained why organizations should be making the shift toward the Digital Master model and the advantages that it confers in the digital age. To learn more about how to make the transition, our whitepaper will help executives understand the evolving landscape and assess the architectural options.

As indicated earlier, moving from MDM to a Digital Master can be done incrementally, without much disruption. It is the ideal path towards harnessing AI at scale across your operations. If you are at the beginning of this journey, the first steps you should take include the following:

• Inventory your data: Determine which are your most frequently accessed and up-to-date data records. These are the sources of the single version of truth—the golden record—that the Digital Master will build upon.
• Identify existing data hubs: Where in the enterprise do these data records sit? Can their repositories be connected to meet current and specific business requirements?
• Review your use cases: What are the existing use cases for your master data? To what extent do your existing MDM processes and technologies align with these? Can new solutions in the market serve those use cases more effectively? Do they create opportunities to for new cases?
• Prioritize the use cases: Which cases are the most critical to delivering uplift to the business? Do they involve new revenue generation or improved operational efficiency—or, alternatively, improved fraud detection or regulatory compliance? The priority cases should be the initial focus of new Digital Master.
• Find external data sources: Look outside your organization for external data sources that can be incorporated to complement existing reference data and help generate more complete insights through contextual and analytical models.

¹ IDC White Paper, sponsored by Seagate, Data Age 2025: The Digitization of the World from Edge to Core, November 2018