It’s a good time to be an Informationist.

The data generated is richer than ever before. The processing power to harness the huge amounts of data and analytics created is ever increasing. Cloud is helping to scale data instantaneously and storing it for organizations to use when they need it. Data is fueling meaningful insights and delivering business value. Which all has led to data becoming a part of the CEO agenda.

Data is everywhere today. Data is powering AI to predict customer orders and increase sales at companies like McDonald’s. And, it’s even accelerating clinical trials for new therapies at many pharmaceutical firms.

The value of data is growing constantly and delivering unprecedented insights. The enterprises which maximize this value are treating data as a differentiated and prized asset. At the same time, others are struggling to unlock that value. It is clear that becoming a data-driven organization is the new table stakes for success, and essential to gaining a competitive advantage in our post-digital world.

We are seeing three trends powering data into 2020: strategy, architecture and engineering.

Trend #1: Top-down data strategy

Data is finally taking its rightful place in business strategy.

Decision-makers have noticed the tangible value data can deliver. The return on invested capital for data-driven companies is 3.5 times more than a non-data-driven company. Plus, 70% of companies considered "data-driven champions" see tangible and measurable value from data, e.g., operations excellence, optimized customer experience, and innovative products and services.

This is shifting the conversation as data increasingly enters the C-suite and board room. Leadership is not only asking if we have the right data to achieve our strategy, but are we treating the data right? Our responsibility is to make sure everything we do serves the big picture. The questions I have my teams ask themselves every day: How can I improve the quality and trust of the data? How can I use the data to make better decisions?

Data is also steadily becoming a formal part of leadership teams. Nearly 60% of enterprises have appointed a chief data officer (CDO) to guide them on the data-driven journey and growing.

To properly serve C-suite priorities, I can’t stress enough that your data strategy requires as rigorous approach as the business strategy. Your data strategy also needs alignment with the business strategy (e.g., digital), divisional strategies (in each line of business) and the IT strategy.

Trend #2: Data architecture fit for purpose

The major shift driving the future of data architecture is the variety of consumption patterns that need to be supported. When you look at data, it has different shapes and speeds, it exists in different places and it needs to support a variety of consumption patterns.

In the past, there would be what we call the fireworks data projects. You would build a specific data pipeline to service a use case. It would result in an initial flash success, but the sparks would die out, as the work was not reusable. Then came technology-led initiatives using Hadoop and scale-out architecture. It was cost-effective, but it tried to be all things to all people, and that didn’t work. We realized the need to have specific yet adaptable platforms at the enterprise level.

Today’s data architecture is fit for purpose.

Instead of building siloed data pipelines or raw data lakes, organizations are investing in building data platforms, also called Data Hubs. This is a set of integrated, consistent data services across on-premise and public cloud environments. These platforms are built at an enterprise level to support data needs of multiple business functions. The platforms are built for speed and adaptability, enabling the processing of complex, data-driven insights in real-time. The data environment is flexible, able to be consumed in various shapes and schemas. And data access and sharing are federated, able to be accessed at the needed speed.

Trend #3: The age of machine-led data engineering

Finally, using machine learning to curate and wrangle data is becoming a fast reality to deliver business insights.

When you embark on an analytical project, it’s all about the data. If the analytics is the tip of the iceberg, then data and the effort to create those insights is everything under the water. Typically, in an analytics project, 60-80% of the time is spent on wrangling, integrating and curating data.

To meet this challenge, we are driving the trend of machine-led data. Data has been enabling machine learning and AI in recent years. Now, we are working with clients to answer the questions: Can machine learning and AI auto-discover, auto-classify and auto-label my data?

The idea is to create an autonomous environment where data manages and fixes itself and becomes self-healing. This will free up precious talent resources to focus on innovations that drive business value.

As we head further into 2020, I want to leave you with these data imperatives:

  • Understand the operational, talent, culture and technology capabilities you to need to create your data platform.
  • Determine your operating model and the roadmap for how to get there.
  • Invest in data literacy and empower a data-driven culture.
  • Enable self-serve, high-quality data.
  • Build a seamless environment that boosts visibility, veracity, security, scalability, agility and speed.

As an Informationist myself, I can’t wait to see how far we’ve come by this time next year.

For more information on how to drive intelligent outcomes with the right data, please visit our Data Management site.

​Shail Jain

Lead, Cloud First – Data & AI

Subscription Center
Subscribe to Software Engineering Blog Subscribe to Software Engineering Blog