Why AI-assisted data governance is crucial to driving value in AWS

You can have all the data in the world, but it won’t be worth much unless it’s properly managed, of high quality, and relevant to the people who use it.

Today’s connected cars, for example, monitor more than 200 data points every second, with more and more data sent to car manufacturers each year. Potential uses of all this data are skyrocketing, but as the data grows by leaps and bounds, so do the challenges.

A lot of companies started moving to the cloud because of its capacity to handle more data than legacy, on-premises platforms. In fact, AWS provides many capabilities that enable companies to ingest huge volumes from a wide range of sources—whether internal or external, human- or machine-generated. But then the question is how to make all that data useful and valuable to people and processes across your enterprise.

The answer is data governance—policies and processes for how data is collected, stored, accessed, and managed over its lifespan. But at cloud scale, the only way to do this effectively is with the help of advanced technologies like AI and machine learning. Cloud-based AI tools bring the advanced capabilities and scale that helps to automatically cleanse, classify, and secure data in the cloud as it is ingested, supporting high quality, veracity, and ethical handling. One such tool is AWS Macie, an AI/ML service that helps identify and classify sensitive data to protect privacy and security. AWS also has a number of infrastructure security services like AWS Inspector and Shield that help protect data.

<<< Start >>>

<<< End >>>

Accenture has developed a no-code accelerator, Intelligent Data Foundation Accelerator (IDF), is easily  deployed on AWS Native/Serverless technology. IDF automates end-to-end data pipeline design, implementation and deployment.

When done right, a properly governed data foundation on cloud can deliver tangible, sustainable value. Consider West Midlands Police (WMP), the second-largest force in England, serving nearly 2.8 million people. WMP built a modern, scalable data foundation on AWS, which now provides the force with accurate information on people, crime and trends that is easily accessible via smart phones. More importantly, officers have data they can trust, at their fingertips. The solution was designed to automatically reload fresh data from legacy systems every few hours, which is critical any time but especially when lives are at stake.

Key considerations for data governance in AWS

When looking at data governance in AWS, there are three key aspects companies should consider:

1. Securing your data in AWS
   To preserve its value, your data must be highly available and secure—in transit and at rest—with measures to control who can access it. In AWS, you can keep multiple instances of your data in separate locations, called Availability Zones, to protect it and enable high availability. To help secure your data in transit, AWS provides dedicated network connections, and when data is at rest, you can take advantage of AWS KMS or AWS CloudHSM encryption services, so no one except your authorized users can see it. You can also augment these AWS capabilities with security control templates, which allow data access and exploration based on data classification controls and established governance policies. Establishing role-based controls over who can access your data, tagging and classifying the data is key. Carefully managed metadata makes it possible to find and use what’s relevant. At Accenture, we enable these capabilities with metadata management services and a Data Marketplace offering that are part of our Intelligent Data Foundations for AWS.
2. Improving regulatory compliance
   There are many different regulations with which companies must comply and they vary widely by industry. AWS helps simplify compliance by certifying their infrastructure and services specifically in conformance with the requirements of sectors such as finance, healthcare, and defense. You just need to consider your data foundations and applications. We approach this for clients with frameworks and tools within our Intelligent Data Foundations for AWS that help build data privacy into your data foundations from the start. Data sovereignty is also an important part of compliance for many companies, which can be addressed using AWS Local Zones to keep data within specified countries or regions. The key is then to control who can access that data—and from where—which can also be achieved with our metadata management services and Data Marketplace.
3. Managing data lifecycles
   Data volumes are growing extremely quickly but not all data remains relevant as it ages. Therefore, it’s important to continually manage data lifecycles. Some data must be highly available and instantly accessible for critical workloads and AWS provides appropriate storage to meet those needs. As it ages and is accessed less frequently, data can be moved to lower-cost storage tiers or data archives. AWS helps you do this intelligently and cost-effectively across multiple classes of its Simple Storage Service (S3) offerings. However, moving data to the right tier at the right time is crucial. This can be automated with AI, but it requires careful assessment of business needs relative to cost over time. Once again, finding the data you need requires effective metadata management to clearly indicate what the data represents, where it came from, how old it is, its access restrictions, and other descriptive information. AI is essential to do this at the massive scale we’re talking about today. You may want to build a universal metadata store to make it quick and easy for people to explore and find what they’re looking for across all storage tiers.

Accelerate deployment of governed data models

To gain the full worth and value of data, it should be readily accessible by people and processes across your enterprise, but it should  also be of high quality and veracity. This requires effective data governance, enabled at scale using cloud technologies such as AI and machine learning. AWS brings strong data governance capabilities, which can be further enhanced and extended with third-party tools and assets to help accelerate deployment of governed data models on AWS that support a range of industry-specific use cases. When all these elements come together, you have a powerful cloud data foundation to drive growth, innovation and sustainable value for your business.