Search and unstructured data analytics: 5 trends to watch in 2020

Most organizations are making good use of structured data (tables, spreadsheets, etc.), but a lot of untapped business-critical insights are held in unstructured data.

<<< Start >>>

<<< End >>>

Nearly 80% of data in the enterprise is unstructured – work descriptions, résumés, emails, text documents, research and legal reports, voice recordings, videos, images, and social media posts. While this data used to be very difficult to process and use, new technology developments in Neural Networks, Search Engines, and Machine Learning are expanding our ability to use unstructured content for enterprise knowledge discovery, search, business insights, and actions.

Search plus AI is solving real-world problems

Think about the apps on your smartphone – Siri, Alexa, Shazam, Lyft, etc. You may not realize this, but they’re all powered by an army of search engines working behind the scenes. Combining search with AI technologies like Natural Language Processing, Neural Networks, and Machine Learning, these apps can process your voice commands or text input, search across diverse data sources, and return the needed answers, all in real-time and with great accuracy.

Inside the enterprise, these technologies can connect your employees to the content and the answers they need, regardless of where the answer may reside – in a document, a financial system, an HR system, or a policies and procedures database.

<<< Start >>>

<<< End >>>

Search has evolved from finding documents to providing answers

And as we’re starting 2020, we expect to see more AI-powered search and search-based analytics applications supporting enterprises.

Here are the top five trends to watch in the search and unstructured data analytics space.

1.   Neural networks and search engines

Revealed in Accenture’s Fjord Trends 2020 as a key technology supporting innovative enterprise AI systems, neural networks can “learn” to perform tasks through pattern recognition. By analyzing vast amounts of digital data, neural networks can learn to identify photos, recognize voice commands, and respond to natural language search queries. Going beyond simple keyword search, neural networks enable the search engine to understand the user’s meaning and intent so that the most personalized, relevant results are provided.

The latest neural networks (BERT and its derivatives) are able to create a “semantic space” – an abstract understanding of the enterprise content – which can be used for:

• Deeper search: to identify sentences that have the same meaning rather than just containing the same search keywords (e.g. “company expense policy” and “business travel reimbursement”)
• Better classification: to classify content for better navigation or management (e.g. compliance, sifting, remediation, etc.)
• Question/Answer: to extract facts from documents to answer specific questions with reference back to the original source material (e.g. “What was U.S. revenue last quarter?”)

These neural networks are already being used for highly-curated content, such as Knowledge Base Articles, policies and procedures, documentation, testing standards, and so on. In the next couple of years, we expect to see more organizations applying neural networks to better understand their document content and user queries, providing highly-relevant, context-based answers.

2.   Semantic search

Expanding on neural networks, semantic search handles a wide range of enterprise users’ queries and requests and can respond with up-to-the-second answers directly from business systems. This allows semantic search to become a single point of access for documents, questions, facts, and business data required by a user community. Intended to provide precise, accurate, and up-to-the-minute answers to users’ questions, both short-tail and long-tail, semantic search is comprised of four parts:

• Understand the entities (business objects) in the query
• Understand the intent of the query
• Map the request to an answer agent
• Fetch the answer and report it back to the end-user

Semantic search has evolved search engines away from displaying lists of results based on keywords to understanding the intention of those words and surfacing targeted content that users really need. If a user is searching for “Q1 revenue,” he/she probably isn’t looking for a list of results containing “Q1 revenue” but rather, a quick response, such as “$123 million.” What's more? Perhaps the revenue figure can even be broken down by market segment for good measure.

The rise of semantic search is supported by a number of factors:

• The growth of data warehousing, data lakes, and content ingestion technologies are breaking down data silos, making valuable content readily available across organizations.
• The emergence of new tools designed for implementing semantic search for business applications helps organizations solve the integration challenge and dramatically reduce the cost of implementation.
• New machine learning methods, such as advanced neural networks, allow semantic search engines to better understand users’ search requests, analyze the objects in the query, and map queries to intents and identify answer agents.

Read my short article for a closer look at semantic search and example business use cases.

3.   Document understanding

When computers read documents, they don’t pay attention to stylistic details, such as where a word is on a page or how it might relate to other words. But presentation elements – positioning, colors, fonts, graphical elements and so on – contain important semantic information that text alone cannot convey. As humans, we understand all of this without thinking. We know, for instance, that font size can denote importance and that positioning of a headline, paragraph, or image can influence the meaning of these items within the document. However, since computers currently ignore most of these presentational elements, organizations are unable to extract substantial value from their documents.

AI is making it possible to extract insights from unstructured content by examining those presentational elements. Intelligent document processing engines can be trained to read this presentational information and deliver insights to end-users. Imagine the various enterprise use cases in which document understanding can be leveraged:

• Automated PDF invoice processing: extracting tables, totals, name/value pairs
• Converting from paper procedures to electronic procedures: from Batch Records to Electronic Batch Records for pharmaceutical manufacture; or from PDFs to Lab Information Management System records for lab test procedures
• PowerPoint content search: search for slides, highlight searches within the slide, extract titles, remove footers
• Search on geo-scientific reports: find well logs, seismic cross-sections, maps, and other elements, and correlate these items with geographic locations on the globe
• Automated mail routing and form-filling: to reduce processing time for mail items, including both snail mail and e-mail
• Automated conversion of engineering drawings: to bills of materials, and, ultimately, to connectivity and flow graphs
• Policy and procedure document search: searching and matching individual paragraphs, or extracting direct answers from the text
• And many others

Read more about how we built these document understanding applications for enterprises.

4.   Image and voice search

The 2019 Accenture Digital Consumer Survey found that about one-half of respondents already use a Digital Voice Assistant (DVA) and 14 percent plan to purchase in the next 12 months. Virtual assistants – Siri, Alexa, Google Assistant, etc. – are becoming ubiquitous. Powered by AI technologies, they enable conversations between humans and computers in everyday interactions. They bring deeper natural language understanding to not only enhance search but also provide an entirely new way to find information.

Voice assistants have made an entrance into the enterprise, enabling customers and employees to interact more easily with corporate data. For instance, instead of typing in keywords, employees can now ask “Who are our data science experts in Europe?” or “How do I book a conference room in the Paris office?” Externally, voice and image search functionalities go beyond traditional text search to provide customers and partners easier ways to find information on a company’s website. 

<<< Start >>>

<<< End >>>

There is a natural synergy between these tools and semantic search (discussed above). In many cases, chatbots can be removed – the back-end can be completely handled by a robust and comprehensive semantic search engine.

5.   Knowledge graphs

Following our prediction from last year, developments in knowledge graphs will continue to power smarter search interactions across the enterprise.

Aggregating the organization’s existing data into one repository – often an enterprise data lake – is a starting point. But how do we make use of this data? We need to add context, relationships, and meaning to it. From fragmented data records from various enterprise functions, natural language understanding (NLU) algorithms can create an interconnected information network indicating how data records are linked to each other, creating the enterprise knowledge graph. Search engines and Question/Answer systems can then instantaneously pull a snapshot of connected information and deliver relevant insight when the user asks a question.

Note that knowledge graphs can span a wide range of complexity:

• Modestly interconnected:

• • Employees and employee information
  • Business units and key team members
  • Office locations
  • Products and support staff
  • Physical plan machinery location

• Richly interconnected:

• • Organizational hierarchies
  • Office corridors, stairways, and conference room locations
  • Machine parts and their proximity/interconnectedness
  • Product categories, lineage, and matching accessories
  • Physical plant machinery interconnectedness
  • Customers, contacts, sales staff, and products purchased
  • Policies and procedures constraints, conditionals, and requirements

The knowledge graph will be ever-growing as new data points and insightful relationships are added indefinitely.

Beyond search

Looking ahead into 2020 and the coming years, we expect these five developments to further evolve and be leveraged more broadly within the enterprise. The emphasis will be placed on how to apply these intelligent technologies to discover and maximize the use of unstructured data. Going beyond traditional search applications, new search-and-AI-powered use cases are invented every day to deliver more value and better outcomes. As AI technologies and approaches are refined, they can be used by organizations to solve both technical and organizational challenges at lower costs and with more powerful results. With practical strategy, domain expertise, and expert implementation, organizations can unlock endless opportunities for innovation.

What value will you unlock from your enterprise data? Share your use case with us and discuss how we can help.

Reference

*"Gartner Top Strategic Predictions for 2018 and Beyond" authored by Kasey Panetta was published on October 3, 2017. To read more, visit: https://www.gartner.com/smarterwithgartner/gartner-top-strategic-predictions-for-2018-and-beyond/ 

<<< Start >>>

---

What value will you unlock from your enterprise data?

CONNECT WITH US TO DISCUSS YOUR USE CASE

---

<<< End >>>