How to select an enterprise search engine

In the first part of this blog series, we visualized in detail our journey to smarter enterprise search: our starting point, the landmarks to visit, and an envisioned destination. This follow-up blog post is about navigating to one of the landmarks we previously defined: selecting an enterprise search engine.

It’s tempting to think that search engine selection is a technical task: Which engine is better than the rest? Yet, you’d likely find that the differences across search engines tend to be minimal when purely considering search engine functionality. The differences are more noticeable though, when comparing the added AI-powered cognitive functionality offered by intelligent search engines. Still, there are multiple variables to think about throughout this leg of our journey.

I’ll describe the steps that have worked well for our clients when selecting their new search engines.

<<< Start >>>

<<< End >>>

5 steps to select an enterprise search engine

STEP 1: Identify potential search engines

Let’s start with a list of all search engines that might work for your needs.

The first source for the list would be your current vendors. Chances are that you already have two or more search engines functioning somewhere in your organization. Any currently deployed search engine maintained and supported by a vendor or an active open source community should be considered a candidate. It’s fine if your search engine hasn’t been upgraded to the latest stable version. In such case, add the search engine’s latest release to your list for consideration so that you’ll eventually compare the latest version with other options.

The second source could be analyst reports, such as Gartner’s Magic Quadrant for Insight Engines or the Forrester Wave™ Cognitive Search report. Be sure to look for the latest ones. These sources provide good overview information for your research.

If you’re in e-commerce or other specific areas, you might want to look for reports around targeted applications with a strong embedded search, in addition to specialized functionality for your industry. In this case, you may not be looking for an enterprise search engine but a more use case focused search solution. This blog still applies for selecting such search platform.

Industry analysts often create their lists based on certain conditions and may not produce an exhaustive list. Thus, your third source to complete your list would be any search engine that you might have read or heard about. It could be an offering you haven’t used from your existing vendor. Or perhaps something you saw in a marketing email, a conference, a webinar, etc.

STEP 2: Narrow your long list of candidate search engines

If your list has over a dozen search engines, I suggest you narrow it down to a few candidates – meaning five or fewer. For the types of evaluation that we typically do, I prefer to work with three engines maximum.

To remove some candidates from the list, I like to start by checking each candidate against the major deal breakers. Usually, I’m able to disqualify some with little work. The list below illustrates some of the potential showstoppers I’ve seen in the past. Each organization is different, and some may have policies or directives against or in favor of one or more of the items below. So, consider your current circumstances and future expectations when going through each one.

• Self-hosted. This is the Do-it-Yourself (DIY) model. Whether it’s in your data center or cloud-based virtual machine, you’re responsible for deploying, configuring, maintaining, and updating the search engine. Many organizations are walking away from this traditional model to avoid the need to manage the software internally. If you prefer managed services, any self-hosted engine would fall off the list.
• Software-as-a-Service (SaaS) or Platform-as-a-Service (PaaS) from a search engine vendor. These are managed cloud services, such as AWS Elasticsearch or Amazon Kendra, Google Cloud Search, Azure Cognitive Search, etc. I’ve had clients who prefer PaaS over SaaS due to the additional data control provided by the PaaS approach. You might want to check with your security, privacy, or legal teams regarding compliance. This can help disqualify some candidates quickly.
• Closed engine. You may be familiar with the now discontinued Google Search Appliance (GSA). It was great for some applications or organizations, yet inadequate for others. It was mostly a black box solution. While there are offerings like the GSA, the need for customization or more control would disqualify a closed engine.
• Hybrid. There are multiple variants of hybrids. It may be a self-hosted search engine integrated with a recommendations service; a combination of your private cloud with your on-premise; or your private cloud with a third-party cloud service; etc. These are more complicated solutions, but there are valid reasons for organizations to require such deployments. Some search engines don’t perform well in hybrid solutions and thus aren’t candidates for evaluation.

Depending on your organizational requirements, you may have a more specific set of items. Maybe there’s limitation based on a list of pre-approved vendors because onboarding a new one may be too time consuming or complicated. The goal is to quickly, without much analysis, cross off some search engines from the list. Remember, we’re trying to narrow our list to the most promising candidates, hopefully down to three or a manageable list, for a deeper comparison.

STEP 3: Define your evaluation criteria

In my experience, your chances of choosing a search engine that will be useful for many years increase when you engage with multiple stakeholders. Work with your current search stakeholders but don’t forget the future ones. Considering both, current and future search clients, would allow you to better evaluate the options out there.

<<< Start >>>

<<< End >>>

Below are some general categories for your evaluation criteria. I’ll drill down into each category and outline the specific elements that our clients typically require or would like to have. 

• Connectors or crawlers. These are mechanisms for loading data from the sources into the search engine. How many connectors does the search engine have for the data sources you’d need to index? You should include the sources that would likely be indexed in the future in addition to what you must index now. If you’re planning on decommissioning a source in a year or two, you may want to exclude that source as you may not want to go through the effort of indexing it before its data is migrated to a newer source.
• Data processing prior to indexing. Preparing your data for indexing is one of the most valuable activities but often overlooked in a search implementation. Data needs cleansing, normalization, or enrichment for improved findability, search relevance calculations, filtering, sorting, or other needs. Some search engines include out-of-the-box data processors and support custom processors for your specific data processing needs.
• Query processing. Search terms, or in some cases, unstructured text used for querying also benefit from some preparation on the search side. Like how it’s done for indexing, query cleansing, normalization, or enrichment would enable the search engine to be better at finding matching documents or scoring them by relevance. Some search engines provide query parsers out-of-the-box with specific intents you may use. Finally, look for extensibility capabilities as you may need to add custom query components in the future.
• Linguistics support. If your content comes in multiple languages, the support or extensibility capabilities could be a key reason to choose one engine over another. Linguistics are usually applied on both the index side and the query side. Linguistics may be applied as a processing pipeline component or a text analytics feature.
• Third-party systems integration. Some search engines have evolved over time with a strong partnership with a content management system or software, perhaps even powering the search features within that software. In those cases, the search engine probably already has native integration with the other software. This is an accelerator for your specific search-powered requirements.
• Search results security trimming. Enterprise search applications must guarantee that a user can only get search results from the data sets intended for them. Many search engines offer access controls down to a document level or metadata field. Yet, some search engines are flexible enough to allow for field-level security. Some engines do not offer security trimming out-of-the-box but can support it through custom integrations or plugins.
• User Interface (UI) toolkit. While you may have your own UI development team, you may need out-of-the-box UI components that facilitate the integration of your search client applications. Some engines come with such components; others allow you to create standalone search applications or complete SEarch Results Pages (SERP) to embed within your own systems.
• Search analytics and site analytics. Search engines usually generate or allow for the generation of search signals or events. The growing functionality for search and site analytics enables intelligent search engines to serve more relevant and personalized search results. These analytics features can use Machine Learning (ML) or other advanced approaches to analyze the signals or generate insights.
• Advanced Artificial Intelligence (AI) features. Intelligent search engines gain their qualifier based on the AI functionality that they offer. Automated tuning of relevance scoring, ML-based query suggestions, recommendations, query intent, and various other AI-powered features aren’t standard across search engines and may be a reason for choosing one over another.
• Licensing model. As with any software, licensing is critical. The model used by the vendor dictates cost, extensibility, scalability, or other conditions that need to be carefully analyzed for your requirements.
• Testing support. Some engines have built in capabilities to do A/B testing, ML models testing or comparisons, relevance rank evaluation, and others. I’m glad to see these features being added to make relevancy improvement easier for product owners, search administrators, and developers.

You may expand the list above with other criteria, like administration user interface, software development kit (SDK), logging, monitoring, documentation, or other areas that may be of high interest to you.

STEP 4: Evaluate your candidate search engines against the criteria

You should now have your three or so candidates, along with the criteria for evaluation. My peers and I have generated multiple spreadsheets with for search engine evaluations over the years. The general process is as followed:

1. Create a table
2. Enumerate all criteria you’ve defined
3. Determine a weight for each criterion
4. Evaluate each criterion for all candidate search engines
5. Multiply your evaluation for that criterion with the assigned weight, which generates the score of the criterion for each engine
6. Sum up the scores across all criteria for the search engine

After step 4, you should have all criteria evaluated for all potential search engines. This step involves research of the search engines’ documentation, consulting a search engine expert, and, in some cases, reaching out to the vendors.

STEP 5: Review your score card and select the best fit

The purpose of the spreadsheet is to provide an objective evaluation of potential search engines. This step should be simple since the spreadsheet already calculates a score for each category as well as a total score for each search engine.

Often though, the total scores aren’t that different across the options. That’s when the categories come handy. You may base your final engine selection on certain categories that are more important to your needs. If you choose to focus on comparing subtotal scores from some categories, avoid having a highly subjective factor that can potentially cause biases in the final selection.

The next leg of the journey: Planning your search engine implementation

Congratulations! You have selected your next enterprise search engine after careful evaluation. The journey continues but there’s still a lot to do before the implementation begins:

• planning implementation of the new search engine,
• preparing a multi-disciplinary team to ensure a successful implementation,
• planning support for the existing engine(s),
• training your staff on the new engine,
• and various other things.

It could be overwhelming… Thus, it’s essential to plan out the next leg of your journey. Remember the landmarks I described in the first part of this series? During the search engine selection process, you’d likely identify additional landmarks and figure out how to get to them.
I trust that you’d have a better idea of next steps after evaluating candidate search engines against your detailed requirements and expectations. For example, you might need to accommodate your resources to maintain the current search engine(s) while implementing the new one. You might need to decouple search from some existing applications, perhaps even develop an API layer to minimize the effect of changing search engines later. So, make sure you visit those preparation landmarks before implementing your selected search engine.

Accenture’s Search and Content Analytics can add value to your enterprise search strategy, from defining the right path to selecting, implementing, and maintaining your search engine. Connect with us to see how we can help achieve your next enterprise search vision. Happy finding!

---

Planning to implement or improve your enterprise search engine?

CONNECT WITH US TO SEE HOW WE CAN HELP

---