What goes into a great search experience?

The Elements of Search [Infographic]

Does AI Search Impact Site Performance and Cost?

Does AI Search Impact Site Performance and Cost?

AI technology is really coming into its own in 2022. New headlines pop up daily about how machine learning is mastering language or eliminating car crashes

Site search is no different, but AI rollout into site search has been slow. One of the things we learned in the new WBR Insights report on site search, is that many executives are concerned about AI — its impact on search performance and cost. That perception is likely slowing adoption. 

impediments to site search survey chart
Download a copy of the WBR Insights site search report.

So, are the concerns based in reality? 

  • Will AI search impact your site's speed? It shouldn’t. 
  • Will AI cost more? Again, it shouldn’t.

So why are companies concerned about AI costs and performance? I think it’s two things. 

  • Vector search 
  • Vector search


What is vector search?

Vectors use math rather than keywords to power search results. Words are converted into mathematical representations and their relationships are mapped. Vector search can easily handle queries that would be complex or impossible for keyword-based search engines. It also opens up an entirely new frontier of semantic, or concept, search results. 

Vector search has been around since 2013 but never really taken off, however. New entrants in the vector search business are still popping up with better algorithms and APIs that, they claim, are poised to improve search.

Vector search would be great, and everyone would be using it, except for two big problems. 

  • It’s slow
  • It’s hard to scale

To solve both problems, you need a lot of very fast GPUs. That kind of infrastructure gets expensive. So, companies must make a tradeoff between cost or speed. 

Some would say that caching is a good way around this problem. The argument goes that by caching results you can virtually eliminate costs and provide results instantly.

Years ago, we actually tested this idea with a client with a large catalog. What we found was that caching was rarely useful — the cache rate of search was low especially for sites with massive longtail content. 

The need for speed in search is a real challenge. Slower queries impact e-commerce conversion rates and reduce engagement. If your site is lagging, the likelihood that someone bounces from your site goes up.

Amazon showed that every 100 millisecond delay loses 1% of revenue. Similarly, Google showed a 500 msec delay reduced engagement by 20%. (source)

Bottlenecks are also more likely with vector search because longer queries take to process. Machines divide CPU time between various inbound processes. To cope, search engines either need more compute power or must instead process the same queries faster.

Vector search companies have been pushing the benefits of vector AI for nearly 10 years, but the cost and performance issues have impeded its progress and engendered concerns about its viability. 

To get around the problem, some companies now offer vector search modules as add-ons to their regular search solution. It costs a premium, and is typically only run if the keyword search result is poor. The message is that you can have one or the other — keywords or vectors, speed or quality — but not both running at the same time. 

With neural hashing and hybrid search, that’s changed.

Introducing neural hashing

Vectors work, but as mentioned above, have speed and scale limitations that affect performance and cost. We took a different approach, called neural hashing, that leverages vectors without tradeoffs. 

neural search vectorization

Neural hashing makes vector-based search as fast as keyword search and this is done without the need for GPUs or specialized hardware. Neural hashing basically uses neural networks to hash vectors. Specifically they compress the vectors into binary hashes.

This means incredibly intelligent AI search can now be done even for instant search (as you type) with a cost-base in line with existing keyword search technology. Performance-wise, neural hashes can be run on commodity hardware, retain up to 99% of the vector information, and can be calculated ~500 times faster than vectors alone. 

Neural hashes alone provide incredible results for most datasets and queries. We actually take a hybrid approach by combining neural hashes with keyword search. Combined, we call it Neuralsearch®

We don’t charge a premium for Neuralsearch — everyone who signs up for a free trial has Neuralsearch — and it doesn’t impact search performance. Results are as much as 10 times faster than vector search. Plus, it scales massively without any issues or added costs. 

Learn more about how Neuralsearch works in our neural hashing whitepaper

AI for site search

There’s no reason to hold off from upgrading your site search to AI-powered search. With Neuralsearch, performance and cost issues are a thing of the past. 

Neuralsearch is still in beta, but soon we will release a new front-end UI to Neuralsearch that gives customers the ability to tweak results with the click of a button. Neuralsearch eliminates the need to manually create relevance rules, saving your team hours of time. 

Don’t just take our word for it, sign up today for a free 14-day trial, or schedule a custom demo to see how Neuralsearch will work for your site.

WB Insights research report

Similar articles

User Experience (UX)
Best Practice

Drastically Improve Your Website's UX by Improving Site Search

Best Practice

An Introduction to Custom Search Engines

Best Practice

How to Get a Competitive Edge with Smart (and Surprisingly Easy) E-commerce Site Search Solutions