Why search is no longer all about Google

If you could not find what you wanted by typing a few words into that familiar search box and hitting return, then it may as well not exist.

Google was the web.

"For a long time search was all about the bag of words," said Stephen Emmott, an expert in search engines at consultants Gartner.

Google prospered because it had a bigger bag of words than anyone else, and it was able to pluck what you wanted out of its bag quicker than anyone else.

It gave fast, accurate access to the website, blog or Wikipedia page people sought.

In those days searching, as a computer operation, was quite straightforward, said Mr Emmott.

The sheer size of the web meant there was, and still is, a lot of information to index but it tended to stay in the same place. Doing a good job involved analysing the words on the web pages and logging how many other sites saw that page as definitive.

Now, said Mr Emmott, searching has got a lot more complicated thanks to our increasingly complicated online and business lives.

Fast response

Instead of just looking up web pages, modern life can include finding a date or a soul mate, scoring a second-hand bargain in an auction, calling up instant taxi services or streaming more movies than you could watch in a lifetime of utter sloth.

"One way or another if you use applications throughout the day you will be touching a lot of different search engines and services," he said.
 

These days, most of those searches will not involve Google technology. Google declined to comment.

Instead, there are new pretenders to the search crown such as Elastic and Solr.

Searching in the old days was about typing text. Not so today. Searching can involve swiping right, moving a map with your fingers or talking to an app, said Shay Banon, founder of Elastic, which makes the open source search technology used by the likes of Tinder, eBay, Uber, Lyft and Netflix.

Behind the search box, the mechanics of finding the right answer are very different, he said.

For instance, on Tinder when you swipe right on a profile, that is a search in that it involves matching data against a constantly shifting set of parameters. It's just not a search as Google classically defined it.

Uber and Lyft also have to match against location as well as the preferences of both their drivers and riders. Similarly, Netflix and eBay do a lot of number crunching to answer queries and make suggestions for their massive user populations.

Food facts

Mr Banon wrote the first version of Elastic to help his wife who was studying to be a cordon bleu chef.

"I decided to write a recipe app for her and needed to figure out how to add a search box to it to look through all the knowledge she was accumulating," he said.

Just indexing the information in all the recipes, techniques and tricks she was learning was not enough, he said.

"I needed a search engine that was highly curated to her experience and her knowledge from the culinary world," said Mr Banon.

That step involved representing relationships between the different elements and organising the information so it could be queried quickly.

Tinder, for instance, uses Elastic to manage more than 300 million search queries every day.

And just as modern web businesses rely on search to keep them running, almost every business has realised that search is a basic function they have to get right, said Mr Emmott from Gartner.

That is for a couple of reasons, he said. Good analysis of customer data, a search in all but name, can reveal important unseen relationships or snags in a sales process that need smoothing out.
 

Fresh insights

For organisations such as Netflix and Tinder, organising themselves to find fresh insights is straight-forward, said Haydn Jones, founder of data science firm Alqami and a veteran of large-scale engineering projects.

"Netflix, Lyft, Uber and the like started with a blank sheet of paper," he said.

As a result they could choose what technology to use and did not have to worry about the different bits not working well together because they record data in different formats or according to different protocols.

That is a luxury few long-established companies have, he said.

Modern search technologies can help because they are much better at handling different sorts of data and extracting useful information from them - it's rarely just about that big bag of words.

And, he said, it was well worth going through the process to analyse the data and see what squeezing it via search can produce.

Alqami had helped many organisations realise several new use cases for their data because searching through it more efficiently revealed insights they did not know they had.

"Where there is muck there is brass when it comes to data," he said.