Fabio Manganiello

In search of a new search

How to tackle Google's monopoly while making search better for everyone

Published by Fabio Manganiello on Aug 12, 2024
.

Yes, #Google has been a monopolist in the search and ads space for the past two decades.

And its self-fed profit machine should have been stopped before it became to big to fail and to big to compete with.

And yes, Google got such a stronghold that it actively invested energies in enshittifying its own results by prioritizing revenue-making entries that results got noticeably worse for everyone, and yet it's unlikely to see a single percentage point of market share eroded.

And yes, Google is probably going to appeal the decision anyway, using the same arguments they've been using for years ("it's not that we're monopolists, it's that other like our money and want us to be there").

But the crazy part of the story is that, while everybody agrees that Google is a monopolist which gained its position by basically bribing everyone away from competing with it, nobody knows who to deal with it.

That's because everybody knows that crawling the web and building a decent modern search engine compliant with modern constraints is hard. And it's even harder to do it while being profitable. And there's usually a relation of inverse proportionality between the profitability of a search engine and the quality of its results.

That's why Google prefers to pay $21B a year to the likes of Apple, Samsung, Motorola and friends just to be the default search engine - in their browsers or on their devices. It's not necessarily that those companies don't have enough resources to build a competing search engine in house. It's just that they estimated how much it would cost them to make and maintain their own search engine, versus how much Google would pay them to stay out of the business and let it be the default, and they realized that they'd better just shut up and take Google's money.

Now lawmakers are coming and saying "hey, Google has bribed you to stay out of its business, now it's time to stop accepting its money". So what? Force Apple or Samsung to build their own search engine from scratch, and end up like an even worse version of Bing, or like Apple's first version of maps?

On a second thought, why not?

Why not establish that if you're either a big creator or big consumer of online content then you should also contribute your little part in keeping the web searchable? That ensuring that the house is clean enough for people to find things is everybody's job - especially if you make a lot of money with at least a piece of that house?

Metasearch to the rescue

Maybe not why take the meta-search approach of #Searxng and make it the norm?

Maybe we don't need many search engines that are able to compete with the largest one on the market that built its monopoly over 25 years - or, worse, try and reinvent the business model from scratch in a short time and solely forced by regulation, preferably with ads and sponsored content playing the smallest possible part.

But we can all benefit from many companies that all play their little part to keep the web searchable. Public resources need public protocols and low access+sharing barriers if you also want competition.

Large tech companies could all contribute for example by running crawlers that index and aggregate results from their clouds, user-generated content and clients. Or even by running it on larger portions of the Internet. Those crawlers and their algorithms should preferably be open-source, but probably they don't have to - although the APIs that they expose and their format should be open and uniform for compatibility reasons.

That's because their results can then be aggregated by open meta-engines like Searxng. You could also easily fine-tune them, just like you would do with a local Searxng instance - more results from search engine A? less from B? results from C only for Spanish? results from D only for academic papers? Let your search algorithms work the way you like, let thousands of custom search engines based on open protocols flourish. Let them compete on the quality of their results, or on the business niches that they decide to invest on, or experiment with new business models, and let open implementations aggregate their results and provide them to the users.

A given search engine decides to enshittify? It starts boosting extremist results? It starts to return only low-quality sponsored content on its first page? It is purchased by Evil Corp? Then the admin of a search aggregator can simply reduce the relevance on its results or remove them entirely. And users could choose whatever search aggregator they prefer. Or they could even tune relevancy of results themselves from their own settings. No need for regulators to scratch their heads on how to stop a monopoly. No need to ask ourselves how to prevent a single monopolist No need for anyone to be forced into accepting bribes. No need to ask all Big Tech companies to build their general-purpose search engine from scratch rather than providing Google as a default, or worse turn a monopoly into an oligopoly. Give your users all the freedom they want. Let them run their own aggregators. Or sign up to a search aggregator, free or commercial. Let them tune results from multiple search engines the way they like. And let the rebalanced mechanism of demand and supply based on open protocols but competing implementations regulate the market the way a healthy market is supposed to.

Or maybe revive the Semantic Web

Maybe we could even dust off semantic protocols (the "old", and probably real, Web 3.0), such as RDFS and OWL, to make web content really machine-readable. Those could even make the whole concept of a search engine with HTML/natural language scrapers obsolete.

The main obstacle against their implementation two decades ago was the burden of having to manually annotate your content just to make it machine readable - or worse maintain your own ontologies. Modern AI technologies have definitely changes those constraints - and this could be a good application of them.

The dinosaur in the room

Then there's still the #Mozilla problem on the table. If nobody should accept Google's bribes anymore, that includes Mozilla. And we all know that the $500M/year bribe that Google pays to Mozilla to be the default search engine in Firefox is basically the only serious recurring annual revenue that keeps Mozilla (barely) afloat.

Paradoxically, Google needs Mozilla because the existence of Firefox is the only thing that allows them to say "you see? there's an alternative browser with an alternative rendering engine out there! granted, it has <5% of the market, but it's there, so you can't say that Chrome/Blink has a complete market monopoly".

And Mozilla needs Google because, well, without their voluntary bribe they would be bankrupt - making the web actually even less competitive.

To their defense, it's not like Mozilla hasn't tried its best to diversify its offer and break free from its awkward relationship with Google. But whatever they've tried (from MDN, to their VPN, to Pocket, to Relay, to sponsored links on the home page, to the more recent moves in the field of data collection defaults and ads) has either proved to be too ambitious for their resources, or too underwhelming compared to established alternatives, or too controversial for their privacy-aware user base.

So maybe this could be the right chance to acknowledge that not only public resources needs public protocols, but that non-profit organizations that keep both competition and open protocols alive also need public funding - it's not fair to let a non-profit compete with a business giant at the giant's market rules.

Now that we all agree on who the bad guy has been all this time, this can be the right chance to do things right. Pass the "we know it's not right, but we all benefit from it" phase, get bold and think of new solutions. Because all the building blocks are already out there.