Comparison platforms · 3 min read

What 24 Years of Running Review Platforms Taught Me About Reviews

A review is only worth something if you can answer two questions about it: who wrote this, and on what basis. A ranking is only worth something if you also know who paid.

By Jeev TrikaFirst published 4 November 2021Updated 6 October 2026

Most advice about online reviews is written by people who have collected them. I have spent twenty-four years on the other side, building the places where they get collected.

The first one was an accident. In 2002 I was working for a nonprofit in Indiana and built topseos.com in my spare time to keep track of what digital marketing agencies were doing and rate them. I had no plan to make money from it. A year later an agency in Dallas offered me $300 to put their banner on the site, and the thing I had not understood became obvious: the people reading were about to spend money, and the companies they were choosing between would pay to be in front of them at that moment.

That is the whole business. Everything since has been a variation on it, across roughly 250 platforms.

Who wrote it, and on what basis

Here is the problem with reviews as most of the internet shows them. You are given a number. You are not told who produced it, how it was weighted, or whether the person writing had anything at stake. Five stars from an anonymous address and five stars from a verified customer of three years are presented as the same fact.

They are not the same fact.

When I specified the provider profile for CrowdReviews in 2017, the list of what a buyer should see on the page included, alongside the average rating:

the percentage of reviews that were verified;
the percentage that came from actual clients;
how many were written and how many were on video;
and a “penalty box” with yes-or-no flags for fake reviews and platform abuse.

None of that makes a review true. What it does is give the reader the questions to ask about it, which is more than most review pages do.

Who paid

Comparison platforms make money from the companies being compared. Mine did. So do almost all of the others, including several that present themselves as neutral.

Be precise about what the money bought, because this is where these businesses get slippery. On my platforms, providers paid to be considered for inclusion, and many contracted for specific positions in the rankings at a monthly price. That was the commercial model, and it is why the sites could exist at all.

So when you read any ranking, mine included, the useful question is not only “how was this scored?” It is “which of these positions were paid for?” A site that will not answer that in one sentence has answered it.

What I would check before trusting a ranking

Who pays the site? If the companies being ranked are the customers, assume position and payment are connected until you are told otherwise.
Is anything about the reviewers disclosed? Verified, from a client, anonymous: the share of each matters more than the average.
When was it last updated? A page that has not moved in two years is a brochure with numbers on it.
Does it match what you hear elsewhere? A ranking is one input. References from people who actually hired the firm are another, and usually a better one.

If you are collecting reviews for your own business

The 2021 version of this article suggested rewarding customers for leaving reviews. Don’t. Google’s policy for Maps prohibits reviews in exchange for incentives such as payment, discounts or free goods, and since 2024 the FTC’s rule on consumer reviews prohibits offering compensation conditioned, expressly or by implication, on a review expressing a particular sentiment.

Ask every customer, not just the happy ones, and ask while the work is fresh. Answer the bad reviews in public, without arguing. A page of unbroken praise reads as manufactured. A page where somebody complained and you fixed it reads as real.