Review sites rank products on numerical scales across every consumer category. Very few explain how the number was produced.
Comparison and review sites have become a default layer between consumers and markets, particularly in categories where the products are hard to evaluate directly. They produce scores — 8.4, four and a half stars, “excellent” — presented with the authority of measurement.
In most cases the measurement is not described anywhere. The number is an opinion formatted to look like data, and the formatting is doing considerable work.
There is a testable standard for whether a ranking is worth anything, and it applies whether the subject is washing machines, insurance or gambling operators.
Five things a real methodology contains
- Named criteria with declared weights. Not “we consider a range of factors” but a specific list with the contribution of each stated. Without weights, a composite score cannot be reconstructed and therefore cannot be checked.
- Disqualifying criteria. Conditions that rule a product out regardless of how it performs elsewhere. Their presence indicates a framework with a floor rather than one where everything scores acceptably.
- A statement of what cannot be verified. The strongest signal of good faith. Every reviewer has limits — things they cannot test, data they cannot access. A methodology that admits them is more credible than one implying omniscience.
- A stated review cadence. When assessments were made and how often they are revisited. A score from two years ago describing a fast-moving market is decoration.
- Disclosure of commercial relationships. Whether the publisher earns from what it ranks, and whether payment affects position. The second half is the part usually omitted.
How to spot a fake one
| Warning sign | What it indicates |
| Every item scores 4.5+ | No discrimination — the scale is decorative |
| No methodology page at all | Rankings cannot be checked |
| Identical review copy across items | Templated, not assessed |
| No dates on reviews | Currency unknowable |
| Nothing ever scores badly | Coverage limited to commercial partners |
The last is the most revealing and the hardest to detect, because it concerns what is absent. A site can be scrupulously honest about everything it reviews while quietly declining to review anything it cannot earn from.
What better practice looks like
The more credible end of the sector publishes the framework and applies it visibly, including where it produces unflattering results.
In gambling comparison specifically — a category with obvious incentive problems, since operators pay the publishers ranking them — the sites worth reading set out their weighted criteria and their disqualifying conditions explicitly. TopNonUKCasinoSites, for instance, publishes a six-pillar framework covering licensing standing, bonus fairness, payout reliability, game selection, payment variety and support quality, alongside a stated review cadence and an explicit section on what it cannot verify.
Whether a reader agrees with those weightings is a separate question. The point is that a published framework can be argued with. An unexplained 8.7 cannot.
The incentive problem, stated plainly
Any honest discussion of review sites has to address the structural conflict rather than talk around it.
Comparison publishers in most commercial categories are paid by the businesses they rank, usually on a performance basis. That creates an obvious incentive to rank well whoever pays best, and a subtler one to review only businesses that pay at all.
The second is harder to detect and probably more distorting. A publisher can apply its methodology scrupulously to everything it covers while quietly declining to cover anything outside its commercial arrangements. Nothing in the visible output reveals the omission.
There is no clean solution to this within the advertising-funded model. What exists are partial mitigations: publishing the methodology so rankings can be argued with, publishing scores that are unflattering, dating reviews, and disclosing the commercial relationship in terms specific enough to be meaningful.
A reader’s reasonable position is to use these publications for the verifiable factual layer — licensing, terms, stated processing times — and to treat the rankings themselves as informed advocacy. Useful, but produced by a business with an interest in the click.
The general lesson
Treat any consumer score as a starting point, and read the methodology before the ranking. If there is no methodology, you are reading advertising with a number on it — which may still be useful, provided you know that is what it is.
The test generalises beyond any one category. Wherever a score is presented without a stated method, ask what would have to be true for the number to be checkable — and notice that in most cases nothing would be, because no method was ever published. That is not evidence of dishonesty. It is evidence that the number was produced for persuasion rather than for measurement, and it should be read accordingly.
The same test applies to this article and to anything else offering you a framework for evaluating things. Ask what would have to be published for the claim to be checkable, and notice whether it was.
The genuinely valuable material in most review content is the verifiable factual layer underneath the score: which regulator licensed a company, what the terms actually say, what the published processing times are. Those can be checked independently, and a publisher that gets them consistently right has earned some credibility on the judgements that cannot.
Where gambling is concerned, one fact overrides any score: operators outside British licensing are not covered by GAMSTOP and offer no British dispute route. Anyone registered with the scheme should keep that protection. Free confidential support: BeGambleAware. 18+.
