No SEO source is universally “the truth.”
A source is like a witness: it can be accurate and still be unable to testify about an event it did not observe. Write the claim first, then choose the source capable of observing or estimating that exact phenomenon.
Source Reliability Matrix
Can this source support this exact claim?
- Governing rule
- Write the claim first. Source fit controls the permitted wording.
- Signal language
- D Direct · C Conditional · E Estimate · N Not Supported
- Next action
- Inspect fit, keep confidence separate, add corroboration, and publish only supported wording.
- Evidence boundary
- Reviewer reliability is NOT RUN. Permissioned account validation remains NO_DATA.
Claim-by-Claim Source Reliability
How to read it: select a checkpoint to isolate that part of the method. The canvas shows the relationship; the inspector explains what the checkpoint changes in the decision.
The 40-cell source reliability matrix
Use the legend for meaning, scan the grid by claim for the reference answer, and use the checker for the rationale and permitted wording.
| Claim | GSC | Ads | SERP | Crawler | Third-party |
|---|---|---|---|---|---|
| Owned organic clicks | DDirect | NNot Supported | NNot Supported | NNot Supported | EEstimate |
| Market search demand | NNot Supported | CConditional | NNot Supported | NNot Supported | EEstimate |
| Domains visible for a query | NNot Supported | NNot Supported | DDirect | NNot Supported | CConditional |
| Technical implementation | CConditional | NNot Supported | NNot Supported | DDirect | CConditional |
| Competitor actual traffic | NNot Supported | NNot Supported | NNot Supported | NNot Supported | EEstimate |
| Causal impact of a change | CConditional | NNot Supported | CConditional | CConditional | EEstimate |
| Crawl/index eligibility | CConditional | NNot Supported | CConditional | DDirect | CConditional |
| Commercial keyword value | CConditional | CConditional | CConditional | NNot Supported | EEstimate |
The V1.4.0 workbook expands all 40 cells with rationale, confidence, evidence strength, corroboration, permitted wording, reviewer-study fields and an account-validation register.
Run a 30-second source-fit check
This guided mode covers the eight published benchmark claims. It is deterministic, not AI-generated, and does not replace case-specific review.
What Direct, Conditional, Estimate and Not Supported mean
Direct
The source measures or records the claimed phenomenon under stated conditions.
ExampleGSC clicks for a verified property and period.
Permitted wording“The source measured or recorded…”
Conditional
The source contributes material evidence, but another source or qualification is required.
ExampleGSC shows a traffic change but cannot prove the change was causal.
Permitted wording“The source supports the conclusion when combined with…”
Estimate
The source provides a modelled value suitable for comparison or exploration, not an exact fact.
ExampleA third-party platform estimates competitor traffic.
Permitted wording“The provider estimates…”
Not Supported
The source neither observes nor estimates the claim sufficiently.
ExampleA crawler cannot establish market demand.
Permitted wording“This source cannot establish…”
A source mismatch nearly created 70 unnecessary repairs
I built this matrix after the same failure repeated in SERPIFAI’s own editorial work: a source looked precise, but it had observed the wrong scope for the claim.
“Seventy pages have no H1.”
A scoped extractor returned zero because it searched only part of each page.
Replay the complete HTML
The full document, not the summary field, became the evidence unit.
Cancel the sitewide repair
The source could not support the claim it had been used to make.
Zero H1 fixes required
All 70 captured pages contained exactly one H1.
Open the other three documented decision corrections
Different inventory scopes
Claim: seven pages were missing.
Correction: the inventories represented different editorial properties.
Result: no page was falsely restored or declared missing.
Hash change without layer meaning
Claim: the old article was safe to overwrite.
Correction: compare body, head, hero and assets before rebasing.
Result: newer visual-system changes were preserved.
Stale publication status
Claim: the replacement was ready.
Correction: a release gate checked the actual review status.
Result: one inaccurate status was blocked with zero target writes.
What each source can and cannot prove
Google Search Console
Use for: measured Google Search clicks, impressions, CTR and average position on verified properties.
Do not use for: competitor actual traffic, total market demand or causality by itself.
Google Ads / Keyword Planner
Use for: demand planning, seasonality, advertiser competition and bid context under recorded settings.
Do not use for: exact organic traffic, realised commercial value or ranking feasibility.
Recorded SERPs
Use for: domains, pages and result features visible for a defined query, locale, language, device and time.
Do not use for: durable market share, total demand, actual clicks or why a result ranked.
Configured crawlers
Use for: observable implementation such as response, canonical, robots, headings, links, rendering and structured data.
Do not use for: rankings, demand, traffic, conversion or business value.
Third-party SEO platforms
Use for: scalable competitor discovery, keyword databases, backlink indexes, ranking history and modelled estimates.
Do not write: “The competitor received exactly this many visits.” Preserve provider, date, method and estimate language.
What to do when the sources do not agree
Sources conflict
Preserve both records. Check date, geography, device, entity definition, sample, transformation and collection method. Prefer the more direct and current source only for the claim it actually observes.
State: Review required until the difference is bounded.No direct source exists
Use a directional estimate only for comparison or exploration. State the provider and method, reduce claim strength and request corroboration before a material decision.
State: Estimate or unresolved—not fabricated precision.Scope is unknown
Do not assign D/C/E/N. Name the missing scope, owner, evidence request and review date.
State: Review required.Reviewers disagree
Keep ratings independent until every rationale is recorded. Compare the exact boundary interpretation before discussion and re-score only after the evidence packet changes.
State: Review required; classification withheld.What is validated—and what remains open
Method and first-party corrections
The 40-cell classification logic, source boundaries, four decision corrections and deterministic workbook are published.
Independent reviewer reliability
The next gate is blind classification of all 40 cells by three to five independent reviewers, followed by raw agreement, a suitable chance-corrected metric and a D/C/E/N confusion analysis.
Permissioned account validation
No permissioned GSC/Ads account dataset is included in this release. Account-level performance, provider-accuracy and commercial-outcome claims remain withheld.
Validation note: status is reported here; the authoritative publication boundary appears in the final Evidence Boundary section.
Where this matrix fits in the evidence system
Practical questions
Questions SEO teams ask
Is Search Console more accurate than Ahrefs or Semrush?
For measured clicks and impressions on a verified property, Search Console is the direct first-party source. Third-party platforms answer competitor and market questions that Search Console cannot. They are not interchangeable measurements.
Is Keyword Planner search volume exact?
No. It is aggregated planning evidence under selected time, location and network settings. Treat it as conditional support, not an exact count for every wording.
Can a crawler tell me why traffic dropped?
A crawler can identify observable technical changes or defects. It cannot prove that they caused the decline without outcome data and a causal evaluation design.
Can one SERP prove a competitor dominates the market?
No. It proves what appeared in one recorded result environment. Durable dominance requires repeated observations and additional market evidence.
Are third-party estimates useful?
Yes. They are useful for discovery, comparison and directional analysis when they remain explicitly labelled as estimates.
Open methodology, source ledger and limitations
Method, sources and limitations
The benchmark evaluates eight claims across five source classes using official platform documentation, provider methodology pages, data-quality guidance and four documented SERPIFAI decision corrections.
It does not compare every paid tier, test every provider, or claim that all tools inside one source class behave identically. Revalidate the matrix when provider methodology, reporting coverage, platform documentation or the decision context changes.
Primary and methodological sources
- Google Search Console Performance report
- Google: About Search Console data
- Google Search Analytics API query method
- Google Ads Keyword Planner documentation
- Google Search crawling and indexing overview
- Screaming Frog configuration
- Semrush Traffic & Market methodology
- Ahrefs organic traffic calculation
- UK Government Data Quality Framework
- W3C PROV Overview
Research boundary: these sources document provider behaviour and data-quality principles. They do not independently validate the proprietary SERPIFAI classification system.
Do not ask which SEO source is universally best.
Ask which source can support this exact claim, under these conditions, for this decision—and what wording the evidence permits.