Direct answer
Confidence is a bounded evidence judgment, not an invented percentage
Confidence is not a percentage you invent. It is a bounded statement about how well the available evidence supports one assessment and what uncertainty could still change it. Assign confidence only after the claim has passed evidence-use review. Then use High, Moderate or Low according to support, independence, assumptions, context and sensitivity. When the claim is refused, unresolved, contradicted, stale for the decision or missing required evidence, confidence is not assignable.
A complete statement contains the exact claim, decision scope, confidence expression, rationale, principal uncertainty, assumptions, strongest permitted wording, prohibited stronger wording, owner, reviewer and reopen condition.
Start with the distinctions
Confidence is not probability, certainty, risk or consequence
These terms answer different questions. Mixing them creates false precision and unsafe handoffs.
| Term | Question it answers | Treatment in this guide |
|---|---|---|
| Confidence | How well do the available evidence and visible assumptions support this exact assessment? | Expressed qualitatively as High, Moderate or Low after eligibility passes. |
| Likelihood or probability | How likely is an event to happen? | Not calculated. Do not translate confidence into a probability band. |
| Certainty | Is there no meaningful doubt? | Not promised. High confidence remains bounded and reopenable. |
| Consequence | What happens if the assessment is wrong? | Changes reviewer, approval and permitted action—not the confidence label mechanically. |
| Reversibility | How hard is it to undo the decision? | Changes governance and approval requirements. |
| Evidence-use permission | May this evidence support this claim at all? | Owned upstream by What Counts as Reliable Evidence in Competitor Analysis?; it dominates confidence. |
Intelligence guidance explicitly separates analytical confidence from likelihood, while scientific uncertainty guidance shows that confidence sometimes cannot be assigned when evidence is insufficient or ambiguous. This guide adapts those principles to competitor analysis without importing probability bands or a scoring system.123
The existing visual remains part of this guide’s image system. Read it as a qualitative synthesis of evidence quality, consistency, coverage and context fit. The executable protocol below adds the fail-closed eligibility gate: unsupported, unresolved or missing evidence cannot be converted into Low confidence.
Use the framework as an explanation—not as a scoring model
This proprietary model formalises the decision logic without pretending the analysis is mechanically certain. It supports reviewable consulting judgement, not algorithm speculation.
The gate that prevents misuse
Ask whether confidence may be assigned before choosing a level
The sequence is fail-closed. A claim receives no qualitative level when an earlier gate fails. Low confidence is reserved for an eligible but fragile assessment; it is not a softer label for a claim that should have been refused.
Stop on upstream refusal or review
Refuse claim stops at P02. Review required stops at P03. This guide does not reinterpret the upstream reason.
Stop on unresolved status or claim mismatch
Unresolved, No, Unknown and Partial direct fit cannot receive a confidence level.
Stop when required evidence is missing
Any submitted value other than None for Missing required evidence stops confidence at P06 with CONF-MISSING-OR-UNCHECKED. Recoverable evidence is still missing until recovered and reviewed.
Stop on contradiction, stale context or semantic conflict
Resolve contradictions through How to Handle Contradictory Competitor Evidence, preserve claim status through Observed, Inferred and Unresolved Evidence, and correct invalid relationships before classification.
“Low confidence” cannot make an unsupported, refused, unresolved, materially contradicted or missing-evidence claim acceptable. The correct output is Confidence not assignable plus the required upstream action.
What changes an eligible result
Use visible factors, not source counts or hidden weights
Confidence depends on the relationship between the claim and its evidence—not on the number of browser tabs. The protocol examines:
- Direct claim fit: does the evidence support the exact proposition, scope and time boundary?
- Independent origins: do corroborating lines arise separately, or do they repeat one underlying source?
- Assumption burden: are assumptions absent, limited and visible, material but bounded, or material and unresolved?
- Coherence and contradiction: is material counter-evidence resolved or still active?
- Context and freshness: is the evidence fit for the current decision, or deliberately bounded to a historical claim?
- Sensitivity: could plausible new evidence change only wording, or readily change the whole assessment?
Official guidance across intelligence, public health, evidence synthesis and statistics consistently emphasizes visible assumptions, corroboration, gaps, relevance and prominent limitations. This guide adopts that transparency while rejecting numerical confidence theatre.145
The qualitative contract
Use three confidence levels and one non-assignable eligibility result
High confidence
Strong eligible support
Direct, coherent support; established multiple independent origins; no unresolved material contradiction; none or limited visible assumptions; current context fit; low sensitivity to plausible new evidence.
Allowed: “The available evidence strongly supports…”
Not allowed: certain, proven, guaranteed or a percentage.
Moderate confidence
Supported with meaningful uncertainty
One effective line, shared origins, deliberately historical scope, named limitations, limited/weak evidence with independent origins, or sensitivity that could change scope or wording.
Allowed: “The available evidence supports…”
Not allowed: definitive, conclusive or established fact.
Low confidence
Eligible but fragile
Single-origin fragility, material bounded assumptions, weak historical support, support not established or sensitivity likely to change the assessment.
Allowed: “Limited evidence weakly supports the bounded assessment…”
Not allowed: shows, confirms or demonstrates as established.
Confidence not assignable
Eligibility failed
The claim is refused, under review, unresolved, mismatched, missing evidence, materially contradicted, stale for the use, semantically inconsistent or technically invalid.
Required: name the reason and route the record to the correct evidence workflow.
Not a level: do not call this “very low” or “zero confidence.”
Portable output
A confidence label without its reasons is not a usable handoff
Write the statement so another reviewer can reconstruct the boundary without reopening every source immediately.
- Exact claim: one bounded proposition.
- Decision use and scope: who will use it, for what decision, in which market and time window.
- Eligibility: why the claim may—or may not—receive confidence.
- Confidence expression: High, Moderate, Low or not assignable.
- Rationale: decisive support, independent origins, visible assumptions and material limitations.
- Principal uncertainty: the most important factor that could change support, scope or wording.
- Wording boundary: strongest allowed statement and stronger prohibited statement.
- Governance: consequence, reversibility, owner, reviewer, permitted action and reopen condition.
Moderate example
“The available evidence supports the assessment that the competitor publishes a public comparison for the named segment, with Moderate confidence for the current-quarter brief. A second genuinely independent line could change the scope or wording.”
This says what is supported, names the level, preserves the scope and exposes what could change it. It does not pretend that “Moderate” means a numerical probability.
Keep governance separate
A high-stakes decision does not automatically reduce confidence
Suppose the evidence strongly supports a narrow claim. Changing the intended use from a descriptive brief to a hard-to-reverse investment does not make the evidence weaker. It changes the review and approval burden.
Confidence describes
- evidential support;
- uncertainty and assumptions;
- stability within the recorded scope.
Consequence and reversibility control
- who must review;
- whether the reviewer must be independent;
- what action is permitted;
- whether a reversible pilot is preferable.
This separation is consistent with risk-informed decision frameworks that treat evidence, uncertainty and consequence as distinct decision dimensions.8
Confidence can change
Every completed statement needs a reopen condition
A reopen condition names the evidence, context, assumption, claim boundary or decision-use change that requires reassessment. It is not a generic “review periodically” note.
Evidence trigger
Reopen when a second independent source confirms or contradicts the product comparison.
Context trigger
Reopen if the assessment expands from the named segment to all markets.
Assumption trigger
Reopen when first-party pricing data resolves the material bounded assumption.
Material time trigger
Use a review date only when the passage of time itself is material. There is no universal evidence expiry rule.
When the evidence-use review in What Counts as Reliable Evidence in Competitor Analysis? returns No material change, carry forward confidence only if the exact claim and decision use are unchanged. A changed claim or use requires a fresh assessment.
Memorable explanation
The confidence flow keeps eligibility, expression and governance separate
Evidence-reviewed claim
Start with the exact evidence-use review outcome and bounded claim.
Confidence eligibility
P00–P09 stop invalid, incomplete or non-assignable states.
Confidence expression
P10 Low, P11 Moderate or P12 High—without scores.
Permitted wording
State reasons, uncertainty, assumptions and prohibited stronger wording.
Review and reopen
H00–H06 controls ownership, review, dates and PDF eligibility.
Non-assignable branch: return to What Counts as Reliable Evidence in Competitor Analysis?, How to Handle Contradictory Competitor Evidence, Observed, Inferred and Unresolved Evidence, or When Competitor Evidence Becomes Too Old to Use. Do not choose Low.
Governance overlay: consequence and reversibility change review and action, not the Stage 1 label.
Evidence → Wording Confidence Ladder
How to read it: select a checkpoint to isolate that part of the method. The canvas shows the relationship; the inspector explains what the checkpoint changes in the decision.
No-JavaScript method
Apply the same method manually
Phase A — Stage 1
- Reject invalid exact select values (P00).
- Complete every active Stage 1 field (P01).
- Stop refused, review-required, unresolved, mismatched, missing-evidence, contradictory, stale or semantically conflicting records (P02–P09).
- For eligible records, apply the first matching Low predicate (P10), then Moderate predicate (P11), otherwise the exact strong-support conjunction (P12).
- Generate the rationale, principal uncertainty and wording boundaries.
- Add consequence and reversibility requirements without changing confidence.
Phase B — Stage 2
- Name the assessment owner.
- Add an accountable reviewer when consequence or reversibility requires one.
- Use a distinct reviewer for consequential or hard-to-reverse decisions.
- Name the reopen condition.
- State whether it contains a material time trigger.
- Add a real date only when the answer is Yes.
- Permit the completed PDF only at H06.
Exact Stage 1 order
P00 invalid value → P01 incomplete → P02 refusal → P03 review → P04 unresolved → P05 fit failure → P06 missing or unchecked evidence → P07 contradiction → P08 stale/context/assumption failure → P09 remaining semantic conflict → P10 Low → P11 Moderate → P12 High.
Exact Stage 2 order
H00 invalid handoff value → H01 generic missing field → H02 reviewer missing → H03 reviewer not independent → H04 time trigger without date → H05 date without time trigger → H06 complete.
Deep Work
Worked example: one claim changes state as the evidence boundary improves
Fictional claim: “The competitor publishes a public product comparison for the named B2B segment.” Decision use: a current-quarter competitor brief.
| Submission | Controlling state | Result | Allowed action |
|---|---|---|---|
| Overbroad initial claim | Direct claim fit is Partial | P05 · Confidence not assignable | Narrow the proposition, obtain approval through What Counts as Reliable Evidence in Competitor Analysis?, and resubmit the new exact claim. |
| Narrow claim; evidence still recoverable but missing | Missing required evidence is Recoverable missing evidence | P06 · Confidence not assignable · CONF-MISSING-OR-UNCHECKED | Recover the evidence and complete evidence-use review. Do not call it Low. |
| Eligible narrow claim; one fragile origin | Single-origin only; sensitivity could change wording | P10 · Low confidence | Use provisional wording; collect independent corroboration. |
| One established independent line | Fit, supported, one independent line, meaningful uncertainty | P11 · Moderate confidence | Use within the stated brief; name the missing second line. |
| Direct coherent support with multiple independent origins | Fit, supported, current, low sensitivity, limited visible assumptions | P12 · High confidence | Proceed within scope; retain uncertainty and reopen trigger. Do not claim certainty. |
Consequence changes the handoff, not the evidence result
If the same High-confidence claim is used for a consequential, hard-to-reverse decision, Stage 1 stays High. Stage 2 requires an accountable reviewer who is distinct from the owner, formal approval and a complete reopen path. Missing that reviewer produces H02 or H03 and prohibits the completed PDF while preserving the High-confidence assessment.
No-material-change reassessment
An unchanged exact claim and unchanged decision use may retain a prior eligible result. A changed exact claim executes CONF-SC-NMC-CLAIM-CHANGED; a changed use executes CONF-SC-NMC-USE-CHANGED. Both require a fresh assessment.
Fast Action
Build a calibrated confidence statement
This tool starts after the claim has been reviewed under What Counts as Reliable Evidence in Competitor Analysis?. It applies the same qualitative contract used by the manual method and workbook. It does not calculate probability, score evidence or rescue an ineligible claim.
Method: SERPIFAI Confidence Statement Method v1.0.1
Current result
Eligibility:
Rationale
Principal uncertainty
Material assumptions
Strongest wording allowed
Stronger wording prohibited
Next evidence or correction
Review and permitted action
Overlay:
Handoff rule: · Reason:
Method:
Technical record
Portable Handoff
Use the workbook for repeated records and the PDF for one completed handoff
XLSX · repeated use
Confidence Register
Capture multiple assessments with exact dropdowns, case-sensitive formula gates, P00–P12 and H00–H06 outputs, a review queue and completed-record summaries.
- Invalid or pasted unsupported values fail closed.
- Incomplete, non-assignable and draft records are excluded from completed summaries.
- No macros, scores, external queries or universal expiry formula.
Tagged PDF · one handoff
Completed confidence handoff
See a two-page, selectable, tagged fictional example showing the assessment, wording boundary, ownership and reopen condition that should survive a decision handoff.
- Displays an H06 completed eligible result only.
- Never recalculates confidence.
- Omits a review date when no material time trigger exists.
Failure prevention
Common mistakes that create false confidence
Inventing 73% confidence
A precise-looking number is not justified without a documented, validated and calibrated probability model.
Counting repeated pages as corroboration
Five pages that repeat one press release are one origin, not five independent lines.
Using Low to smuggle through an unsupported claim
Eligibility failure means confidence not assignable, not Low.
Letting consequence change the label
High stakes require stronger review, not an automatic downgrade.
Keeping a label but losing the rationale
A portable handoff must preserve reasons, uncertainty, assumptions and wording limits.
Leaving the statement permanently open
Name the trigger that changes the assessment; do not rely on vague periodic review.
Direct answers
Questions analysts and reviewers ask
Is confidence the same as probability?
No. Confidence describes the support and uncertainty behind an assessment. Probability describes how likely an event is. This guide does not convert one into the other.
When is High confidence justified?
Only after eligibility passes and the exact strong-support conjunction is met: fit evidence, supported or clearly inferred status, direct claim fit, no unresolved contradiction, no missing required evidence, established multiple independent origins, none or limited visible assumptions, current context fit and low sensitivity to plausible new evidence.
When should Moderate confidence be used?
Use Moderate for an eligible assessment with meaningful uncertainty, such as one effective line, shared origins, a named limitation, deliberately historical scope, limited/weak but independently originated evidence, or sensitivity that could change scope or wording.
When should Low confidence be used?
Use Low only for an eligible but fragile assessment: single-origin fragility, material bounded assumptions, weak historical support, support not established or high sensitivity to new evidence.
Can an unsupported claim be labelled Low confidence?
No. Unsupported, refused, unresolved, materially contradicted, stale-for-use or missing-evidence claims receive Confidence not assignable and return to the relevant evidence workflow.
Does a high-stakes decision reduce confidence?
No. Consequence and reversibility change review, approval and permitted action. They do not mechanically alter evidential confidence.
What changes an existing confidence assessment?
New evidence, changed claim or decision use, changed context, resolved or new contradiction, changed assumptions, changed independent origins or a named reopen trigger can require reassessment.
Who should own and review the statement?
A named assessment owner maintains the record. Material or difficult-to-reverse uses require an accountable reviewer; consequential or hard-to-reverse uses require a reviewer distinct from the owner.
Research basis and limits
The method adapts transparent uncertainty principles without claiming external validation
The protocol draws on official guidance from intelligence analysis, scientific assessment, public health, evidence synthesis, measurement reporting, government analysis and risk-informed decision-making. Those sources support the distinctions, transparency requirements and need to carry uncertainty with the result. They do not validate SERPIFAI’s exact categorical contract or prove that it improves forecast accuracy.
- Office of the Director of National Intelligence, Intelligence Community Directive 203: Analytic Standards — confidence/likelihood distinction, assumptions, sources, uncertainty and indicators.
- UK Government, Explaining Uncertainty in UK Intelligence Assessment — confidence concerns the soundness and stability of an assessment’s foundations.
- IPCC Guidance Note on Consistent Treatment of Uncertainties — separates confidence and likelihood and permits non-assignment.
- CDC Center for Forecasting and Outbreak Analytics, How CFA Sets Confidence Levels — information quality, corroboration, gaps and assumptions.
- Cochrane / GRADE-CERQual overview — visible methodological limitations, relevance, coherence and adequacy.
- NIST Technical Note 1297 — report the basis and method of an uncertainty statement.
- UK Government Analysis Function, Communicating quality, uncertainty and change — make limitations prominent and connected to use.
- U.S. GAO, A Framework for Risk-Informed Decision-Making — distinguish uncertainty, consequence and decision governance.
SERPIFAI Confidence Statement Method v1.0.1 is a qualitative evidence-and-uncertainty protocol. It is not a probability model, calibrated accuracy measure, scoring system, assurance opinion or guarantee. High confidence remains bounded to the exact claim, decision use, evidence context, assumptions and reopen condition.