Direct answer
A competitor score is a compression layer—not evidence, probability or permission to act
A composite score can help compare options, but it creates false certainty when the model hides its purpose, inputs, evidence states, normalization, weights, missing-data rules, vetoes, floors or sensitivity. A defensible score must remain decomposable: every total must lead back to the exact records, assumptions and decision rules that produced it.
Start with purpose
What decision is the score supposed to support?
The same arithmetic can be acceptable for one purpose and dangerously misleading for another. A descriptive index can summarise a pattern. A screening score can reduce a long list. A prioritisation model can help sequence work. None of those automatically becomes a calibrated probability, a funding rule or a final recommendation.
Describe
Compress several dimensions into a readable summary. The score must not be presented as predictive.
Rank
Order comparable options. Rank stability and tied or near-tied options must be disclosed.
Screen
Remove clearly unsuitable options. Thresholds and exclusion rules must be explicit.
Prioritise
Sequence attention or investment. Weight rationale and resource constraints matter.
Trigger review
Surface models that need human inspection. The score is an alert, not a verdict.
Enforce a gate
Block action when a non-compensating condition fails. The gate operates before aggregation.
A total never overrides an unresolved critical input, a failed gate or a ranking that flips under reasonable assumptions.
Core model
Five ways a competitor score creates false certainty
Hidden weights
The model values some criteria more without explaining why, who approved the choice or whether the weights change by decision.
Mixed evidence states
Direct observations, inferences, unresolved gaps and modelled estimates are converted into the same numeric scale as though they were equivalent records.
Scale distortion
Different ranges, directions or transformations make one criterion dominate for mathematical rather than decision-relevant reasons.
Compensation
Several strong dimensions average away a requirement that should have operated as a veto or minimum floor.
False precision
Decimals imply a level of measurement accuracy, stability and certainty that the underlying evidence cannot support.
Input integrity
Unknown, zero, midpoint and not applicable are different conditions
Before aggregation, every input needs an explicit treatment. The model must not silently replace missing knowledge with a convenient number.
| Input condition | Permitted treatment | Unsafe shortcut | Decision consequence |
|---|---|---|---|
| Observed | Use the bounded value with source, scope, time and method. | Treating the observation as automatically complete or decision-ready. | May enter the model if scale and relevance are valid. |
| Inferred | Disclose the observed inputs, reasoning bridge, assumptions and review test. | Scoring the inference as though it were directly measured. | Usually requires a sensitivity scenario. |
| Unresolved | Record the exact missing or unreconciled condition. | Substituting zero, midpoint or an optimistic estimate. | Block, narrow or produce a range depending on materiality. |
| Unknown | Keep unknown separate and state the chosen exclusion or imputation rule. | Calling unknown “neutral.” | May make the score not decision-ready. |
| Zero | Use only when zero is a measured value on the defined scale. | Using zero to represent no research. | Can materially reduce the score and ranking. |
| Midpoint | Use only when the midpoint is supported by the measurement model. | Using it as a default for uncertainty. | Creates artificial centrality and false stability. |
| Not applicable | Exclude from the denominator with a recorded rationale. | Scoring it as zero or a pass. | Changes comparability and must be disclosed. |
| Stale or out of scope | Remove, refresh or narrow the model. | Keeping the value because it is already numeric. | May invalidate the affected criterion or model. |
| Contradictory | Preserve and reconcile the conflicting records first. | Averaging the disagreement into one number. | Route to How to Handle Contradictory Competitor Evidence. |
Weight governance
Visible weights are not enough
A transparent but arbitrary weight is still arbitrary. Each weight should have a decision-specific rationale, permissible range, owner, reviewer and effective version. Weight totals must be checked after exclusions and “not applicable” treatments.
Rationale
Explain why the criterion should exert this much influence on the decision.
Normalization
Make unlike scales comparable without allowing outliers or direction errors to dominate.
Version
Record the model version and effective date so rankings can be reproduced later.
Range
Define a reasonable alternative weight range for sensitivity analysis.
Owner and reviewer
Separate model construction from accountable approval when the decision is consequential.
Total-weight check
Confirm that active weights sum to the intended denominator after exclusions.
Non-compensating controls
Vetoes and floors must operate before the total is interpreted
A weighted average assumes that strength in one dimension may compensate for weakness in another. That assumption is not always acceptable.
A failed condition blocks the affected option regardless of the total. Example: the team cannot publish credibly in a regulated topic.
A criterion must reach a minimum level before the score can be used. Example: evidence quality must be at least “reviewable.”
A weakness limits the maximum interpretation. Example: unresolved source provenance caps the result at “review required.”
Trade-offs are allowed only when the model explicitly permits them and the consequences are reversible.
Order of operations: validate inputs → apply vetoes and floors → calculate the weighted total → test sensitivity → determine the permitted action.
Robustness
A score is not stable until reasonable assumptions are changed
Sensitivity analysis tests whether the ranking or recommendation survives plausible changes to weights, normalization, missing-data treatment, inclusion rules and uncertain inputs. The goal is not to generate every possible model. It is to identify which assumptions are decision-driving.
Stable
The leading option remains ahead across reasonable scenarios and no critical gate fails.
Close
Totals remain similar. Treat the result as a tied decision requiring qualitative review.
Ranking flip
A reasonable weight or input change produces a different winner. The score requires sensitivity review.
Unknown-driven
The recommendation depends on how missing data were treated. Obtain evidence or disclose a range.
Commercial example
Two competitors can receive similar totals while presenting different decision risks
Consider a fictional B2B SaaS decision: which competitor should receive the next detailed response brief?
| Criterion | Weight | Competitor North | Competitor South | Decision treatment |
|---|---|---|---|---|
| Buyer overlap | 25% | 5/5 — observed | 4/5 — observed | Compensating |
| Product substitutability | 20% | 4/5 — inferred | 5/5 — observed | Compensating with disclosure |
| Search-market availability | 15% | 5/5 — observed | 3/5 — observed | Compensating |
| Evidence quality | 15% | 3/5 — unresolved provenance | 5/5 — observed | Minimum floor: 4/5 |
| Implementation burden | 10% | 4/5 — inferred | 3/5 — inferred | Compensating |
| Regulatory feasibility | 15% | 2/5 — gate failed | 5/5 — observed | Veto |
Canonical outcomes
Separate the score from the model’s decision state
| Decision state | Meaning | Permitted next action |
|---|---|---|
| Decomposable and bounded | Purpose, inputs, evidence states, weights, gates and sensitivity are explicit. | Use the score as one decision aid with the recorded limitations. |
| Sensitivity review required | The ranking changes, weights lack rationale, totals are close or reasonable scenarios have not been tested. | Run the recorded scenarios before acting. |
| Critical gate missing | A non-compensating requirement is absent, undefined or allowed to be averaged away. | Define or resolve the gate before interpreting the total. |
| Missing-data treatment unsafe | Unknown, unresolved or not-applicable values were converted into unsupported numbers. | Correct the input treatment or report a bounded range. |
| Score not decision-ready | The purpose, inputs, evidence ownership or contradiction status is not sufficiently visible. | Decompose and reconstruct the model before use. |
Why a Precise Score Can Mislead
How to read it: select a checkpoint to isolate that part of the method. The canvas shows the relationship; the inspector explains what the checkpoint changes in the decision.
No-JavaScript method
Manual score-integrity review
- Name the decision and model purpose. State what the score may and may not determine.
- List every active input. Preserve raw value, scale, direction, source, scope, time and evidence state.
- Record transformations. Explain normalization, exclusions and missing-data treatment.
- Govern weights. Record rationale, owner, reviewer, version and total-weight check.
- Apply gates first. Identify vetoes, floors, caps and permitted compensation.
- Test reasonable scenarios. Change decisive weights and uncertain inputs; record ranking flips.
- Issue a decision state. Keep the numeric total, limitations, permitted action and reopen condition separate.
60–90-second first pass
Quick Score Integrity Check
Use this for one prepared score. It checks purpose, inputs, active weights, critical gates, contradictions and sensitivity; it does not replace the detailed audit.
First-pass result
- Reason code
- Next action
Provisional first-pass classification. Complete the detailed workflow before a consequential handoff.
Detailed professional workflow
Score Decomposition Audit
Usually 5–15 minutes with prepared evidence. The workflow preserves one audit-ready model record and its decision boundary.
Audit-ready record
- Reason code
- Permitted interpretation
- Prohibited interpretation
- Next action
- Approval status
- Reopen condition
Repeated-use tools
Workbook and portable handoff
Score Decomposition Register
Use the workbook for repeated models, component-level normalization, weight control, vetoes, sensitivity scenarios and auditable decision states.
Download workbookScore Integrity Handoff
Use the tagged, searchable PDF to carry one model’s purpose, decisive inputs, gates, sensitivity and permitted interpretation into review or print.
Download PDFEvidence-system handoffs
Score integrity consumes upstream evidence controls—it does not replace them
| Question | Owning method | What this score audit consumes |
|---|---|---|
| Can the evidence support the claim? | What Counts as Reliable Evidence in Competitor Analysis? | Claim eligibility and source fitness. |
| How strongly may an eligible claim be expressed? | How to Assign Confidence Without Fake Precision | Bounded confidence language—not an automatic numeric weight. |
| How should conflicting records be handled? | How to Handle Contradictory Competitor Evidence | Resolved or still-open contradiction status. |
| Is the claim directly recorded, reasoned or unresolved? | Observed, Inferred and Unresolved Evidence | The evidence state attached to each score input. |
| May dimensions be aggregated and interpreted? | This score-integrity method | Purpose, normalization, weights, missing-data rules, gates and sensitivity. |
Frequently asked questions
Questions about competitor scoring models
Should competitor analysis avoid scores completely?
No. Scores can be useful comparison aids when the model is transparent, decomposable and bounded. The problem is treating the total as evidence or as an automatic decision.
Are weighted scoring models objective?
They are structured, not automatically objective. Criteria, scales, weights, missing-data rules and aggregation choices all embed judgement.
What should happen when one input is unknown?
Keep unknown separate from zero, midpoint and not applicable. Record whether the model blocks, excludes with a revised denominator, produces a range or requests more evidence.
When should a critical failure override the total?
When the decision contract defines a veto or minimum floor. The rule should be established before scores are reviewed, not added after an inconvenient result appears.
What is a sensitivity review?
It tests whether reasonable changes to weights, normalization, inclusion rules or uncertain inputs change the ranking or recommendation.
Can a competitor score be treated as a probability?
Not unless the model was explicitly built, calibrated and validated as a predictive probability model. A weighted total is not a probability by default.
Sources and method boundary
Research foundations
This proprietary SERPIFAI score-integrity method adapts transferable principles from composite-indicator construction, multi-criteria decision analysis, uncertainty treatment and sensitivity analysis. These sources support the general controls; they do not validate the deterministic SERPIFAI rules or outputs.
- OECD and European Commission Joint Research Centre — Handbook on Constructing Composite Indicators
- UK Government Analysis Function — An Introductory Guide to Multi-Criteria Decision Analysis
- European Commission Competence Centre on Composite Indicators — Imputation of missing data
- European Commission Competence Centre on Composite Indicators — Normalisation
- European Commission Competence Centre on Composite Indicators — Weighting
- European Commission Competence Centre on Composite Indicators — Sensitivity analysis
- NIST — Incorporating Attribute Value Uncertainty into Decision Analysis
Decision rule
Decompose the number before trusting it
A useful competitor score makes trade-offs visible and reviewable. It does not erase evidence states, convert unknown into neutral, allow a fatal gap to be averaged away or hide a ranking that depends on one fragile assumption. Preserve the total—but keep the model, limitations and permitted action beside it.