Decision-making & prioritisation
Decision Matrix (Weighted Scoring)
A weighted-criteria table for choosing between options: list the candidates, agree and weight the criteria, score each option against each criterion, and read the weighted totals. Its real product is a decision whose reasoning can be inspected and challenged.
Also known as Weighted decision matrix, Grid analysis, Pugh matrix (datum variant), Multi-criteria scoring. First set out by No single originator; formalised separately by Kepner-Tregoe and Stuart Pugh in 1965.
Where this is contested
No single inventor exists. Weighted scoring descends from decision analysis and multi-attribute utility theory; Kepner and Tregoe formalised a musts-and-wants weighted method for managers in 1965, and Stuart Pugh's 1981 concept-selection matrix, refined in Total Design (1990) as controlled convergence, established the engineering datum variant. Named versions are branches of a common practice.
- Format
- Scoring model
- Level
- Business unit · Product · Team
- Best for
- Evaluate options · Prioritise
- Decision stage
- Explore options · Decide
- Difficulty
- Intermediate
- Time to apply
- Two to four hours for a group decision once evidence is gathered; allow a full day where demonstrations and reference checks feed the scores.
Plate · The model
The components
List the options
Assemble the genuine candidates, including the incumbent or do-nothing baseline. The matrix can only rank what is on the list, so the list is where a decision is most easily rigged.
Signals of strength
Every option is real, available and affordable in principle · The incumbent or do-nothing case is present as a baseline · No credible candidate has been excluded to protect a favourite
Choose the criteria
Agree the handful of factors the decision should actually turn on, screened for overlap and separated from absolute constraints, which are pass or fail rather than scoreable.
Signals of strength
Four to eight criteria, each traceable to a stated objective · Must-pass constraints handled separately from preferences · No two criteria rewarding the same underlying thing
Weight the criteria
Distribute a fixed total across the criteria to express relative importance, before scoring begins. Weights are where values enter the analysis, and where most of the honest disagreement lives.
Signals of strength
Weights fixed before any option is scored · Weights reflect realistic differences between options, and not abstract importance · Dissent about weights recorded rather than averaged into silence
Score each option
Rate every option against every criterion on a consistent scale, anchored in evidence. Scoring criterion by criterion keeps the comparison honest; scoring option by option invites halo effects.
Signals of strength
A defined scale applied identically to all options · Each score attached to cited evidence · Scored across options one criterion at a time
Read the weighted totals
Multiply scores by weights, sum, and rank, then interrogate the ranking rather than obeying it. Sensitivity to weight changes, near-ties and surprises all carry information the single total hides.
Signals of strength
Close totals treated as ties, and settled on other grounds · Ranking tested against plausible alternative weights · Any override of the result recorded with reasons
When it earns its keep
- A significant choice between named alternatives, a supplier, a system, a site, a candidate design, needs to be made by a group rather than one head.
- The decision keeps circling because different people are silently optimising for different things, and the criteria have never been put on the table.
- The choice must survive later scrutiny, from a board, an auditor or a procurement review, and the reasoning has to be reconstructible.
- A gut preference already exists and you want to test it honestly against the factors everyone claims to care about.
And when it doesn't
- One criterion genuinely dominates. If nothing matters beside price, or safety is absolute, a weighted blend of everything else only obscures that.
- The options are not yet worth comparing. Weighted scoring evaluates a shortlist; generating better options is design work, and Pugh's own method exists partly to stop premature convergence on a weak set.
- Deep uncertainty, rather than competing preferences, is the problem. Scores imply you can assess outcomes; where you cannot, scenario planning or decision trees fit better.
- The numbers would be theatre. If the decision is already made and the matrix exists to justify it, the tool's authority is being borrowed, and everyone in the room usually knows.
How to run it
Before starting, gather the inputs the analysis depends on:
- A shortlist of real, available options, including the incumbent or do-nothing baseline where one exists.
- The people who own the decision and the people who will live with it, both represented when criteria and weights are set.
- Evidence about each option: demonstrations, references, trials, cost breakdowns, enough that scores can be anchored rather than invented.
- A defined scoring scale and a rule for what weights sum to, agreed before scoring starts.
- A list of absolute constraints, screened out separately, so must-pass requirements are never averaged away by good scores elsewhere.
- 1
Frame the decision and screen the constraints
State what is being decided and by when, then separate absolute requirements from preferences. Kepner and Tregoe's distinction between musts and wants is the working rule: an option that fails a must is out, and no score can buy it back in. Weighted totals are for choosing among survivors.
- 2
List the options
Name the genuine candidates, including the incumbent or do-nothing option as a baseline. A matrix with a rigged shortlist produces a rigorous-looking choice among the wrong things, which is worse than no matrix at all.
- 3
Choose the criteria
Derive four to eight criteria from what the decision must achieve, and check them for overlap. Correlated criteria such as 'ease of use' and 'training burden' quietly double-count the same concern, which is a hidden weight no one agreed to.
- 4
Weight the criteria
Distribute a fixed total, commonly 100 points, across the criteria before any option is scored. Weighting after scoring is the classic route to a reverse-engineered answer. Weights should reflect how much the realistic difference between options on that criterion matters, and the argument this step provokes is usually the most valuable hour of the whole exercise.
- 5
Score each option
Score criterion by criterion across all options, rather than option by option, using the agreed scale and citing evidence for each score. Where the numbers start to feel arbitrary, Pugh's datum method is the honest fallback: pick a reference option and mark each rival better, same or worse against it, which preserves the comparison while dropping the spurious precision.
- 6
Read the totals and test them
Multiply, sum and rank, then attack the result. Vary the contested weights and see whether the ranking survives; a winner that flips with a five-point weight change is a coin toss wearing a spreadsheet. The matrix advises and people decide, and overriding it is legitimate provided the reason is written down next to it.
Reading the result
A ranked comparison of options with explicit criteria, weights, evidence-anchored scores and weighted totals, plus a sensitivity check showing how robust the winner is to the judgements that produced it.
- The total is an argument, and never a verdict. Read the winning margin against the roughness of the inputs: a 5 per cent lead on judgement-based scores is a tie.
- Look at where the winner wins. An option that tops the table on the strength of one heavily weighted criterion is a different proposition from one that scores solidly everywhere.
- Treat instability as a finding. If modest weight changes reorder the ranking, the group disagrees about values rather than facts, and that conversation is the actual decision.
A worked example
A wholesale plumbing supplies firm chooses a warehouse ERP
A family-owned plumbing and heating merchant in the East Midlands, four trade counters and a central warehouse, is replacing a twenty-year-old stock system. Three candidates survive the constraint screen (proven UK VAT handling, supplier viability, go-live inside twelve months): the incumbent vendor's cloud upgrade, a mid-market distribution ERP, and a best-of-breed warehouse system bolted to the existing accounts package. The board runs a weighted matrix across a day of demonstrations and reference calls.
- List the options
- The incumbent upgrade is kept in deliberately as the baseline, despite the warehouse manager's contempt for it, because it is the cheapest and least disruptive path and the others must beat it on the merits. A fourth option, building on spreadsheets and grit, is recorded and excluded at the constraint screen.
- Choose the criteria
- Five criteria: warehouse and stock functionality, trade counter speed, migration and integration risk, five-year total cost, and supplier covenant. An early sixth, 'ease of use', is struck out after someone notices it double-counts both counter speed and training cost, which is already inside five-year cost.
- Weight the criteria
- The managing director opens wanting cost at 40 of 100. The operations director argues the business loses more to stock errors than it would ever save on licences, and the reference calls back her up. Settled weights: warehouse functionality 30, counter speed 20, migration risk 20, five-year cost 20, supplier covenant 10. The debate takes two hours and is, everyone later agrees, the point at which the decision was really made.
- Score each option
- Scored 1 to 5, criterion by criterion, with evidence noted. The incumbent upgrade scores 5 on migration risk and 2 on warehouse functionality, and its counter module demo stumbles on trade credit accounts. The mid-market ERP scores 4s almost across the board. The best-of-breed pairing scores 5 on warehouse functionality and 2 on migration risk after a reference customer describes a painful accounts integration.
- Read the weighted totals
- Weighted totals: mid-market ERP 3.9, best-of-breed pairing 3.6, incumbent upgrade 3.1. Sensitivity testing shows the ranking only flips if cost is pushed above 35 points or migration risk above 30, and nobody will argue for either with a straight face. The consistent all-rounder beats the specialist because the specialist's strength sits on top of an integration risk the firm cannot absorb.
The read. The board chooses the mid-market ERP. The matrix's real contribution was diagnostic: it showed that the disagreement in the room was about weights, values in numerical clothing, rather than about facts, and it forced the cost-versus-capability argument to happen before contract signature instead of after go-live. The scoring table goes into the board minutes, which is exactly where a decision this size should be auditable.
Pitfalls
- Reverse-engineering the weights. Deciding first and tuning weights until the matrix agrees is the tool's commonest abuse, and visible to anyone who asks when the weights were fixed.
- Double-counting through correlated criteria. Two criteria measuring the same underlying concern silently double its weight, and no one agreed to that.
- Averaging away a fatal flaw. A must-pass failure buried under good scores elsewhere is how organisations buy systems that cannot legally be used; screen constraints before scoring.
- Worshipping the decimal. A 3.9 beating a 3.6 on judgement-based inputs is a lean, and never a proof, and presenting it as proof discredits the method.
- Skipping the sensitivity test. An unstable ranking presented as a stable answer converts honest disagreement about values into a fake consensus about arithmetic.
What the critics say
Weighted-sum rankings are vulnerable to rank reversal: adding or removing an option, or changing how scores are normalised, can flip the relative order of two unrelated candidates. First demonstrated against Saaty's analytic hierarchy process by Belton and Gear, the problem generalises to naive scoring matrices, and it means a ranking can hinge on choices that feel like housekeeping.
Belton, V. and Gear, T. (1983) 'On a short-coming of Saaty's method of analytic hierarchies', Omega, 11(3), pp. 228-230.
Weight elicitation is the method's soft underbelly. The MCDA literature shows that weights are only meaningful relative to the ranges of the criteria, yet practitioners routinely elicit abstract 'importance' weights detached from those ranges, producing totals that misrepresent the trade-offs people would actually accept. Belton and Stewart treat naive scoring models as decision support at best, and warn explicitly against reading their outputs as answers.
Belton, V. and Stewart, T. J. (2002) Multiple Criteria Decision Analysis: An Integrated Approach. Boston: Kluwer Academic Publishers.
The additive model assumes the criteria are preferentially independent, so that performance on one does not change how much another matters. Real decisions violate this constantly, a cheap system's cheapness matters less if its migration risk is severe, and the weighted sum has no way to say so.
Keeney, R. L. and Raiffa, H. (1976) Decisions with Multiple Objectives: Preferences and Value Tradeoffs. New York: Wiley.
Work it through
Name the options, agree and weight the criteria before you score, then score each option against each criterion. The weighted totals compute as you go. Read the winner as the start of the argument, not the end of it: a narrow margin on judgement-based scores is a tie.
| Option | Total | |||
|---|---|---|---|---|
| – | ||||
| – | ||||
| – |
Set the weights (they need not sum to 100) and score each option to see the weighted totals.
Sources and further reading
- Kepner, C. H. and Tregoe, B. B. (1965) The Rational Manager: A Systematic Approach to Problem Solving and Decision Making. New York: McGraw-Hill.
- Pugh, S. (1981) 'Concept selection: a method that works', Proceedings of the International Conference on Engineering Design (ICED), Rome, pp. 497-506.
- Pugh, S. (1990) Total Design: Integrated Methods for Successful Product Engineering. Wokingham: Addison-Wesley.
- Belton, V. and Stewart, T. J. (2002) Multiple Criteria Decision Analysis: An Integrated Approach. Boston: Kluwer Academic Publishers. ↗