The evaluation you design before the proposals arrive decides the award, not the proposals themselves. Five bids land, every one polished, every one “uniquely positioned to support Vision 2030,” and without a scoring structure the choice drifts toward whoever presented best in the demo. This post is the framework we run: a mandatory gate, four weighted questions, evidence-anchored scoring, and a panel built to neutralize the loudest voice in the room.
The stakes justify the ceremony. In McKinsey’s study of more than 5,400 IT projects, large IT projects run on average 45% over budget and deliver 56% less value than predicted (McKinsey, 2012). In our experience, most of that outcome is signed for between the proposals arriving and the award.
What a defensible software award looks like
Work backwards from the end. Imagine the losing bidder appeals, or an auditor asks why the cheapest compliant offer lost. The award survives that conversation only if a record exists: the gate decisions and their reasons, the weights as fixed before anyone saw a price, every evaluator’s scores, and the panel’s notes on each disagreement.
Design the evaluation to produce that record and the decision quality follows. We think of it as a chain with five links, four of them written before any envelope is opened.
flowchart LR A(["An RFP with explicit mandatory requirements"]) --> B(["A pass/fail gate, reasons recorded"]) B --> C(["Weights signed off before any price is seen"]) C --> D(["Independent scores against written anchors"]) D --> E(["A panel that argues only the spread"]) E --> F(["An award that survives a challenge"])
For a public entity buying through Etimad this record is not optional hygiene; it is the difference between a clean award and a protest that costs a season.
Gate proposals through mandatory requirements first
Before anything is weighted, run every proposal through a pass/fail gate on the requirements you declared mandatory in the RFP. A proposal that fails one is out, however beautifully it would have scored, and the reason goes into the record. Gate items we insist on for Saudi buyers:
- Data residency and PDPL. Where will personal data live, and who can reach production systems from outside the Kingdom? The law is enforced by SDAIA, and its cross-border transfer rules govern any support model touching personal data from abroad. A proposal that assumes foreign hosting, or an offshore support desk with production access, needs a written compliance answer before it earns a point.
- NCA cybersecurity controls. If your entity falls under the National Cybersecurity Authority’s Essential Cybersecurity Controls, the platform has to let you meet them. Ask which specific controls it supports, in writing, not which certificates the marketing mentions.
- Sector rails. A healthcare buyer should treat NPHIES integration, which runs on the HL7 FHIR standard, as a gate item for anything touching claims or eligibility. A fintech buyer should ask how the vendor fits SAMA’s sandbox and supervision expectations.
- Scope coverage and exit. Does the proposal cover every mandatory requirement, or does it “assume” a few away in the fine print? Is there a clause that returns your data in a usable format when the contract ends?
If the gate keeps producing surprises, the problem is upstream, in an RFP that never made your requirements legible; see our guide to writing a technical RFP that attracts engineers rather than salespeople.
How to weight vendor evaluation criteria
Everything that survives the gate is scored against four questions. The exact weights matter less than the moment you choose them: fixed and signed off before anyone has a favorite, because a weight chosen after reading the proposals is not a weight, it is a justification.
- Can it run here? (Technical fit, heaviest.) Does the architecture match your real constraints: scale, integrations with the systems you already run, data residency, security posture? Score the evidence, not the diagram.
- Have these exact people shipped this before? (Delivery track record, close behind.) The named engineers, not the brand. A comparable deployment in a comparable environment beats any logo.
- What does five years really cost? (Total cost of ownership.) License, implementation, integration, training, support, infrastructure, and the cost of leaving, modeled over the life of the contract.
- What happens when it goes wrong? (Dependency and exit, enough weight to bite.) Key-person risk, escalation paths, and whether a second firm could support the system if this one faltered.
How to score proposals so two evaluators agree
A score is only defensible if two people working alone land within a point of each other. That takes a written definition for every level of the scale (the middle anchor might read “meets the requirement with minor gaps, evidenced by a named reference”) and evidence the proposal either contains or does not.
- Technical fit: a reference architecture and a comparable deployment. The integration diagram against your ERP, identity provider, and data platform, plus one named client where they did the same thing at similar scale.
- Delivery: named CVs and references you choose. The people who will actually staff your project, not the bench, and two references picked by you from their client list, checked by phone.
- Total cost: your template, not their proposal’s. Send every bidder the same sheet covering license, implementation, integration, training, support, infrastructure, and exit. The license fee is the smallest number in the deal; the template is how you see the others.
- Dependency: the exit answer. Who owns the data, in what format, with what notice, and whether a second firm could take over. Score it while you still have leverage.
A worked example: a Riyadh warehouse rollout
A Riyadh logistics group is replacing its warehouse and dispatch platform, and four proposals arrive. The numbers are made up to show the shape; they describe no real vendor. One bid fails the gate outright, customer records hosted outside the Kingdom with no answer on transfer compliance, so it is out before scoring. The weights were signed off weeks earlier at technical fit 40%, delivery 25%, five-year cost 25%, exit 10%, and three evaluators scored each survivor on a zero-to-five scale:
- Proposal A, the global brand: technical 4.0, delivery 2.5, cost 3.0, exit 4.0. Weighted total 3.38.
- Proposal B, the lowest price: technical 2.5, delivery 3.0, cost 4.8, exit 2.0. Weighted total 3.15.
- Proposal C, the local specialist: technical 3.6, delivery 4.5, cost 3.2, exit 3.5. Weighted total 3.72.
Proposal B had the lowest five-year price by a wide margin, yet it finishes last: its assumptions page assigns integration “to the client” and its exit clause returns data in a proprietary format. Proposal A scores beautifully until the panel reads the staffing appendix and finds the “Riyadh delivery team” is an unnamed subcontractor; the score drops and the reasoning is minuted. Proposal C wins because the named team had shipped the same platform for a comparable group and buyer-chosen references confirmed it. The numbers did not make the decision; they made it visible, arguable, and repeatable.
Where software vendor evaluations go wrong
The framework is boring on purpose; the failures are human.
- Weights set after reading. Once someone has a favorite, every weight drafted afterwards favors that favorite. Fix the weights blind.
- The demo built to pass the demo. A scripted walkthrough on the vendor’s own data proves the vendor can give demos. Insist on your data, your scenario, and an hour with your own staff at the keyboard.
- The cheap bid that is cheap because the scope is. Read the assumptions and exclusions page before the price page. Integration “by the client,” migration “out of scope,” and “up to” clauses are where the missing money lives.
- Scoring the logo instead of the team. A global brand with an unnamed local subcontractor doing the work is, delivery-wise, an unnamed subcontractor. Score who shows up.
- One evaluator with a spreadsheet. A single person’s scores are a single person’s anchors. Independent scoring first, panel second, is the entire safeguard against the most confident voice in the room.
The same McKinsey research found that 17% of large IT projects go so badly they threaten the company’s existence (McKinsey, 2012). In our experience, nearly all of those began with a proposal everybody liked and no record of why anybody scored it.
How to start: one procurement, one sheet
Pick one live, mid-sized procurement. Write the gate items and the four weights before the proposals land, signed off by whoever will sign the contract. Build the anchor definitions and the cost template, send the template to every bidder, then run independent scoring and a panel that discusses only the disagreements. File everything with the contract.
Then close the loop: twelve months in, did the winner deliver what the score predicted? If yes, reuse the sheet. If not, the record tells you which dimension misled you. And if the technical dimensions need eyes you do not have in-house, bring in a vendor-neutral evaluation or an independent technology assessment; an assessor with nothing to sell you is the cheapest insurance in the process.
Four proposals on your desk and a committee expecting a recommendation next week? SDCG scores technology proposals as an independent third party and hands you a ranked, evidence-backed recommendation with the working shown. Book a free 30-minute review and bring your shortlist.