Software projects fail for reasons that were visible on day one, and almost never for the reason written in the post-mortem. This post names the six causes we see repeat across every industry and budget size, shows why the earliest ones do the most damage, and gives you a three-question gate to run before you sign anything.
The post-mortem always blames the obvious thing: a missed deadline, a technical problem, a difficult vendor. Those are symptoms. The causes were sitting in the kickoff deck, in plain view.
The odds, honestly
Start with the base rate, because it sets how seriously to take the rest. The Standish Group’s long-running CHAOS research finds that roughly 31% of projects succeed, ~50% are “challenged” (late/over budget/under scope), ~19% fail outright, and small projects succeed far more often than large ones (Standish Group via InfoQ, 2015).
Bigger is worse. McKinsey and the University of Oxford studied more than 5,400 IT projects and found that large IT projects (initial budget > $15M) run on average 45% over budget, 7% over time, and deliver 56% less value than predicted; 17% go so badly they threaten the company’s existence (McKinsey, 2012).
Look at the shape of that McKinsey finding. Only 7% late on average, but 45% over budget and 56% less value. Projects mostly ship close to the date. They ship smaller and dearer. That is the fingerprint of scope nobody understood at the start, not of engineers who typed slowly.
Six causes, and where they sit
We’ve put the six usual causes on one chart, ranked by how much damage they do in our experience (qualitative, not measured) and split into two groups. The big ones happen before the build: the wrong target, requirements nobody understood, and no owner. The smaller ones happen during the build: integration and data bite harder than planned, problems get hidden, and the plan can’t bend when reality shows up.
Before a line of code
- You solved the wrong problem, beautifully. The most expensive failures aren’t badly built. They’re the wrong thing built well, tracing back to a fuzzy, unchallenged idea of what the project was for. A vendor builds exactly what you specified. Whether that was what you needed is on the spec, not on them. The public-sector version is an Etimad tender written around a system somebody saw at a conference, with the scope frozen before anyone asked what the department struggles with.
- The requirements were never really understood. Start on vague or assumed requirements and the project drifts, swells, and misses. Getting at what people need, rather than what they say first, means sitting with the clerk who will use the screen, not just the director who sponsors it. Skip that and you’ve bought a pile of rework.
- Nobody owned the outcome. Split responsibility between a business that “owns the requirements” and a vendor that “owns delivery,” and the gap between them is exactly where projects die. Someone has to own the result the whole way through: one named person with the authority to say no.
During the build
- Integration and data got underestimated. The plan assumed clean data and friendly existing systems. Reality handed over neither. This is the most under-budgeted part of enterprise delivery, and in Saudi Arabia it has a regulatory layer: personal data the PDPL and SDAIA’s transfer rules constrain, and NCA ECC controls the security team will insist on before go-live. Both are cheap to design in and expensive to discover in month nine.
- Problems got hidden, not surfaced. On a troubled project, bad news travels slowly. Status stays green until it can’t, and by the time the truth comes out, the cheap window to fix it has closed. The usual mechanism is a status report that counts features built rather than workflows that run end to end.
- The plan couldn’t absorb change. Requirements always change. A plan and an architecture rigid enough to snap under that change were going to fail the moment reality arrived. The architecture decisions that cost millions later are usually the ones that made change expensive.
The gate: three questions before you sign
Since most of the damage is done before the build, the fix is a gate before the contract. Three questions, in order. A “no” on any of them means you are not ready to sign, however good the proposal looks.
flowchart TD
A(["Before you sign"]) --> B{"Can you state the problem in one sentence a customer or clerk would recognize?"}
B -->|No| X(["Stop. You are about to build the wrong thing well."])
B -->|Yes| C{"Has anyone sat with the people who will use it every day?"}
C -->|No| X
C -->|Yes| D{"Is one named person accountable for the outcome, with authority to say no?"}
D -->|No| X
D -->|Yes| E(["Proceed, with an integration spike in month one"])
The integration spike is the during-the-build protection. Before any features are built, the team takes a real export from every system the new one must talk to and proves the data can be moved and matched. It costs a few weeks and reveals the duplicated customer IDs, the free-text fields that should have been codes, and the residency constraints while fixing them is cheap.
A worked example, with made-up numbers
Say a distributor commissions a SAR 3 million order-management replacement on a 12-month plan. The numbers are made up to show the shape.
Months one to four go well: features are built from the specification, and the weekly status shows 40% complete. In month five the team pulls the first real customer master from the old ERP and finds duplicated IDs and addresses typed into a notes field. The 300 hours budgeted for integration become 1,400. Status stays green because it counts features, and 70% are now done. In month ten the first end-to-end test runs, orders fail to match customers, and the truth surfaces. Go-live slips six months and the final cost lands near SAR 4.4 million, for a system that does less than promised because scope was cut to ship at all.
Now run the same project through the gate. The integration spike in month one finds the same duplicated IDs when they cost three weeks of cleansing, not eight months of rework. Status counts workflows that run end to end, so nobody is surprised in month ten. The project might still land a month late. It will not land nearly 50% over budget with half the value.
Where the rescue goes wrong
When a project is already in trouble, the instinctive fixes usually make it worse.
- Adding people to a late project. New people need onboarding from the people already behind. The delay grows before it shrinks.
- Swapping the vendor. If the specification was the problem, a new vendor inherits it and restarts the clock. Change vendors only when the vendor is genuinely the cause, and re-run the gate first.
- Hardening the contract instead of the requirements. Penalty clauses do not make an ambiguous scope clear. They make the vendor defensive, and defensive vendors hide problems longer.
- Blaming the technology and ordering a rewrite. A rewrite carries every cause on the list a second time, plus a sunk cost.
- Keeping the same status report. If percent-of-features-built got you here, it will not get you out.
The honest rescue is dull: freeze scope, get a real end-to-end test running on real data, and decide from that evidence whether to finish, cut, or stop.
How to start a project that doesn’t fail
The pattern that works is boring and reliable: the same pick-measure-pilot-scale rhythm we recommend for any technology initiative we review.
- Pick one problem. Write it in one sentence on one page, naming the process it replaces. If it’s a tender, that sentence belongs at the top of the RFP, which is most of what writing a technical RFP that gets the right vendor comes down to.
- Measure today. How long the process takes, how many times a month, and the error rate. Success is then a number, not a feeling.
- Pilot before you commit. Run a two-to-four-week discovery with the daily users, plus the integration spike on real data extracts. Sign the build contract after this, and let its findings set the scope.
- Scale with one owner and weekly end-to-end demos. One accountable person, and a weekly demo of a real workflow on real data. Anything else is a slide.
That’s the good news. Projects that start with a sharply defined problem, requirements someone really dug into, one owner, and an honest view of integration risk mostly succeed. The failure modes are knowable, so they’re avoidable.
A project drifting toward trouble, or planning one you can’t afford to get wrong? SDCG reviews troubled projects honestly and de-risks new ones before they start, through our technology assessment and project review work. We own outcomes end to end, so our advice has to hold up in the real world, not just on a slide. Book a free 30-minute review.