Home › Learn › The technical anchor › Discovery

The technical anchor · stage 3 of the data project lifecycle

Discovery: fail fast for the whole project

Discovery is the short stage between "here is a requirement" and "let's build it". Its job is not to fill in a document. Its job is to find out, cheaply and early, whether the project can work, and what it will really take, before anyone commits a budget, a team and a date.

Why spend weeks before building anything?

Why discovery decides the project

Every mistake gets more expensive the later it is found. A wrong definition of revenue costs an hour to fix in discovery, a week in testing, and the business's trust after go-live. A source that can't provide deletes is a design choice in discovery and a crisis in production.

So discovery is the cheapest place in the whole project to be wrong. A good discovery fails fast on purpose: it goes looking for the reasons the project might not work. Ending discovery with "stop" or "not like this" is a success: it saved the budget that a failed build would have spent.

The rough estimate from stage 2 was a guess about the project. Discovery replaces it with evidence.

Who does discovery?

Three owners, one anchor

Three roles share the work. The business analyst owns what the business needs: goals, requirements, scope and definitions. The project manager owns how the work will run: plan, MVPs, team, risks, stakeholders and sign-off. The technical anchor owns whether it will work: architecture, estimate, test strategy and release.

Technical anchorarchitecture, estimate,test, releaseProject managerplan, MVPs, risks,stakeholdersBusiness analystgoals, requirements,scope, definitionsDiscovery documentone document,three owners,the anchor checks it allReviewersFail-fast testsFindings answeredreviewprovefixSign-offgo, changeor stop
One document, three owners. The anchor makes sure the parts agree with each other and with the real data.

The anchor's extra job is to anchor all three. A scope the BA wrote that the architecture can't deliver, a plan the PM drew that ignores the estimate's assumptions, a requirement nobody can test against the data: the anchor is the one who notices the parts don't fit, because the anchor is the one looking at the real data. In particular, the anchor makes sure three things are not wrong: what is in and out of scope, the estimate, and the direction of the architecture.

What goes in the discovery document?

The discovery document, section by section

Use this as the outline. Each section has a lead, a job for the anchor even when someone else leads it, and the people who review it.

SectionLeadThe anchor's jobReviewed by
Business goals and success measuresBACheck every goal can be measured with data the project will actually have.Business owner
Requirements and acceptance criteriaBATurn each requirement into a check the data can be tested against.Business owner, test lead
Sources and data profilingBA and anchorProfile real data from every source: volumes, keys, deletes, late and corrected rows, quality.Source system owners
Definitions and data modelBA and anchorAgree the grain of each table and every business definition, each signed by the person who owns it.Data consumers, the domain owner (for example finance)
In scope and out of scopeBA writes, anchor checksWalk the ten scope traps so nothing is left silent; anything not written out will be assumed in.Business owner, project manager
Application architectureAnchorThe data flows, layers and patterns, the non-functional needs (freshness, volume, recovery), and short decision records, proved by a thin slice.Architect, a peer anchor
Platform architectureAnchor, with the platform teamEnvironments, network, identity and access, security, release pipeline, monitoring and running cost.Platform team, security, operations
EstimationAnchor (effort), project manager (plan)Count the things, use the thin slice's real rate, give a range with assumptions, and include the invisible work.Project manager, a peer anchor
Plan, MVPs and teamProject managerCheck the MVP order follows dependencies and value, and that the plan matches the estimate's assumptions.Delivery lead, business owner
Test strategyAnchor, with the test leadReconciliation, data quality rules, performance at real volume, safe reruns, user acceptance, test environments and test data.Test lead, business users
Release and run approachAnchorHow it deploys, the cut-over and history load, the support model and the runbook plan.Operations, release manager
Risks, assumptions, issues, dependenciesProject manager (log), anchor (technical items)Every technical risk has an owner, a trigger and a mitigation; the riskiest assumptions are tested, not just listed.All reviewers
Stakeholder map and communicationProject managerAdd the people who aren't in the weekly meeting but can block go-live: source owners, operations, security, downstream users.Delivery lead
Decisions, change process and sign-offProject managerStart the decision log; give every open decision an owner, a deadline and a default.Business owner

Two architectures, not one. The application architecture is what the data does: where it comes from, the layers it passes through, the patterns and the rules (the data platform architecture and medallion guides show the shape). The platform architecture is what it runs on: environments, network, identity, security, the release pipeline, monitoring and cost. Projects usually design the first carefully and lose weeks in MVP 1 on the second.

The detail behind several sections is on the main page: estimates, the ten scope traps, decisions and sign-off, risks and the eight lenses for a design.

How do I fail fast in discovery?

Fail fast: the beliefs to test

Every project plan stands on a handful of beliefs. Write them down in the first days, then test the riskiest ones first, in the cheapest way that could prove them wrong. These eight come up on almost every data project.

The beliefThe cheapest test, during discovery
We can get the dataRequest access in week one and pull a real extract. Access that takes six weeks is a project risk, not an admin task.
The data is good enoughProfile it: nulls, duplicates, broken keys, odd values. Show the profile to the business owner.
It fits the platformPush one table through every layer to one report in a production-like environment: the thin slice.
The volume and freshness are achievableLoad a realistic day of data and time it. Check the source can take the extract load.
Everyone means the same thingWrite the top five definitions (revenue, active customer…) and get each signed by its owner.
The estimate holdsRe-estimate from the thin slice's real rate, not from the rough estimate.
The people and skills existName who will build, test, run and support it, and check they are available when the plan needs them.
Nobody outside the room can block itMeet security, operations and source owners during discovery, not at go-live.

Decide before each test what result would change the plan. That is the "kill line": without it, every result gets explained away.

Who should review it, and how?

Reviews

A review only works if reviewers have time to read and a specific question to answer. Hold a mid-discovery review on the direction, and a full review before sign-off. Send the document at least three days ahead, record every finding, and answer each one in writing: fixed, accepted as a risk, or out of scope.

ReviewerWhat they check
Architect or a peer anchorIs the design the simplest that works? Are the decision records honest about the options?
Platform teamCan the environments, network and identity be ready when MVP 1 needs them?
SecurityWhere does personal data flow, who can see it, and how long is it kept?
OperationsCan we monitor it, rerun it safely and support it at night?
Test leadIs every requirement testable, and is there real test data?
Source system ownersCan the source take the load, and will it tell us about changes?
Business owner and key usersDo the scope, definitions and first MVP match what they need?
Delivery lead or project officeDoes the plan match the estimate and its assumptions?

When is discovery done?

Sign-off: when is discovery done?

Discovery is done when all of these are true, not when the calendar says so.

The sign-off records what was decided, who signed, and the assumptions the estimate depends on. When one of those assumptions breaks later, the estimate changes, visibly and without blame.

How long should it take?

A three-week discovery

An example for a medium-sized data project. Small projects can do it in one week, large programmes may need longer, but always set a timebox.

WhenWhat
Week 1Kick-off with everyone on the map. Request all access and environments. Requirements workshops. Pull and profile real data from every source. Start the decision and risk logs.
Week 2Build the thin slice and timeboxed spikes for the riskiest beliefs. Shape both architectures with options. Walk the scope traps. Mid-discovery review with the reviewers.
Week 3Re-estimate from the slice. Write the test strategy, release approach and plan with MVPs. Update risks. Send the full document to reviewers three days ahead. Answer findings. Sign-off: go, change or stop.

What goes wrong in discovery?

Common pitfalls

  1. Discovery as paperwork. A thick document of filled-in templates, written without touching real data. It looks finished and proves nothing.
  2. The rough estimate becomes the promise. The number from stage 2 is quoted as the budget, and discovery is expected to confirm it rather than correct it.
  3. Architecture drawn, never proved. A clean diagram that has never run in the real environment. Network, access and format problems surface in MVP 1 instead.
  4. Platform architecture forgotten. The data flow is designed, but environments, identity, network and the release pipeline aren't, so MVP 1 loses its first weeks waiting.
  5. Scope written only as "in". Without an out-of-scope list, every unstated item is assumed to be included.
  6. BA and anchor working apart. Requirements are written in words nobody can test against the data, and the definitions don't match what the sources hold.
  7. A plan that ignores the estimate's assumptions. The project manager's dates assume access by week one; the estimate assumed week three. Nobody notices until it slips.
  8. Reviewers invited only to the sign-off meeting. They see the document for the first time in the room, so the review is a formality and the real objections come later.
  9. "Test strategy: to be defined". Testing is left for the build, so reconciliation and test data are discovered as problems late.
  10. Operations and security met at go-live. They raise needs that should have shaped the design months earlier.
  11. Risks without owners. A risk log full of worries with no trigger and no mitigation, reviewed once.
  12. Saying yes to keep the peace. Discovery is where "no" and "not like this" are cheapest. Hiding them to keep the sponsor happy moves the failure to the expensive end.
  13. No timebox. Discovery drifts for months, or is squeezed into three days. Both hide the real risks.

Next

Where to go next

Check your own project: the project review skill runs these checks on your documents. The whole lifecycle: all ten stages and what the anchor does in each. The thinking habit underneath: fail fast, own the result.