Home › Learn › The technical anchor › Discovery
The technical anchor · stage 3 of the data project lifecycle
Discovery: fail fast for the whole project
Discovery is the short stage between "here is a requirement" and "let's build it". Its job is not to fill in a document. Its job is to find out, cheaply and early, whether the project can work, and what it will really take, before anyone commits a budget, a team and a date.
Why spend weeks before building anything?
Why discovery decides the project
Every mistake gets more expensive the later it is found. A wrong definition of revenue costs an hour to fix in discovery, a week in testing, and the business's trust after go-live. A source that can't provide deletes is a design choice in discovery and a crisis in production.
So discovery is the cheapest place in the whole project to be wrong. A good discovery fails fast on purpose: it goes looking for the reasons the project might not work. Ending discovery with "stop" or "not like this" is a success: it saved the budget that a failed build would have spent.
The rough estimate from stage 2 was a guess about the project. Discovery replaces it with evidence.
Who does discovery?
Three owners, one anchor
Three roles share the work. The business analyst owns what the business needs: goals, requirements, scope and definitions. The project manager owns how the work will run: plan, MVPs, team, risks, stakeholders and sign-off. The technical anchor owns whether it will work: architecture, estimate, test strategy and release.
The anchor's extra job is to anchor all three. A scope the BA wrote that the architecture can't deliver, a plan the PM drew that ignores the estimate's assumptions, a requirement nobody can test against the data: the anchor is the one who notices the parts don't fit, because the anchor is the one looking at the real data. In particular, the anchor makes sure three things are not wrong: what is in and out of scope, the estimate, and the direction of the architecture.
What goes in the discovery document?
The discovery document, section by section
Use this as the outline. Each section has a lead, a job for the anchor even when someone else leads it, and the people who review it.
| Section | Lead | The anchor's job | Reviewed by |
|---|---|---|---|
| Business goals and success measures | BA | Check every goal can be measured with data the project will actually have. | Business owner |
| Requirements and acceptance criteria | BA | Turn each requirement into a check the data can be tested against. | Business owner, test lead |
| Sources and data profiling | BA and anchor | Profile real data from every source: volumes, keys, deletes, late and corrected rows, quality. | Source system owners |
| Definitions and data model | BA and anchor | Agree the grain of each table and every business definition, each signed by the person who owns it. | Data consumers, the domain owner (for example finance) |
| In scope and out of scope | BA writes, anchor checks | Walk the ten scope traps so nothing is left silent; anything not written out will be assumed in. | Business owner, project manager |
| Application architecture | Anchor | The data flows, layers and patterns, the non-functional needs (freshness, volume, recovery), and short decision records, proved by a thin slice. | Architect, a peer anchor |
| Platform architecture | Anchor, with the platform team | Environments, network, identity and access, security, release pipeline, monitoring and running cost. | Platform team, security, operations |
| Estimation | Anchor (effort), project manager (plan) | Count the things, use the thin slice's real rate, give a range with assumptions, and include the invisible work. | Project manager, a peer anchor |
| Plan, MVPs and team | Project manager | Check the MVP order follows dependencies and value, and that the plan matches the estimate's assumptions. | Delivery lead, business owner |
| Test strategy | Anchor, with the test lead | Reconciliation, data quality rules, performance at real volume, safe reruns, user acceptance, test environments and test data. | Test lead, business users |
| Release and run approach | Anchor | How it deploys, the cut-over and history load, the support model and the runbook plan. | Operations, release manager |
| Risks, assumptions, issues, dependencies | Project manager (log), anchor (technical items) | Every technical risk has an owner, a trigger and a mitigation; the riskiest assumptions are tested, not just listed. | All reviewers |
| Stakeholder map and communication | Project manager | Add the people who aren't in the weekly meeting but can block go-live: source owners, operations, security, downstream users. | Delivery lead |
| Decisions, change process and sign-off | Project manager | Start the decision log; give every open decision an owner, a deadline and a default. | Business owner |
Two architectures, not one. The application architecture is what the data does: where it comes from, the layers it passes through, the patterns and the rules (the data platform architecture and medallion guides show the shape). The platform architecture is what it runs on: environments, network, identity, security, the release pipeline, monitoring and cost. Projects usually design the first carefully and lose weeks in MVP 1 on the second.
The detail behind several sections is on the main page: estimates, the ten scope traps, decisions and sign-off, risks and the eight lenses for a design.
How do I fail fast in discovery?
Fail fast: the beliefs to test
Every project plan stands on a handful of beliefs. Write them down in the first days, then test the riskiest ones first, in the cheapest way that could prove them wrong. These eight come up on almost every data project.
| The belief | The cheapest test, during discovery |
|---|---|
| We can get the data | Request access in week one and pull a real extract. Access that takes six weeks is a project risk, not an admin task. |
| The data is good enough | Profile it: nulls, duplicates, broken keys, odd values. Show the profile to the business owner. |
| It fits the platform | Push one table through every layer to one report in a production-like environment: the thin slice. |
| The volume and freshness are achievable | Load a realistic day of data and time it. Check the source can take the extract load. |
| Everyone means the same thing | Write the top five definitions (revenue, active customer…) and get each signed by its owner. |
| The estimate holds | Re-estimate from the thin slice's real rate, not from the rough estimate. |
| The people and skills exist | Name who will build, test, run and support it, and check they are available when the plan needs them. |
| Nobody outside the room can block it | Meet security, operations and source owners during discovery, not at go-live. |
Decide before each test what result would change the plan. That is the "kill line": without it, every result gets explained away.
Who should review it, and how?
Reviews
A review only works if reviewers have time to read and a specific question to answer. Hold a mid-discovery review on the direction, and a full review before sign-off. Send the document at least three days ahead, record every finding, and answer each one in writing: fixed, accepted as a risk, or out of scope.
| Reviewer | What they check |
|---|---|
| Architect or a peer anchor | Is the design the simplest that works? Are the decision records honest about the options? |
| Platform team | Can the environments, network and identity be ready when MVP 1 needs them? |
| Security | Where does personal data flow, who can see it, and how long is it kept? |
| Operations | Can we monitor it, rerun it safely and support it at night? |
| Test lead | Is every requirement testable, and is there real test data? |
| Source system owners | Can the source take the load, and will it tell us about changes? |
| Business owner and key users | Do the scope, definitions and first MVP match what they need? |
| Delivery lead or project office | Does the plan match the estimate and its assumptions? |
When is discovery done?
Sign-off: when is discovery done?
Discovery is done when all of these are true, not when the calendar says so.
- Real data from every source has been profiled, and the profile was shown to the business owner.
- The thin slice ran end to end in a production-like environment, or the gap is a named, owned risk.
- Scope is written as in and out, including all ten scope traps.
- Both architectures (application and platform) are reviewed, with decision records for the big choices.
- The estimate is a range with assumptions, based on the thin slice's real rate.
- The test strategy, release approach and support model are agreed with the people who will do them.
- The top risks have owners, triggers and mitigations; the riskiest assumptions were tested.
- Every reviewer has had time to read the document, and their findings are answered in writing.
- The sponsor has decided: go, go with changes, or stop.
The sign-off records what was decided, who signed, and the assumptions the estimate depends on. When one of those assumptions breaks later, the estimate changes, visibly and without blame.
How long should it take?
A three-week discovery
An example for a medium-sized data project. Small projects can do it in one week, large programmes may need longer, but always set a timebox.
| When | What |
|---|---|
| Week 1 | Kick-off with everyone on the map. Request all access and environments. Requirements workshops. Pull and profile real data from every source. Start the decision and risk logs. |
| Week 2 | Build the thin slice and timeboxed spikes for the riskiest beliefs. Shape both architectures with options. Walk the scope traps. Mid-discovery review with the reviewers. |
| Week 3 | Re-estimate from the slice. Write the test strategy, release approach and plan with MVPs. Update risks. Send the full document to reviewers three days ahead. Answer findings. Sign-off: go, change or stop. |
What goes wrong in discovery?
Common pitfalls
- Discovery as paperwork. A thick document of filled-in templates, written without touching real data. It looks finished and proves nothing.
- The rough estimate becomes the promise. The number from stage 2 is quoted as the budget, and discovery is expected to confirm it rather than correct it.
- Architecture drawn, never proved. A clean diagram that has never run in the real environment. Network, access and format problems surface in MVP 1 instead.
- Platform architecture forgotten. The data flow is designed, but environments, identity, network and the release pipeline aren't, so MVP 1 loses its first weeks waiting.
- Scope written only as "in". Without an out-of-scope list, every unstated item is assumed to be included.
- BA and anchor working apart. Requirements are written in words nobody can test against the data, and the definitions don't match what the sources hold.
- A plan that ignores the estimate's assumptions. The project manager's dates assume access by week one; the estimate assumed week three. Nobody notices until it slips.
- Reviewers invited only to the sign-off meeting. They see the document for the first time in the room, so the review is a formality and the real objections come later.
- "Test strategy: to be defined". Testing is left for the build, so reconciliation and test data are discovered as problems late.
- Operations and security met at go-live. They raise needs that should have shaped the design months earlier.
- Risks without owners. A risk log full of worries with no trigger and no mitigation, reviewed once.
- Saying yes to keep the peace. Discovery is where "no" and "not like this" are cheapest. Hiding them to keep the sponsor happy moves the failure to the expensive end.
- No timebox. Discovery drifts for months, or is squeezed into three days. Both hide the real risks.
Next
Where to go next
Check your own project: the project review skill runs these checks on your documents. The whole lifecycle: all ten stages and what the anchor does in each. The thinking habit underneath: fail fast, own the result.