Your rules ran. You have eleven flagged deals and a pack that looks convincing. Before it goes anywhere, assume it is wrong somewhere and find out where, in less time than redoing the work. Then do the right thing with what you find: change the rule, not the output. A corrected pack is worth one week. A corrected rule is worth every week after it.
By the end of this lesson you can: verify a flag against source and find the wrong one, fix the rule rather than the row, rebuild one number by hand every week, assemble a pack a manager reads in five minutes, and defend the boundary around the forecast call out loud.
The hunt
The procedure, before the pack goes anywhere:
- Rank by amount and verify the top one by hand. Not a random one: the one a manager will act on first.
- For each flag on it, open the source and read the value. That is what the evidence line is for.
- Cross-check the number against a second source. A single source cannot contradict itself.
- When one is wrong, ask where the fix belongs. Deleting the flag is a patch. Changing the rule is a fix. Only the fix survives to next week.
Try this now
Rank this week's flags by amount. The top is OPP-007017, at 1,000,000 USD, in Solution
Review, no two-way customer activity since 2026-04-30, which is 94 days. By a wide margin the
largest at-risk deal in the book, and the first thing anybody will read.
Verify it. The flag makes two claims, and they are not equally true.
- "No contact in 94 days" is true.
activities.csvhas nothing on this opportunity after 2026-04-30. - "1,000,000 USD" is false. The snapshot from 2026-07-27, six days before the run date, records the same deal at 100,000 USD. The current opportunity row carries an extra zero.
Is it a real increase or an error? You can tell without asking anybody: a jump from 100,000 to
1,000,000 would show in your movement table as an amount increased row of +900,000 USD. It does
not. No week in the 26-week snapshot history shows this deal growing. The number changed in the
opportunity record without ever moving in the pipeline.
answer-keys/PLANTED-ISSUES.md in your pack, which you open only after attempting the lab,
confirms it under the class implausible-amount: "Amount 1000000.0 looks
like a fat finger extra zero (plausible at 100000.0 USD)". A second row in the same class,
OPP-007511, is recorded at 570,000 EUR against a last snapshot of 57,000 EUR.
Where the fix goes. Not in the pack. Add a rule:
R7 amount-plausibility
Compare opportunities.amount against the most recent pipeline_snapshots.amount
for the same opportunity_id.
If they differ by more than 2x and there is no corresponding amount-change row
in the movement table for any week, do not rank the deal.
Route it to exceptions.csv with reason "amount disagrees with last snapshot,
no movement row", and include it in the pack under Data Quality, not under
Risk.
Evidence line: R7 | opportunities.amount vs snapshot <date> | <a> vs <b>
Re-run. OPP-007017 leaves the risk list and appears in a two-line data quality section, where
a manager can act on it: send it to the deal owner to fix. Note what you did not do: delete it,
correct the number yourself, or quietly rank it at 100,000.
Reconcile the roll-up, and rebuild one number by hand
Two checks, every week, both cheap.
The roll-up reconciliation. The sum of deal amounts in your pack must equal the sum in the source for the same filter. "For the same filter" is the load-bearing phrase:
- Open pipeline from
opportunities.csvon 2026-08-02: 62 deals. - Open pipeline from the last snapshot, 2026-07-27: 60 deals.
Both are right. Two deals, OPP-007239 and OPP-007268, were created on 2026-07-28, after the
last snapshot. The pack has to say which basis it used, or a manager comparing your number to
the CRM dashboard concludes one of you is wrong.
The two files also disagree on amount for 14 of the 60 shared deals, plus one where only the
currency label differs (OPP-006697, EUR in the snapshot and USD on the opportunity row): 15
deals differ on what they are worth. Most of that is legitimate rediscounting mid-cycle, with the
snapshot preserving the value as it stood. Two of the fourteen are the fat-finger defect from the
hunt above. A reconciliation that reports "15 deals differ" and lists them beats a single total.
The hand-rebuilt number. Pick one small number every week, vary which one, and reconstruct it yourself before the pack goes out. Dagmar Wexford's book on the 2026-07-27 snapshot is 7 deals at 149,000 USD: a minute of adding up that either matches or does not. Record it in the run log. It is the cheapest tripwire in the pack, and why you can say "I checked it, here is how".
Tell these apart: a signal, and an artifact of how the field is populated
This is the difference between a forecast input and a fact about a colleague's data entry habits.
An artifact
Group pipeline_snapshots.csv by owner and forecast category:
| owner | rows in Commit | total rows | Commit share |
|---|---|---|---|
| Bettina Costa | 43 | 87 | 49.4% |
| Sylvie Dekker | 14 | 82 | 17.1% |
| Dagmar Wexford | 23 | 171 | 13.5% |
| Bettina Farrow | 29 | 217 | 13.4% |
| Aisling Calderon | 26 | 219 | 11.9% |
| Renata Vasquez | 19 | 180 | 10.6% |
| Franka Mertens | 20 | 293 | 6.8% |
| Ursula Ulrich | 8 | 134 | 6.0% |
Across the whole file, about 13% of snapshot rows sit in Commit. One owner is at nearly half. It reads like a signal about deal quality. It is a signal about a person's definition of a word. Break the same rows down by the stage they were in when marked Commit:
| Solution Review | Technical Validation | Proposal | Negotiation | |
|---|---|---|---|---|
| Bettina Costa | 11 | 10 | 6 | 16 |
| Everyone else combined | 0 | 0 | 0 | 139 |
Every Commit row belonging to any of the other seven owners is in Negotiation. All 139 of them. One owner commits from Solution Review onward. "Commit" is not a measurement in this dataset: it is a self-reported category, filled in by one person against a different definition from everybody else, and any roll-up of committed pipeline is measuring that difference.
A signal
Same file, different question: how often does a deal's close date move? Count close-date changes across the snapshot history and divide by the owner's open deals, because owners carry very different numbers of deals and a raw count would rank Franka Mertens worst for having 34 of them.
| owner | pushes | open deals | pushes per deal |
|---|---|---|---|
| Bettina Costa | 7 | 8 | 0.88 |
| Bettina Farrow | 6 | 14 | 0.43 |
| Aisling Calderon | 6 | 18 | 0.33 |
| Ursula Ulrich | 4 | 12 | 0.33 |
| Dagmar Wexford | 3 | 19 | 0.16 |
| Renata Vasquez | 1 | 13 | 0.08 |
| Franka Mertens | 2 | 34 | 0.06 |
| Sylvie Dekker | 0 | 9 | 0.00 |
This is a signal. A close-date push is an observed change in a dated field between two weekly photographs. Nobody chose a category, and it happened twice as often per deal in one book as in the next one down. Still not a conclusion (a book of eight large enterprise deals will push more than a book of thirty-four SMB deals), but it is evidence about deals, not about form filling.
The general test: ask what a row would have to do for the field to change. For a close date, something has to happen to the deal. For a forecast category, somebody has to have an opinion.
What the wrong headline earns you
Suppose your pack's headline reads: "Commit coverage is strongest in Bettina Costa's book at 49% against a team average of 13%, so her forecast is our most reliable." Here is the feedback that sentence gets:
The number is correct and the sentence built on it is not.
forecast_categoryis a field a person sets, and your finding is that one person sets it differently from the other seven. The stage breakdown says so unambiguously: all 139 Commit rows in the rest of the team sit in Negotiation, while this owner's 43 are spread across Solution Review, Technical Validation, Proposal and Negotiation. Half of that book is in Commit because of where the line is drawn, not because those deals are closer to closing.The direction is also backwards. On the one measure in this file that observes deals rather than opinions, this book pushes close dates 0.88 times per open deal, the highest rate on the team and roughly fourteen times the lowest. The pack recommended confidence in the forecast that moves most.
What would have caught it: asking what would have to happen for this field to change. A close date changes when something happens to the deal; a forecast category changes when somebody changes their mind. A metric built on the second kind measures the people filling it in, and must be reported per owner with the definition attached, or not at all.
What to change: in
risk-rules.md, add "forecast_category is self-reported and is never used as a risk signal without an accompanying stage distribution by owner". In the pack, replace the headline with the push-rate table and one line naming the denominator, because the rate is only meaningful next to the deal count.If your team wants Commit to mean one thing, that is a management conversation, and a good one. It is not something this pack can conclude on their behalf.
When this goes wrong
The break. Build the engagement gap rule the obvious way: contacts.csv has a
last_activity_date column sitting right there, so for each open deal take the most recent
last_activity_date among the account's contacts, and flag the deal if it is older than 45 days.
The tell. Take OPP-007409, a 212,000 EUR new business deal at Fairholm Facilities Ltd
(ACC-02908, Ireland, Enterprise), in Technical Validation for 101 days, owned by Renata
Vasquez. Three ways of asking the same question, three different answers:
| Method | Answer | Verdict |
|---|---|---|
Most recent contacts.last_activity_date at the account |
2026-06-17, 46 days | flagged |
Most recent row in activities.csv joined on related_id |
2026-07-16, 17 days | not flagged |
| Most recent two-way customer contact | 2026-06-17, 46 days | flagged |
The second method is the one most people build, and on a 212,000 EUR deal it says everything is fine. Open the rows it is reading:
ACT-0041766 2026-06-17 Email Connected contact CON-008733 "Chasing signature"
ACT-0045630 2026-07-10 Note Completed contact (blank) "Call notes logged"
ACT-0043721 2026-07-16 Note No answer contact (blank) "Handover note"
The two most recent rows are internal notes. contact_id is blank for internal notes and
duration_minutes is 0 for emails and notes (DATA-DICTIONARY.md in your pack's data folder).
Nobody at Fairholm was on
the other end of either. The last time anybody spoke to the customer was 17 June, chasing a
signature.
The first method happens to give the right answer here, and it is still the wrong rule.
contacts.last_activity_date is a denormalised roll-up at contact level, not deal level: it
reports an account as engaged when the engagement was on a different opportunity entirely, and
as dormant when the deal is worked through a contact whose record has not been touched. Right
answer, wrong instrument, and you will not know which you have next week.
A second, quieter failure in the same join. PLANTED-ISSUES.md documents six rows in
activities.csv whose related_id resolves to no opportunity at all, class
broken-foreign-key. An inner join drops them without a word. One, ACT-0044746, is labelled
related_type = Account while carrying an opportunity-shaped related_id of OPP-906685, so a
filter on related_type misses it too. Neither the join nor the filter says anything is missing.
The guard.
R3 engagement-gap, corrected
Source: activities.csv joined on related_id where related_type = 'Opportunity'.
Count a row as customer contact only if contact_id is populated
AND activity_type is not 'Note'
AND outcome is not in ('No answer', 'Bounced').
Flag when the most recent qualifying row is older than 45 days on a deal above
50,000 USD.
Evidence line: R3 | last qualifying activity on <opportunity_id> | <date>, <n> days
Join integrity, reported every run, not optional:
count of activities.csv rows whose related_id does not resolve, and the count
of rows excluded by each of the three conditions above. If the excluded count
is more than 30% of the rows on a deal, name that deal in the pack.
The join integrity line is the part people skip. Reporting how many rows your filter threw away is how you find out that the filter is the problem.
The five-minute pack, and the section that buys you trust
You assemble the pack yourself; a seed layout ships in templates/M6/review-pack.md of your
practice pack. Structure, in this order:
1. Three lines: what changed. Written as sentences, not numbers.
2. The movement table with its two reconciliation lines.
3. Top flagged deals as cards, most exposure first.
4. Data quality: anything R7 or the join integrity check caught.
5. Exclusions. What this pack does not cover, and why.
The exclusions section is a credibility instrument. A pack that says what it does not cover gets caught out less, and stays believed. A real one for this dataset:
This pack does not judge whether any deal will close.
Movement is computed from weekly snapshots only, which start 2026-02-02. Older
weeks have aged out of the warehouse, so there is no year-to-date figure here.
The week of 2026-04-20 is excluded: the snapshot is a duplicate of 2026-04-13,
so 2026-04-27 contains two weeks of movement.
Open pipeline is stated on the 2026-07-27 snapshot basis, 60 deals. The CRM shows
62 today because two deals were created on 2026-07-28.
Engagement is measured from logged activity only. Conversations nobody logged do
not appear, in either direction.
forecast_category is self-reported and is not used as a risk signal here.
Two deals are excluded from the risk ranking under R7 pending an amount
correction: OPP-007017 and OPP-007511.
Ninety seconds to write, and the part a sceptical manager reads first.
What stays human, in words you can say out loud
Hold the line on four things, and here is the sentence for each:
- The forecast call. "Nothing in this pack can be checked against an outcome until the quarter closes, which means there is no way to know whether it was right in time for it to matter. That is a veto under our own fit test, so the call stays mine."
- The decision to inspect a rep. "The pack flags deals. Deciding that a pattern is a coaching problem needs context that is not in any file the agent can read."
- The decision to discount. "It touches money and it is not reversible by deleting a file."
- Whether a champion is real. "The record shows who replied to emails. It does not show who will spend political capital for us."
Two product facts sit under all four, worth naming to your manager once: nothing fires when a stage changes, so the pack is a weekly cadence and a deal that goes wrong on Tuesday is caught the following Monday; and there is no memory between sessions, so anything next week's run needs has to be in the rules file.
What you keep
20-briefs/pipeline-review.brief.md, the brief you started in L6.2, complete.40-reference/risk-rules.md, the file you created in L6.3, now at version 2, with R7 added and R3 corrected. Keep the version history, one line per change naming the false positive that caused it. That history is the part of your portfolio that shows judgment.30-output/pipeline/review-YYYY-WW.mdwith an exclusions section.- A
50-runs/RUN-LOG.mdentry with the reconciliation result and the number you rebuilt by hand.
Check yourself
- You find a flag that is wrong. Deleting it takes ten seconds. Changing the rule takes fifteen minutes and will re-flag three deals that were previously clean. Which do you do, and what do you tell your manager about the three new flags?
- Your manager asks for a predicted close probability per deal. Answer using the fit test, and offer what you can give instead.
- Your pack says 62 open deals. The CRM dashboard says 60. Both are correct. Write the line that goes in the exclusions section.
- A colleague's risk rule flags deals with "no activity in 30 days" and their pack has almost no flags. Name the two questions you would ask about the rule before you believe the pack.
Where this goes next
The exercise sheet in labs/LAB-06 of your practice pack (start with its README) runs this on the shipped dataset, then on two exports of your own book a week apart, behind the redaction gate, with your own measured before and after. M7 moves from the deals you have to the accounts you do not, where the failure mode is not a wrong number but a claim that was never true.