SiSu RevOps

M6  /  L6.4  /  about 14 minutes

Hunt the false positive, then keep the pack

Your rules ran. You have eleven flagged deals and a pack that looks convincing. Before it goes anywhere, assume it is wrong somewhere and find out where, in less time than redoing the work. Then do the right thing with what you find: change the rule, not the output. A corrected pack is worth one week. A corrected rule is worth every week after it.

By the end of this lesson you can: verify a flag against source and find the wrong one, fix the rule rather than the row, rebuild one number by hand every week, assemble a pack a manager reads in five minutes, and defend the boundary around the forecast call out loud.

The hunt

The procedure, before the pack goes anywhere:

  1. Rank by amount and verify the top one by hand. Not a random one: the one a manager will act on first.
  2. For each flag on it, open the source and read the value. That is what the evidence line is for.
  3. Cross-check the number against a second source. A single source cannot contradict itself.
  4. When one is wrong, ask where the fix belongs. Deleting the flag is a patch. Changing the rule is a fix. Only the fix survives to next week.

Try this now

Rank this week's flags by amount. The top is OPP-007017, at 1,000,000 USD, in Solution Review, no two-way customer activity since 2026-04-30, which is 94 days. By a wide margin the largest at-risk deal in the book, and the first thing anybody will read.

Verify it. The flag makes two claims, and they are not equally true.

  • "No contact in 94 days" is true. activities.csv has nothing on this opportunity after 2026-04-30.
  • "1,000,000 USD" is false. The snapshot from 2026-07-27, six days before the run date, records the same deal at 100,000 USD. The current opportunity row carries an extra zero.

Is it a real increase or an error? You can tell without asking anybody: a jump from 100,000 to 1,000,000 would show in your movement table as an amount increased row of +900,000 USD. It does not. No week in the 26-week snapshot history shows this deal growing. The number changed in the opportunity record without ever moving in the pipeline.

answer-keys/PLANTED-ISSUES.md in your pack, which you open only after attempting the lab, confirms it under the class implausible-amount: "Amount 1000000.0 looks like a fat finger extra zero (plausible at 100000.0 USD)". A second row in the same class, OPP-007511, is recorded at 570,000 EUR against a last snapshot of 57,000 EUR.

Where the fix goes. Not in the pack. Add a rule:

R7 amount-plausibility
   Compare opportunities.amount against the most recent pipeline_snapshots.amount
   for the same opportunity_id.
   If they differ by more than 2x and there is no corresponding amount-change row
   in the movement table for any week, do not rank the deal.
   Route it to exceptions.csv with reason "amount disagrees with last snapshot,
   no movement row", and include it in the pack under Data Quality, not under
   Risk.
   Evidence line: R7 | opportunities.amount vs snapshot <date> | <a> vs <b>

Re-run. OPP-007017 leaves the risk list and appears in a two-line data quality section, where a manager can act on it: send it to the deal owner to fix. Note what you did not do: delete it, correct the number yourself, or quietly rank it at 100,000.

Reconcile the roll-up, and rebuild one number by hand

Two checks, every week, both cheap.

The roll-up reconciliation. The sum of deal amounts in your pack must equal the sum in the source for the same filter. "For the same filter" is the load-bearing phrase:

  • Open pipeline from opportunities.csv on 2026-08-02: 62 deals.
  • Open pipeline from the last snapshot, 2026-07-27: 60 deals.

Both are right. Two deals, OPP-007239 and OPP-007268, were created on 2026-07-28, after the last snapshot. The pack has to say which basis it used, or a manager comparing your number to the CRM dashboard concludes one of you is wrong.

The two files also disagree on amount for 14 of the 60 shared deals, plus one where only the currency label differs (OPP-006697, EUR in the snapshot and USD on the opportunity row): 15 deals differ on what they are worth. Most of that is legitimate rediscounting mid-cycle, with the snapshot preserving the value as it stood. Two of the fourteen are the fat-finger defect from the hunt above. A reconciliation that reports "15 deals differ" and lists them beats a single total.

The hand-rebuilt number. Pick one small number every week, vary which one, and reconstruct it yourself before the pack goes out. Dagmar Wexford's book on the 2026-07-27 snapshot is 7 deals at 149,000 USD: a minute of adding up that either matches or does not. Record it in the run log. It is the cheapest tripwire in the pack, and why you can say "I checked it, here is how".

Tell these apart: a signal, and an artifact of how the field is populated

This is the difference between a forecast input and a fact about a colleague's data entry habits.

An artifact

Group pipeline_snapshots.csv by owner and forecast category:

owner rows in Commit total rows Commit share
Bettina Costa 43 87 49.4%
Sylvie Dekker 14 82 17.1%
Dagmar Wexford 23 171 13.5%
Bettina Farrow 29 217 13.4%
Aisling Calderon 26 219 11.9%
Renata Vasquez 19 180 10.6%
Franka Mertens 20 293 6.8%
Ursula Ulrich 8 134 6.0%

Across the whole file, about 13% of snapshot rows sit in Commit. One owner is at nearly half. It reads like a signal about deal quality. It is a signal about a person's definition of a word. Break the same rows down by the stage they were in when marked Commit:

Solution Review Technical Validation Proposal Negotiation
Bettina Costa 11 10 6 16
Everyone else combined 0 0 0 139

Every Commit row belonging to any of the other seven owners is in Negotiation. All 139 of them. One owner commits from Solution Review onward. "Commit" is not a measurement in this dataset: it is a self-reported category, filled in by one person against a different definition from everybody else, and any roll-up of committed pipeline is measuring that difference.

A signal

Same file, different question: how often does a deal's close date move? Count close-date changes across the snapshot history and divide by the owner's open deals, because owners carry very different numbers of deals and a raw count would rank Franka Mertens worst for having 34 of them.

owner pushes open deals pushes per deal
Bettina Costa 7 8 0.88
Bettina Farrow 6 14 0.43
Aisling Calderon 6 18 0.33
Ursula Ulrich 4 12 0.33
Dagmar Wexford 3 19 0.16
Renata Vasquez 1 13 0.08
Franka Mertens 2 34 0.06
Sylvie Dekker 0 9 0.00

This is a signal. A close-date push is an observed change in a dated field between two weekly photographs. Nobody chose a category, and it happened twice as often per deal in one book as in the next one down. Still not a conclusion (a book of eight large enterprise deals will push more than a book of thirty-four SMB deals), but it is evidence about deals, not about form filling.

The general test: ask what a row would have to do for the field to change. For a close date, something has to happen to the deal. For a forecast category, somebody has to have an opinion.

What the wrong headline earns you

Suppose your pack's headline reads: "Commit coverage is strongest in Bettina Costa's book at 49% against a team average of 13%, so her forecast is our most reliable." Here is the feedback that sentence gets:

The number is correct and the sentence built on it is not. forecast_category is a field a person sets, and your finding is that one person sets it differently from the other seven. The stage breakdown says so unambiguously: all 139 Commit rows in the rest of the team sit in Negotiation, while this owner's 43 are spread across Solution Review, Technical Validation, Proposal and Negotiation. Half of that book is in Commit because of where the line is drawn, not because those deals are closer to closing.

The direction is also backwards. On the one measure in this file that observes deals rather than opinions, this book pushes close dates 0.88 times per open deal, the highest rate on the team and roughly fourteen times the lowest. The pack recommended confidence in the forecast that moves most.

What would have caught it: asking what would have to happen for this field to change. A close date changes when something happens to the deal; a forecast category changes when somebody changes their mind. A metric built on the second kind measures the people filling it in, and must be reported per owner with the definition attached, or not at all.

What to change: in risk-rules.md, add "forecast_category is self-reported and is never used as a risk signal without an accompanying stage distribution by owner". In the pack, replace the headline with the push-rate table and one line naming the denominator, because the rate is only meaningful next to the deal count.

If your team wants Commit to mean one thing, that is a management conversation, and a good one. It is not something this pack can conclude on their behalf.

When this goes wrong

The break. Build the engagement gap rule the obvious way: contacts.csv has a last_activity_date column sitting right there, so for each open deal take the most recent last_activity_date among the account's contacts, and flag the deal if it is older than 45 days.

The tell. Take OPP-007409, a 212,000 EUR new business deal at Fairholm Facilities Ltd (ACC-02908, Ireland, Enterprise), in Technical Validation for 101 days, owned by Renata Vasquez. Three ways of asking the same question, three different answers:

Method Answer Verdict
Most recent contacts.last_activity_date at the account 2026-06-17, 46 days flagged
Most recent row in activities.csv joined on related_id 2026-07-16, 17 days not flagged
Most recent two-way customer contact 2026-06-17, 46 days flagged

The second method is the one most people build, and on a 212,000 EUR deal it says everything is fine. Open the rows it is reading:

ACT-0041766  2026-06-17  Email  Connected   contact CON-008733  "Chasing signature"
ACT-0045630  2026-07-10  Note   Completed   contact (blank)     "Call notes logged"
ACT-0043721  2026-07-16  Note   No answer   contact (blank)     "Handover note"

The two most recent rows are internal notes. contact_id is blank for internal notes and duration_minutes is 0 for emails and notes (DATA-DICTIONARY.md in your pack's data folder). Nobody at Fairholm was on the other end of either. The last time anybody spoke to the customer was 17 June, chasing a signature.

The first method happens to give the right answer here, and it is still the wrong rule. contacts.last_activity_date is a denormalised roll-up at contact level, not deal level: it reports an account as engaged when the engagement was on a different opportunity entirely, and as dormant when the deal is worked through a contact whose record has not been touched. Right answer, wrong instrument, and you will not know which you have next week.

A second, quieter failure in the same join. PLANTED-ISSUES.md documents six rows in activities.csv whose related_id resolves to no opportunity at all, class broken-foreign-key. An inner join drops them without a word. One, ACT-0044746, is labelled related_type = Account while carrying an opportunity-shaped related_id of OPP-906685, so a filter on related_type misses it too. Neither the join nor the filter says anything is missing.

The guard.

R3 engagement-gap, corrected
   Source: activities.csv joined on related_id where related_type = 'Opportunity'.
   Count a row as customer contact only if contact_id is populated
   AND activity_type is not 'Note'
   AND outcome is not in ('No answer', 'Bounced').
   Flag when the most recent qualifying row is older than 45 days on a deal above
   50,000 USD.
   Evidence line: R3 | last qualifying activity on <opportunity_id> | <date>, <n> days

   Join integrity, reported every run, not optional:
   count of activities.csv rows whose related_id does not resolve, and the count
   of rows excluded by each of the three conditions above. If the excluded count
   is more than 30% of the rows on a deal, name that deal in the pack.

The join integrity line is the part people skip. Reporting how many rows your filter threw away is how you find out that the filter is the problem.

The five-minute pack, and the section that buys you trust

You assemble the pack yourself; a seed layout ships in templates/M6/review-pack.md of your practice pack. Structure, in this order:

1. Three lines: what changed. Written as sentences, not numbers.
2. The movement table with its two reconciliation lines.
3. Top flagged deals as cards, most exposure first.
4. Data quality: anything R7 or the join integrity check caught.
5. Exclusions. What this pack does not cover, and why.

The exclusions section is a credibility instrument. A pack that says what it does not cover gets caught out less, and stays believed. A real one for this dataset:

This pack does not judge whether any deal will close.
Movement is computed from weekly snapshots only, which start 2026-02-02. Older
  weeks have aged out of the warehouse, so there is no year-to-date figure here.
The week of 2026-04-20 is excluded: the snapshot is a duplicate of 2026-04-13,
  so 2026-04-27 contains two weeks of movement.
Open pipeline is stated on the 2026-07-27 snapshot basis, 60 deals. The CRM shows
  62 today because two deals were created on 2026-07-28.
Engagement is measured from logged activity only. Conversations nobody logged do
  not appear, in either direction.
forecast_category is self-reported and is not used as a risk signal here.
Two deals are excluded from the risk ranking under R7 pending an amount
  correction: OPP-007017 and OPP-007511.

Ninety seconds to write, and the part a sceptical manager reads first.

What stays human, in words you can say out loud

Hold the line on four things, and here is the sentence for each:

  • The forecast call. "Nothing in this pack can be checked against an outcome until the quarter closes, which means there is no way to know whether it was right in time for it to matter. That is a veto under our own fit test, so the call stays mine."
  • The decision to inspect a rep. "The pack flags deals. Deciding that a pattern is a coaching problem needs context that is not in any file the agent can read."
  • The decision to discount. "It touches money and it is not reversible by deleting a file."
  • Whether a champion is real. "The record shows who replied to emails. It does not show who will spend political capital for us."

Two product facts sit under all four, worth naming to your manager once: nothing fires when a stage changes, so the pack is a weekly cadence and a deal that goes wrong on Tuesday is caught the following Monday; and there is no memory between sessions, so anything next week's run needs has to be in the rules file.

What you keep

  • 20-briefs/pipeline-review.brief.md, the brief you started in L6.2, complete.
  • 40-reference/risk-rules.md, the file you created in L6.3, now at version 2, with R7 added and R3 corrected. Keep the version history, one line per change naming the false positive that caused it. That history is the part of your portfolio that shows judgment.
  • 30-output/pipeline/review-YYYY-WW.md with an exclusions section.
  • A 50-runs/RUN-LOG.md entry with the reconciliation result and the number you rebuilt by hand.

Check yourself

  1. You find a flag that is wrong. Deleting it takes ten seconds. Changing the rule takes fifteen minutes and will re-flag three deals that were previously clean. Which do you do, and what do you tell your manager about the three new flags?
  2. Your manager asks for a predicted close probability per deal. Answer using the fit test, and offer what you can give instead.
  3. Your pack says 62 open deals. The CRM dashboard says 60. Both are correct. Write the line that goes in the exclusions section.
  4. A colleague's risk rule flags deals with "no activity in 30 days" and their pack has almost no flags. Name the two questions you would ask about the rule before you believe the pack.

Where this goes next

The exercise sheet in labs/LAB-06 of your practice pack (start with its README) runs this on the shipped dataset, then on two exports of your own book a week apart, behind the redaction gate, with your own measured before and after. M7 moves from the deals you have to the accounts you do not, where the failure mode is not a wrong number but a claim that was never true.