SiSu RevOps

M5  /  L5.2  /  about 13 minutes

Name the defect classes before you count anything

You ask for "a data quality report on the CRM export". You get four paragraphs saying the data has some issues. You cannot act on it, compare it to next month, or tell whether it read every row. It is not wrong. It is not a measurement either.

The fix is not a better model or a longer prompt. You never said what a defect is, so the agent picked. This lesson does that job, once, in a file.

What you will be able to do

  • Write a defect definition that is falsifiable, and spot one that is not.
  • Fix the output shape so this month's audit is comparable to last month's.
  • Produce an audit table with counts, percentages and named example IDs, from a brief you can re-run.

A defect definition is a test, not an adjective

Two definitions of the same class:

Stale records. Records that have not been touched in a long time and look abandoned.

Stale opportunity. An opportunity whose stage is one of the six open stages and whose close_date is earlier than the run date. Counted per opportunity record. Population: all rows in opportunities.csv.

The first cannot be wrong, so it cannot be right either. Two people running it get two answers. The second can be checked by anybody with the file, and argued with. That makes it useful.

A usable definition has four parts:

  1. The fields it reads, by their real column names.
  2. The comparison, with any threshold stated as a value you chose, not a vibe.
  3. The population: "12 records missing an owner" means something different over 528 rows than over 60.
  4. The unit of count: records, or rows. They differ the moment your file has a duplicated primary key. This dataset does.

Write them in 40-reference/field-policy.md, the policy file you create in this lesson, not in the chat window. Cowork carries no memory between sessions. A definition that lives only in a conversation is gone next month, and the next audit silently uses a different one. Rules in a file in the working folder are the memory.

The output shape is half the brief

Say exactly what the output looks like, or you get a different shape every month and can never draw a trend. The shape for this audit:

Column Why it is there
defect_class The name from the field policy. Not free text.
definition_ref The heading in field-policy.md that defines it. Makes the audit auditable.
count Records failing the test.
pct_of_population Count over the stated population, one decimal place.
example_ids Exactly three record IDs, so a reviewer can spot-check in a minute.
population The row count the percentage is over. Printed, not implied.

Three examples, not all, is deliberate. The audit is a summary a manager reads; the full list belongs in proposed-changes.csv and exceptions.csv, two files your pass creates in 30-output/hygiene/. An audit nobody reads has changed nothing.

Ask for defects by name

Do not ask for "data quality issues". Name the classes. This taxonomy covers most CRM exports, each class definable against a real column:

  • missing required fields
  • invalid values against a picklist
  • stale records by last-modified age
  • orphaned records: no owner, or a parent key that does not resolve
  • impossible dates: an open record with a close date in the past, a created date after a modified date
  • inconsistent formatting: country, industry, currency, owner name
  • duplicate candidates (its own lesson, L5.3)

You know which matter in your CRM. That part you cannot delegate, and it takes about twenty minutes. This is why specification cost is a fit-test factor.

See it run

The real brief, run against opportunities.csv in the shipped dataset, in the seven-part shape from Module 4:

OUTCOME
Produce a defect audit of the opportunity export against the definitions in
40-reference/field-policy.md.

INPUTS
10-data/opportunities.csv (528 rows) and 10-data/accounts.csv (308 rows).
accounts.csv is used only to resolve account_id and to read accounts.currency.
Definitions: 40-reference/field-policy.md, sections D1 to D10.

OUTPUT SHAPE
30-output/hygiene/audit-2026-08-06.md
One markdown table, one row per defect class, columns exactly:
defect_class | definition_ref | count | pct_of_population | example_ids | population
Sort descending by count. Percentages to one decimal place.
Below the table, one line: "Distinct opportunity records failing at least one
check: N of 524 (X%)."

CONSTRAINTS
Population for every class is the 524 distinct opportunity_id values unless the
definition says otherwise. Run date is 2026-08-02, which is the dataset's own
"today" (DATA-DICTIONARY.md). Do not clean, modify or rewrite any input file.
Do not propose fixes in this output.

DEFINITION OF DONE
Every class in field-policy.md D1 to D10 appears in the table, including classes
with a count of zero. Every example_ids cell holds exactly three real IDs.

STOPPING CONDITION
Stop after writing the audit file. Do not continue into duplicate detection.

EXCEPTIONS RULE
Any record you cannot classify against a definition goes to
30-output/hygiene/exceptions.csv with columns record_id, reason, definition_ref.
Do not guess.

The result, from the shipped data. Population: the 524 distinct opportunity_id values in a file of 528 rows.

defect_class ref count pct example_ids
open opportunity, close date in the past D4 14 2.7% OPP-006738, OPP-006894, OPP-006905
owner name variant D7 6 1.1% OPP-006696, OPP-006746, OPP-006795
closed, close date blank D5 5 1.0% OPP-006728, OPP-007099, OPP-007174
closed lost, loss reason blank D6 5 1.0% OPP-006670, OPP-006750, OPP-006884
duplicated primary key D1 4 (on 8 rows) 0.8% OPP-006790, OPP-006886, OPP-007099
owner blank D2 4 0.8% OPP-006774, OPP-006841, OPP-007062
currency contradicts account D8 4 0.8% OPP-006697, OPP-006729, OPP-007089
fiscal quarter disagrees with close date D9 4 0.8% OPP-006764, OPP-006773, OPP-006805
amount at or below zero D10 3 0.6% OPP-006774, OPP-006833, OPP-007117
account id does not resolve D3 2 0.4% OPP-006774, OPP-006894

Distinct opportunity records failing at least one check: 46 of 524 (8.8%).

Two lines matter most. A careless run leaves them out.

The class counts add to 51, but only 46 records are involved. Four records fail two checks, one fails three: OPP-006774 has a blank owner, an unresolvable account, and a negative amount. Report "51 defective opportunities" and you overstate by 11%. The first checker will find it. Hence the distinct-record line, exactly what the module quiz in the interactive app asks you to diagnose.

The duplicated primary key class counts 4 and touches 8 rows. That distinction is L5.4's failure section, and where your pipeline total goes wrong.

Every ID above is documented in answer-keys/PLANTED-ISSUES.md in your pack, which you open only after attempting the lab.

Try this now: three definitions, one number back

About eight minutes, one real thing end to end. You are the RevOps analyst at Talvera. Sales leadership wants "how much of the open pipeline is not real" by Monday. You have the export and no definitions, because nobody ever wrote any.

Try this now

  1. Open opportunities.csv in your pack's data folder and read the columns, or the schema in DATA-DICTIONARY.md next to it. The six open stages are Qualification, Discovery, Solution Review, Technical Validation, Proposal, Negotiation.
  2. Create 40-reference/field-policy.md from templates/M5/field-policy.md in your practice pack.
  3. Write three definitions, each with all four parts: an open opportunity whose close date has passed; an opportunity with no owner; an opportunity whose currency contradicts its account's billing currency.
  4. Fill the gaps in the starter brief and run it.
  5. Compare your counts to the reference below. If one differs, look at the definition first, not the run.

Starter brief (fill the gaps marked <<...>>):

OUTCOME
Count the records failing each of the three defect definitions in
40-reference/field-policy.md.

INPUTS
10-data/opportunities.csv
<<which second file do you need, and what exactly do you need from it?>>

OUTPUT SHAPE
A markdown table printed in the chat, not written to a file. Columns:
defect_class | count | pct_of_population | example_ids (exactly 3)

CONSTRAINTS
Population is <<state the population and say whether it is rows or distinct
opportunity_id values>>.
Today's date for any date comparison is <<which date, and why that one?>>.
Do not modify any input file.

DEFINITION OF DONE
Three rows. Every count reproducible from the file.

EXCEPTIONS RULE
<<what should happen to a record you cannot classify?>>

Stuck? Open the hints in order.

Hint 1 (Concept)

Two of your three definitions can be answered from one file. The third cannot: one class is only visible when two files disagree. Single-row defects are the easy half of data quality. Defects in the relationship between two rows are the half that survives a careless run.

Hint 2 (Move)

Join opportunities.account_id to accounts.account_id and compare opportunities.currency against accounts.currency row by row. Then decide what happens to the two opportunity rows whose account_id resolves to no account: they cannot pass this test and cannot fail it. Your exceptions rule answers that.

Hint 3 (Shape)

A correct answer has three counts in single figures and low double figures, all under 3% of the population, plus one thing not in the table: at least one record routed to exceptions because its account did not resolve. If your exceptions output is empty, re-read your currency definition. An empty exceptions file on messy data is a warning sign, not a clean bill of health.

What you should see:

defect_class count pct_of_population example_ids
open opportunity, close date in the past 14 2.7% OPP-006738, OPP-006894, OPP-006905
owner blank 4 0.8% OPP-006774, OPP-006841, OPP-007062
currency contradicts account 4 0.8% OPP-006697, OPP-006729, OPP-007089

Population: 524 distinct opportunity_id values. Run date 2026-08-02. Exceptions: OPP-006774 and OPP-006894, reason "account_id does not resolve in accounts.csv, so the currency test cannot be applied".

Your answer can legitimately differ. A population of 528 rows instead of 524 records gives slightly lower percentages: fine if the audit says which you used. Excluding closed opportunities from "owner blank" gives a count below 4: also fine, since a blank owner on a 2023 closed deal is a different problem from one on live pipeline. Not fine: a count you cannot reproduce, or a population you did not state.

If "open, close date in the past" came back far above 14, check whether you included closed stages. If it came back 0, check whether it used your machine's today instead of the dataset's 2026-08-02.

Worth knowing: the three counts took the agent seconds. The twenty minutes of definition writing is the part still worth something next month.

What you keep

  • 40-reference/field-policy.md, your first defect definitions. The artifact of the module. It survives your CRM, your job and this course.
  • 20-briefs/crm-hygiene-audit.brief.md, started from the brief above (a seed is in templates/M5/crm-hygiene-audit.brief.md of your practice pack).
  • 30-output/hygiene/audit-YYYY-MM-DD.md, the first dated audit.

Check yourself

  1. Which of these is not usable as a defect definition, and rewrite it so it is: (a) accounts with a blank owner; (b) opportunities where amount is at or below zero; (c) records that look wrong; (d) contacts whose email contains no @.
  2. Your audit says 51 defective opportunities. A colleague counts distinct IDs in your proposed-changes file and gets 46. Who is wrong, and what does the audit need to add?
  3. You inherit an audit with counts and percentages but no population column and no definition_ref. Name the two questions you now cannot answer.

Where this goes next

L5.3 takes on duplicates, the class where an agent will produce confident nonsense, and where the control is a written rule rather than a better model.