Decision exercises
E = exercise; Q = exact SQL query. All operating numbers fictional. Personal responses are deliberately blank. Model answers are in a separate file and optional disclosure in the browser.
E01 · Choose the value unit
Decision: Should the North Star count accounts, people, minutes or exports?
Run: sql/Q01.sql; chart charts/Q01.svg; data csv/Q01_result.csv.
Read it: Compare DB01 account reach with task volume and confirmation coverage.
Counterargument: Would one large account or repeated locale versions distort the story?
Guardrails: Filters use date, unlike task cards. Only whole UTC weeks fully inside the selected date range and ended before cutoff are included. Missing confirmations limit interpretation.
Next evidence: Confirm account identity, repeated job frequency and use-confirmation method.
PRD sections: 2, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E02 · Find the important handoff
Decision: Which stage deserves the next investigation?
Run: sql/Q02.sql; chart charts/Q02.svg; data csv/Q02_result.csv.
Read it: Use created tasks as the common denominator, then separate parallel review/export paths.
Counterargument: Could an expected rights or plan block explain the gap?
Guardrails: Review and export are parallel checkpoints; do not force them into a strictly ordered funnel.
Next evidence: Inspect real traces and task sessions; validate missing events before naming a defect.
PRD sections: 3, 6, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E03 · Defend activation
Decision: What qualifies as a first useful result?
Run: sql/Q03.sql; chart charts/Q03.svg; data csv/Q03_result.csv.
Read it: Count users, retain mature zero-task signups and require review/use before day seven.
Counterargument: Does a late review leak future information into the first week?
Guardrails: All mature signups, including zero-task users. Workflow filter means primary intended workflow.
Next evidence: Confirm whether users, accounts or invited seats are the right production denominator.
PRD sections: 6, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E04 · Mature repeat value
Decision: Which cohort merits customer-success investigation?
Run: sql/Q04.sql; chart charts/Q04.svg; data csv/Q04_result.csv.
Read it: Use the complete days 28–41 window, counts and cohort mix.
Counterargument: Are later cohorts censored or jobs naturally infrequent?
Guardrails: Unconditional signup return during days 28–41, not retention conditional on activation.
Next evidence: Interview returning and nonreturning users about need frequency and unobserved offline use.
PRD sections: 3, 14, 17
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E05 · Signal versus cause
Decision: Does activation cause later return?
Run: sql/Q05.sql; chart charts/Q05.svg; data csv/Q05_result.csv.
Read it: Explain the association and a future account-randomized intervention.
Counterargument: Could initial intent or account mix cause both?
Guardrails: Descriptive association; intent, account mix and confirmation coverage may explain it.
Next evidence: Assignment unit, sample/power, coverage and a disconfirming test.
PRD sections: 12, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E06 · Quality evidence first
Decision: Can reviewed acceptance justify expansion?
Run: sql/Q06.sql; chart charts/Q06.svg; data csv/Q06_result.csv.
Read it: Read review coverage, acceptance and critical-error counts together.
Counterargument: Who did not review, and why?
Guardrails: Reviewed acceptance is selected; missing review remains unknown.
Next evidence: Representative bilingual labels, sampling mechanism and hard-case evidence.
PRD sections: 12, 13, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E07 · Latency without arithmetic errors
Decision: Should the team reduce queue time or reviewer wait?
Run: sql/Q07.sql; chart charts/Q07.svg; data csv/Q07_result.csv.
Read it: Compare root ready latency with completion and Q23 human wait.
Counterargument: Are incomplete slow tasks excluded?
Guardrails: Creation to visible-ready includes wait/retry. Completer-only; inspect non-completion in Q02. Not sum of component p95.
Next evidence: Trace spans, timeouts and status UI observations across comparable workloads.
PRD sections: 11, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E08 · Recover value safely
Decision: Which recovery should be built first?
Run: sql/Q08.sql; chart charts/Q08.svg; data csv/Q08_result.csv.
Read it: Compare tasks with failures, recovered renders and recovered useful tasks.
Counterargument: Does render recovery actually restore use?
Guardrails: All failures mature at cutoff except latest-day tasks excluded by generator; recovery-to-render differs from customer value.
Next evidence: Failure eligibility, customer intent, observation window and debit reconciliation.
PRD sections: 6, 11, 17
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E09 · Cost of useful work
Decision: What changes cost per useful result most?
Run: sql/Q09.sql; chart charts/Q09.svg; data csv/Q09_result.csv.
Read it: Include failed/nonmonetized attempts and compare all service scopes.
Counterargument: Does low confirmation coverage inflate the denominator problem?
Guardrails: Includes all selected task costs incl failures; excludes period allocations, shown separately Q18. Review is vendor fraction; customer time separate.
Next evidence: Real invoices, provider rates, vendor/customer labor split and stable use-confirmation sampling.
PRD sections: 2, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E10 · Review burden
Decision: Does fast production save customer time?
Run: sql/Q10.sql; chart charts/Q10.svg; data csv/Q10_result.csv.
Read it: State preparation, review and repair separately; net estimate is not causal savings.
Counterargument: Is the manual baseline measured or estimated?
Guardrails: Manual baseline is an invented estimate; net time is not causal savings.
Next evidence: Matched same-task baseline, quality parity and downstream workload.
PRD sections: 3, 12, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E11 · Hidden cost driver
Decision: Which operational cost deserves attention?
Run: sql/Q11.sql; chart charts/Q11.svg; data csv/Q11_result.csv.
Read it: Rank dollar components, not summed heterogeneous units.
Counterargument: Would suppressing support tickets only hide work?
Guardrails: Do not add heterogeneous units; quantity and rate remain component-specific.
Next evidence: Support resolution quality, invoice reconciliation and utilization assumptions.
PRD sections: 14, 17
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E12 · Explain three meters
Decision: How should a failed job settle usage?
Run: sql/Q12.sql; chart charts/Q12.svg; data csv/Q12_result.csv.
Read it: Reconcile reserved=consumed+released+outstanding within each meter.
Counterargument: Could a retry create a duplicate debit?
Guardrails: Illustrative entitlement model, not claimed production billing. Never sum minute/credit/count units.
Next evidence: Actual plan/contract rules, event dedup and failure-settlement tests.
PRD sections: 10, 11, 13, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E13 · Reject a bad experiment readout
Decision: Can the apparent variant difference support rollout?
Run: sql/Q13.sql; chart charts/Q13.svg; data csv/Q13_result.csv.
Read it: Q13 counts mature accounts with mixed seat assignments; inspect this interference before inference.
Counterargument: Are team members exposed to both variants?
Guardrails: This lab assigns users, so team accounts can have mixed variants. This audit is not a causal comparison; fix assignment before inference.
Next evidence: Account-level assignment and actual exposure, preregistered success and harm criteria.
PRD sections: 12, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E14 · Growth worth buying
Decision: Which channel deserves the next experiment?
Run: sql/Q14.sql; chart charts/Q14.svg; data csv/Q14_result.csv.
Read it: Use mature, matched cost and activated-user cohorts; do not call this paying-account CAC.
Counterargument: Does channel intent or sales assistance explain activation?
Guardrails: Allocated synthetic spend; all users have 42-day follow-up here. Not CAC per paying account or causal channel performance.
Next evidence: Paying-account linkage, channel attribution, incremental acquisition test and retention.
PRD sections: 2, 14, 17
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E15 · Trace the evidence
Decision: Which records support your recommendation?
Run: sql/Q15.sql; chart charts/Q15.svg; data csv/Q15_result.csv.
Read it: Inspect one task per row and follow its review/use/cost IDs.
Counterargument: Do rows duplicate users, accounts or money?
Guardrails: Fictional task IDs; no customer identity or private payload.
Next evidence: Real event provenance and task-version integrity.
PRD sections: 10, 11, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E16 · Include never-starters
Decision: Whose absence is hidden by active-user charts?
Run: sql/Q16.sql; chart charts/Q16.svg; data csv/Q16_result.csv.
Read it: Retain zero-task mature users and avoid assigning a cause from absence.
Counterargument: Could an invitation be intended only for review?
Guardrails: No task does not identify the cause; no session instrumentation in this lab.
Next evidence: Role/intent and consented nonstarter research; session events if necessary.
PRD sections: 3, 6, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E17 · Commercial retention
Decision: Is rising MRR healthy?
Run: sql/Q17.sql; chart charts/Q17.svg; data csv/Q17_result.csv.
Read it: Reconcile opening+new+expansion-contraction-churn=closing; inspect GRR/NRR.
Counterargument: Does new revenue hide losses among existing accounts?
Guardrails: Full calendar months only; exclude October partial. Normalized fictional subscription MRR; not cash or actual price tiers.
Next evidence: Actual contracts, renewals, logo loss, refunds and accounting policy.
PRD sections: 2, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E18 · Full economics
Decision: Can the service sustain this workload?
Run: sql/Q18.sql; chart charts/Q18.svg; data csv/Q18_result.csv.
Read it: Preaggregate billing, task costs and period allocations separately before joining month.
Counterargument: Which costs and cash timing remain excluded?
Guardrails: Not recognized gross margin or profit; customer labor, acquisition and unmodeled overhead excluded. Task-cost month is task creation convention.
Next evidence: Acquisition, overhead, recognition policy and provider commitments.
PRD sections: 2, 14, 17
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E19 · Candidate release
Decision: Does a better average justify deployment?
Run: sql/Q19.sql; chart charts/Q19.svg; data csv/Q19_result.csv.
Read it: Pair heldout source cases and inspect critical errors, latency and cost by slice.
Counterargument: Does a critical slice get worse?
Guardrails: Fictional paired cases, not actual model output; no real release evidence.
Next evidence: Rights-cleared real media, independent bilingual labels and source-isolated split.
PRD sections: 12, 13, 17
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E20 · Trust an automated judge?
Decision: Should automated review replace bilingual checks?
Run: sql/Q20.sql; chart charts/Q20.svg; data csv/Q20_result.csv.
Read it: Compute failure-detection recall and false alarms; inspect critical misses.
Counterargument: How many important failures did the judge miss?
Guardrails: Synthetic labels; compute failure recall, inspect slices and critical misses before using a live blocking judge.
Next evidence: Real adjudicated labels, agreement slices and operational false-positive costs.
PRD sections: 12, 13
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E21 · Choose feasible scope
Decision: Which opportunity fits actual capacity?
Run: sql/Q21.sql; chart charts/Q21.svg; data csv/Q21_result.csv.
Read it: Account for dependencies and separate engineering, design and review weeks.
Counterargument: What if reviewer capacity, not engineering, is binding?
Guardrails: Proposed estimates, no effect size or confidence probability; high reach is not a forecast.
Next evidence: Actual capacity, range estimates, demand evidence and opportunity cost.
PRD sections: 1, 4, 16
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E22 · Release versus readiness
Decision: Which missing gate prevents pilot?
Run: sql/Q22.sql; chart charts/Q22.svg; data csv/Q22_result.csv.
Read it: All gate states are unverified; name the evidence and accountable role.
Counterargument: Does a well-written test count as a passed test?
Guardrails: All proposed gates unverified; no actual product release performed.
Next evidence: Actual test results, support readiness, approval and rollback drill.
PRD sections: 1, 12, 17
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E23 · One elapsed-time story
Decision: Where is the customer's real wait?
Run: sql/Q23.sql; chart charts/Q23.svg; data csv/Q23_result.csv.
Read it: Distinguish ready-to-review wait, active work and total time to use.
Counterargument: Are unconfirmed long tasks censored?
Guardrails: Different completed populations in each duration; missing review/use may censor long waits.
Next evidence: Incomplete durations, reviewer staffing and end-to-end observation.
PRD sections: 6, 11, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time:
E24 · Explain recorded paths
Decision: Where does the last recorded step occur?
Run: sql/Q24.sql; chart charts/Q24.svg; data csv/Q24_result.csv.
Read it: Use timestamp-ordered edges and conservation; review/export can swap order.
Counterargument: Are skipped events instrumentation gaps or true behavior?
Guardrails: Ordered by actual timestamps; review/export can occur in either order. Last recorded is not proven abandonment.
Next evidence: Observed task traces and missing event tests.
PRD sections: 6, 10, 14
Spoken task: State one takeaway, one caveat and one action in 30 seconds. Then stop.
Your answer:
Your score and elapsed time: