Canonical metric dictionary
All definitions are proposed for interview practice, not official AI Studios metrics. All business baselines and targets are unknown. SQL measures fictional records only. DB identifiers are independent of SORI K identifiers.
DB01 · Weekly quality-value accounts
Meaning: Fund repeatable customer value
Business decision: Fund repeatable customer value
Formula: COUNT(DISTINCT account_id)
Unit: accounts
Grain: account-week
Numerator: Accounts with accepted, no-critical-error output and confirmed use
Denominator: None; count
Eligible cohort window: UTC Monday–Sunday of confirmed_use_at; only complete weeks
Exclusions: Internal/test/bot accounts in production; absent in fiction
Data event source: v_task; review + use events
Owner: Product
Leading lagging: lagging value
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Incomplete confirmations undercount; large accounts have more opportunities
Guardrails: DB07 coverage; DB09 errors; DB13 cost
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q01
DB02 · Confirmed quality-use tasks
Meaning: Separate output volume from reach
Business decision: Separate output volume from reach
Formula: COUNT(confirmed_quality_use=1)
Unit: tasks
Grain: task
Numerator: Accepted/no-critical-error tasks confirmed used
Denominator: None; count
Eligible cohort window: Task created cohort, use observed by cutoff
Exclusions: Same as DB01
Data event source: v_task
Owner: Product
Leading lagging: lagging value
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: One source may spawn many locale tasks; duplicates can inflate counts
Guardrails: DB01 account reach
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q01
DB03 · Task milestone counts
Meaning: Find the next handoff to investigate
Business decision: Find the next handoff to investigate
Formula: COUNT(DISTINCT task_id) at each checkpoint
Unit: tasks
Grain: task-checkpoint
Numerator: Tasks with selected recorded checkpoint
Denominator: All created tasks for context
Eligible cohort window: Task created date range; events observed by cutoff
Exclusions: No removal of blocked created tasks
Data event source: v_task / sim_outputs
Owner: Product analytics
Leading lagging: leading
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Missing telemetry differs from abandonment; review/export parallel
Guardrails: Q24 flow conservation
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q02
DB04 · Render completion rate
Meaning: Isolate backend reliability from use
Business decision: Isolate backend reliability from use
Formula: SUM(render_success)/COUNT(tasks)
Unit: fraction
Grain: task
Numerator: Created tasks with >=1 successful attempt
Denominator: All created tasks
Eligible cohort window: Created cohort and cutoff
Exclusions: None; optionally publish eligible-attempt rate separately
Data event source: v_task / sim_attempts
Owner: Engineering
Leading lagging: leading
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Entitlement/consent blocks lower all-task rate without being defects
Guardrails: DB02 actual use
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q02
DB05 · Seven-day confirmed-value activation
Meaning: Assess onboarding
Business decision: Assess onboarding
Formula: SUM(activated7)/COUNT(mature7 users)
Unit: fraction
Grain: user signup cohort
Numerator: Signups with quality-confirmed use in [signup,signup+7d)
Denominator: All signups observed >=7 complete days
Eligible cohort window: Individual UTC signup window, first 7 days
Exclusions: Exclude immature users only; include zero activity
Data event source: v_user_cohort
Owner: Product analytics
Leading lagging: leading for return
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Review must occur before day 7; reported use is incomplete
Guardrails: DB07/09; account interference
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q03
DB06 · Days 28–41 confirmed-value return
Meaning: Evaluate recurring need
Business decision: Evaluate recurring need
Formula: SUM(return28_41)/COUNT(mature42 users)
Unit: fraction
Grain: user signup cohort
Numerator: Signups with quality-confirmed use in [signup+28d,signup+42d)
Denominator: All signups observed >=42 complete days
Eligible cohort window: Full 42-day eligibility
Exclusions: Exclude immature only; include unactivated and zero activity
Data event source: v_user_cohort
Owner: Product analytics
Leading lagging: lagging
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Unconditional signup return is not activated-user retention; natural job frequency varies
Guardrails: DB07 coverage by cohort
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q04
DB07 · Preview review coverage
Meaning: Decide if quality inference is trustworthy
Business decision: Decide if quality inference is trustworthy
Formula: COUNT(reviewed_at)/COUNT(preview_at)
Unit: fraction
Grain: task
Numerator: Previewed tasks with a review
Denominator: All previewed tasks
Eligible cohort window: Created cohort; reviews by cutoff
Exclusions: None; reviewers only after preview in this lab
Data event source: v_task
Owner: Domain quality
Leading lagging: measurement guardrail
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Synthetic reviews sampled; real missingness could be selective
Guardrails: Never impute missing as rejected
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q06
DB08 · Reviewed acceptance
Meaning: Choose editing or quality improvements
Business decision: Choose editing or quality improvements
Formula: SUM(accepted=1)/COUNT(reviewed_at)
Unit: fraction
Grain: reviewed task
Numerator: Reviewed tasks accepted
Denominator: Reviewed tasks
Eligible cohort window: Created cohort; review observed
Exclusions: Unreviewed remain NULL
Data event source: sim_reviews / v_task
Owner: Domain quality
Leading lagging: leading
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Selected sample, not all-task acceptance
Guardrails: DB07 and DB09
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q06
DB09 · Critical content failures
Meaning: Hold unsafe or materially incorrect output
Business decision: Hold unsafe or materially incorrect output
Formula: SUM(critical_error); rate=SUM/COUNT(reviewed)
Unit: tasks and fraction
Grain: reviewed task
Numerator: Reviewed tasks with critical error
Denominator: Reviewed tasks for rate; report counts too
Eligible cohort window: Review cutoff; versioned rubric
Exclusions: Unreviewed are unknown
Data event source: sim_reviews
Owner: Domain quality
Leading lagging: guardrail
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Zero observed does not guarantee zero risk; labels fictional
Guardrails: Numbers/names/rights slice review
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q06
DB10 · Ready latency p50/p95
Meaning: Locate slow completed journeys
Business decision: Locate slow completed journeys
Formula: Nearest-rank percentile(ready_at-created_at)
Unit: seconds
Grain: completed task
Numerator: Ordered elapsed seconds at ceil(p*n)
Denominator: Completed task count n
Eligible cohort window: Task creation to visible-ready; created cohort
Exclusions: Incomplete tasks excluded from percentile, reported separately
Data event source: v_task ready_seconds
Owner: Engineering
Leading lagging: leading
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Completer bias; not full human-review turnaround
Guardrails: Q02 noncompletion and Q23 review wait
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q07
DB11 · Failure recovery to value
Meaning: Choose recovery work
Business decision: Choose recovery work
Formula: COUNT(failed_attempts>0 AND confirmed_quality_use)/COUNT(failed_attempts>0)
Unit: fraction
Grain: task
Numerator: Tasks with failed attempt then quality-use
Denominator: Tasks with >=1 failed attempt
Eligible cohort window: By fixed cutoff; disclose recovery observation time
Exclusions: Pre-attempt entitlement blocks are separate
Data event source: v_task
Owner: Engineering/CS
Leading lagging: leading
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Right censoring and recoverability mix; render recovery differs
Guardrails: DB12 repeated work; DB16 debit correctness
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q08
DB12 · Task retry rate
Meaning: Diagnose repeated work
Business decision: Diagnose repeated work
Formula: COUNT(attempts>1)/COUNT(attempts>0)
Unit: fraction
Grain: task
Numerator: Tasks with multiple render attempts
Denominator: Tasks with any render attempt
Eligible cohort window: Created cohort
Exclusions: Pre-attempt blocks
Data event source: v_task
Owner: Engineering
Leading lagging: diagnostic
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Helpful retry should not be suppressed just to reduce rate
Guardrails: DB11 and total cost
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q08
DB13 · Direct vendor USD per confirmed-use task
Meaning: Choose full task improvements
Business decision: Choose full task improvements
Formula: SUM(direct_vendor_usd)/SUM(confirmed_quality_use)
Unit: USD/task
Grain: task cohort
Numerator: Render/translation/delivery/vendor-review/support costs across all selected tasks
Denominator: Confirmed quality-use tasks in selected cohort
Eligible cohort window: Task costs allocated to task-created month
Exclusions: Period idle/shared excluded here; shown in DB23; customer labor separate
Data event source: sim_costs preaggregated in v_task
Owner: Product/finance
Leading lagging: lagging efficiency
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Low coverage raises denominator uncertainty; rates illustrative
Guardrails: DB07, DB09, DB23 full cost
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q09
DB14 · Customer active review and repair
Meaning: Avoid moving work to people
Business decision: Avoid moving work to people
Formula: AVG(prep_minutes),AVG(review_minutes),AVG(repair_minutes)
Unit: minutes/task
Grain: reviewed task
Numerator: Sum each time category
Denominator: Reviewed tasks
Eligible cohort window: Created cohort
Exclusions: Unreviewed unknown
Data event source: sim_reviews
Owner: Product/CS
Leading lagging: guardrail
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Self-report and selected reviews; customer labor distinct from vendor cost
Guardrails: Quality and elapsed time
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q10
DB15 · Estimated net customer time
Meaning: Form a time-saving hypothesis
Business decision: Form a time-saving hypothesis
Formula: AVG(manual_baseline_estimate-prep-review-repair)
Unit: minutes/task
Grain: reviewed task
Numerator: Estimated baseline less all measured customer task work
Denominator: Reviewed tasks
Eligible cohort window: Same reviewed cohort
Exclusions: Does not include all downstream work; unknown becomes gap
Data event source: sim_reviews
Owner: Research
Leading lagging: proposed outcome proxy
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Invented baseline is not a counterfactual or causal savings
Guardrails: Matched task comparison; quality parity
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q10
DB16 · Entitlement outstanding
Meaning: Find double charge/hold leakage
Business decision: Find double charge/hold leakage
Formula: SUM(reserved-consumed-released) per meter
Unit: meter units
Grain: task-meter
Numerator: Reserved minus consumed minus released
Denominator: None; balance by distinct meter
Eligible cohort window: Closed-task ledger at cutoff
Exclusions: Never combine dubbing minutes, generative credits and video counts
Data event source: sim_entitlements
Owner: Billing engineering
Leading lagging: guardrail
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Simplified reservation model; not public billing implementation
Guardrails: Duplicate and failed-job contract tests
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q12
DB17 · Experiment assignment validity
Meaning: Decide if comparison supports inference
Business decision: Decide if comparison supports inference
Formula: Accounts with >1 assigned variant / assigned accounts
Unit: fraction
Grain: account
Numerator: Accounts exposed to mixed variants
Denominator: Assigned accounts
Eligible cohort window: Same experiment and exposure period
Exclusions: None; diagnose before ITT calculation
Data event source: sim_users variant
Owner: Analytics
Leading lagging: validity guardrail
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Lab intentionally uses seat assignments; Q13 reveals interference
Guardrails: Randomize at account; no treatment conclusion here
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q13
DB18 · Acquisition spend per activated mature user
Meaning: Choose a channel test
Business decision: Choose a channel test
Formula: SUM(acquisition_usd)/SUM(activated7)
Unit: USD/user
Grain: user/channel
Numerator: Allocated signup costs in mature42 cohort
Denominator: Activated users within that mature42 cohort
Eligible cohort window: 42-day mature signups in selected signup period
Exclusions: Immature excluded equally from spend and denominator
Data event source: v_user_cohort
Owner: Growth
Leading lagging: diagnostic
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Not paying-account CAC, no causal channel attribution
Guardrails: Later return and commercial conversion
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q14
DB19 · Mature zero-task share
Meaning: Study never-starting customers
Business decision: Study never-starting customers
Formula: COUNT(no task)/COUNT(mature42 users)
Unit: fraction
Grain: user
Numerator: Mature users with no created task by cutoff
Denominator: All mature42 users
Eligible cohort window: 42-day eligible signup cohort; no-task evaluated to cutoff
Exclusions: Immature users
Data event source: sim_users LEFT/EXISTS tasks
Owner: Research
Leading lagging: leading
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: No sessions or stated intent available; cannot assign cause
Guardrails: Interview nonstarters
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q16
DB20 · Closing subscription MRR
Meaning: Review recurring commercial value
Business decision: Review recurring commercial value
Formula: Opening+new+expansion-contraction-churn
Unit: USD/month
Grain: account-month
Numerator: Normalized monthly subscription value
Denominator: None; amount
Eligible cohort window: Full calendar months only
Exclusions: Usage packs, add-ons and one-off cash not modeled as MRR
Data event source: sim_billing
Owner: Finance
Leading lagging: lagging business
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Illustrative contracts; not GAAP revenue; no reactivation in lab
Guardrails: DB21/22 and cash separately
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q17
DB21 · Gross revenue retention
Meaning: Detect existing-value loss
Business decision: Detect existing-value loss
Formula: (opening-contraction-churn)/opening
Unit: fraction
Grain: existing account-month
Numerator: Retained opening MRR excluding expansion
Denominator: Opening MRR
Eligible cohort window: Full calendar months; mature contract population
Exclusions: Exclude new revenue and expansion
Data event source: sim_billing
Owner: Finance/CS
Leading lagging: lagging
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Plan mix/concentration can conceal logo churn
Guardrails: DB22; account churn needs further instrumentation
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q17
DB22 · Net revenue retention
Meaning: Assess expansion versus losses
Business decision: Assess expansion versus losses
Formula: (opening+expansion-contraction-churn)/opening
Unit: fraction
Grain: existing account-month
Numerator: Existing MRR after expansion and losses
Denominator: Opening MRR
Eligible cohort window: Full calendar months
Exclusions: Exclude new accounts
Data event source: sim_billing
Owner: Finance/CS
Leading lagging: lagging
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Expansion can hide lost accounts
Guardrails: DB21 and customer reasons
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q17
DB23 · Full period service cash contribution
Meaning: Understand sustained service exposure
Business decision: Understand sustained service exposure
Formula: net cash-payment fees-direct costs-idle/shared
Unit: USD
Grain: period
Numerator: Cash contribution after declared service scope
Denominator: None; amount, margin=amount/net cash if >0
Eligible cohort window: Whole calendar months; October partial excluded by default
Exclusions: Acquisition/R&D/corporate overhead excluded from service proxy; R&D separately shown
Data event source: sim_billing + preaggregated sim_costs + sim_period_costs
Owner: Finance
Leading lagging: lagging
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Not accounting gross profit; acquisition needs separate reconciliation
Guardrails: Quality, vendor and customer workload
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q18
DB24 · Heldout task pass by slice
Meaning: Select a candidate without averaging harm
Business decision: Select a candidate without averaging harm
Formula: SUM(task_pass)/COUNT(heldout cases)
Unit: fraction
Grain: case-candidate-slice
Numerator: Candidate passing task cases
Denominator: Same independent heldout cases per candidate
Eligible cohort window: Frozen development/heldout split
Exclusions: Development cases excluded; repeats not independent samples
Data event source: sim_eval_cases/results
Owner: AI/domain evaluation
Leading lagging: pre-release
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Use paired case bootstrap; no real model benchmark.
Known bias: Fictional labels; no media or model runs; paired comparison needed
Guardrails: Critical failures, reviewer disagreement, latency/cost
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q19
DB25 · Judge failure-detection recall
Meaning: Choose the role of automatic checks
Business decision: Choose the role of automatic checks
Formula: true_failure_detected/(true_failure_detected+missed_failure)
Unit: fraction
Grain: heldout case-candidate
Numerator: Human-labeled failures detected by judge
Denominator: All human-labeled failures
Eligible cohort window: Same heldout set
Exclusions: Undefined with no failures
Data event source: sim_eval_results
Owner: Evaluation
Leading lagging: guardrail
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Synthetic judge; errors can correlate with the same model
Guardrails: False alarms and critical misses by slice
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q20
DB26 · Engineering capacity demand
Meaning: Choose bounded scope
Business decision: Choose bounded scope
Formula: SUM(eng_weeks) of selected dependent opportunities
Unit: person-weeks
Grain: opportunity portfolio
Numerator: Estimated engineering person-weeks
Denominator: None; demand; compare to available capacity after reserve
Eligible cohort window: Declared planning horizon, currently unapproved
Exclusions: No conversion to calendar deadline
Data event source: plan_opportunities
Owner: Engineering/PM
Leading lagging: planning input
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Low/high reach is scenario range; no causal effect estimate
Guardrails: Design/reviewer capacity and dependencies
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q21
DB27 · Unresolved release gates
Meaning: Hold launch until evidence exists
Business decision: Hold launch until evidence exists
Formula: COUNT(gate state != verified)
Unit: gates
Grain: requirement-release
Numerator: Unverified required gates
Denominator: All required gates for context
Eligible cohort window: Exact proposed release/version
Exclusions: Do not equate test specification with passing result
Data event source: plan_gates
Owner: Engineering/PM/domain
Leading lagging: release guardrail
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: All are unverified; synthetic labels cannot approve real release
Guardrails: Named approvals and rollback drill
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q22
DB28 · Time from creation to confirmed use
Meaning: Balance machine and human waits
Business decision: Balance machine and human waits
Formula: AVG(confirmed_use_at-created_at); percentile recommended
Unit: hours
Grain: completed useful task
Numerator: Elapsed hours across confirmed-quality-use tasks
Denominator: Confirmed quality-use tasks
Eligible cohort window: Created cohort to use cutoff
Exclusions: Unconfirmed excluded, report coverage
Data event source: v_task
Owner: Product operations
Leading lagging: lagging
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Completer bias; mean hides tail; Q23 reports review wait separately
Guardrails: Missing-use count and active workload
Evidence status: Proposed metric definition; synthetic implementation only
Baseline: None
Target: None
Query: Q23
DB29 · Eligible payment success
Meaning: Separate payment failure from product-value failure
Business decision: Separate payment failure from product-value failure
Formula: successful unique payment intents / eligible submitted payment intents
Unit: fraction
Grain: payment intent
Numerator: Eligible payment intents with confirmed successful settlement
Denominator: Unique eligible submitted intents, including failed attempts under the same intent
Eligible cohort window: Declared checkout cohort and settlement lag
Exclusions: Internal tests, duplicate transport events; cancellations reported separately
Data event source: PROPOSED payment intent and provider settlement events; absent from lab
Owner: Billing/growth
Leading lagging: commercial leading
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Retries/events are not separate buyers; asynchronous settlement can be delayed
Guardrails: Refunds, duplicate debits, customer value
Evidence status: Proposed definition; not instrumented or implemented in this lab
Baseline: None
Target: None
Query: None
DB30 · Mature account paid conversion
Meaning: Test sustainable commercial adoption
Business decision: Test sustainable commercial adoption
Formula: new accounts becoming net paying within 30 days / all new accounts observed 30 days
Unit: fraction
Grain: account signup cohort
Numerator: Eligible new accounts with first settled non-refunded paid subscription in window
Denominator: All eligible new accounts mature 30 days, including zero activity
Eligible cohort window: [account creation, creation+30d), plus explicit refund-lag policy
Exclusions: Internal/test accounts; exclude existing customers; no seat duplication
Data event source: PROPOSED account signup, settlement, subscription and refund linkage; not implemented
Owner: Growth/finance
Leading lagging: commercial lagging
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Sales-led and self-service intent differ; invoice issue is not payment
Guardrails: Quality use, refunds, service contribution and mature return
Evidence status: Proposed definition; not instrumented or implemented in this lab
Baseline: None
Target: None
Query: None
DB31 · Stale approval prevention
Meaning: Protect current-version distribution
Business decision: Protect current-version distribution
Formula: blocked stale-version distribution attempts / all stale-version distribution attempts
Unit: fraction
Grain: distribution attempt
Numerator: Stale approval attempts correctly blocked by server
Denominator: All stale approval attempts, with audited detection coverage
Eligible cohort window: Release/version test suite and production audit period reported separately
Exclusions: Do not mix synthetic test and real operational populations
Data event source: PROPOSED source/target hashes, approval and final distribution events; absent from lab
Owner: Engineering/domain
Leading lagging: quality guardrail
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: No detected attempt does not prove no stale distribution; false blocks need auditing
Guardrails: Unauthorized distribution and reviewer burden
Evidence status: Proposed definition; not instrumented or implemented in this lab
Baseline: None
Target: None
Query: None
DB32 · Confirmed unauthorized disclosure
Meaning: Keep privacy separate from usage growth
Business decision: Keep privacy separate from usage growth
Formula: count of independently confirmed unauthorized content/data disclosures
Unit: incidents
Grain: incident
Numerator: Confirmed incidents by severity and affected tenant
Denominator: No rate denominator required; also report audit coverage and exposure
Eligible cohort window: Declared incident detection/reporting period; revisions versioned
Exclusions: False positives adjudicated separately; missing audit evidence unknown
Data event source: PROPOSED access audit, incident and deletion evidence; absent from lab
Owner: Security/privacy
Leading lagging: privacy guardrail
Uncertainty: Real baseline/target unknown. Fictional outputs are descriptive practice; report n, coverage and slices. Do not interpret a selected cohort as population prevalence.
Known bias: Zero reported incidents is not proof of zero exposure; avoid invasive logging
Guardrails: Permission canaries, consent/deletion tests and response workload
Evidence status: Proposed definition; not instrumented or implemented in this lab
Baseline: None
Target: None
Query: None
Instrumentation still needed
Real traffic-to-qualified-signup conversion, payment attempts/success, refund reasons, account renewal reasons, customer workload sampling, consent/rights audit, deletion propagation, source-version lineage, independent learning/communication outcomes, live-agent resolution and task-specific latency. These are proposed additions, not fabricated measurements. Define eligibility, event contract and owner before baseline collection.