A fintech PM proposes 'total transactions processed (cumulative, all-time)' as the team's North Star metric because it always trends up and impresses the board. What is the strongest structural objection?
- A. It is a lagging indicator and lagging indicators are always unsuitable as North Stars
- B. A monotonic cumulative counter can never decrease, so it cannot signal deteriorating product health or reflect current user value ✓
- C. Transaction count ignores revenue, and North Star metrics must be denominated in currency
- D. It cannot be broken into a driver tree because it is a single aggregate number
Correct answer: B. An ever-accumulating counter mechanically rises even if the product is dying, so it fails the core North Star test of reflecting present, incremental user value.
Daily active users drops 18% overnight with no release shipped. Before forming any user-behavior hypothesis, which first step best reflects disciplined RCA?
- A. Segment the drop by platform, app version, and geography to see if it is uniform or concentrated, which would point to instrumentation or a specific cohort ✓
- B. Immediately survey churned users to understand why they left
- C. Launch a win-back campaign to recover the lost users before the trend worsens
- D. Compare against the same day last year to check for seasonality
Correct answer: A. A sharp, sudden, uniform-looking drop is most often a data/instrumentation break, and slicing by dimensions distinguishes a logging failure in one segment from genuine broad behavior change before you theorize about users.
An A/B test on a new checkout flow shows conversion up +2.1% (p=0.03) but average order value down 3.4% (p=0.04), over 2 weeks. Net revenue-per-visitor is roughly flat. What is the most defensible next move?
- A. Ship it, because the primary metric (conversion) reached statistical significance
- B. Do not ship yet; decompose whether the AOV drop is a novelty/mix artifact and estimate the combined effect on the true business objective before deciding ✓
- C. Do not ship, because any guardrail regression is an automatic no-ship
- D. Ship it, because two significant results outweigh one flat combined metric
Correct answer: B. Conflicting signals where the combined business metric is flat require decomposing the mechanism and evaluating against the overarching objective, not deferring to whichever isolated metric hit significance.
You size the Indian market for a paid B2B invoicing SaaS bottom-up. Early you assume 63 million MSMEs and multiply the full count by your price. An interviewer pushes back. What is the correct fix?
- A. Switch to a top-down approach using a global SaaS market report and take India's GDP share
- B. Narrow the base to the serviceable segment (e.g., MSMEs that are digitized, banked, and willing/able to pay) rather than applying price to all 63 million ✓
- C. Increase the price assumption to compensate for a smaller realistic base
- D. Keep the base but apply a 90% churn discount to the final number
Correct answer: B. A bottom-up estimate must filter the raw universe down to the realistically serviceable/addressable-and-payable segment; applying ARPU to the entire count is the cascading early-assumption error.
Leadership asks you to pick EXACTLY ONE feature to build this quarter with no additional headcount: (A) a retention feature affecting 100% of users with modest per-user lift, or (B) a monetization feature affecting the paying 4% with large per-user lift. Both have equal RICE scores. What resolves the tie best?
- A. Pick neither and ask for more data since RICE is tied
- B. Choose based on the current company stage and constraint: which lever the business most needs now (e.g., retention if the funnel leaks, monetization if retention is already healthy) ✓
- C. Always choose the retention feature because retention compounds
- D. Split the quarter and ship half of each to hedge
Correct answer: B. When a scoring framework ties, the tiebreaker is strategic context and the binding constraint of the current stage, not hedging, splitting, or a universal rule.
You are launching an LLM-powered support assistant. Which acceptance-gate design is the most rigorous before a public rollout?
- A. Require average user satisfaction above 4/5 in a beta and ship if met
- B. Define a numeric max hallucination/factual-error rate on a golden eval set plus a defined fallback (e.g., escalate to human when confidence is low), and gate the launch on both ✓
- C. Ship to 5% of traffic and monitor complaint volume, rolling back if complaints spike
- D. Require the model to pass a fixed list of 50 hand-written prompts with zero errors
Correct answer: B. A responsible AI launch needs a concrete measured error threshold on a representative eval set combined with a defined safe fallback path, not just satisfaction scores or a tiny hand-picked prompt set.
For a RAG-based knowledge assistant, answers are frequently wrong even though the underlying documents contain the correct information. Your eval shows generation faithfulness is high. Where is the fault most likely, and what should you measure?
- A. The generator is hallucinating; measure faithfulness more strictly
- B. Retrieval is failing to surface the right chunks; measure retrieval recall/precision (did the correct passage make it into the context?) ✓
- C. The base model is too small; measure parameter count against a larger model
- D. The prompt template is wrong; measure token length of the system prompt
Correct answer: B. High faithfulness means the model is answering loyally to what it was given, so wrong answers despite correct source docs point to a retrieval-recall problem, which retrieval metrics isolate.
A subscription product reports NRR of 88% but a healthy 5% monthly logo churn and rising new-logo acquisition. Which interpretation is correct?
- A. NRR below 100% means the existing customer base is contracting in revenue net of expansion, so growth depends entirely on new acquisition to mask a leaky base ✓
- B. NRR of 88% is fine because it is close to 90% and logo churn is the metric that matters
- C. NRR below 100% is impossible if new-logo acquisition is rising
- D. NRR measures new customers, so 88% means acquisition is underperforming
Correct answer: A. NRR excludes new logos and captures expansion minus contraction/churn within the existing base; below 100% means that base is shrinking in revenue and acquisition is compensating for a structural leak.
Two weeks after launching a redesigned feed, engagement is up 25%. Your PM instinct says celebrate. What is the most important caveat before declaring success?
- A. The lift may be a novelty effect; you should track the cohort's engagement curve over several weeks to see if it decays toward or below baseline ✓
- B. 25% is within normal variance so it is probably noise
- C. You should immediately roll it to 100% to capture the gains before competitors react
- D. Engagement is a vanity metric and should be ignored entirely
Correct answer: A. Early post-launch spikes are frequently novelty-driven, and only watching the cohort curve over time reveals whether the lift is durable or decays back to (or below) baseline.
You want a growth loop rather than a linear funnel for a document-collaboration tool. Which mechanism qualifies as a self-reinforcing loop?
- A. Running paid ads whose revenue funds more paid ads
- B. Shared documents exposing non-users who sign up, then create and share their own documents, exposing more non-users ✓
- C. A referral bonus paid once per referred user
- D. Improving onboarding so more signups activate
Correct answer: B. A growth loop feeds its own output back as input; collaborative sharing that recruits new users who then share is compounding, whereas paid ads and one-time referrals are linear inputs.