You run an A/B test on a new checkout button. After 3 days, the variant shows a 12% conversion lift with p=0.04, and stakeholders want to ship immediately. You pre-registered a two-week test with a required sample size that hasn't been reached. What is the most rigorous objection to shipping now?
- A. The p-value of 0.04 is not low enough; only p<0.01 justifies a design change of this magnitude
- B. Stopping early inflates the false-positive rate because you're peeking before the pre-computed sample size is reached, making the significance unreliable ✓
- C. A 12% lift is too large to be real, so the test must be misconfigured and should be restarted
- D. Conversion is the wrong metric; you should only be measuring time-on-task for a checkout button
Correct answer: B. Optional stopping (peeking) before the planned sample size dramatically inflates Type I error, so an early 'significant' result cannot be trusted at its nominal p-value.
Your design system's color tokens are referenced directly by hex value across 40 product teams. Leadership wants to introduce dark mode without breaking every consumer. What token architecture change best enables this at scale?
- A. Introduce a semantic aliasing layer (e.g. 'surface.primary') that maps to raw palette tokens per theme, and migrate consumers to reference aliases ✓
- B. Add a second complete set of dark hex values and ask each team to conditionally swap them at build time
- C. Version the entire token package as a major release and force all 40 teams to re-map their hex values simultaneously
- D. Keep raw hex references but add a global CSS filter that inverts colors when dark mode is active
Correct answer: A. A semantic alias layer decouples consumers from raw values so themes can be swapped centrally, which is the standard scalable pattern for multi-theme design systems.
A PM hands you a spec requiring a mandatory account-creation step before users can browse product listings, citing 'we need the emails.' Your research shows this is the top drop-off point. How should you handle the conflict most effectively?
- A. Implement the spec as written since the PM owns requirements, and note your disagreement in the file
- B. Refuse to design the flow until the PM removes the requirement
- C. Reframe around the shared goal (captured users) and propose deferring signup to a later, higher-intent moment, backed by the drop-off data and a measurable test ✓
- D. Quietly ship a version without the gate and let the metrics prove you right
Correct answer: C. The strongest move addresses the underlying business goal with data and a testable alternative rather than either capitulating, obstructing, or acting unilaterally.
You must instrument a brand-new feature that has no historical data, and you want to know if it 'works.' Applying the HEART framework, which pairing of a signal and its risk is handled correctly?
- A. Use raw pageviews as your Happiness metric because more views mean users are happier
- B. Choose task-success rate as an Engagement metric since completing tasks means users are engaged
- C. Select a north-star tied to the original problem, then define per-goal signals (e.g. Task Success for a Goal, Retention for Engagement) and pair each with a counter-metric to catch gaming ✓
- D. Measure only Adoption in week one because a new feature has no other measurable dimension yet
Correct answer: C. HEART requires mapping goals to signals to metrics and guarding against optimizing one dimension at the expense of others, which counter-metrics address; the other options misassign categories or rely on vanity metrics.
In a portfolio review, an interviewer keeps asking 'but why didn't you just use the alternative pattern?' about a case study. What response best demonstrates senior judgment?
- A. Insisting your chosen solution was objectively the only correct one given the constraints
- B. Articulating the specific trade-offs of each path, what you'd have lost with the alternative, and the signal that tipped your decision ✓
- C. Admitting you didn't really consider alternatives and just went with intuition
- D. Pivoting to how much users loved the final visual design
Correct answer: B. Senior designers frame decisions as trade-offs among viable options with explicit reasoning, rather than defending a single 'right' answer or revealing no exploration.
You're designing a payments confirmation screen for a fintech app used heavily on low-end Android phones over patchy 3G. The engineer says a full-screen success animation adds 400ms of jank on those devices. What is the best resolution?
- A. Keep the animation everywhere because motion confirms the transaction and builds trust
- B. Cut the elaborate animation and confirm success with an instant, lightweight state change; reserve richer motion only where it communicates state without harming perceived performance ✓
- C. Show the animation but add a loading spinner before it so users know it's coming
- D. Detect device tier and show a lower-frame-rate version of the same animation on all devices
Correct answer: B. On constrained devices, perceived performance and instant confirmation of a money-moving action outweigh decorative motion, so the motion should be cut where it costs responsiveness without adding informational value.
Card sorting on your navigation produced clear category groupings, but a follow-up tree test shows users still fail to find 'Refund status' 60% of the time. What does this most likely indicate?
- A. The card sort was invalid and should be redone with more participants
- B. Users understand your categories but the specific label or its placement doesn't match where they expect that task to live; findability, not grouping, is the failure ✓
- C. You need to add a mega-menu exposing every item at once
- D. Tree testing is unreliable for transactional tasks and should be ignored here
Correct answer: B. Card sorting validates grouping logic while tree testing validates findability of specific items, so good groupings with poor find rates points to labeling or placement mismatches for that task.
You're adding an AI-generated summary feature. The model is sometimes confidently wrong. Which design approach best calibrates user trust?
- A. Hide the uncertainty so users aren't distracted, since surfacing it undermines confidence in the feature
- B. Show every summary with equal high-confidence styling to keep the interface consistent
- C. Communicate uncertainty and provenance (e.g. show sources, flag low-confidence output, make it easy to verify/correct), keeping a human-in-the-loop for consequential actions ✓
- D. Add a one-time disclaimer at onboarding that the AI may be wrong, then present all output as authoritative
Correct answer: C. Trust calibration requires surfacing uncertainty, provenance, and correction paths so users neither over-trust nor under-trust the AI, especially before consequential actions.
A WCAG audit flags your primary CTA: white text on a brand-blue background at a 3.2:1 contrast ratio. Marketing refuses to change the brand blue. Which solution actually achieves conformance for this normal-size text?
- A. Keep the blue and add a subtle drop shadow to the text to improve legibility
- B. Darken the button background (or the text) enough to reach at least 4.5:1, since normal-size text requires 4.5:1 regardless of brand preference ✓
- C. Increase the font weight to bold, which exempts it from the 4.5:1 requirement
- D. Leave it as-is because 3.2:1 passes the large-text threshold of 3:1
Correct answer: B. WCAG AA requires 4.5:1 for normal-size text; shadows and bold weight don't satisfy the ratio, and 3:1 only applies to large text, so the actual colors must change.
Under a hard two-week deadline you can ship only 3 of 7 planned features. Which prioritization rationale best defends the cut to stakeholders?
- A. Ship the 3 that were easiest to build so you finish comfortably early
- B. Ship the 3 that map most directly to the core user job and the original problem, and sequence the rest, explicitly stating what's deferred and why ✓
- C. Ship all 7 in a rough state so nothing is technically missing at launch
- D. Let each stakeholder pick their favorite feature to keep everyone politically satisfied
Correct answer: B. Defensible scoping ties the cut to the core problem and user job, sequences the remainder, and makes the sacrifice explicit rather than optimizing for ease or politics.