You serve a 7B-parameter LLM (fp16 weights) with a fixed 24 GB GPU. Requests with ~500-token prompts work, but a batch with several 6,000-token prompts triggers CUDA OOM even though weights (~14 GB) plus KV cache seem to fit. What is the most accurate explanation of why long prompts specifically cause the failure?
- A. fp16 weights silently upcast to fp32 during long-sequence prefill, doubling the 14 GB weight footprint
- B. The self-attention score matrix and KV cache grow with sequence length (attention scores scale ~O(L^2) per layer during prefill), so long prompts inflate activation/cache memory far beyond the short-prompt case ✓
- C. Longer prompts increase the parameter count of the model proportionally to the number of input tokens
- D. The optimizer state for Adam is allocated at inference time and scales with prompt length
Correct answer: B. Prefill attention materializes score matrices that scale quadratically with sequence length and the KV cache grows linearly per token, so long prompts blow up transient activation/cache memory rather than the fixed weight size.
A ranking model shows offline NDCG@10 improving by 4% in the new candidate model, but the online A/B test shows flat or slightly negative engagement. Logs confirm no serving bug and identical feature values offline and online. Which is the MOST likely root cause of this offline-online divergence?
- A. The offline evaluation was computed on logged data collected under the old model's policy, so it rewards agreement with past exposure rather than true improvement (presentation/feedback-loop bias) ✓
- B. NDCG@10 is mathematically incapable of correlating with engagement metrics
- C. The online model is using fp16 while offline used fp32, changing the ranking order
- D. The A/B test lacked statistical power, which always fully explains a negative delta
Correct answer: A. Offline metrics computed on logs generated by the incumbent policy suffer exposure/selection bias, so higher offline NDCG can reflect fitting the old policy's choices rather than real online gains.
You must add drift detection to a fraud model with 40 features, several of which are strongly correlated (e.g., transaction_amount, rolling_avg_amount, amount_zscore). Running a per-feature KS test on each marginal shows no significant drift, yet the joint behavior has shifted. Which approach correctly catches this?
- A. Apply a Bonferroni correction to the per-feature KS p-values, which will reveal the joint shift
- B. Use a multivariate method such as Maximum Mean Discrepancy or a domain classifier on the joint feature vector, since shifts in the correlation structure can occur while every marginal looks stable ✓
- C. Increase the KS test sample size until each per-feature test becomes significant
- D. Switch each per-feature KS test to a chi-square test on binned marginals
Correct answer: B. A change in the joint/correlation structure can leave all marginals unchanged, so multivariate detectors (MMD, classifier two-sample tests) on the full vector are required to catch it.
A teammate proudly reports 0.997 validation AUC on a loan-default model, far above the ~0.82 the business expected. The pipeline: load table, StandardScaler.fit_transform on the FULL dataset, then train_test_split, then fit model. Beyond the scaler issue, which additional check most directly exposes classic target leakage here?
- A. Verify the model uses L2 rather than L1 regularization, since L1 causes leakage
- B. Inspect feature importances/SHAP for a feature that is a proxy or downstream artifact of the label (e.g., 'days_past_due' or 'recovery_amount' populated only after default is known) ✓
- C. Confirm the random seed is fixed so the split is reproducible
- D. Re-run with a larger validation set to see if AUC drops
Correct answer: B. Near-perfect AUC usually signals a feature that encodes the outcome; a feature populated only after the label is determined is textbook target leakage, and SHAP/importances that spotlight one such feature reveal it (the fit_transform-before-split is a separate, milder train/test contamination).
For a 70B model you want to reduce KV-cache memory and boost decode throughput without retraining a new attention scheme from scratch. The model uses standard multi-head attention with 64 query heads. Which change most directly targets KV-cache size while preserving most quality?
- A. Grouped-Query Attention: keep 64 query heads but share a smaller number of key/value heads (e.g., 8), shrinking the KV cache proportionally to the number of KV heads ✓
- B. Increase the number of attention heads to 128 so each head's KV projection is smaller
- C. Replace sinusoidal positional encodings with RoPE, which removes the need for a KV cache
- D. Switch from pre-norm to post-norm, which halves KV memory at inference
Correct answer: A. GQA reduces the number of distinct K/V projections that must be cached (cache scales with KV-head count), cutting KV memory and bandwidth while retaining full query-head expressiveness and most quality.
A RAG assistant over policy PDFs returns confident but wrong answers on questions whose ground truth is not in the corpus, and it also occasionally leaks instructions when a document contains 'ignore previous instructions'. Which pair of mitigations targets these two failure modes respectively?
- A. Increase the LLM temperature to reduce hallucination; and enable KV caching to block prompt injection
- B. Add a retrieval-confidence/grounding gate that abstains or says 'not found' when top-k similarity is low; and treat retrieved document text as untrusted data (delimit it, and do not execute instructions found inside it) to blunt prompt injection ✓
- C. Fine-tune on more Q&A pairs to eliminate hallucination; and lower top-k to 1 to prevent injection
- D. Use a larger embedding dimension to eliminate hallucination; and raise the similarity threshold to prevent injection while ignoring the injected text problem
Correct answer: B. Hallucination on out-of-corpus questions is mitigated by grounding/abstention gates on retrieval confidence, while prompt injection is mitigated by treating retrieved content as untrusted data that the model must not obey as instructions.
You deploy a risky new pricing model. You want to catch regressions before they hit real users, then ramp safely with automatic rollback. Which sequence and trigger design is soundest?
- A. Shadow mode (score live traffic, serve old model, compare) → canary to 1-5% real traffic with a guardrail metric and automatic rollback if the metric breaches a pre-set threshold with statistical significance → gradual ramp ✓
- B. Immediately route 50% of traffic to the new model and roll back only if revenue drops for a full week
- C. Canary at 100% traffic first, then shadow mode afterward to confirm
- D. Blue-green swap all traffic at once, relying on users to file support tickets as the failure trigger
Correct answer: A. Shadow mode validates on live inputs with zero user risk, then a small canary with a pre-defined, significance-gated guardrail and automatic rollback limits blast radius before a gradual ramp.
A model retrained daily on the last 24h of clickstream keeps swinging in quality: some days great, some days poor, tracking short-lived promotions and outages. A model retrained monthly drifts too slowly during real seasonal shifts. Which strategy best addresses this retraining-cadence dilemma?
- A. Always retrain as frequently as possible; more retraining strictly improves freshness
- B. Use a longer, weighted training window (recency-weighted over weeks) plus retraining triggered by monitored drift/performance thresholds rather than a fixed daily clock, and validate each candidate before promotion ✓
- C. Freeze the model permanently once it beats baseline to avoid any noise
- D. Retrain daily but only on data from the single best-performing prior day
Correct answer: B. Daily windows overfit transient noise while monthly ones lag real shifts; a recency-weighted longer window with drift/performance-triggered retraining and validation gates balances stability against adaptation.
A binary classifier for a rare disease (0.5% positive rate) reports 99.4% accuracy and 0.97 ROC-AUC, but clinicians say it misses too many cases. Which evaluation change most honestly reflects performance on the minority class under this extreme imbalance?
- A. Report accuracy at a 0.5 threshold, which is already the fairest summary
- B. Report PR-AUC (precision-recall) and inspect precision/recall at operating thresholds, since ROC-AUC can look optimistic when negatives vastly outnumber positives ✓
- C. Switch to R-squared to capture minority-class quality
- D. Oversample positives in the test set until it is 50/50, then report accuracy on that balanced test set
Correct answer: B. Under severe imbalance ROC-AUC is inflated by the huge true-negative pool; PR-AUC and threshold-specific precision/recall better expose minority-class performance, and rebalancing the test set would distort the real operating conditions.
You are choosing between full fine-tuning, LoRA/PEFT, and pure in-context learning for adapting a 13B model to a specialized legal-summarization task where you have ~3,000 labeled examples, one 24 GB GPU, and a need to serve several different client-specific variants cheaply. Which choice is best justified?
- A. Full fine-tuning per client, because it always generalizes best and storage of full weight copies is negligible
- B. LoRA/PEFT, because low-rank adapters train within the memory budget, and small per-client adapter weights can be swapped over a shared frozen base to serve many variants cheaply ✓
- C. In-context learning only, because 3,000 examples fit in the context window and no training is ever needed
- D. Full fine-tuning once on all clients merged, since client-specific behavior is unnecessary
Correct answer: B. LoRA fits the memory budget, leverages the labeled data better than few-shot prompting, and its tiny swappable adapters over a shared base make multi-variant serving far cheaper than storing full fine-tuned copies per client.
During inference (test time), how does batch normalization compute the mean and variance it uses to normalize?
- A. From the current test mini-batch statistics
- B. From running (moving-average) statistics accumulated during training ✓
- C. It is disabled entirely at inference
- D. From the validation set recomputed each step
Correct answer: B. At inference BatchNorm uses the running mean/variance estimated during training, not batch statistics, for deterministic outputs.
What does the Adam optimizer maintain in addition to the standard gradient to adapt learning rates?
- A. Only a global momentum term
- B. Exponential moving averages of both the first moment (mean) and second moment (uncentered variance) of gradients ✓
- C. The full Hessian matrix of second derivatives
- D. A running average of the loss values
Correct answer: B. Adam tracks bias-corrected first and second moment estimates per parameter to scale each update adaptively.
What is the time and memory complexity of vanilla self-attention with respect to sequence length n?
- A. O(n log n)
- B. O(n^2) ✓
- C. O(n)
- D. O(n^3)
Correct answer: B. Self-attention computes an n×n score matrix, giving quadratic O(n^2) complexity in sequence length.
A known drawback of using SMOTE to oversample a minority class is that it can:
- A. Only work on image data
- B. Generate synthetic points that blur class boundaries and cause overfitting to noisy regions ✓
- C. Guarantee improved test recall in all cases
- D. Remove the majority class entirely
Correct answer: B. SMOTE interpolates between minority neighbors and can create ambiguous points near boundaries or amplify noise, harming generalization.
Why is gradient clipping commonly applied when training RNNs/LSTMs?
- A. To prevent the vanishing gradient problem
- B. To prevent exploding gradients from destabilizing weight updates ✓
- C. To enforce weight sparsity
- D. To increase model capacity
Correct answer: B. RNNs are prone to exploding gradients over long sequences; clipping caps gradient norm to keep updates stable.
In XGBoost, the regularization term in the objective penalizes which quantity to control tree complexity?
- A. The learning rate only
- B. The number of leaves and the L2 norm of the leaf output weights ✓
- C. The depth of the input features
- D. The batch size used for training
Correct answer: B. XGBoost's regularization adds a term over the number of leaves (gamma) and the squared leaf weights (lambda) to penalize complex trees.
What is the main advantage of Bayesian optimization over grid search for hyperparameter tuning?
- A. It guarantees finding the global optimum
- B. It uses a probabilistic surrogate model to choose promising points, needing far fewer expensive evaluations ✓
- C. It requires no objective function
- D. It only works for discrete hyperparameters
Correct answer: B. Bayesian optimization builds a surrogate (e.g., Gaussian process) and an acquisition function to sample efficiently, reducing costly trials.
Why is a learning-rate warmup often used at the start of training large transformers?
- A. To immediately reach the maximum learning rate for speed
- B. To avoid large, unstable updates while adaptive optimizer moment estimates are still poorly conditioned ✓
- C. To disable regularization early
- D. To reduce the model's parameter count
Correct answer: B. Early on, adaptive-optimizer statistics are unreliable; ramping the LR up gradually prevents destabilizing early updates.
In knowledge distillation, what does the student model primarily learn from the teacher?
- A. Only the hard one-hot labels
- B. The teacher's softened probability distribution (soft targets) over classes ✓
- C. The teacher's raw weight matrices copied directly
- D. The teacher's training data order
Correct answer: B. Distillation trains the student on the teacher's temperature-softened soft labels, transferring inter-class 'dark knowledge'.
Post-training INT8 quantization of a neural network primarily trades off what?
- A. Larger model size for faster training
- B. Reduced memory/latency for a small potential loss in numerical precision and accuracy ✓
- C. Higher accuracy for slower inference
- D. More parameters for less memory
Correct answer: B. INT8 quantization shrinks and speeds up the model by using lower-precision weights/activations, at the cost of some accuracy.
The Adam optimizer combines which two ideas?
- A. L1 and L2 regularization
- B. Dropout and batch normalization
- C. Momentum and per-parameter adaptive learning rates ✓
- D. Bagging and boosting
Correct answer: C. Adam uses first-moment (momentum) and second-moment (RMSProp-style) estimates for adaptive step sizes.
A 'dead ReLU' neuron occurs when...
- A. A neuron's input stays negative so its gradient is always zero ✓
- B. Its weights grow without bound
- C. The learning rate is too small
- D. The batch size is too large
Correct answer: A. If a ReLU always outputs zero, its gradient is zero and the neuron stops learning.
Compared to Random Forest, gradient-boosted trees (e.g., XGBoost)...
- A. Build trees fully independently in parallel
- B. Build trees sequentially to correct prior residuals ✓
- C. Can never overfit
- D. Require no learning rate
Correct answer: B. Boosting fits each new tree to the residual errors of the current ensemble, sequentially.
In softmax, dividing the logits by a temperature T > 1 produces...
- A. A sharper, lower-entropy distribution
- B. Identical outputs
- C. Negative probabilities
- D. A softer, higher-entropy distribution ✓
Correct answer: D. Higher temperature flattens the distribution, spreading probability mass more evenly.
At inference time, batch normalization uses...
- A. Running moving-average statistics from training ✓
- B. The current batch's mean and variance
- C. No normalization at all
- D. Per-layer weight statistics
Correct answer: A. Inference uses the accumulated running mean/variance so outputs don't depend on batch composition.
Standard self-attention has a computational complexity in sequence length n of...
- A. O(n)
- B. O(n^2) ✓
- C. O(n log n)
- D. O(1)
Correct answer: B. Each token attends to every other token, producing an n-by-n attention matrix.
Gradient clipping is primarily used to mitigate...
- A. Vanishing gradients
- B. Overfitting
- C. Exploding gradients ✓
- D. Class imbalance
Correct answer: C. Clipping bounds gradient norms to prevent the instability of exploding gradients, common in RNNs.
Transformers typically use Layer Normalization rather than Batch Normalization because...
- A. It normalizes across features per token, independent of batch size and padding ✓
- B. It is faster only on GPUs
- C. It removes the need for residual connections
- D. It requires very large batches
Correct answer: A. Layer norm operates per-token over features, making it robust to variable sequence lengths and small batches.
In the bias-variance decomposition of expected squared error, the irreducible error term represents...
- A. The model's bias
- B. The model's variance
- C. The regularization strength
- D. Noise inherent in the data that no model can remove ✓
Correct answer: D. Irreducible error is the intrinsic noise floor independent of the chosen model.
For an extremely imbalanced dataset, using class weights in the loss versus random oversampling differs in that class weighting...
- A. Duplicates minority samples
- B. Reweights gradient contributions without duplicating data ✓
- C. Removes majority samples
- D. Changes the model architecture
Correct answer: B. Class weights scale each class's loss contribution rather than physically resampling the data.