Reported interview periodSep 2025–Jan 2026
Improve and Evaluate an LLM Classification Pipeline
You are given a dataset of labeled examples and a helper that scores candidate label completions under a language model. Design a prompt-driven binary classifier, evaluate it, and improve it systematically.
Practice focusAssumption: the exact domain and helper signature will be provided during the interview. The core task is to produce a real-valued score for each example, choose a label, and measure quality—not merely call the model once. Requirements - Build prompts from row data without leaking the gold label. - Compare the two label candidates using their model scores, converting log scores correctly when a probability or calibrated score is required. - Return both the chosen label and a confidence-like score.…