Data scientist interview questions — and the CV claims behind them
The modelling, experimentation and judgement questions data scientists actually get, what each is really probing, and which line on your CV an interviewer will press hardest.
What this interview is actually testing
Whether you can tell a model that works from a model that shipped — and whether the accuracy figures on your CV came from a validation set or from production.
The questions, and what each one is really asking
1.Tell me about a model you put into production.
What it probes: The single most discriminating question in this interview. Building a model and running a model are different jobs, and most CVs blur them.
2.How did you know your model was actually working after launch?
What it probes: Monitoring, drift, and whether anyone was watching six months later. Many candidates have never seen the after.
3.Walk me through an experiment you designed.
What it probes: Power, randomisation unit, and what you did about the result you did not like. Interviewers listen for whether the stopping rule existed before the data did.
4.When did you choose the simpler model?
What it probes: Judgement over enthusiasm. A candidate who has never preferred logistic regression to something fashionable has either been lucky or has not been responsible for maintaining anything.
5.How would you explain this model to the person who has to act on it?
What it probes: Whether the work reaches a decision. A model nobody trusts is a model nobody uses, and trust is built in that conversation.
6.Tell me about a time the data was too messy to answer the question.
What it probes: Whether you can say no. Knowing when the data cannot support the claim is the core professional skill, and it is the one this job shares with a truthfulness audit.
7.What would you do differently on that project?
What it probes: Whether you have revisited the work. The candidates with nothing to say here usually handed the model over and never saw it again.
The CV claim they'll press hardest
A model performance figure — "improved accuracy to 94%", "reduced churn by 20%".
There are two very different numbers hiding behind one bullet, and interviewers separate them immediately. Offline metrics on a held-out set are one claim; realised business impact in production is another, and the gap between them is where most data science projects live and die. Expect "was that offline or online?" as the very next sentence. Saying "94% on validation; it never shipped because the ops team could not action it in time" is a strong answer about a real constraint. Presenting a validation number as a business outcome is the fastest way to lose a technical interviewer, because they have made exactly that mistake themselves.
There is one question in a data science interview that reorganises everything after it: did it ship?
Not because unshipped work is worthless — plenty of excellent analysis correctly never reaches production — but because the answer tells the interviewer which conversation you are able to have. A candidate who has only built models talks about features, architectures and validation scores. A candidate who has run one talks about drift, retraining cadence, the ops team that had to act on the output, and the week the upstream schema changed without warning.
Both are real experience. Only one supports the phrase “deployed” on a CV.
Two numbers, one bullet
The performance claim on a data science CV is the most reliably ambiguous line in any technical résumé, and the ambiguity is usually accidental.
“Improved churn prediction accuracy to 94%” could mean an AUC-adjacent metric on a held-out sample from a historical dataset. It could mean a realised reduction in churn measured against a control. These are wildly different achievements, and the interviewer’s next sentence will separate them: offline or online?
The honest answers are all fine:
- “94% on validation. It shipped, and the online lift was smaller — about half.”
- “Offline only. It never shipped; the retention team could not action a weekly score.”
- “We never got a clean control, so I can tell you the model’s metrics but not the business impact.”
Every one of those is a credible professional answer, and the third is the kind of sentence that makes senior people relax, because it demonstrates you know what your evidence supports.
What does not survive is a validation number presented as business impact. Not because it is unusual — it is extremely common, and often introduced by a well-meaning CV rewrite rather than by the candidate — but because the interviewer has made that mistake themselves and can smell it from one question away.
The professional skill this job shares with an audit
“Tell me about a time the data was too messy to answer the question” is the question that most rewards preparation, and it is worth understanding why it is asked.
Data science sits at a peculiar pressure point: someone senior wants a number, and you are the only person in the room who knows the number cannot be trusted. Saying so is the job. Producing it anyway, with a caveat nobody reads, is how organisations end up acting on noise.
Interviewers ask this because they need to know which of those two you do under pressure. It is, incidentally, exactly the same discipline a claim has to pass on a CV: does the evidence support the statement, and what do you do when it does not.
Preparing
Split every number on your CV into offline and online, and write the one sentence of context each needs. Pick one model that shipped, one that did not, and one that should not have — and be ready to say why for each.
Our free claim checker flags the lines likeliest to draw a follow-up; the STAR guide covers the answer shape.
Questions about the interview itself
- What questions are asked in a data scientist interview?
- Typically a technical screen (SQL, statistics, sometimes coding), a modelling or case discussion, an experimentation question, and behavioural questions about stakeholders and shipped work. The production question — did your model ever run against real traffic — is increasingly the pivot the rest of the conversation turns on.
- How technical are data science interviews now?
- Technical enough that inflated tooling claims get caught, but the differentiator has shifted. Teams have plenty of candidates who can fit a model and fewer who can define the problem, design the experiment, and get the output used. Prepare the judgement half at least as much as the algorithms half.
- Should I list a model or framework I only used once?
- List it honestly and be precise about the level. 'Used once, on a prototype' is a perfectly good CV line and an easy interview answer. A framework listed flatly alongside your daily tools invites a question pitched at the level of the surrounding list, which is the mismatch that damages you — not the inexperience itself.