Data Scientist 5 – Infrastructure Experimentation

Job Description:

• Build and maintain machine learning models that predict the infrastructure cost impact of A/B experiments, translating experimentally observed signals (e.g., request volume changes) into business and system metrics (e.g. projected annualized costs)
• Drive adoption of infrastructure metrics within the experimentation community through analysis, consultation with experiment owners, documentation, and training
• Partner with platform teams (Observability, Experimentation Platform) to improve the quality and coverage of infrastructure usage data feeding our models
• Extend our measurement framework to new metrics (e.g., latency) and new experiment types (e.g., infrastructure canary tests)
• Champion an infrastructure lens within the broader experimentation community, helping shift culture toward reasoning about the full ROI and infrastructure impact of experiments
• Connect with the larger analytics and experimentation communities at Netflix to bring visibility to our work and learn from others

Requirements:

• Experienced in experimentation methodology and causal inference, with a strong foundation in A/B testing, treatment effect estimation, and statistical significance
• Experienced in building and maintaining machine learning models in production, including the full lifecycle of training, evaluation, monitoring, and continuous improvement
• Fluent in Python and SQL, with experience engineering data pipelines and working with large-scale data systems
• A strong collaborator who thrives in horizontal roles with broad stakeholder surfaces, comfortable influencing decisions through data and analysis rather than direct authority
• An exceptional communicator who can flex between technical and non-technical audiences, translating statistical concepts for software engineers and business leaders alike
• Comfortable with messy, incomplete data environments and able to balance short term execution with a drive to improve data quality over time
• A strong product thinker who views data science outputs as products, taking an end-to-end ownership mindset from data quality through to user adoption
• Comfortable with ambiguity, and thrive with minimal oversight and process
• Curious about infrastructure systems; prior experience in the infrastructure domain is a strong plus but not required; the ability and motivation to learn is essential

Benefits:

• Health Plans
• Mental Health support
• 401(k) Retirement Plan with employer match
• Stock Option Program
• Disability Programs
• Health Savings and Flexible Spending Accounts
• Family-forming benefits
• Life and Serious Injury Benefits
• Paid leave of absence programs
• Full-time hourly employees accrue 35 days annually for paid time off

Apply tot his job Apply To this Job