Pinterest
Pinterest
1,001 – 5,000 Employees
H1B Visa SponsorB2CMedia
Pinterest is a visual discovery platform where people find inspiration, save ideas, and organize them around interests, projects, and purchases. Its experience combines personalized recommendations and visual search across topics such as home design, fashion, recipes, and events, while also helping creators and retailers reach audiences through commerce tools and advertising.

Senior Staff Software Engineer, AI Platform - Palo Alto Hybrid

Lead the strategy, architecture, and adoption of Pinterest’s AI and machine learning evaluation platform. Help establish reliable evaluation practices for GenAI experiences across the company.

Description

  • Identify modernization opportunities and expand the roadmap for a company-wide evaluation system with technical leadership
  • Set cross-functional technical direction, align partner teams, and increase adoption of shared evaluation capabilities
  • Make long-term architectural decisions that strengthen the user experience and reliability of Pinterest’s AI and ML evaluation platform
  • Develop evaluation methods and standards that combine automated metrics, expert review, human feedback, safety, and business outcomes
  • Lead complex initiatives across teams, manage technical tradeoffs and risks, and deliver measurable impact
  • Evolve Pinterest’s AI and ML evaluation platform into a unified, dependable system used across the company
  • Work with engineering, research, product, and safety teams to improve quality, launch decisions, and confidence in AI-powered experiences

Requirements

  • At least 3 years of hands-on experience designing and operating evaluation systems for GenAI and complex machine learning systems
  • Experience creating benchmarks and golden datasets, applying automated metrics and human evaluation, using model-based grading, and analyzing failures
  • Strong proficiency in Python
  • Working knowledge of distributed systems principles
  • Ability to iterate effectively with AI tools during coding and system design
  • Extensive knowledge of GenAI and recommendation-system evaluation libraries and frameworks, including AutoEvals, OpenEvals, and Ragas
  • At least 5 years of experience developing scalable platforms for distributed experimentation, data and model versioning, orchestration, observability, release management, and production monitoring
  • Sound judgment regarding GenAI risks such as robustness, bias, privacy, security, and the limits of automated evaluation signals
  • Bachelor’s degree in computer science, engineering, or a related discipline, or equivalent experience

Benefits

  • Eligibility for equity
  • Flexible PinFlex work model
  • In-person collaboration once or twice per quarter
  • Equal opportunity workplace
  • Medical or religious accommodation support during the application process

Related Jobs

CrowdStrike

Regional Healthcare Sales Manager, West Coast

CrowdStrike

Lead strategic healthcare accounts across the West Coast, selling CrowdStrike’s AI-native cybersecurity platform to enterprise customers. Build executive relationships, work with channel partners, and grow regional revenue.

Open