Capital One
Capital One
Capital One is a financial services and fintech company offering credit cards, auto loans, banking, and savings products. Its work combines financial expertise with technology and digital tools to improve how customers manage money. The company’s careers focus includes building customer-oriented solutions within a diverse and inclusive workforce.

AI Engineer 5, Foundation Model Hosting and LLM Inference

Lead the development and optimization of foundation-model hosting, LLM inference, and agentic AI infrastructure. Build scalable, responsible systems that improve AI capabilities and banking experiences at Capital One.

Description

  • Collaborate with engineers, research scientists, technical program managers, and product managers to deliver AI-powered products
  • Design, develop, test, deploy, and support AI software spanning foundation-model training, LLM inference, agents, multi-agent workflows, similarity search, guardrails, evaluation, experimentation, governance, and observability
  • Apply open-source and SaaS AI technologies, including AWS Ultraclusters, Hugging Face, vector databases, and PyTorch
  • Develop foundation-model optimization methods that improve the scalability, cost efficiency, latency, and throughput of production AI systems
  • Help shape the technical vision and long-term roadmap for foundational AI systems
  • Architect and optimize multi-model orchestration pipelines that combine LLMs, vector search, and domain-specific models
  • Lead cost and performance governance reviews covering GPU utilization, model throughput, and inference efficiency
  • Chair design councils or review boards to promote technical consistency and adherence to AI engineering standards
  • Mentor Principal- and Manager-level AI engineers while advancing the organization’s technical maturity

Requirements

  • Bachelor’s degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or a related field with at least 6 years of experience developing AI or ML algorithms and technologies, or a relevant master’s degree with at least 4 years of experience
  • At least 6 years of programming experience with Python, Go, Scala, CUDA, or Java
  • Experience leading AI-system development while balancing cost, latency, throughput, and accuracy
  • At least 7 years of experience deploying scalable, responsible AI solutions on cloud platforms
  • Experience designing, developing, delivering, and supporting complex AI systems
  • Experience developing AI or ML technologies such as LLM inference, similarity search, vector databases, guardrails, and memory
  • Experience optimizing training and inference software for hardware utilization, latency, throughput, and cost
  • Experience building agentic AI systems and workflows
  • Experience architecting and integrating heterogeneous AI systems into unified production pipelines
  • Experience defining and enforcing ethical AI deployment standards, including explainability, fairness, and human-in-the-loop review
  • Ability to balance model performance with operating cost through dynamic inference strategies and model compression
  • Experience right-sizing models, instance counts, and hardware types
  • Capital One may sponsor a qualified new applicant for employment authorization

Benefits

  • Performance-based incentive compensation may include cash bonuses and/or long-term incentives
  • Comprehensive, competitive, and inclusive health, financial, and other benefits designed to support overall well-being
  • Reasonable accommodations are available for applicants with disabilities

Related Jobs