CommonAI CIC
CommonAI CIC
CommonAI CIC is a Cambridge-based community interest company and non-profit membership organisation working on the safe, responsible development of foundational AI technologies. Its collaborative engineering model connects AI startups, established businesses, public sector organisations and academic teams to share resources, develop intellectual property and advance deep-tech AI through managed programmes, including high-assurance initiatives and the Scaling Inference Lab. The organisation supports members with access to a protected Digital Commons, specialist engineering and research expertise, and cost-effective compute infrastructure for managed open-source and research projects. Hiring at CommonAI CIC offers the opportunity to contribute to practical, collaborative work at the intersection of artificial intelligence, research and public-interest technology.

Machine Learning Systems Engineer, Inference Benchmarking

Build reproducible benchmarks for LLM serving as part of CommonAI’s UK AI infrastructure research. Evaluate long-context and agentic workloads on physical systems and in simulations.

Description

  • Create and maintain reproducible LLM serving workloads, configurations, and benchmarking tools
  • Build long-context and agentic workloads with suitable existing software
  • Characterise request patterns, prompt and output lengths, caching, batching, tool use, and request dependencies
  • Set performance measures alongside application quality and service requirements
  • Work with measurement engineers to reproduce workloads on physical systems
  • Partner with simulator engineers to represent workloads while preserving key behaviours and dependencies
  • Assess how changes to workloads and serving configurations affect conclusions
  • Keep benchmarks current as inference technology and workloads change
  • Record datasets, model and software versions, configurations, provenance, and permitted uses
  • Compare workload performance across systems against agreed application quality and service requirements
  • Help engineers and investors understand infrastructure choices for specific applications

Requirements

  • Experience building or evaluating machine learning systems, inference services, performance benchmarks, or similar workload-driven systems
  • Strong practical software skills for building dependable benchmark tools, analysing results, and debugging application–infrastructure interactions
  • Understanding of LLM inference and interest in how serving choices affect performance and output quality
  • Care in designing reproducible comparisons and interpreting results against stated application requirements
  • Practical experience with AI coding agents and sound judgement in validating generated code, data transformations, and evaluations
  • Experience with vLLM, SGLang, or similar serving software, long-context workloads, agentic applications, trace analysis, or quality evaluation is an advantage
  • Ability to work with LLM serving; other forms of inference may be considered later
  • Relevant postgraduate study is helpful, but experience and work quality matter more than qualifications
  • Must be based in the UK or attend Cambridge as agreed for hybrid work

Benefits

  • Competitive salary package with pension and benefits
  • Professional development opportunities
  • Networking with influential people across technology and academia
  • Substantial tooling budget for frontier models and coding agents
  • Collaborative, supportive workplace
  • Opportunity to make a significant contribution at a growing organisation
  • Vibrant office near Cambridge train station
  • Hybrid work with Cambridge attendance agreed individually
  • Permanent, part-time, and consultancy arrangements available
  • Flexible start date
  • Inclusive and diverse workplace

Related Jobs