LogoFresh Jobs Ahead
Normal Computing
Normal Computing
51 – 200 Employees
The available profile information identifies Normal Computing as a company founded in 2022, but does not describe its products, services, industry, or hiring context. More company-specific details are needed to provide a useful overview for job seekers.
LinkedIn
Company Jobs

Runtime Engineer, AI Hardware

Design runtime software for Normal Computing’s memory-centric AI hardware, leading architecture, hardware abstraction, and LLM inference performance.

USUnited StatesCACaliforniaNYNew York
$200,000 – $300,000 / year
HybridFull-timeMiddleSeniorEnglish RequiredEngineer
C++Rust

Description

  • Lead runtime architecture across device management, memory allocation, host-device transfers, executable loading, and command submission.
  • Develop the runtime for simulators and FPGA prototypes through a consistent interface ahead of silicon availability.
  • Abstract hardware differences across simulators, FPGA prototypes, silicon, and future generations.
  • Manage device state stored on hardware, including model weights kept resident between requests.
  • Reduce runtime overhead on the LLM inference critical path to improve end-to-end latency and throughput.
  • Define interfaces with compiler, driver, and firmware components.
  • Guide the runtime’s technical direction and review designs that depend on it.
  • Work with compiler and serving engineers to shape effective interfaces.

Requirements

  • Experience developing runtimes, drivers, or hardware abstraction layers for accelerators or other specialized hardware.
  • Strong C++ or Rust systems programming skills, including memory management, concurrency, and low-level performance.
  • Demonstrated ownership of a complex systems component from design through production.
  • Experience leading technical decisions across teams.
  • Understanding of LLM inference workloads and runtime needs.
  • Comfort developing software for changing hardware, from simulation through bring-up.
  • Bonus: familiarity with open runtime designs such as IREE’s HAL or with CUDA or ROCm runtime internals.
  • Bonus: experience with in-memory compute or other non-von-Neumann hardware.
  • Bonus: experience integrating a runtime with an inference server such as vLLM.
  • Bonus: contributions to open-source runtime or systems infrastructure.

Benefits

  • Equity.
  • Performance-based incentives.
  • Comprehensive compensation package.
  • Benefits intended to support employee well-being and professional growth.
  • Competitive, equitable compensation.
  • Reasonable accommodations for employees with disabilities.

Related Jobs