Sciforium
Sciforium
Sciforium — өндірісте қолданылатын мультимодальды қолданбаларға арналған серверсіз құралдар әзірлейтін жасанды интеллект инфрақұрылымы компаниясы. Оның бірыңғай, OpenAI-мен үйлесімді API интерфейсі мәтін, кескін, бейне және аудио қызметтеріне, сондай-ақ модельдерді орналастыруға, модельдер кітапханасына, нақты уақыттағы бағалау конвейерлеріне және басқарылатын агенттерді іске қосуға қолжетімділік береді. Sciforium болжамды өнімділікке, төменірек шығындарға және жоғарырақ құпиялылыққа қол жеткізе отырып, командаларға AI жүйелерін әзірлеуге, бағалауға және іске қосуға көмектесуге бағытталған, тігінен интеграцияланған AMD GPU жабдығын басқарады.

Foundation Model Data Engineer

Build petabyte-scale pre-training and post-training datasets for Sciforium’s multimodal AI models. Develop high-performance data pipelines for language, vision, and video training.

Сипаттама

  • Build pre-training datasets for large language models from end to end.
  • Select and balance web, code, book, and technical-paper data to improve downstream model performance.
  • Create pipelines to clean, deduplicate, and extract signals from petabytes of unstructured data.
  • Lead post-training dataset development, including supervised fine-tuning instructions, multi-turn dialogues, and RLHF/DPO preference data.
  • Oversee the acquisition and processing of vision and video data.
  • Write high-throughput Python data-processing scripts with multiprocessing and multithreading.
  • Analyze training corpora statistically to uncover bias, knowledge gaps, and quality regressions.
  • Build synthetic-data pipelines to fill gaps in naturally collected datasets.
  • Address multimodal alignment, video compression, and temporal consistency challenges.

Талаптар

  • At least five years of industry experience in data science or machine learning.
  • Demonstrated experience building and managing datasets for foundation models.
  • Advanced Python skills, including multiprocessing, multithreading, and efficient memory management.
  • Experience working with petabyte-scale datasets for production large language or large vision models.
  • Experience creating large LLM training datasets from scratch using raw web crawls and specialized domain data.
  • Practical experience building datasets for RLHF, DPO, and multi-turn instruction following.
  • Experience managing human-labeling workflows and high-quality gold-standard datasets.
  • Strong command of Spark, Ray, WebDataset, Parquet, or comparable frameworks and formats for large-scale data.
  • Experience building large image or video datasets from scratch is a plus.
  • Familiarity with multimodal crawling, video processing, codecs, and compression is a plus.
  • Experience designing complex labeling schemas for reasoning, coding, and mathematics benchmarks is a plus.
  • A master’s or PhD in a quantitative field focused on data-centric AI or information retrieval is a plus.
  • Legal authorization to work in the United States.

Артықшылықтар

  • Medical, dental, and vision coverage.
  • 401(k) plan.
  • Daily lunch, snacks, and beverages.
  • Flexible time off.
  • Competitive pay.
  • Equity.

Ұқсас жұмыс орындары

Cardinal Health

Data Engineering Manager at Cardinal Health

Cardinal Health

Lead enterprise data transformation teams at Cardinal Health, overseeing Google Cloud platforms and analytics and AI/ML operations. Guide delivery, platform improvements, and talent development.

Ашу
Verana Health

Senior/Staff Backend Engineer, Platform and Data Ingestion

Verana Health

Build and scale clinical data ingestion, OCR, OpenSearch, and oncology platform services at Verana Health. Design APIs, ETL pipelines, and microservices that support real-world healthcare insights.

Ашу
Berkshire Hathaway Homestate Companies - Workers Compensation Division

Senior Data Platform Engineering Manager

Berkshire Hathaway Homestate Companies - Workers Compensation Division
501 – 1 000 Қызметкерлер
B2BҚаржыСақтандыру

Lead an insurance company’s enterprise data platform, engineering team, and operations across Azure, SQL, NoSQL, and analytics. The role covers reliability, automation, compliance, and platform strategy.

Ашу
Berkshire Hathaway Homestate Companies - Workers Compensation Division

Senior Data Platform Engineering Manager

Berkshire Hathaway Homestate Companies - Workers Compensation Division
501 – 1 000 Қызметкерлер
B2BҚаржыСақтандыру

Lead the data platform engineering team supporting Berkshire Hathaway Homestate Companies’ insurance operations. The role spans database and analytics platforms, cloud infrastructure, automation, reliability, and people management.

Ашу
Wealth.com

Senior Data Engineer

Wealth.com
51 – 200 Қызметкерлер
КонсалтингҚұқықСақтандыру

Lead Wealth.com’s AWS data platform, working across dbt, Redshift, and Airflow. Build dependable data systems for its estate planning platform.

Ашу