Anyone AI
Anyone AI
Anyone AI is a Latin America–focused education and talent platform helping software engineers develop careers in artificial intelligence and machine learning. Its remote, mentor-led programs combine technical training, including a Machine Learning Developer track, with career preparation such as resume and LinkedIn guidance, interview coaching, and English practice. The company also operates a talent marketplace that connects its trained community with employers seeking AI professionals, alongside a placement-oriented model that includes income-share payments and employer partnerships.

Senior Software Engineer, SWE-Bench Evaluation

Review open-source software tasks from GitHub issues and pull requests for Anyone AI. Assess their clarity, test quality, reproducibility, complexity, and suitability for benchmarking.

Description

  • Review and assess software engineering tasks based on real GitHub issues and pull requests.
  • Check that task descriptions and success criteria are clear and complete.
  • Evaluate unit tests for accuracy, coverage, and resilience.
  • Find flaky tests, missing dependencies, version mismatches, and environment problems.
  • Check whether tasks can be reproduced consistently across environments.
  • Judge each task’s practical difficulty and technical complexity.
  • Recommend whether tasks should be accepted, revised, or removed.
  • Review bug fixes and feature work, including repository setup, dependency handling, changes across files and modules, and overall technical quality.

Requirements

  • At least three years of professional software engineering experience.
  • Strong experience working in large codebases spanning multiple files.
  • Experience reviewing pull requests, debugging problems, and maintaining production software.
  • A solid understanding of unit testing and test coverage.
  • Ability to judge whether tests validate a solution correctly without unnecessarily limiting implementation choices.
  • Experience managing dependencies, configuring environments, and ensuring reproducibility.
  • Strong command of Git and GitHub-based development workflows.
  • Ability to analyze complex technical issues and provide clear written feedback.
  • Contributions to or maintenance of open-source projects are a plus.
  • Experience with SWE-Bench, SWE-Bench Verified, or comparable coding benchmarks is a plus.
  • Experience with major Python open-source projects, such as Django, Flask, scikit-learn, SymPy, matplotlib, requests, or pytest, is a plus.
  • Experience with Docker, CI/CD, pip, conda, or dependency pinning is a plus.
  • Knowledge of test fixtures, test isolation, or property-based testing is a plus.
  • Experience designing technical assessments or reviewing coding challenges is a plus.
  • Experience in AI/ML evaluation, data curation, RLHF, or benchmark development is a plus.

Benefits

  • Work remotely.
  • Part-time, project-based consulting engagement.

Related Jobs

Hewlett Packard Enterprise

Senior Channel Presales Technical Architect, Storage — HPE Canada Remote

Hewlett Packard Enterprise

Help HPE channel partners position, design, and sell storage and data protection solutions. Support solution architecture, demonstrations, proofs of concept, pipeline development, and partner technical advocacy.

Open
Hewlett Packard Enterprise

Senior Network Operations Cloud Engineer (Secret Clearance)

Hewlett Packard Enterprise

Operate HPE’s secure private cloud and datacenter networks supporting federal customers. Troubleshoot classified infrastructure and VMware NSX across Oklahoma, Pennsylvania, and Utah.

Open
Hewlett Packard Enterprise

Senior Network Operations Cloud Engineer (Secret Clearance)

Hewlett Packard Enterprise

Operate and secure Hewlett Packard Enterprise private-cloud networks supporting federal customers. Troubleshoot classified infrastructure, VMware NSX, ServiceNow changes, and mission-critical connectivity.

Open