AI Safety Red Team Expert – English and Tamil, Remote Contract
Test conversational AI models and agents for vulnerabilities through adversarial probing and structured red-team analysis. Create attack cases, vulnerability data, and reproducible reports for Mercor’s AI lab customers.
Description
- Probe conversational AI models and agents using jailbreaks, prompt injections, misuse scenarios, bias exploitation, and multi-turn manipulation
- Create human-labeled data by documenting failures, categorizing vulnerabilities, and identifying systemic risks
- Use established taxonomies, benchmarks, and playbooks to maintain consistent evaluations
- Develop reproducible reports, datasets, and attack cases for customers
- Assess AI outputs involving bias, misinformation, harmful behavior, and other sensitive topics
- Identify weaknesses that automated testing may overlook
- Broaden evaluation coverage and help prevent unexpected production issues
- Improve customer AI systems and contribute to safer, more reliable models
Requirements
- Native-level fluency in both English and Tamil
- Sound judgment when evaluating language and content
- Ability to determine whether AI responses are accurate, complete, and appropriate, and to explain the assessment
- Careful attention to subtle errors, inconsistencies, and omissions
- Ability to apply guidelines and quality standards consistently
- Clear communication of reasoning to technical and non-technical audiences
- Flexibility across projects, task formats, and customers
- Preferred: adversarial machine learning experience with jailbreak datasets, prompt injection, RLHF or DPO attacks, or model extraction
- Preferred: cybersecurity background in penetration testing, exploit development, or reverse engineering
- Preferred: experience with socio-technical risks such as harassment and disinformation probing, abuse analysis, or conversational AI testing
- Preferred: creative probing experience in psychology, acting, or writing
- Must work as an independent contractor
- H-1B and STEM OPT candidates are not supported
Benefits
- Fully remote work
- Flexible scheduling with the ability to set your own hours
- Weekly payments through Stripe or Wise
- Competitive hourly compensation
- Opportunity to gain experience in human-data-driven AI red teaming focused on frontier safety
- Direct contribution to more robust, safe, and trustworthy AI systems
- Optional participation in higher-sensitivity projects
- Guidelines and wellness resources for projects involving sensitive content
- Reasonable accommodations available upon request
- Referral program offering up to $90 per successful referral, subject to referral limits