Gramian Consulting
Gramian Consulting
Gramian Consulting is a remote-first consulting firm focused on engineering, data, and AI talent. It helps organizations expand technical capacity through talent augmentation, recruiting, dedicated teams, and contractor management, while providing services such as LLM training and fine-tuning, AI agents and assistants, MLOps, and AI infrastructure. The firm also supports individual career development through mentorship and education programs covering career readiness, interview preparation, and international market orientation. Its work combines hands-on engineering and recruiting experience to help clients build technical teams and apply AI to business needs.

AI Conversation Evaluation Research Specialist

Review human–AI conversations for Gramian Consulting, assessing whether the AI gathered enough context to make sound recommendations. Identify missing information and follow-up questions that could change its advice.

Description

  • Review AI conversations about relationships, work, education, family, planning, wellbeing, and other real-world decisions.
  • Assess whether the AI gathered enough context before offering advice, recommendations, or solutions.
  • Spot missing information and propose follow-up questions that could improve the response.
  • Determine whether additional context could materially change a recommendation or decision.
  • Separate outcome-changing information from details that are non-essential or merely descriptive.
  • Review conversations against structured qualitative evaluation guidelines and document findings consistently.
  • Identify missed chances to clarify, understand context, or gather relevant information.
  • Account for personal, psychological, social, financial, practical, and organizational factors when assessing conversations.
  • Work with research teams to apply consistent standards and produce high-quality assessments.
  • Complete assigned work accurately and within project deadlines.

Requirements

  • Bring at least 3–5 years of direct professional experience in one or more relevant fields.
  • For licensed social work: hold a Master of Social Work (MSW) or equivalent graduate qualification.
  • For licensed social work: hold an LCSW, LICSW, LMSW, or equivalent applicable credential or license.
  • Have experience in client assessment, case review, interviewing, counseling, or related social-work practice.
  • For counseling, therapy, or psychology: hold a relevant master's degree and an LMFT, LPC, LCPC, or equivalent counseling or therapy credential; alternatively, hold a PhD or PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology.
  • Have relevant experience in counseling, therapy, psychological assessment, interviewing, case review, or other client-facing professional work.
  • For qualitative research, HCI, or communication: hold an MA, PhD, or equivalent advanced expertise in Human-Computer Interaction, Communication, Anthropology, Sociology, Qualitative Research, or a closely related discipline.
  • Have experience conducting qualitative research, interviews, user research, conversation analysis, thematic analysis, qualitative coding, or similar research activities.
  • Be able to analyze conversations systematically and identify missing context or patterns in human communication.
  • For mediation, coaching, student services, organizational consulting, or related advisory fields: have a relevant professional qualification, credential, or demonstrated subject-matter expertise.
  • For the advisory-field pathway, bring at least five years of direct advisory or client-facing experience.
  • Have experience in coaching, mediation, student advising, organizational consulting, or comparable professional work.
  • Show the ability to gather context, understand individual or organizational needs, and make informed recommendations.

Related Jobs

No jobs found

Try adjusting or clearing your filters to find more opportunities.