Customer Support Specialists, AI Response Evaluation

About mpathic.ai

Keeping the human in AI. mpathic is a trusted leader in advancing quality and safety in AI systems through expert-led evaluation and human data. We partner with leading technology companies to support red teaming, trust & safety, expert annotation, and model evaluation across high-stakes domains.

Our reviewers bring deep expertise in behavioral analysis, conversational design, mental health, and increasingly, financial and enterprise decision-making contexts.

About the Role

mpathic is seeking part-time Customer Support Specialists to support a red-teaming and quality assurance (QA) campaign focused on evaluating AI system behavior in customer-facing interactions.

In this role, you will review AI-generated responses and multi-turn conversations to assess quality, accuracy, tone, and safety—similar to evaluating customer support agents. You will identify issues such as incorrect information, poor customer experience, inappropriate tone, or failure to resolve user needs effectively.

This is not a traditional support role. Instead, it focuses on evaluating AI performance, applying your customer support expertise to assess how well AI systems handle real-world customer interactions.

What You’ll Be Working On 

You will help identify, prevent, and characterize risks that emerge when users interact with AI-powered support systems.

Responsibilities may include:

• Reviewing AI-generated customer support responses for accuracy, clarity, and helpfulness
• Evaluating tone, empathy, and professionalism in customer interactions
• Identifying poor support behaviors (e.g., dismissiveness, lack of resolution, incorrect guidance)
• Assessing whether AI responses appropriately address customer intent and resolve issues
• Evaluating how AI handles frustration, confusion, or escalating user behavior
• Identifying gaps in responses, including missing steps, unclear instructions, or lack of follow-through
• Detecting overconfident, misleading, or incorrect responses
• Evaluating multi-turn conversations for consistency, escalation handling, and resolution quality
• Identifying patterns of agreement without problem-solving (e.g., empty empathy or unhelpful reassurance)
• Participating in or reviewing red teaming exercises to surface failure modes in support scenarios
• Supporting quality assurance (QA) to ensure consistency across evaluations
• Documenting edge cases, failure patterns, and customer experience risks
• Providing structured written feedback to internal teams
• Collaborating with interdisciplinary teams on AI quality, safety, and evaluation frameworks
• Maintaining strict confidentiality and quality standards

This role requires strong judgment, attention to detail, and the ability to evaluate nuanced customer interactions.

What We’re Looking For

Successful candidates are detail-oriented, customer-focused, and experienced in evaluating or delivering high-quality support in fast-paced environments.

Professional experience in one or more of the following:

• Customer support (chat, email, or phone)
• Call centers or contact centers
• Technical support or help desk roles
• Online support environments (SaaS, e-commerce, fintech, etc.)

Strong understanding of:

• High-quality customer experience and service standards
• Effective communication in customer-facing interactions
• Common support workflows (triaging, troubleshooting, escalation)
• How customers interpret and respond to support interactions

Ability to identify:

• Poor or incomplete resolutions
• Incorrect or misleading information
• Inappropriate tone (e.g., robotic, dismissive, overly casual)
• Lack of empathy or ineffective de-escalation
• Failure to follow support best practices

Strong written communication skills and ability to clearly explain reasoning

Experience with or interest in:

• Evaluating conversations or QA for support teams
• Working with AI tools, chatbots, or automated systems
• Assessing how systems perform under real customer behavior

Comfort with:

• Tech tools and platforms (Slack, spreadsheets, dashboards)
• Evaluating AI-generated responses (no coding required, but must be tech-comfortable)
• Ambiguity, iteration, and feedback-driven workflows

Willingness to:

• Sign NDAs and work with sensitive content

Nice to Have (Not Required)

• Experience in QA, coaching, or training within support teams
• Background in trust & safety or content moderation
• Experience with chatbot or AI-assisted support tools
• Familiarity with CX metrics (CSAT, QA scoring, resolution rate)
• Interest in AI, automation, or improving customer experience systems

Compensation

$30-40/hour, depending on experience and specific project tasks/difficulty