AI Model Policy Trainer, Mental Health (Seattle)

About this role

ABOUT HANDSHAKE

Handshake was founded on a simple belief that everyone deserves a path to a great career, regardless of where they went to school or who they know. Today, we power 25 million job seekers, 1 million+ employers, and 1,600 educational institutions.

Handshake AI works directly with frontier AI lab researchers to create evaluations, publish benchmarks, and improve AI models through human expertise.

ROLE DETAILS

Location: Onsite in Seattle, WA, Monday-Friday.

Compensation: $45-$65/hr. Placement within the range depends on experience.

Employer: TCWGlobal. This is a W-2 assignment supporting Handshake AI.

Employment: Full time, 40 hours per week, non-exempt and eligible for overtime pay.

Schedule: Monday-Friday, 8 a.m.-5 p.m. PT.

ABOUT THE ROLE

As an AI Model Policy Trainer focused on mental health, you will help evaluate how conversational AI systems respond to people discussing emotional distress, mental-health conditions, unusual beliefs, possible crises, and other sensitive experiences.

You will review conversations and model behavior against a detailed mental-health safety policy. Your work will help determine whether an AI response is empathetic, appropriately grounded, and sensitive to risk without reinforcing unsupported or potentially harmful beliefs.

This work requires an understanding of why people turn to AI for conversation, reflection, emotional support, or help during difficult moments. Strong candidates will recognize that AI can be useful and accessible while also understanding risks such as inappropriate validation, missed signs of crisis, overreliance, misleading guidance, or reinforcement of harmful beliefs.

Direct experience supporting individuals or families through mental-health or behavioral-health challenges is required. This experience may come from clinical care, social work, nursing, crisis support, professional caregiving, certified peer support, family advocacy, care navigation, or another structured human-services setting. AI or technology experience is helpful, but it does not substitute for this domain experience.

WHAT YOU WILL DO

- Review user conversations and AI responses involving mental health, emotional distress, crisis, unusual beliefs, paranoia, hallucinations, mania, and related experiences

- Apply detailed policy taxonomies and evaluation criteria to classify user requests and model behavior; evaluate individual messages and changes across longer conversations

- Distinguish between validating a person’s emotions and validating an unsupported or potentially harmful explanation; assess whether responses reinforce or escalate harmful beliefs

- Evaluate actions taken by AI systems through tools, including searches, messages, scheduling, purchases, file creation, and publication

- Write clear rationales that connect each evaluation decision to evidence from the conversation and policy

- Participate in calibration exercises and work with other trainers to improve consistency and provide evidence-based peer feedback

- Identify ambiguous examples, gaps, and edge cases that may require policy clarification

- Collaborate with researchers, policy specialists, operations teams, and other experts; incorporate coaching as policies and model capabilities evolve

WHO MAY BE A GOOD FIT

Qualified candidates must have direct experience in one or more of the following areas:

- Mental health or behavioral health

- Social work or community-based services

- Nursing or patient care

- Professional caregiving

- Certified peer support

- Family advocacy or care navigation

- Crisis support or related human services

Clinical licensure is valuable but is not required. Relevant professional caregiving, advocacy, certified peer-support, or other structured human-services experience may qualify. Interest or experience in AI is helpful, but it does not replace direct experience supporting people through mental-health or behavioral-health challenges.

DESIRED CAPABILITIES

- Required: direct experience supporting individuals or families through mental-health or behavioral-health challenges

- Familiarity with conversational AI tools and an interest in how people use them for emotional support and a balanced understanding of their benefits and risks

- Ability to interpret sensitive conversations with empathy, precision, and cultural awareness

- Ability to follow detailed policies and distinguish between closely related categories

- Strong written communication and the ability to explain decisions using specific evidence

- Sound judgment when information is incomplete, ambiguous, or context-dependent

- Openness to feedback, calibration, and evolving guidance, with constructive discussion of policy disagreements

- Strong professional boundaries and comfort working independently with sensitive material

EXTRA CREDIT

- Experience with structured review, case documentation, assessment, triage, quality assurance, or rubric-based evaluation

- Experience working with people experiencing crisis, psychosis, paranoia, mania, hallucinations, suicidal ideation, or severe emotional distress

- Experience as a certified peer specialist, family advocate, patient advocate, case manager, crisis counselor, social worker, nurse, caregiver, or behavioral-health professional

- Familiarity with ChatGPT, Claude, Gemini, or other conversational AI systems

- Experience evaluating AI outputs, annotating data, applying content policies, or conducting quality reviews

- Understanding of how culture, language, disability, socioeconomic circumstances, and access to care can affect mental-health conversations

- Experience participating in case consultation, supervision, audit, or calibration processes

YOU WILL THRIVE HERE IF

- You are curious about why people turn to AI for immediate, private, accessible emotional support

- You can acknowledge the potential value of AI without overlooking its limitations or risks

- You…