Labeler / Annotator – AI Response Evaluation (Castilian Spanish)

About this role

Who is Blueprint Technologies?

We are a technology solutions firm headquartered in Bellevue, Washington, with a strong presence across the United States and an expanding footprint across Latin America (LATAM). Our teams are united by a shared passion for solving complex problems.

At Blueprint, we use technology as a bridge between strategy and execution. Our people bring diverse perspectives, deep expertise, and real-world experience across industries to help organizations grow, transform, and innovate.

About the Role

We are seeking a detail-oriented Labeler / Annotator to evaluate responses generated by AI systems in Castilian Spanish.

This role focuses on side-by-side (SBS) evaluation of outputs from different AI models across real-world scenarios. You will play a key role in improving how AI systems understand and communicate in French.

This is not a translation role. It is an evaluation and analysis role requiring strong judgment and attention to detail.

What You’ll Work On

You will evaluate AI responses across scenarios such as:

• General-purpose Q&A

• Web search results

• File-based and image-based responses

• Image and file generation tasks

• Single-turn and multi-turn conversations

Responsibilities

• Perform side-by-side (SBS) comparisons of AI-generated responses

• Evaluate outputs based on:

• Accuracy

• Relevance

• Clarity

• Instruction-following

• Identify nuances in tone, meaning, and cultural context across French

• Apply detailed, scenario-specific annotation guidelines

• Maintain consistency and high-quality evaluations

Required Qualifications

• Native or professional fluency in Castilian Spanish

• Strong English reading comprehension (required for guidelines)

• Prior experience with side-by-side (SBS) labeling or annotation

• Excellent analytical thinking and attention to detail

• Ability to quickly understand and apply structured evaluation frameworks

Preferred Qualifications