Labeler / Annotator – AI Response Evaluation (German)

About this role

<h3 data-start="343" data-end="392" data-section-id="tennbi">Who is <span class="hover:entity-accent entity-underline inline cursor-pointer align-baseline"><span class="whitespace-normal">Blueprint Technologies</span></span>?</h3> <p data-start="394" data-end="643">We are a technology solutions firm headquartered in Bellevue, Washington, with a strong presence across the United States and an expanding footprint across Latin America (LATAM). Our teams are united by a shared passion for solving complex problems.</p> <p data-start="645" data-end="872">At Blueprint, we use technology as a bridge between strategy and execution. Our people bring diverse perspectives, deep expertise, and real-world experience across industries to help organizations grow, transform, and innovate.</p> <h3 data-start="879" data-end="897" data-section-id="zmeluu">About the Role</h3> <p data-start="899" data-end="1051">We are seeking a detail-oriented <strong data-start="932" data-end="955">Labeler / Annotator</strong> to evaluate responses generated by AI systems in <strong>German</strong>.</p> <p data-start="1053" data-end="1264">This role focuses on <strong data-start="1074" data-end="1107">side-by-side (SBS) evaluation</strong> of outputs from different AI models across real-world scenarios. You will play a key role in improving how AI systems understand and communicate in German.</p> <p data-start="1266" data-end="1390"><strong data-start="1266" data-end="1301">This is not a translation role.</strong> It is an evaluation and analysis role requiring strong judgment and attention to detail.</p> <h3 data-start="1397" data-end="1420" data-section-id="3i09bq">What You’ll Work On</h3> <p data-start="1422" data-end="1478">You will evaluate AI responses across scenarios such as:</p> <ul data-start="1480" data-end="1648"> <li data-start="1480" data-end="1503" data-section-id="2qh5a3">General-purpose Q&A</li> <li data-start="1504" data-end="1526" data-section-id="12bl4i4">Web search results</li> <li data-start="1527" data-end="1567" data-section-id="1y27mvm">File-based and image-based responses</li> <li data-start="1568" data-end="1603" data-section-id="1g34sai">Image and file generation tasks</li> <li data-start="1604" data-end="1648" data-section-id="1lakvzi">Single-turn and multi-turn conversations</li> </ul> <h3 data-start="1655" data-end="1675" data-section-id="t2pyjw">Responsibilities</h3> <ul data-start="1677" data-end="2045"> <li data-start="1677" data-end="1745" data-section-id="811khk">Perform side-by-side (SBS) comparisons of AI-generated responses</li> <li data-start="1746" data-end="1847" data-section-id="w7q3hx">Evaluate outputs based on: <ul data-start="1777" data-end="1847"> <li data-start="1777" data-end="1789" data-section-id="1o67qyd">Accuracy</li> <li data-start="1792" data-end="1805" data-section-id="1aid3u1">Relevance</li> <li data-start="1808" data-end="1819" data-section-id="13o2be8">Clarity</li> <li data-start="1822" data-end="1847" data-section-id="ma6oi4">Instruction-following</li> </ul> </li> <li data-start="1848" data-end="1931" data-section-id="dmg7bd">Identify nuances in tone, meaning, and cultural context across German</li> <li data-start="1932" data-end="1991" data-section-id="rj446b">Apply detailed, scenario-specific annotation guidelines</li> <li data-start="1992" data-end="2045" data-section-id="1fr7mu7">Maintain consistency and high-quality evaluations</li> </ul> <h3 data-start="2052" data-end="2079" data-section-id="8b3p4d">Required Qualifications</h3> <ul data-start="2081" data-end="2438"> <li data-start="2081" data-end="2164" data-section-id="1tm4f6t">Native or professional fluency in German</li> <li data-start="2165" data-end="2231" data-section-id="4lyz8x">Strong English reading comprehension (required for guidelines)</li> <li data-start="2232" data-end="2303" data-section-id="91006e"><strong data-start="2234" data-end="2301">Prior experience with side-by-side (SBS) labeling or annotation</strong></li> <li data-start="2304" data-end="2361" data-section-id="xjh9er">Excellent analytical thinking and attention to detail</li> <li data-start="2362" data-end="2438" data-section-id="1323d76">Ability to quickly understand and apply structured evaluation frameworks</li> </ul> <h3 data-start="2445" data-end="2473" data-section-id="fktgjl">Preferred Qualifications</h3> <ul data-start="2475" data-end="2644"> <li…