Sentient Jobs

As part of our fast-moving team, you'll help shape the quality and reliability of how our AI interacts with consumers.

Role Overview

We're looking for a highly independent AI QA Engineer who thrives at the intersection of traditional QA and cutting-edge AI evaluation. This role spans frontend automation, backend API testing, and the challenge of evaluating AI output for quality, correctness, and speed.

Responsibilities

• Design and implement automated UI regression tests

• Write and maintain robust API test suites to validate backend functionality with each release

• Create and run AI evaluation tests to assess:

•  Quality and correctness of chatbot responses compared to ground truth and other products

• TTFT, end-to-end-latency, and other timing performance

• Build internal tools and datasets for testing and benchmarking

• Collaborate closely with product, engineering, and AI teams to ensure end-to-end quality of AI product

Requirements

• Strong experience with frontend testing tools (e.g., Playwright, Cypress)

• Proficient in backend/API testing (e.g., Postman, pytest, REST/GraphQL)

• Familiarity with evaluating LLM/AI-generated content

• Comfortable designing and analyzing test datasets for AI performance and functionality

• Strong scripting skills (e.g., Python)

• Highly autonomous with excellent problem-solving skills