AI EVAL/QE

ROLE

AI EVAL/QE

TORONTO - HYBRID - FULLTIME

10+ YRS EXP.

PRIMARY SKILLS

Gen AI Automation Testing, AI evaluator/ quality engineer

Define and execute the evaluation approach for generative-AI solutions, proving that models, agents and AI-assisted workflows are accurate, safe, reliable, performant and fit for production under TD controls. Requires 10+ years in quality engineering and at least 2 years evaluating AI/ML systems, with strong Python, test automation, API and integration testing, prompt and response test-set design, groundedness/relevance and hallucination assessment, regression, bias/safety, performance, penetration and non-functional testing; candidates should be comfortable with GitHub/GitHub Actions, GitHub Copilot, Claude Code, Copilot Studio, Foundry or LLM Gateway patterns, LangSmith, AppInsights, Log Analytics and Dynatrace, and must produce auditable evidence, defect analytics and go/no-go recommendations for architecture, privacy, cyber and risk reviewers.