Full-Stack Engineer, Agent Evaluation & Reliability

About the Role

Join a small, technical AI company building infrastructure that helps enterprise AI agents improve through production feedback. This backend-leaning full-stack role focuses on reliable evaluation systems and includes ownership of product features from design through delivery.

What You'll Do

• Build backend systems for agent evaluation and continuous improvement.
• Design data models for evaluation datasets, experiment runs, agent traces, knowledge versions, findings, and customer-defined quality criteria.
• Develop and operate enterprise integrations, including authentication, OAuth, webhooks, incremental syncs, backfills, rate limits, retries, and monitoring.
• Take features from technical design through implementation, deployment, monitoring, customer feedback, and iteration.
• Work with customers to diagnose system failures and turn specific needs into reusable product capabilities.
• Build product interfaces across the full stack while contributing to architecture and product decisions.

What We're Looking For

• Two to four years of experience building and operating production software, including experience at a fast-paced startup.
• Hands-on machine learning engineering experience that includes model development.
• Strong production experience with TypeScript and Node.js or Bun, and the ability to work across the full stack.
• Strong PostgreSQL knowledge and practical experience with Redis, job queues, background workers, or workflow systems.
• Understanding of why evaluating nondeterministic AI agent systems differs from conventional software testing.
• High ownership, fast learning, and the ability to take projects from start to finish.

Compensation & Benefits

Base salary

$120,000 to $180,000 annually. Visa sponsorship is available.

Location

On-site in San Francisco, California, United States.