Software Engineer - RL Environments

Software Engineer - RL Environments

Company

AfterQuery

Location

San Francisco, CA (in person)

Compensation

$260,000 - $290,000 base + profit share (around 150% of base, per the company) + competitive equity

Employment Type

Full-time

Visa Sponsorship

H-1B, O-1, OPT

About AfterQuery

AfterQuery is an applied research lab that curates the datasets, RL environments and evaluation infrastructure frontier AI labs use to build better models, working with leading AI labs and companies including Nvidia, Thinking Machines Lab and Legora.

Founded in 2025, AfterQuery reached a $100M+ revenue run rate within about a year and recently raised at a $3.2B valuation, making it the fastest unicorn in Y Combinator history.

The Role

As a Software Engineer on Environments, you will design the datasets and evaluation rubrics that directly shape how frontier models learn. You will work hands-on with research teams at top AI labs, experiment with data-collection strategies, diagnose model failure modes and build metrics that show whether a model is actually improving, moving from hypothesis to live experiment quickly, with output feeding directly into training runs at scale.

What You Will Do

• Design data slices that expose meaningful model failure modes across domains such as finance, code and enterprise workflows.
• Build and refine evaluation rubrics and reward signals for RLHF and RLVR pipelines.
• Model annotator behavior and run experiments to improve model capabilities.
• Develop quantitative frameworks for dataset quality, diversity and downstream impact on alignment and capability.
• Create and manage real-world and synthetic data pipelines.
• Translate lab research teams' training objectives into concrete data and evaluation specifications.

What You Bring

• 1-4 years of engineering experience
• Strong software engineering fundamentals and comfort running fast, hypothesis-driven experiments
• Interest in RL, evaluation and data for frontier model training
• Ability to work in person in San Francisco

Benefits

Profit share (around 150% of base per the company), competitive equity, and direct work with the world's leading AI labs.